Types conversion configuration¶

Connection can be preconfigured to parse IPC messages according to a specified settings, e.g.: temporal vectors can be represented as raw vectors or converted to numpy datetime64/timedelta64 representation.

# temporal values parsed to QTemporal and QTemporalList classes
q = qconnection.QConnection(host = 'localhost', port = 5000, numpy_temporals = False)

# temporal values parsed to numpy datetime64/timedelta64 arrays and atoms
q = qconnection.QConnection(host = 'localhost', port = 5000, numpy_temporals = True)

Conversion options can be also overwritten while executing synchronous/asynchronous queries (sync(), async()) or retrieving data from q (receive()).

Custom IPC protocol serializers/deserializers¶

Default IPC serializers (QWriter and _pandas.PandasQWriter) and deserializers (QReader and _pandas.PandasQReader) can be replaced with custom implementations. This allow users to override the default mapping between the q types and Python representation.

q = qconnection.QConnection(host = 'localhost', port = 5000, writer_class = MyQWriter, reader_class = MyQReader)

Refer to Custom type mapping for details.

Synchronous queries¶

Executes a q expression:

>>> print(q.sync('til 10'))
[0 1 2 3 4 5 6 7 8 9]

Executes an anonymous q function with a single parameter:

>>> print(q.sync('{til x}', 10))
[0 1 2 3 4 5 6 7 8 9]

Executes an anonymous q function with two parameters:

>>> print(q.sync('{y + til x}', 10, 1))
[ 1  2  3  4  5  6  7  8  9 10]
>>> print(q.sync('{y + til x}', *[10, 1]))
[ 1  2  3  4  5  6  7  8  9 10]

The QConnection class implements the __call__() method. This allows QConnection instance to be called as a function:

>>> print(q('{y + til x}', 10, 1))
[ 1  2  3  4  5  6  7  8  9 10]

Asynchronous queries¶

Calls a anonymous function with a single parameter:

>>> q.async('{til x}', 10)

Executes a q expression:

>>> q.async('til 10')

In order to retrieve query result (for the async() or query() methods), one has to call:

• receive() method, which reads next message from the remote q service.

For example:

• Retrieves query result along with meta-information:

>>> q.query(qconnection.MessageType.SYNC,'{x}', 10)
>>> print(q.receive(data_only = False, raw = False))
QMessage: message type: 2, data size: 13, is_compressed: False, data: 10

• Retrieves parsed query result:

>>> q.query(qconnection.MessageType.SYNC,'{x}', 10)
>>> print(q.receive(data_only = True, raw = False))
10

>>> q.sync('asynchMult:{[a;b] res:a*b; (neg .z.w)(res) }')
>>> q.async('asynchMult', 2, 3)
>>> print(q.receive())
6

• Retrieves not-parsed (raw) query result:

>>> from binascii import hexlify
>>> q.query(qconnection.MessageType.SYNC,'{x}', 10)
>>> print(hexlify(q.receive(data_only = True, raw = True)))
fa0a000000

Type conversions configuration¶

Type conversion options can be overwritten while:

• executing synchronous query: sync()
• executing asynchronous query: async()
• retrieving data from q: receive()

These methods accepts the options keywords arguments:

>>> query = "{[x] 0Nd, `date$til x}"

>>> # retrieve function call as raw byte buffer
>>> from binascii import hexlify
>>> print(binascii.hexlify(q(query, 5, raw = True)))
0e0006000000000000800000000001000000020000000300000004000000

>>> # perform a synchronous call and parse dates vector to numpy array
>>> print(q.sync(query, 5, numpy_temporals = True))
['NaT' '2000-01-01' '2000-01-02' '2000-01-03' '2000-01-04' '2000-01-05']

>>> # perform a synchronous call
>>> q.query(qconnection.MessageType.SYNC, query, 3)
>>> # retrieve query result and represent dates vector as raw data wrapped in QTemporalList
>>> print(q.receive(numpy_temporals = False))
[NaT [metadata(qtype=-14)] 2000-01-01 [metadata(qtype=-14)]
 2000-01-02 [metadata(qtype=-14)] 2000-01-03 [metadata(qtype=-14)]]

>>> # serialize single element strings as q characters
>>> print(q.sync('{[x] type each x}', ['one', 'two', '3'], single_char_strings = False))
[ 10,  10, -10]

>>> # serialize single element strings as q strings
>>> print(q.sync('{[x] type each x}', ['one', 'two', '3'], single_char_strings = True))
[10, 10, 10]

Atoms¶

While parsing IPC message atom q types are translated to Python types according to this table:

During the serialization to IPC protocol, Python types are mapped to q as described in the table:

String and symbols¶

In order to distinguish symbols and strings on the Python side, following rules apply:

• q symbols are represented as numpy.string_ type,
• q strings are mapped to plain Python strings in Python 2 and bytes in Python 3.

# Python 2
# `quickbrownfoxjumpsoveralazydog
<type 'numpy.string_'>
numpy.string_('quickbrownfoxjumpsoveralazydog')

# "quick brown fox jumps over a lazy dog"
<type 'str'>
'quick brown fox jumps over a lazy dog'

# Python 3
# `quickbrownfoxjumpsoveralazydog
<class 'numpy.bytes_'>
b'quickbrownfoxjumpsoveralazydog'

# "quick brown fox jumps over a lazy dog"
<class 'bytes'>
b'quick brown fox jumps over a lazy dog'

>>> # serialize single element strings as q characters
>>> print(q.sync('{[x] type each x}', ['one', 'two', '3'], single_char_strings = False))
[ 10,  10, -10]

>>> # serialize single element strings as q strings
>>> print(q.sync('{[x] type each x}', ['one', 'two', '3'], single_char_strings = True))
[10, 10, 10]

Lists¶

qPython represents deserialized q lists as instances of qcollection.QList are mapped to numpy arrays.

# (0x01;0x02;0xff)
qlist(numpy.array([0x01, 0x02, 0xff], dtype=numpy.byte))

# <class 'qpython.qcollection.QList'>
# numpy.dtype: int8
# meta.qtype: -4
# str: [ 1  2 -1]

Generic lists are represented as a plain Python lists.

# (1;`bcd;"0bc";5.5e)
[numpy.int64(1), numpy.string_('bcd'), '0bc', numpy.float32(5.5)]

While serializing Python data to q following heuristic is applied:

• instances of qcollection.QList and qcollection.QTemporalList are serialized according to type indicator (meta.qtype):

  qlist([1, 2, 3], qtype = QSHORT_LIST)
  # (1h;2h;3h)
  
  qlist([366, 121, qnull(QDATE)], qtype=QDATE_LIST)
  # '2001.01.01 2000.05.01 0Nd'
  
  qlist(numpy.array([uuid.UUID('8c680a01-5a49-5aab-5a65-d4bfddb6a661'), qnull(QGUID)]), qtype=QGUID_LIST)
  # ("G"$"8c680a01-5a49-5aab-5a65-d4bfddb6a661"; 0Ng)
  

• numpy arrays are serialized according to type of their dtype value:

  numpy.array([1, 2, 3], dtype=numpy.int32)
  # (1i;2i;3i)
  

• if numpy array dtype is not recognized by qPython, result q type is determined by type of the first element in the array,

• Python lists and tuples are represented as q generic lists:

  [numpy.int64(42), None, numpy.string_('foo')]
  (numpy.int64(42), None, numpy.string_('foo'))
  # (42;::;`foo)
  

qPython provides an utility function qcollection.qlist() which simplifies creation of qcollection.QList and qcollection.QTemporalList instances.

The qtype module defines QSTRING_LIST const which simplifies creation of string lists:

qlist(numpy.array(['quick', 'brown', 'fox', 'jumps', 'over', 'a lazy', 'dog']), qtype = QSTRING_LIST)
qlist(['quick', 'brown', 'fox', 'jumps', 'over', 'a lazy', 'dog'], qtype = QSTRING_LIST)
['quick', 'brown', 'fox', 'jumps', 'over', 'a lazy', 'dog']
# ("quick"; "brown"; "fox"; "jumps"; "over"; "a lazy"; "dog")

>>> v = q.sync("2001.01.01 2000.05.01 0Nd", numpy_temporals = False)
>>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
<class 'qpython.qcollection.QTemporalList'> dtype: int32 qtype: -14: [2001-01-01 [metadata(qtype=-14)] 2000-05-01 [metadata(qtype=-14)]
 NaT [metadata(qtype=-14)]]

>>> v = q.sync("2000.01.04D05:36:57.600 0Np", numpy_temporals = False)
>>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
<class 'qpython.qcollection.QTemporalList'> dtype: int64 qtype: -12: [2000-01-04T05:36:57.600000000+0100 [metadata(qtype=-12)]
 NaT [metadata(qtype=-12)]]

>>> v = q.sync("2001.01.01 2000.05.01 0Nd", numpy_temporals = True)
>>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
<class 'qpython.qcollection.QList'> dtype: datetime64[D] qtype: -14: ['2001-01-01' '2000-05-01' 'NaT']

>>> v = q.sync("2000.01.04D05:36:57.600 0Np", numpy_temporals = True)
>>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
<class 'qpython.qcollection.QList'> dtype: datetime64[ns] qtype: -12: ['2000-01-04T05:36:57.600000000+0100' 'NaT']

Dictionaries¶

qPython represents q dictionaries with custom qcollection.QDictionary class.

Examples:

QDictionary(qlist(numpy.array([1, 2], dtype=numpy.int64), qtype=QLONG_LIST),
            qlist(numpy.array(['abc', 'cdefgh']), qtype = QSYMBOL_LIST))
# q: 1 2!`abc`cdefgh


QDictionary([numpy.int64(1), numpy.int16(2), numpy.float64(3.234), '4'],
            [numpy.string_('one'), qlist(numpy.array([2, 3]), qtype=QLONG_LIST), '456', [numpy.int64(7), qlist(numpy.array([8, 9]), qtype=QLONG_LIST)]])
# q: (1;2h;3.234;"4")!(`one;2 3;"456";(7;8 9))

The qcollection.QDictionary class implements Python collection API.

Tables¶

The q tables are translated into custom qcollection.QTable class.

qPython provides an utility function qcollection.qtable() which simplifies creation of tables. This function also allow user to override default type conversions for each column and provide explicit q type hinting per column.

Examples:

qtable(qlist(numpy.array(['name', 'iq']), qtype = QSYMBOL_LIST),
      [qlist(numpy.array(['Dent', 'Beeblebrox', 'Prefect'])),
       qlist(numpy.array([98, 42, 126], dtype=numpy.int64))])

qtable(qlist(numpy.array(['name', 'iq']), qtype = QSYMBOL_LIST),
      [qlist(['Dent', 'Beeblebrox', 'Prefect'], qtype = QSYMBOL_LIST),
       qlist([98, 42, 126], qtype = QLONG_LIST)])

qtable(['name', 'iq'],
       [['Dent', 'Beeblebrox', 'Prefect'],
        [98, 42, 126]],
       name = QSYMBOL, iq = QLONG)

# flip `name`iq!(`Dent`Beeblebrox`Prefect;98 42 126)


qtable(('name', 'iq', 'fullname'),
       [qlist(numpy.array(['Dent', 'Beeblebrox', 'Prefect']), qtype = QSYMBOL_LIST),
        qlist(numpy.array([98, 42, 126]), qtype = QLONG_LIST),
        qlist(numpy.array(["Arthur Dent", "Zaphod Beeblebrox", "Ford Prefect"]), qtype = QSTRING_LIST)])

# flip `name`iq`fullname!(`Dent`Beeblebrox`Prefect;98 42 126;("Arthur Dent"; "Zaphod Beeblebrox"; "Ford Prefect"))

The keyed tables are represented by qcollection.QKeyedTable instances, where both keys and values are stored as a separate qcollection.QTable instances.

For example:

# ([eid:1001 1002 1003] pos:`d1`d2`d3;dates:(2001.01.01;2000.05.01;0Nd))
QKeyedTable(qtable(['eid'],
                   [qlist(numpy.array([1001, 1002, 1003]), qtype = QLONG_LIST)]),
            qtable(['pos', 'dates'],
                   [qlist(numpy.array(['d1', 'd2', 'd3']), qtype = QSYMBOL_LIST),
                    qlist(numpy.array([366, 121, qnull(QDATE)]), qtype = QDATE_LIST)]))

Functions, lambdas and projections¶

IPC protocol type codes 100+ are used to represent functions, lambdas and projections. These types are represented as instances of base class qtype.QFunction or descendent classes:

• qtype.QLambda - represents q lambda expression, note the expression is required to be either:

  • q expression enclosed in {}, e.g.: {x + y}
  • k expression, e.g.: k){x + y}

• qtype.QProjection - represents function projection with parameters:

  # { x + y}[3]
  QProjection([QLambda('{x+y}'), numpy.int64(3)])
  

Errors¶

The q errors are represented as instances of qtype.QException class.

Null values¶

Please note that q null values are defined as:

_QNULL1 = numpy.int8(-2**7)
_QNULL2 = numpy.int16(-2**15)
_QNULL4 = numpy.int32(-2**31)
_QNULL8 = numpy.int64(-2**63)
_QNAN32 = numpy.fromstring('\x00\x00\xc0\x7f', dtype=numpy.float32)[0]
_QNAN64 = numpy.fromstring('\x00\x00\x00\x00\x00\x00\xf8\x7f', dtype=numpy.float64)[0]
_QNULL_BOOL = numpy.bool_(False)
_QNULL_SYM = numpy.string_('')
_QNULL_GUID = uuid.UUID('00000000-0000-0000-0000-000000000000')

Complete null mapping between q and Python is represented in the table:

The qtype provides two utility functions to work with null values:

• qnull() - retrieves null type for specified q type code,
• is_null() - checks whether value is considered a null for specified q type code.

Custom type mapping¶

Default type mapping can be overwritten by providing custom implementations of QWriter and/or QReader and proper initialization of the connection as described in Custom IPC protocol serializers/deserializers.

QWriter and QReader use parse time decorator (Mapper) which generates mapping between q and Python types. This mapping is stored in a static variable: QReader._reader_map and QWriter._writer_map. In case mapping is not found in the mapping:

• QWriter tries to find a matching qtype in the ~qtype.Q_TYPE dictionary and serialize data as q atom,
• QReader tries to parse lists and atoms based on the type indicator in IPC stream.

While subclassing these classes, user can create copy of the mapping in the parent class and use parse time decorator:

class PandasQWriter(QWriter):
    _writer_map = dict.copy(QWriter._writer_map)    # create copy of default serializer map
    serialize = Mapper(_writer_map)                 # upsert custom mapping

    @serialize(pandas.Series)
    def _write_pandas_series(self, data, qtype = None):
        # serialize pandas.Series into IPC stream
        # ..omitted for readability..
        self._write_list(data, qtype = qtype)


class PandasQReader(QReader):
    _reader_map = dict.copy(QReader._reader_map)   # create copy of default deserializer map
    parse = Mapper(_reader_map)                    # overwrite default mapping

    @parse(QTABLE)
    def _read_table(self, qtype = QTABLE):
        # parse q table as pandas.DataFrame
        # ..omitted for readability..
        return pandas.DataFrame(data)

Refer to Custom type IPC deserialization for complete example.

Data conversions¶

If pandas flag is set, qPython converts the data according to following rules:

• q vectors are represented as pandas.Series:
  • pandas.Series is initialized with numpy.ndarray being result of parsing with numpy_temporals flag set to True (to ensure that temporal vectors are represented as numpy datetime64/timedelta64 arrays).
  • q nulls are replaced with numpy.NaN. This can result in type promotion as described in pandas documentation.
  • pandas.Series is enriched with custom attribute meta (qpython.MetaData), which contains qtype of the vector. Note that this information is used while serializaing pandas.Series instance to IPC protocol.
• tables are represented as pandas.DataFrame instances:
  • individual columns are represented as pandas.Series.
  • pandas.DataFrame is enriched with custom attribute meta (qpython.MetaData), which lists qtype for each column in table. Note that this information is used during pandas.DataFrame serialization.
• keyed tables are backed as pandas.DataFrame instances as well:
  • index for pandas.DataFrame is created from key columns.
  • pandas.DataFrame is enriched with custom attribute meta (qpython.MetaData), which lists qtype for each column in table, including index ones. Note that this information is used during pandas.DataFrame serialization.

Type hinting¶

qPython applies following heuristic to determinate conversion between pandas and q types:

• pandas.DataFrame are serialized to q tables,
• pandas.Series are serialized to q lists according to these rules:
  • type of q list is determinate based on series dtype,
  • if mapping based on dtype is ambiguous (e.g. dtype is object), q type is determined by type of the first element in the array.

User can overwrite the default type mapping, by setting the meta attribute and provide additional information for the serializer.

pandas.Series(numpy.array([numpy.datetime64('2000-01-04T05:36:57.600Z', 'ms'), numpy.datetime64('nat', 'ms')]))
# 2000.01.04D05:36:57.600000000 0N (type 12h)

l = pandas.Series(numpy.array([numpy.datetime64('2000-01-04T05:36:57.600Z', 'ms'), numpy.datetime64('nat', 'ms')]))
l.meta = MetaData(qtype = QDATE_LIST)
# 2000.01.04 0N (type 14h)

l = pandas.Series([1, numpy.nan, 3])
# 1 0n 3 (type 9h)

l = pandas.Series([1, numpy.nan, 3])
l.meta = MetaData(qtype = QINT_LIST)
# 1 0N 3i (type 6h)

t = pandas.DataFrame(OrderedDict((('pos', pandas.Series(['A', 'B', 'C'])),
                                  ('dates', pandas.Series(numpy.array([numpy.datetime64('2001-01-01'), numpy.datetime64('2000-05-01'), numpy.datetime64('NaT')], dtype='datetime64[D]'))))))

# pos dates
# ---------------------------------
# A   2001.01.01D00:00:00.000000000
# B   2000.05.01D00:00:00.000000000
# C
#
# meta:
# c    | t f a
# -----| -----
# pos  | c
# dates| p

t = pandas.DataFrame(OrderedDict((('pos', pandas.Series(['A', 'B', 'C'])),
                                  ('dates', pandas.Series(numpy.array([numpy.datetime64('2001-01-01'), numpy.datetime64('2000-05-01'), numpy.datetime64('NaT')], dtype='datetime64[D]'))))))

t.meta = MetaData(pos = QSYMBOL_LIST, dates = QDATE_LIST)

# pos dates
# --------------
# A   2001.01.01
# B   2000.05.01
# C
#
# meta:
# c    | t f a
# -----| -----
# pos  | s
# dates| d

t = pandas.DataFrame(OrderedDict((('eid', pandas.Series(numpy.array([1001, 1002, 1003]))),
                                  ('pos', pandas.Series(numpy.array(['d1', 'd2', 'd3']))),
                                  ('dates', pandas.Series(numpy.array([numpy.datetime64('2001-01-01'), numpy.datetime64('2000-05-01'), numpy.datetime64('NaT')], dtype='datetime64[D]'))))))
t.reset_index(drop = True)
t.set_index(['eid'], inplace = True)
t.meta = MetaData(pos = QSYMBOL_LIST, dates = QDATE_LIST)

# pos dates
# --------------
# d1  2001.01.01
# d2  2000.05.01
# d3
#
# meta:
# c    | t f a
# -----| -----
# pos  | s
# dates| d

t = pandas.DataFrame(OrderedDict((('eid', pandas.Series(numpy.array([1001, 1002, 1003]))),
                                  ('pos', pandas.Series(numpy.array(['d1', 'd2', 'd3']))),
                                  ('dates', pandas.Series(numpy.array([numpy.datetime64('2001-01-01'), numpy.datetime64('2000-05-01'), numpy.datetime64('NaT')], dtype='datetime64[D]'))))))
t.reset_index(drop = True)
t.set_index(['eid'], inplace = True)
t.meta = MetaData(pos = QSYMBOL_LIST, dates = QDATE_LIST, qtype = QKEYED_TABLE)

# eid | pos dates
# ----| --------------
# 1001| d1  2001.01.01
# 1002| d2  2000.05.01
# 1003| d3
#
# meta:
# c    | t f a
# -----| -----
# eid  | j
# pos  | s
# dates| d

Synchronous query¶

Following example presents how to execute simple, synchronous query against a remote q process:

from qpython import qconnection


if __name__ == '__main__':
    # create connection object
    q = qconnection.QConnection(host='localhost', port=5000)
    # initialize connection
    q.open()

    print(q)
    print('IPC version: %s. Is connected: %s' % (q.protocol_version, q.is_connected()))

    # simple query execution via: QConnection.__call__
    data = q('{`int$ til x}', 10)
    print('type: %s, numpy.dtype: %s, meta.qtype: %s, data: %s ' % (type(data), data.dtype, data.meta.qtype, data))

    # simple query execution via: QConnection.sync
    data = q.sync('{`long$ til x}', 10)
    print('type: %s, numpy.dtype: %s, meta.qtype: %s, data: %s ' % (type(data), data.dtype, data.meta.qtype, data))

    # low-level query and read
    q.query(qconnection.MessageType.SYNC, '{`short$ til x}', 10) # sends a SYNC query
    msg = q.receive(data_only=False, raw=False) # retrieve entire message
    print('type: %s, message type: %s, data size: %s, is_compressed: %s ' % (type(msg), msg.type, msg.size, msg.is_compressed))
    data = msg.data
    print('type: %s, numpy.dtype: %s, meta.qtype: %s, data: %s ' % (type(data), data.dtype, data.meta.qtype, data))
    # close connection
    q.close()

This code prints to the console:

:localhost:5000
IPC version: 3. Is connected: True
type: <class 'qpython.qcollection.QList'>, numpy.dtype: int32, meta.qtype: 6, data: [0 1 2 3 4 5 6 7 8 9]
type: <class 'qpython.qcollection.QList'>, numpy.dtype: int64, meta.qtype: 7, data: [0 1 2 3 4 5 6 7 8 9]
type: <class 'qpython.qreader.QMessage'>, message type: 2, data size: 34, is_compressed: False
type: <class 'qpython.qcollection.QList'>, numpy.dtype: int16, meta.qtype: 5, data: [0 1 2 3 4 5 6 7 8 9]

Asynchronous query¶

Following example presents how to execute simple, asynchronous query against a remote q process:

import random
import threading
import time

from qpython import qconnection
from qpython.qtype import QException
from qpython.qconnection import MessageType
from qpython.qcollection import QDictionary


class ListenerThread(threading.Thread):

    def __init__(self, q):
        super(ListenerThread, self).__init__()
        self.q = q
        self._stopper = threading.Event()

    def stop(self):
        self._stopper.set()

    def stopped(self):
        return self._stopper.isSet()

    def run(self):
        while not self.stopped():
            print('.')
            try:
                message = self.q.receive(data_only = False, raw = False) # retrieve entire message

                if message.type != MessageType.ASYNC:
                    print('Unexpected message, expected message of type: ASYNC')

                print('type: %s, message type: %s, data size: %s, is_compressed: %s ' % (type(message), message.type, message.size, message.is_compressed))
                print(message.data)

                if isinstance(message.data, QDictionary):
                    # stop after 10th query
                    if message.data[b'queryid'] == 9:
                        self.stop()

            except QException as e:
                print(e)


if __name__ == '__main__':
    # create connection object
    q = qconnection.QConnection(host = 'localhost', port = 5000)
    # initialize connection
    q.open()

    print(q)
    print('IPC version: %s. Is connected: %s' % (q.protocol_version, q.is_connected()))

    try:
        # definition of asynchronous multiply function
        # queryid - unique identifier of function call - used to identify
        # the result
        # a, b - parameters to the query
        q.sync('asynchMult:{[queryid;a;b] res:a*b; (neg .z.w)(`queryid`result!(queryid;res)) }');

        t = ListenerThread(q)
        t.start()

        for x in range(10):
            a = random.randint(1, 100)
            b = random.randint(1, 100)
            print('Asynchronous call with queryid=%s with arguments: %s, %s' % (x, a, b))
            q.async('asynchMult', x, a, b);

        time.sleep(1)
    finally:
        q.close()

Interactive console¶

This example depicts how to create a simple interactive console for communication with a q process:

import qpython
from qpython import qconnection
from qpython.qtype import QException

try:
    input = raw_input
except NameError:
    pass


if __name__ == '__main__':
    print('qPython %s Cython extensions enabled: %s' % (qpython.__version__, qpython.__is_cython_enabled__))
    with qconnection.QConnection(host = 'localhost', port = 5000) as q:
        print(q)
        print('IPC version: %s. Is connected: %s' % (q.protocol_version, q.is_connected()))

        while True:
            try:
                x = input('Q)')
            except EOFError:
                print('')
                break

            if x == '\\\\':
                break

            try:
                result = q(x)
                print(type(result))
                print(result)
            except QException as msg:
                print('q error: \'%s' % msg)

Twisted integration¶

This example presents how the qPython can be used along with Twisted to build asynchronous client:

import struct
import sys

from twisted.internet.protocol import Protocol, ClientFactory

from twisted.internet import reactor
from qpython.qconnection import MessageType, QAuthenticationException
from qpython.qreader import QReader
from qpython.qwriter import QWriter, QWriterException



class IPCProtocol(Protocol):

    class State(object):
        UNKNOWN = -1
        HANDSHAKE = 0
        CONNECTED = 1

    def connectionMade(self):
        self.state = IPCProtocol.State.UNKNOWN
        self.credentials = self.factory.username + ':' + self.factory.password if self.factory.password else ''

        self.transport.write(self.credentials + '\3\0')

        self._message = None

    def dataReceived(self, data):
        if self.state == IPCProtocol.State.CONNECTED:
            try:
                if not self._message:
                    self._message = self._reader.read_header(source=data)
                    self._buffer = ''

                self._buffer += data
                buffer_len = len(self._buffer) if self._buffer else 0

                while self._message and self._message.size <= buffer_len:
                    complete_message = self._buffer[:self._message.size]

                    if buffer_len > self._message.size:
                        self._buffer = self._buffer[self._message.size:]
                        buffer_len = len(self._buffer) if self._buffer else 0
                        self._message = self._reader.read_header(source=self._buffer)
                    else:
                        self._message = None
                        self._buffer = ''
                        buffer_len = 0

                    self.factory.onMessage(self._reader.read(source=complete_message, numpy_temporals=True))
            except:
                self.factory.onError(sys.exc_info())
                self._message = None
                self._buffer = ''

        elif self.state == IPCProtocol.State.UNKNOWN:
            # handshake
            if len(data) == 1:
                self._init(data)
            else:
                self.state = IPCProtocol.State.HANDSHAKE
                self.transport.write(self.credentials + '\0')

        else:
            # protocol version fallback
            if len(data) == 1:
                self._init(data)
            else:
                raise QAuthenticationException('Connection denied.')

    def _init(self, data):
        self.state = IPCProtocol.State.CONNECTED
        self.protocol_version = min(struct.unpack('B', data)[0], 3)
        self._writer = QWriter(stream=None, protocol_version=self.protocol_version)
        self._reader = QReader(stream=None)

        self.factory.clientReady(self)

    def query(self, msg_type, query, *parameters):
        if parameters and len(parameters) > 8:
            raise QWriterException('Too many parameters.')

        if not parameters or len(parameters) == 0:
            self.transport.write(self._writer.write(query, msg_type))
        else:
            self.transport.write(self._writer.write([query] + list(parameters), msg_type))



class IPCClientFactory(ClientFactory):

    protocol = IPCProtocol

    def __init__(self, username, password, connect_success_callback, connect_fail_callback, data_callback, error_callback):
        self.username = username
        self.password = password
        self.client = None

        # register callbacks
        self.connect_success_callback = connect_success_callback
        self.connect_fail_callback = connect_fail_callback
        self.data_callback = data_callback
        self.error_callback = error_callback


    def clientConnectionLost(self, connector, reason):
        print('Lost connection.  Reason: %s' % reason)
        # connector.connect()

    def clientConnectionFailed(self, connector, reason):
        if self.connect_fail_callback:
            self.connect_fail_callback(self, reason)

    def clientReady(self, client):
        self.client = client
        if self.connect_success_callback:
            self.connect_success_callback(self)

    def onMessage(self, message):
        if self.data_callback:
            self.data_callback(self, message)

    def onError(self, error):
        if self.error_callback:
            self.error_callback(self, error)

    def query(self, msg_type, query, *parameters):
        if self.client:
            self.client.query(msg_type, query, *parameters)



def onConnectSuccess(source):
    print('Connected, protocol version: %s' % source.client.protocol_version)
    source.query(MessageType.SYNC, '.z.ts:{(handle)(`timestamp$100?1000000000000000000)}')
    source.query(MessageType.SYNC, '.u.sub:{[t;s] handle:: neg .z.w}')
    source.query(MessageType.ASYNC, '.u.sub', 'trade', '')


def onConnectFail(source, reason):
    print('Connection refused: %s' % reason)


def onMessage(source, message):
    print('Received: %s %s' % (message.type, message.data))


def onError(source, error):
    print('Error: %s' % error)


if __name__ == '__main__':
    factory = IPCClientFactory('user', 'pwd', onConnectSuccess, onConnectFail, onMessage, onError)
    reactor.connectTCP('localhost', 5000, factory)
    reactor.run()

Subscribing to tick service¶

This example depicts how to subscribe to standard kdb+ tickerplant service:

import numpy
import threading
import sys

from qpython import qconnection
from qpython.qtype import QException
from qpython.qconnection import MessageType
from qpython.qcollection import QTable


class ListenerThread(threading.Thread):

    def __init__(self, q):
        super(ListenerThread, self).__init__()
        self.q = q
        self._stopper = threading.Event()

    def stopit(self):
        self._stopper.set()

    def stopped(self):
        return self._stopper.is_set()

    def run(self):
        while not self.stopped():
            print('.')
            try:
                message = self.q.receive(data_only = False, raw = False) # retrieve entire message

                if message.type != MessageType.ASYNC:
                    print('Unexpected message, expected message of type: ASYNC')

                print('type: %s, message type: %s, data size: %s, is_compressed: %s ' % (type(message), message.type, message.size, message.is_compressed))

                if isinstance(message.data, list):
                    # unpack upd message
                    if len(message.data) == 3 and message.data[0] == b'upd' and isinstance(message.data[2], QTable):
                        for row in message.data[2]:
                            print(row)

            except QException as e:
                print(e)


if __name__ == '__main__':
    with qconnection.QConnection(host = 'localhost', port = 17010) as q:
        print(q)
        print('IPC version: %s. Is connected: %s' % (q.protocol_version, q.is_connected()))
        print('Press <ENTER> to close application')

        # subscribe to tick
        response = q.sync('.u.sub', numpy.string_('trade'), numpy.string_(''))
        # get table model
        if isinstance(response[1], QTable):
            print('%s table data model: %s' % (response[0], response[1].dtype))

        t = ListenerThread(q)
        t.start()

        sys.stdin.readline()

        t.stopit()

Data publisher¶

This example shows how to stream data to the kdb+ process using standard tickerplant API:

import datetime
import numpy
import random
import threading
import sys
import time

from qpython import qconnection
from qpython.qcollection import qlist
from qpython.qtype import QException, QTIME_LIST, QSYMBOL_LIST, QFLOAT_LIST


class PublisherThread(threading.Thread):

    def __init__(self, q):
        super(PublisherThread, self).__init__()
        self.q = q
        self._stopper = threading.Event()

    def stop(self):
        self._stopper.set()

    def stopped(self):
        return self._stopper.isSet()

    def run(self):
        while not self.stopped():
            print('.')
            try:
                # publish data to tick
                # function: .u.upd
                # table: ask
                self.q.sync('.u.upd', numpy.string_('ask'), self.get_ask_data())

                time.sleep(1)
            except QException as e:
                print(e)
            except:
                self.stop()

    def get_ask_data(self):
        c = random.randint(1, 10)

        today = numpy.datetime64(datetime.datetime.now().replace(hour=0, minute=0, second=0, microsecond=0))

        time = [numpy.timedelta64((numpy.datetime64(datetime.datetime.now()) - today), 'ms') for x in range(c)]
        instr = ['instr_%d' % random.randint(1, 100) for x in range(c)]
        src = ['qPython' for x in range(c)]
        ask = [random.random() * random.randint(1, 100) for x in range(c)]

        data = [qlist(time, qtype=QTIME_LIST), qlist(instr, qtype=QSYMBOL_LIST), qlist(src, qtype=QSYMBOL_LIST), qlist(ask, qtype=QFLOAT_LIST)]
        print(data)
        return data


if __name__ == '__main__':
    with qconnection.QConnection(host='localhost', port=17010) as q:
        print(q)
        print('IPC version: %s. Is connected: %s' % (q.protocol_version, q.is_connected()))
        print('Press <ENTER> to close application')

        t = PublisherThread(q)
        t.start()

        sys.stdin.readline()

        t.stop()
        t.join()

Custom type IPC deserialization¶

This example shows how to override standard deserialization type mapping with two different QReader sub-classes. Please refer to Custom type mapping on implementation aspects:

import numpy

from qpython import qconnection
from qpython.qreader import QReader
from qpython.qtype import QSYMBOL, QSYMBOL_LIST, Mapper


class StringQReader(QReader):
    # QReader and QWriter use decorators to map data types and corresponding function handlers
    _reader_map = dict.copy(QReader._reader_map)
    parse = Mapper(_reader_map)

    def _read_list(self, qtype):
        if qtype == QSYMBOL_LIST:
            self._buffer.skip()
            length = self._buffer.get_int()
            symbols = self._buffer.get_symbols(length)
            return [s.decode(self._encoding) for s in symbols]
        else:
            return QReader._read_list(self, qtype = qtype)

    @parse(QSYMBOL)
    def _read_symbol(self, qtype = QSYMBOL):
        return numpy.string_(self._buffer.get_symbol()).decode(self._encoding)



class ReverseStringQReader(QReader):
    # QReader and QWriter use decorators to map data types and corresponding function handlers
    _reader_map = dict.copy(QReader._reader_map)
    parse = Mapper(_reader_map)

    @parse(QSYMBOL_LIST)
    def _read_symbol_list(self, qtype):
        self._buffer.skip()
        length = self._buffer.get_int()
        symbols = self._buffer.get_symbols(length)
        return [s.decode(self._encoding)[::-1] for s in symbols]

    @parse(QSYMBOL)
    def _read_symbol(self, qtype = QSYMBOL):
        return numpy.string_(self._buffer.get_symbol()).decode(self._encoding)[::-1]



if __name__ == '__main__':
    with qconnection.QConnection(host = 'localhost', port = 5000, reader_class = StringQReader) as q:
        symbols = q.sync('`foo`bar')
        print(symbols, type(symbols), type(symbols[0]))

        symbol = q.sync('`foo')
        print(symbol, type(symbol))


    with qconnection.QConnection(host = 'localhost', port = 5000, reader_class = ReverseStringQReader) as q:
        symbols = q.sync('`foo`bar')
        print(symbols, type(symbols), type(symbols[0]))

        symbol = q.sync('`foo')
        print(symbol, type(symbol))

qpython.qconnection module¶

exception qpython.qconnection.QConnectionException¶

Bases: exceptions.Exception

Raised when a connection to the q service cannot be established.

exception qpython.qconnection.QAuthenticationException¶

Bases: qpython.qconnection.QConnectionException

Raised when a connection to the q service is denied.

class qpython.qconnection.MessageType¶

Bases: object

Enumeration defining IPC protocol message types.

ASYNC = 0¶

SYNC = 1¶

RESPONSE = 2¶

class qpython.qconnection.QConnection(host, port, username=None, password=None, timeout=None, encoding='latin-1', reader_class=None, writer_class=None, **options)¶

Bases: object

Connector class for interfacing with the q service.

Provides methods for synchronous and asynchronous interaction.

The QConnection class provides a context manager API and can be used with a with statement:

with qconnection.QConnection(host = 'localhost', port = 5000) as q:
    print(q)
    print(q('{`int$ til x}', 10))

Parameters:

host (string) - q service hostname port (integer) - q service port username (string or None) - username for q authentication/authorization password (string or None) - password for q authentication/authorization timeout (nonnegative float or None) - set a timeout on blocking socket operations encoding (string) - string encoding for data deserialization reader_class (subclass of QReader) - data deserializer writer_class (subclass of QWriter) - data serializer

Options:

raw (boolean) - if True returns raw data chunk instead of parsed data, Default: False numpy_temporals (boolean) - if False temporal vectors are backed by raw q representation (QTemporalList, QTemporal) instances, otherwise are represented as numpy datetime64/timedelta64 arrays and atoms, Default: False single_char_strings (boolean) - if True single char Python strings are encoded as q strings instead of chars, Default: False

protocol_version¶

Retrieves established version of the IPC protocol.

Returns:	integer – version of the IPC protocol

open()¶

Initialises connection to q service.

If the connection hasn’t been initialised yet, invoking the open() creates a new socket and performs a handshake with a q service.

Raises:	QConnectionException, QAuthenticationException

close()¶

Closes connection with the q service.

is_connected()¶

Checks whether connection with a q service has been established.

Connection is considered inactive when:

• it has not been initialised,
• it has been closed.

Returns:	boolean – True if connection has been established, False otherwise

query(msg_type, query, *parameters, **options)¶

Performs a query against a q service.

In typical use case, query is the name of the function to call and parameters are its parameters. When parameters list is empty, the query can be an arbitrary q expression (e.g. 0 +/ til 100).

Calls a anonymous function with a single parameter:

>>> q.query(qconnection.MessageType.SYNC,'{til x}', 10)

Executes a q expression:

>>> q.query(qconnection.MessageType.SYNC,'til 10')

Parameters:	msg_type (one of the constants defined in MessageType) - type of the query to be executed query (string) - query to be executed parameters (list or None) - parameters for the query
Options:	single_char_strings (boolean) - if True single char Python strings are encoded as q strings instead of chars, Default: False
Raises:	QConnectionException, QWriterException

sync(query, *parameters, **options)¶

Performs a synchronous query against a q service and returns parsed data.

In typical use case, query is the name of the function to call and parameters are its parameters. When parameters list is empty, the query can be an arbitrary q expression (e.g. 0 +/ til 100).

Executes a q expression:

>>> print(q.sync('til 10'))
[0 1 2 3 4 5 6 7 8 9]

Executes an anonymous q function with a single parameter:

>>> print(q.sync('{til x}', 10))
[0 1 2 3 4 5 6 7 8 9]

Executes an anonymous q function with two parameters:

>>> print(q.sync('{y + til x}', 10, 1))
[ 1  2  3  4  5  6  7  8  9 10]

>>> print(q.sync('{y + til x}', *[10, 1]))
[ 1  2  3  4  5  6  7  8  9 10]

The sync() is called from the overloaded __call__() function. This allows QConnection instance to be called as a function:

>>> print(q('{y + til x}', 10, 1))
[ 1  2  3  4  5  6  7  8  9 10]

Parameters:	query (string) - query to be executed parameters (list or None) - parameters for the query
Options:	raw (boolean) - if True returns raw data chunk instead of parsed data, Default: False numpy_temporals (boolean) - if False temporal vectors are backed by raw q representation (QTemporalList, QTemporal) instances, otherwise are represented as numpy datetime64/timedelta64 arrays and atoms, Default: False single_char_strings (boolean) - if True single char Python strings are encoded as q strings instead of chars, Default: False
Returns:	query result parsed to Python data structures
Raises:	QConnectionException, QWriterException, QReaderException

async(query, *parameters, **options)¶

Performs an asynchronous query and returns without retrieving of the response.

In typical use case, query is the name of the function to call and parameters are its parameters. When parameters list is empty, the query can be an arbitrary q expression (e.g. 0 +/ til 100).

Calls a anonymous function with a single parameter:

>>> q.async('{til x}', 10)

Executes a q expression:

>>> q.async('til 10')

Parameters:	query (string) - query to be executed parameters (list or None) - parameters for the query
Options:	single_char_strings (boolean) - if True single char Python strings are encoded as q strings instead of chars, Default: False
Raises:	QConnectionException, QWriterException

receive(data_only=True, **options)¶

Reads and (optionally) parses the response from a q service.

Retrieves query result along with meta-information:

>>> q.query(qconnection.MessageType.SYNC,'{x}', 10)
>>> print(q.receive(data_only = False, raw = False))
QMessage: message type: 2, data size: 13, is_compressed: False, data: 10

Retrieves parsed query result:

>>> q.query(qconnection.MessageType.SYNC,'{x}', 10)
>>> print(q.receive(data_only = True, raw = False))
10

Retrieves not-parsed (raw) query result:

>>> from binascii import hexlify
>>> q.query(qconnection.MessageType.SYNC,'{x}', 10)
>>> print(hexlify(q.receive(data_only = True, raw = True)))
fa0a000000

Parameters:	data_only (boolean) - if True returns only data part of the message, otherwise returns data and message meta-information encapsulated in QMessage instance
Options:	raw (boolean) - if True returns raw data chunk instead of parsed data, Default: False numpy_temporals (boolean) - if False temporal vectors are backed by raw q representation (QTemporalList, QTemporal) instances, otherwise are represented as numpy datetime64/timedelta64 arrays and atoms, Default: False
Returns:	depending on parameter flags: QMessage instance, parsed message, raw data
Raises:	QReaderException

qpython.qcollection module¶

class qpython.qcollection.QList(spec=None, side_effect=None, return_value=sentinel.DEFAULT, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, **kwargs)¶

Bases: Mock

An array object represents a q vector.

class qpython.qcollection.QTemporalList(spec=None, side_effect=None, return_value=sentinel.DEFAULT, wraps=None, name=None, spec_set=None, parent=None, _spec_state=None, _new_name='', _new_parent=None, **kwargs)¶

Bases: qpython.qcollection.QList

An array object represents a q vector of datetime objects.

raw(idx)¶

Gets the raw representation of the datetime object at the specified index.

>>> t = qlist(numpy.array([366, 121, qnull(QDATE)]), qtype=QDATE_LIST)
>>> print(t[0])
2001-01-01 [metadata(qtype=-14)]
>>> print(t.raw(0))
366

Parameters:	idx (integer) - array index of the datetime object to be retrieved
Returns:	raw representation of the datetime object

qpython.qcollection.get_list_qtype(array)¶

Finds out a corresponding qtype for a specified QList/numpy.ndarray instance.

Parameters:	array (QList or numpy.ndarray) - array to be checked
Returns:	integer - qtype matching the specified array object

qpython.qcollection.qlist(array, adjust_dtype=True, **meta)¶

Converts an input array to q vector and enriches object instance with meta data.

Returns a QList instance for non-datetime vectors. For datetime vectors QTemporalList is returned instead.

If parameter adjust_dtype is True and q type retrieved via get_list_qtype() doesn’t match one provided as a qtype parameter guessed q type, underlying numpy.array is converted to correct data type.

qPython internally represents (0x01;0x02;0xff) q list as: <class 'qpython.qcollection.QList'> dtype: int8 qtype: -4: [ 1  2 -1]. This object can be created by calling the qlist() with following arguments:

• byte numpy.array:

  >>> v = qlist(numpy.array([0x01, 0x02, 0xff], dtype=numpy.byte))
  >>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
  <class 'qpython.qcollection.QList'> dtype: int8 qtype: -4: [ 1  2 -1]
  

• int32 numpy.array with explicit conversion to QBYTE_LIST:

  >>> v = qlist(numpy.array([1, 2, -1]), qtype = QBYTE_LIST)
  >>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
  <class 'qpython.qcollection.QList'> dtype: int8 qtype: -4: [ 1  2 -1]
  

• plain Python integer list with explicit conversion to QBYTE_LIST:

  >>> v = qlist([1, 2, -1], qtype = QBYTE_LIST)
  >>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
  <class 'qpython.qcollection.QList'> dtype: int8 qtype: -4: [ 1  2 -1]
  

• numpy datetime64 array with implicit conversion to QDATE_LIST:

  >>> v = qlist(numpy.array([numpy.datetime64('2001-01-01'), numpy.datetime64('2000-05-01'), numpy.datetime64('NaT')], dtype='datetime64[D]'))
  >>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
  <class 'qpython.qcollection.QList'> dtype: datetime64[D] qtype: -14: ['2001-01-01' '2000-05-01' 'NaT']
  

• numpy datetime64 array with explicit conversion to QDATE_LIST:

  >>> v = qlist(numpy.array([numpy.datetime64('2001-01-01'), numpy.datetime64('2000-05-01'), numpy.datetime64('NaT')], dtype='datetime64[D]'), qtype = QDATE_LIST)
  >>> print('%s dtype: %s qtype: %d: %s' % (type(v), v.dtype, v.meta.qtype, v))
  <class 'qpython.qcollection.QList'> dtype: datetime64[D] qtype: -14: ['2001-01-01' '2000-05-01' 'NaT']
  

Parameters:	array (tuple, list, numpy.array) - input array to be converted adjust_dtype (boolean) - determine whether data type of vector should be adjusted if it doesn’t match default representation. Default: True Note numpy datetime64 and timedelta64 arrays are not converted to raw temporal vectors if adjust_dtype is True
Kwargs:	qtype (integer or None) - qtype indicator
Returns:	QList or QTemporalList - array representation of the list
Raises:	ValueError

class qpython.qcollection.QDictionary(keys, values)¶

Bases: object

Represents a q dictionary.

Dictionary examples:

>>> # q: 1 2!`abc`cdefgh
>>> print(QDictionary(qlist(numpy.array([1, 2], dtype=numpy.int64), qtype=QLONG_LIST), 
...                    qlist(numpy.array(['abc', 'cdefgh']), qtype = QSYMBOL_LIST)))
[1 2]!['abc' 'cdefgh']

>>> # q: (1;2h;3.234;"4")!(`one;2 3;"456";(7;8 9))
>>> print(QDictionary([numpy.int64(1), numpy.int16(2), numpy.float64(3.234), '4'], 
...                    [numpy.string_('one'), qlist(numpy.array([2, 3]), qtype=QLONG_LIST), '456', [numpy.int64(7), qlist(numpy.array([8, 9]), qtype=QLONG_LIST)]]))
[1, 2, 3.234, '4']!['one', QList([2, 3], dtype=int64), '456', [7, QList([8, 9], dtype=int64)]]

Parameters:	keys (QList, tuple or list) - dictionary keys values (QList, QTable, tuple or list) - dictionary values

items()¶

Return a copy of the dictionary’s list of (key, value) pairs.

iteritems()¶

Return an iterator over the dictionary’s (key, value) pairs.

iterkeys()¶

Return an iterator over the dictionary’s keys.

itervalues()¶

Return an iterator over the dictionary’s values.

qpython.qcollection.qtable(columns, data, **meta)¶

Creates a QTable out of given column names and data, and initialises the meta data.

QTable is represented internally by numpy.core.records.recarray. Data for each column is converted to QList via qlist() function. If qtype indicator is defined for a column, this information is used for explicit array conversion.

Table examples:

>>> # q: flip `name`iq!(`Dent`Beeblebrox`Prefect;98 42 126)
>>> t = qtable(qlist(numpy.array(['name', 'iq']), qtype = QSYMBOL_LIST), 
...     [qlist(numpy.array(['Dent', 'Beeblebrox', 'Prefect'])), 
...      qlist(numpy.array([98, 42, 126], dtype=numpy.int64))])
>>> print('%s dtype: %s meta: %s: %s' % (type(t), t.dtype, t.meta, t))
<class 'qpython.qcollection.QTable'> dtype: [('name', 'S10'), ('iq', '<i8')] meta: metadata(iq=-7, qtype=98, name=-11): [('Dent', 98L) ('Beeblebrox', 42L) ('Prefect', 126L)]

>>> # q: flip `name`iq!(`Dent`Beeblebrox`Prefect;98 42 126)
>>> t = qtable(qlist(numpy.array(['name', 'iq']), qtype = QSYMBOL_LIST),
...           [qlist(['Dent', 'Beeblebrox', 'Prefect'], qtype = QSYMBOL_LIST), 
...            qlist([98, 42, 126], qtype = QLONG_LIST)])
>>> print('%s dtype: %s meta: %s: %s' % (type(t), t.dtype, t.meta, t))
<class 'qpython.qcollection.QTable'> dtype: [('name', 'S10'), ('iq', '<i8')] meta: metadata(iq=-7, qtype=98, name=-11): [('Dent', 98L) ('Beeblebrox', 42L) ('Prefect', 126L)]

>>> # q: flip `name`iq!(`Dent`Beeblebrox`Prefect;98 42 126)
>>> t = qtable(['name', 'iq'],
...            [['Dent', 'Beeblebrox', 'Prefect'], 
...             [98, 42, 126]],
...            name = QSYMBOL, iq = QLONG)
>>> print('%s dtype: %s meta: %s: %s' % (type(t), t.dtype, t.meta, t)) 
<class 'qpython.qcollection.QTable'> dtype: [('name', 'S10'), ('iq', '<i8')] meta: metadata(iq=-7, qtype=98, name=-11): [('Dent', 98L) ('Beeblebrox', 42L) ('Prefect', 126L)]

>>> # q: flip `name`iq`fullname!(`Dent`Beeblebrox`Prefect;98 42 126;("Arthur Dent"; "Zaphod Beeblebrox"; "Ford Prefect"))
>>> t = qtable(('name', 'iq', 'fullname'),
...            [qlist(numpy.array(['Dent', 'Beeblebrox', 'Prefect']), qtype = QSYMBOL_LIST), 
...             qlist(numpy.array([98, 42, 126]), qtype = QLONG_LIST),
...             qlist(numpy.array(["Arthur Dent", "Zaphod Beeblebrox", "Ford Prefect"]), qtype = QSTRING_LIST)])
<class 'qpython.qcollection.QTable'> dtype: [('name', 'S10'), ('iq', '<i8'), ('fullname', 'O')] meta: metadata(iq=-7, fullname=0, qtype=98, name=-11): [('Dent', 98L, 'Arthur Dent') ('Beeblebrox', 42L, 'Zaphod Beeblebrox') ('Prefect', 126L, 'Ford Prefect')]

Parameters:	columns (list of strings) - table column names data (list of lists) - list of columns containing table data
Kwargs:	meta (integer) - qtype for particular column
Returns:	QTable - representation of q table
Raises:	ValueError