Pointer to c_int16 array buffer in Ctypes

Question

I'm writing a wrapper for a C dll in Python Ctypes. In the C library I have a function of the kind

int32 Transfer  ( ..., PIN_PARAMS_TRANSFERDATA  pInData, ... )

where PIN_PARAMS_TRANSFERDATA is a pointer to this structure:

typedef struct
{
    ...
    void *  pDataBuffer;        //!< Pointer to application-allocated buffer to hold requested data.
} IN_PARAMS_TRANSFERDATA, *PIN_PARAMS_TRANSFERDATA;

The data type of the DataBuffer depends can change, but is usually int16 in my case. In ctypes I defined the structure and the function as follows:

class IN_PARAMS_TRANSFERDATA(Structure):
    _fields_ = [
        ...,
        ('pDataBuffer', c_void_p)
    ]

_lib = CDLL('mydll.dll')

_lib.Transfer.argtypes = [..., POINTER(IN_PARAMS_TRANSFERDATA), ...]
_lib.Transfer.restype = c_int32

def Transfer(system, in_data):
    status = _lib.Transfer(..., byref(in_data), ...)
    return in_data

Now I have the problem that I have to define a data buffer before calling the Transfer function, which I don't manage. I tried the following (the 2000 is just for testing and should rather be a variable):

in_data = IN_PARAMS_TRANSFERDATA()
data_buffer = (c_int16*2000)()
print(data_buffer)
in_data.pDataBuffer = byref(data_buffer)
in_data = Transfer(hsystem, in_data)

The output message is:

<__main__.c_short_Array_2000 object at 0x00000000053D2EC8>
Traceback (most recent call last):
  File ".../test.py", line 48, in <module>
    in_data.pDataBuffer = byref(data_buffer)
TypeError: cannot be converted to pointer

Can anybody help me, how to create the data buffer properly and pass it to the structure (and finally to the C function), such that the C function can write data into it and I can read the data out (e.g. as a numpy.array).

In case this helps, this is how the creation of the buffer pointer is done in an example C program (unfortunately I don't really understand what's going on):

void* pBuffer = NULL;
pBuffer  = VirtualAlloc(NULL, (size_t)((i64TransferLength + i64Padding) * u32SampleSize), MEM_COMMIT, PAGE_READWRITE);

EDIT

Following J.F. Sebastian's suggestion, I changed the creation of the buffer slightly:

data_buffer = (c_int16*2000)()
p_data_buffer = ctypes.cast(data_buffer, POINTER(c_int16*2000))
in_data.pDataBuffer = p_data_buffer

which gives me a slightly different error now:

Traceback (most recent call last):
  File ".../test.py", line 50, in <module>
    in_data.pDataBuffer = p_data_buffer
TypeError: incompatible types, LP_c_short_Array_2000 instance instead of c_void_p instance

instead of byref(data_buffer), try ctypes.cast(data_buffer, ctypes.POINTER(ArrayType)) where ArrayType = c_int16*2000 See Pointers and arrays in Python ctypes ` — jfs
– jfs, Commented Jun 7, 2016 at 13:36
Thanks for your hint. Indeed, now the conversion to the pointer works. However, I still cannot assign this pointer to in_data.pDataBuffer, because the type doesn't match c_void_p. I'll update the question. — erik
– erik, Commented Jun 7, 2016 at 17:52
I tried it now, and it runs. But I have problems accessing the data in the buffer now. I would assume that I can just do list(in_data.pDataBuffer) to obtain a list with all the elements in the buffer. But I get the error, that 'long' object is not iterable. I tried to cast back to POINTER(c_int16*2000) and then apply list() but this runs forever without any result. — erik
– erik, Commented Jun 7, 2016 at 18:54

Community · Accepted Answer · 2017-05-23 12:08:12Z

4

ctypes.c_void_p is represented as an integer. You could pass it the address of the array:

>>> from ctypes import *
>>> class IN_PARAMS_TRANSFERDATA(Structure):
...     _fields_ = [('pDataBuffer', c_void_p)]
... 
>>> ArrayType = c_int16 * 2000
>>> in_data = IN_PARAMS_TRANSFERDATA()
>>> data_buffer = ArrayType(1,2,3) # set the first three values
>>> in_data.pDataBuffer = c_void_p(addressof(data_buffer)) 
>>>                  # or cast(data_buffer, c_void_p)

Get the values back:

>>> cast(in_data.pDataBuffer, POINTER(ArrayType)).contents[:3]
[1, 2, 3]

Here're the first three values.

To get numpy array from the pointer, see How to convert pointer to c array to python array:

>>> import numpy
>>> pa = cast(in_data.pDataBuffer, POINTER(ArrayType))
>>> a = numpy.frombuffer(pa.contents, dtype=c_int16)
>>> a
array([1, 2, 3, ..., 0, 0, 0], dtype=int16)

edited May 23, 2017 at 12:08

CommunityBot

11 silver badge

answered Jun 7, 2016 at 21:13

jfs

417k210 gold badges1k silver badges1.7k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

erik Over a year ago

This method works very well. For reference, I will also write an answer using VirtualAlloc, that I found.

erik · Accepted Answer · 2016-06-08 07:17:14Z

0

Another possibility is to use VirtualAlloc, like in the original C code.

p_data_buffer = ctypes.windll.kernel32.VirtualAlloc(c_int(0), c_int(sizeof(c_int16)*2000), c_int(0x00001000), c_int(0x04))
in_data.pDataBuffer = p_data_buffer

And for getting the data back:

target = (c_int16*2000)()
ctypes.windll.kernel32.RtlMoveMemory(target, in_data.pDataBuffer, c_int(sizeof(c_int16)*2000))
data = numpy.frombuffer(target, dtype=c_int16)

Here you can find more details. I assume though, that this code is specific for Windows.

answered Jun 8, 2016 at 7:17

erik

6591 gold badge11 silver badges21 bronze badges

1 Comment

jfs Over a year ago

1- you don't need to constantly write c_int: VirtualAlloc(NULL, sizeof(ArrayType), MEM_COMMIT, PAGE_READWRITE) where NULL=None, MEM_COMMIT, PAGE_READWRITE = 0x1000, 0x4 should work. 2- use target = ArrayType() and pass sizeof(ArrayType). Avoid magic constants in your code or give them meaningful names.

Collectives™ on Stack Overflow

Pointer to c_int16 array buffer in Ctypes

2 Answers 2

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related