Using ctypes to call a C++ method with parameters from Python results in "Don't know how to convert parameter" error

Question

I'm trying to use the following C++ class from Python3.7, but can't get the first method 'Set' to work, much less the operator overload methods. I've tried many variations of the Python wrapper and the extern block but I either get the "Don't know how to convert parameter 5" error or a segmentation fault. The examples online and SO answers I've found are too basic or address other issues.

Should my argtypes 1st argument be a pointer to the object? I don't know what syntax would be used to indicate that.

CMatrix.h:

#include <stdio.h>

class CMatrix
{
public:
    CMatrix(int d1);
    CMatrix(int d1, int d2);
    CMatrix(int d1, int d2, int d3);
    CMatrix (float f1, float f2, float f3);
    ~CMatrix();
    
    void Set(int x, int y, int z, float f);
    void operator=(const float f);
    CMatrix& operator=(const CMatrix &cm);
    inline float& operator()(const int i) {
        return m[i];}
    inline float& operator()(const int i, const int j) {
        return m[i*s2+j];}
    inline float& operator()(const int i, const int j, const int k) {
        return m[i*s23+j*s3+k];}
    int s1, s2, s3;     // dimensions of array
    int s23;            // s2*s3;
    float *m;           // pointer to first element of matrix.
    int dimensions;     // 1, 2, or 3.
};

extern "C" {
    CMatrix* CMatrix_new1(int i) {return new CMatrix(i); }
    CMatrix* CMatrix_new2(int i, int j) {return new CMatrix(i, j); }
    CMatrix* CMatrix_new3(int i, int j, int k) {return new CMatrix(i, j, k); }
    void CMatrix_Set(CMatrix* cm, int x, int y, int z, float f) {cm->Set(x, y, z, f); }
}

cmWrapper.py:

import ctypes as c
lib = c.cdll.LoadLibrary('./libCMatrix.o')
lib.CMatrix_Set.argtypes = [c.c_int, c.c_int, c.c_int, c.c_float]

class CMatrix(object):
    def __init__(self, i, j, k):
        if j==0 and k==0:
            self.obj = lib.CMatrix_new1(i)
        elif k==0:
            self.obj = lib.CMatrix_new2(i, j)
        else:
            self.obj = lib.CMatrix_new3(i, j, k)

    def Set(self, x, y, z, f):
        lib.CMatrix_Set(self.obj, x, y, z, f)

cm = CMatrix(2, 3, 0)
cm.Set(1, 2, 0, 99.0)

The traceback:

>>> import cmWrapper
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/.../ctypes/cmWrapper.py", line 18, in <module>
    cm.Set(1, 2, 0, 99.0)
  File "/.../ctypes/cmWrapper.py", line 15, in Set
    lib.CMatrix_Set(self.obj, x, y, z, f)
ctypes.ArgumentError: argument 5: <class 'TypeError'>: Don't know how to convert parameter 5

If it matters, I compiled the C++ code using:

g++ -c -fPIC CMatrix.cpp -o CMatrix.o
g++ -shared -Wl -o libCMatrix.o CMatrix.o

This is on a Mac running 10.15.7.

From lldb:

Executable module set to "/Library/Frameworks/Python.framework/Versions/3.7/Resources/Python.app/Contents/MacOS/Python".
Architecture set to: x86_64h-apple-macosx-.
(lldb) 
There is a running process, detach from it and attach?: [Y/n] n
(lldb) thread list
Process 57460 stopped
* thread #1: tid = 0x47364c, 0x00007fff202f5656 libsystem_kernel.dylib`__select + 10, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
(lldb) thread continue
Resuming thread 0x47364c in process 57460
Process 57460 resuming
Process 57460 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x13835f20)
    frame #0: 0x00000001015f19b2 libCMatrix.o`CMatrix::Set(int, int, int, float) + 34
libCMatrix.o`CMatrix::Set:
->  0x1015f19b2 <+34>: movl   (%rax), %esi
    0x1015f19b4 <+36>: movl   0x4(%rax), %edx
    0x1015f19b7 <+39>: movl   0x8(%rax), %ecx
    0x1015f19ba <+42>: movl   0xc(%rax), %r8d
Target 0: (Python) stopped.

Mark Tolonen · Accepted Answer · 2021-10-18 17:05:55Z

You have to match the arguments exactly. Set the .argtypes and .restype of every function you use so ctypes can properly marshal the parameters to C and back again. If you do not set .restype ctypes assumes the return value is c_int (typically a signed 32-bit integer) instead of a (possibly 64-bit) pointer.

Here's a working example. I didn't flesh out every function because one should be sufficient. Tested on both 32- and 64-bit Python.

test.cpp (built with MS compiler, cl /LD /EHsc /W4 test.cpp):

#include <stdio.h>

// Needed to export functions on Windows
#ifdef _WIN32
#   define API __declspec(dllexport)
#else
#   define API
#endif

class CMatrix
{
public:
    CMatrix(int d1) : s1(d1) { m = new float[d1]; }
    ~CMatrix() { delete [] m; }
    const float* Get(int& s) { s = s1; return m; }
    void Set(int x, float f) { m[x] = f; }
    int s1;
    float *m;
};

extern "C" {
    API CMatrix* CMatrix_new(int i) {return new CMatrix(i); }
    API const float* CMatrix_Get(CMatrix* cm, int& x) { return cm->Get(x); }
    API void CMatrix_Set(CMatrix* cm, int x, float f) { cm->Set(x, f); }
    API void CMatrix_delete(CMatrix* cm) { delete cm; }
}

test.py

import ctypes as ct

# For type checking the returned pointer.
class _CMatrix(ct.c_void_p) : pass
PCMatrix = ct.POINTER(_CMatrix)

class CMatrix:

    _dll = ct.CDLL('./test')
    _dll.CMatrix_new.argtypes = ct.c_int, 
    _dll.CMatrix_new.restype = PCMatrix
    _dll.CMatrix_Get.argtypes = PCMatrix, ct.POINTER(ct.c_int)
    _dll.CMatrix_Get.restype = ct.POINTER(ct.c_float)
    _dll.CMatrix_Set.argtypes = PCMatrix, ct.c_int, ct.c_float
    _dll.CMatrix_Set.restype = None
    _dll.CMatrix_delete.argtypes = PCMatrix, 
    _dll.CMatrix_delete.restype = None

    def __init__(self, i):
        self.obj = self._dll.CMatrix_new(i)

    def Set(self, x, f):
        self._dll.CMatrix_Set(self.obj, x, f)

    def Get(self):
        size = ct.c_int()
        m = self._dll.CMatrix_Get(self.obj, ct.byref(size))
        return m[:size.value]

    def __del__(self):
        self._dll.CMatrix_delete(self.obj)

cm = CMatrix(2)
cm.Set(0, 1.5)
cm.Set(1, 2.5)
print(cm.Get())

Output:

[1.5, 2.5]

There was a bug in the original C++ code that was causing the segmentation fault. After following your example and making a minor correction, I got it working.

Joseph Sible-Reinstate Monica · Accepted Answer · 2021-10-18 01:57:16Z

1

Here's the problem:

lib.CMatrix_Set.argtypes = [c.c_int, c.c_int, c.c_int, c.c_float]

You forgot the entry for the self.obj argument. Change it to this:

lib.CMatrix_Set.argtypes = [c.c_void_p, c.c_int, c.c_int, c.c_int, c.c_float]

Also, you should probably set that as the restype of the other functions you call, so that the return value doesn't get truncated on systems where int is narrower than void *.

Your other option is to make a Python class that inherits from ctypes.Structure corresponding to your C++ class, and then use a pointer to that as the type, but then you'd have to be very careful with your C++ code or the layout might end up different than it would be in C, and what Python is expecting.

answered Oct 18, 2021 at 1:57

Joseph Sible-Reinstate Monica

49k7 gold badges55 silver badges118 bronze badges

6 Comments

Tom Spargo Over a year ago

Now it immediately does a segmentation fault.

Joseph Sible-Reinstate Monica Over a year ago

@TomSpargo When you used a debugger, what did it say triggered the segfault?

Tom Spargo Over a year ago

'Process finished with exit code 139 (interrupted by signal 11: SIGSEGV'

Joseph Sible-Reinstate Monica Over a year ago

@TomSpargo You're not using the debugger right. If you use it right, it'll pause the program when the segfault happens instead of letting it exit.

Tom Spargo Over a year ago

Put screenshots of variable values in the initial post. The debugger just stops, no error message.

|

Collectives™ on Stack Overflow

Using ctypes to call a C++ method with parameters from Python results in "Don't know how to convert parameter" error

2 Answers 2

1 Comment

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related