0

I am trying to compile an extension module where I define a custom type. However, I got hold of some python 2 code for this purpose, but I cannot for the life of me convert it into a python 3 module.

So far, I have tweaked the original code I found in the book 'Python in a nutshell' in order to attempt to compile it for python 3, by merging some bits from https://docs.python.org/3/extending/newtypes_tutorial.html,but I am still getting nowhere. I know the 3rd edition of this book has a python 3 version of the code I am trying to compile but I don't have access to that edition of the book.

So, here is the module source code test.c:

#include "Python.h"
#include "structmember.h"

/* per-instance data structure */
typedef struct {
    PyObject_HEAD
    int first, second;
} intpair;

static int
intpair_init(PyObject *self, PyObject *args, PyObject *kwds)
{
 static char* nams[] = {"first","second",NULL};
 int first, second;
 if(!PyArg_ParseTupleAndKeywords(args, kwds, "ii", nams, &first, &second))
 return -1;
 ((intpair*)self)->first = first;
 ((intpair*)self)->second = second;
 return 0;
}

static void
intpair_dealloc(PyObject *self)
{
 self->ob_type->tp_free(self);
}


static PyObject* nothing_method(PyObject* self, PyObject* args)
{
    printf("This does literally nothing!\n");

    Py_RETURN_NONE;
}

static PyMemberDef intpair_members[] = {
 {"first", T_INT, offsetof(intpair, first), 0, "first item" },
 {"second", T_INT, offsetof(intpair, second), 0, "second item" },
 {NULL}
};

static PyTypeObject t_intpair = {
 PyObject_HEAD_INIT(0) /* tp_head */
 0, /* tp_internal */
 "test.intpair", /* tp_name */
 sizeof(intpair), /* tp_basicsize */
 0, /* tp_itemsize */
 intpair_dealloc, /* tp_dealloc */
 0, /* tp_print */
 0, /* tp_getattr */
 0, /* tp_setattr */
 0, /* tp_compare */
 0, /* tp_as_number */
 0, /* tp_as_sequence */
 0, /* tp_as_mapping */
 0, /* tp_hash */
 0, /* tp_call */
 0, /* tp_str */
 PyObject_GenericGetAttr, /* tp_getattro */
 PyObject_GenericSetAttr, /* tp_setattro */
 0, /* tp_as_buffer */
 Py_TPFLAGS_DEFAULT,
 "two ints (first,second)",
 0, /* tp_traverse */
 0, /* tp_clear */
 0, /* tp_richcompare */
 0, /* tp_weaklistoffset */
 0, /* tp_iter */
 0, /* tp_iternext */
 0, /* tp_methods */
 intpair_members, /* tp_members */
 0, /* tp_getset */
  0, /* tp_base */
 0, /* tp_dict */
 0, /* tp_descr_get */
 0, /* tp_descr_set */
 0, /* tp_dictoffset */
 intpair_init, /* tp_init */
 PyType_GenericAlloc, /* tp_alloc */
 PyType_GenericNew, /* tp_new */
 PyObject_Del, /* tp_free */
};


static PyMemberDef Noddy_members[] = {
    {"PI", T_INT, 3, 0, "noddy number"},
    {NULL, NULL, 0, NULL}
};

static PyMethodDef CosMethods[] =
{
     {"does_nothing", nothing_method, METH_VARARGS, "This really does nothing"},
     {NULL, NULL, 0, NULL}
};

static PyModuleDef testmodule = {
    PyModuleDef_HEAD_INIT,
    .m_name = "test",
    .m_doc = "Example module that creates an extension type.",
    .m_size = -1,
    CosMethods,
};

PyMODINIT_FUNC
PyInit_test(void)
{
    PyObject *m;
    if (PyType_Ready(&t_intpair) < 0)
        return NULL;

    m = PyModule_Create(&testmodule);
    if (m == NULL)
        return NULL;

    Py_INCREF(&t_intpair);
    PyModule_AddObject(m, "Custom", (PyObject *) &t_intpair);
    return m;
}

And the file setup.py:

from distutils.core import setup, Extension
setup(name="test", version="1.0",
      ext_modules=[
         Extension("test", ["test.c"]),
         ])

I can successfully compile this module using python3 setup.py build_ext --inplace, however, when I import this module in python 3 I get the following error:

Python 3.7.2 (default, Jan 10 2019, 23:51:51) 
[GCC 8.2.1 20181127] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import test
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: Type does not define the tp_name field.

I do not understand this error message, since there is a tp_name field "test.intpair" defined in PyTypeObject. What am I missing here?

Also, what is necessary to change in this code to make it compile for python 3?

Thanks!

0

2 Answers 2

2

I'm pretty sure the flaw is the line 0, /* tp_internal */ in:

 PyObject_HEAD_INIT(0) /* tp_head */
 0, /* tp_internal */
 "test.intpair", /* tp_name */

If you search for this line the only references to it on the internet are this question and the book you're using - it's completely non-standard and I have no idea where it's come from. I half suspect that the Python 2 code uses PyVarObject and then uses tp_internal to fill in the size rather than PyVarObject_HEAD_INIT, but I can't see the relevant page of the book...

The current documentation recommends using C99 initialization like

static PyTypeObject t_intpair = {
    PyObject_HEAD_INIT(0)
    .tp_name = "test.intpair",
    // etc
 };

so that getting the order of the attributes right no longer matters (anything you don't specify is 0 initialized).


Building your code provides some pretty clear warnings about mismatched types:

warning: initialization of ‘long int’ from ‘char *’ makes integer from pointer without a cast [-Wint-conversion] "test_mod.intpair", /* tp_name */`

You should pay attention to these

Sign up to request clarification or add additional context in comments.

Comments

0

Not a direct answer to your question, so posting this is probably against site rules. I hit this page when looking for advice about how to do a similar thing, converting a python2 - C extension module to python3 - C.

This is generally not a trivial endeavor, as the interface changed significantly. I think you will find the resources at this link helpful:

https://docs.python.org/3/howto/cporting.html

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.