Updating python variable from c

Question

I am having an intermittent error causing my Python module to crash, and I'm assuming it's because of a memory error occurring by not getting the refcounts correct in the c code. I have a bit of code that gets a response at a random time from a remote location. Based on the data received, it needs to update a data variable which I should have access to in Python. What's the best way to accomplish this? The following code runs most of the time, and it works correctly when it does, but when it doesn't it crashes Python (bringing up the visual studio debug box). Thanks.

if (event == kResponseEvent) {
    list = PyList_New(0);

    for (i = 0; i < event->count; i++) {
        PyList_Append(list, Py_BuildValue("{s:i, s:s}",
                                          "id", event->id,
                                          "name", event->name));
    }

    PyModule_AddObject(module, "names", list);
}

I'm posting this as a comment rather than an answer but I've generally felt that compiling the C part as a shared lib and using ctypes was more manageable than using the extension library. — Noufal Ibrahim
– Noufal Ibrahim, Commented Feb 9, 2010 at 18:09
Thanks - I will look into this, especially since I'm much more well versed in Python than C. — Jeff
– Jeff, Commented Feb 9, 2010 at 18:15
Cython is an even slicker option to provide Python bindings for a C library. — Mike Graham
– Mike Graham, Commented Feb 9, 2010 at 18:18

Ignacio Vazquez-Abrams · Accepted Answer · 2010-02-09 18:07:29Z

1

PyModule_AddObject() steals a reference. As such, you should not be decrefing list after.

answered Feb 9, 2010 at 18:07

Ignacio Vazquez-Abrams

804k160 gold badges1.4k silver badges1.4k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jeff Over a year ago

I've updated the code per your suggestion (that was actually how I originally had it), but it still crashes randomly.

Thomas Wouters · Accepted Answer · 2010-02-09 21:02:22Z

1

PyList_New() can return NULL to indicate an error, which you aren't checking for. Py_BuildValue() can return NULL to indicate an error, which you aren't checking for. PyList_Append() can return -1 to indicate an error, which you're also not checking for. PyList_Append() doesn't steal the reference, so you're leaking the reference to the dict returned by Py_BuildValue(). The latter may be causing you to run out of memory, which can cause Py_BuildValue() or PyList_Append() to fail, and your failure to handle the error can cause a crash.

(Something else can also cause Py_BuildValue() or PyList_Append() to fail, but that's hard to guess at from just this snippet.)

answered Feb 9, 2010 at 21:02

Thomas Wouters

134k23 gold badges153 silver badges123 bronze badges

1 Comment

Jeff Over a year ago

Thanks for the info related to the memory management. Interestingly enough, when I manually called the command to get a response, it never failed, but when I ran it from a for loop to hit it really quickly, it crashed. This leads me to believe that there may be something else wrong in the system that's actually causing the crash. I will look more into this issue.

Collectives™ on Stack Overflow

Updating python variable from c

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related