Segfault in multithreaded python extension in C

Question

I have a very trimmed down example that creates a segfault that I can't seem to get rid of. Python script calls a C function in an extension, which creates a new thread using pthreads. I use PyGILState_Ensure and PyGILState_Release around my python call (PyRun_SimpleString) in the new thread, but perhaps I'm not using them correctly or have missed some other step. Commenting out the python calls in the receive_audio function, the segfault no longer happens. Any ideas?

Output:

python lib/test.py
(Main Thread) initmodule complete
(Main Thread) Calling run_thread()
(Main Thread) Creating thread
(New Thread) In receive_audio() - acquiring GIL
(New Thread) python print!
(New Thread) In receive_audio() - released GIL
(New Thread) Looping 0
Segmentation fault

C Code as follows:

PyMODINIT_FUNC streamaudio() {
    PyObject *m = Py_InitModule("streamaudio", methods);
    PyEval_InitThreads();
    mainThreadState = PyThreadState_Get();
    PyEval_ReleaseLock();
    printf("(Main Thread) initmodule complete\n");
}

static PyObject* run_thread(PyObject* self, PyObject* args)
{
    int ok, stream_id;
    PyGILState_STATE gstate;
    gstate = PyGILState_Ensure();
    ok = PyArg_ParseTuple(args, "i", &stream_id);
    PyRun_SimpleString("print '(Main Thread) Creating thread'\n");
    int rc = pthread_create(&thread, NULL, receive_audio, (void*)stream_id);
    PyRun_SimpleString("print '(Main Thread) Thread created'\n");
    PyGILState_Release(gstate);
    return Py_BuildValue("i", rc);
}

void* receive_audio(void *x)
{
    printf("(New Thread) In receive_audio() - acquiring GIL\n");
    PyGILState_STATE gstate;
    gstate = PyGILState_Ensure();
    PyRun_SimpleString("print '(New Thread) python print!'\n");
    PyGILState_Release(gstate);
    printf("(New Thread) In receive_audio() - released GIL\n");
    int i;
    for (i = 0; i < 100; i++) {
    printf("(New Thread) Looping %d\n", i);
    sleep(1);
    }
}

One note - the PyGILState_Ensure() and PyGILState_Release calls around the pthread_create function seem to be inconsequential. I think that is to be expected since it is being run by the main thread. — cursemyziti
– cursemyziti, Commented Mar 19, 2010 at 16:37

Will Robinson · Accepted Answer · 2010-03-19 18:20:23Z

3

I'm not positive that this will be relevant to your question, but one thing that looks suspicious is the PyEval_ReleaseLock() call in your module initializer function. I doubt that Python expects your module initializer to release the GIL out from underneath it, and a quick look at some example code here doesn't show anything along these lines. Could you try removing that PyEval_ReleaseLock() call and tell us what happens?

BTW, I agree that the PyGILState_*() calls within run_thread() should have no effect; you should be able to just remove them.

answered Mar 19, 2010 at 18:20

Will Robinson

2,3191 gold badge14 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

cursemyziti Over a year ago

Nice! That did it. I was following code from here: linuxjournal.com/article/3641?page=0,2 His example is for embedding python in C, whereas I'm doing the reverse, so that call is disastrous. Thanks!

Collectives™ on Stack Overflow

Segfault in multithreaded python extension in C

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related