Can't load python extension module that exists

Question

I'm trying to use thanos, which creates an extension module at runtime. Thanos is constantly failing as it cannot import that said extension it created at runtime.

I am baffled as to why this is the case.

At runtime (after compilation) the "cutils_ext" dir contains cutils_ext.so

import cutils_ext works, but import cutils_ext.cutils_ext fails with

ImportError: No module named cutils_ext

I've tested with with imp directly. Say that cutils is the imported cutils_ext package. When I do:

imp.find_module('cutils_ext', cutils.__path__)

I again get the :

ImportError: No module named cutils_ext

I am completely baffled. Is there any way to debug this? I cannot run python in verbose mode due to my inability to start the python interpreter (using picloud), but if I could enable verbose at runtime, that would help (haven't figured out how to do that).

Is the Python version that Thanos linked its module with the same as you are using? For example, the entry point protocol differes between Python 2 and Python 3. — cdarke
– cdarke, Commented Jun 29, 2012 at 7:03
Yes, all are python2.7. I was even able to download the extension to my own computer and load it directly. Either way, it seems the find_module is failing, even before the load occurs — UsAaR33
– UsAaR33, Commented Jun 29, 2012 at 22:59
Wow, my bad. Looks like I had another cutils_ext elsewhere on sys,path and Theano was preloading it. Filed a bug on Theano ( github.com/Theano/Theano/issues/728) — UsAaR33
– UsAaR33, Commented Jun 30, 2012 at 2:16

score 3 · Accepted Answer · 2012-06-29 10:18:54Z

3

could you try:

import cutils_ext
import cutils_ext.cutils_ext

I don't know why, but sometimes I also have to import a module in order to import the sub modules in it.

Or maybe, just maybe, the second cutils_ext doesn't exist at all in the first cutils_ext?

EDIT: Wait a minute.. could you please post your folder structure and what the PYTHONPATH is containing?

I guess you have something like:

upper_folder (a folder)
    cutils_ext (folder)
        __init__.py (file)
        cutils_ext.so (obviously file)

if your PYTHONPATH points to upper_folder then you should be able to do what you have posted Note that if you have this situation the __ init __.py must be there.

If however your PYTHONPATH is pointing directly to the cutils_ext folder, then I supposed you can only import the file, so cutils_ext.cutils_ext doesn't make sense for the interpreter (unless you have a variable called cutils_ext in cutil_ext.so...)

edited Jun 29, 2012 at 10:18

answered Jun 29, 2012 at 10:12

user1336619

Sign up to request clarification or add additional context in comments.

1 Comment

UsAaR33 Over a year ago

"Or maybe, just maybe, the second cutils_ext doesn't exist at all in the first cutils_ext?" Good call. It was very subtle as theano was preloading my cutils_ext package from another location, then patching sys.path. cutils_ext therefore referred to the wrong package. ugh.. thanks for the post.. helped a lot!

Collectives™ on Stack Overflow

Can't load python extension module that exists

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related