How do I install a Python extension module using distutils?

Question

I'm working on a Python package named "lehmer" that includes a bunch of extension modules written in C. Currently, I have a single extension module, "rng". I am using Python's Distutils to build and install the module. I can compile and install the module, but when I try to import the module using import lehmer.rng or from lehmer import rng, the Python interpreter throws an ImportError exception. I can import "lehmer" fine.

Here are the contents of my setup.py file:

from distutils.core import setup, Extension

exts = [Extension("rng", ["lehmer/rng.c"])]

setup(name="lehmer",
      version="0.1",
      description="A Lehmer random number generator",
      author="Steve Park, Dave Geyer, and Michael Dippery",
      maintainer="Michael Dippery",
      maintainer_email="[email protected]",
      packages=["lehmer"],
      ext_package="lehmer",
      ext_modules=exts)

When I list the contents of Python's site-packages directory, I see the following:

th107c-4 lehmer $ ls /scratch/usr/lib64/python2.5/site-packages/lehmer
__init__.py  __init__.pyc  rng.so*

My PYTHONPATH environment variable is set correctly, so that's not the problem (and as noted before, I can import lehmer just fine, so I know that PYTHONPATH is not the issue). Python uses the following search paths (as reported by sys.path):

['', '/scratch/usr/lib64/python2.5/site-packages', '/usr/lib/python25.zip', '/usr/lib64/python2.5', '/usr/lib64/python2.5/plat-linux2', '/usr/lib64/python2.5/lib-tk', '/usr/lib64/python2.5/lib-dynload', '/usr/lib64/python2.5/site-packages', '/usr/lib64/python2.5/site-packages/Numeric', '/usr/lib64/python2.5/site-packages/PIL', '/usr/lib64/python2.5/site-packages/SaX', '/usr/lib64/python2.5/site-packages/gtk-2.0', '/usr/lib64/python2.5/site-packages/wx-2.8-gtk2-unicode', '/usr/local/lib64/python2.5/site-packages']

Update

It works when used on an OpenSUSE 10 box, but the C extensions still fail to load when tested on Mac OS X. Here are the results from the Python interpreter:

>>> sys.path
['', '/usr/local/lib/python2.5/site-packages', '/opt/local/lib/python25.zip', '/opt/local/lib/python2.5', '/opt/local/lib/python2.5/plat-darwin', '/opt/local/lib/python2.5/plat-mac', '/opt/local/lib/python2.5/plat-mac/lib-scriptpackages', '/opt/local/lib/python2.5/lib-tk', '/opt/local/lib/python2.5/lib-dynload', '/opt/local/lib/python2.5/site-packages']
>>> from lehmer import rng
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name rng
>>> import lehmer.rngs
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named rngs
>>> import lehmer.rng 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named rng
>>> from lehmer import rngs
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name rngs

Actually, you don't know that PYTHONPATH is set correctly. Your local directory is always first on the sys.path list. Could you run Python and run 'import sys; print sys.path` and add that to your question? — S.Lott
– S.Lott, Commented Nov 19, 2008 at 18:29
sorry for the stupid question, but what exactly is the ImportError's message? iirc ImportErrors are thrown for problems when loading extension modules so it could be caused by forgetting to link a library fex. — TFKyle
– TFKyle, Commented Nov 19, 2008 at 18:37
Whoa! When I tried to import it again, it magically worked. I swear I didn't touch anything, or do anything differently than what I had been doing...although I guess I must've done something differently. Cookie to anyone who might offer an explanation as to why. :) — mipadi
– mipadi, Commented Nov 19, 2008 at 18:42
@mipadi: Yes, '' is usually first (not always, but usually) and that means that the local directory is examined for your module before any other directory is checked. Developers often test their imports in the same place they're developing, masking PATH problems. — S.Lott
– S.Lott, Commented Nov 19, 2008 at 18:55

mipadi · Accepted Answer · 2009-01-12 16:41:37Z

4

For the record (and because I am tired of seeing this marked as unanswered), here were the problems:

Since the current directory is automatically added to the Python packages path, the interpreter was first looking in the current directory for packages; since some C modules were not compiled in the current directory, the interpreter couldn't find them. Solution: Don't launch the interpreter from the same directory in which your working copy of the code is stored.
Distutils did not install the module with the correct permissions on OS X. Solution: Fix the permissions.

answered Jan 12, 2009 at 16:41

mipadi

414k91 gold badges538 silver badges488 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Setjmp Over a year ago

Is there a way to make sure the installation proceeds with proper permissions instead of changing them after the fact?

merwok Over a year ago

Yes, please define what the correct permissions should be and what distutils does. If there is a distutils bug, it should be reported to bugs.python.org.

Collectives™ on Stack Overflow

How do I install a Python extension module using distutils?

Update

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

Update

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related