How to statically link a library when compiling a python module extension

Question

I would like to modify a setup.py file such that the command "python setup.py build" compiles a C-based extension module that is statically (rather than dynamically) linked to a library.

The extension is currently dynamically linked to a number of libraries. I would like to leave everything unchanged except for statically linking to just one library. I have successfully done this by manually modifying the call to gcc that distutils runs, although it required that I explicitly listed the dependent libraries.

Perhaps this is too much information, but for clarity this is the final linking command that was executed during the "python setup.py build" script:

gcc -pthread -shared -L/system/lib64 -L/system/lib/ -I/system/include build/temp.linux-x86_64-2.7/src/*.o -L/system/lib -L/usr/local/lib -L/usr/lib -ligraph -o build/lib.linux-x86_64-2.7/igraph/core.so

And this is my manual modification:

gcc -pthread -shared -L/system/lib64 -L/system/lib/ -I/system/include build/temp.linux-x86_64-2.7/src/*.o -L/system/lib -L/usr/local/lib -L/usr/lib /system/lib/libigraph.a -lxml2 -lz -lgmp -lstdc++ -lm -ldl -o build/lib.linux-x86_64-2.7/igraph/core.so

Section 2.3.4 of Distributing Python Modules discusses the specification of libraries, but only "library_dirs" is appropriate and those libraries are dynamically linked.

I'm using a Linux environment for development but the package will also be compiled and installed on Windows, so a portable solution is what I'm after.

Can someone tell me where to look for instructions, or how to modify the setup.py script? (Thanks in advance!)

I'm new to StackOverflow, so my apologies if I haven't correctly tagged this question, or if I have made some other error in this posting.

SWPhantom · Accepted Answer · 2023-12-06 15:49:31Z

39

6 - 7 years later, static linking with Python extensions is still poorly documented. As the OP points out in a comment, the usage is OS dependent.

On Linux / Unix

Static libraries are linked just as object files and should go with the path and its extension into extra_objects.

On Windows

The compiler sees if the linked library is static or dynamic and the static library name goes to the libraries list and the directories to library_dir

Solution for both platforms

For the example below, I will use the same library scenario as OP, linking igraph static and z, xml2 and gmp dynamic. This solution is a bit hackish, but at least does the right thing for each platform.

static_libraries = ['igraph']
static_lib_dir = '/system/lib'
libraries = ['z', 'xml2', 'gmp']
library_dirs = ['/system/lib', '/system/lib64']

if sys.platform == 'win32':
    libraries.extend(static_libraries)
    library_dirs.append(static_lib_dir)
    extra_objects = []
else: # POSIX
    extra_objects = ['{}/lib{}.a'.format(static_lib_dir, l) for l in static_libraries]

ext = Extension('igraph.core',
                 sources=source_file_list,
                 libraries=libraries,
                 library_dirs=library_dirs,
                 include_dirs=include_dirs,
                 extra_objects=extra_objects)

On MacOS

I guess this also works for MacOS (using the else path), but I have not tested it.

edited Dec 6, 2023 at 15:49

SWPhantom

6885 silver badges19 bronze badges

answered Mar 6, 2018 at 20:14

oekopez

1,0901 gold badge12 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Bob Barcklay Over a year ago

Thanks for this. I can confirm that it does indeed work on MacOS. The top answer did not work for me.

detly · Accepted Answer · 2011-01-05 03:01:44Z

16

If all else fails, there's always the little-documented extra_compile_args and extra_link_args options to the Extension builder. (See also here.)

You might need to hack in some OS-dependent code to get the right argument format for a particular platform though.

answered Jan 5, 2011 at 3:01

detly

30.6k20 gold badges101 silver badges161 bronze badges

3 Comments

Matthew Walker Over a year ago

I added an "extra_objects" parameter to the Extension instance specifying libigraph.a (contrary to the documentation, which says dont add the file's extension) and added all the dependencies to the "libraries" parameter. This does produce a statically linked library. But, as you point out this is very OS-dependent so if this is the only option then I'll have to pay attention to what Windows wants. Thanks for your assistance.

detly Over a year ago

@Matthew Walker - if you do find a better solution, you should post it back here, 'cause I'd be very interested :)

Mike T Over a year ago

this works: Extension('my.mod', ['src/file.c'], extra_link_args=['-static'])

Brian Bruggeman · Accepted Answer · 2012-06-05 18:52:14Z

0

Any possibility that this might work?

g++ -Wl,-Bstatic -lfoo -Wl,-Bdynamic -lbar -Wl,--as-needed

answered Jun 5, 2012 at 18:52

Brian Bruggeman

5,3343 gold badges43 silver badges57 bronze badges

Collectives™ on Stack Overflow

How to statically link a library when compiling a python module extension

3 Answers 3

On Linux / Unix

On Windows

Solution for both platforms

On MacOS

1 Comment

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

On Linux / Unix

On Windows

Solution for both platforms

On MacOS

1 Comment

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related