Compiling multiple C files with Cython

Question

How do I compile using Cython, a C function from file simulate_fast.c, where this also depends on further C files matrix.c and random_generator.c. This seems like it must be a common use of Cython but after reading through the documentation, I still cannot figure out how to do it. My directory contains the following files

matrix.c
matrix.h
random_generator.c
random_generator.h
simulate_fast.c
simulate_fast.h
test.c
simulate_fast_c.pyx
setup.py

The matrix.c and random_generator.c files contain standalone functionality. The simulate_fast.c file uses both of these and contains the function I want to be exposed to Python, simulate(). The test.c file tests that all the C functionality runs correctly i.e. I can execute

$ gcc test.c simulate_fast.c matrix.c random_generator.c -o test

to compile into a test executable that works.

My issue now is trying to compile this with Cython. My .pyx file is

cimport cython

cdef extern from "simulate_fast.h":
    int simulate()

def simulate_cp():
    return simulate()

I then use the basic setup.py

from distutils.core import setup
from Cython.Build import cythonize
from Cython.Distutils import build_ext
import numpy as np

setup(
    name='simulate_fast',
    ext_modules = cythonize(["simulate_fast_c.pyx"]),
    include_dirs=[]
)

However if I try and compile this using

python3 setup.py build_ext --inplace

I get an error

In file included from simulate_fast_c.c:492:0:
simulate_fast.h:89:28: error: field ‘RG’ has incomplete type
    struct RandomGenerator RG;

where structure RandomGenerator is declared in random_generator.h.

How do I tell the compiler we must also consider the matrix and random_generator files when compiling.

Update

If doing as ead said in the comments, I include random_generator.h and matrix.h in simulate_fast.h then the program now compiles. However when I try to import the simulate_fast_c module in Python I get an ImportError:

undefined symbol: simulate

Further, if I change the extern declaration line in simulate_fast.pyx to

cdef extern from "simulate_fast.c":
    int simulate()

I then get an import error

undefined symbol: get_random_number

which is a function in random_generator.h

It looks as if simulate_fast.h is not self-contained, include random_generator.h in simulate_fast.h — ead
– ead, Commented Sep 19, 2017 at 14:36
That's true although I still get an error so I can update the question. Out of curiosity, why would I need to include the random_generator.h in simualte_fast.h header when using Cython but not with gcc? — rwolst
– rwolst, Commented Sep 19, 2017 at 14:47
One way or another simulate_fast.c includes random_generator.h - it does not care whether it comes from simulate_fast.h or somewhere else... — ead
– ead, Commented Sep 19, 2017 at 15:05
I understand that (why it works), what I don't understand is why it didn't compile before with Cython yet was fine when compiling test.c with gcc. — rwolst
– rwolst, Commented Sep 19, 2017 at 15:09
Look at my setup.py in this answer: stackoverflow.com/a/46271641/5769463 you need to add all c-files to sources (but do not need the whole c++ stuff) — ead
– ead, Commented Sep 19, 2017 at 15:09

Pierre de Buyl · Accepted Answer · 2017-09-19 15:12:13Z

1

The cythonized module must either be linked to a shared library containing the compiled C code or embed it. One way to do the latter is to list the C sources as being an "Extension", then pass this extension to the cythonize command, as mentioned in Cython's documentation

The example setup.py file in the link can be summarized by (modulo the imports):

setup(
    ext_modules = cythonize(Extension("simulate_fast", ["matrix.c", "random_generator.c", "simulate_fast_c.pyx"]))
)

answered Sep 19, 2017 at 15:12

Pierre de Buyl

7,3232 gold badges18 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Compiling multiple C files with Cython

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related