0

I am trying to import an extension module contained inside a nested package. For the purposes of this example, it's the only element of the root module, but the real case has others. When I attempt to test the following module setup using tox (or cibuildwheel/pytest, it's the same result) I get the following error:

ImportError: cannot import name 'dummyext' from 'rootpack'

Here is the code for setup.py:

import setuptools
import numpy

PACKAGE_NAME = 'rootpack'

setuptools.setup(
    name=PACKAGE_NAME,
    version="0.0.1",
    description="Testing extension module",
    packages=[PACKAGE_NAME],
    #ext_package=PACKAGE_NAME,
    ext_modules = [
        setuptools.Extension(
            'rootpack.dummyext',
            ['ext/dummy.c'],
            extra_compile_args=["-std=c99"],
            include_dirs=[numpy.get_include()],
        ),
    ],
    install_requires=[
        'numpy',
    ],
)

And dummy.c:

#include <Python.h>
#include <numpy/arrayobject.h>

static PyObject* dummy_method(PyObject* self, PyObject* args)
{
    npy_int rows = 3;
    npy_int cols = 4;
    npy_intp dims[2] = {rows, cols};
    PyObject* arr_obj = PyArray_ZEROS(2, dims, NPY_INTP, 0);
    int* arr_data = PyArray_DATA(arr_obj);

    for(int i = 0; i < rows; ++i)
    {
        for(int j = 0; j < cols; ++j)
        {
            arr_data[i*cols + j] = i+j;
        }
    }
    return arr_obj;
}

static PyMethodDef dummyext_methods[] = {
    {"dummy_method", dummy_method, METH_VARARGS, "Returns a dummy array"},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef dummyext_module = {
    PyModuleDef_HEAD_INIT,
    "dummyext",
    NULL,
    -1,
    dummyext_methods
};

PyMODINIT_FUNC PyInit_dummyext(void)
{
    PyObject* module = PyModule_Create(&dummyext_module);
    import_array();
    return module;
}

Note that __init__.py files exist in both the rootpack and test directories. There is a trivial test file test/dummy.py that contains the from rootpack import dummyext line triggering the error.

I've tried using ext_package and changing the ext module name to just dummyext, but that results in the same error. If I remove the ext module from the nested hierarchy, it can be imported.

7
  • What is your directory structure? If I have a root directory that contains: i) your setup.py, ii) a directory called rootpack (containing an empty __init__.py), iii) a directory called ext containing dummy.c, then running python setup.py install installs the package correctly, and doing from rootpack import dummyext in a Python terminal (outside that directory) works for me. Commented Dec 19, 2022 at 16:30
  • The structure is as you've guessed. I can do the same thing, assuming I change my working directory to some other place (I guess it's getting confused by the local directory). But why doesn't it work in self-contained environments like tox or cibuildwheel? The goal here is to have a binary distribution, not just a local install. Commented Dec 19, 2022 at 17:26
  • Do you have a MANIFEST.in file that includes include ext/dummy.c? Commented Dec 19, 2022 at 21:27
  • No, what needs to be in that file? The c file? The .so? Commented Dec 19, 2022 at 21:27
  • You might also want to try having pytest-runner as in this question stackoverflow.com/questions/49033350/… Commented Dec 19, 2022 at 21:27

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.