Bundling C++ extension headers with a Python package source distribution

Question

I'm writing a Cython wrapper to a C++ library that I would like to distribute as a Python package. I've come up with a dummy version of my package that looks like this (full source here).

$ tree
.
├── bogus.pyx
├── inc
│   └── bogus.hpp
├── setup.py
└── src
    └── bogus.cpp
$
$ cat inc/bogus.hpp 
#ifndef BOGUS
#define BOGUS

class bogus
{
protected:
    int data;

public:
    bogus();
    int get_double(int value);
};

#endif
$
$ cat src/bogus.cpp 
#include "bogus.hpp"

bogus::bogus() : data(0)
{

}

int bogus::get_double(int value)
{
    data = value * 2;
    return data;
}
$ cat bogus.pyx 
# distutils: language = c++
# distutils: sources = src/bogus.cpp
# cython: c_string_type=str, c_string_encoding=ascii

cdef extern from 'bogus.hpp':
    cdef cppclass bogus:
        bogus() except +
        int get_double(int value)

cdef class Bogus:
    cdef bogus b
    def get_double(self, int value):
        return self.b.get_double(value)

With the following setup.py file, I can can confirm that the library installs correctly with python setup.py install and that it works correctly.

from setuptools import setup, Extension
import glob

headers = list(glob.glob('inc/*.hpp'))

bogus = Extension(
    'bogus',
    sources=['bogus.pyx', 'src/bogus.cpp'],
    include_dirs=['inc/'],
    language='c++',
    extra_compile_args=['--std=c++11', '-Wno-unused-function'],
    extra_link_args=['--std=c++11'],
)

setup(
    name='bogus',
    description='Troubleshooting Python packaging and distribution',
    author='Daniel Standage',
    ext_modules=[bogus],
    install_requires=['cython'],
    version='0.1.0'
)

However, when I build a source distribution using python setup.py sdist build, the C++ header files are not included and the C++ extension cannot be compiled.

How can I make sure the C++ header files get bundled with the source distribution?!?!

<rant>

Troubleshooting this has uncovered a tremendously convoluted and inconsistent mess of documentation, suggestions, and hacks, none of which have worked for me. Put a graft line in MANIFEST.in? Nope. The package_data or data_files options? Nope. Python packaging seems to have improved a lot in the last few years, but it is still nigh impenetrable for those of us that don't live and breathe Python packaging!

</rant>

Daniel Standage · Accepted Answer · 2016-09-27 22:47:31Z

3

Short answer

Put include inc/*.hpp in the MANIFEST.in file.

Long answer

Based on various blog posts and SO threads, I had tried the suggestion of declaring the files in a MANIFEST.in file. Following these instructions, I added a graft inc/ line to MANIFEST.in to include the entire directory. This did not work.

However, replacing this line with include inc/*.hpp did work. Arguably this should have been the first thing I tried, but being unfamiliar with the intricacies and warts of setuptools and distutils, I had no reason to expect that graft wouldn't work.

answered Sep 27, 2016 at 22:47

Daniel Standage

8,37419 gold badges75 silver badges123 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Bundling C++ extension headers with a Python package source distribution

1 Answer 1

Short answer

Long answer

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Short answer

Long answer

Comments

Your Answer

Sign up or log in

Post as a guest

Related