Include file from root level in setup.py

Question

I'am trying to build a python package using a bamboo pipeline within our working environment.

To trigger the build I have to include a file build.info.txt that contains the version number. This file is at the top level in the repository, thus the repo structure is the following

build.info.txt
setup.py
MANIFEST.in
README.md
package/
   __init__.py
   file1.py
   file2.py

The file build.info.txt has the version number

version=x.y.z

I would like to use this file to trigger the version number, thus in the __init__.py I included a simple parser

from pathlib import Path
path = Path(__file__).absolute().parent / "build.info.txt"
__version__ = open(path).readline()[0].split("=")[-1]

This works fine in local mode. However, when I create a package and push it on our pypi server I get the following error at runtime (i.e. pip install package works):

FileNotFoundError: [Errno 2] No such file or directory: '</PATH/TO/ENVIRONMENT>/python3.8/site-packages/build.info.txt'

I changed the behavior of setup.py (I use setuptools) including:

setup(**other_params, package_data={"": ["../build.info.txt"]}, include_package_data=True)

while MANIFEST.in looks like

include build.info.txt

The issue is that build.info.txt using pip install is located at top level:

</PATH/TO/ENVIRONMENT>/python3.8/site-packages/build.info.txt

this file is shared among several packages, while I would like to have the file in

</PATH/TO/ENVIRONMENT>/python3.8/site-packages/<PACKAGE>/build.info.txt

(then I have to manage the parser in __init__ in order to parse the right file, but this is straightforward)

How can I manage this?

Thank you

sinoroc · Accepted Answer · 2020-10-08 08:46:44Z

1

My recommendation is to use importlib.metadata to read the version of the project.

There was an attempt to formalize and standardize the convention of setting __version__ in PEP 396, but it never reached full acceptance. And also nowadays this habit goes away, because the need for it isn't that pressing anymore, thanks to importlib.metadata. So you could remove __version__ entirely and 3rd party pieces of code that want to know the version number of your application or library can use importlib.metadata.version('MyProject') to read it.

If you insist on having __version__ in your library, you can turn this around and do the following in your __init__.py:

__version__ = importlib.metadata.version('MyProject')

Asides:

If you insist on working with the file build-info.txt, then you could symlink it in the package, for example as package/build-info.txt (linking to build-info.txt), and __init__.py could read that file at run-time.
- Or, as suggested in the comments, symlink build-info.txt to package/build_info.py, and in __init__.py: from build_info import version as __version__ (this assumes that the content of that file is also valid Python syntax).
You should not use __file__ to read package data resources, this is unreliable, use importlib.resources instead (it will do this in a much cleaner way, and if needed take care of extracting the files in case the package is zipped, etc.).
There are tools that can take care of all this version number issues, setuptools-scm comes to mind and might fit your needs.

edited Oct 8, 2020 at 8:46

answered Oct 7, 2020 at 10:44

sinoroc

23.4k4 gold badges59 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Jean-Didier Over a year ago

Another suggestion for the "asides", package/build-info.txt could be renamed package/version.py, so the line in __init__.py becomes as simple as: from .version import version as __version__. And then have setup.py automatically generate this file using e.g. scm info.

Jean-Didier Over a year ago

To complete my previous comment, see for instance numpy's setup.py, esp. the function write_version_py

sinoroc Over a year ago

@demi-lune From what I understood of the question, the name and location of the file is a given. I understood it as it is a convention from their "bamboo" pipeline environment, and it can't be changed. I might be wrong (my answer is also a bit misleading on that, I will fix it). -- But yeah, ./setup.py could read ./build-info.txt and generate ./package/version.py, that could work.

Andrea Di Iura Over a year ago

@sinoroc, you are right: name and the path are fixed.

theannouncer · Accepted Answer · 2024-06-19 03:40:00Z

0

Another alternative, which I use instead is to add a MANIFEST.in file. setup.py already includes things from the root level, like READMEs there is a valid use case to include files at the root level.

For the example above a MANIFEST.in could look like:

include build.info.txt

answered Jun 19, 2024 at 3:40

theannouncer

1,22418 silver badges28 bronze badges

Collectives™ on Stack Overflow

Include file from root level in setup.py

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related