Circular dependency issue with setup script

Question

I'm writing a python setup.py script for an own package which needs a version constant from the package. However, the package needs to have some dependencies installed. Therefore I specified install_requires in the setup.py. However, when I generate the package via python setup.py sdist and install it in another project I get a dependency error. What am I doing wrong, here? When I dismiss the import statements from the __init__.py file, the dependencies (in this case pyzmq and jsonpickle, but could be any other dependencies) are installed correctly in the other project.

folder-structure

myproject/
   | setup.py
   | mypackage/
     | __init__.py
     | some_code.py
     | version.py

__init__.py

from . import some_code
from . import version

some_code.py

import jsonpickle
import zmq

from mypackage.version import VERSION

print(f"Version is {VERSION}")

setup.py

import [...]
from distutils.dir_util import remove_tree
from setuptools import setup, find_packages

# Globals definitions used more than one time
PACKAGE_NAME = "mypackage"

[...]

 
from mypackage.version import VERSION  # <<<<---- This is the command which is problematic !!!!!


setup(name = PACKAGE_NAME,
    version = VERSION,
    author = "Me",
    author_email = "[email protected]",
    description='mypackage test',
    include_package_data = True,
    packages=find_packages(),
    install_requires=[
    'pyzmq', 'jsonpickle'
    ],
    zip_safe = False)

In another project

(.venv) user@l-user:~/PycharmProjects/using_mypackage$ python -m pip install mypackage-3.4.6.tar.gz 
Looking in indexes: https://pypi.org/simple
Processing ./mypackage-3.4.6.tar.gz
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 255:
   command: /home/user/PycharmProjects/using_mypackage/.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-4upmamin/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-4upmamin/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-w7plcisg
       cwd: /tmp/pip-req-build-4upmamin/
  Complete output (9 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-req-build-4upmamin/setup.py", line 3, in <module>
      from mypackage.version import VERSION  # <<<<---- This is the command which is problematic !!!!!
    File "/tmp/pip-req-build-4upmamin/mypackage/__init__.py", line 1, in <module>
      from . import some_code
    File "/tmp/pip-req-build-4upmamin/mypackage/some_code.py", line 1, in <module>
      import jsonpickle
  ModuleNotFoundError: No module named 'jsonpickle'

AKX · Accepted Answer · 2022-06-14 09:32:16Z

1

The legacy "canonical" solution has been to use a regexp to find the line with VERSION and ast.literal_eval the string on that line with something like

with open(os.path.join(os.path.dirname(__file__), "mypackage", "__init__.py")) as infp:
    version = ast.literal_eval(
        re.search("^VERSION = (.+?)$", infp.read(), re.M).group(1)
    )

This works until it doesn't (but happily it will break loudly when someone tries to run setup.py).

However, if you don't need setup.py at all (though based on the fact that you're referring to remove_tree and probably using it in the part you've elided, you might?), you could switch altogether to having no imperative setup.py at all, by switching to PEP 517 builds; setuptools has nice machinery to let you just do version = attr:mypackage.VERSION.

packaging.python.org has a great step-by-step tutorial on that. You may also find my setuppy2cfg converter tool useful.

answered Jun 14, 2022 at 9:32

AKX

171k16 gold badges147 silver badges229 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Wör Du Schnaffzig Over a year ago

I think my misunderstanding was that I've overseen that the setup.py script is actually used two times. At first to generate the tar.gz, at second to install the package in the other project. I've overseen the second.

AKX Over a year ago

Yeah, setup.py has this twin (or more) personality. With PEP517-compliant packages and build from PyPI, all you need is python -m build . to get a .tar.gz and a .whl, and modern versions of pip deal fine with PEP517 too, setup.py or no setup.py.

Wör Du Schnaffzig Over a year ago

I tried your script and it provided me a setup.cfg in my test project (the version string is VERSION = "3.4.5"). However in the real project I got the error message: *** Unable to get value for ('metadata', 'version'): malformed node or string: <_ast.Attribute object at 0x7f4582da17f0>. The version in this case is: BLABLA_VERSION = "0.8.90". Did I broke some naming convention?

AKX Over a year ago

It's not perfect in the least – you will need to modify those bits by hand; for your case, add version = attr:blabla.BLABLA_VERSION (if the package is blabla).

Wör Du Schnaffzig Over a year ago

That's OK. I'm glad I can use your tool, anyway. It gave me a good starting point to fiddle out those details. Thanks.

|

Collectives™ on Stack Overflow

Circular dependency issue with setup script

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related