1

I have a project structure as follows:

python-test » tree
.
├── asdf
│   ├── __init__.py
│   ├── mycode2.py
│   ├── mycode.py
│   └── scripts
│       └── __init__.py
├── __init__.py
└── scripts
    ├── __init__.py
    └── mymod.py

asdf/mycode.py contains:

import scripts.mymod

and scripts/mymod.py contains:

print('hello world')

All the __init__.pys are empty.

I would like to import scripts.mymod.

I am running asdf/mycode.py which fails because it is looking inside asdf/scripts instead of the root scripts folder for mymod.

> PYTHONPATH=: python asdf/mycode.py
Traceback (most recent call last):
  File "asdf/mycode.py", line 1, in <module>
    import scripts.mymod
ModuleNotFoundError: No module named 'scripts.mymod'

As a workaround, I can manually modify the path, which I have done in asdf/mycode2.py.

import sys
sys.path = [p for p in sys.path if 'asdf' not in p]
import scripts.mymod

This works correctly:

> PYTHONPATH=: python asdf/mycode2.py
hello world

Is there a way to import scripts.mymod in asdf/mycode.py without modifying the sys.path manually, and without changing the PYTHONPATH which has to be set to :?

1
  • you can use a main entrypoint in the project dir. python-test/main.py if you run the code from main dir. you can import with from asdf.scripts import ... or from scripts import mymod. Commented Oct 29, 2021 at 21:00

1 Answer 1

1

TL;DR

Python imports are weird and cumbersome and you are probably best off either creating an entrypoint in the top of the directory or using a tool to handle the path manipulations.

Python Imports

Python has two different ways of handling imports, the first is the standard: import numpy as np, which is resolved against the environment PYTHONPATH in order until it finds the requested module. Included in that path is the directory of the file that is currently being executed, which you see in your example in that you are needing to manually remove that from the sys.path.

The other way that python handles imports is via relative paths, which always lead with . like from . import a or import .a or from .. import b. Unfortunately, relative import only work if the file that they are in is not being directly run (This happens if you have two files within a module where one of them imports an object from another, but both of them are intended to be imported by and external script). This is because python uses the builtin name global to resolve the relative path, which if the file is being directly run from a shell gets overridden as "__main__".

Consider a file structure:

.
├── a.py
└── b.py

where a.py is

import b

and b.py is

print(__name__)

If you run:

python3 b.py    # prints "__main__"
python3 a.py    # prints "b"

So, if you want to be able to import the scripts/mymod.py from asdf/mycode2.py, you could put change the import to:

from .. import scripts.mymod

But, if you do that, you will not be able to run the asdf/mycode2.py file directly, you will need to create a third file somewhere else that imports asdf/mycode2.py and run that third script (these files are called entrypoints). For example:

python-test » tree
.
├── asdf
│   ├── __init__.py
│   ├── mycode2.py
│   ├── mycode.py
│   └── scripts
│       └── __init__.py
├── __init__.py
├── entrypoint.py
└── scripts
    ├── __init__.py
    └── mymod.py

where entrypoint.py is

import asdf.mycode2

An Alternative Approach

The alternative approach would be to develop a tool which handles manipulating python's sys.path to allow relative imports even when the current file is being run a "__main__". Full disclosure, I am currently working on such a tool based off nodejs's require function, called repyrer, which is pip-installable as

pip3 install repyrer

It allows you to do this kind of thing:

from repyrer import repyre

mymod = repyre('../scripts/mymod')

which will work even if you run the file directly.

Hope that helps!

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.