Python adaptor for working around relative imports in my QGIS plugin?

Question

I am writing a QGIS plugin. During early development, I wrote and tested the Qt GUI application independently of QGIS. I made use of absolute imports, and everything worked fine.

Then, I had to adapt everything to the quirks of QGIS. I can't explain why and haven't been able to find any supporting documentation, but nonetheless: apparently, QGIS needs or strongly prefers relative imports. The majority of other plugins I've looked at (completely anecdotal of course) have all used a flat plugin directory and relative imports.

My team decided to keep the hierarchical structure. The plugin now works in QGIS, with the same hierarchical structure, using relative imports.

My goal: I would like to still be able to run the GUI independently of QGIS, as it does not (yet) depend on any aspects of QGIS. With the relative imports, this is completely broken.

My project directory has the following hierarchy:

.
├── app
│   └── main.py
├── __init__.py
├── justfile
├── metadata.txt
├── plugin.py
├── README.md
├── resources
│   ├── name_resolver.py
│   └── response_codes.py
├── processing
│   ├── B_processor.py
│   ├── A_processor.py
│   ├── processor_core.py
│   ├── processor_interface.py
│   ├── processor_query_ui.py
│   └── processor_query_ui.ui
└── tests
    ├── __init__.py
    ├── test_A_processor.py
    └── test_processor_core.py

The app directory and its main.py module are where I'm trying to run the GUI independently of QGIS. The GUI is in processing/processor_query_ui.py.

app/main.py is as follows:

if __name__ == "__main__":
    import sys
    from PyQt5 import QtWidgets
    from processing.processor_query_ui import UI_DataFinderUI

    app = QtWidgets.QApplication(sys.argv)
    ui = UI_DataFinderUI()
    ui.show()
    sys.exit(app.exec_())

When running main from the top level, all imports within main.py work:

$ python app/main.py

What does NOT work are the subsequent imports:

Traceback (most recent call last):
  File "/path/to/app/main.py", line 4, in <module>
    from processing.processor_query_ui import UI_DataFinderUI
  File "/path/to/processing/processor_query_ui.py", line 2, in <module>
    from .A_processor import Aprocessor
  File "/path/to/processing/A_processor.py", line 5, in <module>
    from ..resources.response_codes import RESPONSE_CODES
ImportError: attempted relative import beyond top-level package

This shows that all imports in main.py are working correctly. But when processor_query_ui tries to do its imports, those ones fail.

I have tried adding __init__.py files to all first level directories as well (e.g. {app,resources,processing}/__init__.py) to no vail. Running python -m app/main{.py} doesn't work, though I didn't really expect it to.

For pytest to work, the tests directory must have an init.py file; then, pytest works as either pytest or python -m pytest.

My goal is to be able to run the GUI in processing/processor_query_ui.py as a standalone app, by writing some kind of adaptor such that I do not have to change the current directory structure or relative imports (which, as above, make QGIS happy).

Any advice is greatly appreciated.

Mr_and_Mrs_D · Accepted Answer · 2025-01-25 16:36:57Z

2

To add to the accepted answer you should be running your project from the parent of the app folder (as this folder still contains py code) using the -m switch as in:

$ python -m app.main # note no py

This tells python that app is a package - your "top-level package" mentioned in the traceback - containing a module named main. Now python will scan your top level package for subpackages/modules and relative imports will work - for the absolute imports to work though you need to rewrite them (or again use relative):

if __name__ == "__main__":
    from app.processing.processor_query_ui import UI_DataFinderUI # absolute import
    from .processing.processor_query_ui import UI_DataFinderUI # relative import
    ...

answered Jan 25 at 16:36

Mr_and_Mrs_D

34.5k45 gold badges193 silver badges373 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

skytwosea Jan 25 at 20:04

Yep, that's how I have it set in my justfile. Thanks for taking the time!

couteau · Accepted Answer · 2025-01-25 04:07:20Z

1

What does your python path look like? For those relative imports to work, you need the directory containing your repository directory (not just the repository directory itself) to be on the python path. The reason the relative imports work when you run the code as a QGIS plugin is that the directory containing your repo —- i.e., the QGIS plugin directory —- is already on the python path in the QGIS python environment.

To avoid that issue, I would suggest restructuring your repo as follows, with all of the plugin code in a subdirectory off of the top-level directory (I’d suggest a more descriptive name than ‘plugin’, but I don’t know what your plugin is called).

.
├── app
│   └── main.py
├── justfile
├── README.md
├── plugin
│   ├── __init__.py 
│   ├── plugin.py
│   ├── metadata.txt
│   ├── resources
│   │   ├── name_resolver.py
│   │   └── response_codes.py
│   └── processing
│       ├── B_processor.py
│       ├── A_processor.py
│       ├── processor_core.py
│       ├── processor_interface.py
│       ├── processor_query_ui.py
│       └── processor_query_ui.ui
└── tests
    ├── __init__.py
    ├── test_A_processor.py
    └── test_processor_core.py

This has a couple of advantages:

First, it lets you distribute just the plugin subdirectory as your plugin. Your plugin users don’t need your unit tests or the app directory or the README.

Second, if you now add your top level repository directory to the python path (which I assume it already is or your absolute imports wouldn’t work), you should be able to do the absolute imports from outside the plugin folder (you’ll have to change those to import plugin.processing and plugin.resources and their contents instead of just processing etc), and you should then be able to do relative imports within plugin directory.

Your tests will also need to use absolute imports if they don’t already.

This structure has worked for me for QGIS plugins with multiple levels of submodules/subfolders.

answered Jan 25 at 4:07

couteau

3642 silver badges8 bronze badges

2 Comments

skytwosea Jan 25 at 15:31

Thank you very much, this did the trick! Re: my python path: In my development dir I have a .venv installed. So when cd'd into the dev dir root, calling python means the dev dir is on my python path: I tested that with import sys; print(sys.path) (don't mind the semicolon). Restructuring also cleaned up my loader scripts. I had a bash script (supplanted by the justfile) which was selectively rsync-ing to the qgis plugin dir; now, instead of having all those --excludes I just rsync all contents of plugin.

skytwosea Jan 25 at 15:31

Follow-up: do you have an explanation for why qgis prefers/requires(?) relative imports? Or any documentation on that? I'll keep looking but so far have not found anything with a satisfying explanation. Thanks again, I appreciate your time

Collectives™ on Stack Overflow

Python adaptor for working around relative imports in my QGIS plugin?

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related