How to structure a python project with three applications that use common module

Question

My project contains three Python applications. Application 1 is a web app. Applications 2 and 3 contain scripts downloading some data.

All three apps need to use a module Common containing a "model" (classes that are saved to database) and common settings.

I have no clue how to structure this project. I could create three directories, one for each application, and copy Common three times into their directories (doesn't seem right).

Another idea that comes to mind is; create a main directory and put there all files from Common, including __init__.py. Then, crete three subdirectories (submodules), one for each application.

Another way would be installing Common using pip, but that means I would have to reinstall every time I change something in that module.

In previous projects I used .NET - the equivalent in that world would be a Solution with four projects, one of them being Common.

Any suggestions?

Install Common using pip in editable mode (option "-e") and then you don't have to reinstall every time. — Jan Bednařík
– Jan Bednařík, Commented Sep 5, 2014 at 8:24

Andy · Accepted Answer · 2014-08-23 18:23:41Z

10

+50

I have a similar project that is set up like this

project_root/
    App1/
        __init__.py
    FlaskControlPanel/
        app.py
        static/
        templates/
    models/
        __init__.py
        mymodels.py

Then, I run everything from project_root. I have a small script (either batch or shell depending on my environment) that sets PYTHONPATH=. so that imports work correctly. This is done because I usually develop using PyCharm, where the imports "just work", but when I deploy the final product the path doesn't match what it did in my IDE.

Once the PYTHONPATH is set to include everything from your project root, you can do standard imports.

For example, from my FlaskControlPanel app.py, I have this line:

from models.mymodels import Model1, Model2, Model3

From the App1 __init__.py I have the exact same import statement:

from models.mymodels import Model1, Model2, Model3

I can start the Flask application by running this from my command line (in Windows) while I am in the project_root directory:

setlocal
SET PYTHONPATH=.
python FlaskControlPanel\app.py

The setlocal is used to ensure the PYTHONPATH is only modified for this session.

answered Aug 23, 2014 at 18:23

Andy♦

50.8k62 gold badges181 silver badges240 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jon Jay Obermark Over a year ago

Can the PYTHONPATH edit be replaced by a *.pth file? It is easier to include in source control.

Jan Bednařík · Accepted Answer · 2014-09-04 21:38:20Z

Make standard Python modules from your apps. I recommend structure like this:

apps/
    common/
        setup.py
        common/
            __init__.py
            models.py
    app1/
        setup.py
        app1/
            __init__.py
            models.py
project/
requirements.txt

Basic setup.py for app common:

#!/usr/bin/env python

from setuptools import setup, find_packages

setup(
    name='common',
    version='1.0.0',
    packages=find_packages(),
    zip_safe=False,
)

Make similar setup.py for other apps.

Set editable "-e" option for your apps in requirements.txt:

-e apps/common
-e apps/app1

Install requirements with pip:

$ pip install -r requirements.txt

Editable option means that source files will be linked into Python enviroment. Any change in source files of your apps will have immediate effect without reinstalling them.

Now you can import models from your common app (or any other app) anywhere (in other apps, project files, ...).

Nodari L · Accepted Answer · 2014-09-02 12:36:41Z

2

I like this approach

projects/
  __init__.py
  project1/
    __init__.py
  project2/
    __init__.py
  lib1/
    __init__.py
    libfile.py
  lib2/
    __init__.py

So, I need to cd into the projects folder. To start a projects use

python -m project_name

This allows me to easily import from any external lib like

from lib1.libfile import [imoprt what you want]

or

from lib1 import libfile

answered Sep 2, 2014 at 12:36

Nodari L

1,1783 gold badges15 silver badges24 bronze badges

Comments

Louis · Accepted Answer · 2014-09-22 19:16:46Z

0

I would create a structure like this:

project_root/
    app1/
        __init__.py
        script.py
    common/
        __init__.py
        models.py (all "common" models)

app1/script.py

import os, sys

# add parent directory to pythonpath
basepath = os.path.join(os.path.dirname(os.path.abspath(__file__)), '..')
if basepath not in sys.path:
    sys.path.append(basepath)

from common.models VeryCommonModel
print VeryCommonModel

If you don't want to set the python path at runtime, set the python path before running the script:

$ export PYTHONPATH=$PYTHONPATH:/path/to/project_root

And then you can do:

python app1/script.py

edited Sep 22, 2014 at 19:16

answered Aug 31, 2014 at 4:39

Louis

6845 silver badges18 bronze badges

4 Comments

user44 Over a year ago

so let's say I run file app1.py located in project_root/app1; the file starts with import models.py; but models.py imports common.models which is not in module search path (ImportError: No module named common.models)

Louis Over a year ago

Updated my answer, maybe that is what you were going for?

user44 Over a year ago

yes, this should work if you are ok with modifying path at runtime

Louis Over a year ago

I guess you can also set the PYTHONPATH before running the script to make sure "project_root" is in the python path. run: "export PYTHONPATH=$PYTHONPATH:/path/to/project_root"

Collectives™ on Stack Overflow

How to structure a python project with three applications that use common module

4 Answers 4

1 Comment

Comments

Comments

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related