Installing my own python module inside a virtual environment

Question

What I have:

local Python3 files that I want to turn into a module test_module
test_module folder containing an empty __init__.py, a setup.py file (see below) and subdirectories with several source files

What I want:

continuously work on and improve test_module locally
have an easy way to install test_module and all its dependencies locally in my own virtual environment (created using python3 -m venv my_environment)
run files that make use of the module via python myexample.py, without having to take care of adapting my local PYTHONPATH variable each time i enter or exit the my_environment
share my python code with others via git, and allow them to install their code locally on their machines using the same procedure (as simple as possible)
learn best practices on how to create my own module

How I'm doing it at the moment:

pip freeze > requirements.txt and pip install -r requirements.txt for installing dependencies
adding export PYTHONPATH="${PYTHONPATH}:." to my_environment/bin/activate, to have my own module in the search path (as found here: How do you set your pythonpath in an already-created virtualenv?)

I'd like to know if there are "cleaner" solutions based on setup.py, possibly involving something like pip install ./test_module or similar that takes care of 2.-3. automagically.

My current setup.py file looks as follows

 from setuptools import setup

 setup(
    name='test_module',
    version='0.1',
    description='Some really good stuff, that I am still working on',
    author='Bud Spencer',
    author_email='[email protected]',
    packages=['test_module'],  # same as name
    install_requires=['numpy', 'scipy', 'sklearn', 'argparse'], # external packages as dependencies
  )

why you decided to share python package with others via git? why you can't just upload package into pypi? I think that pip install test_module is eases way. — Danila Ganchar
– Danila Ganchar, Commented Sep 28, 2018 at 8:55
I am not very familiar with pypi, so I didn't even consider it. But as of now, I only want to share the package wit ha couple of collaborators for development, not with everyone. And git seemed to be the natural choice for that, particularly since the project was started on git in the first place. — mattu
– mattu, Commented Sep 29, 2018 at 19:07
ok. could you explain details? don't understand the problem. you can run all functionality without installing package into virtualenv. I mean it would be something like this: git clone... + virtualenv env + source env/bin/activate + pip install -r requirements.txt + python run something. Besides, you do not need to do cloning and installing twice. don't understand for what install package into virtualenv. I mean if you need to share package pip + virtualenv is good way. But I'm not sure that you need it. — Danila Ganchar
– Danila Ganchar, Commented Sep 30, 2018 at 0:29
The main point is that test_module is my own module, i.e. it only exists locally. As far as I understand, I cannot install such modules using pip. If there is such a way, I'd like to know how to do it (that is one part of the question). Right now, the way I make it work is to add export PYTHONPATH... to my bin/activate script, but this feels a little hacky and I'm thinking there must be a better way, without me having to ask other people to mess with their activate script to run my code in their virtualenv — mattu
– mattu, Commented Oct 1, 2018 at 7:08
I don't understand why you can't use PyPI. anyway you can install package into any virtualenv using setup.py. source env/bin/activate, cd path_to_your_repo, python setup.py install — Danila Ganchar
– Danila Ganchar, Commented Oct 1, 2018 at 10:57

Svet · Accepted Answer · 2019-06-06 19:32:27Z

4

It sounds like you want to run pip install -e <path/url> from within your virtual env, which will install a package (with a setup.py file as you have) from either a local path or a Git repo. See https://pip.pypa.io/en/stable/reference/pip_install/#vcs-support for an explanation on the syntax of the latter.

Example:

pip install -e git+https://github.com/me/test_module/#egg=test-module

If you have already installed and want to pull the latest code from the repo, add an --upgrade switch to the above.

answered Jun 6, 2019 at 19:32

Svet

1,7601 gold badge13 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

martin Over a year ago

thanks, what if the repo is private?

q.undertow · Accepted Answer · 2024-08-02 17:56:28Z

What I do:

make the venv a subdirectory of my project directory, with an exclusion line in .gitignore if needed. For example, if my project lives in /home/me/project I create the venv with python3.12 -m venv /home/me/project/venv.
python libraries developed as part of the project go in /home/me/project/local/lib.
in python scripts developed as part of the project, after importing the "external" modules i.e. those from the standard library plus any installed into the venv, and before importing the "local" modules, I put this:

import sys
from pathlib import Path

sys.path.append(str(Path(sys.prefix).parent / 'local/lib'))

As you understand, this relies on sys.prefix being set by the interpreter to the correct pathhname for the virtual environment.

Collectives™ on Stack Overflow

Installing my own python module inside a virtual environment

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related