6

My Python application can be install the normal way, or in development/editable mode with pip, like this:

virtualenv my_app
source my_app/bin/activate
pip install -e my_app

How can I make a function to introspect my virtualenv and check if my application runs in development/editable mode?

Is there any "flag" in sys module for that?

Motivation: have different configuration in development mode, and in production.

EDIT: I compare the virtualenv and the package directories.

import os
import sys

import pkg_resources

main_pkg = 'my_app'
package_dir = pkg_resources.resource_filename(main_pkg, '__init__.py')
virtualenv_dir = os.path.dirname(os.path.dirname(sys.executable))
common_path = os.path.commonprefix([package_dir, virtualenv_dir])
is_dev_mode = not common_path.startswith(virtualenv_dir)

I test if the package_dir is a subdirectory of the virtualenv_dir: if it is not a subdirectory, then I am on development mode.

EDIT2:

Is there a more reliable solution?

I want to know if there isn’t a data/a flag in the environment that would clearly indicate to me that my application is running in development mode.

What will happen if another dependency is in development mode too?

Improve this question
1
  • Note that Python development mode is something different than what this question is asking about. Commented Oct 23, 2020 at 13:48

3 Answers 3

Reset to default
5
+50

Using code from pip we can determine this:

import pip
import pkg_resources

# I've done `pip install -e .` for a git repo I'm working in inside
# a virtualenv
distributions = {v.key: v for v in pkg_resources.working_set}
# >>> distribution
# pre-commit 0.9.3 (/home/asottile/workspace/pre-commit)
distribution = distributions['pre-commit']

# Below is approximately how `pip freeze` works, see the end of
# the answer for a simpler approach, still using pip

# Turn into a pip FrozenRequirement (I'm using pip 9.0.1, it may
# be different for your version)
# I've passed an empty list for the second argument (dependency_links)
# I don't think it's necessary?
frozen_requirement = pip.FrozenRequirement.from_dist(distribution, [])

# Query whether the requirement is installed editably:
print(frozen_requirement.editable)

The magic of this comes from a little function inside pip (pip.utils):

def dist_is_editable(dist):
    """Is distribution an editable install?"""
    for path_item in sys.path:
        egg_link = os.path.join(path_item, dist.project_name + '.egg-link')
        if os.path.isfile(egg_link):
            return True
    return False

The dist in this is a pkg_resources distribution (as we acquired above). You of course can use the dist_is_editable function directly instead of going through FrozenRequirement:

# With `distribution` as above:
from pip.utils import dist_is_editable
print(dist_is_editable(distribution))  # True in my case ;)
Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

It sounds good ;-) Are the keys of pkg_resources.working_set packages names (import pkg_name) or library names (like `name="lib_name" in setup.py)?
The keys will end up being "normalized" package names (so for yaml you'll look for pyyaml, for pre_commit you'll look for pre-commit). The normalization rules follow python.org/dev/peps/pep-0440 (PEP440) (though generally it ends up being .replace('_', '-').lower())
From pip docs- you must not use pip’s internal APIs in this way. And the info is outdated anyway, now whether a distribution is editable or not can be recorded in direct_url.json instead of an egg-link.
@wim well of course this was written in a different time, 5 years ago now :)
Yeah, although pip didn't really have any public API at that time either (ref: #4696). More to the point, it would be good to update the answer with a way that is currently working/supported.
|
3

The top solution needs updating for a more recent version of pip, where the FrozenRequirement class is no longer accessible in pip's namespace.

I used this workaround in the relevant lines.

from pip._internal.operations.freeze import FrozenRequirement 
frozen_requirement = FrozenRequirement.from_dist(distribution)
Improve this answer

1 Comment

even this doesn't work anymore with pip==22.3.1
1

I am not sure if there's a reliable way to determine this. In fact, I would advise you against testing paths like that because you might run into into different special cases on various operating systems or environments.

There are a few alternatives I recommend instead:

1) If this is a command-line utility, I'd recommend to allow loading of a custom configuration file configurable via a command-line flag:

from argparse import ArgumentParser
import sys
import json

parser = ArgumentParser(description='...')
parser.add_argument('-c', '--config', default='config.json')

def main(argv):
    args = parser.parse_args(argv)
    print('loading config file:', args.config)
    with open(args.config, 'r') as config:
        config = json.loads(config.read())
    print('loaded config', config) 
    # TODO do something with the config

if __name__ == '__main__':
    main(sys.argv[1:])

Run with: python3 test1.py -c config-dev.json

2) If this is not a CLI app, you can achieve a similar thing by using an environment variable:

import os
import json

os.environ.get('CONFIG', 'config.json')

def main():
    config_file = os.environ.get('CONFIG', 'config.json')
    print('loading config file:', config_file)
    with open(config_file, 'r') as config:
        config = json.loads(config.read())
    print('loaded config', config) 
    # TODO do something with the config

if __name__ == '__main__':
    main()

Run with: CONFIG=config-dev.json python3 test2.py, or:

export CONFIG=config-dev.json
python3 test2.py

You could also make a shell script to help with set your dev environment, let's call it customenv:

source env/bin/activate
export CONFIG=config-dev.json

And then you could use this file to activate the dev environment:

source customenv

3) If you really wish to have a special case in your code for the development environment, you could specify this via an environment variable too:

import os
is_dev_mode = 'MY_APP_DEV' in os.environ and os.environ['MY_APP_DEV'] == '1'
if is_dev_mode:
    print('dev mode!')

Run with MY_APP_DEV=1 python3 test3.py, or:

export MY_APP_DEV=1
python3 -m test3.py

4) For more customization:

import os
app_mode = os.environ.get('MY_APP_MODE', 'prod')
print(app_mode)
Improve this answer

1 Comment

I agree with you but this is not what I want. Sorry, but, I really need to check if I am in development mode or not.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.