417 questions
6
votes
1
answer
403
views
Python import mechanism and module mocks
This is more an effort to understand how Python (3.9 in this instance) works than an effort to solve an actual problem so please bear with me and disregard the nonsensical way of m3. I just wanted to ...
- 2,058
2
votes
1
answer
65
views
Dynamically Loading Modules of the Same Name but Different Paths
I'm making a sort-of mod loader that automatically loads certain classes with a specific variable defined from any files in the root directories' mods folder.
My file structure is as follows:
mods
- ...
- 21.2k
11
votes
3
answers
3k
views
Enum comparison become False after reloading module
I have MyEnum, an enumeration derived from enum, defined in a myenum.py file such as:
# myenum.py
import enum
class MyEnum(enum.Enum):
ONE = 1
TWO = 2
Then, I import this file using the ...
- 24.2k
5
votes
3
answers
424
views
How to handle extra requirements when also used as type hints?
I want to update a library so that a requirement is optional (using extras_require).
Unfortunately, imported classes from the optional requirement are used everywhere in the lib as type hints. Here is ...
- 24.2k
51
votes
2
answers
41k
views
Import class from module dynamically
I have class called 'my_class' placed in 'my_module'. And I need to import this class. I tried to do it like this:
import importlib
result = importlib.import_module('my_module.my_class')
but it says:
...
- 6,537
17
votes
1
answer
2k
views
How to import a package given full path?
My question is similar to How to import a module given the full path? however I'm not importing a .py source file, I'm importing a package with a .pyd.
During runtime I am creating new package ...
3
votes
2
answers
699
views
Convert module path to import_module name format
Is there a built-in or simple way to convert a file path (module path) to a name argument to be used by the importlib.import_module function?
For example:
'path/module.py' -> 'path.module'
I ...
- 17.3k
0
votes
1
answer
105
views
Use importlib.resources for files inside the top level parent module
I want to load a file from my python module using importlib.resources (see also this question).
If the file is inside a submodule, this is straight forward (run using python -m, see here):
import ...
- 581
0
votes
1
answer
63
views
Why does inspect fail to get source file for classes in a dynamically-imported module?
inspect.getsource() and inspect.getsourcefile() can access source info for a function, but not for a class, when they are in a module that is imported dynamically with importlib.
Here are two files, ...
- 123k
23
votes
6
answers
22k
views
Get app version from pyproject.toml inside python code
I am not very familiar with python, I only done automation with so I am a new with packages and everything.
I am creating an API with Flask, Gunicorn and Poetry.
I noticed that there is a version ...
- 23.3k
1
vote
0
answers
90
views
importlib resources files -> not a package, empty module
I tend to alwasy run the testers from inside the module folder, but this breaks the "resources.files" functionality, as it seems not to be able to find the module any more
module folder
...
- 1,446
0
votes
0
answers
100
views
How to properly import python module from another python file using importlib.import_module?
I'm working on Alembic migration github action and need to import base metadata class inside python migration script which locates in another directory. I can't import it directly inside python ...
- 1,197
7
votes
1
answer
3k
views
How to reference the current package for use with importlib.resources?
I have an installed package (typically located inside …/lib/pythonX.Y/site-packages/my-package) which contains (among other things) the resources …/my-package/a.txt and …/my-package/b.png. I would ...
- 2,292
2
votes
1
answer
68
views
What's `_distutils_hack.DistutilsMetaFinder` in `sys.meta_path`?
The document of Python's import system tells that there are 3 default finders/importers (built-in, frozen, and the import path one). But when trying to get the repr of sys.meta_path, I got another ...
- 1,784
28
votes
2
answers
20k
views
How to implement an import hook that can modify the source code on the fly using importlib?
Using the deprecated module imp, I can write a custom import hook that modifies the source code of a module on the fly, prior to importation/execution by Python. Given the source code as a string ...
- 421
0
votes
1
answer
91
views
Check if a python module exists with specific venv path
If I have a venv path, how can I find out if a Python module is installed inside that venv?
I normally use importlib.util.find_spec. But this only works for the current venv that is active and does ...
- 87
0
votes
1
answer
106
views
How to manually create a namesapce package import from a list of paths?
According to PEP420, namespace packages have the lowest precedence. For example, import xxx will not create a namespace package xxx if there are any xxx/__init__.pys or xxx.{py,pyc,so,pyd}s in any ...
- 169
0
votes
1
answer
65
views
Why does importlib.resources read_text raise UnicodeDecodeError on Windows but not mac and RPi?
I am writing a small web app (https://github.com/r-owen/base_loom_server) and it runs on macOS and Raspberry Pi, but a Windows user reports an exception when Python reads a UTF-8-encoded resource file ...
- 181k
26
votes
6
answers
28k
views
How to modify imported source code on-the-fly?
Suppose I have a module file like this:
# my_module.py
print("hello")
Then I have a simple script:
# my_script.py
import my_module
This will print "hello".
Let's say I want to "override" the print(...
- 641
0
votes
1
answer
52
views
python importlib from subdirectory, module not found
I'v been struggling with importing a sub module. In main.py two modules are imported located under 220_04_070. I'm using importlib since the directory name only consist of numbers. common.py just ...
- 57
0
votes
2
answers
176
views
How to dynamically import functions from a file in a subdirectory? [duplicate]
I am trying to dynamically import functions from a file in a subdirectory.
For example:
workspace
--|main.py
----|subfolder1
------|subfolder2
--------|file1.py
--------|file2.py
The file main.py ...
- 21.3k
4
votes
1
answer
20k
views
Getting ModuleNotFoundError: No module named 'importlib.resources'
So I'm working on a project where I should work with Netmiko. I had installed the module using pip3 install netmiko command and it installed successfully. But, when I try to import netmiko module in ...
- 1
0
votes
2
answers
299
views
Python load tempfile as module
I can't seem to load a tempfile with contents using importlib. It works fine if I have the content saved under a python file (with .py extension), then provide that file path.
Is there a way to make ...
- 1
2
votes
3
answers
5k
views
Is there any way to get the installed modules without version info with pip?
I tried pip freeze. I need it to output just the modules delimited by newlines, like pip freeze, except without the version number, because I am trying to create an auto upgrader, and want to do pip ...
- 47
1
vote
0
answers
52
views
Overriding pip package
I am trying to setup Octoprint on a new orangepi 3b board. It has been a bit problematic.
One problem is about access to gpio for all the existing plugins. The standard RPi.GPIO python library does ...
- 23.3k
3
votes
1
answer
76
views
Is it possible to create a dynamic module with a custom `__dict__` class?
When doing module = types.ModuleType('mymodule'), we don't get any chance of doing further customization, and we can't override module.__dict__ since it's readonly...
The same seems to apply with all ...
- 109k
5
votes
2
answers
712
views
Django autoreload raises: TypeError: unhashable type: 'types.SimpleNamespace'
When I upgrade importlib_meta from version 8.4.0 to 8.5.0 (released just yesterday, Sep 11 2024), I get the following error when I start running the development server with python manage.py runserver:
...
- 116
1
vote
0
answers
1k
views
Reloading torch triggers RuntimeError
When trying to reload the torch library in Python, by running the following lines of code, I encountered the error below:
import torch
import importlib
importlib.reload(torch)
------------------------...
- 4,894
0
votes
1
answer
89
views
OSError: Bad file descriptor
I'm trying to run celery beat worker and a normal celery worker as docker containers. I have my tasks in flask application.
The normal celery worker is working fine but getting following error in beat ...
- 815
0
votes
2
answers
78
views
Why undeclared variables are present in `globals()` after a reload and is it safe to use them to identify a reload? [duplicate]
I've found that the snippet below when reloading a module test unexpectedly has all variables already defined in globals()/locals().
Why this happens?
I've noticed this "xxx" in locals() ...
- 20.2k
0
votes
1
answer
64
views
How do I dynamically define a package level __all__ without running any slow code in the package's constituent modules?
TL;DR: How do I read a module's __all__ definition and dynamically add it to the package-level __init__.py without actually running any slow code in the module itself?
I am writing a library and have ...
- 23.3k
44
votes
6
answers
17k
views
Python 3.5+: How to dynamically import a module given the full file path (in the presence of implicit sibling imports)?
Question
The standard library clearly documents how to import source files directly (given the absolute file path to the source file), but this approach does not work if that source file uses ...
- 170
-1
votes
1
answer
68
views
Reload custom python function
I'm trying to reload from a python script a custom function because i'm editing the function constantly and i dont want to reload the kernel each time i make a change on the file. I have read that i ...
7
votes
1
answer
2k
views
Typehint importing module dynamically using importlib
Give something as follows:
import importlib
module_path = "mod"
mod = importlib.import_module(module_path, package=None)
print(mod.Foo.Bar.x)
where mod.py is:
class Foo:
class Bar:
...
- 12.2k
1
vote
2
answers
4k
views
AttributeError: 'EntryPoints' object has no attribute 'get'
I keep on getting the following error when testing a simple celery example.
AttributeError: 'EntryPoints' object has no attribute 'get'
using the following CLI command:
celery -A ctest worker --...
- 79
5
votes
2
answers
3k
views
How do I embed the version from pyproject.toml so my package can use it?
The version in my project is stored in pyproject.toml:
[tool.poetry]
name = "package_name"
version = "1.2.3"
# ...
I now want to have a __version__ in package_name/__init__.py as ...
- 23.3k
2
votes
1
answer
706
views
How can I replace the imp module in existing code to load .pyc files?
I am doing a CS course from the MIT Open Courseware website that has been taped in 2016 (course 6.0002 - intro to computational thinking and data science).
In one of the problem sets (ps3) the staff ...
- 611
12
votes
2
answers
4k
views
Find which Python package provides a specific import module
Without getting confused, there are tons of questions about installing Python packages, how to import the resulting modules, and listing what packages are available. But there doesn't seem to be the ...
- 5,750
0
votes
2
answers
169
views
Import a script from a parent's subdir, with a filename and dirname starting with digits
I know the usual Python import system, packages, __init__.py, the __import__ function (see In python, how to import filename starts with a number), etc. I know the classic How do I import other Python ...
1
vote
0
answers
116
views
Is there a way to get the python package distribution (wheel) name for a given module?
Is there a way to get the python package distribution (wheel) name for a given module (python object given by import ...)?
I know that it's possible to get meta-information about installed packages ...
- 15.2k
0
votes
1
answer
52
views
How do I carry over an import carried out in a 2nd file back to the calling file in python?
The code below is a utility I wrote that installs and restarts a script automatically without having the user need to parse through errors to install. Ideal usage would be below:
#file1.py
from file2 ...
- 2,459
0
votes
0
answers
30
views
Multiple tests for output of a module reading standard input
In one coding exercise in my Udemy course, a student is required to write a program that checks whether the total length of three strings read from the standard input is equal to 10, i.e. a one-liner ...
- 8,211
1
vote
1
answer
191
views
How to Override a Package's __init__.py Without Breaking Submodule Imports in Python?
I'm working on a project where I need to override files located in the ./lib/python3.9/site-packages" directory. I have the following hook that does just that using importlib.util and sys....
- 125
0
votes
1
answer
817
views
How to get the distribution name of an imported package in Python3.10+
I have multiple distributions that each provide a Python package of the same name (because they have the same API and same functionality, just different implementations). I want to be able to report ...
- 42.3k
1
vote
1
answer
272
views
Get package name of caller script from imported python file
I have a project that will be zipped and run from Spark, let's call it "client" project.
I would like, on a library imported by a script within this client project, to read some static ...
- 11
2
votes
1
answer
2k
views
How to use pyfakefs in conjunction with importlib.resources.path
Given some installed package, the following code can be used to print its location on the filesystem:
import importlib.resources
def print_package_path(pkg):
with importlib.resources.path(pkg, &...
- 20.7k
0
votes
0
answers
301
views
Installing geopandas in a QGIS Plugin?
I'm trying to install geopandas for use in a QGIS Plugin.
create a venv
install geopandas
add site-packages to path
import geopandas
# sys.executable - path to qgis python interpreter
# venv_path - ...
1
vote
0
answers
472
views
How to annotate a variable of an existing object? (working with mypy)
def get_source(import_name: str) -> Optional[str]:
spec = get_spec(import_name)
# Now, here. The spec.loader might be one of several values.
# But I *know* that it is gonna be importlib....
- 12.2k
1
vote
0
answers
604
views
Access files within python package with python 3.12 and importlib
I'm trying to figure out what the best practice is when it comes to reading data from within a package.
I understood that in python 3.12 I should use importlib.resources, see e.g. How to read a (...
- 503
1
vote
3
answers
3k
views
ImportError: No module named importlib
I'm trying to install scrapy on python 2.7. The problem is that when it's time to compile it I get this error
$ sudo python2 -m pip install scrapy
DEPRECATION: Python 2.7 reached the end of its life ...
- 105