load python code at runtime

Question

I would like to load a .py file at runtime. This .py file is basically a config file with the following format:

var1=value  
var2=value  
predicate_function=func line : <return true or false>

Once this file is loaded, I would like to be able to access var1, var2 and predicate_function. For each line, I'll pass it to the predicate function, and if it returns false, I'll ignore it.

In any case, I'm not sure how to load a python file at runtime and access its variables.

Clarification: there may be any number of these config files that I need to pass to the main program and I won't know their names until runtime. Google tells me I should use __import__. I'm not sure how to correctly use that method and then access the variables of the imported file.

You can append the file path to sys.path at runtime: >>>import sys Then >>>sys.path.append(r'C:\path_to_file') and finally >>>import myfile — Hank Lenzi
– Hank Lenzi, Commented Jun 11, 2022 at 13:06

Clément · Accepted Answer · 2010-02-14 09:47:18Z

15

As written in the python official documentation, if you just want to import a module by name, you can look it up in the sys.modules dictionary after using __import__.

Supposing your configuration is in myproject.mymodule, you would do like that :

module_name = 'myproject.mymodule'

import sys
__import__(module_name)
mymodule = sys.modules[module_name]

# Then you can just access your variables and functions
print mymodule.var1
print mymodule.var2
# etc...

You can also use the return value of __import__ statement but you will have to understand fully how python works with namespaces and scopes.

answered Feb 14, 2010 at 9:47

Clément

6,7381 gold badge32 silver badges22 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Clément Over a year ago

mmh, nope, in my example, it would return myproject module and not myproject.mymodule. Straight from the doc : When the name variable is of the form package.module, normally, the top-level package (the name up till the first dot) is returned, not the module named by name.

forefinger · Accepted Answer · 2010-02-14 09:29:29Z

You just need to be able to dynamically specify the imports and then dynamically get at the variables.

Let's say your config file is bar.py and looks like this:

x = 3
y = 4
def f(x): return (x<4)

Then your code should look like this:

import sys

# somehow modnames should be a list of strings that are the names of config files
#
# you can do this more dynamically depending on what you're doing                                                                                                     
modnames = ['bar']

for modname in modnames:
  exec('import %s' % modname)

for modname in modnames:
  mod = sys.modules[modname]
  for k in mod.__dict__:
    if k[:2] != '__':
      print modname, k, mod.__dict__[k]

I get this output:

bar f <function f at 0x7f2354eb4cf8>
bar x 3
bar y 4

Then you at least have all the variables and functions. I didn't quite get what you wanted from the predicate functions, but maybe you can get that on your own now.

Mike Graham · Accepted Answer · 2010-02-14 00:00:42Z

7

To access another Python module, you import it. execfile has been mentioned by a couple people, but it is messy and dangerous. execfile clutters your namespace, possibly even messing up the code you are running. When you want to access another Python source file, use the import statement.

Even better would be not to use a Python file for configuration at all, but rather to use the builtin module ConfigParser or a serialization format like JSON. This way your configuration files don't allow execution of arbitrary (possibly malicious) code, doesn't require people to know Python to configure your program, and can easily be altered programatically.

answered Feb 14, 2010 at 0:00

Mike Graham

77.2k16 gold badges105 silver badges131 bronze badges

1 Comment

Shahbaz Over a year ago

Actually the reason I am making my config a python file is because I want the config file to contain a couple of (predicate) functions. Otherwise I would simple read in a text file, split the strings and read into a dict.

oefe · Accepted Answer · 2010-02-14 09:18:46Z

7

If the imported module is on the regular search path, you can use __import__.

If you need to load the module from an arbitrary path in the filesystem, use imp.load_module.

Be sure to consider the security implications of loading arbitrary user-specified code.

answered Feb 14, 2010 at 9:18

oefe

20.1k7 gold badges49 silver badges66 bronze badges

Comments

Alex Martelli · Accepted Answer · 2010-02-13 23:05:31Z

3

In Python 2.*, execfile works (I recommend passing a specific dictionary and accessing the variables from there -- as the note in the docs says, execfile can't affect the calling function's locals() dictionary).

In Python 3.*, execfile has been removed, so do, instead:

with open('thefile.py') as f:
  exec(f.read(), somedict)

answered Feb 13, 2010 at 23:05

Alex Martelli

887k175 gold badges1.3k silver badges1.4k bronze badges

Comments

Tom · Accepted Answer · 2016-10-02 17:52:30Z

3

Since the Python version hasn't been clearly mentioned, it is worth pointing out that the imp module has been deprecated in newer Python versions in favor of the importlib module. Example here.

answered Oct 2, 2016 at 17:52

Tom

3,2036 gold badges35 silver badges38 bronze badges

Comments

Dolda2000 · Accepted Answer · 2014-04-08 21:37:56Z

2

I'm kinda late to the party, but I want to present an alternative answer nonetheless.

If you want to import code without affecting the global module namespace, you can create an anonymous module (using types.ModuleType) and load arbitrary code in it (using compile and exec). For instance, like this:

import types

filename = "/path/to/your/file.py"
with open(filename) as fp:
    code = compile(fp.read(), filename, "exec")
config_module = types.ModuleType("<config>")
exec code in config_module.__dict__

You can then access the variables as config_module.var1, &c.

answered Apr 8, 2014 at 21:37

Dolda2000

26k4 gold badges58 silver badges94 bronze badges

1 Comment

weberc2 Over a year ago

What's the Python3 equivalent? I'm getting a syntax error on the last line.

Simon Callan · Accepted Answer · 2010-02-13 23:02:44Z

1

If you want to have a configuration file that will only be edited by the user when the program isn't running, just import it as a normal python file

ie.

main.py:

import config
print config.var1

config.py:

var="var12"
var2 = 100.5

answered Feb 13, 2010 at 23:02

Simon Callan

3,1301 gold badge24 silver badges35 bronze badges

1 Comment

Shahbaz Over a year ago

I may have any number of such config files, so I can't just do an "import configFile." Otherwise this would be the simplest solution :)

user221209 · Accepted Answer · 2011-02-04 23:29:56Z

0

try the imp module : http://docs.python.org/library/imp.html

answered Feb 4, 2011 at 23:29

user221209

Collectives™ on Stack Overflow

load python code at runtime

9 Answers 9

1 Comment

Comments

1 Comment

Comments

Comments

Comments

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

9 Answers 9

1 Comment

Comments

1 Comment

Comments

Comments

Comments

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related