Lambda function cannot access import of calling function?

Question

I wrote a lambda function to procedurise calls to a dynamically loaded library but I seem to be coming up against namespace problems with respect to the import. My testcase is below:

def dot(*args):
    ''' return a dot notation string
    '''
    return '.'.join(map(str, args))


def test(import_name, classname):
    exec('import ' + import_name)
    get_global = lambda glob: eval(dot(import_name, classname, glob))
    technodename = get_global('technodename')
    metalstack = get_global('metalstack')
    return technodename, metalstack

print(test('PROJECT', 'Quabanatu'))

the error which comes when executing the lambda function is:

NameError: name 'PROJECT' is not defined

however if I execute:

    technodename = eval(dot(import_name, classname, 'metalstack' ))

instead of calling get_global it works just fine. Also if I import the PROJECT library at the start of the program the lambda function works fine. What am I missing?

BTW I know this is not the best way of doing it in this case since 'technodname' is a constant but in some of my code I am forced to look up a class variable based on another variable hence the need to use the eval function.

bruno desthuilliers · Accepted Answer · 2018-09-10 14:55:38Z

3

Sorry but : you're doing it wrong

As a general rule, when you're typing eval() or exec, you are doing it wrong (the real use case for those "features" are so rare I never needed them in 20 years). Most python statements are executable (they are executed at runtime) and are syntactic sugar for features exposed as part of the builtins or in the stdlib.

In your case, the proper solutions here are import_lib.import_module(module_name) and getattr(obj, attrname):

import importlib

def get_global(module, *names):
   obj = module
   for name in names:
       obj = getattr(obj, name)
   return obj

def test(module_name, classname):
    module = importlib.import_module(module_name)
    technodename = get_global(module, classname, "technodename")
    metalstack = get_global(module, classname, "metalstack")
    return technodename, metalstack

answered Sep 10, 2018 at 14:55

bruno desthuilliers

78.3k6 gold badges103 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Stephen Ellwood Over a year ago

Thanks for the education. I have done some more experiments and realised that its eval which is at fault rather than lambda. In my defense I inherited this code and am doing my best to tidy it up - I never used eval or exec before this week either. I assume you are copying module to obj just so you don't change the input parameter, correct?

bruno desthuilliers Over a year ago

nope it's just so I don't have to special-case the first iteration. Rebinding a local name (arguments names are local) as no impact on the input param whatsoever...

Stephen Ellwood Over a year ago

How would you extend this to import a tree of libraries? The legacy code I am trying to improve imports (for instance) a project called Quabanatu and PROJECT is one of the libraries inside the folder i.e. Quabanatu is a class in the file PROJECT.py inside the directory Quabanatu and he whole directory is imported using the exec function.

Dunes · Accepted Answer · 2018-09-10 16:04:53Z

2

The specific reason for this problem is down to the way you have called exec inside a function. When exec is used in the one argument form it uses the locals of the calling context. This is fine for modules and classes, but it gets complicated inside a function. exec works with local variables that are stored in a dict (as do modules and classes). However, function locals are stored differently[1]. As such exec must create a dictionary view of the function locals and work with that. This means though that any changes to this dictionary are not reflected in the actual locals of the function.

def f():
    # NB. do not rely on any behaviour that results from mutating the dict returned by locals
    x = 'value' 
    assert locals()['x'] == 'value'
    assert 'builtins' not in locals()
    exec('import builtins')
    try:
        builtins
    except NameError as e:
        print(e)
    else:
        assert False
    print(locals()['builtins'])

# prints
# name 'builtins' is not defined
# <module 'builtins' (built-in)>

The correct solution in your instance is as given by bruno desthuilliers. However, to get exec and eval working in a reliable way you should provide the globals and locals for them to use where possible. They can be any dictionaries you want, not just the globals or locals of your context.

eg.

def g(module_name, attr):
    locals_ = {}
    globals_ = {}
    # two-arg form where locals and globals are the same
    exec('import ' + module_name, globals_)
    assert module_name not in locals_
    assert module_name in globals_
    exec('result = {}.{}'.format(module_name, attr), globals_, locals_)
    assert 'result' in locals_
    assert 'result' not in globals_
    return eval('{}.{}'.format(module_name, attr), globals_)

assert g('builtins', 'str') is str

[1] They are stored in a fixed-length array, and the index for each local variable is computed at compile time of the function. Functions are not equipped to have variable local variables -- they have no where to store them and no way to know how to retrieve them.

answered Sep 10, 2018 at 16:04

Dunes

42.1k7 gold badges87 silver badges107 bronze badges

3 Comments

Stephen Ellwood Over a year ago

Thanks for the explanation about eval. As stated I never used it before I came across it in this legacy code. Unfortunately the code is littered with them and my lambda function was a first attempt at isolating them. The problem in this case is that the import exec function imports a whole tree of libraries....

Dunes Over a year ago

I'm not quite sure what you're asking. The original question was about why exec wasn't working as you expected. Are you trying to debug a problem this legacy import code, or are tring to understand exec so you can extend your legacy code, or are you trying to replace it the legacy code entirely?

Stephen Ellwood Over a year ago

Actually my original question was why the lamda function was not working.... Ideally I would like to replace exec and eval in the code entirely since there are over 50 of them used and the code is very non-Pythonic and messy. The first step was to isolate them into a few places using a lambda function so that they are still within the related code and make sure that the code still works, then I was going to try to remove them entirely. @bruno desthuilliers code works nicely for a single library but I do not know how to extend it for a package of libraries.

Collectives™ on Stack Overflow

Lambda function cannot access import of calling function?

2 Answers 2

3 Comments

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related