python Environment Variables not changed for object of imported module

Question

I am puzzled with this behavior where environment variable is not updated correctly for class of a module I import on second or later calls.

I have a module which have some class I initialize to get a function (and totally works fine). To control and Test some behaviors on an automated manner locally (which in prod used by Docker environment variables) have some environment variables which can be set or unset.

To test some behaviors I am trying to make a script that tests some permutations of these environment variables importing the relevant module and initializing an object, reloading the module each call.

Each object initialization shows environment variable value and also if it affected the call as in below code:

#inside MyModule.SubModule

SOME_ENV_VAR = os.getenv("SOME_ENV_VAR", "0")
# convert environment variable to integer
try:
    SOME_ENV_VAR = int(SOME_ENV_VAR)
except ValueError:
    # if non integer given set to 0
    SOME_ENV_VAR = 0
print("SOME_ENV_VAR", SOME_ENV_VAR)

class MyClass:
    def __init__(self):
        # Logic to set Cuda device depending on Environmental variable
        print("Value of SOME_ENV_VAR: ", SOME_ENV_VAR) 

import sys, importlib

def get_object(env_set=True):
    os.environ["SOME_ENV_VAR"] = "1" if env_set else "0"
    importlib.reload(sys.modules['MyModule'])
    from MyModule.SubModule import MyClass

    p = MyClass()
    return p

On first call it is fine and environment variable is set correctly as in logs:

get_object(env_set=False)
#log output
SOME_ENV_VAR 0
Value of SOME_ENV_VAR:  0

But on second (and later calls) when I try to change environment variable, log output show environment variable is set correctly for module but on object gets the previously set environment variable.

get_object(env_set=True)
#log output
SOME_ENV_VAR 1
SOME_ENV_VAR:  0

I tried using multiprocessing too (to create another process), which had same result. Why is this behaviour happening, and how can I fix it some simple logic?

What is MyClass.init()? I don't see that method in the class. Do you mean p = MyClass()? — Barmar
– Barmar, Commented Dec 16, 2022 at 22:21
If you import a module multiple times, it doesn't reload the module. — Barmar
– Barmar, Commented Dec 16, 2022 at 22:22
Instead of reloading the module, put the code that does what you want into a function. Then call that function. — Barmar
– Barmar, Commented Dec 16, 2022 at 22:24
@Barmar thank you , but seems I have wrongly written my problem and there is also another submodule involved, editing the answer — Gorkem
– Gorkem, Commented Dec 16, 2022 at 23:28
You mean the question, right? Why do so many posters say that wrong? Is it a language issue? — Barmar
– Barmar, Commented Dec 16, 2022 at 23:29

Gorkem · Accepted Answer · 2023-01-27 07:12:47Z

0

Apparently it seems it is due to python not deleting previous instances of "submodule" which this answer https://stackoverflow.com/a/7123146/1562772 explains.

Instead of only reloading the main module, I should be deleting reference to submodule and reoading the actual submodule where the Class exists

def get_object(env_set=True):
    os.environ["SOME_ENV_VAR"] = "1" if env_set else "0"
    
    del sys.modules['MyModule.SubModule'] 
    importlib.reload(sys.modules['MyModule.SubModule']) 
    from MyModule.SubModule import MyClass
    
    p = MyClass()
    return p

edited Jan 27, 2023 at 7:12

answered Dec 16, 2022 at 23:33

Gorkem

99111 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

python Environment Variables not changed for object of imported module

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related