4

I've been learning me some dynamic 'plugin loading' for python and noticed a not really problematic but rather an interesting difference between import module and from package import module.

I've created a test script consisting of four files (resembles my own setup for what I want to achieve)

The file tree looks like this:

  • test (main package)
    • sup (package, plugin folder)
      • __init__.py
      • uber.py (plugin)
    • __init__.py
    • bar.py ('main'-program)
    • foo.py (object that needs dynamically added functionality)
    • poo.py (decorator)

poo.py:

from test import foo

def decorate(method):
    print "before:", method.__name__ in dir(foo.Foo)
    setattr(foo.Foo, method.__name__, method)
    print "after :", method.__name__ in dir(foo.Foo)
    return method

foo.py:

import os

class Foo(object):

    def __init__(self):
        self.__loadplugins("sup")

    @classmethod
    def __loadplugins(cls, plugindir):
        for f in os.listdir(os.path.join(os.path.dirname(__file__), plugindir)):
            if f.endswith(".py"):
                __import__(("%s.%s" % (plugindir, f))[0:-3])

uber.py:

from test import poo

@poo.decorate
def aFunction(self, anArg):
    print anArg

I have two versions of bar.py, this one does not work:

import foo

f = foo.Foo()

f.aFunction("print goes here") # pylint: disable-msg=E1101

This one does work:

from test import foo

f = foo.Foo()

f.aFunction("print goes here") # pylint: disable-msg=E1101

The only difference between the two bars is the import. One is relative the other one is not. But the relative one does not work, and the absolute one does work. Is there anyone who can replicate this on his machine and could give some kind of explanation of why it is happening?

update
Thought it would be useful to also note my python version: Using normal python release 2.7.2 x86

update
Output of the 'wrong' bar.py:

before: False
after : True
Traceback (most recent call last):
  File "C:\Users\Daan\workspace\python\mytests\src\test\bar.py", line 6, in <module>
    f.aFunction("print goes here") # pylint: disable-msg=E1101
AttributeError: 'Foo' object has no attribute 'aFunction'

Output of the 'correct' bar.py:

before: False
after : True
print goes here
Improve this question
4
  • Try adding an empty file to your test directory named __init__.py Commented Mar 22, 2012 at 21:48
  • They are there. Bit lazy to put them in that List. But specially for you I'll add them. Commented Mar 22, 2012 at 21:52
  • What do you mean by saying that first one doesn't work? If you see an exception, please include backtrace. Commented Mar 22, 2012 at 22:33
  • @subdir done, it is quite a standard error message. Hope this clarifies this a bit more. Commented Mar 22, 2012 at 22:40

3 Answers 3

Reset to default
3

I assume that you are inside 'test' directory and have environment variable PYTHONPATH=.. (so you can do both 'import foo' and 'from test import foo').

In this case foo and test.foo are two different modules, loaded separately (in spite of the fact that they are loaded from the same file).

'decorate' function from test.poo module adds method to class 'Foo' that lives in test.foo (first line of poo.py: "from test import foo"), at the same time 'Foo' class from foo module remains unchanged.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Yes, your first assumptions are correct. From within Eclipse There is a 'src' directory which is in the pythonpath. test is a package and I run the bar.py as a python-run.
Ok, so I guess I answered your question?
I have to test them first, haven't got time to try all answers yet :-)
I indeed was mixing up PYTHONPATH's. My 'test' package was residing in a source dir called src. I was running from within test. but src was also added to the pythonpath. I have now restructured my code/layout and now it works just fine. Thanks.
1

How are you executing bar.py? I am assuming you are going to run it as a package, because that's how you designed it. Or else, from test import foo in the second version of bar.py doesn't make much sense as test would not be recognized by bar.py unless it is run as a package.

Have a look at the python docs for Intra-package references. They talk mainly about two kinds of references. Say, you are trying to import uber.py from foo.py. One way to do this is by, explicit relative reference of the kind from .sup import uber. Another way to do this, is by absolute reference and this would be of the form from test.sup import uber or import test.sup.uber

It looks like you are using absolute references through out except in foo.py. Here, you are effectively calling __import__('sup.uber') where as it should have been __import__('test.sup.uber'). I am not sure if this is what is causing the error that you reported, but I was able to get both versions of bar.py to run using,

I got both versions of bar.py to run by using this in foo.py:

__import__(("%s.%s" % ('test.'+plugindir, f))[0:-3])

Also, how are you running bar.py as a package? One way to do this would be to include a import test.bar line in a script outside the test dir and then run it.

Improve this answer

Comments

0

The short answer to your questions:

Is there anyone who can replicate this on his machine and could give some kind of explanation of why it is happening?

Yes. :-)

Somewhat longer:

Yes - I was able to reproduce it.

Here my explanation:

In the first case (where the error occurs), python includes the module foo twice (you can check that easily using a print statement). The class Foo is created a second time, the aFunction is added to the second incarnation - still the main program uses the first.

In the second case, the module foo is only read in once - and therefore the Foo class exists only once. The same incarnation of the class is used adding the aFunction and in the main.

The reason of importing the file twice is IMHO somewhat strange arrangement of modules. Looks that for python the internal used path / name for the foo module differs in the first example.

(I used a lot of print of id(foo.Foo) and globals() to get this sorted out.)

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.