Python 3 Strange Metaclass Behaviour

Question

I was playing with metaclasses in Python 3:

class M(type):
    def __new__(cls, clsname, bases, attrs):
        for name, attr in attrs.items():
            if callable(attr):
                attrs[name] = attr
        return type.__new__(cls, clsname, bases, attrs)

class C(metaclass=M):
    def f(self, x):
        print(x)

if __name__ == '__main__':
    c = C()
    c.f(1)
    c.f(2)

Nothing special so far, I just hook into the creation of a class, and substitute its method with... well, itself, so no wonder everything works. But with:

class M(type):
    def __new__(cls, clsname, bases, attrs):
        for name, func in attrs.items():
            if callable(func):
                attrs[name] = lambda *args, **kwds: func(*args, **kwds)
        return type.__new__(cls, clsname, bases, attrs)

It sometime works, and sometimes doesn't:

user$ python test.py
1
2
user$ python test.py
Traceback (most recent call last):
  File "./meta.py", line 23, in <module>
    main()
  File "./meta.py", line 19, in main
    instance.method(1)
  File "./meta.py", line 9, in <lambda>
    attrs[name] = lambda *args, **kwds: func(*args, **kwds)
TypeError: 'str' object is not callable

But I just substituted its method with a lambda wrapper! What does 'str' have to do with anything? What am I doing wrong?

(Just in case it's some weird platform-dependent implementation issue, I'm using Ubuntu Server 12.04.3...)

UPDATE: fixed the name mismatch in the traceback.

Your traceback doesn't match your code; attribute != func, attributes != attrs. — Martijn Pieters
– Martijn Pieters, Commented Feb 5, 2014 at 22:53
In this case attribute appears to be a string object, so the callable test appears to be missing. — Martijn Pieters
– Martijn Pieters, Commented Feb 5, 2014 at 22:54
Also, your next problem: func isn't going to point to what you think it's pointing to inside the lambda. (It will refer to the last attribute you saw in your loop, which may not be callable.) To fix this, write a helper function to create the lambda, passing in the function, rather than creating it in your loop. — kindall
– kindall, Commented Feb 5, 2014 at 23:04
possible duplicate of Local variables in Python nested functions — Martijn Pieters
– Martijn Pieters, Commented Feb 5, 2014 at 23:06

kindall · Accepted Answer · 2014-02-06 15:43:18Z

To further elaborate on my comment:

def makelambda(func):
    return lambda *args, **kwds: func(*args, **kwds)

class M(type):
    def __new__(cls, clsname, bases, attrs):
        for name, func in attrs.items():
            if callable(func):
                attrs[name] = makelambda(func)
        return type.__new__(cls, clsname, bases, attrs)

This is necessary because, in your original code, inside the lambda, func refers to whatever value func had when your __new__ method returned, not the value it had when the lambda was created. This is counterintuitive, but you can verify it:

lambdas = [lambda: x for x in range(10)]
print(lambdas[0]())    # prints 9, as do lambdas[1] through [9]

To fix that, we use a separate function to create the lambda, and thus "freeze" the value of the func variable at the time the lambda was created. You can also do this with a default argument value on the lambda, but since you're using * and ** here, this is a little problematic.

(The behavior has nothing to do with metaclasses, you'd see the same behavior anywhere you defined lambdas and changed the value of variables used in them after creating them. And lambdas are no different from any other function in this respect.)

Collectives™ on Stack Overflow

Python 3 Strange Metaclass Behaviour

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related