Python iter and for loops

Question

As I understand it, I can use the for loop construction on an object with a __iter__ method that returns an iterator. I have an object for which I implement the following __getattribute__ method:

def __getattribute__(self,name):
    if name in ["read","readlines","readline","seek","__iter__","closed","fileno","flush","mode","tell","truncate","write","writelines","xreadlines"]:
        return getattr(self.file,name)
    return object.__getattribute__(self,name)

I have an object of this class, a for which the following happens:

>>> hasattr(a,"__iter__")
True
>>> for l in a: print l
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'TmpFile' object is not iterable
>>> for l in a.file: print l
...
>>>

So python sees that a has an __iter__ method, but doesn't think it is iterable. What have I done wrong? This is with python 2.6.4.

Glenn Maynard · Accepted Answer · 2011-03-14 16:08:33Z

12

There's a subtle implementation detail getting in your way: __iter__ isn't actually an instance method, but a class method. That is, obj.__class__.__iter__(obj) is called, rather than obj.__iter__().

This is due to slots optimizations under the hood, allowing the Python runtime to set up iterators faster. This is needed since it's very important that iterators be as fast as possible.

It's not possible to define __getattribute__ for the underlying class type, so it's not possible to return this method dynamically. This applies to most __metamethods__; you'll need to write an actual wrapper.

answered Mar 14, 2011 at 16:08

Glenn Maynard

57.9k11 gold badges123 silver badges133 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user395760 Over a year ago

+1 So that's why iter doesn't call __getattribute__... as they say, you learn something new on SO every day :)

Glenn Maynard Over a year ago

Note that you can see a side-effect of this (at least in CPython): if you define __getattribute__, you'll notice it gets called for __class__ when you use iter, as the runtime looks up the class of the object to call __iter__. (This should probably be considered an implementation detail.)

Duncan · Accepted Answer · 2011-03-14 16:12:49Z

4

Some of the special methods are optimised when a class is created and cannot be added later or overridden by assignment. See the documentation for __getattribute__ which says:

This method may still be bypassed when looking up special methods as the result of implicit invocation via language syntax or built-in functions.

What you need to do in this case is provide a direct implementation of __iter__ that forwards the call:

def __iter__(self):
    return self.file.__iter__()

answered Mar 14, 2011 at 16:12

Duncan

96.4k15 gold badges129 silver badges160 bronze badges

Collectives™ on Stack Overflow

Python iter and for loops

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related