Multiple iterators (using enumerate) for the same iterable, what is going on?

Question

Consider the following example:

s = 'abc'
[(c1, c2) for j, c2 in enumerate(s) for i, c1 in enumerate(s)]

Output:

[('a', 'a'),
 ('b', 'a'),
 ('c', 'a'),
 ('a', 'b'),
 ('b', 'b'),
 ('c', 'b'),
 ('a', 'c'),
 ('b', 'c'),
 ('c', 'c')]

I would expected the same output if enumerate is called outside the list comprehension and the iterators are assigned to variables:

it1, it2 = enumerate(s), enumerate(s)
[(c1, c2) for j, c2 in it1 for i, c1 in it2]

But I get:

[('a', 'a'), ('b', 'a'), ('c', 'a')]

What is going on? I use Python 3.6.9.

In the first way after every iteration a new iterator is created while in the second it is created only once. That's what I think it's happening. — dcg
– dcg, Commented Mar 27, 2020 at 13:57

norok2 · Accepted Answer · 2020-03-27 14:43:42Z

7

What is happening is that the inner iterator gets exhausted after the first iteration of the outer iterator:

s = 'abc'
it1 = enumerate(s)
it2 = enumerate(s)

for i, x in it1:
    for j, y in it2:  # <-- gets consumed when i = 0 and stays empty
        ...

By contrast:

s = 'abc'

for i, x in enumerate(s):
    for j, y in enumerate(s):  # <-- gets recreated at each iteration
        ....

If you need persistence, enclose it in a list or tuple:

itr = list(enumerate(s))
print([(c1, c2) for j, c2 in itr for i, c1 in itr])
# [('a', 'a'), ('b', 'a'), ('c', 'a'), ('a', 'b'), ('b', 'b'), ('c', 'b'), ('a', 'c'), ('b', 'c'), ('c', 'c')]

although note the different memory footprint of using enumerate() multiple times versus having it enclosed in a list or tuple.

edited Mar 27, 2020 at 14:43

answered Mar 27, 2020 at 13:58

norok2

27.1k6 gold badges83 silver badges110 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

jsbueno · Accepted Answer · 2020-03-27 13:58:11Z

4

The difference is that in:

s = 'abc'
[(c1, c2) for j, c2 in enumerate(s) for i, c1 in enumerate(s)]

A new c1 enumerator is created for each value yielded on the first for. While on your second example, the same enumerator is used (it2) - and it gets exausted once it reaches "c" - when the first for advances to the next iteration (c2 = "b") and tries to iterate "it2", it is already exhausted - and the whole expression ends.

answered Mar 27, 2020 at 13:58

jsbueno

114k11 gold badges159 silver badges239 bronze badges

Comments

kederrac · Accepted Answer · 2020-03-27 14:16:13Z

enumerate build-in function returns an iterator, once the elements are exhausted it stops

in your first version, you are building a new enumerate(s) for each iteration of the first loop and in the second loop you are consuming it,

in your second version, it2 has finished his elements from the first iteration of the first loop

s = 'abc'

def my_enumerate(s, name):
    print('a new enumerate: ', name)
    for i in enumerate(s):
        yield i
    print(f'the enumerate {name} is exhausted ')

[(c1, c2) for j, c2 in my_enumerate(s, 'it1') for i, c1 in my_enumerate(s, 'it2')]

output:

a new enumerate:  it1
a new enumerate:  it2
the enumerate it2 is exhausted 
a new enumerate:  it2
the enumerate it2 is exhausted 
a new enumerate:  it2
the enumerate it2 is exhausted 
the enumerate it1 is exhausted 
[('a', 'a'),
 ('b', 'a'),
 ('c', 'a'),
 ('a', 'b'),
 ('b', 'b'),
 ('c', 'b'),
 ('a', 'c'),
 ('b', 'c'),
 ('c', 'c')]

and for:

it1, it2 = my_enumerate(s, 'it1'), my_enumerate(s, 'it2')
[(c1, c2) for j, c2 in it1 for i, c1 in it2]

output:

a new enumerate:  it1
a new enumerate:  it2
the enumerate it2 is exhausted 
the enumerate it1 is exhausted 
[('a', 'a'), ('b', 'a'), ('c', 'a')]

Collectives™ on Stack Overflow

Multiple iterators (using enumerate) for the same iterable, what is going on?

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest