Multiple usage of generator from list in Python

Question

Basically, I'm in following situation - I generate a list, e.g.

l = [2*x for x in range(10)]

which I iterate through later on multipletimes, e.g.

for i in l: print i    # 0,2,4,6,8,10,12,14,16,18
for i in l: print i    # 0,2,4,6,8,10,12,14,16,18
for i in l: print i    # 0,2,4,6,8,10,12,14,16,18

The problem is that the list is way too large to fit into memory, hence I use its generator form, i.e.:

l = (2*x for x in range(10))

However, after this construction only first iteration works:

for i in l: print i    # 0,2,4,6,8,10,12,14,16,18
for i in l: print i    #
for i in l: print i    #

Where is the problem? How may I iterate through it multipletimes?

JuniorCompressor · Accepted Answer · 2015-04-18 10:41:50Z

6

Your generator is exhausted the first time. You should recreate your generator each time to renew it:

l = (2*x for x in range(10))
for i in l: print i
l = (2*x for x in range(10))
for i in l: print i

(Note: you should use xrange in python 2 because range creates a list in memory)

You can create also a shortcut function to help you or even a generator function:

def gen():
    for i in range(10):
        yield 2 * i

and then:

 for i in gen(): print i
 for i in gen(): print i

edited Apr 18, 2015 at 10:41

answered Apr 18, 2015 at 9:02

JuniorCompressor

20.1k4 gold badges33 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

mgallagher Over a year ago

Should probably note that xrange is the generator version of range in python 2, whereas range in python 3+ is a generator and there is no xrange.

JuniorCompressor Over a year ago

@MylesGallagher yes I updated. I guessed we were talking only for python 2.

PM 2Ring Over a year ago

I like the generator function. Alternatively, for such a simple generator, a lambda would be acceptable, eg gen = lambda : (2*x for x in xrange(10)). And of course either of these functions could be given multiplier and length args to make them more versatile.

salparadise · Accepted Answer · 2015-04-18 09:05:27Z

0

You can also iterate on the generator directly:

for i in (2*x for x in range(10)): print i
for i in (2*x for x in range(10)): print i
...

answered Apr 18, 2015 at 9:05

salparadise

5,8751 gold badge28 silver badges32 bronze badges

1 Comment

PM 2Ring Over a year ago

True, but that's not DRY.

Collectives™ on Stack Overflow

Multiple usage of generator from list in Python

2 Answers 2

3 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related