filter object becomes empty after iteration? [duplicate]

Question

I'm learning how to use the filter function.

This is the code I've written:

people = [{'name': 'Mary', 'height': 160},
          {'name': 'Isla', 'height': 80},
          {'name': 'Sam'}]

people2 = filter(lambda x: "height" in x, people)

As you can see what I'm trying to do is to remove all the dictionaries that don't contain the 'height' key.

The code works properly, in fact if I do:

print(list(people2))

I get:

[{'name': 'Mary', 'height': 160}, {'name': 'Isla', 'height': 80}]

The problem is that if I do it twice:

print(list(people2))
print(list(people2))

the second time, I get an empty list.

Can you explain me why?

Related: stackoverflow.com/questions/21715268/… , stackoverflow.com/questions/40960036/… — Mark Dickinson
– Mark Dickinson, Commented Jun 7, 2017 at 18:25

cs95 · Accepted Answer · 2017-06-07 18:26:06Z

23

This is a classic python3 doh!.

A filter is a special iterable object you can iterate over. However, much like a generator, you can iterate over it only once. So, by calling list(people2), you are iterating over each element of the filter object to generate the list. At this point, you've reached the end of the iterable and nothing more to return.

So, when you call list(people2) again, you get an empty list.

Demo:

>>> l = range(10)
>>> k = filter(lambda x: x > 5, l)
>>> list(k)
[6, 7, 8, 9]
>>> list(k)
[]

I should mention that with python2, filter returns a list, so you don't run into this issue. The problem arises when you bring py3's lazy evaluation into the picture.

answered Jun 7, 2017 at 18:26

cs95

406k106 gold badges744 silver badges797 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

L'ultimo Over a year ago

oh, ok, so if I want to use it more than once, I just have to save it on another variable, like people3 = list(people2), right?

cs95 Over a year ago

@L'ultimo Yep, you got it.

L'ultimo Over a year ago

(Or even better I can directly do people2 = filter(lambda x: "height" in x, people))

cs95 Over a year ago

You'll need to do people2 = list(filter(lambda x: "height" in x, people)) but yes.

Ehsan Kia Over a year ago

I don't understand the logic for this. filter and map are special objects, can they not have logics for reseting it back to the start after it's been parsed? You can still have lazy evaluation AND also be usable multiple times. Every time iter is called, return a new lazy iterator from the start.

Don Hatch · Accepted Answer · 2024-06-07 23:14:30Z

1

It's because what filter really returns is an iterator. This iterator doesn't really do anything until you start to use its results-- in this case, when you cast it to a list. people2 is this thing that's ready to filter the list of people; then when list is called on it, it iterates through the list of people and delivers the filtered result. Now that iterator is done, there's nothing left for it to iterate over, so when you call list on it a second time, there's nothing there.

Read this for some more details - Lazy evaluation in Python

edited Jun 7, 2024 at 23:14

Don Hatch

5,6045 gold badges36 silver badges56 bronze badges

answered Jun 7, 2017 at 18:27

OldGeeksGuide

2,91815 silver badges23 bronze badges

Collectives™ on Stack Overflow

filter object becomes empty after iteration? [duplicate]

2 Answers 2

5 Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

Comments

Linked

Related