How to create an iterator pipeline in Python?

Question

Is there a library or recommended way for creating an iterator pipeline in Python?

For example:

>>>all_items().get("created_by").location().surrounding_cities()

I also want to be able to access attributes of the objects in the iterators. In the above example, all_items() returns an iterator of items, and the items have different creators.

The .get("created_by") then returns the "created_by" attribute of the items (which are people), and then location() returns the city of each person, and then pipes it to surrounding_cities(), which returns all an iterator of the surrounding cities for each location (so the end result is a large list of surrounding cities).

I'm trying to see what it would take to emulate Gremlin in Python:youtube.com/watch?v=5wpTtEBK4-E — espeed
– espeed, Commented May 31, 2011 at 14:40

John Gaines Jr. · Accepted Answer · 2011-05-31 14:06:32Z

4

Aren't you just processing an iterator? The natural way to use an iterator in Python is the for loop:

for item in all_items():
    item.get("created_by").location().surrounding_cities()

There are other possibilities such as list comprehensions which may make more sense depending upon what you're doing (usually makes more sense if you're trying to generate a list as your output).

answered May 31, 2011 at 14:06

John Gaines Jr.

11.6k1 gold badge28 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

bpgergo · Accepted Answer · 2011-05-31 14:22:23Z

1

I suggest you look up how to implement pipes with coroutines in python, more specifically this pipe example

If you implemented your functions according to the example above, then your code would be something like this (for simplicity, I suppose you'll want to print those cities):

all_items(get_location(get_creators(get_surrounding_cities(printer()))))

edited May 31, 2011 at 14:22

answered May 31, 2011 at 13:57

bpgergo

16.1k5 gold badges48 silver badges70 bronze badges

2 Comments

slowdog Over a year ago

Why not all_items(get_creators(get_location(get_surrounding_cities(printer())))) ? Is there a reason for all_items and get_creators to behave differently than get_location and get_surrounding_cities?

bpgergo Over a year ago

Actually you're right, no need to behave differently, I edited my answer

Anentropic · Accepted Answer · 2013-01-30 15:31:41Z

1

in your example you only really have two iterator methods, all_items and surrounding_cities so you could do pretty well with just itertools.chain:

from itertools import chain

cities = chain.from_iterable(
    item.get("created_by").location().surrounding_cities() for item in all_items()
)

cities would then be an iterator listing all the surrounding cities for the items

answered Jan 30, 2013 at 15:31

Anentropic

34.2k13 gold badges111 silver badges170 bronze badges

Collectives™ on Stack Overflow

How to create an iterator pipeline in Python?

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related