summing nested iterables with map-reduce/itertools

Question

I've been stuck with this data structure for a while now:

iter([iter([1,0]),iter([1,1]),iter([0,0])])

I want to get to sum of the inner-most elements using map-reduce/itertools.

I am able to get to the answer fairly quickly using for loops:

outer_iter = iter([iter([1,0]),iter([1,1]),iter([0,0])])

for inner_iter in outer_iter:
    for inner_list in inner_iter:
        total = total + inner_list

I am struggling to "translate" the code.

Can the iterables be nested arbitrary deep, or only two levels? — willeM_ Van Onsem
– willeM_ Van Onsem, Commented Dec 1, 2017 at 9:47

willeM_ Van Onsem · Accepted Answer · 2017-12-01 09:52:02Z

2

If the data is nested two levels deep, We can use the chain function to concatenate the iterables together, and then let sum(..) calculate the sum of the iterable. So:

from itertools import chain

sum(chain.from_iterable(outer_iter))

chain.from_iterable takes as input an iterable of iterables, and converts this into an iterable that lazily obtains the elements from the iterables one at a time. We can use iterable unpacking on chain, but if the outer iterable is an infinite list, the algorithm would get stuck (and eventually run out of memory).

answered Dec 1, 2017 at 9:52

willeM_ Van Onsem

482k33 gold badges483 silver badges624 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

lainglo Over a year ago

something I noticed when playing with this code: stackoverflow.com/questions/29403401/…

willeM_ Van Onsem Over a year ago

@lainglo: yes, the iterator is indeed consumed. But there is not really a way to not consume the iterator. You can use tee etc. that will produce two iterators out of one (by using shared memory)

remykarem · Accepted Answer · 2021-11-24 13:28:50Z

0

This is an iterator of iterator of int's.

Flatten to obtain just an iterator of int's
Sum this iterator

We can use Pyterator for this (disclaimer: I'm the author).

from pyterator import iterate

iterate(outer_iter).flatten().sum()

answered Nov 24, 2021 at 13:28

remykarem

2,51626 silver badges30 bronze badges

Collectives™ on Stack Overflow

summing nested iterables with map-reduce/itertools

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related