Python iterating over a list in parallel?

Question

I have a list created with no more than 8 items in it, right now I loop over the item with a standard for loop "for item in list:" I am wondering if there is a way to check all of the elements at the exact same time? I have seen some posts about using zip to iterate over 2 lists at the same time which could help.

If there is no way to iterate over a lists elements at the same time I would guess I need to split one single list into 4 separate lists and iterate 2 at the same time using zip and do the other 2 on a separate thread, that seems like it could be a clunky fix though.

Any help would be appreciated.

Edit Sorry to be vague, I have a list

apples = ['green', 'green', 'green', 'red']

right now I use a for loop to go over each element,

for apple in apples:
    checkAppleColour(apple)
    #Do some more stuff depending on colour....

right now it loops over each element in apple one by one, what I would like to be able to do is check each of the items at the exact same time for a colour check as every millisecond counts (example code used, but principle is exactly the same for what I am trying to achieve).

Performance is key here I don't mind if its threading I have to use, but I need maximum speed / efficiency for checking each element and my guess is parallelism is the way to do it, I am just not quite sure how.

what do you mean exactly by check all of the elements at the exact same time ? What do you actually want to do ? — Overdrivr
– Overdrivr, Commented Feb 10, 2016 at 15:21
Can you add some example code to clarify exactly what you want to do. — Håken Lid
– Håken Lid, Commented Feb 10, 2016 at 15:23
I'll second @Overdrivr what are you trying to acheive/do? are you looking for multiprocessing solution? are you trying to compare several elements to one another? — Roy
– Roy, Commented Feb 10, 2016 at 15:23
my guess is parallelism is the way to do it do not guess, profile and measure it. — Łukasz Rogalski
– Łukasz Rogalski, Commented Feb 10, 2016 at 17:04

wegry · Accepted Answer · 2016-02-10 15:31:29Z

7

If you can map a function onto the list, parallelism is easy.

def addOne (x):
    return x + 1

The default map does things on one thread.

map(addOne, range(1, 4)) #When iterated over produces [2, 3, 4]

If your problem is a map, it means that it's trivially parallel.

from multiprocessing import Pool

pool = Pool()
pool.map(addOne, range(1, 4)) ##When iterated over produces [2, 3, 4]

This uses multiple processes. multiprocessing.dummy.Pool is a thread pool. See this article for a great introduction to the rationale behind both.

answered Feb 10, 2016 at 15:31

wegry

8,2977 gold badges46 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python iterating over a list in parallel?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related