How to use concurrent.futures in Python

Question

Im struggling to get multithreading working in Python. I have i function which i want to execute on 5 threads based on a parameter. I also needs 2 parameters that are the same for every thread. This is what i have:

from concurrent.futures import ThreadPoolExecutor

def do_something_parallel(sameValue1, sameValue2, differentValue):
    print(str(sameValue1))        #same everytime
    print(str(sameValue2))        #same everytime
    print(str(differentValue))    #different

main(): 

    differentValues = ["1000ms", "100ms", "10ms", "20ms", "50ms"]

    with ThreadPoolExecutor(max_workers=5) as executor:
        futures = [executor.submit(do_something_parallel, sameValue1, sameValue2, differentValue) for differentValue in differentValues]

But i don't know what to do next

rAntonioH · Accepted Answer · 2018-11-26 15:22:54Z

6

If you don't care about the order, you can now do:

from concurrent.futures import as_completed

# The rest of your code here

for f in as_completed(futures):
    # Do what you want with f.result(), for example:
    print(f.result())

Otherwise, if you care about order, it might make sense to use ThreadPoolExecutor.map with functools.partial to fill in the arguments that are always the same:

from functools import partial

# The rest of your code...

with ThreadPoolExecutor(max_workers=5) as executor:
    results = executor.map(
        partial(do_something_parallel, sameValue1, sameValue2),
        differentValues
    ))

edited Nov 26, 2018 at 15:22

rAntonioH

1493 silver badges12 bronze badges

answered Mar 19, 2018 at 8:47

samfrances

3,8273 gold badges28 silver badges48 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Buzz B Over a year ago

How can this be done using keyword arguments? Trying executor.map(partial(fun, param1=arg1, param3=arg3), param2=arg2) throws an error, and removing param2= leads to an error as the iterable argument is passed to the first parameter.

Collectives™ on Stack Overflow

How to use concurrent.futures in Python

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related