How to multiprocess async/await functions in python?

Question

I need to run an event every 60 seconds to all my cars. My problem with the code below is the while loop doesn't end until the timeout (60) does and hence only the first car in cars is run.

class RunCars(BaseEvent):

    def __init__(self):
        interval_seconds = 60  # Set the interval for this event
        super().__init__(interval_seconds)

    # run() method will be called once every {interval_seconds} minutes
    async def run(self, client, cars):
        for car in cars:        
            channel = get_channel(client, "general")
            await client.send_message(channel, 'Running this '+str(car))
            await msg.add_reaction(str(get_emoji(':smiley:'))) 
            
            reaction = None
            while True:
                if str(reaction) == str(get_emoji(':smiley:'))
                    await client.send_message(channel, 'Finished with this '+str(car))
                try:
                    reaction, user = await client.wait_for('reaction_add', timeout=60, check=check)
                except:
                    break

I tried changing the code into a multithreaded process but had trouble with async/await inside the function and problems with pickling the function itself.

I'd appreciate any suggestions for how to go about this..

mackorone · Accepted Answer · 2021-01-06 08:28:58Z

1

The asyncio module lets you execute multiple async method concurrently using the gather method. I think you can achieve the behavior you want by defining a method that runs a single car, and then replacing your for-loop with a call to gather, which will execute multiple run_one coroutines (methods) concurrently:

import asyncio

class RunCars(BaseEvent):

    def __init__(self):
        interval_seconds = 60  # Set the interval for this event
        super().__init__(interval_seconds)


    async def run(self, client, cars):
        coroutines = [self.run_one(client, car) for car in cars]
        asyncio.gather(*coroutines)

    async def run_one(self, client, car):
        channel = get_channel(client, "general")
        await client.send_message(channel, 'Running this '+str(car))
        await msg.add_reaction(str(get_emoji(':smiley:'))) 
          
        reaction = None
        while True:
            if str(reaction) == str(get_emoji(':smiley:'))
                await client.send_message(channel, 'Finished with this '+str(car))
            try:
                reaction, user = await client.wait_for('reaction_add', timeout=60, check=check)
            except:
                break

In general, when writing async code, you should try to replace sequential calls to async methods - basically for-loops that call async methods - with gather statements so they execute concurrently.

edited Jan 6, 2021 at 8:28

answered Jan 6, 2021 at 6:37

mackorone

1,0766 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Pynchia Over a year ago

I agree in principle, but this answer doesn't explain why his code isn't working. Admittedly it's difficult to tell, as the question should be more focused. As it is it implies too many aspects

echan00 Over a year ago

@mackorone thank you very much for your answer. this what i was trying to figure out. it really sucks that stackoverflow has become such a negative place. Some people assume it needs to work a particular way, the truth is never so obvious

mackorone Over a year ago

@Pynchia yea, I wasn't about to try to explain asyncio just to answer the question. And echan00, happy to help!

Collectives™ on Stack Overflow

How to multiprocess async/await functions in python?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related