I want use the Python 3 asyncio module to create a server application.
I use a main event loop to listen to the network, and when new data is received it will do some compute and send the result to the client. Does 'do some compute' need a new event loop? or can it use the main event loop?
-
are you looking for something like the producer consumer pattern ? en.wikipedia.org/wiki/Producer%E2%80%93consumer_problemAndrew Scott Evans– Andrew Scott Evans2014-12-04 09:43:25 +00:00Commented Dec 4, 2014 at 9:43
-
In that case you could use the producer as the process from the main loop and only need one loop. It depends on the network use a computer has a lot of network ports.Andrew Scott Evans– Andrew Scott Evans2014-12-04 09:48:46 +00:00Commented Dec 4, 2014 at 9:48
Add a comment
|
1 Answer
You can do the compute work in the main event loop, but the whole event loop will be blocked while that happens - no other requests can be served, and anything else you have running in the event loop will be blocked. If this isn't acceptable, you probably want to run the compute work in a separate process, using BaseEventLoop.run_in_executor. Here's a very simple example demonstrating it:
import time
import asyncio
from concurrent.futures import ProcessPoolExecutor
def cpu_bound_worker(x, y):
print("in worker")
time.sleep(3)
return x +y
@asyncio.coroutine
def some_coroutine():
yield from asyncio.sleep(1)
print("done with coro")
@asyncio.coroutine
def main():
loop = asyncio.get_event_loop()
loop.set_default_executor(ProcessPoolExecutor())
asyncio.async(some_coroutine())
out = yield from loop.run_in_executor(None, cpu_bound_worker, 3, 4)
print("got {}".format(out))
loop = asyncio.get_event_loop()
loop.run_until_complete(main())
Output:
in worker
done with coro
got 7
cpu_bound_worker gets executed in a child process, and the event loop will wait for the result like it would any other non-blocking I/O operation, so it doesn't block other coroutines from running.
