Strange behaviour when task added to empty loop in different thread

Question

I have an app which adds coroutines to an already-running event loop. The arguments for these coroutines depend on I/O and are not available when I initially start the event loop - with loop.run_forever(), so I add the tasks later. To demonstrate the phenomenon, here is some example code:

import asyncio
from threading import Thread
from time import sleep

loop = asyncio.new_event_loop()

def foo():
    loop.run_forever()

async def bar(s):
    while True:
        await asyncio.sleep(1)
        print("s")

#loop.create_task(bar("A task created before thread created & before loop started"))
t = Thread(target=foo)
t.start()
sleep(1)
loop.create_task(bar("secondary task"))

The strange behaviour is that everything works as expected when there is at least one task in the loop when invoking loop.run_forever(). i.e. when the commented line is not commented out.

But when it is commented out, as shown above, nothing is printed and it appears I am unable to add a task to the event_loop. Should I avoid invoking run_forever() without adding a single task? I don't see why this should be a problem. Adding tasks to an event_loop after it is running is standard, why should the empty case be an issue?

user4815162342 · Accepted Answer · 2018-03-04 11:03:45Z

Adding tasks to an event_loop after it is running is standard, why should the empty case be an issue?

Because you're supposed to add tasks from the thread running the event loop. In general one should not mix threads and asyncio, except through APIs designed for that purpose, such as loop.run_in_executor.

If you understand this and still have good reason to add tasks from a separate thread, use asyncio.run_coroutine_threadsafe. Change loop.create_task(bar(...)) to:

asyncio.run_coroutine_threadsafe(bar("in loop"), loop=loop)

run_coroutine_threadsafe accesses the event loop in a thread-safe manner, and also ensures that the event loop wakes up to notice the new task, even if it otherwise has nothing to do and is just waiting for IO/timeouts.

Adding another task beforehand only appeared to work because bar happens to be an infinite coroutine that makes the event loop wake up every second. Once the event loop wakes up for any reason, it executes all runnable tasks regardless of which thread added them. It would be a really bad idea to rely on this, though, because loop.create_task is not thread-safe, so there could be any number of race conditions if it executed in parallel with a running event loop.

TankorSmash · Accepted Answer · 2021-02-07 05:27:22Z

0

Because loop.create_task is not thread safe, and if you set loop._debug = True, you should see the error as

Traceback (most recent call last):
  File "test.py", line 23, in <module>
    loop.create_task(bar("secondary task"))
  File "/Users/soulomoon/.pyenv/versions/3.6.3/lib/python3.6/asyncio/base_events.py", line 284, in create_task
    task = tasks.Task(coro, loop=self)
  File "/Users/soulomoon/.pyenv/versions/3.6.3/lib/python3.6/asyncio/base_events.py", line 576, in call_soon
    self._check_thread()
  File "/Users/soulomoon/.pyenv/versions/3.6.3/lib/python3.6/asyncio/base_events.py", line 615, in _check_thread
    "Non-thread-safe operation invoked on an event loop other "
RuntimeError: Non-thread-safe operation invoked on an event loop other than the current one

edited Feb 7, 2021 at 5:27

TankorSmash

12.9k6 gold badges70 silver badges108 bronze badges

answered Mar 4, 2018 at 9:36

patpat

7141 gold badge5 silver badges15 bronze badges

Collectives™ on Stack Overflow

Strange behaviour when task added to empty loop in different thread

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related