Removing async pollution from Python

Question

How do I remove the async-everywhere insanity in a program like this?

import asyncio


async def async_coro():
    await asyncio.sleep(1)


async def sync_func_1():
    # This is blocking and synchronous
    await async_coro()


async def sync_func_2():
    # This is blocking and synchronous
    await sync_func_1()


if __name__ == "__main__":
    # Async pollution goes all the way to __main__
    asyncio.run(sync_func_2())

I need to have 3 async markers and asyncio.run at the top level just to call one async function. I assume I'm doing something wrong - how can I clean up this code to make it use async less?

FWIW, I'm interested mostly because I'm writing an API using asyncio and I don't want my users to have to think too much about whether their functions need to be def or async def depending on whether they're using a async part of the API or not.

async.sleep() does not sleep in the classical sense. It schedules a continuation (in this case for an implicit return None) and hands back control to the loop. — Klaus D.
– Klaus D., Commented Oct 28, 2019 at 1:38
I mean, call it whatever you want, but the program takes 1 second to run. It definitely does not "do nothing" — Thomas Johnson
– Thomas Johnson, Commented Oct 28, 2019 at 1:41

Thomas Johnson · Accepted Answer · 2019-10-28 03:35:08Z

1

After some research, one answer is to manually manage the event loop:

import asyncio


async def async_coro():
    await asyncio.sleep(1)


def sync_func_1():
    # This is blocking and synchronous
    loop = asyncio.get_event_loop()
    coro = async_coro()
    loop.run_until_complete(coro)


def sync_func_2():
    # This is blocking and synchronous
    sync_func_1()


if __name__ == "__main__":
    # No more async pollution
    sync_func_2()

answered Oct 28, 2019 at 3:35

Thomas Johnson

11.9k20 gold badges66 silver badges104 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user4815162342 Over a year ago

That won't work if one sync function implemented in this fashion at any point needs to call another such function - you'll get an "event loop is running" RuntimeError. Also, setup and teardown of an event loop has a cost which is non-negligible if done behind the scenes - an empty asyncio.run takes around 0.15ms on my machine.

user4815162342 Over a year ago

Added an alternative answer that avoids this issue.

user4815162342 · Accepted Answer · 2019-10-28 19:36:44Z

If you must do that, I would recommend an approach like this:

import asyncio, threading

async def async_coro():
    await asyncio.sleep(1)

_loop = asyncio.new_event_loop()
threading.Thread(target=_loop.run_forever, daemon=True).start()

def sync_func_1():
    # This is blocking and synchronous
    return asyncio.run_coroutine_threadsafe(async_coro(), _loop).result()

def sync_func_2():
    # This is blocking and synchronous
    sync_func_1()

if __name__ == "__main__":
    sync_func_2()

The advantage of this approach compared to one where sync functions run the event loop is that it supports nesting of sync functions. It also only runs a single event loop, so that if the underlying library wants to set up e.g. a background task for monitoring or such, it will work continuously rather than being spawned each time anew.

Collectives™ on Stack Overflow

Removing async pollution from Python

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related