Can I create a helper function for both async and sync environment?

Question

I'm trying to create helper function for my projects, but some legacy projects are in sync environment.

my helper function looks like:

def func_for_async_and_sync(session, data):
    statement = select(Obj).where(Obj.id == data['id'])
    # and more code
    if isinstance(session, AsyncSession):
        obj = await session.execute(statement)
    else:
        obj = session.execute(statement)
    # edit obj and return
    return obj

Of course it not work. How can I call AsyncSession.execute in normal function?

The main difference between How to call a async function from a synchronized code Python is I need return value from async function.

If I rewrite my function to:

        loop = asyncio.get_running_loop()
        obj = asyncio.run_coroutine_threadsafe(
            session.get(model_class, primary_key_value), loop
        ).result(timeout=5)

I will always get TimeoutError. Future.result() will not return correctly, and this is my minimal test code:

import asyncio


async def main():
    """main is async and started as normal with asyncio.run"""
    print("BEGIN main")

    loop = asyncio.get_running_loop()
    timeout = 3
    # Create a coroutine
    coro = asyncio.sleep(1, result=3)

    # Submit the coroutine to a given loop
    future = asyncio.run_coroutine_threadsafe(coro, loop)

    # Wait for the result with an optional timeout argument
    assert future.result(timeout) == 3


if __name__ == "__main__":
    asyncio.run(main())

"I need return value from async function" -- the question you linked to has multiple answers that facilitate returning values. Have you tried those? The example you've provided does not align with those answers. If you're running an ASGI webserver, consider looking at how django handles this (specifically, the async_to_sync and sync_to_async functions). — sytech
– sytech, Commented Jun 27, 2024 at 7:30
@sytech, I have tried that, see: gist.github.com/PaleNeutron/3c85fb5f2bb7d2f9b76e1ebef2afe82e , it will raise RuntimeError: You cannot use AsyncToSync in the same thread as an async event loop - just await the async function directly — PaleNeutron
– PaleNeutron, Commented Jun 27, 2024 at 8:26

sytech · Accepted Answer · 2024-06-27 07:52:40Z

0

Since your function is an 'edge' interface, and your session argument can be a Session or AsyncSession object whose execute method accepts the same arguments, you can just return session.execute without concern for whether it's a coroutine or a normal function.

If the session is an AsyncSession, your function will return a coroutine (which can in turn be awaited by the caller). If session is a Session the function works as normal and returns the result directly.

class Session:
    def execute(self, statement):
        time.sleep(1)
        return 'sync response'


class AsyncSession:
    async def execute(self, statement):
        await asyncio.sleep(1)
        return 'async response'

def func_for_async_and_sync(session, data):
    statement = ...
    return session.execute(statement)

def main():
    # sync example
    session = Session()
    result = func_for_async_and_sync(session, {'id': 'foo'})
    print(result)

async def a_main():
    # async example
    session = AsyncSession()
    result = await func_for_async_and_sync(session, {'id': 'foo'})
    print(result)


if __name__ == '__main__':
    main()
    asyncio.run(a_main())

You can also look at projects like unasync which can be used to code-generate syncronous counterparts to async code.

I need return value from async function.

The methods described in the linked question do facilitate returning a response from the async function.

If your callers are sync functions wanting to use async, you might do something like this, applying the asyncio.run strategy described in the question you linked:

def func_for_async_and_sync(session, data):
    statement = ...

    if asyncio.iscoroutinefunction(session.execute):
        return asyncio.run(session.execute(statement))
    else:
        return session.execute(statement)


def main():
    # using sync session (from sync function)
    session = Session()
    result = func_for_async_and_sync(session, {'id': 'foo'})
    print(result)

def a_main():
    # using async session (from sync function)
    session = AsyncSession()
    result = func_for_async_and_sync(session, {'id': 'foo'})
    print(result)

if __name__ == '__main__':
    main()
    a_main()

As you can see in each of these cases, func_for_async_and_sync can be called both with an async session object or session object and can be called from async or sync functions as needed.

edited Jun 27, 2024 at 7:52

answered Jun 27, 2024 at 7:40

sytech

42.7k8 gold badges77 silver badges127 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

PaleNeutron Over a year ago

Sorry, I edit my question, I need to mutate the object not use return it in the edge, and I really confused about why future.result not work.

sytech Over a year ago

@PaleNeutron you're trying to use it in an async function. There's no point in doing that, since you can just use await the coroutine. It really only makes sense to use run_coroutine_threadsafe in a regular sync function.

Collectives™ on Stack Overflow

Can I create a helper function for both async and sync environment?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related