Output of one request used as a parameter in next request using Asyncio + Aiohttp

Question

I'm new to Async programming, Asyncio, Aiohttp world.

I have multiple requests where - output of 1st request is being used as a parameter in 2nd request.

So far I have tried this much -

import aiohttp
import asyncio


async def get_response(session, url, params):
    async with session.get(url=url, params=params) as response:
        response = await response.json()
        return response['output']


async def main():

    tasks = []

    url = "https://example.api.com/"

    url_1 = url + 'path_1'
    url_2 = url + 'path_2'

    params = {'name': "Hello"}

    async with aiohttp.ClientSession() as session:

        a1 = get_response(session, url_1, params)
        tasks.append(a1)

        params = {'name': a1}
        b1 = get_response(session, url_2, params)
        tasks.append(b1)

        responses = await asyncio.gather(*tasks, return_exceptions=True)

        print(responses)

loop = asyncio.get_event_loop()
loop.run_until_complete(main())

I'm getting this error which is obvious -

TypeError("Invalid variable type: value should be str, int or float, got <coroutine object get_response at 0x00000000037FA6C0> of type <class 'coroutine'>")

So how do I pass the output of 1st request and not the coroutine object to the 2nd request -

params = {'name': a1}
b1 = get_response(session, url_2, params)

Hi, I'm curious if my answer helped with your question?

user4815162342
– user4815162342

2021-02-18 21:13:46 +00:00
Commented Feb 18, 2021 at 21:13 — user4815162342
– user4815162342, Commented Feb 18, 2021 at 21:13

user4815162342 · Accepted Answer · 2021-02-13 11:09:52Z

If the result of the first coroutine is needed for the second one, then you cannot run both coroutines in parallel, you will need to await the response of the first one. For example:

async def get_both(session, url_1, url_2, params_1):
    a1 = await get_response(session, url_1, params_1)
    params_2 = {'name': a1}
    b1 = await get_response(session, url_2, params_2)
    return a1, b1  # or just b1 if you don't need a1

Your main() would then look like this:

async def main():
    url = "https://example.api.com/"
    url_1 = url + 'path_1'
    url_2 = url + 'path_2'
    params = {'name': "Hello"}

    async with aiohttp.ClientSession() as session:
        responses = await get_both(session, url_1, url_2, params)
        print(responses)

Of course, you don't get parallel fetching, but that's a direct consequence of the fact that one request depends on the result of the other. What you could easily do is parallelize multiple instances of get_both - e.g. if url_1 and url_2 were lists, you could do this:

async def main():
    url_list_1 = [...]
    url_list_2 = [...]
    params = {'name': "Hello"}

    async with aiohttp.ClientSession() as session:
        coros = [get_both(session, url_1, url_2, params)
                 for url_1, url_2 in zip(url_list_1, url_list_2)]
        all_responses = await asyncio.gather(*coros)
        print(all_responses)

Collectives™ on Stack Overflow

Output of one request used as a parameter in next request using Asyncio + Aiohttp

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related