Async Django 3.1 with aiohttp client

Question

Is it possible now, after the start of porting django to async to use aiohttp client with django?

My case:

server receives a request;
server sends another request to another server;
server decodes the response, saves it into DB, and returns that response to the client;

Andrew Svetlov mentioned that aiohttp "was never intended to work with synchronous WSGI" two years ago. (https://github.com/aio-libs/aiohttp/issues/3357)

But how the situation looks now? Django seems to almost support async views. Can we use aiohttp with asgi django?

I know, I can create an aiohttp server that handles requests, then populates some queue, and queue handler that saves responses into database, but here I am missing a lot from django: ORM, admin, etc.

Eduard Gan · Accepted Answer · 2021-01-30 22:58:19Z

2

You can implement thhis scenario in Django 3.1 like this:

async def my_proxy(request):
    async with ClientSession() as session:
        async with session.get('https://google.com') as resp:
            print(resp.status)
            print(await resp.text())

But the biggest question for me is how to share one session of aiohttp within django project because it is strongly not recommended to spawn a new ClientSession per request.

Important note: Of course you should run your django application in ASGI mode with some compatible application server(for example uvicorn):

uvicorn my_project.asgi:application --reload

UPDATE: I found a workaround. You can create a module(file *.py) with shared global objects and populate it with ClientSession instance from project settings at project startup:

shared.py:

from typing import Optional
from aiohttp import ClientSession


AIOHTTP_SESSION: Optional[ClientSession] = None

settings.py:

from aiohttp import ClientSession
from my_project import shared
...
shared.AIOHTTP_SESSION = ClientSession()

views.py:

from my_project import shared

async def my_proxy(request):
    async with shared.AIOHTTP_SESSION.get('https://google.com') as resp:
         print(resp.status, resp._session)
         await resp.text()

Important - imports should be EXACTLY the same. When i change them to form "from my_project.shared import AIOHTTP_SESSION" my test brakes :(

tests.py:

from asyncio import gather
from aiohttp import ClientSession
from django.test import TestCase
from my_project import shared


class ViewTests(TestCase):

    async def test_async_view(self):
        shared.AIOHTTP_SESSION = ClientSession()
        await gather(*[self.async_client.get('/async_view_url') for _ in range(3)])

Run test by ./manage.py test

edited Jan 30, 2021 at 22:58

answered Jan 30, 2021 at 21:14

Eduard Gan

6356 silver badges7 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

RcoderNY Over a year ago

That's good you point out that aiohttp.ClientSession should be created once for the lifetime of the server. This is in order to benefit from connection pooling. One thing though is that per docs the ClientSession should be created inside an async function: "To avoid import dependency hell aiohttp encourages creation of ClientSession from an async function."

Collectives™ on Stack Overflow

Async Django 3.1 with aiohttp client

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related