Django slow performance on processing requests

Question

I have a django platform and I've found a strange slow performance when the server tries to response some http requests.

I've managed to isolate a little useless code that shows a strange delay, from which all the others can be derived.

case 1) If I have this function:

def get_n_events(request):
    a = request.user
    return HttpResponse("5")

the response is very fast, as expected:

Only 12.61ms, very fast, as expected.

case 2) But if I have this function:

def get_n_events(request):
    a = request.user.username
    return HttpResponse("5")

the response:

it takes more than 2 seconds!

How can be this delay explained and how can I avoid this huge time when I need to access the data inside the request? (For example, in my case I need to know the request.user.group_name, but it takes too much tome to get a response.

UPDATE

I made some tests with Django Debug Toolbar and I saw that the huge time is not due to SQL runtime.

As we can see, the SQL takes only 183ms, but the request has 1354msecs. Where does this time come from and how can I reduce it?

request.user is a lazy objects, so as long as you do not ask for an attribute, or call str(..), etc. over it, no database query is made. — willeM_ Van Onsem
– willeM_ Van Onsem, Commented May 13, 2020 at 18:13
And is it normal to take more than 2 seconds to access the database and retrieve the name? Is there any way to reduce that time? — Sergio Ferrer Sánchez
– Sergio Ferrer Sánchez, Commented May 13, 2020 at 18:14
no, but if this is the first query, then it can indeed take some time, for example because the connection pool is initialized, etc. If you run this a second time, then often the performance is better. — willeM_ Van Onsem
– willeM_ Van Onsem, Commented May 13, 2020 at 18:15
Yes, I notice that the first requests is a bit slower, but more than 2 seconds is too much time for a single database SELECT. We need to reduce that time somehow :S — Sergio Ferrer Sánchez
– Sergio Ferrer Sánchez, Commented May 13, 2020 at 18:19
well likely there is still something not configuered correctly. Perhaps you better start from an existing PostgreSQL database config in Django, (and thus alter hostname, etc.), also check for firewall issues, etc. — willeM_ Van Onsem
– willeM_ Van Onsem, Commented May 13, 2020 at 18:35

willeM_ Van Onsem · Accepted Answer · 2020-05-13 18:15:13Z

3

request.user is a lazy object, so as long as you do not ask for an attribute, or call str(..), etc. over it, no database query is made.

Indeed, if we take a look at the AuthenticationMiddleware [GitHub], we see:

class AuthenticationMiddleware(MiddlewareMixin):
    def process_request(self, request):
        assert hasattr(request, 'session'), (
            "The Django authentication middleware requires session middleware "
            "to be installed. Edit your MIDDLEWARE setting to insert "
            "'django.contrib.sessions.middleware.SessionMiddleware' before "
            "'django.contrib.auth.middleware.AuthenticationMiddleware'."
        )
        request.user = SimpleLazyObject(lambda: get_user(request))

So request.user presents itself as if it was loaded, but as long as you do not need an attribute, etc. from it, it is just a proxy object that promises to make a query if necessary.

answered May 13, 2020 at 18:15

willeM_ Van Onsem

482k33 gold badges483 silver badges624 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

kevlar Jun 9 at 15:31

We're running into the same issue and what I don't understand is why evaluating the request.user (which I understand may result in a database call due to it being a SimpleLazyObject) is so slow. There's no reason that getting a user from the database should take 2s. For us, sometimes it takes 10s+ or even longer (50s+) in some cases. Does anyone know why?

willeM_ Van Onsem Jun 9 at 15:34

: so how do you know it is when fetching request.user?

kevlar Jun 17 at 1:01

you're right in that we didn't. It turns out the slowness is in the CSRF middleware somewhere, which is crazy to me. We've added instrumentation and it 100% confirms this. We're adding further instrumentation to nail it down 100%

kevlar Jun 25 at 16:37

Turns out that wasn't the issue. Rather, it was because the middleware CSRF check was accessing request.POST and trying to parse JSON data as x-www-form-urlencoded, which would take ungodly amounts of time for "large" payloads. This is kind of a bug in the Django CSRF middleware because it doesn't check the Content-Type, so it runs the wrong parser and bricks the server for a long time.

Collectives™ on Stack Overflow

Django slow performance on processing requests

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related