1

I am using django framework and ran into some performance problems.

There is a very heavy (which costs about 2 seconds) in my views.py. And let's call it heavy().

The client uses ajax to send a request, which is routed to heavy(), and waits for a json response.

The bad thing is that, I think heavy() is not concurrent. As shown in the image below, if there are two requests routed to heavy() at the same time, one must wait for another. In another word, heavy() is serial: it cannot take another request before returning from current request. The observation is tested and proven on my local machine.

synchronous django

I am trying to make the functions in views.py concurrent and asynchronous. Ideally, when there are two requests coming to heavy(), heavy() should throw the job to some remote worker with a callback, and return. Then, heavy() can process another request. When the task is done, the callback can send the results back to client. The logic is demonstrated as below:

asynchronous django

However, there is a problem: if heavy() wants to process another request, it must return; but if it returns something, the django framework will send a (fake)response to the client, and the client may not wait for another response. Moreover, the fake response doesn't contain the correct data. I have searched throught stackoverflow and find less useful tips. I wonder if anyone have tried this and knows a good way to solve this problem.

Thanks,

Improve this question
1
  • 1
    What server are you using? The Django development server? Commented Nov 30, 2015 at 22:54

2 Answers 2

Reset to default
2

First make sure that 'inconcurrency' is actually caused by your heavy task. If you're using only one worker for django, you will be able to process only one request at a time, no matter what it will be. Consider having more workers for some concurrency, because it will affect also short requests.

For returning some information when task is done, you can do it in at least two ways:

  • sending AJAX requests periodicaly to fetch status of your task
  • using SSE or websocket to subscribe for actual result

Both of them will require to write some more JavaScript code for handling it. First one is really easy achievable, for second one you can use uWSGI capabilities, as described here. It can be handled asynchronously that way, independently of your django workers (django will just create connection and start task in celery, checking status and sending it to client will be handled by gevent.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I use polling myself. That article you link to is a must-read for people who'd like to have websocket work with Django.
0

To follow up on GwynBliedD's answer:

celery is commonly used to process tasks, it has very simple django integration. @GwynBlieD's first suggestion is very commonly implemented using celery and a celery result backend.

https://www.reddit.com/r/django/comments/1wx587/how_do_i_return_the_result_of_a_celery_task_to/

A common workflow Using celery is:

  1. client hits heavy()
  2. heavy() queues heavy() task asynchronously
  3. heavy() returns future task ID to client (view returns very quickly because little work was actually performed)
  4. client starts polling a status endpoint using the task ID
  5. when task completes status returns result to client
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.