Django: can I make db queries async?

Question

In my app I have a template that I wish to make it like a control center that would display a ton of information, so I have to make a ton of queries to grab that info.

So of course, the page is slow as heck and now I'm thinking how to fix this. I thought about making indexes but I don't have the space for this. I already optimized my queries to the minimun waste possible but it still takes like 1:30 minutes to load the page.

I'm looking into assynchronous support on Django, but the version of Django I'm using doesn't seem to support (2.1).

Does anybody got any tip that could help me in this?

I didn't showed my code because I figured it's not nescessary, for this question is more abstract. It's not about my problem especifics but more about optimization in general.

Celery/Django Channels

Andrey Borzenko
– Andrey Borzenko

2020-10-01 21:14:38 +00:00
Commented Oct 1, 2020 at 21:14 — Andrey Borzenko
– Andrey Borzenko, Commented Oct 1, 2020 at 21:14
Sounds more like a scaling problem.

yeg
– yeg

2020-10-02 07:50:57 +00:00
Commented Oct 2, 2020 at 7:50 — yeg
– yeg, Commented Oct 2, 2020 at 7:50

Ashkan Farhadi · Accepted Answer · 2020-12-07 12:26:22Z

1

The best idea is to use Celery.
Although it is not recommended, you can use python Thread Pools. Something like this:

queries = {
    '1': query_1, # These are your queries.
    '2': query_2,
    '3': query_3,
    '4': query_4,
}
# Since queries are lazy in Django, they have not been evaluated, yet.

# creating a threadpool
pool = ThreadPool(processes=int(len(queries) * 0.5))
threads = dict()
results = {}
for key in queries.keys():
    threads[key] = pool.apply_async(lambda f: f, args=(queries[key], ))

# wait for all threads to finish
for key, item in threads.items():
    results[key] = item.get()

Now your queries are being evaluated asynchronously.

answered Dec 7, 2020 at 12:26

Ashkan Farhadi

1621 silver badge11 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Django: can I make db queries async?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related