Getting started with Celery in Django

Question

I'm currently working on a project and I'd like to integrate asynchronous task processing as well as some sort of message queue early on so that I'll be able to scale up quickly by simply adding message queue processor servers to the cluster.

I came across Celery a while back and it caught my eye. Since it's pretty well integrated with Django, I figured I'd get pretty good support with it. I'm just not really sure how to start, as there's a lot of configuration involved.

For now, I'm running just about everything out of my Django project (serving static files, pipeline, etc.) so I'd like to have a messaging queue built in to run with django runserver if possible. (Don't worry, this is only for development.) How can I get started using Celery with my existing Django project?

github.com/ask/django-celery might help. Also django-celery.readthedocs.org/en/latest/getting-started/… — Marius Grigaitis
– Marius Grigaitis, Commented Mar 11, 2012 at 22:13

Holger Drewes · Accepted Answer · 2012-05-21 18:46:29Z

3

djkombu is now deprecated, the django transport is now directly integrated in the kombu package.

For defining the backend in your Django settings.py, you can use:

BROKER_BACKEND = "django"

You can find different transport aliases from Kombu here.

This was tested with django-celery 2.5.5, celery 2.5.3 and kombu 2.1.8.

answered May 21, 2012 at 18:46

Holger Drewes

612 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Bernhard Vallant · Accepted Answer · 2012-03-26 10:12:30Z

1

Celery has quite a nice documentation, also for those getting started, but two facts worth being mentioned for beginners:

Use djkombu as the BROKER_BACKEND. This will give you a pretty simple message queue for development, where all messages are stored in the SQL database used by django. Due to celery's api you can easily replace it with a "real" message queue for production:

BROKER_TRANSPORT = "kombu.transport.django"
Django-celery has a setting CELERY_ALWAYS_EAGER. If set to True there will be no asynchronous background processing, all tasks that are getting called via celery will be run synchronously (so no need to start any additional celery workers - very useful for debugging as well).

edited Mar 26, 2012 at 10:12

answered Mar 11, 2012 at 22:15

Bernhard Vallant

50.9k21 gold badges126 silver badges149 bronze badges

2 Comments

Naftuli Kay Over a year ago

Unfortunately, things have changed a lot in the Celery library. Can you edit your answer to work with the new APIs? Other than installing the right apps, what do I need to do to run a messaging queue directly in my Django app? It can block for now, that's fine, the main thing is being able to migrate my application to use it, so that I'm "future-proofed" so to speak.

Naftuli Kay Over a year ago

ie: djkombu is deprecated, BROKER_BACKEND has become BROKER_TRANSPORT, etc.

Collectives™ on Stack Overflow

Getting started with Celery in Django

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related