Some confusions regarding celery in python

First, just to explain how it works briefly. You have a celery client running in your code. You call tasks.add(1,2) and a new Celery Task is created. That task is transferred by the Broker to the queue. Yes the queue is persisted in Rabbimq or SQS. The Celery Daemon is always running and is listening for new tasks. When there is a new task in the queue, it starts a new Celery Worker to perform the work.

To answer your questions:

1. Celery daemon is always running and it's starting celery workers.

2. Yes Rabitmq or SQS is doing the work of a queue.

3. With the celery monitor you can monitor how many tasks are running, how many are completed, what is the size of the queue, etc.

I think the answer from nstoitsev has good intention but create some confusion. So let's try to clarify a bit.

• A Celery worker is the celery process responsable of executing the tasks, when configured to run in background than is often called celery daemon. So you can consider the two the same thing. To clarify the confusion of he answer of nstoitsev, each worker can have a concurrency parameter that can be bigger than 1. When this is the case each celery worker is capable of create N child worker till reaching the concurrency parameter to execute the task in parallel, this are often also called worker.
• The broker holds queues and exchanges this means that a celery worker is able to connect to to the broker using a protocol called AMQP and publish or consume messages.
• Flower is able to monitor a celery cluster using the broker itself. Basically is capable to receive events from all the workers. Flower works also if you have the Result Backend disabled that btw is default behavior with celery Celery result backend.

Hope this helps.