Is it possible to redeliver a Celery worker's tasks as soon as the worker stops sending heartbeat?

Question

According to the celery documentation on Celery using Redis broker,

The visibility timeout defines the number of seconds to wait for the worker to acknowledge the task before the message is redelivered to another worker

This means that if a worker crashes during execution of a task, the same task will be delivered to another worker after this timeout. However, I also noticed that Celery workers sent heartbeats to each other(probably via the broker). Is it possible to leverage these heartbeats to redeliver task to healthy workers as soon as an unhealthy worker stops sending heartbeats? The advantage of this is that tasks handled by a failed worker can be picked up much sooner than if we had to wait for the visibility timeout(which defaults to 1 hour).

DejanLekic · Accepted Answer · 2022-05-26 12:44:08Z

1

That would require some complex logic to be set in place... I would rather rely on task_reject_on_worker_lost instead.

answered May 26, 2022 at 12:44

DejanLekic

20k4 gold badges53 silver badges84 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Jessica Over a year ago

I tested doing this on RabbitMQ and it works as expected. It doesn't seem to work with Redis as message broker though. Do you have any suggestions about how to solve that problem?

Collectives™ on Stack Overflow

Is it possible to redeliver a Celery worker's tasks as soon as the worker stops sending heartbeat?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related