Is it possible to query state of a celery tasks using django-celery-results during the execution of a task?

Question

I am using Celery + RabbitMQ for queuing tasks in my Django App,

I want to track the state of a task using the task_id and the task_state.

For that i created a TaskModel(Model) to store the task_id, task_state and some additional data in the database. On task execution, a new TaskModel object is save and updated as the task progresses. Everything is working fine.

However, i still need to add a lot of functionality and features and error protections etc. That's when i remembered the celery documentation mentions the django-celery-results.

So i followed the django-celery-results documentation instructions. Tasks results get stored in the default django database in a dedicated table, However only after the task concludes... and not during the PENDING, STARTED states.

Is it possible to use django-celery-results to store and query tasks during the PENDING and STARTED states? or not?

Thanks

Ouss · Accepted Answer · 2017-10-16 13:20:19Z

5

After reviewing the source code of django-celery-result it turns out the code is pretty simple and straight-forward.

In order to use django-celery-result to store tasks after the task function is called use the following:

from django_celery_results.models import TaskResult
import json

@shared_task(bind=True)
def foo(self, count):
 print('hello')
 task_result = TaskResult.objects.get(self.request.id)
 counter = 1
 interval = 20 #Limit updates to reduce database queries
 interval_count = count/interval
 for i in range(count):
  print(i)
  if counter>= interval_count:
   interval_count+=count/interval
   task_result.meta = json.dumps({'progress': counter/count})
   task_result.save()
  counter+=1
 task_result.save()
 return

def goo()
 task = foo.delay(1000)
 task_result = TaskResult(task_id=task.task_id)
 task_result.save()

edited Oct 16, 2017 at 13:20

answered Jan 31, 2017 at 3:09

Ouss

4,0563 gold badges34 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Alexey Over a year ago

task_result = TaskResult.objects.get(self.request.id) what is self in this line?

Alexey Over a year ago

Is it an error? Why do you use self outside any class?

Ouss Over a year ago

Hello, you are partly right, the function definition should declare self, otherwise it will produce an error. When bind is set to True (bind=True), in the decorator @shared_task. self is references the current task instance. I am editing the function definition accordingly.

Alexey Over a year ago

In fact no, it did't, don't know why but TaskReults are always pending and have no results, it must be my fault, but i don't know where

Ouss Over a year ago

Is your task running and displaying output? Make sure you store the task_result using task_result.save() every time you update its value. The entry in the database will not be updated unless you force the model to update using .save()

|

Collectives™ on Stack Overflow

Is it possible to query state of a celery tasks using django-celery-results during the execution of a task?

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related