Celery task does not timeout

Question

I would like to verify that setting time limits for celery tasks work.

I currently have my configuration looking like this:

CELERYD_TASK_SOFT_TIME_LIMIT = 30
CELERYD_TASK_TIME_LIMIT = 120

task_soft_time_limit = 29
task_time_limit = 44_LIMIT = 120

I am overloading the timeout parameters because it appears that there name change coming and I just want to be sure that I hit at least one timeout.

But when I run a test in the debugger the cellery app.conf dictionary looks like this:

(Pdb) app.conf['task_time_limit'] == None
True
(Pdb) app.conf['task_soft_time_limit'] == None
True
(Pdb) app.conf['CELERYD_TASK_SOFT_TIME_LIMIT']
30
(Pdb) app.conf['CELERYD_TASK_TIME_LIMIT']
120

I've written a test which I believe would trigger the timeout but no error is ever raised:

@app.task(soft_time_limit=15)
def time_out_task():
    import time
    count = 0
    #import pdb; pdb.set_trace()
    while count < 1000:
        time.sleep(1)
        count += 1
        print(count)

My questions are as follows:

What are the conical settings I should set for a soft and hard time limit?
How could I execute a task in a test which proves to me that the time limits are in place.

Thanks

Matt · Accepted Answer · 2017-11-27 22:53:09Z

1

I solved the issue by changing the way I was testing and by changing the way I was importing the Celery configuration.

Initially, I was setting the configuration by importing a Django settings object:

app = Celery('groot')
app.config_from_object('django.conf:settings', namespace='CELERY')

But this was ignoring the settings with the CELERYD_... prefix. Thus I used the new notation and called the following method:

app.conf.update(
    task_soft_time_limit=30,
    task_time_limit=120,
)

I also changed from testing this in the Django test environment to spinning up an actual Celery worker and sending the task to the worker. If someone would supply a solution for how to test the timeout settings in the unit test it would be much appreciated.

answered Nov 27, 2017 at 22:53

Matt

5,1982 gold badges32 silver badges57 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

ibuler · Accepted Answer · 2019-03-09 08:21:25Z

0

In django settings CELERY_TASK_TIME_LIMIT is working for me

CELERY_TASK_TIME_LIMIT = 60

answered Mar 9, 2019 at 8:21

ibuler

134 bronze badges

Collectives™ on Stack Overflow

Celery task does not timeout

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related