Django celery: scheduling recurring task but with fixed start date

Question

At the moment djcelery allows me to schedule a recurring task via the PeriodicTask model. For example a task that runs on an interval like every minute, or an interval specified by a crontab like every 1st of the month at noon. What I'd really like to do however is schedule a task for a fixed date that then repeats on an interval. For example first run on 3 March 2016 at 2:00 and then every hour thereafter.

Is there a way to achieve this within django and celery(with or without djcelery)? Thanks

ldmberman · Accepted Answer · 2016-02-13 14:22:11Z

4

As it is stated in the docs, you may implement your own custom scheduler. You should override the is_due method, which decides whether it is time to run the task.

Below is a proof-of-concept (I haven't checked it for errors). Note, that the __reduce__ method is also overridden so that the new parameter gets serialised as well.

import celery.schedules.schedule

class myschedule(celery.schedules.schedule):

    def __init__(self, *args, **kwargs):
        super(myschedule, self).__init__(*args, **kwargs)
        self.start_date = kwargs.get('start_date', None)

    def is_due(self, last_run_at):
        if self.start_date is not None and self.now() < self.start_date:
            return (False, 20)  # try again in 20 seconds
        return super(myschedule, self).is_due(last_run_at)

    def __reduce__(self):
        return self.__class__, (self.run_every, self.relative, self.nowfun, self.start_date)

And then you use it in the config:

CELERYBEAT_SCHEDULE = {
    'add-every-30-seconds': {
        'task': 'tasks.add',
        'schedule': myschedule(timedelta(seconds=30), start_date=start_date),
        'args': (16, 16)
    },
}

answered Feb 13, 2016 at 14:22

ldmberman

991 silver badge3 bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

fpghost Over a year ago

Thanks. This is helpful and close to what I want, however I also need it to be a database scheduler (as currently I have a frontend interface for users to create PeriodicTask model instances to launch jobs). I guess I need to do something similar but perhaps with the celery.beat.Scheduler being overriden? (similar to how djcelery does it now in djcelery.schedulers.DatabaseScheduler but maybe overriding is_due there?)

ldmberman Over a year ago

Yes, the database backend makes things a bit more complicated. Indeed, in order to avoid the mess of teaching django-celery to work with custom schedules like in my answer above I would rather update/override the PeriodicTask model to contain a start_date field and override DatabaseScheduler so that it takes it into account: ``` class MyScheduler(DatabaseScheduler): def is_due(self, entry): if entry.schedule.now() < entry.model.start_date: return False, 20 # try again in 20s return super(MyScheduler, self).is_due(entry) ```

fpghost Over a year ago

Where would be the correct place to overide PeriodicTask? Should I just fork djcelery or is there a better way?

fpghost Over a year ago

Or I guess if I just overide DatabaseScheduler I can point to it in settings.py, and in my CustomDatabaseScheduler, I set Model and Entry to my custom versions..

ldmberman Over a year ago

You can create your own model without touching djcelery and add the start_date field for it. Also, you would need to register the signals in this case. Alternatively, you can update djcelery (PeriodicTask and DatabaseScheduler) because this looks like a useful feature. The latter looks like a better option to me. The only problem though is djcelery seems to be abandoned - your PR will hardly get through any time soon.

|

Collectives™ on Stack Overflow

Django celery: scheduling recurring task but with fixed start date

1 Answer 1

11 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

11 Comments

Your Answer

Sign up or log in

Post as a guest

Related