How to execute tasks in Celery using datetime from MySQL?

Question

News are stored in database MySQL with datetime of publication.

Any time user can add a new in the table with delay date publishing.

How to use Celery for listeting database table and check if it is time to publish data?

For publishing data is responsible another process(script Python). So Celery should call this script for each rows in MySQL table according datetime.

How to do that using Celery?

Another way I think to create queue with date and id of publication then directly add data from user form in this queue. Therefore sholud be process(Celery) that observes a queue for further executing tasks by date.

Celery provides documentation for Periodic tasks and Scheduled tasks, but there is no explanation and example how to use it.

Sorry if question is trivial, I am a newcomer in Python Celery.

You can schedule a daily task that publishes all publications whose publication date is today() — user2390182
– user2390182, Commented Nov 25, 2017 at 18:31
I dont need that, I need to schedule tasks using data that stored in database MySQL, and immediately to execute this in time. Also database can be extended by new dates. So Celery should be aware about that immidiatly. You sugest to use static concret date. It is not suitable for me. — Oleg
– Oleg, Commented Nov 25, 2017 at 18:33
You could hook into the save or connect to the post_save signal, and call your publish task via apply_async(eta=instance.publish_date). Of course the task itself needs to recheck if the date hasn't changed. — user2390182
– user2390182, Commented Nov 25, 2017 at 18:45
That is possible too, hooking to save is more secure though, e.g. if publishing dates are changed any other way, like through the console. — user2390182
– user2390182, Commented Nov 25, 2017 at 18:59

HQM · Accepted Answer · 2017-11-26 03:45:06Z

1

you can execute tasks by using paramater eta on apply_async() (refer to http://docs.celeryproject.org/en/latest/userguide/calling.html#eta-and-countdown)

eta parameter must be in datetime format, so it will be run on the exact millisecond precision.

I recommend that you use ORM, so the datetime data type from your database will convert automatically by ORM :D

let's say we have an Article model:

class Article(models.Model):
    title = models.CharField(max_length=100)
    published = models.BooleanField(default=False)
    created_date = models.DateTimeField()

and then our tasks module:

@app.task(bind=True)
def set_published(*args, **kwargs):
    article = Article.objects.get(id=args[1])
    article.published = True
    article.save()

    print("article %d has been published" % args[1])

Save the new article and call set_published with ETA+1minute

article = Article(title='This is a new article', published=False, created_date=datetime.utcnow())
article.save()

delta = timedelta(minutes=1)
set_published.apply_async(args=(article.id,), eta=article.created_date + delta)

edited Nov 26, 2017 at 3:45

answered Nov 25, 2017 at 23:02

HQM

5861 gold badge4 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

HQM Over a year ago

edited ! this is might be the simple solution, you can use signal (like post_save) in model for calling the set_published function so it will be invoked everytime Article saved

Collectives™ on Stack Overflow

How to execute tasks in Celery using datetime from MySQL?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related