Celery Beat not Scheduling Task in Django Project

Question

I'm working on a Django project where I'm using Celery for task scheduling. I have a periodic task that should run every minute, but it seems that Celery Beat is not picking up this task. Here's my setup:

tasks.py:

from celery.schedules import crontab
from ..apps.scraper.scrapers.updates import get_updates
from .settings import app

app.conf.beat_schedule = {
    "get_updates": {
        "task": "..apps.scraper.scrapers.updates.get_updates",
        "schedule": crontab(minute="*/1"),  # Run every 1 minute
    },
}

celery.py:

from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "manga_project.settings")

app = Celery("manga_project")

app.config_from_object("django.conf:settings", namespace="CELERY")

app.autodiscover_tasks()

app.conf.broker_url = "redis://localhost:6379/0"

updates.py:

@app.task
def get_updates():
    print("Looking for manga updates...")
    mangas = Manga.objects.all()
    for manga in mangas:
        print("Checking ", manga.title, " ...")
        scrape_chapters("str", manga.url)

I expect the get_updates task to run every minute, but it's not happening. When I start Celery Beat, it doesn't seem to pick up this task when I check the debbuger.

I tried restarting celery and adding some error handling, but the tasks just simply doesnt get picked up so I am not sure how to handle this.

Any advice would be appreciated.

k4anubhav · Accepted Answer · 2023-10-22 23:06:29Z

1

autodiscover_tasks Searches a list of packages for a "tasks.py" module which is default related_name for search.

As your tasks reside in updates.py

so change app.autodiscover_tasks() to app.autodiscover_tasks(related_name='updates') to check for updates.py

read autodiscover_tasks parameters and what it does

You can take a look here if you have task files are not in the same relative location Celery discover tasks in files with other filenames

and of course, you can change the updates.py to tasks.py if that works

answered Oct 22, 2023 at 23:06

k4anubhav

1301 silver badge9 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Celery Beat not Scheduling Task in Django Project

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related