How to keep each celery class based task in separate file in django project?

Question

I have setup celery in my django project using official documentation at
http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html#using-celery-with-django

So my project structure is

└── mysite
    ├── db.sqlite3
    ├── manage.py
    ├── mysite
    │   ├── celery.py
    │   ├── __init__.py
    │   ├── settings.py
    │   ├── urls.py
    │   └── wsgi.py
    └── polls
        ├── admin.py
        ├── apps.py
        ├── forms.py
        ├── __init__.py
        ├── migrations
        │   ├── 0001_initial.py
        │   └── __init__.py
        ├── models.py
        ├── tasks.py
        ├── tests.py
        └── views.py

polls is application
polls/tasks.py have class based celery task.
Currently tasks.py have many tasks so that file is too big. I want to keep each task in separate file like

mysite/polls/
├── admin.py
├── apps.py
├── forms.py
├── __init__.py
├── migrations
│   ├── 0001_initial.py
│   └── __init__.py
├── models.py
├── tasks # I want to keep easy task in separate file like this
│   ├── __init__.py
│   ├── download_task.py
│   ├── process_task.py
│   └── upload_task.py
├── tests.py
└── views.py

How to make this setup working?

2ps · Accepted Answer · 2018-12-12 19:13:24Z

3

That is 100% correct. In your tasks/__init__.py file, make sure to import the tasks from the other files:

    from .download_task import *  
    from .process_task import *
    # etc...

And then make sure you have the autodiscover_tasks call in your celery.py file to discover the tasks in each of your INSTALLED_APPS.

answered Dec 12, 2018 at 19:13

2ps

16k2 gold badges34 silver badges53 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

tpalves · Accepted Answer · 2023-06-16 13:35:57Z

0

2023 Update

You can set the autodiscover_tasks in your celery.py to look up other files.

eg. app.autodiscover_tasks(related_name='tasksv2')

This will search for 'taskv2.py' in your apps folder.

answered Jun 16, 2023 at 13:35

tpalves

1

1 Comment

Community Over a year ago

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

Fan Chen · Accepted Answer · 2024-05-30 19:17:52Z

0

In the file where you instantiated Celery app.

Try this:

def make_celery():
    celery = Celery()
    celery.conf.broker_url = os.environ.get(
        "CELERY_BROKER_URL", "redis://localhost:6379"
    )
    celery.conf.result_backend = os.environ.get(
        "CELERY_RESULT_BACKEND", "redis://localhost:6379"
    )
    return celery

celery = make_celery()

# for class based task
# from tasks.download_task import DownloadTask 
celery.register_task(DownloadTask)

# for function based task
# from tasks.download_task import download_task
celery.task(download_task)

answered May 30, 2024 at 19:17

Fan Chen

413 bronze badges

Collectives™ on Stack Overflow

How to keep each celery class based task in separate file in django project?

3 Answers 3

Comments

2023 Update

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2023 Update

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related