1,746 questions
2
votes
0
answers
513
views
Long running Django Celery task close DB connection
I have a Django Celery task that performs long-running operations and sometimes loses connection to the database (Postgres).
the task looks like something like this:
@app.task(name='...
- 8,760
0
votes
1
answer
629
views
AttributeError: 'Logger' object has no attribute 'warn'
I'm trying to connect celery to my django project via docker.
But when starting the worker container, I get the following error -
File "/usr/local/lib/python3.13/site-packages/kombu/transport/...
- 49
0
votes
0
answers
137
views
What is difference between Celery Beats, Celery worker, Threads and Multiprocessors?
What is the difference between:-
Celery beats and Celery Worker, and if Celery worker works just like threads or multiprocessors, why do we use celery worker especially?
Thank you.
I tried working ...
user22996319
1
vote
1
answer
2k
views
How to configure celery with SQS as backend?
I'm trying to setup a SQS broker with a celery app, configured in a django project.
Here's my setup:
celery.py:
import os
from celery import Celery
os.environ.setdefault("DJANGO_SETTINGS_MODULE&...
- 347
1
vote
3
answers
2k
views
Django Celery Results Table Missing periodictaskname and taskname Fields
I have set up Django with Celery, using django_celery_results and django_celery_beat to schedule tasks and store the results. The tasks are running successfully, but I've encountered an issue with the ...
0
votes
1
answer
71
views
celery beat ExpressionDescriptor.py get_full_description
I hope your day is going well. I've hit a bit of a snag with a Django project I’m working on. I'm relatively new to Django and would deeply appreciate it if you could spare a moment to help me ...
- 61
0
votes
0
answers
406
views
How to assign user for Celery tasks?
I'm planning a project that will make use of Celery tasks that will perform CRUD operations on my models. In addition, I have Signals in place that will listen for those CRUD operations and I want to ...
- 3,258
0
votes
1
answer
354
views
Celery Beat not Scheduling Task in Django Project
I'm working on a Django project where I'm using Celery for task scheduling. I have a periodic task that should run every minute, but it seems that Celery Beat is not picking up this task. Here's my ...
- 1
1
vote
0
answers
99
views
Using celery group throws NotImplementedError
I`m trying to call celery tasks from another celery task, here is a example code:
from celery import group, allow_join_result
@shared_task
def task1():
some_code1()
@shared_task
def task2():
...
- 11
0
votes
0
answers
188
views
How save the result from task in database using django_celery_results?
I am beginner with Celery, and i need to save the result of the task in the database
here is tasks.py:
from __future__ import absolute_import, unicode_literals
from celery import shared_task
import ...
- 118
0
votes
1
answer
197
views
Python Django cannot run celery when methods are defined in a class
I am new to celery, and right now I am using it compute Django tasks in the background. The problem I am having can be demonstrated using the below class method, where I am defining 3 functions inside ...
- 725
1
vote
1
answer
573
views
Access request or request.user in celery task
I am trying to access request.user in django celery task function but it is unable to access as the function is not accepting any request instance, so How can I access it?
@shared_task
def ...
- 45
2
votes
0
answers
187
views
There is always that one celery worker process, can't use kill command because the PID is constantly changing
I'm in the process of developing a project and I'm working on implementing background tasks and notifications for user feedback. I'm using Django in conjunction with Celery for task management and ...
0
votes
1
answer
149
views
wrong with CELERY_BEAT_SCHEDULER in setting.py
setting.py
# celery
CELERY_TIMEZONE = "UTC"
CELERY_BROKER_URL = "redis://127.0.0.1:6379"
CELERY_ACCEPT_CONTENT = ["application/json"]
CELERY_RESULT_SERIALIZER = "...
1
vote
0
answers
147
views
Celery task not getting aborted using AbortableTask
I have a long running celery task. I want to terminate/stop/revoke that task if user hits /cancel route. I used multiple ways of doing this, but none of them worked. I am using RabbitMQ as a message ...
- 107
0
votes
1
answer
348
views
how to setup celery beat for scrapy project?
I have a scrapy project and I want to run my spider every day so I use celery to do that. this is my tasks.py file:
from celery import Celery, shared_task
from scrapy.crawler import CrawlerProcess
...
- 3
2
votes
0
answers
118
views
Django Celery Beat Periodic Tasks Running Twice
I trying to send scheduled emails to users but my Django celery periodic tasks running 2 times within a half-hour span. Every day I scheduled my email task at 7.00 AM but it is running 1st task at 7....
- 4,122
1
vote
0
answers
339
views
Celery in Django: celery beat doesn't work
settings.py:
#CELERY
CELERY_BROKER_URL = 'redis://localhost:6379' # URL для Redis брокера
CELERY_RESULT_BACKEND = 'redis://localhost:6379' # URL для Redis бэкенда результатов
# Настройки для Celery ...
- 77
0
votes
1
answer
126
views
how to integrate django and scrapy and celery?
I have a django project and in its root directory i have a scrapy project that has a spider that should be run every 24 hours and it scrapes provided urls and saves data to a mysql database using ...
- 3
1
vote
0
answers
38
views
how to make celery able to handle custom import
hello every one how do i make celery able to handle custom models, i have a model i want to perform some tasks on i tried importing it but i keeps throwing errors, below is the code.
the error tend to ...
3
votes
2
answers
12k
views
Error response from daemon: failed to create task for container
I have a django app. That also has redis, celery and flower.
Everything is working on my local machine.
But When I am trying to dockerize it The redis and django app is starting. But celery and flower ...
- 2,417
0
votes
1
answer
650
views
Time Limit not Working for Django Celery 5.3.1
I am having a trouble with trying time_limit and soft_time_limit feature in celery and django app.
import os
from celery import Celery
# Set the default Django settings module for the 'celery' ...
1
vote
0
answers
179
views
Celery worker not executing task properly(tasks are being received from beat)
I am trying to add celery-redis combo to my django project. Installed celery, django-celery-beat and made other set ups. I wrote a simple task to sent me a message in telegram(added logger.info at the ...
0
votes
1
answer
72
views
How to stop/cancel the celery task by passing 'task_id' to a python function in django?
I have celery task Id, I want to send it as query param to a django url endpoint and cancel/revoke the task using that task_id. Is it possible? If so, how?
- 107
3
votes
1
answer
1k
views
Stop the currently running task by celery worker in Django
I have a Django application and React as my frontend framework and I am using celery for running lengthy tasks which do take a lot of time and everything is working fine.
Also I am providing real time ...
- 157
0
votes
1
answer
157
views
Django-Celery Daemon is unable to connect to Redis on Elasticbeanstalk
I am using celery with django for may scheduled tasks. Locally it worked. When i deployed it to Elasticbeanstalk the Celery Daemon is always connecting to rabbitmq and failing since there is no ...
- 1,041
2
votes
0
answers
385
views
Celery signature/delay AttributeError: 'UUID' object has no attribute 'game_id'
First I chain tasks like this
tasks_chain = chain(create_game_folder.si(instance.id))
tasks_chain |= download_game.si(instance.id).set(
task_id=str(task_id)
)
But I get this error
...
- 1,601
0
votes
1
answer
1k
views
docker-compose unable to build
This is the Docker file I am building using docker-compose build, and its always stuck on 9/38 and its building process is not finishing. Here is the main code of the docker file:
`###########
## ...
0
votes
1
answer
77
views
Celery not running new task
I've readded an old Celery task to my Django app running on Heroku, but it's not being run by the worker. The task appears in the tasks when the worker starts, and I can see the scheduler sending it ...
- 13
0
votes
1
answer
1k
views
Using Celery with Amazon SQS, messages are waiting in 'Messages Avaliable'
I have a problem with Celery. I am using Celery with Amazon SQS. I set up everything about Celery and SQS.
The function in tasks.py works without delay() but not working with delay().
I succesfully ...
- 59
3
votes
1
answer
870
views
Same Message Group Tasks executing in parallel in Celery with Amazon SQS FIFO
I have a setup where I'm using Celery as the task queue with Amazon SQS FIFO. My goal is to ensure sequential processing of tasks within the same message group ID, while allowing tasks with different ...
- 33
0
votes
1
answer
315
views
Upgrading Redis on Heroku causing SSL errors in Celery
I've recently upgraded our Heroku Redis from version 5.0 to 7.0.11 and I'm now receiving the following error when running Celery
Traceback (most recent call last):
File "/app/.heroku/python/...
- 1
3
votes
1
answer
156
views
Celery Eventlet OSError: Load averages are unobtainable
Our django celery server is running with concurrency value of 500 with eventlet as execution pool. Recently, we encountered this issue and restarting the server fixed it. I am attaching the traceback ...
- 31
0
votes
0
answers
243
views
Unable to send email using celery in DRF
I'm unable to send email via smtp using celery task and DRF.
I have the following constants in settings.py
# Celery settings
CELERY_BROKER_URL = 'redis://127.0.0.1:6379/0'
CELERY_RESULT_BACKEND = '...
- 106
0
votes
1
answer
3k
views
Unable to connect to redis running on docker from celery
I'm using the django application where i want to run celery workers. But when i hit python -m celery -A cartpe worker, i get the following error
consumer: Cannot connect to redis://redis:6379/0: Error ...
- 106
3
votes
1
answer
325
views
Celery is not consuming with two tasks together
I have celery snippets as follows:
first define the Task Class in celery_tasks.tasks.py
import celery
class LoggerDefine(celery.Task):
name = 'message-logger'
def run(self, payload):
...
- 81
1
vote
1
answer
568
views
Celery chain of groups and tasks doesn't call next task if group fails
I have created a celery chain that consists of group of tasks and normal tasks, shown below
@app.task
def task_initial(id):
# do something
print(id)
@app.task
def task_to_group(id):
# do ...
- 1,430
14
votes
1
answer
6k
views
How does Celery worker run the code defined elsewhere in a task?
I tried reading official documentation as well as other SO threads, but it is still not clear how Celery works.
From what I understand:
Django app: Celery is installed in Django (or any app) where @...
- 1,086
1
vote
1
answer
1k
views
connection Refused for client AWS RabbitMQ with celery & Django
I am a noob in AWS RabbitMQ, Celery and I have some issues going on. Please have a look:
pip install celery==5.2.7
My settings.py:
CELERY_BROKER_URL = 'amqps://username:[email protected]....
0
votes
1
answer
377
views
Unable to call "celery status" in Django, RabbitMQ, Celery container on Docker. KeyError: 'No such transport'
I have a docker container setup with Django, where I'd like to use Celery. The broker is RabbitMQ, and for the results backend I'm using django-celery-backend. Once everything is up and running, I get ...
- 107
0
votes
1
answer
398
views
Unable to test a celery chain when tasks are executed by different celery apps
I have two celery apps in my environment that are :
app1 = Celery('app1', broker=BROKER_URL1, backend=BROKER_URL1)
app2 = Celery('app2', broker=BROKER_URL2, backend=BROKER_URL2)
From a Django app ...
- 400
1
vote
0
answers
162
views
Celery workers receive the task but do not process it
I’m trying to implement Celery and RabbitMQ in my Django project with poor results. So I’m trying to create a simple script that simply prints the result of the sum.
from celery import Celery
from ...
0
votes
1
answer
583
views
setting monitor_slug="slug-name" in @sentry_sdk.crons.monitor() doesn't work
When using sentry to monitor my django-celery cron job.
I add a Monitor to sentry.io/corns
Then use @sentry_sdk.crons.monitor to decorate my task
@shared_task
@sentry_sdk.crons.monitor(monitor_slug=&...
- 5,748
1
vote
0
answers
641
views
Using Celery + SQS with Django - Celery not respecting broker_transport_options
I'm trying to set up a celery worker with AWS SQS as the message broker. However, celery does not use the queue I have defined, nor does it even use the correct region. It instead creates a us-east-1 ...
0
votes
1
answer
685
views
Issue with file write using Django and Celery
I have such a model for representing posts in my system.
class Post(models.Model):
caption = models.CharField(max_length=256)
text = models.CharField(max_length=256, null=True, blank=True)
...
- 137
0
votes
1
answer
355
views
Broken pipe error when using @shared_task in Django
I am utilizing the celery library to execute tasks in the background even when the user closes or refreshes the page. Since the task takes some time, I opted to use celery.
The algorithm works fine as ...
- 252
0
votes
0
answers
176
views
How to update status of Celery's tasks in Django?
Friends I have a question: I want to update the status of celery's tasks in Django. An example of what I want:
1- Create a task in tasks say to extract numbers from multiple text files:
@shared_task()
...
- 23
1
vote
1
answer
1k
views
Celery is not able to find my django module
I’m facing the below error when I try to start the celery using the command celery -A blogger.celery worker. I’m running the command from my package directory where I have my celery.py
...
- 13
0
votes
1
answer
485
views
Celery vs. AWS Lambda funcions for lot of long running tasks
Is better to use for a lot of long running tasks Celery or AWS Lambda function or something different? I have a function that makes http requests. each function need to run separately and parallel, ...
- 1
2
votes
2
answers
1k
views
Trying to Run Celery Tasks on Remote Server
I have been scouring the Internet for some sort of guide for setting up django, celery, and redis so the celery task runs on a remote server, and I cannot seem to find out how to do this.
I have three ...
- 935