1,746 questions
1
vote
1
answer
464
views
Celery with Mongodb as brocker and backend
I followed the code from the below
Working example of celery with mongo DB
Getting the below error
CRITICAL/MainProcess] Unrecoverable error: TypeError("'Collection' object is not callable. If ...
- 303
2
votes
2
answers
1k
views
Django + DRF + Celery: execute celery_task.delay() only after database transaction completed (model.save() reached the database)
I tried to use the on_commit method as they tell us in the docs, but I still get ModelDoesNotExistError inside the task launched via delay().
views.py:
class SomeViewset(viewsets.ModelViewset):
......
- 719
16
votes
2
answers
20k
views
Django + Celery + Redis: kombu.exceptions.OperationalError: [Errno 111] Connection refused
Although celery reports no problems at start and says it successfully connected to redis (see log), I get this error running celery inspect ping
Traceback (most recent call last):
File "/usr/...
- 719
0
votes
1
answer
756
views
How to connect redis for celery to a django app deployed in digital ocean
I hae recently deployed a django app in digitalocean. My app uses celery and redis for several email sending puposes and scheduled tasks. I have been searching for a documentation on how to connect ...
0
votes
0
answers
237
views
django TimeField getting saved with wrong time zone
I have a simple Group model:
class Group(models.Model):
leader = models.ForeignKey(User, on_delete=models.CASCADE)
name = models.CharField(max_length=55)
description = models.TextField()
...
- 133
1
vote
0
answers
331
views
Django Celery issues within Docker
Having a major issue I cannot track down and I'm spinning my wheels. Any help would be greatly appreciated. I have django, celerybeat and celeryworker containers that all use the same base image of ...
- 554
1
vote
1
answer
429
views
Change the celery workers' code in production without task loss
I have a system that has important long-running tasks which are executed by Celery workers. Assume that we have deployed our application using k8s or docker-compose.
How can I change the celery ...
1
vote
1
answer
2k
views
Celery scheduler not performing the task
I was trying to use Celery to query an external api at a regular frequency and update my database in my Django project with the new data. Celery schedules the task correctly and sends it to the celery ...
- 17
7
votes
1
answer
4k
views
How to make a celery worker stop receiving new tasks (Kubernetes)
So we have a kubernetes cluster running some pods with celery workers. We are using python3.6 to run those workers and celery version is 3.1.2 (I know, really old, we are working on upgrading it). We ...
- 2,163
0
votes
2
answers
370
views
How to run parallel tasking with celery django?
I am looking to run tasks in parallel with django celery.
Let's say the following task:
@shared_task(bind=True)
def loop_task(self):
for i in range(10):
time.sleep(1)
print(i)
...
- 85
1
vote
1
answer
484
views
Django celery with redis is executing same task multiple times
I'm trying to creating a background task with Django celery and Redis as broker. this task is being sent from models when model post save. But the problem is that, the same task is getting executed ...
- 141
4
votes
0
answers
329
views
celery re-running the same tasks over and over again indefinitely if there are any issues with rabimq/celery server in Python
I have searched about this but did not get an expected answer.
There is the same question asked but it applies to redis which can be solved by setting,
BROKER_TRANSPORT_OPTIONS = {'visibility_timeout':...
- 344
0
votes
1
answer
46
views
How to pass data between views and forms using form_valid
I have a simple UserRegistration view:
class UserRegistration(generic.CreateView):
form_class = RegisterForm
template_name = 'registration/registration.html'
success_url = reverse_lazy('...
- 133
0
votes
1
answer
313
views
How to send emails with Django/Celery based on DateTimes stored in the database?
I want to send reminder emails based on different dates saved in the database. I saw some links that was kinda related to that but most of them was really messy.
Table
send_email_at: [2022/07/12, 2022/...
- 367
0
votes
3
answers
2k
views
sending periodic emails over django using celery tasks
I have a Group model:
class Group(models.Model):
leader = models.ForeignKey(User, on_delete=models.CASCADE)
name = models.CharField(max_length=55)
description = models.TextField()
joined = ...
- 133
0
votes
1
answer
1k
views
How to kill a Celery task if it's not finished after a set period of time?
I have a Celery task that runs the main function of external Python scripts. However, some of the scripts may take a very very long time to execute and I'd like to set a time limit after which the ...
- 1,105
0
votes
0
answers
667
views
Want to call a 2 get api's using celery in django at sheduled time everyday byitself
dear community members. I have created a Django project which working totally fine with whatever API I called.
But here I have one small query that there are 2 APIs from which from both APIs while get ...
- 43
0
votes
1
answer
430
views
Celery Send Email on Due_date
Sending email to the users based on the due date in model using the celery, the due_date is something different from date when a task is created
Models.py
class Task(BaseModel):
name = models....
- 75
1
vote
1
answer
1k
views
How to get nodename of running celery worker?
I want to shut down celery workers specifically. I was using app.control.broadcast('shutdown'); however, this shutdown all the workers; therefore, I would like to pass the destination parameter.
When ...
1
vote
0
answers
1k
views
I want to route to 2 different CELERY_BROKER_URL
I want to use 2 different CELERY_BROKER_URL for 2 different IP's and specific task should go in the another URL (ex - redis://127.0.0.2:6379).
environment file
CELERY_TASK_TRACK_STARTED=True
...
- 13
0
votes
0
answers
107
views
How can I see how many child-tasks a pool worker is currently executing?
I'm using Celery in Django and the workers are eating up all of the memory of my Dynos in Heroku. Setting the --max-tasks-per-child seems to be a valid solution but I can't seem to find what the ...
- 4,222
6
votes
2
answers
5k
views
Django Celery Redis
I have this error A rediss:// URL must have parameter ssl_cert_reqs and this must be set to CERT_REQUIRED, CERT_OPTIONAL, or CERT_NONE
settings
CELERY_BROKER_URL = os.environ.get('REDIS_URL', "...
- 355
2
votes
0
answers
150
views
Limit different number of workers for different groups of celery tasks
For different celery job groups, how can we limit the number of workers asynchronously differently.
Here is a good reference on how to group different celery tasks together : https://sayari3.com/...
- 149
0
votes
1
answer
680
views
Tell completion for a set of celery tasks
How I can run a another method action() automatically when a set of celery tasks is finished. Are there any simple way to trigger another function call on completion?
#tasks.py
@app.task
def rank(item)...
- 149
1
vote
1
answer
1k
views
Celery 'update_status' function is not working
I am using celery and django-celery-results in my Django application. My celery results are being stored in the backend and this is what celery.py looks like.
os.environ.setdefault("...
- 1,086
1
vote
1
answer
1k
views
Celery task suddenly missing
I have a long running celery task which I can see it in flower at the beginning of execution but after some hours, it's like it doesn't exist! All I see in task details page is Unknown task '894a8b45-...
- 3,362
0
votes
1
answer
1k
views
Getting OperationalError: (2000, 'Unknown MySQL error') when accessing mysql database using celery and django
I am using celery with my django application. My application works fine with mysql database, but I am getting (2000, 'Unknown MySQL error') when celery tries to access the database. This happens only ...
- 700
0
votes
0
answers
134
views
AttributeError: 'name_Funcition/task' object has no attribute 'GET' - Django, Celery + RabbitMQ
I'm starting Celery+RabbitMQ to use async functions on my Django system and I'm facing some issues.
As I understand it, when using Celery+RabbitMQ, the function that I insert in my tasks.py file will ...
0
votes
1
answer
240
views
Prevent celery from running my Django app threads
I have a Django app and I am using celery (with redis) for running processing tasks in the background. In addition, I have a python thread which runs some periodic checks as part of my Django app. ...
- 700
0
votes
1
answer
1k
views
Error while running celery worker : ModuleNotFoundError: No module named 'mysite'
I was using celery and rabbitmq to send email to the user who has logged in. However when I am running my celery worker I am getting :
My project name is myproject and the appname is app
celery -A ...
- 7,901
0
votes
1
answer
465
views
Django: Celery only works on local but not production using the cookiecutter project
I set up a project using the django cookie cutter and deployed it with the docker option https://cookiecutter-django.readthedocs.io/en/latest/deployment-with-docker.html
Celery perfectly works on my ...
- 223
2
votes
0
answers
236
views
how can I avoid time out in django
I am creating a site that downloads videos from other sites and converts them to GIF when requested.
The problem is that it takes too long to download and convert videos.
This causes a 504 timeout ...
- 169
2
votes
1
answer
2k
views
Celery's task_reject_on_worker_lost doesn't work with Redis as message broker
I'm currently using version 5.2.6 of Celery and version 6.2.6 of Redis. When I turn on the task_reject_on_worker_lost flag, I am expecting Celery to redeliver a task executed by a worker that died ...
- 811
1
vote
2
answers
1k
views
Celery Limit the Number of Tasks Running per User
I have a task in Celery that looks like this:
@app.task(name='task_one')
def task_one(user_id, *args, **kwargs):
# Long running task
This task is created in views every time a user submits a form,...
- 1,159
0
votes
1
answer
867
views
Is it possible to redeliver a Celery worker's tasks as soon as the worker stops sending heartbeat?
According to the celery documentation on Celery using Redis broker,
The visibility timeout defines the number of seconds to wait for the worker to acknowledge the task before the message is ...
- 811
-1
votes
1
answer
189
views
Has there been a fix for this issue related to celery? https://github.com/celery/celery/issues/3519
Celery seems to be picking both my old code and new code. I have tried clearing cache, clearing the broker queue(redis), restarting celery etc. But none of them seem to be fixing this issue.
For ...
- 94
1
vote
0
answers
155
views
Celery / Celery beat stuck on old version of a script
I've got a django application running on AWS EC2 using Celery for periodic updates. I recently had to update a data processing script on the Django app, but no matter what I seem to try, Celery ...
- 169
0
votes
0
answers
190
views
Django-celery can't work in another environment but work in localhost
I try to learn and pratice celery with Django and it works only in a localhost environment. The problem that I have is when I build it on another environment it returns an error like this
web_1 ...
4
votes
1
answer
2k
views
upgrading celery 4.x.x to 5.x.x in Django app - execute_from_commandline() replacement
the usage in 4.x.x was as following:
from tenant_schemas_celery.app import CeleryApp
class TenantCeleryApp(CeleryApp):
def create_task_cls(self):
return self.subclass_with_self('...', ...
user2552806
0
votes
1
answer
398
views
Official celery project for django is not working
I am using the official celery project for django, but it is not working on my machine.
I have installed all the necessary modules, and am using the example given in the link: Example Django project ...
- 15
0
votes
1
answer
650
views
Django Export Excel Save File In Server (Celery + RabbitMQ)
I have a Django view that exports an Excel File and prompts a Download Dialog when the file is ready. I am installing Celery and RabbitMQ to make this task a background task. This means that the excel ...
- 357
0
votes
1
answer
2k
views
Dynamically change celery beat schedule params
I get schedule values from .env file. And sometimes parameters in .env file change.
Is it possible to change schedule values of already running celery beat tasks?
My celery.py:
import os
from celery ...
0
votes
1
answer
3k
views
Many tasks at once in Celery?
If we use celery beat and run about 1000 tasks by same crontab schedule, will tasks run one by one or some tasks will not run (cause of out of time)?
redis as MQ
- 45
1
vote
0
answers
224
views
How to reduce/specify number of connections in celery flower?
I am running an app on a limited service host, where my Redis server has a maximum of 50 connections. I have configured my web app to be using a consistent 20 Redis connections. However, if I launch ...
- 145
0
votes
1
answer
245
views
Change saved filepath in a django celery task
I have a model named Flights
class Flights(models.Model):
field = models.ForeignKey(Field, on_delete=models.CASCADE)
datetime = models.DateTimeField(blank=True, null=True, default=timezone.now(...
- 848
-1
votes
1
answer
940
views
How to Print celery worker details on windows and mac
I actually had issues printing mine on command prompt because I was using the wrong command but I found a link to a project which I forked Project
(If on Mac ) celery -A Project worker --loglevel=...
- 1,087
1
vote
1
answer
1k
views
How to retry Django Celery task on Exception when Internal Server Error is raised?
I am trying to use Celery to send anywhere from 1 to about 25 consecutive requests to a third-party API. The measure of success is whether I get a URL back in the response payload: response_json["...
-1
votes
2
answers
546
views
Django import my app models AppRegistryNotReady
I will import my models in celery.py. But when I import and run the runserver command, I get the following error:
File "/directory/manage.py", line 22, in <module>
main()
File &...
- 109
0
votes
1
answer
819
views
Terminate previous Celery task with same task id and run again if created
In my django project, I have made a view class by using TemplateView class. Again, I am using django channels and have made a consumer class too. Now, I am trying to use celery worker to pull queryset ...
0
votes
1
answer
782
views
Celery chord failing with ValueError: not enough values to unpack (expected 2, got 1)
I am running celery v5.2.3 in a docker container running ubuntu.
Here I am trying to get celery chords to work but every attempt I make give me:
File "/usr/local/lib/python3.7/site-packages/...
