1,746 questions
2
votes
1
answer
262
views
Django celery page is giving 404
I'm trying to acces https://docs.celeryproject.org/en/stable/django/first-steps-with-django.html but getting 404. Is celery not officially available?
- 191
0
votes
0
answers
107
views
Django Celery Beat - Runaway Process on Ubuntu
I have celerybeat running as a service on ubuntu.
Generally speaking, everything works great. I have the same setup on numerous servers.
Last night, I noticed the frequency at which a certain task ...
- 355
0
votes
2
answers
1k
views
Celerybeat Multiple schedules of the same task
I got following Celery beat task which cleans 1000 items daily at 1 AM:
from celery.schedules import crontab
from .celery import app as celery_app
celery_app.conf.beat_schedule['maintenance'] = {
...
- 2,051
0
votes
1
answer
2k
views
Need to return the object from the celery shared task queue process
I have faced the error "kombu.exceptions.EncodeError: Object of type 'JsonResponse' is not JSON serializable "
In my settings.py :
CELERY_BROKER_URL = 'amqp://localhost'
...
- 7
3
votes
0
answers
729
views
Objects created from celery task not firing django post_save signal
I have a celery task that creates a modal object. I'm trying to run some code after the object is created using django post_save signal. but for some reason, the signals are not firing.
models.py
...
- 396
0
votes
0
answers
548
views
WebSocket not connecting |. Django Channels | WebSocket
I am using the WebsocketConsumer consumer and when I disconnect the websocket connection it gives the below error.
Also when I have already one open connection and when I try to create a connection ...
- 711
0
votes
0
answers
120
views
How to fix issue with celery workers writing payloads to the same file but output is often malformed
A legacy django & celery based service has been writing payloads in a newline to a file using celery workers. But I noticed a lot of malformed payloads ... A payload would start in the middle of ...
- 402
0
votes
0
answers
1k
views
Import "celery" could not be resolved
I know this has something to do with python not recognizing the standard library celery but I haven't found anything helpful online so far:
pip freeze output:
amqp==5.0.9
asgiref==3.5.0
backports....
- 483
2
votes
1
answer
1k
views
worker_concurrency configuration not valid for celery
environment: Django3.1, celery5.2
Django setting.py
CELERY_RESULT_BACKEND = 'django-db'
CELERY_BROKER_URL = 'redis://127.0.0.1/0'
CELERYD_CONCURRENCY = 2 # 并发worker数
# CELERYD_PREFETCH_MULTIPLIER=1
...
- 41
0
votes
1
answer
983
views
How to check if the Django-background process is running in server?
The command to start the background process is,
nohup python manage.py process_tasks &
Similarly in Linux which is the command used to check running status?
- 81
0
votes
2
answers
1k
views
Call Celery class-based task in sync mode
I have celery class-based task
from celery import Task
from django.db import transaction
from config import celery_app
class RefreshData(Task):
name = "refresh-data"
@transaction....
- 2,352
0
votes
0
answers
280
views
django settings does not affect after changes(CELERY_BROKER_URL)
CELERY_BROKER_URL = 'redis://127.0.0.1:6379/0'
CELERY_ACCEPT_CONTENT = ['json']
TASK_SERIALIZER = 'json'
[2022-02-14 12:02:26,910: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**@192....
1
vote
0
answers
148
views
celery not importing to app django app even though celery.py exists in project directory
I can't get celery to import to my app in django project.
Here is tasks.py in apis app directory:
from __future__ import absolute_import, unicode_literals
from celery import shared_task
@shared_task
...
- 483
0
votes
1
answer
2k
views
How to parallelize subtasks in celery
I have the following code. This is working fine. But I am iterating metrics in a for loop in get_host_types(). I want to create subtasks from get_host_types() function for each metric which will call ...
- 331
1
vote
0
answers
175
views
Django Celery apply_async doesn't execute all queries
I have used celery apply_async (also tried delay) with django.
The task executes only 3 out of 10 queries where I am performing an update operation on mongo db database. (I am using pymongo)
Only the ...
2
votes
1
answer
3k
views
How to run Scrapy spiders in Celery
There are several posts regarding how to setup Scrapy within a Celery task avoiding to restart the Twister reactor to prevent the twisted.internet.error.ReactorNotRestartable error. I have tried ...
- 5,754
0
votes
1
answer
373
views
Request to Django views that start Celery tasks time out
I'm deploying a Django app with Docker.
version: '3.1'
services:
b2_nginx:
build: ./nginx
container_name: b2_nginx
ports:
- 1904:80
volumes:
- ./app/cv_baza/static:/...
- 678
1
vote
0
answers
1k
views
Django Celery Beat not sending tasks to Celery Worker
I am using Celery in Django in order to run tasks in specific time intervals. When I first start the Docker, all the tasks run without any issue. If I stop the docker (i.e. docker-compose down) and ...
- 1,021
1
vote
2
answers
3k
views
Run periodic celery task with a dynamic schedule in django application
I am wondering if it is possible to have my end users dynamically adjust the schedule of a periodic task.
So something along these lines:
# celery.py
def get_schedule():
config = get_user_config()...
- 355
0
votes
1
answer
2k
views
Django Celery: Clocked task is not running
In a Django app, I have a form that schedules an email to be sent out. It has four fields: name, email, body, send_date. I want to dynamically create a Celery task (email) to run another Celery task ...
- 35
2
votes
0
answers
189
views
Celery/django - chunk tasks
I have a lot of tasks that are being generated that I would like to group in to chunks, but I think in the opposite way as Celery does. From my understanding, I can use the chunks feature to split up ...
- 8,538
0
votes
2
answers
824
views
celery Django task is not working in mock.patch how to get updated value of DB which is being performed in the celery task
I am working on django app, I made a service file which is called service.py and in that file i am calling a celery task i.e.
# start celery task in the apps>myapp>tasks.py
@app.task()
def ...
0
votes
0
answers
361
views
django celery delay function use guest user
Hello I want to use celery tasks in my Django project
When I run the celery -A proj worker -l INFO all good it connects to the rabbitmq server
But I have a problem when I run the task add.delay(1, 1) ...
- 3
1
vote
1
answer
1k
views
Fetching results from Celery backend is abnormally slow
I'm using Celery with a Redis broker to do some "heavy" processing for my Django app. Everything is running locally in Docker containers on WSL2.
The tasks output a JSON which is roughly 2.5 ...
- 93
3
votes
2
answers
9k
views
Django, ImportError: cannot import name 'task' from 'celery'
I have Django application that was using Celery version 4.4.2, which was working fine.
from celery import task
import logging
@task(ignore_result=True)
def log_user_activity(user_id):
try:
...
- 1,615
0
votes
1
answer
517
views
TypeError("predictions() missing 1 required positional argument: 'solvent'") while using celery
I am doing inference on two strings using celery and django rest framework but I am getting error while using .delay() or apply_async(). I am not getting the predicted response
#views.py
@api_view(['...
- 595
0
votes
0
answers
244
views
ImportError: cannot import name 'solvent' from partially initialized module 'api.views' in django rest api
I am using Django rest framework to do prediction on two strings and using celery to run inference. But I am getting error when trying to take solute and solvent from the result function into tasks.py ...
- 595
0
votes
0
answers
221
views
unable to start workers in celery
I am using Django rest framework to do prediction on two strings and using celery to run inference. But I was unable to make use of celery workers even though I have done everything perfectly. I am ...
- 595
4
votes
1
answer
6k
views
Django +docker-compose + Celery + redis - How to use Redis deployed in my own remote server?
I have a Django app deployed in Docker containers.
I have 3 config environnements: dev, preprod and prod.
dev is my local environnement (localhost) and preprod/prod are remote linux environnements.
It ...
- 400
6
votes
2
answers
5k
views
How to deploy Celery worker on DigitalOcean App Platform
I am trying to add celery + redis to my django app hosted on DO App Platform. I understand there is an issue I need to work around that is documented here:
https://www.digitalocean.com/community/...
- 61
1
vote
1
answer
360
views
Django causes error when I'm trying to send email. Error message: [Errno 61] Connection refused
I'm trying to send email from Django. My Django settings configurations are as follows:
# SMTP Settings
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
EMAIL_HOST = "smtp....
- 439
2
votes
1
answer
8k
views
How to set retry tasks in case of failure in Django-Celery
I'm trying to run a task using celery.
I need to send post requests to a remote server while the user presses the send button, So I tried using celery with Redis here with this configuration in ...
- 415
3
votes
1
answer
3k
views
uploading videos in Djnago rest framework
I have created an post api which will upload images and videos for blogs. I can handle multiple images, not a problem there. But I have to send a video as well from the frontend. I have used Filefield ...
- 1,074
0
votes
1
answer
628
views
django import-export-celery cannot import resource
I'm following this repo but I got this error:
Error: Import error cannot import name 'ProfileResource' from 'crowdfunding.models' (C:\_\_\_\_\_\crowdfunding\models.py)
which supposedly makes an ...
- 2,822
0
votes
1
answer
5k
views
OperationalError, Error 111 connecting to 127.0.0.1:6379. Connection refused. After deploying in heroku
I am getting the below error after I deployed my website on heroku.
Error 111 connecting to 127.0.0.1:6379. Connection refused.
Request Method: POST
Request URL: https://website.herokuapp.com/...
1
vote
0
answers
402
views
Periodic Task not running using Celery
I have setup Celery to run a periodic task every 10 seconds that sends a post request to my Django Rest API Framework.
When I run the Celery worker it picks up the task correctly:
[tasks]
. ...
- 69
0
votes
0
answers
630
views
how to load django settings for celery app
This is my tree (hidding all the unnecesary stuff):
src/
├─ api/
│ ├─ app1/
│ │ ├─ tasks.py
│ │ ├─ models.py
│ ├─ celeryworker/
│ │ ├─ celery.py
│ ├─ settings.py
│ ├─ urls.py
├─ manage.py
├─ ...
70
votes
3
answers
38k
views
Celery unable to use redis
Trying to start Celery first time but issues error as below,
i have installed redis and its starting fine , but still somehow django seems to have issues with it ,
File "<frozen importlib....
- 1,186
0
votes
1
answer
1k
views
Schedule emails on a specific time using Django and celery
This is the use case I am looking for :
User will register on my application
It is a note making and remainder application
While creating remainder users will enter the time my application wants to ...
- 57
1
vote
0
answers
297
views
Celery concurrency
I have a celery task that needs to be sequential, i.e the messages need to be evaluated one by one. For this, I have used the concurrency=1 option in the worker startup. But the issue is that the ...
- 711
1
vote
1
answer
465
views
Route celery tasks
In some Django-project there are tasks:
@app.task
def task1():
# processing
@app.task
def task2():
# processing
@app.task
def long_task():
# very long processing
settings.py:
...
- 1,039
1
vote
1
answer
909
views
Django - long running tasks
Im looking for and advice for long running tasks in Django. This is my use case:
I have an eshop order with over 200 products (rows). When I click on "process" Django runs a function (which ...
- 13
1
vote
4
answers
1k
views
Django Celery IntegrityError
I want to create a progress bar for my project. I have a class and this class has some functions. Especially, one of them takes a long time (def download_all) and this is my main reason for wanting to ...
- 680
0
votes
0
answers
185
views
How to run more poll process/task in parallel than number of CPU in celery Prefork
I have django application using celery as task scheduler.
In my scenario, I have to read some data from teradata using pyodbc which range average > 100000 records and need to load that data back ...
0
votes
0
answers
561
views
Celery AttributeError : 'download_all' object has no attribute 'location'
I want to create a progress bar for my project. I have a class and this class has some functions. Especially, one of them takes a long time (def download_all) and this is my main reason for wanting to ...
- 680
1
vote
1
answer
2k
views
Celery beat: Change to individual timezone tasks causing validation error "Invalid timezone"
celery --version
5.1.2 (sun-harmonics)
django --version
3.2.8
I have a celery schedule that has four tasks that run in different timezones. I am using nowfun for setting the timezones and have set ...
- 171
1
vote
1
answer
2k
views
django -celery worker not receiving tasks
I am trying to send emails via Django celery but the tasks are not getting received by the celery.
settings.py
CELERY_BROKER_URL = 'redis://127.0.0.1:6379'
CELERY_RESULT_BACKEND = 'redis://127.0.0.1:...
- 195
1
vote
1
answer
2k
views
billiard.exceptions.WorkerLostError: Worker exited prematurely: signal 9 (SIGKILL) without Docker
[2021-10-27 02:05:09,365: ERROR/MainProcess] Process 'ForkPoolWorker-5' pid:12871 exited with 'signal 9 (SIGKILL)'
[2021-10-27 02:05:09,412: ERROR/MainProcess] Task handler raised error: ...
- 1,613
0
votes
1
answer
1k
views
Avoid Celery retries for unregistered task
We have a Django app with celery to handle an asynchronous tasks. We use AWS SQS as the task broker.
We ended up with a bad task to be processed (removed the task implementation without removing the ...
- 4,388
2
votes
0
answers
253
views
How to quit selenium webdriver when celery task is revoked in Python
I'm running a selenium webdriver in a celery task. However, in some cases, I need to revoke/terminate the celery task. I successfully terminate the celery task using the following
>>> from ...
- 584