celery with redis backend

Question

Ok, I got everything installed properly to get celery + redis in django

I got

INSTALLED_APPS = (
    'djcelery',
    'kombu.transport.django',
)

These are in my settings

CELERY_REDIS_HOST = 'localhost'
CELERY_REDIS_PORT = 8889
CELERY_REDIS_DB = 0
CELERY_RESULT_BACKEND = 'redis'

BROKER_URL = "redis://localhost:8889/0"

REDIS_CONNECT_RETRY = True
CELERY_IGNORE_RESULT = True
CELERY_SEND_EVENTS = True
CELERY_TASK_RESULT_EXPIRES = 60
CELERYBEAT_SCHEDULER= 'djcelery.schedulers.DatabaseScheduler'

I got redis working with

./redis-server

It outputs to the terminal every second or so I can also run

./manage.py celeryd -E -B --loglevel=INFO -n w1.d820

Without errors I can even see my tasks being added and successfully finished with

./manage celeryev

So even thought it gets logged by celeryev this code doesnt work:

from celery.task import task
@task
def add(x, y):
    return x + y

res = add.apply_async(args=[1,5])
print res.wait()

It just hangs without ever returning the result, I can see its actually trying to get it from redis because if I do

./redis-cli
MONITOR

I get lots of GETs trying to get an non existance key So my conclusion is that django is not saving to the backend Whats wrong with my settings? I think Iam missing something obvious and I can't see it because I've been too long fighting this. Help

Btw I can't use the standard port 6379 because I am on a shared server.

UPDATE using these

celery==2.5.3
django-celery==2.5.5
django-celery-with-redis==2.5

asksol · Accepted Answer · 2012-06-09 12:27:30Z

7

Do you have import djcelery; djcelery.setup_loader() in your settings.py?

You can find out where it is trying to connect by running the following:

 $ python manage.py celeryctl shell

 >>> celery.broker_connection().as_uri()

The 'setup_loader' is very important, so you must make sure that you include it.

(the good news is that celery will support Django out of the box for version 2.7).

answered Jun 9, 2012 at 12:27

asksol

19.6k5 gold badges67 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Guillermo Siliceo Trueba Over a year ago

i did have those lines in my settings. +1 for how to know where is trying to connect celery.

Guillermo Siliceo Trueba · Accepted Answer · 2012-06-09 01:49:46Z

4

In ubuntu 10.10 when you install redis it automatically starts at boot, and it was conflicting with celery because apparently even if you tell it to use this or that port if celery can connect to redis in 6379 port it will use it as a backend. I've not seen the code but is my conclusion. So i closed all the redis instances ran my own with my redis.conf on whatever port i want and then i put these on my settings.

BROKER_URL = 'redis://localhost:8889/0'
REDIS_DB = 0
REDIS_CONNECT_RETRY = True
CELERY_RESULT_BACKEND = 'redis'
CELERY_REDIS_PORT = 8889
BROKER_PORT = 8889
CELERY_RESULT_PORT = 8889
CELERY_TASK_RESULT_EXPIRES = 10
CELERYBEAT_SCHEDULER= 'djcelery.schedulers.DatabaseScheduler'

answered Jun 9, 2012 at 1:49

Guillermo Siliceo Trueba

4,6496 gold badges38 silver badges49 bronze badges

Comments

Michael · Accepted Answer · 2012-06-09 01:01:16Z

0

What version of celery are you using? In the development version it looks like they use a different Decorator;

from celery.task import task

@celery.task()
def add(x, y):
    return x + y

answered Jun 9, 2012 at 1:01

Michael

834 bronze badges

2 Comments

sberry Over a year ago

His decorator is right. Where are you seeing that? That wouldn't even work. You would have to do import celery for this to work.

Michael Over a year ago

I took that straight from the development versions example :) See here: celery.github.com/celery/userguide/tasks.html

Kalail · Accepted Answer · 2012-06-09 02:01:29Z

0

Are you starting the Redis server on the port 8889?

answered Jun 9, 2012 at 2:01

Kalail

1557 bronze badges

1 Comment

Kalail Over a year ago

Hmmmm. Try using: add.delay(1, 5) Instead of: add.apply_async(args=[1,5]) See if that works?

Collectives™ on Stack Overflow

celery with redis backend

4 Answers 4

1 Comment

Comments

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related