58

I went through this example here:

http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html

All my tasks are in files called tasks.py.

After updating celery and adding the file from the example django is throwing the following error, no matter what I try:

ImportError: cannot import name Celery

Is the problem possibly caused by the following?

app.autodiscover_tasks(settings.INSTALLED_APPS, related_name='tasks')

Because it goes through all tasks.py files which all have the following import.

from cloud.celery import app

cloud/celery.py:

from __future__ import absolute_import

import os, sys
from celery import Celery
from celery.schedules import crontab
from django.conf import settings

BROKER_URL = 'redis://:PASSWORD@localhost'

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'cloud.settings')
app = Celery('cloud', broker=BROKER_URL)
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(settings.INSTALLED_APPS, related_name='tasks')

if "test" in sys.argv:
    app.conf.update(
        CELERY_ALWAYS_EAGER=True,
    )
    print >> sys.stderr, 'CELERY_ALWAYS_EAGER = True'

CELERYBEAT_SCHEDULE = {
    'test_rabbit_running': {
        "task": "retail.tasks.test_rabbit_running",
        "schedule": 3600, #every hour
    },
    [..]

app.conf.update(
    CELERYBEAT_SCHEDULE=CELERYBEAT_SCHEDULE
)

retail/tasks.py:

from cloud.celery import app

import logging
from celery.utils.log import get_task_logger
logger = get_task_logger('tasks')
logger.setLevel(logging.DEBUG)

@app.task
def test_rabbit_running():
    import datetime
    utcnow = datetime.datetime.now()
    logger.info('CELERY RUNNING')

The error happens, when I try to access a url that is not valid, like /foobar.

Here is the full traceback:

Traceback (most recent call last):
  File "/opt/virtenvs/django_slice/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 126, in handle_request
    respiter = self.wsgi(environ, resp.start_response)
  File "/opt/virtenvs/django_slice/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 255, in __call__
    response = self.get_response(request)
  File "/opt/virtenvs/django_slice/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 178, in get_response
    response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
  File "/opt/virtenvs/django_slice/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 220, in handle_uncaught_exception
    if resolver.urlconf_module is None:
  File "/opt/virtenvs/django_slice/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 342, in urlconf_module
    self._urlconf_module = import_module(self.urlconf_name)
  File "/opt/virtenvs/django_slice/local/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/opt/src/slicephone/cloud/cloud/urls.py", line 52, in 
    urlpatterns +=  patterns('', url(r'^search/', include('search.urls')))
  File "/opt/virtenvs/django_slice/local/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 25, in include
    urlconf_module = import_module(urlconf_module)
  File "/opt/virtenvs/django_slice/local/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/opt/src/slicephone/cloud/search/urls.py", line 5, in 
    from handlers import SearchHandler
  File "/opt/src/slicephone/cloud/search/handlers.py", line 15, in 
    from places import handlers as placeshandler
  File "/opt/src/slicephone/cloud/places/handlers.py", line 23, in 
    import api as placesapi
  File "/opt/src/slicephone/cloud/places/api.py", line 9, in 
    from djapi import *
  File "/opt/src/slicephone/cloud/places/djapi.py", line 26, in 
    from tasks import add_single_place, add_multiple_places
  File "/opt/src/slicephone/cloud/places/tasks.py", line 2, in 
    from cloud.celery import app
  File "/opt/src/slicephone/cloud/cloud/celery.py", line 4, in 
    from celery import Celery
  File "/opt/src/slicephone/cloud/cloud/celery.py", line 4, in 
    from celery import Celery
ImportError: cannot import name Celery
Improve this question
1
  • please provide full traceback, not just the last line Commented Nov 12, 2013 at 11:07

14 Answers 14

Reset to default
73

Adding the following lines to cloud/celery.py:

import celery
print celery.__file__

gave me the file itself and not the celery module from the library. After renaming celery.py to celeryapp.py and adjusting the imports all errors were gone.

Note:

That leads to a change in starting the worker:

celery worker --app=cloud.celeryapp:app

For those running celery==3.1.2 and getting this error:

TypeError: unpack_from() argument 1 must be string or read-only buffer, not memoryview

Apply the patch mentioned here: https://github.com/celery/celery/issues/1637

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

And don't forget to delete the compiled cloud/celery.pyc in the directory. If not, you'll continue to get the error.
I've seen this exact error, I don't see why renaming the file should be necessary when using from __future__ import absolute_import but that's what I did.
Clever solution, I don't know why developers choose celery.py for the default name, creating this problems and having to rely on from __future__ import absolute_import everywhere.
You can't name it any other than celery, otherwise you won't be able to call it via celery worker -A src
51

With Django 1.7.5, Celery 3.1.17, and Python 2.7.6 I found that I was still getting these ImportError: cannot import name Celery. But only when running tests under PyCharm 4.0.4.

I found that a solution was not to rely on from __future__ import absolute_import as described in First Steps with Django. Instead I renamed proj/proj/celery.py to proj/proj/celery_tasks.py and then changed the content of __init__.py to match: from .celery_tasks import app as celery_app. No more multiple instances of files named celery.py to cause import confusion seemed to be a simpler approach.

Improve this answer

4 Comments

This is exactly what I did to remove any confusion for both the code and myself.
The above worked for me in django 1.8, but I left from __future__ import absolute_import in there and it worked fine. Though I also had to rm *.pyc and restart the dev server.
I had the same error and a slightly different experience. In PyCharm 2017.2 I started getting errors about the imports not working. Turns out it's because I launched PyCharm with the charm command on the command line ... bad idea. That command line inherits from the shell's environment, and on the shell I made my default Python 3.6 and PYTHONPATH leaked into my Python2 project in PyCharm. The solution is to launch PyCharm outside of the terminal, and my problems went away.
This is exactly the solution for me. from __future__ import absolute_import does not work.
17

got the same error

my celery settings filename which was(celery.py) was conflicting with 'celery' package...

so while doing this-> from celery import Celery , it was raising error- cannot import name Celery

solution->just change the 'celery.py' to something else like 'celery-settings.py'

Improve this answer

2 Comments

Also remove celery.pyc that in the end caused my problem.
nicely work for me.
14

In October 2022, importlib-metadata==5.0.0 was released. In python 3.7, this breaks kombu==5.2.4 (see issue#1600) which is used by the current version of celery.

Pin importlib-metadata==4.13.0 or another version less than 5, or update to python 3.8.

Improve this answer

Comments

9

Work for me ( some bug after deploy in server ): Remove all *.pyc files from project and restart him.

Improve this answer

2 Comments

Been trying for hours. Built the entire code in MacOs, then tried running the same application on Windows and I've only got circular imports. Erased all *.pyc files and now it worked!!! Thank you
You can do it with find . -name "*.pyc" -exec rm -f {} \;
8

Did you add the line:

from __future__ import absolute_import

to the top of your cloud/celery.py module?

Read the breakdown of the example here: http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html

Improve this answer

Comments

5

For someone who want to know what cause this error:
I have meet this problem just now, then I found the problem --- sys.path.
Maybe you add some path to sys.path like me, I add below code in manage.py,

ROOT_PATH = os.path.dirname(os.path.abspath(__file__))
SRC_PATH = os.path.join(ROOT_PATH, 'src')
CONF_PATH = os.path.join(ROOT_PATH, 'conf')

sys.path.insert(0, SRC_PATH)
sys.path.insert(0, CONF_PATH)

so, from celery import Celery would search celery in SRC_PATH and CONF_PATH first, that's the problem.

change to 

sys.path.append(SRC_PATH)
sys.path.append(CONF_PATH)

It would search in python's lib and site-packages first. Solved perfectly.

Improve this answer

Comments

4

I got the same error. It turns out there was a problem with my Celery version. I upgraded to 3.1 and celeryd is now deprecated for this version (http://celery.readthedocs.org/en/latest/whatsnew-3.1.html). So I had to downgrade to the version 3.0.19 that was the previous stable version used for the project, and it works good so far. 

    pip install celery==3.0.19

Anyway, if you don't want to downgrade, the replacement for celeryd in the version 3.1 is celery worker. Check here for more info: http://celery.readthedocs.org/en/latest/userguide/workers.html.

Hope this helps! :)

Improve this answer

Comments

2

I have face similar type of issue: from celery import Celery ImportError: cannot import name 'Celery' from 'celery'

Another simple way to solve this: If your package have celery configuration in celery.py this is the reason that it is causing problems. Rename it something like celery_settings.py

Improve this answer

1 Comment

If you rename it you can't then run it as celery worker -A src
2

If the above error i.e. ImportError: cannot import name 'Celery' from 'celery' (/usr/local/airflow/.local/lib/python3.7/site-packages/celery/__init__.py) is coming and the python version is being used as 3.7.*, then due to a bug in importlib-metadata library because of its latest release.

So, to fix the above issue please use this version in your python dependencies list:

  • pip install importlib-metadata==4.13.0

Alternately, if you are using Docker file to build the project, change the python version from 3.7 to 3.8 and it should work fine.

For me, it worked. Please use the below link for a detailed look:

Django_celery_python_3.7_issue

Improve this answer

Comments

1

Note that older Django projects have the manage.py script in the same directory as the project directory. That is, the structure looks like this:

- proj/
  - proj/__init__.py
  - proj/celery.py
  - proj/urls.py
  - proj/manage.py
  - proj/settings.py

instead of this:

- proj/
  - proj/__init__.py
  - proj/celery.py
  - proj/settings.py
  - proj/urls.py
- manage.py

In this case, you will just have to rename the celery.app file to something different, like celeryapp.py as suggested in the accepted answer above.

Improve this answer

Comments

1

I faced the same issue on a FastAPI app running on Python 3.7. None of the solutions from this thread were working for me. I fixed the issue on my app by upgrading Python to 3.8.

Improve this answer

Comments

0

I got the same error.

Seems that from __future__ import absolute_import DOES NOT work for Python 2.6.1, still not raising an error.

Upgraded to Python 2.7.5 and it just worked.

Improve this answer

Comments

0

In my django application I had the same error. Version Django 4, Celery 5. My my problem is that I run Celery from directory with file celery.py, but it is necessary from directory with manage.py. Slava Ukraini!

Improve this answer

1 Comment

If you have a new question, please ask it by clicking the Ask Question button. Include a link to this question if it helps provide context. - From Review

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.