0

i have a django project with celery task (running inside), but i have a question about logging, what i have done is:

Task logging get_task_logger :

from celery.utils.log import get_task_logger
logger = get_task_logger('celery')

Django logging:

LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'root': {
    'level': 'DEBUG',
    'handlers': ['sentry', 'file'],
},
'formatters': {
    'verbose': {
        'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
    },
},
'handlers': {
    'console': {
        'level': 'DEBUG',
        'class': 'logging.StreamHandler',
        'formatter': 'verbose',
    },
    'celery': {
        'level': 'INFO',
        'class': 'logging.StreamHandler',
        'stream': sys.stdout,
    },
 ........
},
'loggers': {
    'celery': {
        'handlers': ['celery'],
        'level': 'INFO',
    },
......
},

supervisord celery config:

  • [program:celery] *command=/usr/local/bin/celery worker -A app --autoscale=20,10 -E -l INFO -Ofair
  • directory=/xxx/celerydir
  • numprocs=1
  • stdout_logfile=/xxx/logs/celery_worker.log
  • stderr_logfile=/xxx/logs/celery_worker_err.log
  • autostart=true
  • autorestart=true
  • startsecs=10
  • stopwaitsecs=600
  • stopasgroup=true
  • priority=998

My problem is as you can see, inside Django -> celery handler i've specified "'stream': sys.stdout", but when i start celery with supervisord i will see both logs celery_worker.log and celery_worker_err.log that store all logs for all level : INFO, WARNING, ERROR... why?

How can i configure logger for celery to redirect all celery log only to stdout and let supervisord store level info only on celery_worker.log ?

thanks

1 Answer 1

3

I think it's because by default celery reports things to stderr instead of stdout, you can use supervisord redirect_stderr = true flag to combine two files into one if that helps.

There's a settings in celery that let's you specify a log-file if log-file is not specified it uses stderr.

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.