Python logging: different logging destination per process

Question

I just started to use Python logging module and I cannot understand something.

I'm writing a script that works in the following way:

1st part (single process): it get some data to compute (it's not important how). Here I create a logger in the following way:

import logging
logging.basicConfig(format='%(asctime)-6s: %(name)s - %(levelname)s - %(message)s')
logger = logging.getLogger('Pipeline')
logger.setLevel(logging.INFO)
logger.warning('Pipeline started')

In other words I log to the screen.

2nd part (multiprocessing): I create several processes (the data analysis is really time and CPU consuming) to analyze the data found before.

Now I would like that each process logs to a different file ONLY without logging to the screen.

What I wrote is :

fh = logging.FileHandler('/tmp/'+multiprocessing.current_process().name+'_worker.log')
fmt = logging.Formatter(%(asctime)-6s: %(name)s - %(levelname)s - %(message)s)
fh.setFormatter(fmt)
local_logger = logging.getLogger(multiprocessing.current_process().name+'_worker')
local_logger.addHandler(fh)
local_logger.warning(multiprocessing.current_process().name + ' (worker) Process started')

What I got is that each process logs to a different file but logs ALSO to the screen!

How can I fix this?

Art Swri · Accepted Answer · 2012-05-18 17:40:39Z

4

Suspect that your local logger is passing its log messages upwards to the top level, where it gets output to stdout. Try turning off propagation on the local logger. I believe you can do it like this:

local_logger.propagate = False

answered May 18, 2012 at 17:40

Art Swri

2,8243 gold badges30 silver badges37 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Art Swri Over a year ago

I would probably add that using basicConfig() may not be the most suitable choice for your situation IMHO. As my colleague Jason pointed out to me, it's really meant for very simple logging setup. I'd recommend looking into using a somewhat more 'sophisticated' approach; easy to find several approaches by googling.

Vinay Sajip · Accepted Answer · 2012-05-18 19:00:25Z

3

You could do it Art Swri's way, or just omit the basicConfig() call. That's what adds a console handler to the root logger.

answered May 18, 2012 at 19:00

Vinay Sajip

100k15 gold badges184 silver badges196 bronze badges

Collectives™ on Stack Overflow

Python logging: different logging destination per process

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related