Unexpected python logger output when using several handlers with different log levels

Question

I am trying to log data to stderr and into a file. The file should contain all log messages, and to stderr should go only the log level configured on the command line. This is described several times in the logging howto - but it does not seem to work for me. I have created a small test script which illustrates my problem:

#!/usr/bin/env python

import logging as l

l.basicConfig(level=100)
logger = l.getLogger("me")

# ... --- === SEE THIS LINE === --- ...
logger.setLevel(l.CRITICAL)

sh = l.StreamHandler()
sh.setLevel(l.ERROR)
sh.setFormatter(l.Formatter('%(levelname)-8s CONSOLE %(message)s'))
logger.addHandler(sh)

fh = l.FileHandler("test.dat", "w")
fh.setLevel(l.DEBUG)
fh.setFormatter(l.Formatter('%(levelname)-8s    FILE %(message)s'))
logger.addHandler(fh)

logger.info("hi this is INFO")
logger.error("well this is ERROR")

In line 5th code line I can go for logger.setLevel(l.CRITICAL) or logger.setLevel(l.DEBUG). Both results are unsatisfying.

With logger.setLevel(l.CRITICAL) I get ...

$ python test.py
$ cat test.dat  
$

Now with logger.setLevel(l.DEBUG) I get ...

$ python test.py
INFO:me:hi this is INFO
ERROR    CONSOLE well this is ERROR
ERROR:me:well this is ERROR
$ cat test.dat  
INFO        FILE hi this is INFO
ERROR       FILE well this is ERROR
$

In one case I see nothing nowhere, in the other I see everything everywhere, and one message is being displayed even twice on the console.

Now I get where the ERROR CONSOLE and ERROR FILE outputs come from, those I expect. I don't get where the INFO:me... or ERROR:me... outputs are coming from, and I would like to get rid of them.

Things I already tried:

Creating a filter as described here: https://stackoverflow.com/a/7447596/902327 (does not work)
Emptying handlers from the logger with logger.handlers = [] (also does not work)

Can somebody help me out here? It seems like a straightforward requirement and I really don't seem to get it.

Padraic Cunningham · Accepted Answer · 2016-02-01 17:12:08Z

3

You can set the root level to DEBUG, set propagate to False and then set the appropriate level for the other handlers.

import logging as l

l.basicConfig()
logger = l.getLogger("me")

# ... --- === SEE THIS LINE === --- ...
logger.setLevel(l.DEBUG)
logger.propagate = False
sh = l.StreamHandler()
sh.setLevel(l.ERROR)

sh.setFormatter(l.Formatter('%(levelname)-8s CONSOLE %(message)s'))
logger.addHandler(sh)

fh = l.FileHandler("test.dat", "w")
fh.setLevel(l.INFO)
fh.setFormatter(l.Formatter('%(levelname)-8s    FILE %(message)s'))
logger.addHandler(fh)

logger.info("hi this is INFO")

logger.error("well this is ERROR")

Output:

~$ python test.py
ERROR    CONSOLE well this is ERROR
~$ cat test.dat
INFO        FILE hi this is INFO
ERROR       FILE well this is ERROR

edited Feb 1, 2016 at 17:12

answered Feb 1, 2016 at 17:06

Padraic Cunningham

181k30 gold badges264 silver badges327 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Padraic Cunningham Over a year ago

No worries, when you set the level to critical that means only critical level message will be sent to the other handlers.

Collectives™ on Stack Overflow

Unexpected python logger output when using several handlers with different log levels

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related