python: modify level of logs from a module

Question

TLDR

if a module uses

log.error("something happened")

we would like to see these logs, but as warnings, so that the net effect for us would be the same as if that module had used

log.warning("something happened")

More details

We use the aiokafka module which logs errors when the connection with confluent.cloud has trouble. However these are transient problems and after a while connection is re-established, so we would have preferred these logs to be warning instead of error, yet we don't want to lose those logs.

Is there a way to modify these log records "on the fly", to change their log level? I know I could

logger = logging.getLogger("aiokafka")
logger.setLevel(logging.CRITICAL)

but then all logs would get lost.

Could you overwrite the error function to the warning function, i.e. logging.getLogger("aiokafka").error = logging.getLogger("aiokafka").warning — Tom McLean
– Tom McLean, Commented Oct 4, 2021 at 11:17

Vinay Sajip · Accepted Answer · 2021-10-04 17:42:38Z

3

You can attach a filter function to the logger which downgrades the level. Here'a a working example you can use to build from:

import logging

def downgrade_filter(record):
    if record.levelno == logging.ERROR:
        record.levelno = logging.WARNING
        record.levelname = logging.getLevelName(logging.WARNING)
    return True

if __name__ == '__main__':
    logging.basicConfig(level=logging.DEBUG, format='%(levelname)-8s|%(name)-8s|%(message)s')
    logger = logging.getLogger('aiokafka')
    logger.setLevel(logging.WARNING)
    logger.addFilter(downgrade_filter)
    logger.debug('This should not appear')
    logger.info('This should not appear')
    logger.warning('This should appear as a warning')
    logger.error('This should appear as a warning, though logged as an error')
    logger.critical('This should appear as a critical error')

When run, it should print

WARNING |aiokafka|This should appear as a warning
WARNING |aiokafka|This should appear as a warning, though logged as an error
CRITICAL|aiokafka|This should appear as a critical error

(This is on a recent version of Python 3.x)

answered Oct 4, 2021 at 17:42

Vinay Sajip

100k15 gold badges184 silver badges196 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

balderman Over a year ago

This will turn every ERROR to WARNING - right? So we better look inside the log record attributes and find the attributes that will "filter" only messages we want to modify.

Vinay Sajip Over a year ago

Not every ERROR, just the ones logged to that logger. One can of course tweak the filter function to do whatever is needed, but that's the basic approach.

balderman Over a year ago

Agreed - I voted up.

Collectives™ on Stack Overflow

python: modify level of logs from a module

TLDR

More details

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

TLDR

More details

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related