How to set up multiple loggers with different settings using logging.config.dictConfig()

Question

I'm trying to set up three different loggers using dictConfig and for some reason the last logger always seems to overwrite the configuration of the two loggers created before it. Here is the code I'm using:

import logging
import logging.config

def setup_logger(name, level, ContentFormat='%(asctime)s %(levelname)s %(message)s', DateTimeFormat='%Y-%m-%d %H:%M:%S'):
    logging.config.dictConfig({
        'version': 1,
        'disable_existing_loggers': True,
        'formatters': {
            'default': {'format': ContentFormat, 'datefmt': DateTimeFormat},
        },
        'handlers': {
            'console': {
                'class': 'logging.StreamHandler',
                'level': level,
                'formatter': 'default',
                'stream': 'ext://sys.stdout'
            }
        },
        'loggers': {
            'a': {
                'level': level,
                'handlers': ['console']
            },
            'b': {
                'level': level,
                'handlers': ['console']
            },
            'c': {
                'level': level,
                'handlers': ['console']
            }
        }
    })
    return logging.getLogger(name)

logger_a = setup_logger(name='a', level=logging.INFO, ContentFormat='A: %(message)s')
logger_b = setup_logger(name='b', level=logging.INFO, ContentFormat='B: %(message)s')
logger_c = setup_logger(name='c', level=logging.INFO, ContentFormat='C: %(message)s')

logger_a.info('logger_a')
logger_b.info('logger_b')
logger_b.info('logger_c')

And this is the output:

C: logger_a
C: logger_b
C: logger_c

Instead what I'd like to see is:

A: logger_a
B: logger_b
C: logger_c

Any ideas what I'm doing wrong? I've even tried making multiple handlers and I still run into the same problem.

I have another version of this function that uses the following code and I'm able to call it multiple times to create multiple separate loggers with different settings, but I really wanted to figure out how to do this with dictConfig instead, or at least understand where I'm going wrong:

logger = logging.getLogger(name)
logger.setLevel(level)

# create console handler for printing logging output to the screen as well
formatter = logging.Formatter(ContentFormat, DateTimeFormat)
handler = logging.StreamHandler()
handler.setLevel(level)
handler.setFormatter(formatter)
logger.addHandler(handler)
return logger

MartenCatcher · Accepted Answer · 2019-08-11 16:55:56Z

6

It happens because you created and overrode the global configuration 3 times. You used default formatter to configure specific logger and the last one C used to log you records.

You can easily check it with this code:

def setup_logger(name, level, DateTimeFormat='%Y-%m-%d %H:%M:%S'):
    cfg = {
        'version': 1,
        'disable_existing_loggers': False,
        'formatters': {
            'default_for_a': {'format': 'A: %(message)s', 'datefmt': DateTimeFormat},
            'default_for_b': {'format': 'B: %(message)s', 'datefmt': DateTimeFormat},
            'default_for_c': {'format': 'C: %(message)s', 'datefmt': DateTimeFormat}
        },
        'handlers': {
            'console_for_a': {
                'class': 'logging.StreamHandler',
                'level': level,
                'formatter': 'default_for_a',
                'stream': 'ext://sys.stdout'
            },
            'console_for_b': {
                'class': 'logging.StreamHandler',
                'level': level,
                'formatter': 'default_for_b',
                'stream': 'ext://sys.stdout'
            },
            'console_for_c': {
                'class': 'logging.StreamHandler',
                'level': level,
                'formatter': 'default_for_c',
                'stream': 'ext://sys.stdout'
            },
        },
        'loggers': {
            'a': {
                'level': level,
                'handlers': ['console_for_a']
            },
            'b': {
                'level': level,
                'handlers': ['console_for_b']
            },
            'c': {
                'level': level,
                'handlers': ['console_for_c']
            }
        }
    }
    logging.config.dictConfig(cfg)
    return logging.getLogger(name)


logger_a = setup_logger(name='a', level=logging.INFO)
logger_b = setup_logger(name='b', level=logging.INFO)
logger_c = setup_logger(name='c', level=logging.INFO)

logger_a.info('logger_a')
logger_b.info('logger_b')
logger_c.info('logger_c')

Or make it more generic with the code

import logging
import logging.config


def setup_logger(name, level, ContentFormat='%(asctime)s %(levelname)s %(message)s', DateTimeFormat='%Y-%m-%d %H:%M:%S'):
    cfg = {
        'version': 1,
        'disable_existing_loggers': False,
        'formatters': {
            f'default_for_{name}': {'format': ContentFormat, 'datefmt': DateTimeFormat}
        },
        'handlers': {
            f'console_for_{name}': {
                'class': 'logging.StreamHandler',
                'level': level,
                'formatter': f'default_for_{name}',
                'stream': 'ext://sys.stdout'
            },
        },
        'loggers': {
            name: {
                'level': level,
                'handlers': [f'console_for_{name}']
            }
        }
    }
    logging.config.dictConfig(cfg)
    return logging.getLogger(name)


logger_a = setup_logger(name='a', level=logging.INFO, ContentFormat='A: %(message)s')
logger_b = setup_logger(name='b', level=logging.INFO, ContentFormat='B: %(message)s')
logger_c = setup_logger(name='c', level=logging.INFO, ContentFormat='C: %(message)s')

logger_a.info('logger_a')
logger_b.info('logger_b')
logger_c.info('logger_c')

The output is:

A: logger_a
B: logger_b
C: logger_c

answered Aug 11, 2019 at 16:55

MartenCatcher

2,8858 gold badges29 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Mike Over a year ago

Ohhhhh.... I didn't realize that you just call it once to configure everything (or like the last example, multiple times using variables). That helps a lot!

Mike Over a year ago

when I try your example, it doesn't seem to like the f-strings as IDs for the formatter: File "logger.py", line 10 f'default_for_{name}': {'format': ContentFormat, 'datefmt': DateTimeFormat} ^ SyntaxError: invalid syntax - the syntax error arrow is pointing at the first bracket in {name}

MartenCatcher Over a year ago

@Mike what the version of Python do you use? This syntax is available since 3.6. If you use a lower python 3 version, you have to replace an expression with the old version of the string format. "{name}".format(name=name) for example.

Mike Over a year ago

Ahh... 3.5 - I switched it over to format strings and that fixed it. Thanks!

Collectives™ on Stack Overflow

How to set up multiple loggers with different settings using logging.config.dictConfig()

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related