python django logging: One single logger with more than one logging level

Question

tl;dr: Is there any way to have one single logger that manages all loglevels?

To log in my project, I am currntly running:

logger = logging.getLogger('info.contract')
logger.info("Some info message...")

Which prints 2021-01-24 03:05:02,063 [contract_generator] Some info message... to console.

The info.contract logger is defined in my settings.py:

    'loggers': {
        'info': {
            'handlers' : ['info_logfile'],
            'level': 'INFO',
        },
        'info.contract': {
            'handlers': ['contract_info_logfile'],
            'level': 'INFO',
            'propagate': True,
        }
    },
    'handlers': {
        'info_logfile': {
            'class': 'logging.FileHandler',
            'filename': 'info.log',
            'formatter': 'default',
        },
        'contract_info_logfile': {
            'class': 'logging.FileHandler',
            'filename': 'contract.info.log',
            'formatter': 'default',
        }
    }
    'formatters': {
        'default': {
            'format': '%(asctime)s [%(module)s] %(message)s',
        }
    }
}

The problem I am having is the following:

In order to log to different files depending on the loglevel, I would have to create a new logger for each loglevel. Which is to say, the following doesn't work:

logger = logging.getLogger('info.contract')
logger.info("Some info message...")

## this part is new
logger.debug("Some debug message...")

Since logger uses the info.contract logger defined in settings.py, which has its 'level' setting set to 'INFO' (as shown above). Interestingly though, I don't get any errors. The logger simply doesn't log the entries when I use the .debug() function.

So in order to log to different files depending on the loglevel, I would indeed have to create a new logger for each loglevel, even though that produces a lot of duplicate code:

##---snip---##
'loggers': {
        'info': {
            'handlers' : ['std_err', 'info_logfile'],
            'level': 'INFO',
        },
        'info.contract': {
            'handlers': ['contract_info_logfile'],
            'level': 'INFO',
            'propagate': True,
        }
        'debug': {                                     ## duplicate code
            'handlers' : ['std_err', 'debug_logfile'],
            'level': 'DEBUG',
        },
            'debug.contract': {                        ## duplicate code
            'handlers': ['contract_debug_logfile'],
            'level': 'DEBUG',
            'propagate': True,
        }
},
##---snip---##

So then in my code I would have to do:

info_logger = logging.getLogger('info.contract')
debug_logger = logging.getLogger('debug.contract')

info_logger.info("Some info message...")
debug_logger.debug("Some debug message...")

Which implies that it is necessary to instantiate a new logger object for every loglevel I need to use. This seems cumbersome and is maybe inefficient.

My question is:

Is there any way to have one single logger that manages all loglevels? In other words, is it possible to do the following:

some_multilevel_logger = logging.getLogger('some_multilevel_logger')
some_multilevel_logger.info("Some info message...")
some_multilevel_logger.debug("Some debug message...")

And that would produce the same results as the previous snippet, in which 2 loggers are defined.

If that behavior is not possible, is it considered bad practice to create one logger for every loglevel? What about massive projects with many loglevels?

Thank you!

Fabián Montero · Accepted Answer · 2021-01-24 05:50:04Z

Best solution I found was to create multiple handlers for one single logger:

LOGGING = {
    'version': 1,
    'disable_existing_loggers' : False,
    'loggers': {
        'general': {
            'handlers': ['error', 'info', 'debug'],
            'level': 1
        }
    },
    'handlers': {
        'std_err': {
            'class': 'logging.StreamHandler'
        },
        'info': {
            'class': 'logging.FileHandler',
            'filename': 'info.log',
            'level': 'INFO',
            'formatter': 'default',
        },
        'error': {
            'class': 'logging.FileHandler',
            'filename': 'error.log',
            'level': 'ERROR',
            'formatter': 'error',
        },
        'debug': {
            'class': 'logging.FileHandler',
            'filename': 'debug.log',
            'level': 'DEBUG',
            'formatter': 'default',
        },
    },
    'formatters': {
        'default': {
            'format': '%(asctime)s [%(module)s | %(levelname)s] %(message)s',
        },
        'error': {
            'format': '%(asctime)s [%(module)s | %(levelname)s] %(message)s @ %(pathname)s : %(lineno)d : %(funcName)s',
        },
    },
}

Which allows the following:

logger = logging.getLogger('general')
logger.error("Some error message...")
logger.debug("Debug message...")
logger.info("Some info message..")

it even allows:

logger.exception("Exception message", fmt=std.FAIL)

Roham · Accepted Answer · 2021-01-24 06:21:06Z

I think one way to use one log file for different purposes is to change your log level in your logger configs because based on this document you can not log debug level in to info level. For example look at the following configs which I usually use as my best practice:

LOG_DIR = BASE_DIR / 'logs'
LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'simple': {
            'format': '[%(asctime)s] %(levelname)s %(message)s',
            'datefmt': '%Y-%m-%d %H:%M:%S'
        },
        'verbose': {
            'format': '[%(asctime)s] %(levelname)s [%(filename)s.%(funcName)s:%(lineno)d] %(message)s',
            'datefmt': '%Y-%m-%d %H:%M:%S'
        },
    },
    'filters': {
        'require_debug_false': {
            '()': 'django.utils.log.RequireDebugFalse',
        },
        'require_debug_true': {
            '()': 'django.utils.log.RequireDebugTrue',
        },
    },
    'handlers': {
        'console': {
            'level': 'DEBUG',
            'filters': ['require_debug_true'],
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        'mail_admins': {
            'level': 'ERROR',
            'filters': ['require_debug_false'],
            'class': 'django.utils.log.AdminEmailHandler',
            'formatter': 'verbose'
        },
        'file': {
            'level': 'DEBUG' if DEBUG else 'INFO',
            'class': 'logging.FileHandler',
            'filename': LOG_DIR / 'django.log',
            'formatter': 'verbose' if DEBUG else 'simple',
        },
        'db_queries': {
            'level': 'DEBUG',
            'filters': ['require_debug_true'],
            'class': 'logging.FileHandler',
            'filename': LOG_DIR / 'db_queries.log',
        },
        'null': {
            'level': 'DEBUG',
            'class': 'logging.NullHandler',
        },
    },
    'loggers': {
        'django.db.backends': {
            'level': 'DEBUG',
            'handlers': ['db_queries'],
            'propagate': False,
        },
        'accounts': {
            'handlers': ['file', 'console'],
            'level': 'DEBUG',
        },
    },
}

And for logging just use following lines:

logger = logging.getLogger('accounts')

logger.info("some info logs")
logger.debug("some debug logs")

Farhad Hossain · Accepted Answer · 2021-01-24 05:43:13Z

One solution might be this.

import logging

class MultiLogger(logging.Logger):
    def __init__(self, name, level=logging.INFO, formatter=None):
        self.name = name
        self.level = level
        self.formatter = logging.Formatter("%(asctime)s %(levelname)s %(message)s")
        logger = logging.getLogger(self.name)
        logger.setLevel(self.level)
        self.logger = logger
        super().__init__(name=name, level=level)

    def debug(self, msg, log_file="debug.log", *args, **kwargs):
        handler = logging.FileHandler(log_file)
        handler.setFormatter(self.formatter)

        for h in self.handlers:
            self.removeHandler(h)

        self.addHandler(handler)
        self.level = 10
        return super().debug(msg, *args, **kwargs)
    
    def error(self, msg, log_file="error.log", *args, **kwargs):
        handler = logging.FileHandler(log_file)
        handler.setFormatter(self.formatter)
        for h in self.handlers:
            self.removeHandler(h)
        self.addHandler(handler)
        self.level = 40
        return super().error(msg, *args, **kwargs)
    
    def warning(self, msg, log_file="warning.log", *args, **kwargs):
        handler = logging.FileHandler(log_file)
        handler.setFormatter(self.formatter)
        for h in self.handlers:
            self.removeHandler(h)
        self.addHandler(handler)
        self.level = 30
        return super().warning(msg, *args, **kwargs)
    
    def info(self, msg, log_file="info.log", *args, **kwargs):
        handler = logging.FileHandler(log_file)
        handler.setFormatter(self.formatter)
        for h in self.handlers:
            self.removeHandler(h)
        self.addHandler(handler)
        self.level = 20
        return super().info(msg, *args, **kwargs)
    
    

logger = MultiLogger(name="info_logger", level=40)
logger.debug("This is debug message", log_file="debug.log")
logger.error("This is error message", log_file="error.log")
logger.info("This is info message", log_file="info.log")
logger.warning("This is warning message", log_file="warn.log")

Collectives™ on Stack Overflow

python django logging: One single logger with more than one logging level

tl;dr: Is there any way to have one single logger that manages all loglevels?

My question is:

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

tl;dr: Is there any way to have one single logger that manages all loglevels?

My question is:

3 Answers 3

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related