Python logging show 3 lines for 1 log in log file

Question

I have a python project where I have 3 python files out of which app.py is the main file and other two files are tracker.py and celerio.py. I have the logging enabled for all the files and below is its code:

def setup_logger(logger_name, log_file, level=logging.ERROR):
    log_formatter = logging.Formatter('%(asctime)s %(message)s')
    my_handler = RotatingFileHandler(log_file, mode='a', maxBytes=100 * 1024 * 1024, backupCount=5, encoding=None,
                                     delay=0)
    my_handler.setFormatter(log_formatter)
    my_handler.setLevel(level)
    l = logging.getLogger(logger_name)
    l.addHandler(my_handler)

dir = Path(__file__).parents[1]
log_file = os.path.join(dir, 'logs', 'application.log')
setup_logger('debug', log_file)
log = logging.getLogger('debug')

and then I use it like log.error("Hello world") in the python file and it works fine. Now as I have another 2 files, I have also copied the same above code in those two files tracker.py and celerio.py. But the problem is, lets if I log log.error("Hello world from app.py") in app.py. It is saved as 3 times in application.log file. Same is the case with other files as well.

2019-11-15 13:32:00,288 Hello world from app.py
2019-11-15 13:32:00,288 Hello world from app.py
2019-11-15 13:32:00,288 Hello world from app.py

I dont know why it is logged 3 times for just 1 log. Can anyone please help me in this. Thanks

The line log = logging.getLogger('debug') is not in a function. So when you import those files, you import the reference to that. Hence calling log.error('whatever') prints three time. You can test this by removing it from one file and it will only log twice — Rashid 'Lee' Ibrahim
– Rashid 'Lee' Ibrahim, Commented Nov 15, 2019 at 13:57

ababak · Accepted Answer · 2019-11-15 13:57:44Z

1

While you set up your logger you add new handlers over and over again. Just try to clear them before adding:

l.handlers[:] = []

answered Nov 15, 2019 at 13:57

ababak

1,8031 gold badge13 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

S Andrew Over a year ago

So you mean before this line l.addHandler(my_handler), I should add l.handlers[:] = [].?

ababak Over a year ago

Yes, exactly. Or you may check if you have already added a handler before.

ababak Over a year ago

I just want to add that you may want to consider setting all your logs configuration in one place (maybe via logging.config.dictConfig) and then set the logging.initialized = True (you create a logging module variable that is persistent for your Python session). Then you may check if hasattr(logging, 'initialized') to prevent from setting the configuration several times.

Daniel Hepper · Accepted Answer · 2019-11-15 14:11:40Z

The logging system has global state. By adding this code to all three of your modules, you are adding three handlers to the logger named "debug".

Only setup your logging once in app.py, only use log = logging.getLogger('debug') in your other modules.

app.py

def setup_logger(logger_name, log_file, level=logging.ERROR):
    log_formatter = logging.Formatter('%(asctime)s %(message)s')
    my_handler = RotatingFileHandler(log_file, mode='a', maxBytes=100 * 1024 * 1024, backupCount=5, encoding=None,
                                     delay=0)
    my_handler.setFormatter(log_formatter)
    my_handler.setLevel(level)
    l = logging.getLogger(logger_name)
    l.addHandler(my_handler)

dir = Path(__file__).parents[1]
log_file = os.path.join(dir, 'logs', 'application.log')
setup_logger('debug', log_file)
log = logging.getLogger('debug')

tracker.py and celerio.py

import logging
log = logging.getLogger('debug')

Collectives™ on Stack Overflow

Python logging show 3 lines for 1 log in log file

2 Answers 2

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related