I am very new to Python and am trying to add logs(in a log file) to a project. As soon as I added some log codes, the runtime tripled, so I was suggested to use a functionality that could perhaps store the logs in Python's memory and then print it out to a logfile - in the hope for the runtime to not increase so much.
I started searching for ways to do that and found this resource on MemoryHandler, which I interpreted as something that could help me achieve my purpose:
Base1.py (Attempt1)
I tried this code first, without the MemoryHandler:
import logging
import logging.config
import logging.handlers
formatter = logging.Formatter('%(asctime)s — %(name)s — %(levelname)s — %(funcName)s:%(lineno)d — %(message)s')
def setup_logger(__name__,log_file,level=logging.INFO):
'''to set up loggers'''
handler= logging.FileHandler(log_file)
handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.setLevel(level)
logger.addHandler(handler)
return logger
checking = setup_logger(__name__,'run_logfile.log')
checking.info("this is a test")
It took around 11 seconds to run for the entire project. This is actually a lot - because currently the data volumes are Nil and it should be around 3 seconds, which is what it was before I added these log codes. So next, I tried the below code in the hope of making the code faster:
Base2.py (Attempt2)
import logging
import logging.config
import logging.handlers
formatter = logging.Formatter('%(asctime)s — %(name)s — %(levelname)s — %(funcName)s:%(lineno)d — %(message)s')
def setup_logger(__name__,log_file,level=logging.INFO):
'''to set up loggers'''
handler= logging.FileHandler(log_file)
handler.setFormatter(formatter)
memoryhandler = logging.handlers.MemoryHandler(
capacity=1024*100,
flushLevel=logging.INFO,
target=handler,
flushOnClose=True
)
logger = logging.getLogger(__name__)
logger.setLevel(level)
logger.addHandler(handler)
logger.addHandler(memoryhandler)
return logger
checking = setup_logger(__name__,'run_logfile.log')
checking.info("this is a test")
This too takes 11 seconds - it does work, but my problem is that it is not at all faster than when I did not use the MemoryHandler previously, so I was wondering if my code was very wrong still?
Am I doing anything wrong here? Or if there is way to have logs without making the runtime longer?
