Call a function everytime logging module in python is called

Question

I would like to call a function everytime, I do any sort of logging using the logging module in python.

As can be seen below, instead of calling copyfiles at each instance of logging, I would like to see if there is way to injest copyfiles to the filehandler in someway or even a wrapper.

from sys import argv
import shutil
import logging
logger = logging.getLogger('')

sh = logging.StreamHandler(sys.stdout)
logfile = 'logs/log-run-{}.log'.format(date.today().strftime('%d-%m-%Y'))
fh = logging.FileHandler(logfile)

formatter = logging.Formatter('[%(asctime)s] %(levelname)s \
                            [%(filename)s.%(funcName)s:%(lineno)d] \
                            %(message)s', datefmt='%a, %d %b %Y %H:%M:%S')
logger.setLevel(logging.INFO)

sh.setFormatter(formatter)
fh.setFormatter(formatter)

logger.addHandler(sh)
logger.addHandler(fh)

LOGPATH = args[1]
def copyfiles():
    """
    Copy files to azure blob storage
    """
    fh.flush()
    shutil.copy(logfile, LOGPATH)

logging.info('test')
copyfiles()
logging.info('foobar')
copyfiles()

I tried digging into invoking copyfiles each time logging is called, but I ended up no where.

In case you are wondering why I am copy files from logging, this is why.

So you want to copy the whole file every time a new line is added to it? — Julien Palard
– Julien Palard, Commented Mar 16, 2021 at 12:47
I have suggested a solution, let me know if that would work. Just curious, wouldn't this cause a ton of extra File I/O's and become a bottleneck ? — the23Effect
– the23Effect, Commented Mar 16, 2021 at 14:38
This might become a bottleneck, but this is the only way out as stdout and stderr aren't given until the end of the job. so, i don't even know what i am waiting for — tandem
– tandem, Commented Mar 16, 2021 at 14:54
I really think this is a very bad idea. Why not using another handler like an HTTPHandler, a SocketHandler, a DatagramHandler, whatever? — Julien Palard
– Julien Palard, Commented Mar 16, 2021 at 15:20

the23Effect · Accepted Answer · 2021-03-16 14:34:21Z

Currently this is what I could think of:

Override the flush() of FileHandler by inheriting it into a utility class like FlushCopyFileHandler class shown in the code below.
Instead of using the FileHandler use the FlushCopyFileHandler class and all you have to do is call this overridden flush().

"""

from logging import FileHandler
from shutil

class FlushCopyFileHandler(FileHandler):
    
    # These arguments are passed along to the parent class FileHandler.
    def __init__(self, *args, **kwargs):
        super(FlushCopyFileHandler, self).__init__(filename, *args, **kwargs)
        self.copy_destination = "some default destination path"   # can also be set from args.
        
    # When this flush() is called it will call the flush() of FileHandler class.
    # After that it will call the shutil.copy() method.
    def flush(self):
        super().flush()
        shutil.copy(self.filename, self.copy_destination)
        
    # Use this to change destination path later on.
    def set_copy_path(self, destination):
        self.copy_destination = destination

"""

Collectives™ on Stack Overflow

Call a function everytime logging module in python is called

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related