1

I'm creating centralised logging. That basically looks like the scripts below.

The logit module will create a file, based on the scripts name which called it. in this case apiCaller.

Originally i had this manually defined when calling logit, however i was searching for away for logit to determine the origin of the log itself.

There are 3 modules at play here:

main.py:

def runAnalytic(script):
    importlib.import_module("monitoringScripts."+script["package"]+"."+script["module"], package=None)

packageModule = [{"package":"awesome","module":"apiCaller"}]

with concurrent.futures.ThreadPoolExecutor() as executor:
    results = executor.map(runAnalytic, packageModule)

apiCaller.py (module above)

from adminTools.logger import logit
logit.create(results[i]["items"][r]["userId"],"apiCaller") #How i currently pass the script name, i want to get rid of this.

logit.py Handles all log requires from all my other scripts (centralised logging)

import sys, logging, logging.handlers, pathlib
#Path for all log files for scriptHub
logdir = str(pathlib.Path(__file__).parent.absolute())

#Creates the log file based on a given name from the script
def create(logMessage,scriptName, level="DEBUG"):
    #create filename
    log_filename = logdir+"/sysLogs/"+scriptName+".logs"
    #Creates the logging object
    my_logger = logging.getLogger(scriptName)
    my_logger.setLevel(logging.DEBUG)
    #Formats the log:
    formatter = logging.Formatter('%(asctime)s - %(message)s - %(name)s')
    #Gives the log file a limit for 100mb if it goes bigger than this, it will create another file, but keep the old one
    handler = logging.handlers.RotatingFileHandler(log_filename, maxBytes=100000000, backupCount=1)
    handler.setFormatter(formatter)
    #Handlers need to be cleared to stop duplicated logs.
    if (my_logger.hasHandlers()):
        my_logger.handlers.clear()
    my_logger.addHandler(handler)
    #creates the log message
    my_logger.debug(logMessage)

So, I'm not sure if that helps or hinders you all lol

Essentially, instead of providing logit with the script name, i want logit to get it from the module it's called from. E.g in this case "apiCaller" would be the name that's passed through to logit.

Improve this question
7
  • you can use the %(module)s log formatter, see the section on LogRecord Attributes of docs.python.org/2/library/logging.html Commented Feb 5, 2020 at 16:20
  • When you do logging.getLogger, you should put __name__ as the parameter. __name__ is the name of the module itself if imported or "__main__" if the file was run as a script. Then the log formatter does the rest. Commented Feb 5, 2020 at 16:21
  • 1
    how are your scripts called from other scripts ? And what is logit.py and why did it print test above ? Commented Feb 5, 2020 at 16:21
  • @Dan That "module" returns the logit module not the one that called the logit module. Commented Feb 5, 2020 at 18:10
  • @h4z3 name just returns the logit package structure (parents etc) Commented Feb 5, 2020 at 18:10

2 Answers 2

Reset to default
3

The question isn't very clear, but you can use inspect.stack().

loggy.py

import inspect

def log(s):
    caller = inspect.stack()[1]
    print(f"{caller.filename} line {caller.lineno} says: {s}")

thing.py

import loggy

loggy.log("Hey!")

/v/f/n/8/T/tmp.ZiRzgsqi $ python3 thing.py
thing.py line 3 says: Hey!
/v/f/n/8/T/tmp.ZiRzgsqi $
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

already found this answer somewhere else, but since you wrote it first here, you get the credit :-) thank you. For some reason i can't find the other post, I did up vote it though. :-)
1

Okay, with the rewritten question:

I've seen it done the other way around than you do it - get a logger, then set it up (two lines in the module, not one). The logger is per module thing and always is there.

In your case, you re-get the logger and remake the handlers each time.

This way you can't make use of the beautiful possibilities logging module offers!

So basically, this other approach is:

In each script you do logger = logging.getLogger(__name__), usually somewhere near the top, below imports.

+You just call logit.setupLogger(logger). (In your case, in the next line. In case of scripts, I keep it in main function - so that if I ever import the script as a module, I will call whatever logging setup I need on imported_module.logger so it doesn't spam the wrong log file. :D)

Rewritten logit.py:

import sys, logging, logging.handlers, pathlib
#Path for all log files for scriptHub
logdir = str(pathlib.Path(__file__).parent.absolute())

#Creates the log file based on a given name from the script
def create(my_logger, level=logging.DEBUG):
    #create filename
    log_filename = logdir+"/sysLogs/"+logger.name+".logs"

    my_logger.setLevel(level)
    #Formats the log:
    formatter = logging.Formatter('%(asctime)s - %(message)s - %(name)s')
    #Gives the log file a limit for 100mb if it goes bigger than this, it will create another file, but keep the old one
    handler = logging.handlers.RotatingFileHandler(log_filename, maxBytes=100000000, backupCount=1)
    handler.setFormatter(formatter)
    #Handlers need to be cleared to stop duplicated logs.
    if (my_logger.hasHandlers()):
        my_logger.handlers.clear()
    my_logger.addHandler(handler)

This way, you only set up insides of the logger - including file handler - in the logit, and you can use standard logging things:

  • you can use any level logging in your module:
logger.info("like")
logger.warning("this")
  • you write every logging message in the code like above - make the code full of logging messages! - remember that debug messages should have everything needed for debug, this sometimes includes huge chunks of information + remember that debug messages may exist next to info messages, just with different details (e.g. "Getting info from X" is info, "Sent request 'some/address/here' and received '''chunk of data here'''" is debug).
  • when you need to cut the level - e.g. you debugged your message and got tired of seeing so.much.info (or just going from dev to prod) - you just change logit.setupLogger(logger) to logit.setupLogger(logger, logging.INFO) or whatever level you need.

Logging might seem like a good idea to do your way, but logging module is quite powerful when you learn how to use it. :) Here's a How-To from Python's docs, but it has a lot of info, so simpler tutorials on python logging are a better thing to start with.

Tbh I started with my own logging practices as well, even after reading the docs and tutorials because I didn't get it at all. I only switched to the approach above when I saw it used in a lib I'm using. :)

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.