Python logging module doesn't write in log file

Question

I have an issue with my script, I'm trying to create a log file for every call of this function:

@begin.subcommand
def sample_mode(sample, folder = None, reference = None, dbsnp = None):
    """ creates a config file for 1 sample

    - setup logging system: {folder}/logs/{sample}_init.log
    - takes in a sample name
    - creates a file: {folder}/{sample}_config.yaml
    """
    # get the current folder if none is given
    if folder == None:
        folder = os.getcwd() 

    # create logs folder if necessary
    if not os.path.isdir("{}/logs".format(folder)):
        os.mkdir("{}/logs".format(folder))

    log_file = setup_log(sample, folder)

    for i in range(1,3):
        try:
            os.path.getsize("{}/{}.{}.fq.gz".format(folder, sample, i))
        except FileNotFoundError:
            log_file.error("{}/{}.{}.fq.gz doesn't exist".format(folder, sample, i))
            return -1

        if os.path.getsize("{}/{}.{}.fq.gz".format(folder, sample, i)) == 0:
            log_file.error("\"{}/{}.{}.fq.gz\" is empty\n".format(folder, sample, i))
            return -1

    # set up logging system
    # logging.basicConfig(filename = "{}/logs/{}_init.log".format(folder, sample), level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

    # create the config file
    with open("{}/{}_config.yaml".format(folder, sample), "w") as config:
        config.write("sample: \"{}\"\n\n".format(sample))
        log_file.info("Sample name: {}".format(sample))

For reference, the setup_log function:

def setup_log(sample, folder):
    logger = logging.getLogger(__name__)

    log_format = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')

    filename = "{}/logs/{}_init.log".format(folder, sample)

    log_handler = logging.FileHandler(filename)
    log_handler.setLevel(logging.DEBUG)
    log_handler.setFormatter(log_format)

    logger.addHandler(log_handler)

    return logger

The log file is created but it's empty. However, if i put a wrong sample name, the log file is created and the error message is written.

What an i doing wrong?

olinox14 · Accepted Answer · 2019-06-24 13:52:49Z

3

There are a few mistakes in your code, this should work:

import logging

def setup_log(name):
    logger = logging.getLogger(name)   # > set up a new name for a new logger

    logger.setLevel(logging.DEBUG)  # here is the missing line

    log_format = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s')
    filename = f"./test_{name}.log"
    log_handler = logging.FileHandler(filename)
    log_handler.setLevel(logging.DEBUG)
    log_handler.setFormatter(log_format)

    logger.addHandler(log_handler)

    return logger

def test_log(name):
    logger = setup_log(name)
    logger.info("Just logged from %s", name)   # > old formatting syntax is advised by the logging cookbook

test_log("log1")
test_log("log2")
test_log("log3")

The log files were not written because you have to set the logger's level, before to set the level of the handler.

Also, you have to know that logging.getLogger(name) returns a singleton. If you call the getLogger mfunction with the same name you used before, this will return the existing logger, which could leads you to a very weird situation here... To illustrate it:

def test_log(name):
    logger = setup_log(name)

is equivalent to

def test_log(name):
    setup_log(name)
    logger = logging.getLogger(name)

More here: https://docs.python.org/3/howto/logging-cookbook.html

answered Jun 24, 2019 at 13:52

olinox14

6,7223 gold badges27 silver badges43 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Yujin Kim Over a year ago

Thank you, it works now. The examples i found online used the setLevel method on the handler object, i guess that's wrong?

olinox14 Over a year ago

That's not wrong, you can set the level at the root (logging) level, at the logger's level and at the handler's level. If the logger's level is ERROR, the handler will never receive the message, no matter what is its level

Collectives™ on Stack Overflow

Python logging module doesn't write in log file

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related