Python logging module prints unnecessary messages

Question

Please consider this dummy code.

$ cat dummy.py 
import logging
import time

from boto3.session import Session

# Logging Configuration
fmt = '%(asctime)s [%(levelname)s] [%(module)s] - %(message)s'
logging.basicConfig(level='INFO', format=fmt, datefmt='%m/%d/%Y %I:%M:%S')
logger = logging.getLogger()

def main():
    session = Session(region_name='us-west-2')
    client = session.client('ec2')
    response = client.describe_instances(InstanceIds=['i-11111111111111111'])

    logger.info('The instnace size is: %s', response[
                'Reservations'][0]['Instances'][0]['InstanceType'])

if __name__ == '__main__':
    main()

Output:

$ python3 dummy.py 
03/03/2017 08:47:00 [INFO] [credentials] - Found credentials in shared credentials file: ~/.aws/credentials
03/03/2017 08:47:01 [INFO] [connectionpool] - Starting new HTTPS connection (1): ec2.us-west-2.amazonaws.com
03/03/2017 08:47:02 [INFO] [dummy] - The instnace size is: t2.micro

Question: How to avoid below lines from being printed?

03/03/2017 08:47:00 [INFO] [credentials] - Found credentials in shared credentials file: ~/.aws/credentials
03/03/2017 08:47:01 [INFO] [connectionpool] - Starting new HTTPS connection (1): ec2.us-west-2.amazonaws.com

If I change logging.basicConfig(level='INFO',... to logging.basicConfig(level='WARNING',... then Those messages are not printed, but then all my messages get logged with WARNING severity.

I just want the logging module to print the messages that I explicitly write using logger.info .... and nothing else. Hence I need any pointers on how to avoid the unnecessary messages from being printed.

While maybe not an exact duplicate, you should be able to find what you're looking for in this question. — glibdud
– glibdud, Commented Mar 3, 2017 at 15:43
@glibdud, the question you referred is not a solution/workaround for me and I have already mentioned about it in my post. Thanks for your Time. — slayedbylucifer
– slayedbylucifer, Commented Mar 3, 2017 at 16:51
The linked question shows you how to access other modules' loggers (e.g. credentials, connectionpool) and set them to a different log level. I don't see anywhere in your question where you address that. — glibdud
– glibdud, Commented Mar 3, 2017 at 16:56
@glibdud, I have resolved. this. I will post the answer soon. — slayedbylucifer
– slayedbylucifer, Commented Mar 3, 2017 at 17:01

Aubrey Lavigne · Accepted Answer · 2022-02-05 00:30:48Z

4

boto3 defines a function set_stream_logger that overrides the logging level for a specific boto3 namespace.

Prototype:

boto3.set_stream_logger(name='boto3', level=logging.DEBUG, format_string=None)

Example:

import boto3
boto3.set_stream_logger(name='botocore.credentials', level=logging.WARNING)
boto3.set_stream_logger(name='urllib3.connectionpool', level=logging.WARNING)

botocore/credentials.py loads the logger with this line:

logger = logging.getLogger(__name__)

In this file __name__ resolves to botocore.credentials, so that will be the logger name when calling set_stream_logger.

Note that the Starting new HTTPS connection message comes from urllib3, which isn't part of boto3. The boto3.set_stream_logger function still works because it acts on the logging module.

edited Feb 5, 2022 at 0:30

answered Jan 27, 2022 at 17:32

Aubrey Lavigne

6827 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

slayedbylucifer · Accepted Answer · 2017-03-03 17:05:57Z

Solution:

import logging
import time

from boto3.session import Session

# Logging Configuration
fmt = '%(asctime)s [%(levelname)s] [%(module)s] - %(message)s'
logging.basicConfig(format=fmt, datefmt='%m/%d/%Y %I:%M:%S')
logger = logging.getLogger('LUCIFER')
logger.setLevel(logging.INFO)


def main():
    COUNTER = 3
    session = Session(region_name='us-west-2')
    client = session.client('ec2')
    response = client.describe_instances(InstanceIds=['i-0a912622af142b510'])

    logger.info('The instnace size is: %s', response[
                'Reservations'][0]['Instances'][0]['InstanceType'])

if __name__ == '__main__':
    main()

Output:

$ python3 dummy.py 
03/03/2017 10:30:15 [INFO] [dummy] - The instnace size is: t2.micro

Explanation: Earlier, I set the INFO level on root logger. Hence, all other loggers who do not have a level set, get this level propagated and start logging. In the solution, I am specifically enabling this level on a logger LUCIFER.

Reference: from: https://docs.python.org/3/howto/logging.html

Child loggers propagate messages up to the handlers associated with their ancestor loggers. Because of this, it is unnecessary to define and configure handlers for all the loggers an application uses. It is sufficient to configure handlers for a top-level logger and create child loggers as needed. (You can, however, turn off propagation by setting the propagate attribute of a logger to False.)

AND

In addition to any handlers directly associated with a logger, all handlers associated with all ancestors of the logger are called to dispatch the message (unless the propagate flag for a logger is set to a false value, at which point the passing to ancestor handlers stops).

Collectives™ on Stack Overflow

Python logging module prints unnecessary messages

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related