How to add custom data in Python logs

Question

I am using python logging.

Once I get a handle to logger object, I do the following to log

logger.info("this is my log message")

My requirement is this:

In the above custom log info message, I would want to pass more parameters like the following:
```
custom_name = "DISPLAY"
logger.info("this is my log message", custom_name)
```
custom_name parameter should be consumed by the log formatter's custom_name placeholder as shown below
```
formatter = logging.Formatter('%(asctime)s %(custom_name)s:%(name)s %(message)s')
```

I read articles, stackoverflow posts, python logging documentation. I could only find how to create custom placeholders in the formatter. I am not sure how to pass those custom placeholder values from the log statement itself. Can someone point me to a resource or let me know how to accomplish this ?

Edit:

I could do something like this

import logging

CUSTOM_VAR= 'myCustomVarValue'


class ContextFilter(logging.Filter):

    def filter(self, record):
        record.CUSTOM_VAR= CUSTOM_VAR
        return True

FORMAT = '%(CUSTOM_VAR)s %(asctime)s - %(levelname)s - %(message)s'
logging.basicConfig(format=FORMAT, datefmt='%d/%m/%Y %H:%M:%S')

logger = logging.getLogger(__name__)
logger.addFilter(ContextFilter())

logger.info("'this is info message")

The above works fine. However, CUSTOM_VAR is a static value. How can I dynamically pass this value from log.info ??

Seems already answered here:stackoverflow.com/questions/16203908/… — Muposat
– Muposat, Commented May 15, 2017 at 21:09
not really as that one talks about a static parameter. I found a way to do it which I have posted in the answer below — Hary
– Hary, Commented May 15, 2017 at 22:00

Kapil Ahuja · Accepted Answer · 2022-07-06 20:13:12Z

2

In config.py we can set CUSTOM_VAR using contextvars class and we can update dynamically inside the main code.

import contextvars
CUSTOM_VAR = contextvars.ContextVar("CUSTOM_VAR", default=None)

Inside main code we can update CUSTOM_VAR like this

CUSTOM_VAR.set("DISPLAY")

we can use set CUSTOM_VAR inside logger file

import logging
from config import CUSTOM_VAR


class ContextFilter(logging.Filter):

    def filter(self, record):
        if not hasattr(record, 'CUSTOM_VAR'):
            record.custom_name= CUSTOM_VAR.get()
            return True

FORMAT = '%(custom_name)s %(asctime)s - %(levelname)s - %(message)s'
logging.basicConfig(format=FORMAT, datefmt='%d/%m/%Y %H:%M:%S')

logger = logging.getLogger(__name__)
logger.addFilter(ContextFilter())

logger.info("'this is info message",extra=CUSTOM_VAR)

edited Jul 6, 2022 at 20:13

answered Jul 5, 2022 at 20:29

Kapil Ahuja

214 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

chepner · Accepted Answer · 2017-05-15 22:11:30Z

0

I was able to accomplish it the following way:

In log config code, have your formatter declared like this:

 formatter = logging.Formatter(
   '%(asctime)s %(levelname)s %(name)s,%(traceId)s %(packageName)s %(message)s')

Now in the code, where you have the log statements, have such statements written like this where you pass extra param values dynamically:

logger.info("custom log message goes here.....",
            extra={'traceId': 'XDDF4543GG', 'packageName': 'myPackage'})

corresponding log entry would look like this:

2017-05-15 17:59:24,101 INFO [my-proj-name,XDDF4543GG] myPackage custom log message goes here.....

Also, if you don't put all extra params in log.info then you would get a keyError error message.

edited May 15, 2017 at 22:11

chepner

538k77 gold badges595 silver badges747 bronze badges

answered May 15, 2017 at 22:09

Hary

1,2386 gold badges29 silver badges55 bronze badges

Collectives™ on Stack Overflow

How to add custom data in Python logs

Edit:

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Edit:

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related