Pythonic way to prevent execution of functions for logging purposes

Question

Let's consider, for debugging purposes, we would like to log the result of costly_metric():

logger.info("some function")
logger.debug(f"result of costly metric: {costly_metric()}"

We only ever need this result for debugging and due to the computing resource costs involved in calculating costly_metric() it should not be executed if we set log level > debug.

However, logging, of course will execute the statement and then just hide the result. Minimal example:

l = logging.getLogger("logger")
l.setLevel(logging.INFO)

def foo():
    l.debug(f"costly metric: {costly_metric()}")
    l.info("test")

def costly_metric():
    l.warning("costly metric executed")
    return 42

If we call foo() now, this will result in:

> WARNING:logger:costly metric executed
> INFO:logger:test

Thus, costly metric: 42 is just hidden in the logger's output but the method costly_metric() still is getting executed.

In order to prevent this behavior, we could check for the level with an if statement:

def foo():
    if l.level <= logging.DEBUG:
        l.debug(f"costly metric: {costly_metric()}")
    l.info("test")

What would be the most pythonic way to resolve this issue?

see orbifold.xyz/logging-expensive.html

balderman
– balderman

2020-11-06 12:50:18 +00:00
Commented Nov 6, 2020 at 12:50 — balderman
– balderman, Commented Nov 6, 2020 at 12:50

MisterMiyagi · Accepted Answer · 2020-11-06 12:56:21Z

3

The logging module is intended to be used with separate log format and log content.

l.debug("costly metric: {}", costly_metric())

In order to defer evaluating not just the formatting, but also the content, a helper can be used:

class LazyStr:
    """Lazily evaluate a callable when its result string representation is needed"""
    def __init__(self, call):
        self.call = call

    # only perform the call when ``str`` or ``repr`` is used,
    # e.g. as part of log formatting
    def __str__(self):
        return str(self.call())

    def __repr__(self):
        return repr(self.call())

    # allow format field syntax, e.g. ``{:7.2f}``
    def __format__(self, format_spec):
        return format(self.call(), format_spec)

l.debug("costly metric: {}", LazyStr(costly_metric))

edited Nov 6, 2020 at 12:56

answered Nov 6, 2020 at 12:49

MisterMiyagi

53.4k14 gold badges131 silver badges138 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Pythonic way to prevent execution of functions for logging purposes

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related