UnicodeDecodeError on interpolating str message to Unicode logging template in Python

Question

I'm trying to create a utf-8 log file with Python 2.7 on Win 8. I'm always getting UnicodeDecodeError: 'ascii' codec can't decode byte 0xfc in position 20: ordinal not in range(128) when I'm trying to log Windows paths that contain non-latin characters in them. The following setup invariably throws the dreaded UnicodeDecodeError:

import logging
from logging import handlers

def GetLogger():
    logger = logging.getLogger('log')
    if not len(logger.handlers):
        logger.setLevel(logging.DEBUG)
        logger.propagate = False
        # create a file handler
        fileHandler = logging.handlers.TimedRotatingFileHandler('mylog.log', when = 'midnight', backupCount = 10, encoding = 'utf-8')
        fileHandler.setLevel(logging.DEBUG)
        # unicode template here
        fileHandler.setFormatter(logging.Formatter(u'[%(levelname)-8s: %(asctime)s - %(filename)s:%(lineno)s - %(funcName)s()] - %(message)s'))
        logger.addHandler(fileHandler)
    return logger

logger = GetLogger()
path = 'e:\\\xcf\xf0\xee' + 'foo.bar' # actually I'm just calling os.getcwd() + 'foo.bar' here, and that's what it returns
print type(path) # prints <type 'str'>
logger.warning('Didn\'t find path %s' % path) # <- POOF!
# logger.warning('Didn\'t find path %s' % 'abcde') <- This would work

I tried wrapping path in unicode(path), but it didn't help. I'm at my wit's end. How do I log these tricky paths?

Did you try path = u'e:\\\xcf\xf0\xee' + u'foo.bar'?

Burhan Khalid
– Burhan Khalid

2017-01-08 04:54:22 +00:00
Commented Jan 8, 2017 at 4:54 — Burhan Khalid
– Burhan Khalid, Commented Jan 8, 2017 at 4:54

Mark Tolonen · Accepted Answer · 2018-02-15 17:30:45Z

2

Call os.getcwdu() for the Unicode version of the current path and use Unicode strings in your logger.warning.

When dealing with Unicode, it's best to use Unicode strings for all text, or use Python 3.x where it becomes the default type for 'xxxx' and b'xxxx' is the syntax for byte strings. You can also use from __future__ import unicode_literals in Python 2.7 to mimic the Python 3.x behavior. See PEP 3112 for more info.

edited Feb 15, 2018 at 17:30

answered Jan 8, 2017 at 4:42

Mark Tolonen

181k26 gold badges182 silver badges278 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

UnicodeDecodeError on interpolating str message to Unicode logging template in Python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related