UTF-8 encoding issue with logging imodule in python 3

Question

In my python (3.7.4) app I'm trying to analyze a few xml files (encoded with utf-8) in parallel (a few threads) and log a few messages regarding the rows in my log.

my code :

import logging
....
logging.basicConfig(filename='app.log', filemode='w',format='[%(asctime)s] [%(process)d][%(threadName)s - %(thread)d]- %(message)s', level=logging.INFO,encoding='utf-8')
xmldoc = minidom.parse(xml_file)
logging.info("starting to parse file : {}".format(xml_file))
itemsList = xmldoc.getElementsByTagName("Item")
for item in itemsList:
    currentItemName = item.getElementsByTagName("itemName")[0].firstChild.data
    logging.info("parsing item :{}".format(currentItemName))
    ...

the error that I'm getting regarding the logging row :

UnicodeEncodeError: 'charmap' codec can't encode characters in position 81-86: character maps to <undefined>

The file that I'm parsing is an xml file encoded in utf8 :

<?xml version="1.0" encoding="utf-8"?>

stack trace :

--- Logging error ---
Traceback (most recent call last):
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\logging\__init__.py", line 1028, in emit
    stream.write(msg + self.terminator)
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 82-87: character maps to <undefined>
Call stack:
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 890, in _bootstrap
    self._bootstrap_ifnner()
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 926, in _bootstrap_inner
    self.run()
  File "C:\Users\myuser\AppData\Local\Programs\Python\Python37-32\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\myuser\PycharmProjects\myproj\pars\xmlparsers.py", line 88, in parseXML
    logging.info("parsing item :{}".format(currentItem))
Message: 'parsing item לד500גר'

Also tried(didnt solve my problem) :

logging.basicConfig(
    handlers=[logging.FileHandler('main.log', 'w', 'utf-8')],
    format='[%(asctime)s] [%(process)d][%(threadName)s - %(thread)d]- %(message)s',
    level=logging.INFO,
    encoding='UTF-8')

I will be happy if someone can help me solve this problem..

Possible duplicate of python 3.6 *logging modul error* UnicodeEncodeError: 'charmap' codec can't encode characters — snakecharmerb
– snakecharmerb, Commented Aug 31, 2019 at 10:09
@snakecharmerb - I tried, didnt help (also updated my main comment) — JeyJ
– JeyJ, Commented Aug 31, 2019 at 12:40
It seems that once u configure the root logger you cant edit its setting. I configured the basicConfig of the logger in a different file and thats why all the changes in the current file (like enabling encoding for the handler) didnt take effect. — JeyJ
– JeyJ, Commented Aug 31, 2019 at 16:27

halfer · Accepted Answer · 2020-12-05 21:28:47Z

0

It seems that once you configure the root logger you can't edit its setting. I configured the basicConfig of the logger in a different file and that's why all the changes in the current file (like enabling encoding for the handler) didn't take effect.

edited Dec 5, 2020 at 21:28

halfer

20.2k20 gold badges110 silver badges207 bronze badges

answered Sep 16, 2019 at 7:12

JeyJ

4,1409 gold badges47 silver badges108 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

UTF-8 encoding issue with logging imodule in python 3

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related