python 3.6 logging modul error UnicodeEncodeError: 'charmap' codec can't encode characters

Question

On Windows 10 the logging module send this error (using scrapy)

# --- Logging error ---
...
# UnicodeEncodeError: 'charmap' codec can't encode characters in position 175-176: character maps to <undefined>

~~I have read that I should add encoding='utf-8' but I did not find how I could add it in the code below.~~ EDIT: According to the tuto it isnt needed.

configure_logging(install_root_handler=False) #override default log settings
logging.basicConfig(
    filename='logfile.log',
    format='%(levelname)s: %(message)s',
    datefmt='%m-%d %H:%M',
    level=logging.INFO #CRITICAL ERROR WARNING  INFO    DEBUG    NOTSET 
)

I found many questions on the topics, but mostly on python 2 (or not related to the logging module). And the logging tutorial don't even talk about utf-8. (Notice that I can print UTF8 characters without any problem. The problem only occur with the logging module)

Cukic0d · Accepted Answer · 2018-05-08 17:42:35Z

41

I'm not using Scrapy but I ran into the same issue with vanilla Python 3.6 logging of not being able to write UTF-8 characters via logging to a file (console worked just fine).

Based on this comment I added 'utf-8' to my FileHandler initialization. I'm configuring logging via an INI file so it looks like this:

[handler_file]
class = FileHandler
args = (r'log.txt', 'a', 'utf-8')
level = NOTSET
formatter = generic

To use logging.basicConfig() I think that you should be able to do the following:

configure_logging(install_root_handler=False) #override default log settings
logging.basicConfig(
    handlers=[logging.FileHandler('logfile.log', 'w', 'utf-8')],
    format='%(levelname)s: %(message)s',
    datefmt='%m-%d %H:%M',
    level=logging.INFO #CRITICAL ERROR WARNING  INFO    DEBUG    NOTSET 
)

edited May 8, 2018 at 17:42

Cukic0d

5,4882 gold badges25 silver badges53 bronze badges

answered May 7, 2018 at 22:35

smsearcy

5836 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Pixel78 Over a year ago

The answer with logging.basicConfig() doesn't work for me.

smsearcy Over a year ago

@SomJura, can you be a little more specific? I just tried it on Python 3.8 and it seemed to work (I did skip the configure_logging() call).

Pixel78 Over a year ago

It seems that it didn't work because I was using import logging, which is not necessay anymore with newer versions of Python.

Abhirami V S Over a year ago

Is it possible in windows? Facing the same issue, I have tried the above solution but it's not working. I'm using Python 3.8.10. I tried removing the import statement, 'import logging', but that's causing errors. 'NameError: name 'logging' is not defined'

smsearcy Over a year ago

@Abhi, I initially had this problem in Windows so it should work in Windows. What error are you having? Reviewing the second logging.basicConfig() example, I'm not sure what that first line with configure_logging() is doing there, what if you delete that? You will need import logging.

|

Mohsen HNSJ · Accepted Answer · 2024-11-24 14:20:54Z

1

While using dictconfig, with any file handler (like RotatingFileHandler) you can use the encoding parameter and set it to utf-8.

Sample YAML file as logging config:

    ...
    handlers:
     file_output:
      class: logging.handlers.RotatingFileHandler
      level: INFO
      filename: logs/info.log
      maxBytes: 100000
      backupCount: 60
      encoding: utf-8
    ...

edited Nov 24, 2024 at 14:20

answered Nov 24, 2024 at 14:17

Mohsen HNSJ

238 bronze badges

Collectives™ on Stack Overflow

python 3.6 logging modul error UnicodeEncodeError: 'charmap' codec can't encode characters

2 Answers 2

6 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related