Exception in Python Trace Module: "Charmap Can't Encode..."

Question

I am using the python trace module in order to figure out why my program is exiting early.

However, there seems to be a bug with the trace module itself.

When i run py -m trace -t src/main.py > temp/trace.txt in powershell, I get the following error:

Traceback (most recent call last):
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\trace.py", line 740, in <module>
    main()
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\trace.py", line 728, in main
    t.runctx(code, globs, globs)
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\trace.py", line 450, in runctx
    exec(cmd, globals, locals)
  File "src/main.py", line 11, in <module>
    from project_core.image_processing import *
  File "G:\My Drive\Swamynathan Lab Image Processing\ImageProcessingProjectLatestCodeWithGitVersionTracking\project\project_core\image_processing.py", line 7, in <module>
    import numpy as np
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\__init__.py", line 154, in <module>
    from . import polynomial
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\polynomial\__init__.py", line 116, in <module>
    from .polynomial import Polynomial
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\polynomial\polynomial.py", line 87, in <module>
    from ._polybase import ABCPolyBase
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\polynomial\_polybase.py", line 18, in <module>
    class ABCPolyBase(abc.ABC):
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\polynomial\_polybase.py", line 73, in ABCPolyBase
    "0": "⁰",
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\site-packages\numpy\polynomial\_polybase.py", line 73, in ABCPolyBase
    "0": "⁰",
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\trace.py", line 575, in localtrace_trace
    print("%s(%d): %s" % (bname, lineno,
  File "C:\Users\micha\AppData\Local\Programs\Python\Python310\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2070' in position 32: character maps to <undefined>

Is there any straightforward way to fix this problem?

Your title is misleading, there's nothing about UTF-8 in that error message. But you might be able to fix it by setting the environment variable PYTHONIOENCODING=utf-8 before running the program. — Mark Ransom
– Mark Ransom, Commented Jul 23, 2022 at 3:06
@Mark Ransom Title changed. Also your solution worked. Thanks! — Michael Sohnen
– Michael Sohnen, Commented Jul 23, 2022 at 3:22
I’m voting to close this question because the question was simple enough to answer in the comments. — Michael Sohnen
– Michael Sohnen, Commented Jul 23, 2022 at 3:29
No need to close it, it's still a question that could be useful to someone with the same problem. But the title's still wrong, it says "decode" but your error says "encode". — Mark Ransom
– Mark Ransom, Commented Jul 23, 2022 at 3:31
@Mark Ransom. Got it. But as a new-ish so member, what is the best way to manage a question if it was answered in the comments? — Michael Sohnen
– Michael Sohnen, Commented Jul 23, 2022 at 3:39

Mark Ransom · Accepted Answer · 2022-07-23 14:46:21Z

4

The problem is that the trace module is opening a file with the default encoding for your Python implementation. On Windows this is often cp1252 which you can see in the error dump. CP1252 is incapable of encoding every possible Unicode character, so sometimes you'll run into this problem.

Python will use the environment variable PYTHONIOENCODING to override the default. If you set it before you start Python to something that can encode all the Unicode characters such as utf-8 or utf-8-sig it will eliminate the error. Unfortunately this will affect all other text files opened by your program.

answered Jul 23, 2022 at 14:46

Mark Ransom

310k44 gold badges423 silver badges660 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Exception in Python Trace Module: "Charmap Can't Encode..."

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related