python input UnicodeDecodeError:

Question

python 3.x

>>> a = input()
hope
>>> a
'hope'
>>> b = input()
håpe
>>> b
'håpe'
>>> c = input()

start typing hå... delete using backspace... and change to hope

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc3 in position 1: invalid continuation byte
>>>

The situation is not terrible, I am working around it, but find it strange that when deleting, the bytes get messed up. Has anyone else experienced this?

the terminal history shows that it thought that I entered h?ope

any ideas?

in the script that is using this, I do import readline to give command line history.

standard one, 6-7 year old macbook, 10.6.8, although I am running visor. — beoliver
– beoliver, Commented Jul 9, 2012 at 13:44
readline can cause the Mac terminal to crash. On a second thought, the problem is that the Mac terminal can crash at all, so still a Mac problem. — Imperishable Night
– Imperishable Night, Commented Jun 25, 2019 at 23:09

phihag · Accepted Answer · 2012-07-08 21:21:25Z

8

It looks like backspace is deleting the last byte instead of the last character. What ends up happening is that you type:

68 c3 a5
h |  å

68 c3 6f 70 65
h |  |o |p |e

In UTF-8, a byte with the first bit set (c3) means that the next byte must have its first bit set as well (see Wikipedia's description).

Make sure that your terminal emulator and readline understand you're using UTF-8.

answered Jul 8, 2012 at 21:21

phihag

289k75 gold badges475 silver badges489 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

beoliver Over a year ago

this certainly makes sense. At least I know what is going on!

mathsyouth Over a year ago

@beoliver The same problem happed to me. How to solve it? Thanks.

phihag Over a year ago

@mathsyouth As written in the answer: Set your terminal emulator, as well as readline itself, to UTF-8. In your terminal emulator, it's likely in the settings somewhere. readline will read the environment variables, namely LC_ALL and friends.

Collectives™ on Stack Overflow

python input UnicodeDecodeError:

1 Answer 1

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related