How to get the system default encoding in Python 2.x?

Question

I tried sys.getdefaultencoding() but unfortunately that doesn't work. It returns ascii on my system which has the system locale set to ja_JP (i.e. the encoding should be Shift-JIS).

I'm trying to parse CJK text (on Windows). I need to read some text from keyboard, determine the system encoding, and convert it to utf8. I would often change my system locale between zh_CN (GBK encoding) and ja_JP (Shift-JIS encoding) so hard-coding the system encoding (encoding of keyboard-input text) is not an option. Any solutions?

jack980517 · Accepted Answer · 2016-05-29 05:46:10Z

6

Solved: sys.stdin.encoding

Also, for anyone trying sys.getdefaultencoding(), it would almost never work and would always be ascii according to https://wiki.python.org/moin/DefaultEncoding:

Its value should be 'ascii' and it is used when converting byte strings to unicode strings.

and

If you put non-ascii characters into byte string then .decode(sys.getdefaultencoding()) method will fail with UnicodeDecodeError, therefore byte strings should not contain non-ascii characters.

answered May 29, 2016 at 5:46

jack980517

1011 silver badge7 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Holger Joukl · Accepted Answer · 2020-01-10 15:22:35Z

3

Can't comment to the answer above due to reputation, but:

Be careful with sys.[stdin|stdout].encoding when running in non-tty mode:

$ tty
/dev/pts/2
$ python2 -c'import locale, sys; print(sys.stdout, type(sys.stdout), sys.stdin.encoding, locale.getpreferredencoding())'
(<open file '<stdout>', mode 'w' at 0x7fa77abff150>, <type 'file'>, 'UTF-8', 'UTF-8')

$ ssh $HOSTNAME tty
not a tty
$ ssh $HOSTNAME "python2 -c'import locale, sys; print(sys.stdout, type(sys.stdout), sys.stdin.encoding, locale.getpreferredencoding())'"
(<open file '<stdout>', mode 'w' at 0x7f7991c43150>, <type 'file'>, None, 'UTF-8')

As you can see it might be safer to use locale.getpreferredencoding().

(Not a problem on Py3 so this is just for historical record as of 2020 ;-)).

answered Jan 10, 2020 at 15:22

Holger Joukl

915 bronze badges

1 Comment

Hedede Over a year ago

Well, there are still people who use python2 in 2022, so this is actually useful ; )

Collectives™ on Stack Overflow

How to get the system default encoding in Python 2.x?

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related