importing binary strings using json

Question

I like to import regular expressions using binary strings like

    ReplaceChars = (
    ('UTF8 HYPHEN '   , b'\xe2\x80\x90', b'\x2d'),
    ('UTF8 EN DASH'   , b'\xe2\x80\x93', b'\x2d'),
    ('UTF8 EM DASH'   , b'\xe2\x80\x94', b'\x2d'),
  )

Currently this is hardcoded in my python source, but I like to to have it in a different file to be more flexible.

I normally use json.load(jsonfile) to do this, but it seems that json is not working with the binary strings...

I tried to dump ReplaceChars into a json file:

with open('result.json', 'w') as f:
    json.dump(ReplaceChars, f)

but it causes the following error:

    Traceback (most recent call last):
[...]
    TypeError: Object of type 'bytes' is not JSON serializable

Is there a workaround in json?

mortenvp · Accepted Answer · 2020-04-27 07:29:52Z

1

You can use e.g. Base64 encoding to represent your binary data as strings. See here: https://docs.python.org/3/library/base64.html

answered Apr 27, 2020 at 7:29

mortenvp

1,1151 gold badge7 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

anthony sottile · Accepted Answer · 2020-04-27 07:41:06Z

0

It might be better to store the characters and encode to bytes in your application as needed (rather than hand-hard-coding the UTF-8 representation)

for example, considering the UTF8 HYPHEN:

>>> '\u2010'
'‐'
>>> '\u2010'.encode()
b'\xe2\x80\x90'
>>> '-'.encode()
b'-'
>>> {'UTF8 HYPHEN': ['\u2010', '-']}
{'UTF8 HYPHEN': ['‐', '-']}
>>> print(json.dumps({'UTF8 HYPHEN': ['\u2010', '-']}))
{"UTF8 HYPHEN": ["\u2010", "-"]}
>>> json.loads('{"UTF8 HYPHEN": ["\\u2010", "-"]}')['UTF8 HYPHEN'][0].encode()
b'\xe2\x80\x90'

answered Apr 27, 2020 at 7:41

anthony sottile

72.1k19 gold badges199 silver badges248 bronze badges

2 Comments

user3884301 Over a year ago

I have also CP1252 EN DASH as 'b/x96'... how to handle them?

anthony sottile Over a year ago

.encode('cp1252') will take a unicode string and give you the bytes you need

Ketan Krishna Patil · Accepted Answer · 2020-04-27 07:35:36Z

0

Instead of using a JSON file, you can use the pickle file to store binary strings data.

import pickle
from ast import literal_eval as make_tuple
ReplaceChars = (
    ('UTF8 HYPHEN '   , b'\xe2\x80\x90', b'\x2d'),
    ('UTF8 EN DASH'   , b'\xe2\x80\x93', b'\x2d'),
    ('UTF8 EM DASH'   , b'\xe2\x80\x94', b'\x2d'),
  )

with open('result.pkl', 'wb') as file:
    pickle.dump(ReplaceChars, file)

with open('result.pkl', 'rb') as file:
    Result_ReplaceChars = pickle.load(file)

print(Result_ReplaceChars)

answered Apr 27, 2020 at 7:35

Ketan Krishna Patil

811 silver badge6 bronze badges

1 Comment

user3884301 Over a year ago

This results in a binary file... since I need to have some human readable file for better handling this is not running for me

Collectives™ on Stack Overflow

importing binary strings using json

3 Answers 3

Comments

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related