How to convert pickle.loads from Python 2 to Python 3?

Question

I am trying to convert a Python 2 code into Python 3. I used the pickle.loads function in Python 2.7, which according to its documentation does the following (https://docs.python.org/2.7/library/pickle.html):

pickle.loads(string)
Read a pickled object hierarchy from a string. Characters in the 
string past the pickled object’s representation are ignored.

However its behavior changes in Python 3 (https://docs.python.org/3/library/pickle.html):

pickle.loads(bytes_object, *, fix_imports=True, encoding="ASCII", errors="strict")
Read a pickled object hierarchy from a bytes object and return the 
reconstituted object hierarchy specified therein.

In the database, I have a string x which was the output of pickle.dumps(obj) performed in Python 2. I want to retrieve obj in Python 3. When I do pickle.loads(x), I receive the following error:

a bytes-like object is required, not 'str'

Changing pickle.loads(x) into pickle.loads(bytes(x, 'utf-8')) instead gives the following error:

invalid load key, '\x5c'.

How can I obtain obj from x in Python 3?

If you're trying to read a pickle created by Python 2 into Python 3, you need to set the right protocol: docs.python.org/3/library/pickle.html#data-stream-format — jonrsharpe
– jonrsharpe, Commented Apr 22, 2019 at 9:30
This article outlines the entire process by which one would do this, including gotchas. — Alec
– Alec, Commented Apr 22, 2019 at 9:32
@jonrsharpe according to its documentation: The protocol version of the pickle is detected automatically, so no protocol argument is needed. Is it still necessary to explicitly set the protocol when I call python.loads? — Irvan
– Irvan, Commented Apr 22, 2019 at 9:33
@alec_a The article talks mostly about pickle.load instead of pickle.loads. I couldn't resolve my problem by trying a similar fix for pickle.loads. — Irvan
– Irvan, Commented Apr 22, 2019 at 9:34

Irvan · Accepted Answer · 2019-04-23 09:17:12Z

2

Change pickle.loads(x) to pickle.loads(bytes(x, 'latin-1')).

Change pickle.dumps(o) to str(pickle.dumps(o, protocol=0), 'latin-1').

answered Apr 23, 2019 at 9:17

Irvan

4494 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Paul Over a year ago

Or leveraging the internal of python, passing the encode arg would do it. pickle.load(x, encoding='latin1') Checkout this answer stackoverflow.com/questions/28218466/…

Collectives™ on Stack Overflow

How to convert pickle.loads from Python 2 to Python 3?

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related