How does Python pickle save objects?

Question

I want to know if I can save an object with multiple values/properties into one pickle file or do I have to save each value independently? Here is the code I have now:

def __init__(self, id, targets, binaries):
    self.id = id
    self.targets = targets
    self.binaries = binaries

with open('PI0_Electron_data.pickle', 'wb') as output:
    pickle.dump(PiElectron, output)

For better understanding id is an integer and both targets and binaries are numpy arrays. Will I be able to get the id, targets, and binaries of the object from this single pickle file or must I create three pickle files? Also how would I extract the data from the pickle file?

I am trying it now but it takes like 30 min because there is a lot of data — Troll_Hunter
– Troll_Hunter, Commented Aug 5, 2015 at 16:32
It should work. Use pickle.load to get the object from file. — saulspatz
– saulspatz, Commented Aug 5, 2015 at 16:37
First work on a sample set. Understand the concept and move onto your real data set. — ssapkota
– ssapkota, Commented Aug 5, 2015 at 16:38

Mike McKerns · Accepted Answer · 2015-08-05 17:20:12Z

2

Your code should look something like this:

>>> class Thing(object):
...   def __init__(self, id, targets, binaries):
...     self.id = id
...     self.targets = targets
...     self.binaries = binaries
... 
>>> import numpy as np
>>> t = Thing(1, np.arange(3), np.arange(3,9,2))
>>> 
>>> import dill
>>> with open('electron_data.pkl', 'w') as f:
...   dill.dump(t, f)
... 
>>>

Where I've used dill to give you better serialization… essentially, dill can pickle class instances easily -- as well as most python objects. Then when you want the object back you need to load.

Python 2.7.10 (default, May 25 2015, 13:16:30) 
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> with open('electron_data.pkl', 'r') as f:
...   t = dill.load(f)
... 
>>> t
<__main__.Thing object at 0x100394410>
>>> t.id
1
>>> t.targets
array([0, 1, 2])
>>> t.binaries
array([3, 5, 7])
>>>
>>> print dill.source.getsource(t.__class__)
class Thing(object):
  def __init__(self, id, targets, binaries):
    self.id = id
    self.targets = targets
    self.binaries = binaries

>>>

I hope that helps answer your question… if your actual code is more complex (I'm assuming it is), it should still work unless there's an unpicklable object.

answered Aug 5, 2015 at 17:20

Mike McKerns

35.5k8 gold badges126 silver badges149 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Troll_Hunter Over a year ago

Better than I could have hoped. Thanks

user3666197 Over a year ago

Incredible package, Mike. You should accept in public it is your, authored, package. Cool job! After a brief listing of supported features, could you confirm / deny a dill.dump() can save a state-full matplotlib 3D-view of aDataSET plot on a Figure() instance and allow to stream it / receive it / restore it on a remote system with still mouse-interactive 3D-viewing of a DataSET?

Mike McKerns Over a year ago

Yes, it's my package, and thanks. I can't confirm that every matplotlib object pickles with dill, but there has been an effort across several teams to make sure there is good coverage. See: github.com/uqfoundation/dill/issues/4

Collectives™ on Stack Overflow

How does Python pickle save objects?

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related