Python pickle instance variables

Question

I am doing some calculations on an instance variable, and after that is done I want to pickle the class instance, such that I don't have to do the calculations again. Here an example:

import cPickle as pickle

class Test(object):
    def __init__(self, a, b):
        self.a = a
        self.b = b
        self.c = None
    def compute(self, x):
        print 'calculating c...'
        self.c = x * 2

test = Test(10, 'hello')
test.compute(6)

# I have computed c and I want to store it, so I don't have to recompute it again:

pickle.dump(test, open('test_file.pkl', 'wb'))

After test.compute(6) I can check to see what test.__dict__ is:

>>> test.__dict__
{'a': 10, 'c': 12, 'b': 'hello'}

I thought that is what going to get pickled; however,

When I go to load the class instance:

import cPickle as pickle

from pickle_class_object import Test

t2 = pickle.load(open('test_file.pkl', 'rb'))

I see this in the shell:

calculating c...

Which means that I did not pickle c and I am computing it over again.

Is there a way to pickle test how I want to? So I don't have to compute c over again. I see that I could just pickle test.__dict__, but I am wondering if there is a better solutions. Also, my understanding about what is going on here is weak, so any comment about what is going would be great. I've read about __getstate__ and __setstate__, but I don't see how to apply them here.

Martijn Pieters · Accepted Answer · 2013-05-19 19:32:29Z

3

You are importing the pickle_class_object module again, and Python runs all code in that module.

Your top-level module code includes a call to .compute(), that is what is being called.

You may want to move the code that creates the pickle out of the module, or move it to a if __name__ == '__main__': guarded section:

if __name__ == '__main__':
    test = Test(10, 'hello')
    test.compute(6)

    pickle.dump(test, open('test_file.pkl', 'wb'))

Only when running a python file as the main script is __name__ set to __main__; when imported as a module __name__ is set to the module name instead and the if branch will not run.

answered May 19, 2013 at 19:32

Martijn Pieters

1.1m326 gold badges4.2k silver badges3.4k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

mgilson · Accepted Answer · 2013-05-19 19:50:41Z

2

Pickling works as you expect it to work. The problem here is when you run the new script, you import the module that contains the class Test. That entire module is run including the bit where you create test.

The typical way to handle this sort of thing would be to protect the stuff in a if __name__ == "__main__: block.

class Test(object):
    def __init__(self, a, b):
        self.a = a
        self.b = b
        self.c = None
    def compute(self, x):
        print 'calculating c...'
        self.c = x * 2

if __name__ == "__main__":
    import cPickle as pickle

    test = Test(10, 'hello')
    test.compute(6)

    # I have computed c and I want to store it, so I don't have to recompute it again:

    pickle.dump(test, open('test_file.pkl', 'wb'))

edited May 19, 2013 at 19:50

answered May 19, 2013 at 19:32

mgilson

312k70 gold badges656 silver badges722 bronze badges

Comments

7stud · Accepted Answer · 2013-05-19 19:44:46Z

1

That isn't what's happening. You import a python module that has code in it at the top level, which executes when you import the module. You can see that your code works as you intended:

import cPickle as pickle

class Test(object):
    def __init__(self, a, b):
        self.a = a
        self.b = b
        self.c = None
    def compute(self, x):
        print 'calculating c...'
        self.c = x * 2

test = Test(10, 'hello')
test.compute(6)

pickle.dump(test, open('test_file.pkl', 'wb'))

t2 = pickle.load(open('test_file.pkl', 'rb'))
print t2.c


--output:--
calculating c...
12

If your code worked as you describe, then you would see "calculating c..." twice.

edited May 19, 2013 at 19:44

answered May 19, 2013 at 19:39

7stud

48.8k14 gold badges107 silver badges136 bronze badges

Collectives™ on Stack Overflow

Python pickle instance variables

3 Answers 3

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related