ipython parallel push custom object

Question

I am unable to send object to direct view workers. Here is what I want to do:

class Test:
  def __init__(self):
    self.id = 'ddfdf'

from IPython.parallel import Client
rc = Client()
dv = rc[:]
t = Test()
dv['t'] = t
print dv['t']

NameError: name 't' is not defined

This would work if I try to push pandas object or any of the build in objects. What is the way to do it with custom object?

I tried:

dv['Test'] = Test
dv['t'] = t
print dv['t']
UnpicklingError: NEWOBJ class argument isn't a type object

Knowing something about how pickle works, try pushing the class (Test) before the instance (t). — Thomas K
– Thomas K, Commented Jun 20, 2014 at 0:31

minrk · Accepted Answer · 2014-06-20 01:37:09Z

2

For interactively defined classes (in __main__), you do need to push the class definition, or use dill. But even this doesn't appear to work for old-style classes, which is a bug in IPython's handling of old-style classes[1]. This code works fine if you use a new-style class:

class Test(object):
    ...

instead of on old-style class. Note that old-style classes are not available in Python 3. It's generally a good idea to always use new-style classes anyway.

answered Jun 20, 2014 at 1:37

minrk

38.9k9 gold badges98 silver badges90 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Ivelin Over a year ago

minrk: Your suggestion works for the example but when I import the Test class from file: 'from test import Test' then dv['Test'] = Test will fail. Do you have any suggestions?

minrk Over a year ago

You shouldn't need to push Test if it's in a module, but the module itself does need to be available. Is the module a local file? If so, it's probably a good idea to send the file to all of your machines. If your engines are local, then you may just want to make sure the engines have the same CWD as the Client.

Collectives™ on Stack Overflow

ipython parallel push custom object

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related