How to define an instance attribute in a pandas DataFrame subclass?

Question

I am subclassing pandas DataFrame and I want to have an attribute.

class MyFrame(pd.DataFrame):


    _metadata = ['myattr']
    myattr = []


    def __init__(self, *args, **kwargs):
        pd.DataFrame.__init__(self, *args, **kwargs)

        self.myattr.append(0)


    @property
    def _constructor(self):
        return AutoData

My issue is that myattr is a class attribute. When I modify it in an instance of my class, every instances got modified:

mf2 = mf
mf2.myattr.append(1)
print(mf.myattr)
>>> [0, 1]

But I want the attribute to be attached with its instance. In other word, modify myattr only for mf2 but not for mf. Thank you.

You can simply define attribute in __init__ self.myattr = [], by this you are sure this is instance's attribute ;) — Damian
– Damian, Commented Jan 27, 2019 at 2:07
Yes but then it raises 2 issues: (1) myattr will not be attached to copies of my object and (2) it gives this warning: UserWarning: Pandas doesn't allow columns to be created via a new attribute name — Adrien Pavao
– Adrien Pavao, Commented Jan 27, 2019 at 2:15
Well, I don't know how much this will be problematic for you but consider creating class which will not inherit from DataFrame but instead will contain it as for example self.df = pd.DataFrame. Composition in this case looks like better solution because it will not need from you to adjust to DataFrame implementation at cost of wrapping it. Of course this isn't best solution at all cases but still, consider it :) — Damian
– Damian, Commented Jan 27, 2019 at 2:22
Thank you for you advice. I finally manage to find a solution by defining the attribute in __init__ and redefining the copy method which now copy also my attribute to the new DataFrame. Then I use copy() to duplicate my object. — Adrien Pavao
– Adrien Pavao, Commented Jan 28, 2019 at 1:18

K.Kbz · Accepted Answer · 2019-03-22 15:47:48Z

1

Assigning instance attributes to a pd.DataFrame subclass can be done as following:

class MyFrame(pd.DataFrame):
    _metadata = ['myattr']

    def __init__(self, *args, **kwargs):
        pd.DataFrame.__init__(self, *args, **kwargs)
        self.myattr = [0]

    @property
    def _constructor(self):
        return MyFrame

The _metadata list sets the attributes that should not be considered as columns. Names listed in _metadata are considered in the __setattr__() and __getattr__() methods of the parentclass of pd.DataFrame(which is NDFrame) and set as object attributes without raising the UserWarning.

While _metadata are normal properties there is also the possibility to set temporary properties with _internal_names as described in the pandas documentation. Temporary properties are not kept after dataframe modifictions.

edited Mar 22, 2019 at 15:47

answered Mar 22, 2019 at 15:42

K.Kbz

264 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to define an instance attribute in a pandas DataFrame subclass?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related