Using a For Loop to modify a dictionary of DataFrames in Pandas

Question

I was going through a tutorial on Pandas. I decided to experiment halfway with what I thought should be straight forward. I condensed it to a simple code for others to reproduce personally and help me see what is my error or the error in Python.

df = pd.DataFrame({'A': 1.,
                   'B': pd.Timestamp('20130102'),
                   'C': pd.Series(1, index = list(range(4)), dtype = 'float32'),
                   'D': np.array([3] * 4, dtype = 'int32'),
                   'E': pd.Categorical(["test", "train", "test", "train"]),
                   'F': 'foo'
                   })

# Made copy of df and modified it individually to show that it works.
df2 = df
df2.drop([1,3], inplace=True) # Dropping 2nd and 5th row.
print(df2)

# Now trying to do the same for multiple dataframes in a 
# dictionary keeps giving me an error.

dic = {'1900' : df, '1901' : df, '1902' : df} # Dic w/ 3 pairs.
names = ['1900', '1901', '1902']              # The dic keys in list.

# For loop to drop the 2nd and 4th row.
for ii in names:
    df_dic = dic[str(ii)]
    df_dic.drop([1,3], inplace=True)
    dic[str(ii)] = df_dic

The output I get is:

     A          B    C  D     E    F
0  1.0 2013-01-02  1.0  3  test  foo
2  1.0 2013-01-02  1.0  3  test  foo
--------------------------------------------------------------------------
ValueError                               Traceback (most recent call last)
<ipython-input-139-8236a9c3389e> in <module>()
     21 for ii in names:
     22     df_dic = dic[str(ii)]
---> 23     df_dic.drop([1,3], inplace=True)

C:\Anaconda3\lib\site-packages\pandas\core\generic.py in drop(self, labels, axis, level, inplace, errors)
   1905                 new_axis = axis.drop(labels, level=level, errors=errors)
   1906             else:
-> 1907                 new_axis = axis.drop(labels, errors=errors)
   1908             dropped = self.reindex(**{axis_name: new_axis})
   1909             try:

C:\Anaconda3\lib\site-packages\pandas\indexes\base.py in drop(self, labels, errors)
   3260             if errors != 'ignore':
   3261                 raise ValueError('labels %s not contained in axis' %
-> 3262                                  labels[mask])
   3263             indexer = indexer[~mask]
   3264         return self.delete(indexer)

ValueError: labels [1 3] not contained in axis

So obviously the dropping the rows when doing it individually works since it gave me the desired output. Why is implementing in a For Loop making it behave strangely?

Thanks in advance.

I think you need add copy like df_dic = dic[str(ii)].copy() — jezrael
– jezrael, Commented Dec 23, 2016 at 11:26

jezrael · Accepted Answer · 2016-12-23 11:36:01Z

2

You need copy DataFrame:

for ii in names:
    df_dic = dic[str(ii)].copy()
    df_dic.drop([1,3], inplace=True)
    dic[str(ii)] = df_dic

print (dic)
{'1900':      A          B    C  D     E    F
0  1.0 2013-01-02  1.0  3  test  foo
2  1.0 2013-01-02  1.0  3  test  foo, '1902':      A          B    C  D     E    F
0  1.0 2013-01-02  1.0  3  test  foo
2  1.0 2013-01-02  1.0  3  test  foo, '1901':      A          B    C  D     E    F
0  1.0 2013-01-02  1.0  3  test  foo
2  1.0 2013-01-02  1.0  3  test  foo}

Copying in docs.

edited Dec 23, 2016 at 11:36

answered Dec 23, 2016 at 11:30

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

ssurendr Over a year ago

That makes a lot of sense. But after adding the copy(), I still get the exact same error. And it points to the line after df_dic = dic[str(ii)].copy() in the error message. I appreciate that you replied very quickly.

jezrael Over a year ago

Maybe in your code df2 = df is important add copy too. ;)

ssurendr Over a year ago

High Five!! That was it. Thank you very much! By the way, I could have swore you had a link to something. It is now gone. Was it something that would help me out?

jezrael Over a year ago

I think no, I only add link to the end of answer.

ssurendr Over a year ago

Yeah, that was the link I was referring to. I guess my weary eyes lost track of it the second time. Haha. And no problem. You answered it correctly, so it is a given. :)

|

Collectives™ on Stack Overflow

Using a For Loop to modify a dictionary of DataFrames in Pandas

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related