Pandas DataFrame Apply

Question

I have a Pandas DataFrame with four columns, A, B, C, D. It turns out that, sometimes, the values of B and C can be 0. I therefore wish to obtain the following:

B[i] = B[i] if B[i] else min(A[i], D[i])
C[i] = C[i] if C[i] else max(A[i], D[i])

where I have used i to indicate a run over all rows of the frame. With Pandas it is easy to find the rows which contain zero columns:

df[df.B == 0] and df[df.C == 0]

however I have no idea how to easily perform the above transformation. I can think of various inefficient and inelegant methods (for loops over the entire frame) but nothing simple.

Wouter Overmeire · Accepted Answer · 2012-08-05 20:07:32Z

8

A combination of boolean indexing and apply can do the trick. Below an example on replacing zero element for column C.

In [22]: df
Out[22]:
   A  B  C  D
0  8  3  5  8
1  9  4  0  4
2  5  4  3  8
3  4  8  5  1

In [23]: bi = df.C==0

In [24]: df.ix[bi, 'C'] = df[bi][['A', 'D']].apply(max, axis=1)

In [25]: df
Out[25]:
   A  B  C  D
0  8  3  5  8
1  9  4  9  4
2  5  4  3  8
3  4  8  5  1

answered Aug 5, 2012 at 20:07

Wouter Overmeire

69.7k10 gold badges67 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Freddie Witherden Over a year ago

Quite neat. However, I think that you can get away with .max(axis=1) instead of apply(...).

Wouter Overmeire Over a year ago

max() is ok too of course, i think i got biased towards apply by the way you asked the question :-)

THM · Accepted Answer · 2012-08-03 19:05:11Z

2

Try 'iterrows' DataFrame class method for efficiently iterating through the rows of a DataFrame.See chapter 6.7.2 of the pandas 0.8.1 guide.

from pandas import *
import numpy as np

df = DataFrame({'A' : [5,6,3], 'B' : [0,0,0], 'C':[0,0,0], 'D' : [3,4,5]})

for idx, row in df.iterrows():
    if row['B'] == 0:
        row['B'] = min(row['A'], row['D'])
    if row['C'] == 0:
        row['C'] = min(row['A'], row['D'])

answered Aug 3, 2012 at 19:05

THM

8052 gold badges9 silver badges14 bronze badges

Collectives™ on Stack Overflow

Pandas DataFrame Apply

2 Answers 2

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related