Conditional on pandas DataFrame's

Question

Let df1, df2, and df3 are pandas.DataFrame's having the same structure but different numerical values. I want to perform:

res=if df1>1.0: (df2-df3)/(df1-1) else df3

res should have the same structure as df1, df2, and df3 have.

numpy.where() generates result as a flat array.

Edit 1:

res should have the same indices as df1, df2, and df3 have.

For example, I can access df2 as df2["instanceA"]["parameter1"]["paramter2"]. I want to access the new calculated DataFrame/Series res as res["instanceA"]["parameter1"]["paramter2"].

JohnE · Accepted Answer · 2015-07-30 13:45:14Z

Actually numpy.where should work fine there. Output here is 4x2 (same as df1, df2, df3).

df1 = pd.DataFrame( np.random.randn(4,2), columns=list('xy') )
df2 = pd.DataFrame( np.random.randn(4,2), columns=list('xy') )
df3 = pd.DataFrame( np.random.randn(4,2), columns=list('xy') )

res = df3.copy()
res[:] = np.where( df1 > 1, (df2-df3)/(df1-1), df3 )

          x         y
0 -0.671787 -0.445276
1 -0.609351 -0.881987
2  0.324390  1.222632
3 -0.138606  0.955993

Note that this should work on both series and dataframes. The [:] is slicing syntax that preserves the index and columns. Without that res will come out as an array rather than series or dataframe.

Alternatively, for a series you could write as @Kadir does in his answer:

res = pd.Series(np.where( df1>1, (df2-df3)/(df1-1), df3 ), index=df1.index)

Or similarly for a dataframe you could write:

res = pd.DataFrame(np.where( df1>1, (df2-df3)/(df1-1), df3 ), index=df1.index,
                                                              columns=df1.columns)

Community · Accepted Answer · 2017-05-23 12:14:01Z

1

Integrating the idea in this question into JohnE's answer, I have come up with this solution:

res = pd.Series(np.where( df1 > 1, (df2-df3)/(df1-1), df3 ), index=df1.index)

A better answer using DataFrames will be appreciated.

edited May 23, 2017 at 12:14

CommunityBot

11 silver badge

answered Jul 30, 2015 at 6:30

Kadir

1,7153 gold badges17 silver badges28 bronze badges

Comments

AZhao · Accepted Answer · 2015-07-29 16:03:18Z

0

Say df is your initial dataframe and res is the new column. Use a combination of setting values and boolean indexing.

Set res to be a copy of df3:

 df['res'] = df['df3']

Then adjust values for your condition.

df[df['df1']>1.0]['res'] = (df['df2'] - df['df3'])/(df['df1']-1)

answered Jul 29, 2015 at 16:03

AZhao

14.5k8 gold badges33 silver badges55 bronze badges

1 Comment

JohnE Over a year ago

I believe df1/df2/df3 should be dataframes, not columns.

Collectives™ on Stack Overflow

Conditional on pandas DataFrame's

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related