Pandas dataframe row-wise formula with function

Question

So I have quite big DataFrame and I create a new column by some equation based on other columns:

df['F'] = (params.a * params.b * df.A/1000 - param.C * (df.B + df.C - df.D) + param.D * df.E

and it works perfectly fine. Except I want to repeat this function throughout the code, so instead of error-prone copying and pasting I want to cast it into a reusable function.

So I casted it into lambda:

def fun(r):
     return (params.a * params.b * r.A/1000 - param.C * (r.B + r.C - r.D) + param.D * r.E   
df['F'] = r.apply(funy,axis =1)

yet this is 5x slower now (1.2s vs 6s for 10k rows).

What should I do if I want to have a neat function and still benefit from speed?

Second one is not recommendable. first one is vectorized approach. — Mohamed Thasin ah
– Mohamed Thasin ah, Commented Aug 29, 2019 at 7:18
@AniketDixit but this means repeating the code everytime I use it, which I want to avoid. — Intelligent-Infrastructure
– Intelligent-Infrastructure, Commented Aug 29, 2019 at 7:26

Stef · Accepted Answer · 2019-08-29 07:44:17Z

1

What's wrong with:

def fun():
    return params.a * params.b * df.A/1000 - param.C * (df.B + df.C - df.D) + param.D * df.E

df['F'] = fun()

So you get a reusable vectorized function.

edited Aug 29, 2019 at 7:44

answered Aug 29, 2019 at 7:32

Stef

30.9k3 gold badges34 silver badges60 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Intelligent-Infrastructure Over a year ago

nice shortcut, let me try it.

Intelligent-Infrastructure Over a year ago

works like charm, %timeit gives exactly the same computation times. Only limitation (not for my case) is that it has to be in the same namespace and using variables that are already there. So best way is to define it inside a function from which you call

Collectives™ on Stack Overflow

Pandas dataframe row-wise formula with function

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related