Applying functions declared as strings to a pandas dataframe

Question

I have a pandas dataframe. I want to create new columns in the dataframe with mathematical functional values of the existing columns.

I know how to do it for simple cases:

import pandas as pd
import numpy as np

# Basic dataframe
df = pd.DataFrame(data={'col1': [1,2], 'col2':[3,5]})
for i in df.columns:
    df[f'{i}_sqrt'] = df[i].apply(lambda x :np.sqrt(x))

produces

Now I want to extend it to the cases where the functions are written as strings like:

one_func = ['(x)', '(np.sqrt(x))']
two_func = ['*'.join(i) for i in itertools.product(one_func, one_func)]

so that two_func = ['(x)*(x)','(x)*(np.sqrt(x))','(np.sqrt(x))*(x)', '(np.sqrt(x))*(np.sqrt(x))']. Is there any way I can create columns like the first example with these new functions?

rafaelc · Accepted Answer · 2019-04-05 12:35:51Z

2

That looks like a bad design, but I won't go down that road.

Answering your question, you can use df.eval

First of all, set

one_func = ['{x}', '(sqrt({x}))']

with {} instead of () such that you can replace {x} for your actual column name.

Then, for instance,

expr = two_func[0].format(x='col1')
df.eval(expr)

The food loop your look like

for col in df.columns:
    for func in two_func: df[func] = df.eval(func.format(x=col))

edited Apr 5, 2019 at 12:35

answered Apr 5, 2019 at 12:33

rafaelc

59.4k15 gold badges64 silver badges87 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

anotherone Over a year ago

I know it is bad, it feels so but I cannot pinpoint the problem. I will be really grateful if you can suggest a better way of doing this.

rafaelc Over a year ago

Well, design problems are usually posted in code review SE, so maybe you could give it a go there ;}

Graipher Over a year ago

@RafaelC: Pure design questions are off-topic on Code Review. Giving suggestions on how to improve some working piece of code is on-topic (including what a better design for it would be). Have a look at their off-topic help center, where "Higher-level architecture and design of software systems" is explicitly listed. Those questions might be on-topic on Software Engineering, though.

rafaelc Over a year ago

@Graipher well this is not a design problem per se, but rather a 5 lines of code for loop that needs revision ;} These is no system or architecture in this problem, not even a function or a class ! But thanks for chiming in and contributing ! ;}

Collectives™ on Stack Overflow

Applying functions declared as strings to a pandas dataframe

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related