Python: How to create a new dataframe from existing dataframe according to certain criteria?

Question

I have a dataframe called df_new and this is the current state:

       State_of_Discharge  Internal_Resistance
60                  0.308               -1.077
61                  0.308              307.745
182                 0.934                1.175
304                 1.560                0.392
305                 1.560              328.992
426                 2.186               -0.783
427                 2.186              329.384
548                 2.812                0.000
549                 2.813              323.705
670                 3.439                0.000
671                 3.439              320.180

I wish to take this dataframe and remove the values to exclude any values in Internal_Resistance that are near zero. The new data frame df_sorted shall look like this:

       State_of_Discharge  Internal_Resistance
61                  0.308              307.745
305                 1.560              328.992
427                 2.186              329.384
549                 2.813              323.705
671                 3.439              320.180

near_zero is a vague term. Can you tell the exact threshold of Internal_Resistance for which it should be omitted? — arshovon
– arshovon, Commented Jun 12, 2020 at 5:02
Let's say absolute value of '50'. So Anything within -50 to 50 shall be omitted. — encer
– encer, Commented Jun 12, 2020 at 5:42
@encer But the currently accepted answer only check for values greater than 50 but disregards negative values. — Ch3steR
– Ch3steR, Commented Jun 12, 2020 at 9:29

NotAName · Accepted Answer · 2020-06-12 05:03:45Z

1

The best way is to use loc:

df_sorted = df_new.loc[df_new['Internal_Resistance'] > 1.175]

answered Jun 12, 2020 at 5:03

NotAName

4,4744 gold badges39 silver badges59 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Ch3steR · Accepted Answer · 2020-06-12 05:18:32Z

0

You can use np.isclose here and define atol parameter set to the whatever deviation range is accepted from 0, i.e say -5 to 5 are values near to 0 then atol is set to 5.

from functools import partial

isclose = partial(np.isclose,0,atol=5)
df[~isclose(df['Internal_Resistance'])]

     State_of_Discharge  Internal_Resistance
61                0.308              307.745
305               1.560              328.992
427               2.186              329.384
549               2.813              323.705
671               3.439              320.180

Note: This can be done without using partial, df[~np.isclose(df['Internal_Resistance'],0,atol=5)] gives same results but it's more verbose IMO.

answered Jun 12, 2020 at 5:18

Ch3steR

20.8k4 gold badges34 silver badges66 bronze badges

Collectives™ on Stack Overflow

Python: How to create a new dataframe from existing dataframe according to certain criteria?

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related