23

In Pandas, I can use df.dropna() to drop any NaN entries. Is there anything similar in Pandas to drop non-finite (e.g. Inf) entries?

Improve this question
1

4 Answers 4

Reset to default
30

You can use:

with pd.option_context('mode.use_inf_as_null', True):
   df = df.dropna()
Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

this causes my dataframe to go from 130 rows to 0. The na/inf issue I'm investigating is a column of all blanks, i believe
You can specify the axis you wish to drop NaNs over as a parameter to dropna
This will be deprecated in pandas as of 2.1.0.
As @mwaskom said, this option has now been renamed to mode.use_inf_as_na
10 
df[np.isfinite(df) | np.isnan(df)]
Improve this answer

1 Comment

There's an np.isinfinity
3

You can use .dropna() after a DF[DF==np.inf]=np.nan, (unless you still want to keep the NANs and only drop the infs)

Improve this answer

2 Comments

Thanks. Surprisingly df.drop(df==np.inf) doesn't work. Any thoughts why?
I think, as the docstring goes: .dropna(self, axis=0, how='any', thresh=None, subset=None), df==np.inf will get passed as axis argument, with should raise an exception.
0

The mode.use_inf_as_null option has been deprecated, and its replacement mode.use_inf_as_na is also deprecated as well (will be removed in pandas 3.0). So future comers, you should first replace inf with NaN and then drop them:

df = df.replace([np.inf, -np.inf], np.nan)
df = df.dropna()
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.