pandas filter row null and

Question

I have a pandas dataframe as

my_df = pd.DataFrame({"months":[0,1,2,3,4,5], "value":[12,123,np.nan,234,345,456]})

I wanted to check for specific months(such as 0, 1, 3) any value is null or 0

I tried to do following way but does not work

if my_df[(my_df["months"].isin([0, 1, 3])) & (my_df["value"].isnull() | my_df["value"] == 0)].empty:

this still gives output as True

I want to check value for months 0,1,3 is not Nan or 0

Rachana Mahajan
– Rachana Mahajan

2021-07-21 17:09:58 +00:00
Commented Jul 21, 2021 at 17:09 — Rachana Mahajan
– Rachana Mahajan, Commented Jul 21, 2021 at 17:09

not_speshal · Accepted Answer · 2021-07-21 18:24:24Z

1

You forgot to add the "not" ~ part while checking for null values. Try this instead:

>>> my_df[(my_df["months"].isin([0,1,3])) & ~(my_df["value"].isnull()|my_df["value"]==0)]
   months  value
0       0   12.0
1       1  123.0
3       3  234.0

With your empty check:

>>> my_df[(my_df["months"].isin([0,1,3])) & ~(my_df["value"].isnull()|my_df["value"]==0)].empty
False

Edit:

To check if any value for the given months is not zero or null, you could do:

>>> any((my_df["months"].isin([0,1,2]))&((my_df["value"].isnull())|(my_df["value"]==0)))
True

edited Jul 21, 2021 at 18:24

answered Jul 21, 2021 at 17:09

not_speshal

23.2k2 gold badges18 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Rachana Mahajan Over a year ago

but if I change months and add 2 it give me the wrong output as False again >>> my_df[(my_df["months"].isin([0, 1, 2])) & ~(my_df["value"].isnull()|my_df["value"]==0)].empty

not_speshal Over a year ago

The output should be False. You have rows (0, 12) and (1, 123) which have months in [0, 1, 2] where the values are not null. What do you expect?

Rachana Mahajan Over a year ago

Oh, apologies (Changed the question removed not), I mean to say if any of given months have null or 0 values then I need to log them in error

not_speshal Over a year ago

@RachanaMahajan - Edited

Collectives™ on Stack Overflow

pandas filter row null and

1 Answer 1

Edit:

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Edit:

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related