Pandas DataFrame indexing, Selecting rows with specific columns that are NaN values

Question

Considering I have a pandas dataframe as the following example (with more rows/columns in the real dataset):

   | t1 | val1 | val2 | val3 | val4
------------------------------------
  0| 1  | 1    | NaN  | NaN  | NaN
  1| 2  | 12   |  5   | NaN  | 4
  2| 3  | 104  |  6   | NaN  | NaN
  3| 4  | -1   |  7   | 6    | NaN
  4| 1  | -3   |  8   | 7    | 10

I would like to extract only the rows where t1 == 1 and val2, val3 and val4 are NaN values and only some of the columns.

For instance, in the dataframe above I would like to get only the first row.

So far I have tried the following and some variations of it with no lack:

I have defined a list of labels for the columns I am interested in:

labels = [ 't1', 'val2', 'val3', 'val4']

Then I run the following code to get all values with t1 == 1 and only the specified columns.

df2 = df.loc[df.t1 == 1, labels]

Afterwards I am trying to get only the rows that val2, val3 and val4 are NaN at the same time. I have written the following code but it does not seem to work:

df3 = df2.loc[df2[labels].isnull() == True, labels]

But it returns the following error:

ValueError: Cannot index with multidimensional key

Do you know what is wrong? Or another way of getting the results I would like to?

Thanks in advance.

BENY · Accepted Answer · 2018-05-01 13:46:02Z

2

You should using all

df2[df2[['val2','val3','val4']].isnull().all(1)]
Out[544]: 
   t1  val2  val3  val4
0   1   NaN   NaN   NaN

answered May 1, 2018 at 13:46

BENY

324k22 gold badges176 silver badges250 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

mkanakis Over a year ago

You are indeed correct, could you care to explain what all with parameter 1 stands for?

BENY Over a year ago

@Davelis4 all mean all True, 1 standard for axis =1 which check all row

Collectives™ on Stack Overflow

Pandas DataFrame indexing, Selecting rows with specific columns that are NaN values

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related