Row selection in data frame based on condition

Question

The dataframe below has time series data

| datetime                | mc | quality | v0 | v1 | v2 | v5 | v6 | v7   | v8 | v9 | v10 |
|-------------------------|----|---------|----|----|----|----|----|------|----|----|-----|
| 2021-12-27 14:00:56.420 | L2 | 192     | 1  | 0  | 0  | 0  | 0  | 5515 | 0  | 0  | 0   |
| 2021-12-27 14:01:25.267 | L2 | 192     | 1  | 0  | 0  | 0  | 0  | 5507 | 0  | 0  | 0   |
| 2021-12-27 14:01:54.113 | L2 | 192     | 1  | 0  | 0  | 0  | 0  | 5524 | 0  | 0  | 0   |
| 2021-12-27 14:02:22.897 | L2 | 192     | 1  | 5  | 6  | 1  | 0  | 5513 | 0  | 0  | 0   |
| 2021-12-27 14:02:51.747 | L2 | 192     | 1  | 0  | 0  | 0  | 0  | 5455 | 0  | 0  | 0   |
| 2021-12-27 14:03:51.743 | L2 | 192     | 1  | 0  | 0  | 0  | 0  | 5522 | 7  | 3  | 9   |
| 2021-12-27 14:04:51.743 | L2 | 192     | 1  | 4  | 0  | 0  | 0  | 5522 | 7  | 3  | 9   |

Requirement: I need to select rows in which either v1 or v9 is not equal '0'

| datetime                | mc | quality | v0 | v1 | v2 | v5 | v6 | v7   | v8 | v9 | v10 |
|-------------------------|----|---------|----|----|----|----|----|------|----|----|-----|
| 2021-12-27 14:02:22.897 | L2 | 192     | 1  | 5  | 6  | 1  | 0  | 5513 | 0  | 0  | 0   |
| 2021-12-27 14:03:51.743 | L2 | 192     | 1  | 0  | 0  | 0  | 0  | 5522 | 7  | 3  | 9   |
| 2021-12-27 14:04:51.743 | L2 | 192     | 1  | 4  | 0  | 0  | 0  | 5522 | 7  | 3  | 9   |

Trials:

df = df[df.v1 != 0]

If I select individually it works

df = df[(df.v1 != 0) & (df.v9 != 0)]

But if I put condition it doesn't work, I get empty data frame.

Would like to know the efficient method

jezrael · Accepted Answer · 2021-12-27 11:51:39Z

1

Use | for bitwise OR:

df1 = df[(df.v1 != 0) | (df.v9 != 0)]
print (df1)
                 datetime  mc  quality  v0  v1  v2  v5  v6    v7  v8  v9  v10
3 2021-12-27 14:02:22.897  L2      192   1   5   6   1   0  5513   0   0    0
5 2021-12-27 14:03:51.743  L2      192   1   0   0   0   0  5522   7   3    9
6 2021-12-27 14:04:51.743  L2      192   1   4   0   0   0  5522   7   3    9

Or test both columns with DataFrame.any for get rows if at least one True in boolean DataFrame:

df1 = df[(df[['v1','v9']] != 0).any(axis=1)]
print (df1)
                 datetime  mc  quality  v0  v1  v2  v5  v6    v7  v8  v9  v10
3 2021-12-27 14:02:22.897  L2      192   1   5   6   1   0  5513   0   0    0
5 2021-12-27 14:03:51.743  L2      192   1   0   0   0   0  5522   7   3    9
6 2021-12-27 14:04:51.743  L2      192   1   4   0   0   0  5522   7   3    9

edited Dec 27, 2021 at 11:51

answered Dec 27, 2021 at 11:45

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

user_v27 Over a year ago

I have one question. Suppose if the data frame is of large volume, is it better have condition using '|' or 2d one

jezrael Over a year ago

@user_v27 - first. Second is better if need test e.g. 100 columns,

user_v27 Over a year ago

oh! ok. Thank you!

Collectives™ on Stack Overflow

Row selection in data frame based on condition

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related