How to subset a DataFrame, using loc(), based on multiple columns having a particular value with Python?

Question

If I want to create a subset of a DataFrame, based on a condition where a specified column can have multiple specified values, I can do this:

df = df.loc[df[column_name].isin(list_of_acceptable_values)]

If I have a list of column names, what is the best way to create a subset of a DataFrame, based on a condition, which checks if these columns contain a particular value. For example, the list of column names is:

['column_1', 'column_2', 'column_3']

And I want to create a new DataFrame, which only has rows from the initial dataframe that contain 0 in column_1, column_2 or column_3

df[(df[list_of_cols] == 0).any(axis=1)] should work

EdChum
– EdChum

2017-05-10 15:43:25 +00:00
Commented May 10, 2017 at 15:43 — EdChum
– EdChum, Commented May 10, 2017 at 15:43

EdChum · Accepted Answer · 2017-05-11 08:15:47Z

3

You can pass a list of the column names of interest to subset, then compare with 0 and test for any col values for a row using any(axis=1):

In [9]:
df = pd.DataFrame({'a':[1,1,1,0],'b':[1,1,1,1],'c':[1,0,1,1]})
df

Out[9]:
   a  b  c
0  1  1  1
1  1  1  0
2  1  1  1
3  0  1  1

In [10]:
df[(df[['a','c']]==0).any(axis=1)]

Out[10]:
   a  b  c
1  1  1  0
3  0  1  1

For the case where you have a predefined list you don't need to enclose again using square brackets:

In [12]:
col_list = ['a','c']
df[(df[col_list]==0).any(axis=1)]

Out[12]:
   a  b  c
1  1  1  0
3  0  1  1

edited May 11, 2017 at 8:15

answered May 10, 2017 at 15:47

EdChum

397k204 gold badges836 silver badges583 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Richard Gray Over a year ago

Great answer, thanks. A problem I'm having is that I can't do it this way when the list values are not known in advance. E.g. columns_list = ['cooking', 'pets'] df[(df[[columns_list]]==0).any(axis=1)] gives me a TypeError: unhashable list. I need to pass a pre-populated list. Sorry for not being clearer in the question.

EdChum Over a year ago

See updated answer, basically if you already have a list then you don't need the additional square brackets

Collectives™ on Stack Overflow

How to subset a DataFrame, using loc(), based on multiple columns having a particular value with Python?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related