Using the condition, select the desired columns in pandas DataFrame

Question

I have a DataFrame I created using pandas and want to create new table based on the original, but filtered based on certain conditions.

df = pd.DataFrame(
    [['Y', 'Cat', 'no', 'yes', 6],
    ['Y', 4, 7, 9, 'dog'],
    ['N', 6, 4, 6, 'pig'],
    ['N', 3, 6, 'beer', 8]],
    columns = ('Data', 'a', 'b', 'c', 'd')
)

My condition that doesnt work:

if (df['Data']=='Y') & (df['Data']=='N'):
    df3=df.loc[:,['Data', 'a', 'b', 'c']]
else:
    df3=df.loc[:,['Data', 'a', 'b']]

I want the new table to contain data that matches the following criteria:

If df.Data has value 'Y' and 'N', the new table get columns ('Data', 'a', 'b')

If not, the new table gets columns ('Data', 'a', 'b', 'c')

 Data    a   b
0    Y  Cat  no
1    Y    4   7
2    N    6   4
3    N    3   6

  Data    a   b     c
0    Y  Cat  no   yes
1    Y    4   7     9
2    Y    6   4     6
3    Y    3   6  beer

Your first output has Y and N but no column c while second output has only Y but it has column c. That differs from your verbal description. — harpan
– harpan, Commented Jun 6, 2018 at 15:28

jpp · Accepted Answer · 2018-06-06 15:33:11Z

1

You are comparing a series with a character rather than checking existence for a single Boolean result. You can, instead, use pd.Series.any which returns True if any value in a series is True:

if (df['Data']=='Y').any() & (df['Data']=='N').any():
    # do something

An alternative method is to use pd.DataFrame.drop with a ternary statement:

df = df.drop(['d'] if set(df['Data']) == {'Y', 'N'} else ['c', 'd'], 1)

print(df)

  Data    a   b     c
0    Y  Cat  no   yes
1    Y    4   7     9
2    N    6   4     6
3    N    3   6  beer

edited Jun 6, 2018 at 15:33

answered Jun 6, 2018 at 15:27

jpp

166k37 gold badges301 silver badges363 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

nimrodz · Accepted Answer · 2018-06-06 15:39:39Z

0

if all(df.Data.unique() == ['Y','N']) == True:
    df3 = df[['Data', 'a', 'b', 'c']]
else:
    df3 = df[['Data','a','b']]

edited Jun 6, 2018 at 15:39

answered Jun 6, 2018 at 15:32

nimrodz

1,6091 gold badge15 silver badges19 bronze badges

1 Comment

jpp Over a year ago

While this may work, worth noting this is more lax. For example, 'Y' and 'O' would also satisfy the condition. You can also use pd.Series.nunique for this method.

Collectives™ on Stack Overflow

Using the condition, select the desired columns in pandas DataFrame

2 Answers 2

Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related