created a new column in a pandas dataframe based on multiple different conditions

Question

I found a similar question here, but it did not help me because my case is different.

I have a huge dataframe that looks more or less like the example below:

        x   y   st  mt  ast sr  c7  z   w
    0   mt      2   1   4   2   2   a   yes
    1       b   3   3   3   3   3       yes
    2           1   1   2   4   3       yes
    3       d   3   3   1   2   4   d   
    4       e   2   3   2   1   4       
    5   nlp     5   5   5   5   5   f   yes

my goal is to create another column named "other_flagged" based on the following condition:

'if columns x, y and z are NOT filled with a string, but w has a "yes", fill column "other_flagged" with the string 'flagged'. So, my desired output would be:

        x   y   st  mt  ast sr  c7  z   w   other flagged
    0   mt      2   1   4   2   2   a   yes     
    1       b   3   3   3   3   3       yes    
    2           1   1   2   4   3       yes     flagged
    3       d   3   3   1   2   4   d   
    4       e   2   3   2   1   4       
    5   nlp     5   5   5   5   5   f   yes

in this case, only row 2 receives the string "flagged" because x, y and z are empty but column w has the string 'yes'. I tried the code below but the new column created is being completely populated with the string 'flagged' but not based on the conditions:

 survey.loc[(survey['x'] != '') & (survey['y'] !='') & (survey['z'] != '') & 
(survey['w'] == 'yes'), 'other_flagged'] ='flagged'

also tried:

survey.loc[(survey['x'] != '[a-zA-Z0-9-]+') & (survey['y'] != '[a-zA-Z0-9-]+') & 
(survey['z'] != '[a-zA-Z0-9-]+') & (survey['w'] == 'yes'), 'other_flagged']='flagged'

How can I achieve this goal?

jezrael · Accepted Answer · 2022-05-19 11:41:51Z

1

Use if not exist values are empty strings is possible test all columns together use DataFrame.all:

m = (survey[['x', 'y', 'z']] == '').all(axis=1) & (survey['w'] == 'yes')
survey.loc[m, 'other_flagged'] ='flagged'

print (survey)
     x  y  st  mt  ast  sr  c7  z    w other_flagged
0    m  t   2   1    4   2   2  a  yes           NaN
1       b   3   3    3   3   3     yes           NaN
2           1   1    2   4   3     yes       flagged
3       d   3   3    1   2   4  d                NaN
4       e   2   3    2   1   4                   NaN
5  nlp      5   5    5   5   5  f  yes           NaN

Your solution - change != to ==:

m = (survey['x'] == '') & (survey['y'] =='') & (survey['z'] == '') & (survey['w'] == 'yes')
survey.loc[m, 'other_flagged'] ='flagged'

If values are missing values:

m = survey[['x', 'y', 'z']].isna().all(1) & (survey['w'] == 'yes')
survey.loc[m, 'other_flagged'] ='flagged'

print (survey)
     x    y  st  mt  ast  sr  c7    z    w other_flagged
0    m    t   2   1    4   2   2    a  yes           NaN
1  NaN    b   3   3    3   3   3  NaN  yes           NaN
2  NaN  NaN   1   1    2   4   3  NaN  yes       flagged
3  NaN    d   3   3    1   2   4    d  NaN           NaN
4  NaN    e   2   3    2   1   4  NaN  NaN           NaN
5  nlp  NaN   5   5    5   5   5    f  yes           NaN

edited May 19, 2022 at 11:41

answered May 19, 2022 at 11:38

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Natália Resende Over a year ago

I'll try this. What does the .all(1) do?

Mayank Porwal · Accepted Answer · 2022-05-19 11:36:31Z

0

Use numpy.where:

import numpy as np

survey['other flagged'] = np.where((survey['x'].eq('') & survey['y'].eq('') & survey['z'].eq('') & survey['w'].eq('yes')) , 'flagged', 'not flagged')

answered May 19, 2022 at 11:36

Mayank Porwal

34.2k9 gold badges45 silver badges65 bronze badges

Collectives™ on Stack Overflow

created a new column in a pandas dataframe based on multiple different conditions

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related