Check duplicate values in some of columns in dataframe

Question

I have a dataframe which is organized similar to this

I am trying to make a new column which details if the GUID matches and the 'DL', 'DRI', 'DS' columns match. So from the image above the new column would have picked up that this data is a match.

I've tried:

cols = {['DL','DRI','DS']}
df['match'] = df[cols].eq(df.col1.shift())

But am getting 'TypeError: unhashable type: 'list''

How do you plan to let a column contain information about multiple rows? — gosuto
– gosuto, Commented Nov 29, 2019 at 6:28
Its more like it checks the row above because earlier in the script I've sorted the set by column A — cd-6
– cd-6, Commented Nov 29, 2019 at 6:29

gbruenjes · Accepted Answer · 2019-11-29 06:44:58Z

1

Maybe this is what you want. Maybe you should rephrase your question because it isn't clear what you actually want.

df = pd.DataFrame({'GUID':['00059','00059','123'], 'DL':['','','123'], 'DRI':[True,True,True], 'DS':['','','123'], 'Model':['asd','qwe','123']})
df

    GUID   DL   DRI   DS Model
0  00059       True        asd
1  00059       True        qwe
2    123  123  True  123   123

df['match'] = df.duplicated(['GUID', 'DL', 'DRI', 'DS'], keep=False)
df

    GUID   DL   DRI   DS Model  match
0  00059       True        asd   True
1  00059       True        qwe   True
2    123  123  True  123   123  False

answered Nov 29, 2019 at 6:44

gbruenjes

4,2251 gold badge18 silver badges32 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

cd-6 Over a year ago

yep that worked luigigi, thanks a lot. I made it work by combining the three column values in a new column then removing where they matched but this is much better.

oppressionslayer · Accepted Answer · 2019-11-29 06:34:42Z

1

cols = {['DL','DRI','DS']}

is in the wrong format, it should be:

cols = [['DL','DRI','DS']]

answered Nov 29, 2019 at 6:34

oppressionslayer

7,2242 gold badges11 silver badges26 bronze badges

1 Comment

cd-6 Over a year ago

didn't seem to do the trick, still coming out as false with the change

Collectives™ on Stack Overflow

Check duplicate values in some of columns in dataframe

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related