how to merge two data frame with out adding duplicates in Python?

Question

I have a dataframe looks like:

    x   y   z
1   3   7   1
2   7   5   2
3   8   2   3

and other dataframe looks like:

    x   y   z
1   7   5   4
2   8   2   8
3   4   5   5
4   7   3   7
5   9   5   3
6   3   7   9

Now I want to merge the two frames, and the frame after merge to be looked like this:

    x   y   z
1   3   7   1
2   7   5   2
3   8   2   3
4   4   5   5
5   7   3   7
6   9   5   3

jezrael · Accepted Answer · 2017-06-08 13:25:01Z

4

Use concat + drop_duplicates with define columns for check duplicates x and y + reset_index for remove duplicated index values:

df = pd.concat([df1,df2]).drop_duplicates(['x','y']).reset_index(drop=True)
print (df)
   x  y  z
0  3  7  1
1  7  5  2
2  8  2  3
3  4  5  5
4  7  3  7
5  9  5  3

edited Jun 8, 2017 at 13:25

answered Jun 8, 2017 at 13:14

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

bikuser Over a year ago

Thank you @jezrael, that is what i was wanted.

ILostMySpoon · Accepted Answer · 2017-06-08 13:14:03Z

2

Concatenate, then dedupe

pandas.concat([df1,df2]).drop_duplicates().reset_index(drop=True)

answered Jun 8, 2017 at 13:14

ILostMySpoon

2,3992 gold badges21 silver badges26 bronze badges

1 Comment

bikuser Over a year ago

yes it add all value to the dataframe did not remove duplicates on x and y

Collectives™ on Stack Overflow

how to merge two data frame with out adding duplicates in Python?

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related