pandas create new column after merge

Question

Thank you in advance for taking the time to help me

I have 2 dataframes: 1 - dataframe with notifications that i want to send to users

notifications_to_send = pd.DataFrame({'user_id': ['201', '207', '223', '212', '112', '311'],
                   'id_notification': ['1', '1', '1', '1', '1', '1'],
                     'email':['[email protected]','[email protected]','[email protected]','[email protected]','[email protected]','[email protected]']
                     })

   user_id   id_notification    email
0   201             1           [email protected]
1   207             1           [email protected]
2   223             1           [email protected]
3   212             1           [email protected]
4   112             1           [email protected]
5   311             1           [email protected]

2 - dataframe with info about users who set option to don't limit the quantity of sending notification to them

unlimited_notifications = pd.DataFrame({'user_id': ['201', '212'],
                   'id_notification': ['1', '2']})

  user_id   id_notification
0   201           1
1   212           2

so I want to add column 'send_push' to first dataframe, that will determine that push will be sent if user set unlimited option I thought about merge 2 dataframes on user_id and id_notification

notifications_to_send.merge(unlimited_notifications, on=['user_id','id_notification'], how='left')

, and for row which fits with this condition 'send_push' will be set on True, if condition don't execute then set it on Null, so it would look like this:

  user_id    id_notification    email             send_push
0   201             1           [email protected]    True
1   207             1           [email protected]    null
2   223             1           [email protected]    null
3   212             1           [email protected]    null
4   112             1           [email protected]    null
5   311             1           [email protected]    null

so the questions is how to add new column while merge or maybe there is any other way to make it? Compare two dataframes?

Not sure I'm following - does the unlimited_notifications dataframe contains only the users that you want the send_push column to be True? Does the id_notification column on that dataframe have any meaning? — jfaccioni
– jfaccioni, Commented Aug 10, 2021 at 14:42
@jfaccioni I have different notifications, so user can choose which one of them he will always get, if he didn't set unlimited option, for example, notification with id = 1 he will get only once a week, but if he would set this option, i would send this notification 5 times a week — Bimbo
– Bimbo, Commented Aug 10, 2021 at 15:26
Hi! Is any one of the answers below working? If so & if you wish, you might consider accepting one of them to signal others that the issue is resolved. If not, you can provide feedback so they can be improved (or removed altogether) — Anurag Dabas
– Anurag Dabas, Commented Aug 11, 2021 at 10:18

Anurag Dabas · Accepted Answer · 2021-08-10 14:54:20Z

3

Perform left merge then compare the 'id_notification' of both df's and finally assign values according to that:

import numpy as np

notifications_to_send=notifications_to_send.merge(unlimited_notifications,how='left',on='user_id',suffixes=('','_y'))
notifications_to_send['send_push']=np.where(notifications_to_send.eval("id_notification==id_notification_y"),True,pd.NA)
notifications_to_send=notifications_to_send.drop(columns='id_notification_y')

OR

without np.where():

notifications_to_send=notifications_to_send.merge(unlimited_notifications,how='left',on='user_id',suffixes=('','_y'))
m=notifications_to_send['id_notification'].eq(notifications_to_send.pop('id_notification_y'))
notifications_to_send.loc[m,'send_push']=True
notifications_to_send.loc[~m,'send_push']=pd.NA

edited Aug 10, 2021 at 14:54

answered Aug 10, 2021 at 14:47

Anurag Dabas

24.3k9 gold badges25 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Hanalia · Accepted Answer · 2021-08-10 14:54:45Z

1

You can just add send_push column to the 2nd dataframe before merging.

When you merge, only the rows where userid and id_notification match the first dataframe will return True.

answered Aug 10, 2021 at 14:54

Hanalia

1971 silver badge9 bronze badges

Collectives™ on Stack Overflow

pandas create new column after merge

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related