fill null values in a column of pandas dataframe

Question

I have a pandas dataframe which has more than 4 columns. Some values in the col1 are missing and I want to set those missing values based on the following approach:

try to set it based on the average of values of col1 of the records that have the same col2,col3,col4 values
if there is no such record, set it based on the average of values of col1 of the records that have the same col2,col3 values
if there is still no such record, set it based on the average of values of col1 of the records that have the same col2 values
If none of the above could be found, set it to the average of all other non-missing values in col1

What's the best way to do this?

Without some data, not easy to help. Please take a moment to read about how to post pandas questions: stackoverflow.com/questions/20109391/… — YOLO
– YOLO, Commented Jul 29, 2020 at 20:34

Quang Hoang · Accepted Answer · 2020-07-29 20:40:42Z

2

Based on your logic, you can do something as follows, where each row of fillna corresponds to a bullet point in your question, in the same order:

df['col1'] = (df['col1']
               .fillna(df.groupby(['col2','col3','col4'])['col1'].transform('mean'))
               .fillna(df.groupby(['col2','col3'])['col1'].transform('mean'))
               .fillna(df.groupby(['col2'])['col1'].transform('mean')
               .fillna(df['col1'].mean())
             )

answered Jul 29, 2020 at 20:40

Quang Hoang

151k11 gold badges64 silver badges86 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Dinesh vishe · Accepted Answer · 2024-04-14 15:58:35Z

0

--- filling null null value with zero

df_with_dummies.fillna(value = 0, inplace = True)

answered Apr 14, 2024 at 15:58

Dinesh vishe

3,6581 gold badge27 silver badges26 bronze badges

Collectives™ on Stack Overflow

fill null values in a column of pandas dataframe

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related