1

If I have data something like this, a missing value in 'Date4' column, its a datetime64[ns] dtype.

  • How to handle missing values in this type of situation?
  • What if I want to fill it with most_frequent date, how can it be done for the dates?

I have searched for the solution on several websites but couldn't get proper answer yet.

   No  Name      Date1      Date2      Date3      Date4
0   1  Per1 2015-05-25 2016-03-20 2016-03-22 2017-01-01
1   2  Per2 2015-06-26 2016-05-22 2016-06-22 2017-02-02
2   3  Per3 2015-09-28 2016-07-24 2016-07-26 2017-05-22
3   4  Per4 2015-11-21 2016-09-02 2016-05-09 2017-05-22
4   5  Per5 2015-12-25 2016-11-11 2016-11-14        NaT
Improve this question
1
  • 1
    Don't post the image of data, post the sample data as text Commented Jul 29, 2020 at 16:50

2 Answers 2

Reset to default
2 
In [135]: df
Out[135]:
       Date4
0 2017-01-01
1 2017-02-02
2 2017-05-22
3 2017-05-22
4        NaT

In [136]: df["Date4"].replace(np.nan, df["Date4"].mode().iloc[0])
Out[136]:
0   2017-01-01
1   2017-02-02
2   2017-05-22
3   2017-05-22
4   2017-05-22
Name: Date4, dtype: datetime64[ns]
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thank you so much. I was trying with fillna method : df['Date4'] = df['Date4'].fillna(df["Date4"].mode()) like this. But I didn't use .iloc[0], that was the mistake and I got the solution from your code. Now fillna is also working like this : df['Date4'] = df['Date4'].fillna(df["Date4"].mode().iloc[0]) . Thanks a lot.
Can you suggest what other methods can we use to fill missing values in datetime[ns] datatype?
There are several methods like bfill, ffill but it mostly depends on the data you are dealing with
1

What you just described is called Imputation. Sklearn's SimpleImputer() does the job well. You can even specify how you want the missing values to be filled.

imp=SimpleImputer(missing_values=np.nan, strategy = 'most_frequent')
df=pd.DataFrame(imp.fit_transform(df))
Improve this answer

1 Comment

SimpleImputer does not support datetime64[ns] data type. Otherwise mostly I use SimpleImputer only.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.