How to convert String to Numpy Datetime64 from a dataframe

Question

I'm dealing with a DataFrame that contains column of time like "hh:mm:ss" and I need to convert those values to the NumPy datetime64 type.

import pandas as pd
data = [dict(voie="V3", Start="06:10", End='06:20'),
            dict(voie="V5", Start='06:26', End='06:29'),
            dict(voie="V3", Start='06:20', End='06:30'),
            dict(voie="V5", Start='06:32', End='06:35')]
df = pd.DataFrame(data)
#df = df['Start'].to-datetime64()

I need to convert the column Start and End from type string to datetime64

Grr · Accepted Answer · 2019-05-16 11:58:26Z

2

Just use pandas.to_datetime for each column. For example:

df.End = pd.to_datetime(df.End)
df.End
0   2019-05-15 06:20:00
1   2019-05-15 06:29:00
2   2019-05-15 06:30:00
3   2019-05-15 06:35:00
Name: End, dtype: datetime64[ns]

You can also use the pandas.DataFrame.astype method of the DataFrame.

df.End = df.End.astype('datetime64[ns]')
df.End
0   2019-05-15 06:20:00
1   2019-05-15 06:29:00
2   2019-05-15 06:30:00
3   2019-05-15 06:35:00
Name: End, dtype: datetime64[ns]

Regarding `pd.Timestamp` and `np.datetime64`

That is a complicated relationship. The .values attribute of the series will be an array of type np.datetime64, while the type of a single entry will be pd.Timestamp. As far as I know there is nothing you can do with np.datetime64 that you can't with pd.Timestamp. There is a nice little graphic in Converting between datetime, Timestamp and datetime64 that might help. Deep down within the pd.to_datetime code you will see that in fact when passed a pd.Series each entry is converted to np.datetime64. It isn't until you access an item in the series that it is converted into a pd.Timestamp (see pandas._libs.index.get_value_at).

edited May 16, 2019 at 11:58

answered May 15, 2019 at 15:23

Grr

16.2k7 gold badges72 silver badges91 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Mohamed Over a year ago

It only changes the the type from String to Timestamp.

Grr Over a year ago

@Mohammed see my edits. I hope that helps make it more clear. The Timestamp you see is only a top layer that pandas uses when you are accessing elements. The actual data itself is of datetime64 type.

Collectives™ on Stack Overflow

How to convert String to Numpy Datetime64 from a dataframe

1 Answer 1

Regarding `pd.Timestamp` and `np.datetime64`

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Regarding pd.Timestamp and np.datetime64

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related

Regarding `pd.Timestamp` and `np.datetime64`