Why is this datetime string not converting to a datetime object in a Pandas dataframe?

Question

I am attempting to convert a column in a dataframe from an object to a datetime64[ns]. I am using to_datetime to accomplish this, yet at the end of my code, that column is still an object.

import pandas as pd
from StringIO import StringIO

DATA = StringIO("""id;Date of Event
3574;2015-12-12 22:03:28Z
0657;2015-08-25 17:48:03Z
0408;2015-10-13 12:01:32Z
3043;2015-09-08 16:55:43Z
9397;2015-09-09 09:33:31Z
9291;2015-07-15 08:13:48Z
4263;2015-12-30 09:25:55Z
0200;2015-10-25 13:38:35Z
8576;2015-09-01 02:01:47Z
6023;2015-08-29 20:47:20Z
9975;2015-10-05 15:11:32Z
5202;2015-12-21 23:44:10Z
9278;2015-12-22 05:56:03Z
8520;2015-09-05 01:27:07Z
9048;2015-08-29 18:38:26Z
9624;2015-12-09 01:49:15Z
2659;2015-10-03 01:43:50Z
6230;2015-10-16 11:43:40Z
2272;2015-11-18 14:15:52Z
""")

df = pd.DataFrame.from_csv(DATA, sep=";")
pd.to_datetime(df['Date of Event'], format="%Y-%m-%d %H:%M:%SZ")
print df['Date of Event'].dtype

That final print shows:

object

df.info() returns this:

Int64Index: 19 entries, 3574 to 2272
Data columns (total 1 columns):
Date of Event    19 non-null object
dtypes: object(1)
memory usage: 304.0+ bytes

Why did my pd.to_datetime(df['Date of Event'], format="%Y-%m-%d %H:%M:%SZ") fail to convert the column to datetime objects and how can I correct it?

The format is valid, and I can utilize the datetime library to test that:

>>> import datetime
>>> s = "2015-11-18 14:15:52Z"
>>> dt = datetime.datetime.strptime(s, "%Y-%m-%d %H:%M:%SZ")
>>> dt
datetime.datetime(2015, 11, 18, 14, 15, 52)

Why did the conversion fail on the entire Pandas column?

you have not assigned pd.to_datetime(df['Date of Event'], format="%Y-%m-%d %H:%M:%SZ") to anything — The6thSense
– The6thSense, Commented Jan 14, 2016 at 3:54

DSM · Accepted Answer · 2016-01-14 03:55:50Z

7

to_datetime returns a new result, it doesn't modify its argument in place. Reassign it:

>>> df['Date of Event'] = pd.to_datetime(df['Date of Event'], format="%Y-%m-%d %H:%M:%SZ")
>>> df.dtypes
Date of Event    datetime64[ns] 
dtype: object

Or use parse_dates and have it converted at the start (note that it's more common to use read_csv than pd.DataFrame.from_csv):

>>> df = pd.read_csv(DATA, sep=";", parse_dates=["Date of Event"])
>>> df.dtypes
id                        int64
Date of Event    datetime64[ns]
dtype: object

answered Jan 14, 2016 at 3:55

DSM

355k67 gold badges606 silver badges504 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

DSM Over a year ago

@The6thSense: when necessary, you can pass a date_parser function, although I tend to use to_datetime directly when I need that.

Collectives™ on Stack Overflow

Why is this datetime string not converting to a datetime object in a Pandas dataframe?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related