Pandas Convert Timestamp Column to Datetime

Question

Given the following data frame and necessary wrangling:

import pandas as pd
df=pd.DataFrame({'A':['a','b','c'],
        'dates':['2015-08-31 00:00:00','2015-08-24 00:00:00','2015-08-25 00:00:00']})
df.dates=df.dates.astype(str)
df['dates'] = pd.to_datetime(df.dates.str.split(',\s*').str[0])
set(df['dates'])

I end up with:

{Timestamp('2015-08-24 00:00:00'),
 Timestamp('2015-08-25 00:00:00'),
 Timestamp('2015-08-31 00:00:00')}

I need to convert the time stamps back to datetime (really, just date) format.

I've tried this based on the answer to this post:

df['dates'].to_pydatetime()

But that returns:

AttributeError: 'Series' object has no attribute 'to_pydatetime'

In my real data, the data type is: <M8[ns]

Possible duplicate of Converting between datetime and Pandas Timestamp objects — Nickil Maveli
– Nickil Maveli, Commented Jun 5, 2016 at 16:46
Please see edited post. I tried the method suggested in that post but it did not work for me. — Dance Party
– Dance Party, Commented Jun 5, 2016 at 17:21

EdChum · Accepted Answer · 2016-06-05 20:43:38Z

9

You can use dt.date to return a datetime.date object:

In [3]:
set(df['dates'].dt.date)

Out[3]:
{datetime.date(2015, 8, 24),
 datetime.date(2015, 8, 25),
 datetime.date(2015, 8, 31)}

answered Jun 5, 2016 at 20:43

EdChum

397k204 gold badges836 silver badges583 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Steven G Over a year ago

This extract only the date from the timestamp, not the entire datetime.

turbonate Over a year ago

set type is unordered. if trying to apply back to the dataframe then @RexBarker's answer using apply with lambda is the right answer.

RexBarker · Accepted Answer · 2020-03-08 08:23:15Z

9

You can convert directly using apply:

df.dates = df.dates.apply(lambda x: x.date())

This makes an in-place conversion of the previous 'dates' (as a timestamp) to the truncated 'datetime' only portion

answered Mar 8, 2020 at 8:23

RexBarker

1,89922 silver badges16 bronze badges

Comments

M.abr · Accepted Answer · 2019-08-17 16:26:51Z

2

I have a similar issue where I need to convert timestamp to datetime in numpy though, but I believe it can be apply in Pandas as well. I think using function under Pandas.Timestamp would be better to convert timestamp as below.

==============================

np1=pd.DataFrame.to_numpy(df2)
print(np1)
[[Timestamp('2019-01-31 00:00:00') 'UCHITEC'  2000 2.56 5129.54]
 [Timestamp('2019-01-16 00:00:00') 'UCHITEC'  1000 2.61 2618.79]]

np2= np.asarray(np1)
Timestamp('2019-01-16 00:00:00')

np3 = pd.Timestamp.to_datetime64(np2[0][0])
np4 = pd.Timestamp.to_pydatetime(np2[1][0])

print(np3)
print(np4)

2019-01-31T00:00:00.000000000
2019-01-16 00:00:00

edited Aug 17, 2019 at 16:26

answered Aug 17, 2019 at 16:21

M.abr

613 bronze badges

Comments

jpobst · Accepted Answer · 2022-08-07 16:56:07Z

2

You can convert a whole Timestamp column to an array of datetime.datetime objects like this:

dt_array = df['dates'].dt.to_pydatetime()
# dt_array is an array of datetime.datetime objects

BUT as soon as you try to use that array to create/overwrite a pandas column it will end up as a dtype='datetime64[ns]'). For instance, columns A and B below will be dtype='datetime64[ns]').

dt_array = df['dates'].dt.to_pydatetime()
# dt_array is an array of datetime.datetime objects
df['A'] = dt_array

# or in one line
df['B'] = df['dates'].dt.to_pydatetime()

edited Aug 7, 2022 at 16:56

answered Aug 7, 2022 at 16:23

jpobst

3,7413 gold badges27 silver badges25 bronze badges

Comments

Nickil Maveli · Accepted Answer · 2016-06-05 18:32:57Z

1

If you are keen on extracting only the date from a given Timestamp object, you can get the raw datetime.date objects by calling the unbound Timestamp.date method as shown:

import pandas as pd
from pandas import Timestamp, Series, date_range

start = Timestamp('2016-01-01')
stop = Timestamp('2016-01-02')
s = Series(date_range(start, stop, freq = 'H'))

print s[0]
2016-01-01 00:00:00

print s.map(Timestamp.date)[0]
2016-01-01
dtype: object

answered Jun 5, 2016 at 18:32

Nickil Maveli

29.8k10 gold badges86 silver badges88 bronze badges

Comments

taveltjuven · Accepted Answer · 2022-02-03 23:36:46Z

1

this worked for me. df['time'] consist of a column of timestamps

df['time'] = df['time'].apply(lambda x: datetime.datetime.fromtimestamp(x).strftime('%Y-%m-%d'))

# i.e. x is a timestamp such as 1641772800 (or in date 2022-01-10)

edited Feb 3, 2022 at 23:36

answered Feb 3, 2022 at 23:30

taveltjuven

112 bronze badges

Collectives™ on Stack Overflow

Pandas Convert Timestamp Column to Datetime

6 Answers 6

2 Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

2 Comments

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related