Convert unix time to readable date in pandas dataframe

Question

I have a dataframe with unix times and prices in it. I want to convert the index column so that it shows in human readable dates.

So for instance I have date as 1349633705 in the index column but I'd want it to show as 10/07/2012 (or at least 10/07/2012 18:15).

For some context, here is the code I'm working with and what I've tried already:

import json
import urllib2
from datetime import datetime
response = urllib2.urlopen('http://blockchain.info/charts/market-price?&format=json')
data = json.load(response)   
df = DataFrame(data['values'])
df.columns = ["date","price"]
#convert dates 
df.date = df.date.apply(lambda d: datetime.strptime(d, "%Y-%m-%d"))
df.index = df.date

As you can see I'm using df.date = df.date.apply(lambda d: datetime.strptime(d, "%Y-%m-%d")) here which doesn't work since I'm working with integers, not strings. I think I need to use datetime.date.fromtimestamp but I'm not quite sure how to apply this to the whole of df.date.

Thanks.

Jeff · Accepted Answer · 2013-10-07 18:25:59Z

397

These appear to be seconds since epoch.

In [20]: df = DataFrame(data['values'])

In [21]: df.columns = ["date","price"]

In [22]: df
Out[22]: 
<class 'pandas.core.frame.DataFrame'>
Int64Index: 358 entries, 0 to 357
Data columns (total 2 columns):
date     358  non-null values
price    358  non-null values
dtypes: float64(1), int64(1)

In [23]: df.head()
Out[23]: 
         date  price
0  1349720105  12.08
1  1349806505  12.35
2  1349892905  12.15
3  1349979305  12.19
4  1350065705  12.15
In [25]: df['date'] = pd.to_datetime(df['date'],unit='s')

In [26]: df.head()
Out[26]: 
                 date  price
0 2012-10-08 18:15:05  12.08
1 2012-10-09 18:15:05  12.35
2 2012-10-10 18:15:05  12.15
3 2012-10-11 18:15:05  12.19
4 2012-10-12 18:15:05  12.15

In [27]: df.dtypes
Out[27]: 
date     datetime64[ns]
price           float64
dtype: object

answered Oct 7, 2013 at 18:25

Jeff

130k21 gold badges223 silver badges189 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

CDspace · Accepted Answer · 2017-07-07 20:18:23Z

114

If you try using:

df[DATE_FIELD]=(pd.to_datetime(df[DATE_FIELD],***unit='s'***))

and receive an error :

"pandas.tslib.OutOfBoundsDatetime: cannot convert input with unit 's'"

This means the DATE_FIELD is not specified in seconds.

In my case, it was milli seconds - EPOCH time.

The conversion worked using below:

df[DATE_FIELD]=(pd.to_datetime(df[DATE_FIELD],unit='ms'))

edited Jul 7, 2017 at 20:18

CDspace

2,69919 gold badges32 silver badges39 bronze badges

answered Jul 7, 2017 at 17:54

Parachute

1,2781 gold badge9 silver badges13 bronze badges

Comments

Michael Dorner · Accepted Answer · 2018-09-17 09:01:43Z

44

Assuming we imported pandas as pd and df is our dataframe

pd.to_datetime(df['date'], unit='s')

works for me.

edited Sep 17, 2018 at 9:01

Michael Dorner

20.7k16 gold badges96 silver badges132 bronze badges

answered Mar 17, 2018 at 19:55

Anj Raju

6857 silver badges7 bronze badges

Comments

Carlos Ferreira · Accepted Answer · 2021-12-03 10:54:07Z

23

The Pandas Documentation gives this and other format examples and wasn't included in any of the above previous answers. Link: https://pandas.pydata.org/docs/reference/api/pandas.to_datetime.html

Code

pd.to_datetime(1490195805, unit='s')

Timestamp('2017-03-22 15:16:45')

pd.to_datetime(1490195805433502912, unit='ns')

Timestamp('2017-03-22 15:16:45.433502912')

answered Dec 3, 2021 at 10:54

Carlos Ferreira

2,1162 gold badges16 silver badges19 bronze badges

Comments

ipramusinto · Accepted Answer · 2019-03-18 06:21:18Z

2

Alternatively, by changing a line of the above code:

# df.date = df.date.apply(lambda d: datetime.strptime(d, "%Y-%m-%d"))
df.date = df.date.apply(lambda d: datetime.datetime.fromtimestamp(int(d)).strftime('%Y-%m-%d'))

It should also work.

answered Mar 18, 2019 at 6:21

ipramusinto

2,6884 gold badges17 silver badges25 bronze badges

Collectives™ on Stack Overflow

Convert unix time to readable date in pandas dataframe

5 Answers 5

Comments

Comments

Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related