5

I am trying to convert a column of timestamps (YYYY-MM-DD HH-MM-SS) from a pandas dataframe to seconds.

Here is my current code:

df['recorded_time'] = pd.to_datetime(df['recorded_time'])
df['timestamp'] = df['recorded_time'].datetime.total_seconds() #creating a new column

The error I keep getting is:

AttributeError: 'Series' object has no attribute 'datetime'

Can anyone point out where I may be missing a step?

3
  • 1
    Because it doesn't. to_datetime will return a datetime64 value that doesn't have the same methods/attributes of a regular python datetime. You'll need to use the .dt accessor, so something like df['timestamp'] = df['recorded_time'].dt. but then total_seconds() is a datetime.timedelta method from python, so I don't really follow what you expect that to be doing, even if we translated it to pandas. Seconds from when? Commented Nov 26, 2019 at 19:32
  • @roganjosh 2019/01/01 00:00:00 would work... I was trying to follow this: stackoverflow.com/questions/56430069/… However, I was still getting the same attribute error. Commented Nov 26, 2019 at 19:36
  • What roganjosh is trying to say is that total seconds is a timedelta, this means it takes the total seconds between two timepoints. In this case you just have a timestamp (one time point). So it does not make sense to take the "total seconds". Total seconds from when or what? Commented Nov 26, 2019 at 19:47

2 Answers 2

10

I'm using the example you gave in a comment for the df. You cannot use regular datetime.datetime methods on pandas datetime64 values without using the .dt accessor. In addition to the example you linked to, you said that you want total_seconds to refer to the base datetime of 2019/01/01 00:00:00. A timedelta must always have some kind of reference point, otherwise it could be any arbitrary value.

import pandas as pd

df1 = pd.DataFrame({'lat':[15.13,12.14,13.215,11.1214,12.14], 
              'lon': [38.52, 37.536,39.86,38.536,37.536],
              'Datetime': pd.to_datetime(['2019-03-09 16:05:07',
                                          '2019-03-15 09:50:07',
                                          '2019-03-09 11:03:47',
                                          '2019-03-10 16:41:18',
                                          '2019-03-09 06:15:27']),
              'temp':[23,22,21,22,19]})

# We can just call it once to get a reference to your datum point
base_dt = pd.to_datetime('2019/01/01 00:00:00')

df1['seconds'] = (df1['Datetime'] - base_dt).dt.total_seconds()
Sign up to request clarification or add additional context in comments.

Comments

2

One method to convert datetime into numbers is to view it as 'int64':

df['ns_since_UNIX_epoch'] = df['recorded_time'].view('int64')

# this is equivalent to 
(df['recorded_time'] - pd.Timestamp('1970-01-01')).dt.total_seconds() * 10**9

In general, to access datetime methods, use the .dt accessor on a pandas column; and to access string methods, use the .str accessor.

To see a list of all attributes of a pandas column or Series, call dir(). You can check the following:

'datetime' in dir(pd.Series)          # False
'dt' in dir(pd.Series)                # True
'total_seconds' in dir(pd.Series.dt)  # True

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.