python pandas TypeError: Cannot compare type 'Timestamp' with type 'float'

Question

I have a pandas data frame, df_data, want to use the pandas index.asof() method to find the nearest row to a specified time. The times I have are in seconds (type = float64) (see below).

Originally, the index was made to be a DateTimeIndex:

In [12]: df_data.index = pd.to_datetime(df_data.index, coerce=True) 
         df_data.index.dtype
Out[12]: dtype('<M8[ns]')

Then, I changed the index to be in seconds from the initial time:

In [22]: ## Convert the index from DateTimeIndex to a float64 
         ## that is the number of seconds from the initial measurement
         df_data.index = (df_data.index - df_data.index[0])/np.timedelta64(1,'s')
In [23]: df_data.index.dtype
Out[23]: dtype('float64')

But when I try to use the asof method with a float, I get a TypeError:

In [24]: df_data.index.asof(10.0)
...
TypeError: Cannot compare type 'Timestamp' with type 'float'

I have tried to use datetime, datetime.fromtimestamp, etc., but haven't been able to resolve the issue.

Although this is not stated clearly in the docs, I think asof only works with a DatetimeIndex, and not with other index types. — joris
– joris, Commented Nov 13, 2014 at 22:45
@joris: I think you are right about the asof being limited to DateTimeIndex. I have edited my code to use the method (and save the results) before changing my index from DateTime to float. I have edited the question above. — dhltp
– dhltp, Commented Nov 14, 2014 at 1:09

dhltp · Accepted Answer · 2014-11-14 05:07:06Z

Thanks to @joris for the insightful comment.

SOLUTION

Before changing the index from DateTimeIndex to a float (i.e., seconds from initial measurement as described in the question), you need to identify the times (in this case I use a simple example with one time time_float) at which you'd like to find the nearest index. Then, those datetime indices can be transformed to float indices:

In [21]: time_float = 10.0
         time_td = df_data.index[0]+ datetime.timedelta(0,time_float)
         ## convert from the DateTimeIndex type to time from start in seconds as type float64
         time_index = (df_data.index.asof(time_td) - df_data.index[0]).total_seconds()
         time_index
Out[21]: 9.86296

Now, after the overall conversion of the index (given above) to seconds from the initial time, I can refer to the index closest to time_float, which is time_index:

In [24]: df_data.ix[time_index,0]
Out[24]: 0.00075450129999999997

Collectives™ on Stack Overflow

python pandas TypeError: Cannot compare type 'Timestamp' with type 'float'

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related