how can i find a not timestamp data in dataframe python?

Question

Here is the timestamp column:

TIME
2018-03-02 11:57:37
2018-03-12 10:36:16
2018-03-29 12:02:21
2018-03-23 16:37:08
2018-03-09 22:22:28
         .
         .

And I tried merge and faced the following error.

TypeError: '<' not supported between instances of 'datetime.datetime' and 'int'

So I tried to find a column with a different data type like under code, but it didn't worked.

for i in range(len(ds)):
    if type(ds['TIME'].loc[i]) != type(ds['TIME'].loc[1]) : 
        ds = ds.drop(i)
# type(ds['TIME'].loc[1]) was confirmed that it was a timestamp type

how can i solve this problem?

I would be grateful if you give me some good advice.

this is what i try.

raw_data = pd.merge_ordered(ds,ds2)
#ds2 is similar data like ds

+I think it might be a parse problem in db.

jezrael · Accepted Answer · 2019-02-11 09:19:50Z

2

I believe you need create default index for prevent duplicated by reset_index and then call drop_duplicates:

ds['TIME'].reset_index(drop=True).drop_duplicates()

If possible multiple columns:

ds.reset_index(drop=True).drop_duplicates(subset=['TIME'])

edited Feb 11, 2019 at 9:19

answered Feb 11, 2019 at 8:10

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

송준석 Over a year ago

Thank you for giving me a good comparison. Still, the errors that appear above are not resolved. Is there a way to resolve this error?

송준석 Over a year ago

i also didn't worked. and i check all data type using type(), all data is same(<class 'pandas._libs.tslibs.timestamps.Timestamp'>). it so hard to solve it ㅜㅜ

송준석 Over a year ago

in my real data it has not duplicate date(check ds.drop_duplicate()). TypeError: '<' not supported between instances of 'datetime.datetime' and 'int' <--Does this error occur when the same data exists?

jezrael Over a year ago

@송준석 - Do youi think ds['TIME'].drop_duplicates() ? If failed, try ds['TIME'].reset_index(drop=True).drop_duplicates()

송준석 Over a year ago

i think it didn't worked. if you give your email, i sen real data. Looking at the actual data, the problem seems to be clearer.

|

Madriot · Accepted Answer · 2019-02-11 08:41:52Z

1

Try to use built-in type() function. I don't know if this is a correct way, but is simple:

if 'datetime' in str(type(ds)):
    print('Is a datetime format')

answered Feb 11, 2019 at 8:41

Madriot

765 bronze badges

1 Comment

송준석 Over a year ago

in case of my data, str(type(ds)) 's result is "<class 'pandas._libs.tslibs.timestamps.Timestamp'>" so i use upper code using 'timestamps' but it also didn't worked

Collectives™ on Stack Overflow

how can i find a not timestamp data in dataframe python?

2 Answers 2

8 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

8 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related