Python convert multiple columns to datetime using for loop

Question

I'm stuck on this and looking for community help. I have a few columns within a dataframe that are labeled as objects but would like to convert to datetime. All columns have year,month,day criteria.

time = np.array(['time1','time2','time3'])

def cols_to_datetime(df):

   cols_to_datetime = time

   for col in cols_to_datetime:
       df[col] = pd.to_datetime(df[col])

   return df

Although all have year, month, day i'm receiving this error

ValueError: to assemble mappings requires at least that [year, month, day] be specified: [day,month,year] is missing

df -

time1                      time2                      time3 
2020-06-06 20:01:10.327    2020-06-06 22:08:14.832    2020-06-06

There may be nulls in the df that would need to be skipped.

I did test without them and didn't have luck. So I'm not quite sure why this approach doesn't work. Thanks!

beware using things like time for variable names... plus it will be data specific — Rob Raymond
– Rob Raymond, Commented Feb 13, 2021 at 20:58

chvolkmann · Accepted Answer · 2021-02-13 21:19:56Z

2

You can use DataFrame.astype().

import pandas as pd

def convert_times(df, cols=None):
    if not cols:
        # if no columns are specified, use all
        cols = df.columns
    df[cols] = df[cols].astype('datetime64')
    return df

df = pd.DataFrame({
    'created_at': ['2020-06-06 20:01:10.327'],
    'updated_at': ['2020-06-06 22:08:14.832']
})
print(df.dtypes)
# created_at    object
# updated_at    object
# dtype: object

df2 = convert_times(df)
print(df2.dtypes)
# created_at    datetime64[ns]
# updated_at    datetime64[ns]
# dtype: object

edited Feb 13, 2021 at 21:19

answered Feb 13, 2021 at 21:14

chvolkmann

5242 silver badges10 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

paranormaldist Over a year ago

if one of the dfs does not have all of the columns, how can i account for that easily?

chvolkmann Over a year ago

Do you mean the case in which your dataframe has a lot of columns, but you only want to make a subset of those into datetime? Use convert_times(df, ['col1', 'col2']). All other columns of the dataframe won't be touched.

paranormaldist Over a year ago

if a df has time1 only but another df has time1,time2,time3. i'm running this on a few dfs, so i get a not in index error if not all times columns are in the df

chvolkmann Over a year ago

Then you would have to use convert_times(df1, ['time1', 'time2']); convert_times(df2, ['anothertime1', 'anothertime2'])

Collectives™ on Stack Overflow

Python convert multiple columns to datetime using for loop

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related