Improve performance of Datetime operation in pandas

Question

I have a big data set that I need to date operation, and as it is taking too long, I was wondering if there is any other way to boost up the speed. Data frame looks like following:

Date, Month
2017-01-01, 0
2017-01-01, 1
2017-01-01, 2

I need to create another column that adds month column to date column, so it would look like following:

Date, Month, newDate
2017-01-01, 0, 2017-01-01
2017-01-01, 1, 2017-02-01
2017-01-01, 2, 2017-03-01

My current method is using apply function and relativedelta method like:

def newDateCalc(self, row):
return row[0] + relativedelta(months = row[1])

df['newDate'] = df[['Date', 'Month']].apply(lambda row: newDateCalc(row), axis = 1)

Thank you for your help in advance,

MaxU - stand with Ukraine · Accepted Answer · 2017-07-25 21:13:38Z

1

Here is my vectorized attempt:

df['newDate'] = (df.Date.values.astype('M8[M]') + 
                 df.Month.values * np.timedelta64(1, 'M')).astype('M8[D]')

Result:

In [106]: df
Out[106]:
        Date  Month    newDate
0 2017-01-01      0 2017-01-01
1 2017-01-01      1 2017-02-01
2 2017-01-01      2 2017-03-01

answered Jul 25, 2017 at 21:13

MaxU - stand with Ukraine

212k37 gold badges402 silver badges436 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

MaxU - stand with Ukraine Over a year ago

@Hojin, glad I could help :)

cs95 · Accepted Answer · 2017-07-25 21:06:21Z

1

You can use df.transform with relativedelta:

In [960]: df.transform(lambda x: x['Date'] + relativedelta(months=x['Month']), axis=1)
Out[960]: 
0   2017-01-01
1   2017-02-01
2   2017-03-01
dtype: datetime64[ns]

answered Jul 25, 2017 at 21:06

cs95

406k106 gold badges744 silver badges797 bronze badges

Collectives™ on Stack Overflow

Improve performance of Datetime operation in pandas

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related