OverflowError: date value out of range when converting matlab datenum to python datetime

Question

I am trying to convert a pandas dataframe column of matlab datenum format to python date time format using the function below:

def datenum_to_datetime(datenum):
    """
    Convert Matlab datenum into Python datetime.
    :param datenum: Date in datenum format
    :return:        Datetime object corresponding to datenum.
    """
    days = datenum % 1
    date = datetime.fromordinal(int(datenum)) \
           + timedelta(days=days) \
           - timedelta(days=366)
 
    return date.year

A sample of the values

0         693726
1         693726
2         693726
3         693726
4         693726
           ...  
460718    726831
460719    726831
460720    726831
460721    726831
460722    726831
Name: dob, Length: 460723, dtype: int32

But I get the error below when I try to apply the function.

df['dob'] = df['dob'].apply(datenum_to_datetime)

OverflowError                             Traceback (most recent call last)
<ipython-input-67-cc4fb607b497> in <module>()
----> 1 df['dob'] = df['dob'].apply(datenum_to_datetime)
      2 df.head()

1 frames
pandas/_libs/lib.pyx in pandas._libs.lib.map_infer()

<ipython-input-66-bab915eae5ff> in datenum_to_datetime(datenum)
      6     """
      7     days = datenum % 1
----> 8     date = datetime.fromordinal(int(datenum))            + timedelta(days=2)            - timedelta(days=366)
      9 
     10     return date.year

OverflowError: date value out of range

solves the problem for me: Converting Matlab's datenum format to Python — FObersteiner
– FObersteiner, Commented May 4, 2021 at 6:42

Chiebukar · Accepted Answer · 2021-05-04 04:50:13Z

0

Didn't find a single solution following this Converting Matlab's datenum format to Python, but this function worked without errors

def datenum_to_datetime(datenum):
    """
    Convert Matlab datenum into Python datetime.
    :param datenum: Date in datenum format
    :return:        Datetime object corresponding to datenum.
    """
    origin = np.datetime64('0000-01-01', 'D') - np.timedelta64(1, 'D')
    date = datenum * np.timedelta64(1, 'D') + origin
    year = date.astype('datetime64[Y]').astype(int) + 1970
 
    return year

answered May 4, 2021 at 4:50

Chiebukar

9310 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

FObersteiner Over a year ago

I checked your code, it works with pandas 1.2.4 / numpy 1.20.2 but seems to fail with older versions. Besides, the method described here seems to be much simpler to me (basically just pd.to_datetime(df['datenum']-719529, unit='D')). Once you have a column of type datetime, you can get the year like df['datetime'].dt.year.

Chiebukar Over a year ago

Your proposed method seems better, but I get this error when using it on pandas version 1.1.5. OutOfBoundsDatetime: cannot convert input with unit 'D'

Chiebukar Over a year ago

Same issue,it actually works for some datenum inputs and not all.

FObersteiner Over a year ago

perhaps the resulting datetime actually is out of range?

Chiebukar Over a year ago

it works for

datenum =    656178                                                                                               date = pd.to_datetime((datenum-719529)*86400, unit='s')     year = date.year

results in 1796 but datenum = 600000 gives the OutOfBoundsDatetime: cannot convert input with unit 's'. and I do have very low values like 146, 25 etc in my dataframe. I wonder this error. you could probably try converting 600000 and see its result.

|

Collectives™ on Stack Overflow

OverflowError: date value out of range when converting matlab datenum to python datetime

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related