numpy datetime64 in recarray

Question

I'm having trouble creating a record array with the datetime64 type. I'm running Python 2.7, Numpy 1.7.

Here's a minimal example:

p_dtype = np.dtype({"names": ['trns_id', 'trns_date', 'qty', 'price', 'amount', 'description', 'commission', 'fees'],
                    "formats": [long, "M8", float, float, float, "S40", float, float]})

p_row = (8609132959, np.datetime64('2012-05-01'), 337.574, 4.86, -1640.61, 'Bought 337.574 XYZ @ 4.86', 0.0, 0.0)

print p_list, p_dtype

p_array = np.array(p_row, dtype=p_dtype)

I get the following error (& output):

TypeError                                 Traceback (most recent call last)
<ipython-input-137-0b4de45b819c> in <module>()
      6 print p_list, p_dtype
      7 
----> 8 p_array = np.array(p_row, dtype=p_dtype)
      9 
     10 print "Array: %s, dtype: %s" % (p_array, p_array.dtype)

TypeError: Cannot cast NumPy timedelta64 scalar from metadata [D] to  according to the rule 'same_kind'

(8609132959.0, numpy.datetime64('2012-05-01'), 337.574, 4.86, -1640.61, 'Bought 337.574 PIMSX @ 4.86', 0.0, 0.0) [('trns_id', '<i8'), ('trns_date', '<M8'), ('qty', '<f8'), ('price', '<f8'), ('amount', '<f8'), ('description', 'S40'), ('commission', '<f8'), ('fees', '<f8')]

Hints, anyone?

askewchan · Accepted Answer · 2013-05-17 22:37:49Z

Specify a "date" datetime dtype. That is, "M8[D]" instead of "M8", or 'datetime64[D]' instead of 'datetime64'.

In [80]: np.array([(0,np.datetime64('2012-05-17'))],
   ....:          dtype=[('i',np.int),('date','datetime64[D]')])
Out[80]: 
array([(0, datetime.date(2012, 5, 17))], 
      dtype=[('i', '<i8'), ('date', '<M8[D]')])

Note you can also feed in your data as simply the string (ie '2012-05-17', instead of the np.datetime('2012-05-17') object)

In [81]: np.array([(0,'2012-05-17')],
   ....:          dtype=[('i',np.int),('date','datetime64[D]')])
Out[81]: 
array([(0, datetime.date(2012, 5, 17))], 
      dtype=[('i', '<i8'), ('date', '<M8[D]')])

It seems that these types are interpreted differently in the single dtype case from the structured dtype case. You would not run into the problem you are having with a single dtype like here:

In [84]: np.array([np.datetime64('2012-05-17')], dtype='datetime64')   # no need for [D]
Out[84]: array(['2012-05-17'], dtype='datetime64[D]')

In [85]: np.array(['2012-05-17'], dtype='datetime64')   # no need for [D]
Out[85]: array(['2012-05-17'], dtype='datetime64[D]')

But make it structured and you do have the problem:

In [87]: np.array([(0,'2012-05-17')],
   ....:          dtype=[('i',np.int),('date','datetime64')])
---------------------------------------------------------------------------
ValueError: Cannot create a NumPy datetime other than NaT with generic units

In [88]: np.array([(0,np.datetime64('2012-05-17'))],
   ....:          dtype=[('i',np.int),('date','datetime64')])
---------------------------------------------------------------------------
TypeError: Cannot cast NumPy timedelta64 scalar from metadata [D] to  according to the rule 'same_kind'

shlomoa · Accepted Answer · 2015-06-27 09:05:05Z

numpy has a page on datetime, it is rather heavy but answers most questions.

Two things to note:

the separation between date and time just like in python's datetime
The context of use which is specific to numpy (the [*] suffix)

The issue encountered above is of the second kind,

dtnow = datetime.datetime.now()
numpy.datetime64(dtnow, '[D]')

Traceback (most recent call last):
  File "", line 1, in 
TypeError: Cannot cast datetime.datetime object from metadata [us] to [D] according to the rule 'same_kind'

numpy.datetime64(dtnow, '[s]')

numpy.datetime64('2015-06-27T14:53:21+0300')

If your datetime will never have any time component than datetime64[D] would be sufficient.

however, if it will have, I would suggest using datetime64[s] a second level context.

Collectives™ on Stack Overflow

numpy datetime64 in recarray

2 Answers 2

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related