Why cannot numpy arrays convert from datetime to np.datetime64 implicitly?

Question

Say, I have a datetime:

given_time = datetime(2013, 10, 8, 0, 0, 33, 945109,
                      tzinfo=psycopg2.tz.FixedOffsetTimezone(offset=60, 
                                                             name=None))

I would like to transform it into np.datetime64:

np.datetime64(given_time)
> numpy.datetime64('2013-10-08T00:00:33.945109+0100')

It works well. However, if I have an array of given_time:

given_times = np.array([given_time]*3) # dtype is object

Both given_times.astype('datetime64') and given_times = np.array([given_time] * 3, dtype=np.datetime64) would trigger TypeError: Cannot cast datetime.datetime object from metadata [us] to [D] according to the rule 'same_kind'

So, I have to specify the unit:

given_times.astype('datetime64[us]')
# or
given_times = np.array([given_time]*3, dtype='datetime64[us]')

My question is, why do I have to specify the unit here? It doesn't require unit in np.datatime64 constructor.

Because a numpy array is a homogeneous collection of variables. The same is true of array.array. — Nafiul Islam
– Nafiul Islam, Commented Feb 24, 2014 at 13:10
@GamesBrainiac I am asking why cannot numpy infer the unit in this case, like the way it works here: np.array(['2007-07-13', '2006-01-13', '2010-08-13'], dtype='datetime64') — colinfang
– colinfang, Commented Feb 24, 2014 at 13:12
Good question. What happens with np.array(np.datetime64(d) for d in [given_time] * 3)? — salezica
– salezica, Commented Feb 24, 2014 at 13:22
@uʍopǝpısdn np.array(list(np.datetime64(d) for d in [given_time] * 3)) works — colinfang
– colinfang, Commented Feb 24, 2014 at 13:29

aldanor · Accepted Answer · 2016-05-25 18:05:03Z

I know it's an old question, but I'd try to answer in case anyone else comes across this.

As of 1.11, numpy doesn't try to automatically convert iterables of date/datetime objects to datetime64 arrays, this is pretty clear from this excerpt in the test suite:

# at the moment, we don't automatically convert these to datetime64

dt = datetime.date(1970, 1, 1)
arr = np.array([dt])
assert_equal(arr.dtype, np.dtype('O'))

dt = datetime.datetime(1970, 1, 1, 12, 30, 40)     
arr = np.array([dt])
assert_equal(arr.dtype, np.dtype('O'))

Ideally, numpy would figure that datetime64 with correct units could be used; see this issue.

When constructing datetime64 from a scalar, the unit it set to M8[D] for date objects and to M8[us] for datetime objects (a relevant test).
When you specify dtype='datetime64', or, similarly, dtype='M8', the units are set to "generic", which later resolves to M8[D] (although it would be logical to have it resolve to M8[D], see this issue):

>>> np.datetime_data(np.dtype('datetime64'))
('generic', 1)
>>> np.datetime_data(np.dtype('M8'))
('generic', 1)
>>> np.datetime_data(np.dtype('M8[D]'))
('D', 1)
>>> np.datetime_data(np.dtype('M8[us]'))
('us', 1)

given_times.astype('datetime64') no longer raises an exception -- this was fixed in 1.11.
Starting from 1.11, datetime64 objects are timezone-naive, so passing a datetime object with tzinfo set like in the provided example will trigger a deprecation warning.

Collectives™ on Stack Overflow

Why cannot numpy arrays convert from datetime to np.datetime64 implicitly?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related