Numpy functions return array class instance when called on subclass of ndarray

Question

Some numpy functions (logically) return scalars:

>>> my_arr = np.ndarray(shape=(1,))
>>> type(np.max(my_arr))
<type 'numpy.float64'>

but only when called with an ndarray, rather than a subclass:

>>> class CustomArray(np.ndarray):
...     pass
>>> my_arr = CustomArray(shape=(1,))
>>> type(np.max(my_arr))
<class '__main__.CustomArray'>

Why is this? I'd expect either both to return a scalar (of type <type 'numpy.float64'>, or the former to return a np.ndarray instance and the latter a CustomArray instance. But instead, I get a combination of these two behaviours. Can I change this behaviour through changing my own class?

I don't see anything that would explain this on the doc page discussing subclassing ndarray (http://docs.scipy.org/doc/numpy-1.9.2/user/basics.subclassing.html).

(Running Python 2.7.10, numpy 1.9.2, in case it matters.)

Sudeep Juvekar · Accepted Answer · 2016-05-17 14:06:56Z

1

This is because max() is not overloaded in CustomArray. If you try it, my_array.max() returns an object of CustomArray instead of scalar.

my_array = CustomArray(shape=(1,))
print my_array.max()
>> CustomArray(9.223372036854776e+18)

np.max internally calls np.amax, which ends up calling np.maximum.reduce. This is the standard reduce of map-reduce and returns a base-object returned by max. Hence, the type returned by np.max is in fact the type returned by max() method called on your object. You can override it as:

class CustomArray(np.ndarray):
   def max(self, axis, out):
      return np.ndarray(self.shape, buffer=self).max(axis, out)

type(np.max(my_arr))
>> numpy.float64

The trick is to upcast self as an np.ndarray and find max using it.

edited May 17, 2016 at 14:06

answered May 17, 2016 at 14:01

Sudeep Juvekar

5,1183 gold badges31 silver badges35 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

acdr Over a year ago

I'm guessing then that I have to change pretty much every method that ndarray instances get? (I was just using max as an example.)

acdr Over a year ago

Also, this doesn't answer the question of why the instance of a subclass behaves differently from an instance of ndarray, if the subclass doesn't actually alter any behaviour.

Sudeep Juvekar Over a year ago

Yes, you probably have to update every relevant method, if you want the right return type. The class body does not update anything, but the default overrides of max() (and additional methods like sum()) are different from base np.ndarray.

acdr Over a year ago

Why are they different though? Is there an explicit check in ndarray.max to check that self is actually an instance of ndarray and not a subclass or something?

hpaulj Over a year ago

Look at the code for np.matrix or masked to see how they handle this.

|

nfrasser · Accepted Answer · 2022-09-13 20:14:05Z

0

I also ran into this, solved it for all aggregation/reduction operations in NumPy by implementing a custom __array_wrap__:

import numpy as np

class CustomArray(np.ndarray):
    def __array_wrap__(self, obj, **kwargs):
        if obj.shape == ():
            return obj[()]
        else:
            return super().__array_wrap__(obj, **kwargs)

Example return types for various operations:

>>> a = CustomArray(shape=(3,))
>>> type(np.max(a))
<class 'numpy.float64'>
>>> type(np.median(a))
<class 'numpy.float64'>
>>> type(np.exp(a))
<class '__main__.CustomArray'>
>>>

answered Sep 13, 2022 at 20:14

nfrasser

4151 gold badge5 silver badges13 bronze badges

Collectives™ on Stack Overflow

Numpy functions return array class instance when called on subclass of ndarray

2 Answers 2

6 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related