1

Here is the code:

import numpy as np

x = np.arange(2*32*32*8*16)
x = x.reshape(2, 32, 32, 8, 16)

print("x-shape=", x.shape)

y = x[0, 3:5, 8:10, [1, 2, 3, 4, 5], :]
z = x[0][3:5, 8:10, [1, 2, 3, 4, 5], :]

print("y-shape=", y.shape)
print("z-shape=", z.shape)

and the output:

x-shape= (2, 32, 32, 8, 16)
y-shape= (5, 2, 2, 16)
z-shape= (2, 2, 5, 16)

I would expect the shape of array y and z to be equal but they are not. What I need is actually the shape of z.
Can someone explain or point me to the standard?
Thanks!

1
  • 1
    y is mixed basic (slices) and advanced indexing, with the slices in the middle. The advanced dimension, 5, is put first. It's awkward but documented (and occasionally appears on SO). Commented Aug 13, 2023 at 14:02

2 Answers 2

1

Your arrays:

In [24]: x.shape
Out[24]: (2, 32, 32, 8, 16)

In [25]: y=x[0,:,:,:,[1,2,3,4,5]]
In [26]: y.shape
Out[26]: (5, 32, 32, 8)

Let's try y without the initial 0, i.e. no slices 'in the middle':

In [27]: z=x[:,:,:,:,[1,2,3,4,5]]    
In [28]: z.shape
Out[28]: (2, 32, 32, 8, 5)

y is a copy, its own 'base`:

In [29]: y.base

But z is a transpose of another array. Note that the 5 dimension is first:

In [30]: z.base.shape
Out[30]: (5, 2, 32, 32, 8)

The fact that advanced indexing produces a copy, followed by transpose to get the indexing in the right order, is not documented. But I suspect it is key to the unexpected mixed basic/advanced shape.

We also see this in the strides:

In [34]: x.strides
Out[34]: (1048576, 32768, 1024, 128, 8)
In [35]: y.strides
Out[35]: (65536, 2048, 64, 8)
In [36]: z.base.strides
Out[36]: (131072, 65536, 2048, 64, 8)
In [37]: z.strides
Out[37]: (65536, 2048, 64, 8, 131072)

With an initial index of 0, the ambiguity that the mixed docs talks about is not obvious. It's more so if we try to select a 'diagonal' from x. Should the size 2 dimension be first, or last?

In [54]: z = x[[1,0],:,:,:,[1,2]]
In [55]: z.shape
Out[55]: (2, 32, 32, 8)
Sign up to request clarification or add additional context in comments.

Comments

0

Refer to the numpy docs on indexing:

Single element indexing works exactly like that for other standard Python sequences. It is 0-based, and accepts negative indices for indexing from the end of the array. [...] Note that if one indexes a multidimensional array with fewer indices than dimensions, one gets a subdimensional array.

Thus x[0] gets the first multidimensional subarray of x. x[0] will have the shape (32, 32, 8, 16). Then, you slice this again, using multidimensional slicing with as many slices as there are dimensions. You obtain an array of the same dimensionality where all dimensions have been sliced: [3:5, 8:10, [1, 2, 3, 4, 5], :] projects the first 32 down to 2 (5 - 3), the second 32 down to 2 as well (10 - 8), the 8 down to 5 (selecting indices 1 - 5), and keeps the 16 as 16 (:). You get (2, 2, 5, 16).

What's probably confusing you here is that advanced indexing rules (even worse, you are combining advanced and basic indexing) apply since one of your slices is [1, 2, 3, 4, 5]. Thus numpy won't create a view but rather a copy. If you use the proper slice 1:6 instead, advanced indexing won't apply, and y.shape will be (2, 2, 5, 16) as expected.

The reason for this is that there is a fundamental difference between views ("fat pointers") and advanced indexing: When using advanced indexing, you don't have to use consecutive ranges with constant differences (arithmetic progressions). You could use prime numbers, your favorite numbers, whatever for indexing. Thus a simple view consisting of a pointer and a shape (which tells you how to move the pointer when you move by one in each direction, and where the array ends) won't suffice. numpy actually needs to go through your list of indices and select those indices you want, creating a new array.

1 Comment

The reason why y has a different shape, isn't about view vs copy. It goes deeper into how advanced indexing is done - in compiled code.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.