Why does indexing this Numpy matrix cause an error?

Question

I'm using:

Python 3.5.4
Numpy 1.16.2

Given the code:

import numpy as np
num = 3
a = np.asmatrix(np.eye(num, num))
b = np.asmatrix(range(0, num))
print(a[b].transpose())

I get the result:

[[1.00000000e+000 0.00000000e+000 0.00000000e+000]
 [1.77658241e-307 0.00000000e+000 0.00000000e+000]
 [3.47328271e-310 0.00000000e+000 0.00000000e+000]]

But by either changing the definition of b to np.asarray(...), or by performing a second indexing to get the first item in the list (print(a[b][0].transpose()) I get the expected result:

[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]

What is going on? I've not experienced this bug before don't have a clue to the underlying cause. If its something fundamental to Python (I'm quite new to it still) I'd quite like to learn about it so that I don't sink quite as much time into debugging it as I have this time. Many thanks in advance.

it is not an error. Basically you see a representation of integer into float (decimal form) and it is not exactly the same with integer value. See similar post here: stackoverflow.com/questions/286061/… — jose_bacoy
– jose_bacoy, Commented Apr 18, 2019 at 21:05
@âńōŋŷXmoůŜ If that were the case I'd expect it to produce the same error irrespective of how I manipulated it before hand. In both cases, without the transpose, there are no rounding errors and I get the identity matrix back with only 1.0 and 0.0 values showing. — user3303504
– user3303504, Commented Apr 18, 2019 at 21:09

user2357112 · Accepted Answer · 2019-04-18 21:15:30Z

1

NumPy matrix objects try to ensure they're always 2D, but they don't quite do a complete job of it.

When you do

b = np.asmatrix(range(0, num))

range(0, num) is a 1D sequence, but asmatrix creates a single-row 2D matrix instead of a 1D object. asarray would produce a 1D array.

When you do

a[b]

this is a fancy indexing operation that produces a 3D matrix. numpy.matrix objects are never supposed to be 3D, and the following transpose doesn't know how to handle that, producing nonsensical results. Particularly, the strange, almost-0 numbers in the output are not due to rounding error; they're due to the nonsensical strides of the resulting matrix, which cause unaligned memory access and try to read fragments of separate floats as a single float:

In [13]: a[b].transpose().strides
Out[13]: (1, 24)

That's a 1-byte stride in the first dimension.

The key takeaway here: don't use numpy.matrix.

edited Apr 18, 2019 at 21:15

answered Apr 18, 2019 at 21:08

user2357112

286k32 gold badges490 silver badges571 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

user3303504 Over a year ago

I guess that makes sense. Wonder why there isn't an error message / warning appearing to indicate as much. Took me ages to figure out it was that code messing things up.

user3303504 Over a year ago

I presume you mean "don't use numpy.matrix for indexing"?

user2357112 Over a year ago

@user3303504: Don't use it at all if you have any other option. It causes all kinds of problems, not just this problem, and the only benefit it ever had is gone with the introduction of the @ operator.

user2357112 Over a year ago

Also see the warning in its docs: "It is no longer recommended to use this class, even for linear algebra. Instead use regular arrays. The class may be removed in the future."

hpaulj Over a year ago

@user3303504, in your case it's the fact that a is np.matrix and b is 2d (matrix or array) that produces the false 3d matrix.

Collectives™ on Stack Overflow

Why does indexing this Numpy matrix cause an error?

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related