2

I seem to have had an XY problem in this question regarding how to tell if arrays share the same memory. The ways I was checking were wrong and I'm not sure why.

Let's take a few examples

test = np.ones((3,3))
test2 = np.array(test, ndmin = 2, copy = False)
test3 = np.array(test, ndmin = 3, copy = False)

First, let's check if they're sharing memory using .base

test2.base is test
False

test3.base is test
True

So it seems like test3 is sharing data with test but test2 isn't. In fact test2.base is None => True, which I thought meant that it is seperate memory (i.e. a copy).

This impression is reinforced when I check with .flags

test2.flags['OWNDATA']
True

test3.flags['OWNDATA']
False

It seems again like only test3 is sharing data, and test2 is a copy.

But if I check with the python builtin id(...)

id(test)
248896912

id(test2)
248896912

id(test3)
248897352

Now the id (which is supposedly the adress of the object in memory) of test and test2 match but test3 does not, which gives the exact opposite impression from the above methods.

And of course, both of those impressions are wrong because:

test.__array_interface__['data']
(209580688, False)

test2.__array_interface__['data']
(209580688, False)

test3.__array_interface__['data']
(209580688, False)

The actual buffer addresses all match. Indeed:

test[0,0] = 2

test, test2, test3

(array([[ 2.,  1.,  1.],
        [ 1.,  1.,  1.],
        [ 1.,  1.,  1.]]),  
array([[ 2.,  1.,  1.],
        [ 1.,  1.,  1.],
        [ 1.,  1.,  1.]]), 
array([[[ 2.,  1.,  1.],
         [ 1.,  1.,  1.],
         [ 1.,  1.,  1.]]]))

So if ndarray.base, ndarray.flags['OWNDATA'], and id(ndarray) don't actually tell me if memory is shared, what are they telling me?

Improve this question
1

1 Answer 1

Reset to default
2

Most of your confusion comes from the fact that test2 is test and test3 is a view of test.

ndarray.base

If an array is created as a view of another object, its base will point to the other object... or NumPy will follow that object's base chain and use the "root base". Sometimes one option, sometimes the other. There were a number of bugs and backward compatibility issues.

test2 is not a view of test. test2 is test. Its base is None because it's not a view.

test3 is a view of test, and its base is set to test.

ndarray.flags['OWNDATA']

This flag is set if the array is not a view. As previously stated, test3 is a view and test2 isn't, because test2 is test.

id

id gives a numeric identifier for an object, guaranteed not to be the same as the identifier for any object whose lifetime overlaps with the first. This function does not care about NumPy and views; two different arrays with overlapping lifetimes will have different IDs, regardless of any view relationships.

test2 is test, so of course it has the same ID as itself. test3 is a different object with an overlapping lifetime, so it has a different ID.

ndarray.__array_interface__['data']

This is a tuple whose first element is an integer representing the memory address of the array's first element. (The second element is unimportant for the question.) test, test2, and test3 all use the same data buffer, with their first elements at the same offset, so they give the same value here. However, two arrays that share memory won't always have the same value for this, because they might not have the same first element:

>>> import numpy
>>> x = numpy.arange(5)
>>> y = x[1:]
>>> z = x[:-1]
>>> x.__array_interface__['data']
(39692208L, False)
>>> y.__array_interface__['data']
(39692212L, False)
>>> z.__array_interface__['data']
(39692208L, False)
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Yeah, just figured out test2 is test == True about a minute before you answered. That makes everything make a bit more sense. I forgot that arrays can share identity as well as memory.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.