creating a new array from a big array using a small array to select values numpy

Question

I have two numpy arrays - a big 6000x4 array and small 10x1 array.

big_array = array[[1,0.2,0.3,0.4],
                  [1,10.0,20.0,30.0],
                  [2,7.0,0.4,0.8]
                  [2,3,4,5]
                  ...
                  [150,56,57,89]
                  [150,78,27,64]]
small_array = array([17, 12,  5, 12, 18, 11,  1,  4,  3,  9])

I want to create a new array using the values in the small array to select values from the big array. I can achieve this by using inputting single values in the following line:

new_array_0 = big_array[big_array[:,0]] == small_array[0]
new_array_1 = big_array[big_array[:,0]] == small_array[1]

I tried running this in a for loop:

subSet = np.empty((0,4))
for i in small_array:
    a = big_array[big_array[:,0]] == small_array[i]
    subSet = np.append(subSet,a,axis=0)

But this results in an error. Is there a python/numpy way to achieve this?

Could you add a complete example: a big_array, a small_array and the expected output? — JohanC
– JohanC, Commented Jan 23, 2021 at 12:39
I think the error appears because subSet and a have different shapes. Dont know how to fix it though, unless maybe making subsets shape ((,4)) instead of ((0, 4)) — Canbach
– Canbach, Commented Jan 23, 2021 at 14:14
What is the error? Don't just say there's an error and expect us to guess. Do some of the debugging work yourself! — hpaulj
– hpaulj, Commented Jan 23, 2021 at 17:10

Ivan · Accepted Answer · 2021-01-23 14:10:58Z

I took a small sample from your post:

>>> big_array = np.array([[ 1,  0,  0,  0],
                          [ 1, 10, 20, 30],
                          [ 2,  7,  0,  0],
                          [ 2,  3,  4,  5],
                          [ 2, 56, 57, 89],
                          [ 1, 78, 27, 64]])

>>> small_array = torch.array([20, 10, 30])

The way I see it, you have a two dimensional array big_array[big_array[:,0]] which you want to compare to scalar values contained in small_array. Comparing an array with a single scalar will return an array of same shape where the conditional will have been called element-wise, leaving you with a boolean mask:

>>> big_array[big_array[:,0]] == small_array[0]
array([[False, False, False, False],
       [False, False, False, False],
       [False, False, False, False],
       [False, False, False, False],
       [False, False, False, False],
       [False, False, False, False]])

Of course, if you compare to small_array straight away, it will check array-to-array, which will return False and perharps even a DeprecationWarning warning:

>>> big_array[big_array[:,0]] == small_array
False

What you could do though, is broadcast your small_array:

>>> small_array[:, None, None].shape
(10, 1, 1)

This way you can compare the indexed array with your small_array's scalar values:

>>> big_array[big_array[:,0]] == small_array[:, None, None]
array([[[False, False,  True, False],
        [False, False,  True, False],
        [False, False, False, False],
        [False, False, False, False],
        [False, False, False, False],
        [False, False,  True, False]],

       [[False,  True, False, False],
        [False,  True, False, False],
        [False, False, False, False],
        [False, False, False, False],
        [False, False, False, False],
        [False,  True, False, False]],

       [[False, False, False,  True],
        [False, False, False,  True],
        [False, False, False, False],
        [False, False, False, False],
        [False, False, False, False],
        [False, False, False,  True]]])

Which is the same result as the one yielded by big_array[big_array[:,0]] == small_array[0], big_array[big_array[:,0]] == small_array[1], and big_array[big_array[:,0]] == small_array[2].

Collectives™ on Stack Overflow

creating a new array from a big array using a small array to select values numpy

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related