move values of 2D array knowing new coordinates with mask

Question

I would like to "move" elements of a 2D array to new coordinates which are stored in 2 other arrays. I'm looking to automate this, because in reality my arrays are large (400x200x100). Some values wont find his coordinates and wont be used, Some of these coordinates are masked, which I have indicated in the example below by using the value 0. If the coordinate is masked, the elements in the array I want to reshuffle won't be used.

import numpy as np

#My new coordinates in X and Y directions   
mx = np.array([[ 1.,  2.,  3.,  4.,  0.],
       [ 1.,  2.,  3.,  4.,  0.],
       [ 1.,  2.,  3.,  4.,  0.],
       [ 1.,  2.,  3.,  4.,  0.],
       [ 1.,  2.,  3.,  4.,  0.]])

my = np.array([[ 0.,  2.,  2.,  2.,  2.],
       [ 0.,  3.,  3.,  3.,  3.],
       [ 0.,  4.,  4.,  4.,  4.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])

#The array with values to move
IRtest = np.array([[-0.07383495, -0.08606554, -0.08480594, -0.08099556, -0.08218414],
       [-0.07866761, -0.08373   , -0.08253587, -0.08106102, -0.08220205],
       [-0.07727436, -0.08271511, -0.0807254 , -0.07832416, -0.08021686],
       [-0.07612349, -0.08190446, -0.07996929, -0.07842754, -0.08024891],
       [-0.07488144, -0.08150557, -0.08038229, -0.07895656, -0.07997815]])

#Creation of zeros array to get new array
b = np.zeros((5,5))    

# I tried this but it doesn't work...
for i in range(IRtest.shape[0]):
    for j in range(IRtest.shape[1]):
        b[my[i,j], mx[i,j]] = IRtest[i,j]

plt.imshow(b)
plt.colorbar()
plt.show()

So the array expected looks like :

array_expected = np.array([[-0.08271511, -0.0807254 , -0.07832416, -0.08021686, 0],
       [-0.08190446, -0.07996929, -0.07842754, -0.08024891, 0],
       [-0.08150557, -0.08038229, -0.07895656, -0.07997815, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0]])

-----EDIT LATER -----------------

It s better in this "orientation" :

for i in range(IRtest.shape[0]):
    for j in range(IRtest.shape[1]):
        b[j, i] = IRtest[my[j,i],mx[j,i]]

And i get that :

array([[-0.08606554, -0.0807254 , -0.07832416, -0.08021686, -0.07727436],
       [-0.08606554, -0.07996929, -0.07842754, -0.08024891, -0.07612349],
       [-0.08606554, -0.08038229, -0.07895656, -0.07997815, -0.07488144],
       [-0.08606554, -0.08480594, -0.08099556, -0.08218414, -0.07383495],
       [-0.08606554, -0.08480594, -0.08099556, -0.08218414, -0.07383495]])

So the last problem it s to deal with the masked values...

So i try :

mask_mx = np.array([[False, False, False, False, True],
        [False, False, False, False, True],
        [False, False, False, False, True],
        [False, False, False, False, True],
        [False, False, False, False, True]], dtype=int)

mask_my = np.array([[True, False, False, False, False],
        [True, False, False, False, False],
        [True, False, False, False, False],
        [True, True, True, True, True],
        [True, True, True, True, True]], dtype=int)


mx3 = np.where(mask_mx, 'nan', mx)

my3 = np.where(mask_my, 'nan', my)

for i in range(IRtest.shape[0]):
    for j in range(IRtest.shape[1]):
        b[j, i] = IRtest[my3[j,i],mx3[j,i]]

But i get the error below, it doesn t like 'nan' as coordinates : invalid literal for int() with base 10: 'nan'

I cant let zero values and apply after a mask because i will loose several values... — user3601754
– user3601754, Commented Nov 18, 2014 at 10:23
You should not have edited your question so drastically as to make my answer invalid. I gave the correct answer, 2 days later you updated your question, thereby implementing my comments. In the future, ask a new question and give credit where credit's due. Because the question was eventually answered to your satisfaction, I won't roll back your edit, but this is not the way things are done on StackOverflow. — Oliver W.
– Oliver W., Commented Mar 1, 2015 at 14:50

Ben · Accepted Answer · 2014-11-18 21:38:28Z

1

+50

You probably want to use -1 as mask and not 0, so you can access the 0 index in IRtest.

mx = np.array([[ 1.,  2.,  3.,  4.,  -1.],
               [ 1.,  2.,  3.,  4.,  -1.],
               [ 1.,  2.,  3.,  4.,  -1.],
               [ 1.,  2.,  3.,  4.,  -1.],
               [ 1.,  2.,  3.,  4.,  -1.]])

my = np.array([[ 2.,  2.,  2.,  2.,  2.],
               [ 3.,  3.,  3.,  3.,  3.],
               [ 4.,  4.,  4.,  4.,  4.],
               [ -1.,  -1.,  -1.,  -1.,  -1.],
               [ -1.,  -1.,  -1.,  -1.,  -1.]])

b = np.zeros_like(IRtest)

for i in range(IRtest.shape[0]):
    for j in range(IRtest.shape[1]):
        b[j, i] = IRtest[my[j,i],mx[j,i]]*(mx[j,i]!=-1)*(my[j,i]!=-1)

answered Nov 18, 2014 at 21:38

Ben

3221 silver badge4 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user3601754 Over a year ago

And if i want to add a third dimension to my array, can i add a another loop?

Oliver W. · Accepted Answer · 2014-11-16 13:56:53Z

1

Your code does work, but your arrays of coordinates are not what you have in mind. Let's analyze your double for-loop:

During the iteration where i is 2 and j is 1, you're effectively handling the case:

b[my_test[2,1], mx_test[2,1]] = IRtest[2,1]

IRtest[2,1] is -0.08271511. my[2,1] is 4 and mx[2,1] is 2, so the line above boils down to:

b[4, 2] = -0.08271511

which is exactly what you see in the output.

Basically the problem is that you should change your indices arrays my and mx to get the output you desire.

answered Nov 16, 2014 at 13:56

Oliver W.

13.6k3 gold badges41 silver badges52 bronze badges

1 Comment

Oliver W. Over a year ago

I want to add here that this post was valid, up till the point where the original poster decided to change his original question. It's not the first time... Please take the time to revise your questions before you post them on StackOverflow. When someone correctly pinpoints the problem and you then change your question drastically, is not done. You should've considered asking a new question.

shx2 · Accepted Answer · 2015-01-02 09:15:06Z

Thanks to numpy's fancy-indexing, the loop isn't necessary, and in fact, your code will run much faster without it.

Also, note that:

your index-arrays are best represented as int-arrays (fancy-indexing wouldn't work with floats)
masks are best represented as bool-arrays (which allow logical operations as bitwise operations, are more memory efficient, and much faster).

So some preprocessing is required:

mx = np.array(mx, dtype=int)
my = np.array(my, dtype=int)
mask_mx = np.array(mask_mx, dtype=bool)
mask_my = np.array(mask_my, dtype=bool)
unified_mask = mask_my | mask_mx  # logical-or as bitwise-or

So, first without the mask, you can simply fancy-index your array:

b = IRtest[my,mx]

Then to apply the mask, you can either do:

b *= ~unified_mask

or explicitly assign zeros (again, fancy-indexing, this time using a mask):

b[unified_mask] = 0

As a one-liner:

b = IRtest[my,mx] * ~unified_mask

This is shorter, and way faster, than looping. Neat, eah?

With numpy, more often than not, you can and should avoid the loops.

Collectives™ on Stack Overflow

move values of 2D array knowing new coordinates with mask

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related