4

When I was trying to solve a scientific problem with Python (Numpy), a 'shape mismatch' error came up: "shape mismatch: objects cannot be broadcast to a single shape". I managed to reproduce the same error in a simpler form, as shown below:

import numpy as np
nx = 3; ny = 5
ff = np.ones([nx,ny,7])
def test(x, y):
    z = 0.0
    for i in range(7):
        z = z + ff[x,y,i]
    return z

print test(np.arange(nx),np.arange(ny))

When I tried to call test(x,y) with x=1,y=np.arange(ny), everything works fine. So what's going on here? Why can't the both parameters be numpy arrays?

UPDATE

I have worked out the problem with some hints from @Saullo Castro. Here's some updated info for you guys who tried to help but feel unclear about my intention:

Basically I created a mesh grid with dimension nx*ny and another array ff that stores some value for each node. In the above code, ff has 7 values for each node and I was trying to sum up the 7 values to get a new nx*ny array.

However, the "shape mismatch" error is not due to the summing process as many of you might have guess now. I have misunderstood the rule of functions taking ndarray objects as input parameters. I tried to pass np.arange(nx), np.arange(ny) to test() is not gonna give me what I desired, even if nx==ny.

Back to my original intention, I solve the problem by creating another function and used np.fromfunction to created the array:

def tt(x, y):  
    return np.fromfunction(lambda a,b: test(a,b), (x, y))

which is not perfect but it works. (In this example there seems to be no need to create a new function, but in my actual code I modified it a bit so it can be used for slice of the grid)

Anyway, I do believe there's a much better way compared to my kind of dirty solution. So if you have any idea about that, please share with us :).

Improve this question
2
  • I think you are misunderstanding how indexing multi-dimensional arrays works, have a look at the docs; it is very clear explanation docs.scipy.org/doc/numpy/user/basics.indexing.html Commented May 16, 2014 at 3:20
  • @Akavall Could you be more specific about that? BTW I'm aware that I'm mixing the x[i] and i since they are the same in this simplified case. Commented May 16, 2014 at 4:22

4 Answers 4

Reset to default
3

Let's look into an array similar to your ff array:

nx = 3; ny = 4
ff = np.arange(nx*ny*5).reshape(nx,ny,5)
#array([[[ 0,  1,  2,  3,  4],
#        [ 5,  6,  7,  8,  9],
#        [10, 11, 12, 13, 14],
#        [15, 16, 17, 18, 19]],
#
#       [[20, 21, 22, 23, 24],
#        [25, 26, 27, 28, 29],
#        [30, 31, 32, 33, 34],
#        [35, 36, 37, 38, 39]],
#
#       [[40, 41, 42, 43, 44],
#        [45, 46, 47, 48, 49],
#        [50, 51, 52, 53, 54],
#        [55, 56, 57, 58, 59]]])

When you index using arrays of indices a, b, c like in ff[a, b, c], a, b, c must have the same shape, and numpy will build a new array based on the indices. For example:

ff[[0, 0, 1, 1, 2, 2], [0, 1, 0, 1, 2, 3], [0, 0, 0, 1, 1, 1]]
#array([ 0,  5, 20, 26, 51, 56])

This is called fancy indexing, which is like building an array with:

np.array([ff[0, 0, 0], ff[0, 1, 0], ff[1, 0, 0], ..., ff[2, 3, 1]])

In your case the f[x, y, i] will produce a shape mismatch error since a, b, c do not have the same shape.

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

Thank you Saullo Castro. I have updated my original post. Please take a look.
@seekiu I read you update, you could probably achieve the same result with ff.sum(axis=2)...
@seekiu you showld consider accepting one of the answers if they fit your needs...
I'd like to accept your answer since I worked out my problem based on it, even though not exactly what I was asking for. Pity I cannot vote others' useful because my reputation is too low... Anyway, thanks for your help.
1

Looks like you want to sum ff over the last dimension, with the 1st 2 dimensions covering their whole range. : is used to denote the whole range of a dimension:

def test():
    z = 0.0
    for i in range(7):
        z = z + ff[:,:,i]
    return z
print test()

But you can get the same result without looping, by using the sum method.

print ff.sum(axis=-1)

: is shorthand for 0:n

ff[0:nx, 0:ny, 0]==ff[:,:,0]

It is possible to index a block of ff with ranges, but you have to be much more careful about the shapes of the indexing arrays. For a beginner it is better to focus on getting slicing and broadcasting correct.

edit -

You can index an array like ff with arrays generated by meshgrid:

I,J = meshgrid(np.arange(nx),np.arange(ny),indexing='ij',sparse=False)
I.shape # (nx,ny)
ff[I,J,:]

also works with

I,J = meshgrid(np.arange(nx),np.arange(ny),indexing='ij',sparse=True)
I.shape # (nx,1)
J.shape # (1, ny)

ogrid and mgrid are alternatives to meshgrid.

Improve this answer

2 Comments

Thank you hpaulj. I have updated my original post. Please take a look.
I wonder if the kinds of grids produced by meshgrid are useful in your case?
0

Let's reproduce your problem in 2D case, so it is easier to see:

import numpy as np

a = np.arange(15).reshape(3,5)

x = np.arange(3)
y = np.arange(5)

Demo:

>>> a
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14]])
>>> a[x, y] # <- This is the error that you are getting
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: shape mismatch: objects cannot be broadcast to a single shape

# You are getting the error because x and y are different lengths,
# If x and y were the same lengths, the code would work: 

>>> a[x, x]
array([ 0,  6, 12])

# mixing arrays and scalars is not a problem

>>> a[x, 2]
array([ 2,  7, 12])
Improve this answer

1 Comment

Thank you Akavall. I have updated my original post. Please take a look.
0

It is not clear in your question what you are trying to do or what result are you expecting. It seems, though, that you are trying to calculate a total with your variable z.

Check if the sum method produces the result that you need:

import numpy as np
nx = 3; ny = 5

ff = ff = np.array(np.arange(nx*ny*7)).reshape(nx,ny,7)

print ff.sum()          # 5460

print ff.sum(axis=0)    # array([[105, 108, 111, 114, 117, 120, 123],
                        #        [126, 129, 132, 135, 138, 141, 144],
                        #        [147, 150, 153, 156, 159, 162, 165],
                        #        [168, 171, 174, 177, 180, 183, 186],
                        #        [189, 192, 195, 198, 201, 204, 207]]) shape(5,7)

print ff.sum(axis=1)    # array([[ 70,  75,  80,  85,  90,  95, 100],
                        #        [245, 250, 255, 260, 265, 270, 275],
                        #        [420, 425, 430, 435, 440, 445, 450]]) shape (3,7)

print ff.sum(axis=2)    # array([[ 21,  70, 119, 168, 217],
                        #        [266, 315, 364, 413, 462],
                        #        [511, 560, 609, 658, 707]]) shape (3,5)
Improve this answer

1 Comment

Thank you. I have updated my original post. Please take a look.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.