0

I am trying to parallelize a for-loop with numba. I'm new to this library, but after some research I crafted this code, which looked correct in comparison to the examples I studied:

@njit(nopython=True, parallel=True)
def tempFtemp(ftemp_pte, func_F, numPointsEval, pointsToEval):
for i in prange(0, numPointsEval):
    ftemp_pte[:,i] = np.hstack(func_F(np.vstack(pointsToEval[:,i])))
return ftemp_pte

ftemp_pte= tempFtemp(ftemp_pte, func_F, numPointsEval, pointsToEval)

When i compile it in my program thought, i get the error "non-precise type pyobject" at the line for i in prange(0, numPointsEval):. Both ftemp_pteand pointsToEvalare 2d arrays, numPointsEvalis an integer and func_Fis a random function which will produce the 1d arrays to be stored in ftemp_pte[:,i].

Any help on how to figure out what is producing this error would be greatly appreciated.

[EDIT]

The sequential code i have initially (which works) is the following:

def func_F(x):
    f= np.zeros((1,2))
    f[0,0]= x[0,0]
    n= max(np.size(x,0), np.size(x,1))    
    g    = 1 + 9* np.sum(x[1:n])/(n-1)
    h    = 1 - np.sqrt(f[0,0]/g)
    f[0,1] = g * h
    F= np.transpose(f)
    return F

for i in range(0, numPointsEval):
   ftemp_pte[:,i] = np.hstack(func_F(np.vstack(pointsToEval[:,i])))

I also should mention that the use of hstackand vstack is needed, so that the format of the arrays created can match ftemp_ptearray. Removing those instructions would result in a mismatch of dimensions.

The variable ftemp_pte always has 2 rows and x columns. One example of the correct values is

[[0.21875   0.21875   0.21875   0.21875   0.21875   0.21875   0.21875
  0.21875   0.21875   0.21875   0.21875   0.21875   0.21875   0.21875
  0.21875   0.21875   0.21875   0.21875   0.21875   0.21875   0.21875
  0.21875   0.21875   0.21875   0.21875   0.21875   0.21875   0.21875
  0.21875  ]
 [0.5397286 0.5397286 0.5397286 0.5397286 0.5397286 0.5397286 0.5397286
  0.5397286 0.5397286 0.5397286 0.5397286 0.5397286 0.5397286 0.5397286
  0.5397286 0.5397286 0.5397286 0.5397286 0.5397286 0.5397286 0.5397286
  0.5397286 0.5397286 0.5397286 0.5397286 0.5397286 0.5397286 0.5397286
  0.5397286]]

The original purpose of my code is to translate to Python the following Matlab's parfor instruction

parfor i=1:numPointsEval
     ftemp_pte(:,i) = feval(func_F,pointsToEval(:,i));

Any help would be greatly appreciated

Improve this question

2 Answers 2

Reset to default
1

Answer to problem #2.

As long as you are not really stacking arrays, but reshaping them, you should avoid hstack() and vstack() and consider using reshape() or ravel(). This way for example:

ftemp_pte[:, i] = func_F(pointsToEval[:, i].reshape(1, -1)).ravel()

However, reshape() on non-contiguous arrays is not supported by numba.

So I have managed to make your code run with numba by transposing everything to avoid reshaping arrays. The following code does work and may give you some ideas:

@nb.njit
def func_F(x):
    f = np.zeros(2)    # Simple 1d array
    f[0] = x[0]
    n = max(x.shape)
    g = 1 + 9 * np.sum(x[1:n]) / (n - 1)
    h = 1 - np.sqrt(f[0] / g)
    f[1] = g * h
    return f

@nb.njit(parallel=True)
def tempFtemp(ftemp_pte, func_F, numPointsEval, pointsToEval):
    for i in nb.prange(numPointsEval):
        ftemp_pte[i] = func_F(pointsToEval[i])
    return ftemp_pte

ftemp_pte = np.zeros((2, 5)).T
pointsToEval = np.zeros((2, 5)).T
numPointsEval = 5
ftemp_pte = tempFtemp(ftemp_pte, func_F, numPointsEval, pointsToEval)
print(ftemp_pte.T)
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thank you very much for your help. The code did not work but i will try to find another alternative to doing so and i will be changing the vstackand hstackoperations for your suggested ones
0

The docs say that first-class function objects can be Numba cfunc compiled functions, JIT compiled functions, and objects that implement the Wrapper Address Protocol.

You can pass a JITted function, as in this simplified example:

@nb.njit
def cos(a):
    return np.cos(a)

@nb.njit(parallel=True)
def tempFtemp(ftemp_pte, func_F, numPointsEval, pointsToEval):
    for i in nb.prange(numPointsEval):
        ftemp_pte[:, i] = func_F(pointsToEval[:, i])
    return ftemp_pte

ftemp_pte = tempFtemp(ftemp_pte, np.cos, numPointsEval, pointsToEval)  # Error
ftemp_pte = tempFtemp(ftemp_pte, cos, numPointsEval, pointsToEval)     # Works

This solves the "non-precise type pyobject" problem, but I have deleted the hstack and vstack operations from the example because they produce their own problems and inefficiencies.

Improve this answer

10 Comments

Thank you for the help, sadly it still does not work. Imma edit my post with the following problem after trying your suggestion
According to the new error message, the problem happens when func_F() is called in tempFunc_F(). This may be because of the function itself or the parameters you are passing to it. You should post the simplest version of func_F() and data passed to it that the problem can be reproduced.
Ok thank for very much. I will edit the post so i provide all the necessary info. Really appreciate your help
I updated the question and gave all the details needed i think. I would trully appreciate your help, been debating with this problem for days
could you give me some feedback please?
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.