Cython allocate memory for array without using python

Question

I'm trying to Cythonize a simple function and I want to be able to compile it with the nogil statement. What I have (in a jupyter notebook) is:

%%cython -a
import numpy as np
cimport numpy as np
cimport cython
from libc.math cimport exp, pi

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.cdivision(True)
cdef double[:, ::1] _gauss(Py_ssize_t yw, Py_ssize_t xw, double y0, double x0, double sy, double sx):
    """Simple normalized 2D gaussian function for rendering"""
    # for this model, x and y are seperable, so we can generate
    # two gaussians and take the outer product
    cdef double amp = 1 / (2 * pi * sy * sx)

    cdef double[:, ::1] result = np.empty((yw, xw), dtype=np.float64)

    cdef Py_ssize_t x, y

    for y in range(yw):
        for x in range(xw):
            result[y, x] = exp(-((y - y0) / sy) ** 2 / 2 - ((x - x0) / sx) ** 2 / 2) * amp

    return result

def gauss(yw, xw, y0, x0, sy, sx):
    return _gauss(yw, xw, y0, x0, sy, sx)

Which compiles fine. If I change the first cdef line to read:

...
cdef double[:, ::1] _gauss(Py_ssize_t yw, Py_ssize_t xw, double y0, double x0, double sy, double sx) nogil:
...

Then the compilation fails because the first and third cdef lines interact with the python interpreter and I'm not sure why (especially for the first one).

CodeSurgeon · Accepted Answer · 2019-07-12 00:24:38Z

The creation of a typed memoryview object results in interaction with the GIL because it is a python object. For that reason, you would not be able to return a new typed memoryview object from a cdef nogil function. However, there are several ways to get around this limitation with the GIL.

One option would be to just release the GIL within the function. This can be done using a with nogil: block that can be placed around the iterating code. This function would look as follows:

@cython.boundscheck(False)
@cython.wraparound(False)
@cython.cdivision(True)
cdef double[:, ::1] _gauss(Py_ssize_t yw, Py_ssize_t xw, double y0, double x0, double sy, double sx):
    #Do gil-interacting, python stuff here
    cdef double amp = 1 / (2 * pi * sy * sx)
    cdef double[:, ::1] result = np.empty((yw, xw), dtype=np.float64)
    cdef Py_ssize_t x, y

    with nogil:
        #And now basically write C code
        for y in range(yw):
            for x in range(xw):
                result[y, x] = exp(-((y - y0) / sy) ** 2 / 2 - ((x - x0) / sx) ** 2 / 2) * amp
    return result

Another option would be to have the user pass in a numpy array of type double[:, ::1]. That way, memory is not allocated within the function itself. Using that approach, _gauss could be declared cdef nogil.

If you are really concerned about the overhead of allocating memory for the numpy array, you could try to use C-style functions like malloc, calloc, realloc, and free to manage your memory. This pointer could then be cast to an appropriate typed memoryview; however, doing so will invoke the gil upon creation of said memoryview.

Collectives™ on Stack Overflow

Cython allocate memory for array without using python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related