Functions to initialize and load a SharedMemory array

Question

How do I write functions to be imported in other parts of the code so I can initialize and/or load a shared array? I don't need it to be super fancy atm so I'm trying this simple approach:

# functions.py

import numpy as np
from multiprocessing.shared_memory import SharedMemory


def init_shared_array(shape, dtype, name):
    zeros = np.zeros(shape, dtype=dtype)
    shm = SharedMemory(create=True, size=zeros.nbytes, name=name)
    shared_array = np.ndarray(shape, dtype=dtype, buffer=shm.buf)
    shared_array[:] = zeros[:]
    return name

def load_shared_array(shape, dtype, name):
    shm = SharedMemory(create=False, name=name)
    shared_array = np.ndarray(shape, dtype=dtype, buffer=shm.buf)
    return shared_array

# main.py

from functions import init_shared_array, load_shared_array

shape = (1,)
dtype = int
name = 'MySharedArray'

init_shared_array(shape, dtype, name)

shared_array = load_shared_array(shape, dtype, name)  # I'll run this line on other files to

I wasn't expecting it to fail, but it does when loading the array, with a warning UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up followed by an error Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)

One curious thing I noticed is that if instead of calling the load_shared_array method I just replace it with its code, then the program works:

# main.py

import numpy as np
from multiprocessing.shared_memory import SharedMemory
from functions import init_shared_array, load_shared_array

shape = (1,)
dtype = int
name = 'MySharedArray'

init_shared_array(shape, dtype, name)

shm = SharedMemory(create=False, name=name)
shared_array = np.ndarray(shape, dtype=dtype, buffer=shm.buf)  # works well

L.B. · Accepted Answer · 2022-12-07 18:47:25Z

0

Found out the answer on Segmentation Fault using Python Shared Memory. Turns out I needed to keep a reference of shm and return it together with shared_array inside load_shared_array:

def load_shared_array(shape, dtype, name):
    shm = SharedMemory(create=False, name=name)
    shared_array = np.ndarray(shape, dtype=dtype, buffer=shm.buf)
    return shared_array, shm

answered Dec 7, 2022 at 18:47

L.B.

4563 silver badges14 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Functions to initialize and load a SharedMemory array

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related