0

I got a C library storing its results in memory as a structure of arrays and returning a pointer to it. I'd like to create a writeable numpy array using this memory. Unfortunately, the arrays don't have a common type making this rather difficult.

For a classic array of structures, I'd create a equivalent ctypes Structure, init it via from_address() and pass that to numpy.ctypeslib.as_array(). Doing this with a structure of array results in a zero dimensional array, meaning I can't slice it.

Since this basically is just column major instead of row major layout, I hoped that I could just pass a F_CONTINOUS flag to as_array, but it does not have that option. I looked into NumPy's array interface protocol and to me it seems that this use case is not supported, as I can't provide strides for descr.

Is there a way to create a structured array in column-major order or do I need to create a separate array for each column?

I only found one similar question, but in the answer it was only suggested that it may work with advanced dtypes, which I couldn't confirm.

3
  • 1
    Could you return a dictionary of NumPy arrays, one Numpy array for each array in your struct? That could be converted into a Pandas dataframe in a zero-copy way, and Pandas has strong support for heterogeneous types. Commented Nov 12, 2023 at 0:18
  • That's a great idea. Haven't thought about using Pandas, since I'm not that familiar with it, but I'll give it a try. Commented Nov 12, 2023 at 3:01
  • A dataframe can be composed from Series. Each series is a separate numpy array (along with an index array). I doubt if it provides any memory use advantage over a list of such arrays (or equivalently an object dtype array.) Commented Nov 12, 2023 at 18:44

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.