cython memory view of c array of structure

Question

I'm trying to use cython to extract data from some binary files but I'm coming up against a problem and my knowledge of cython/c is not able to solve it.

The problem: Attempting to make a memoryview of my c array of structures causes a segmentation fault.

What I'm doing: I have a binary file full of data, the first entry is an unsigned int giving the number of snapshots (Snapshot_Counter) contained in the file. The next entries are in the format of Snapshot_Information and this block is repeated Snapshot_Counter times. If I read this in a loop block by block, storing the results in a list it works and I get the right numbers out.

I want to avoid this loop by reading in the array all at once. I'm trying to extract the Snapshot_Information format information into an array, which I think I have managed to allocate properly. However I can't work out how to see into the array to check if I've got the right data in there. I have a feeling it is wrong as the number I'm extracting after this block is wrong.

What I would like to know: How can I see into this array with a memory view? Am I allocating the memory properly? Is there another way of doing this without a c array?

The code:

in extractiontest.pxd

cdef unsigned int Snapshot_Counter

cdef packed struct Snapshot_Information:
     signed int Days
     unsigned int Seconds
     ...more type definitions.


in extractiontest.pyx

rm = fopen("/filepath")

fread(&Snapshot_Counter,4,1,rm) #works fine
cdef Snapshot_Information *snap_info_array = <Snapshot_Information *>malloc(Snapshot_Counter*sizeof(Snapshot_Information))
fread(&snap_info_array, sizeof(Snapshot_Information), Snapshot_Counter, rm)

cdef Snapshot_Information[:] snap_arr_view = <Snapshot_Information[:-1]>snap_info_array

DavidW · Accepted Answer · 2016-12-08 15:10:07Z

2

You've got to tell it the length: it can't work out a length from a malloced array:

cdef Snapshot_Information[:] snap_arr_view = <Snapshot_Information[:Snapshot_Counter]>snap_info_array

I suspect it interprets -1 as a huge positive integer (hence the segmentation fault)

On closer look you also have another bug in your code:

fread(<void*>snap_info_array, sizeof(Snapshot_Information), Snapshot_Counter, rm)

I've removed the & in front of snap_info_array and added a cast to void* (I'm not sure if you need the cast though). You were telling fread to write to the place where the address of your array was kept, rather than into your array.

edited Dec 8, 2016 at 15:10

answered Dec 8, 2016 at 14:37

DavidW

31.2k7 gold badges64 silver badges99 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

R Davies Over a year ago

Thanks DavidW, I have managed to get it to extract the memoryview, and then using np.asarray(snap_array_view) I can see it is Snapshot_Counter long. Sadly if I try to print any of it, I get a seg fault! So I suspect I've not got to the bottom of the problem.

DavidW Over a year ago

@RDavies See edit. I wouldn't like to promise that's the only other issue, but it certainly is an issue

R Davies Over a year ago

excellent, that seems to have fixed it. I'm new to the pointers/address thing. Thankyou for your clear explanation. It does work both with <void *> and nothing in front of the array name.

Collectives™ on Stack Overflow

cython memory view of c array of structure

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related