Python: ctypes + C malloc error. C memory problem or Python/ctypes problem?

Question

everyone. I'm having a memory allocation error using ctypes and a C code. I'm wondering if the memory problem is inside of C, or caused by an improper use of ctypes. The memory error is

python(79698) malloc: * error for object 0x15627ac08: incorrect checksum for freed object >- object was probably modified after being freed. * set a breakpoint in malloc_error_break to debug Abort

or

python(76110,0x7fff70062ca0) malloc: * error for object 0x7d807e1078907df: pointer being >freed was not allocated * set a breakpoint in malloc_error_break to debug Abort

depending on how I write the "free_some_memory" part of the code below. I won't post a bunch of C code here, but assuming I've written the C code correctly, does my ctypes interface look correct?

Also, the failures I'm experiencing don't always happen in the same place in my scripts, for a given version of "free_some_memory". Is it possible that my C code memory management is written poorly such that Python memory management may or may not bungle up (de)allocations?

Dunno if this is important, but I'm working on a Mac, Snow Leopard.
Any help you can offer would be very appreciated.

Best, jkmacc

My C code looks like:

/**** cfunction.c ****/
int cfun (double *y, char datafile[1024], int byteoffset, int num )
{
  allocate_some_memory;
  do_stuff;

  if ( error_condition )
  {
    free_some_memory;
    return ( -1 );
  }

  fill_y_here;
  free_some_memory;
  return ( 0 );
}

My ctypes wrapper looks like:

#pyfunction.py

import ctypes as C
import numpy as np

lib = C.CDLL('mylib.dylib')

def pyfun(DATAFILE, BYTEOFFSET, NUM):
    """
    DATAFILE: a string file name
    BYTEOFFSET: an integer
    NUM: an integer
    """

    #return integer success/failure flag
    lib.cfun.restype = C.c_int 

    #array memory to be filled inside of C
    Y = np.empty(NUM,dtype='double',order='C')

    cDATAFILE = C.create_string_buffer(DATAFILE,1024) 
    cBYTEOFFSET = C.c_int(int(BYTEOFFSET))
    cNUM = C.c_int(int(NUM))

    flag = lib.cfun(Y.ctypes.data_as(C.POINTER(C.c_double)), \ 
               cDATAFILE,cBYTEOFFSET,cNUM)

    if flag == -1:
        print("Something went wrong.")
        return -1
    else:
        return Y

You need to post the actual code for cfunction.c, not pseudo-code. — Heath Hunnicutt
– Heath Hunnicutt, Commented Jun 3, 2011 at 17:51
The Python code is fine. Post the C code. Note that you can pass strings, ints, pointers directly to C functions without putting them in ctypes types (pass DATAFILE, BYTEOFFSET, NUM directly without the cDATAFILE, etc. translations. — Mark Tolonen
– Mark Tolonen, Commented Jun 4, 2011 at 6:10
Thanks for looking this over. I'll produce a minimal example of the C that generates the problem, as I can't post the full code. — jkmacc
– jkmacc, Commented Jun 6, 2011 at 14:27
I'm getting this same issue using the Python GDLA library on OS X with brew installed gdal and pip installed gdal 1.10.1/1.10.0 Any ideas? — James Mills
– James Mills, Commented Nov 29, 2013 at 3:51

Michael Fox · Accepted Answer · 2011-08-26 21:37:29Z

4

I had this problem and here is what I figured out:

If you map something with ctypes and then free it you will get an error when the ctypes object is destroyed. So, before you call free(), you need to make sure that there are no remaining references to it in the python code.

So, your C code will be like:

char* p;
char* allocate() {
 p = asprintf("hello world");
 return(p)
}
void freep() {
 free(p);
}

And your Python code will be like:

allocate = clib.allocate
allocate.restype = c_char_p()
p = allocate()
print p
# This is the key:
del(p)
clib.freep()

If you want to see an error, just omit del(p)

answered Aug 26, 2011 at 21:37

Michael Fox

3,6381 gold badge19 silver badges29 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Python: ctypes + C malloc error. C memory problem or Python/ctypes problem?

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related