How to P/Invoke a function that allocates and returns pointer to new array

Question

I have a C function that I would like to call from C# program. Function compresses inputted byte array and outputs new compressed array. It looks like this:

extern __declspec(dllexport) int Compress(chandle handle, 
unsigned char *inputBuf, unsigned char **outputBuf, unsigned long *outputSize);

I have translated it into this on C# part. But output I get is array with one item.

[DllImport("compresslib.dll", CallingConvention = CallingConvention.Cdecl)]
internal extern static int Compress(IntPtr handle, byte[] input, out byte[] output, out uint outputSize);

What should I do to get this working?

Here is working code I was able to write with the help of Hans Passant

[DllImport("compresslib.dll", CallingConvention = CallingConvention.Cdecl)]
internal extern static int Compress(IntPtr handle, byte[] input, out IntPtr output, out uint outputSize);

// and this is how i call it 
byte[] outputData;
int outputDataSize;
IntPtr outputDataP = IntPtr.Zero;
try
{
    int success = NativeMethods.Compress(handle,
    inputData, out outputDataP, out outputDataSize);
    if (success == -1)
    {
        throw new Exception("Compression failed.");
    }
    outputData = new byte[outputDataSize];
    Marshal.Copy(outputDataP , outputData , 0, (int)outputDataSize);
}
finally
{
    if (outputDataP != IntPtr.Zero)
    NativeMethods.tjFree(outputDataP);// release unmanaged buffer
}

return outputData ;

Hans Passant · Accepted Answer · 2014-02-14 19:12:13Z

2

  ..., unsigned char **outputBuf, ...

There is a pretty serious problem with this function, it cannot reliably be called from a C program either. The caller needs to release the output buffer after using it. That requires using the exact same allocator as was used in your C code. That's very hard to guarantee in a C program and often goes wrong badly. Like when the user of your DLL doesn't use the exact same compiler version as you used. He'll use a different version of the C runtime library, one that uses his own heap. So cannot possibly release the buffer since he doesn't have a handle to the heap that you used.

It becomes next to impossible when you pinvoke, the CLR of course has absolutely no idea what C runtime version you used and is guaranteed to not use the same one that you used since it has its own private copy.

The reason you are just getting one byte is related to this problem, the pinvoke marshaller doesn't know how large the array is since it didn't create the array. That's fixable you need to apply tbe [MarshalAs(UnmanagedType.LPArray), SizeParamIndex = 3] attribute to the parameter. That tells the pinvoke marshaller that the 4th argument contains the size of the array.

You'll need to address the memory management problem, you can't leave it the way it is since you'll leak memory badly on XP and hard-crash on Vista and higher when the pinvoke marshaller tries to release the array. You need to alter the C code to use memory for the returned output buffer that is allocated on a known heap. Which requires using CoTaskMemAlloc().

Another possible fix is that you export a function that allows the caller to release the buffer. In which case you should declare the argument out IntPtr and marshal yourself with, say, Marshal.Copy(), then release the buffer with your added function.

Or a completely different way to use the function, like having Compress() only compress but not return the data yet. With extra functions that the client code can use to discover the required buffer size and to obtain a copy of the data.

answered Feb 14, 2014 at 19:12

Hans Passant

946k151 gold badges1.8k silver badges2.6k bronze badges

Sign up to request clarification or add additional context in comments.

9 Comments

zigzag Over a year ago

Okay so if I understand correctly, the output array that is allocated in the unmanaged code can not be delocated from managed code? I looked in the documentation of the unmanaged library and found two things there related to memory management of outputBuf array. see below...

zigzag Over a year ago

1st: regarding allocation of outputBuf it says I have three options: #1: pre-allocate the output buffer with an arbitrary size using tjAlloc() and let library grow the buffer as needed #2: set *outputBuf to NULL to tell library to allocate the buffer for you (this is I guess what I am doing) #3: pre-allocate the buffer to a "worst case" size determined by calling tjBufSize(). This should ensure that the buffer never has to be re-allocated (setting TJFLAG_NOREALLOC guarantees this. There is also flags parameter that I omitted in code sample I posted to make it more straight to the point.)

zigzag Over a year ago

2nd: there is a function in a library DLLEXPORT void DLLCALL tjFree (unsigned char* buffer) Documentations says: Free an buffer previously allocated by library. If I understand correctly I cannot use [MarshalAs(UnmanagedType.LPArray), SizeParamIndex = 3] because program will crash when GC tries to collect that buffer that was allocated in unmanaged code. What are my options then? To copy array and then destroy original output buffer by calling tjFree? How will this affect performance? Is there any other way to handle this?

Hans Passant Over a year ago

Smells like a solution, a rjRelease() function would be more inspiring. It isn't very clear to me how you are supposed to use these functions. Talk to the vendor or author to find more help with that.

Hans Passant Over a year ago

Okay, so you do have a release function. Then use it as I described, declare the argument as ref IntPtr and initialize the IntPtr to IntPtr.Zero before the call. And Marshal.Copy + tjFree afterwards.

|

Collectives™ on Stack Overflow

How to P/Invoke a function that allocates and returns pointer to new array

1 Answer 1

9 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

9 Comments

Your Answer

Sign up or log in

Post as a guest

Related