3

I'm attempting to return a small (8 byte) struct by value from a delegate bound to a native function, but am running into the following error when targeting the .NET Framework 2.0 (the code seems to work correctly when targeting 4.0+):

An unhandled exception of type 'System.AccessViolationException' occurred in testclient.exe

Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

I suspect I've messed up the managed type annotations such that the return value isn't being marshalled correctly, but I can't see what I'm doing wrong. Below is the code for a small native test DLL and managed client which reproduces the problem.

C/C++ Win32 (x86) test DLL

//Natural alignment, blittable, sizeof(StatusBlock) == 8
struct StatusBlock{
    std::uint32_t statusA;
    std::uint32_t statusB;
};

/*
 * When compiled this function stores the 64bit return value in the
 * eax:edx register pair as expected.
 */
static StatusBlock __cdecl SomeFunction(std::uint32_t const someVal){
    return StatusBlock{ someVal, 0x1234ABCD };
}

//Exported
extern "C" PVOID __stdcall GetFunctionPointer(){
    return &SomeFunction;
}

C# test client

class Program
{

    //Blittable, Marshal.SizeOf(typeof(StatusBlock)) == 8
    [StructLayout(LayoutKind.Sequential)]
    private struct StatusBlock
    {
        public UInt32 statusA;
        public UInt32 statusB;
    }

    [UnmanagedFunctionPointer(CallingConvention.Cdecl)]
    private delegate StatusBlock SomeFunction(UInt32 someVal);

    [DllImport("testlib.dll",CallingConvention = CallingConvention.StdCall)]
    private static extern IntPtr GetFunctionPointer();

    static void Main(string[] args)
    {
        var fnPtr = GetFunctionPointer();

        System.Diagnostics.Debug.Assert(fnPtr != IntPtr.Zero);

        var someFn = (SomeFunction)Marshal.GetDelegateForFunctionPointer(fnPtr, typeof(SomeFunction));
       
        /* 
         * Crashes here with a System.AccessViolationException when targeting .NET Framework 2.0.
         * Works as expected when targeting .NET Framework 4.0 +
         */
        var statusBlock = someFn(22);
    }
}

It's worth noting that if the return type of the delegate is Uint64 the application works as expected in both the .NET 2.0 and 4.0 case. However, I shouldn't have to do this; StatusBlock should marshal correctly.

Have I just been lucky when targeting .NET 4.0? Any insight into what I'm doing wrong would be much appreciated.

Improve this question
3
  • The abi for struct return values is ill defined. You'd be better off with an out param. Commented May 21, 2015 at 6:03
  • I repro. Looks like the CLR generates a bad stub for the delegate, hard to debug however. You'll need help from Microsoft Support to get ahead, difficult with such an old .NET version. Commented May 21, 2015 at 6:05
  • Damn, .NET's stupid. I think I figured the problem. Commented May 23, 2015 at 19:43

1 Answer 1

Reset to default
3

It is by all means fault in .NET.

tl;dr

.NET Framework 2 generates an incorrect (possibly insecure) stub.

How did I find this out?

I conducted some tests:

  1. Instead of a 8-byte long struct, I used 4-byte long struct. It works!
  2. Instead of using x86, I used x64. It works!

Figuring out that it works in all other cases, I decided to native-debug it with windbg to see where it crashes (because Visual Studio won't allow me to "step in" a native call with the disassembly window).

Guess what I found: enter image description here

The .NET framework generated a stub that is calling memcpy, and it fails when it tries to copy into edi, which at that time had the value 0x16 (==22), which was the parameter sent in the C# code!

So, lets see what would happen if I were to send a valid pointer to the function:

unsafe
{
    long* ptr = &something;
    uint ptr_value = (uint)ptr;
    Console.WriteLine("Pointer address: {0:X}", (long)ptr);

    var statusBlock = someFn(ptr_value);
    Console.WriteLine("A: {0}", statusBlock.statusA);

    Console.WriteLine("B: {0:X}", statusBlock.statusB);
}

Output: (it works and doesn't crash when a valid pointer is given!)

Marshal.SizeOf(typeof(StatusBlock)) = 8
Running .NET Version 2
Pointer address: 49F15C
A: 0
B: 0

So, I conclude this is an unsalvageable problem in .NET Framework 2.

Why is it happening?

When a C function is defined to return a struct larger than 8 bytes, the function shall actually return a pointer to the struct in the local function's stack and the caller should use memcpy to copy it to it's own stack (this is a part of C's specifications and is implemented by the compiler - the programmer simply "returns" a struct and the compiler does the heavy lifting).

However, for 8 bytes (either struct or long long), most C compilers return it in eax:edx. Probably .NET's developers have missed that. The mistake is probably that someone wrote size >= 8 instead size > 8...

Edit: What's worse, it writes the result on the pointer given!

before: 0x1111222244445555
after : 0x1234ABCD007BEF5C

It changes the pointer to be the return value! As you can see, the first dword after the call is is 0x1234ABCD (as in the native DLL) and the second dword is the pointer to the value, i.e the parameter someVal that was given!

It's even more funny, because if you pass a pointer to a StatusBlock struct - it will actually work for this specific case (because the first dword in the return value is used as a pointer)

Solution

Return a long variable and make the struct yourself.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I suspected this was a bug, but thanks for proving it with the excellent analysis! As a work around I've been doing exactly what you suggested; marshal the return value as 64 bit int and provide a conversion constructor on StatusBlock to do the mapping.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.