Using DLLs with exports that take C++ classes as their argument is dangerous even in C++. It is impossible to interop with C#. You cannot use the same memory allocator and you can't call the constructor and destructor. Not to mention that your C++ code isn't valid, it doesn't actually return the string.
Use a C string instead. Make it look like this:
extern "C" __declspec(dllexport)
void __stdcall XmlDecrypt(const wchar_t* inputFile, wchar_t* msg, int msgLen)
{
wcscpy_s(msg, msgLen, L"haha");
}
[DllImport("libXmlEncDll.dll", CharSet = CharSet.Auto)]
public static extern void XmlDecrypt(string inputFile, StringBuilder msg, int msgLen)
...
StringBuilder msg = new StringBuilder(666);
XmlDecrypt(someFile, msg, msg.Capacity);
string decryptedText = msg.ToString();
Some notes with these code snippets:
- The __stdcall declarator selects the standard calling convention for DLL exports so that you don't have to use the CallingConvention property in the [DllImport] attribute.
- The C++ code uses wchar_t, suitable to store Unicode strings. This prevents loss of data when the text you get from the XML file is converted to 8-bit characters, a lossy conversion.
- Selecting the proper msgLen argument is important to keep this code reliable. Do not omit it, the C++ code will destroy the garbage collected heap if it overflows the "msg" buffer.
- If you really do need to use std::string then you'll need to write a ref class wrapper in the C++/CLI language so it can convert from System.String to std::string.