Access memory error after unmanaged interop in Vista.

I have got a Access memeroy error when get a memory be malloced from unmage code to manage code in Vista, but it works well in XP SP2.

My manage code are

public partial class Form1 : Form

{

private void button1_Click(object sender, EventArgs e)

{

string name = RunInterop.GetStringFromUnmanage();

}

}

Interop code

public static class RunInterop

{

const string RunDllFileName = "Unmanage.dll";

[DllImport(RunDllFileName)]

public static extern string GetStringFromUnmanage();

}

Unmage code

extern "C"

{

__declspec(dllexport) char* __stdcall GetStringFromUnmanage()

{

char *lpchar = (char *)malloc(strlen("test) + 1);

memcpy(lpchar," test ",strlen("test "));

return lpchar;

}

}

There have a crash in the ntdll!RtlpLowFragHeapFree function in Vista, but it works well in Vista. I am sure lots of developers have meet this error, any suggestion I know it is ok if using StringBuilder to get the string from the unmanage code, but I am

0:000> kb

ChildEBP RetAddr Args to Child

001ce4d4 770a18c3 001ce814 001ce814 00db25f3 ntdll!RtlpLowFragHeapFree+0x31

001ce4e8 76677a7e 00200000 00000000 04183398 ntdll!RtlFreeHeap+0x101

001ce4fc 7653d5d6 00200000 00000000 041833a0 KERNEL32!HeapFree+0x14

001ce510 7653de31 7660e6fc 041833a0 001ce5f4 ole32!CRetailMalloc_Free+0x1c

001ce520 7a0b425b 041833a0 001ce814 7a0b7ab9 ole32!CoTaskMemFree+0x13

001ce52c 7a0b7ab9 001ce7bc 001ce804 f9131f59 mscorwks!DefaultMarshalOverrides<CSTRMarshalerBase>::ReturnCLRFromNative+0x33

001ce768 79ea35c6 00db25f0 001ce804 001ce7c0 mscorwks!RunML+0x77a

001ce7e8 79ea34ba 00245360 001ce894 00757030 mscorwks!NDirectGenericStubPostCall+0x194

001ce850 00db2647 00245360 001ce894 fca439b0 mscorwks!NDirectGenericStubReturnFromCall+0x1f

001ce87c 00d20303 014a0fb8 0148e8c4 0000004b CLRStub[StubLinkStub]@db2647

001ce8d4 7b0693eb 00000000 00000000 00000000 Manage2Un!Manage2Un.Form1.button1_Click(System.Object, System.EventArgs)+0x33 [D:\Fuhaitao\Manage2Un\Manage2Un\Form1.cs @ 23]

001ce96c 7b1065e9 00000000 00000000 00000000

Hi,

When you are allocating a string buffer in unmanaged code and return it to managed code, .NET will try to free your memory allocated in GetStringFromUnmanage() by calling CoTaskMemFree, which leads to a crash because it was allocated by malloc, and is in a different heap than what CoTaskMemFree is using. So you'll have to use CoTaskMemAlloc instead of malloc/new in this case.

The reason why only Vista is failing is most probably because vista has become more sensitive (or robust) to such kind of allocation differences. In fact, you should always rely on CoTaskMemAlloc/CoTaskMemFree, since that's exactly what .NET/CLR is using.

--

Yi Zhang, Microsoft

This posting is provided "AS IS" with no warranties, and confers no rights.

further question, why .NET/CLR was trying to free the memory allocated in GetStringFromUnmamaged() Is it always trying to do so

What I thought is .NET/CLR just needs to make a copy at managed heap, rather than worry about how to free memory in GetStringFromUnmamaged().

If I just change the return type from string to intptr for public static extern string GetStringFromUnmanage(), is there any potential issues It worked, at least.

In this case, since unmanaged function GetStringFromUnmanaged is returning a pointer to a block of memory, the piece of memory by convention is now owned by the CLR. After CLR makes a copy of that memory, it decides that it no longer needs the memory any more (since it already has a copy) and frees it by CoTaskMemoryFree (or SysFreeString in the case of BSTR. SysFreeString basically does the same thing as CoTaskMemoryFree, except that it needs to do some pointer adjustment for the BSTR pointer).

--

Yi Zhang, Microsoft

This posting is provided "AS IS" with no warranties, and confers no rights.

Huh

I don't think so. The CLR always attempts to free that memory allocated at unmanaged world, except a block of memory referenced by a IntPtr type. MSDN did say that.

Anyway, I just took a quick demo and that proved.

It worked at Vista, and if I just passed a string back, evil there!!!

To nobugz:

In some case the CLR will free the memory, when the function signature and marshalling attribute tells the CLR to do so. When the function is written this way: char *GetStringFromUnmanaged(), by convention it is expecting the caller to free the memory. The CLR here is following the COM convention: if you are returning a pointer to something new in a function as out/return value, then the memory that the pointer points to is owned by the caller, and should be released by the caller. If the function doesn't follow this convention, then the function should be re-written or you should do the marshalling manually.

Of course, in pure unmanaged world, what you said is correct. The caller should not expect the returned buffer to be always allocated in heap. To avoid any confusion about the ownership of the memory, the function should be rewritten in the way you suggested: int GetStringFromUnmange(char *buffer, int buflen);

To summarize, by letting CLR marshalling the return value, the marshalling rule must be followed, otherwise you should marshal it by your own.

To Buddhist:

I believe inside your GetStringFromCoAlloc function, it allocates a string by using CoTaskMemAlloc, right Otherwise it won't work in Vista.

To deepnight,

In this case, if the CLR doesn't free the memory, yes, there will be a leak. Using StringBuilder in the argument as [out] would be another different story though.

--

Yi Zhang, Microsoft

This posting is provided "AS IS" with no warranties, and confers no rights.

Right, I did a CotTaskMemAlloc :-)

Using StringBuilder in the argument as [out] would be another different story though.

Could you please talk it a little bit more

I tried this way, and it failed at Vista.

Hi buddhist,

Could you please elaborate on how you were using StringBuilder in your case Some code would be nice

__declspec(dllexport) void __stdcall GetStringFromStrBuilder(char** buffer)
{

buffer = (char**)malloc(1000);

*buffer = malloc(1000);

memset(*buffer, 'a', 1000);
*buffer[999] = '\0';
}

[DllImport("UnmanagedStrAlloc.dll", CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Ansi)]
public extern static void GetStringFromStrBuilder(out StringBuilder str);

......

StringBuilder sb = null;
GetStringFromStrBuilder(out sb);
Console.WriteLine(sb.ToString());

Looks like it's same problem and same reason compared with using String. Right

How do you think about safely using StringBuilder

To buddhist,

There are 3 places in your code that need to be fixed:

1. Similar problem: The memory allocation should be done using CoTaskMemAlloc

2. buffer = (char **)malloc(1000) is incorrect. You should allocate 1000 bytes of memory and assign it to *buffer

3. memset is actually assigning 1000 integer 97, which is a buffer overflow

So, You native function should be written like the following:

In your case it doesn't make much difference whether you are using a string or StringBuilder.

--

Yi Zhang, Microsoft

This posting is provided "AS IS" with no warranties, and confers no rights.

Hi nobugz,

To nobugz,

In Adam Nathan's ".NET and COM - The complete Interoperability Guild", he point out that CLR will do a CoTaskMemFree. And from my debugging experience, at least, CLR on Vista does it.

Deepnight