Clearing large arrays from memory

The GC isn't interested in reclaiming all available memory. This would be slow and wasteful. Instead the GC's goal is to ensure that enough memory is available to handle any memory allocation requests. As a result there are lower limits as to how much memory you can expect to recover during a GC. 15MB on a 512MB machine isn't much so I wouldn't worry about it. If you had allocated 256MB and the GC didn't recover any of it then I'd blame it on an outstanding reference.

Now as to where the 15MB came from. At least part of it comes from the various objects that are sitting on the callstack (the event arguments for example). When you do a GC all objects that aren't freed move to the next generation. Therefore that event arg you allocated before calling btnClick is now in G1. Therefore it won't get freed by the GC for a long time (if ever) since G0 is generally sufficient. This is the primary reason why you should never force a GC. This applies to any of the other objects (even locals) that you allocated in any previous stack frame. So, taking this example a little further imagine that you modified the code as follows:

private void btnClick ( object sender, EventArgs e )
{
int[] arr = new int[1204];

DoSomething();

arr = null;
}

private void DoSomething ( )

{

GC.Collect();

}

The first function allocated 1MB of memory. The second function forced a GC. Since the array was still in use that 1MB is now moved to G1. Until the GC tries to fill a memory request and can't by dumping G0 objects that 1MB will remain around in memory even though you set it to null. Normally you would expect the G0 objects to go away pretty quickly. That is the way generational garbage collection works.

Michael Taylor - 8/21/07

http://p3net.mvps.org

Although there are many who will argue I have yet to find any circumstance where forcing a GC is ever a reasonable idea. In fact in my code reviews with devs I'll wear them out if I find any forced GCs. The general excuse is that it resolves a memory problem (avoidance = resolved, I guess). A close second is after allocating and deallocating large blocks of memory. In this particular case I could be swayed but the GC knows how much memory is available and it gets paid to ensure that memory is available when you need it so even in this particular case I would say you shouldn't force a GC. IMHO.

Michael Taylor - 8/22/07

http://p3net.mvps.org

The VMM in Windows is responsible for meeting the demands of memory across processes. The GC is only worried about memory within its current process. So yes the memory demands of other processes will ultimately impact the GC. .NET works like most traditional programming languages though in the sense that it pre-allocates a block of memory (the heap) through the VMM for allocating objects. The heap can grow over time as memory permits. I suspect that under the hood it reserves a large amount of memory and then only commits it as needed. As a result it has a lot of space to grow but doesn't eat up all the VM for no reason.

As objects are allocated the CLR takes memory from the heap. When the heap runs out of memory a GC occurs. There could be plenty of physical or virtual memory available on the machine and still a GC would occur. So the fact that you just started a native app that eats up 90% of the memory doesn't mean a GC will be needed for an already running .NET app. However if you then try to allocate a large block of memory in the .NET app a GC would probably be triggered.

The other factor for available memory is memory fragmentation. In a traditional heap objects are allocated and removed all the time. As this occurs holes appear in the heap where memory was freed. When an allocation is needed most heap subsystems start walking the heap until they find a hole big enough for the memory allocation. If you allocate a mixture of large and small objects then there comes a point where you might have a lot of holes in the heap that are not contiguous. At this point any allocation that can't find a hole big enough would result in an OOM exception. This is why you here people complaining all the time about a bug in the runtime because they have 20GB of memory yet they can't allocate a 1MB buffer. Memory fragmentation is tough.

.NET gets around memory fragmentation by compressing the heap after a GC. The heap is compressed so no holes exist. In theory this works great and eliminates the problem. In reality there are a few hitches. Firstly memory that is pinned can not be moved so pinned objects will leave holes in the heap. The second hitch is that very large objects are expensive to move. Objects over a fixed size are allocated on a completely separate heap known as the large object heap (LOH). Objects in the LOH are never compressed so fragmentation is an issue there. The compression thing is the reason why people say that .NET references are pointers but that the pointers can be moved around in memory. During a compression the objects are moved and the references (pointers) have to be updated.

Michael Taylor - 8/22/07

http://p3net.mvps.org