Freigeben über

I have a memory leak!!! What do i do? (defining the "where")

  • Artikel

Not that it matters a tremendous lot but just because it is a big pet-peeve of mine I want to differentiate between a real memory-leak and high memory usage.

A memory leak is when you used some memory and lost the pointer to the allocation so you can no longer de-allocate that memory. If you still have a pointer to it, you have high-memory usage, which might be just as bad in terms of what happens to the process but still differentJ Just to make things simple I’ll use the term memory leak for both of these from now on…

I like to think of solving performance and memory leak issues like peeling an onion. You take away one layer at a time, starting with the most obvious one, and then you define a limit where you determine that the problem is resolved.

There are two types of memory leaks, the gradual memory leak (where memory continuously grows at approximately the same rate) and the sudden memory jump. You troubleshoot them approximately the same way, except for in the latter case you also try to figure out if something out of the ordinary happened at the time of the jump, such as extreme load on the server etc.

So where are you leaking? As I mentioned in my previous post the process’ memory space contains a few different types of “objects” such as threads, manged object heaps, managed loader heap, native heaps, dll’s, and virtual allocations, so a good place to start is by running a performance monitor log with the following counters.

   Process/Virtual Bytes
   Process/Private Bytes
   .net CLR Memory/# Bytes in all Heaps
   .net CLR Memory/% Time in GC
   .net CLR Memory/Large Object Heap size
   .net CLR Loading/Bytes in Loader Heap
   .net CLR Loading/Current Assemblies

The main thing to look for is if private bytes grow at approximately the same rate as virtual bytes and if # Bytes in all Heaps seem to follow the same curve.

If the private bytes keep increasing but # Bytes in all Heaps do not, you’re likely looking at a native memory leak (i.e. you are leaking in a COM component or similar) but if # Bytes in all heaps increase at the same rate as private bytes your leak is likely in managed code.

Much the same way, if you see a steady increase of virtual bytes but your private bytes stay pretty steady, your application probably has a problem where it is reserving a lot of virtual memory that it’s not using.

Bytes in Loader Heap and Current Assemblies should stay fairly constant once the process has started up and all app domains are loaded. If this keeps continuously increasing it is very probable that you have an assembly leak. I will write more about this later but for now, if it is an asp.net application, check that debug=false in all web.configs and that you are not running into one of the following issues.

Memory usage is high when you create several XmlSerializer objects
or
Assembly leak because of script blocks in XSLT's

So now you know approximately where you are leaking, the next step is to find out why and I will talk more about this in future posts but knowing the “where” is a big step.

Comments

  • Anonymous
    February 14, 2006
    did we ever get to the future posts ... "why is memory leaking" .. i am interested in native memory leak

    Thanks,
    Maur

  • Anonymous
    February 14, 2006
    There are various case studys in here showing why memory is leaking, no native memory leak yet though... I haven't really decided on a good one to show native debugging techniques with, but you can start by downloading the IIS Diagnostic Toolkit  and run DebugDiag (a "self diagnosing" tool), which works for other processes than IIS as well

    http://www.microsoft.com/downloads/details.aspx?FamilyID=9BFA49BC-376B-4A54-95AA-73C9156706E7&displaylang=en

    /Tess

  • Anonymous
    April 05, 2006
    The following are based on problems I see people having every day in production ASP.NET systems.
    1....

  • Anonymous
    June 05, 2006
    I am particulary interested in investigating native memory leaks. We have one, and would like some pointers on how to g about pinpointing the source. Is there a posting/blog that deals with memory leaks in unmanaged code?

  • Anonymous
    June 05, 2006
    I am particulary interested in investigating native memory leaks. We have one, and would like some pointers on how to g about pinpointing the source. Is there a posting/blog that deals with memory leaks in unmanaged code?

  • Anonymous
    June 11, 2006
    I will try to post something on that shortly, in the meantime take a look at the iis diagnostic toolkit  (available at http://www.microsoft.com/windowsserver2003/iis/diagnostictools/default.mspx)  specifically the debug diagnostics tool,  that is what I normally use for un-managed leaks.

  • Anonymous
    May 10, 2007
    From the article: "Much the same way, if you see a steady increase of virtual bytes but your private bytes stay pretty steady, your application probably has a problem where it is reserving a lot of virtual memory that it’s not using." Can you give an example of how an application reserves memory, but does not use it?   We have this issue (VM increasing , private bytes steady). We create large arrays, but how is "creating" differentiated from "using"?

  • Anonymous
    May 10, 2007
    The comment has been removed

  • Anonymous
    June 18, 2007
    The following are based on problems I see people having every day in production ASP.NET systems. 1 .

  • Anonymous
    September 24, 2008
    W poniższym artykule wykorzystano materiały znajdujące się w następujących artykułach Tess: http://blogs.msdn.com/tess/archive/2005/11/25/i-have-a-memory-leak-what-do-i-do-defining-the-where.aspx

  • Anonymous
    July 14, 2009
    The comment has been removed

  • Anonymous
    July 14, 2009
    Hi Tess, You have posted information about debugging native and unmanaged code. Can you post the links for debugging techniques for Managed code?

  • Anonymous
    July 21, 2009
    Sunny,  basically this whole blog is about debugging managed code,  for memory issues, check out the post index for the articles under "memory issues"

  • Anonymous
    July 21, 2009
    Sakar, that's some seriously strange output, I am guessing it is some kind of problem with sos...  i wouldnt trust it and it is irrelevant for you as the object you are gcrooting is not showing up here

  • Anonymous
    December 20, 2009
    About memory leaks, for diagnosing a problem I'd like to get memory address value of a handle. Do you know how to get that? Obliviously it's not something one should rely on to remain constant over time, but for debugging/logging it could be useful (for example: log it to see if two threads are dealing with the same object) Since the debugger can retrieve this value, there is probably an interface (ICorDebug?) for it :) Object ^ anObject; as the debugger shows it (like 0x01223AA) it is probably the memory address of the GCHandle of anObject. it is not toString() which equals "System::Object" it is not getHashCode() it is not lAddress in (which probably makes sense, since it's probably the address of the newly allocated handle)    GCHandle lHandle = GCHandle::Alloc(aObject);    int lAddress = GCHandle::ToIntPtr(lHandle).ToInt32();    lHandle.Free(); anObject cannot be casted to System::Int32 using dynamic_cast (exception)

  • Anonymous
    February 25, 2010
    The comment has been removed

  • Anonymous
    February 25, 2010
    KR, Yepp, that still exists in 2.0.  You can have a look at perfmon and the .net clr loading/current assemblies counter to see if you are creating a lot of new assemblies. You might also want to check out http://blogs.msdn.com/tess/archive/2008/03/17/net-debugging-demos-lab-6-memory-leak.aspx  for ideas on debugging it