Tracking Down a .NET Windows Service Memory Leak

I was recently involved in tracking down a memory leak in a .NET Windows Service and I wanted to summarize the process we went through. This is really an aggregate of a number of other people’s posts that are referenced at the end of this post.

Firstly, get your machine set up:

1. Install the Debugging Tools for Windows on your workstation.
2. Run WinDbg and set the symbol path (File | Symbol File Path) to SRV*C:\SymbolCache*https://msdl.microsoft.com/download/symbols where C:\SymbolCache is a local directory where you want to cache downloaded symbols.

Secondly, capture a dump from the leaking process (you typically want to do this when the process has been running for a while and has shown signs of the memory leak):

1. Open Task Manager.
2. Right-click the leaking process and choose Create Dump File. Sometime later (depending on the amount of memory on the machine and how much the process is consuming) you will be given the path to the dump file.
3. Move the dump file to your workstation. Dump files can be quite large but they generally compress relatively well if needed.

Thirdly, analyze the dump to find what types are consuming the most space in the process’s memory:

1. Run WinDbg and open the dump file.
2. Load the SOS extensions using the command (don’t leave out the leading period):
   1. .loadby sos mscorwks
3. Dump a list of each type along with it’s total number of instances and size using the command:
   1. !dumpheap -stat
4. Scroll to the bottom of this list to find the types that are consuming the most memory. I usually copy the list into Excel where I can do further sorting or analysis of it.
5. Choose one of the types that is consuming an unexpected amount of memory and dump a list of all of the instances of it using the command:
   1. !dumpheap <type> e.g. !dumpheap System.Byte[]
6. You can then output the object hierarch holding a reference to the object using the command (where 02246da8 is one of the address in the first column of results from the previous command):
   1. !gcroot 02246da8
7. Repeat this for a few different addresses to see if the objects are all being held by one hierarchy or if there are multiple leaks within the application.

This usually gives you enough information to then go back to your source code and investigate the root cause of the memory leak with more direction…

Additional Information

• Use the Microsoft Symbol Server to obtain debug symbol files
• Getting started with WinDbg and managed code
• Debugging memory usage in managed code using WinDbg

Comments

• Anonymous
  September 28, 2015
  Starting from .NET4, use ".loadby sos clr" instead of ".loadby sos mscorwks"