ASP.NET Tips: What to gather to troubleshoot – part 2 – High Memory

Identifying a Memory leak

A Memory Leak in an IIS process (INETINFO.EXE, DLLHOST.EXE, W3WP.EXE) occurs when Memory Usage in Task Manager continues to consume more than 50% of the physical RAM until running out of system memory or until the process stops functioning.

On heavily-used sites, a continuous growth in memory over the first 24 hours of use is normal. IIS caches much of its data and the TTL on the cache is 24 hours. Since IIS typically uses 50% of available physical RAM (per IIS Process; not the total of all IIS Processes), use this as a guide to identify a memory problem. The steps below will help troubleshoot a memory leak by gathering a memory dump of the leaking process.

Important Questions

• How much RAM is on the machine?
• How much RAM does the leaking IIS process take?
• How long does it take to reproduce?

Troubleshooting Steps

1. Install the Debug Diagnostic tool

https://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1286

The default install path is C:\Program Files\DebugDiag and can be changed during the installation.

2. Set up Performance Monitor Logging before the issue occurs:

a) Open DebugDiag (Start -> Programs –> Debug Diagnostics Tool 1.1)

b) Go to the Tools menu -> Options and Settings

c) Select the Performance Log tab

d) Click "Enable Performance Counter Data Logging"

e) Click OK

Note: The Data Sampling Interval and time to start monitoring is subjective to when the memory leak reproduces. Due to the log size, work with your Support Professional on a solution where we get the information needed while not overwhelming the server.

3. Create a Memory and Handle Leak Rule with the following steps:

a) Open DebugDiag (Start -> Programs -> Debug Diagnostics Tool 1.1)

b) Select "Memory and Handle Leak" and click Next

c) Select the leaking Process and click Next

d) In “Configure Tracking Duration”, a Warm-Up time can be specified but if the issue reproduces immediately, select “Start memory tracking immediately”. For “Tracking Time”, specify the time it will take to reproduce the issue. At the end of the specified time, a dump will be created. Make sure “Auto-create a crash rule” is checked.

e) Click Next

f) Click Finish for "Rule Name". The "Userdump Location" can be changed here.

Notice the Status is Active. The Userdump Count will increase each time a dump file is created.

4. Get the Data Manually if you are not sure when the leak will occur. In DebugDiag, go to the Processes tab, right-click the leaking process and select “Monitor For Leaks”. When the process gets up to 60%-80% of RAM, right-click the leaking process and select "Create Full Userdump”.

5. Stop PerfMon Logging after the dump has finished:

a) Open DebugDiag (Start -> Programs –> Debug Diagnostics Tool 1.1 )

b) Go to the Tools menu -> Options and Settings

c) Select the Performance Log tab

d) Click "Disable Performance Counter Data Logging"

e) Click OK

6. Analyze the Dump by selecting the "Advanced Analysis" tab and clicking "Add Data Files". When the .dmp is added, select the "Memory Pressure Analysis” script and click "Start Analysis". When finished, a report (.mht) will be created in C:\Program Files\DebugDiag\Reports and displayed in Internet Explorer with the results and recommendation. If using custom DLL’s, the Symbol path (Tools menu -> Options and Settings -> Symbol Search Path) to the custom PDB files can be added.

Comments

• Anonymous
  May 05, 2008
  The comment has been removed

• Anonymous
  May 05, 2008
  .NET A-Star Pathfinding in C# JIT Optimizations Creating portable ASP.NET applications that work on IIS

• Anonymous
  May 19, 2008
  The comment has been removed

• Anonymous
  May 21, 2008
  So now that I am done with all the individual posts, I thought I would wrap them all together.