How to Collect a Memory Dump Using ADPLUS and DebugDiag

A user mode memory dump is a file typically with a DMP extension that contains a portion of virtual memory and CPU registers. Two important items captured by the memory dump include the call stack for each thread and the exception record. This information is captured for the moment in time that the memory dump is taken.

A call stack is the way the operating system keeps track of the currently executing functions within the thread. Each thread within the process will have its own call stack. The call stack is a LIFO queue with the currently executing function being at the top of the stack.

The call stack reported by the tools used to capture the memory dump will only be as good as the symbols that are present. Without symbols for a module the reports output by the tools will show the memory offset from the beginning of the module, or try to get an export from the binaries, in which case it will be from the nearest function call. In the event that the memory dump is captured on a machine that does not have the symbols, or the symbols are not setup correctly they will be recalculated by memory address when it is analyzed on a machine with the proper files.

In order to try and ensure that these reports have the correct call stack it is recommended to set the _NT_SYMBOL_PATH and _NT_ALT_SYMBOL_PATH environment variables. Assuming you want your symbols to be stored on your local hard drive in a directory called symbols you would create a folder structure that looks like c:\symbols\web and c:\symbols\private. If you are using this directory structure you would set your symbol path (in the environment variables listed above) to “c:\symbols\private;srv*c:\symbols\web*https://msdl.microsoft.com/download/symbols;”. What this path means is the private directory will be checked first for symbol files that match, which is where you add your applications files if desired and then check the web directory. In the event that the web directory does not contain the files go to the Microsoft public file server and check there. If the debugger finds the symbols on the public web server it will download them into the web directory.

If the server being debugged is a public facing machine it is recommended to consult your company’s privacy policy to ensure it is ok to put the private symbols for your application in the private directory. The symbols may contain sensitive intellectual property.

You will need to download ADPLUS as part of the Debugging Tools for Windows from https://www.microsoft.com/whdc/default.mspx. The debugging tools are tied to your application and hardware architecture. If you are on a 32-bit machine or your application runs under Windows on Windows (WOW) then you install the 32-bit version otherwise the install the appropriate 64-bit version. The tools can be installed on the machine you want to later analyze the dump on. Use XCOPY to get the files onto the target machine.

To know what parameters to use when capturing the memory dump it is important to know the difference between a first and second chance exception. A first chance exception is one that the application has handled through some mechanism such as structured exception handling (ex. try / catch) and does not terminate the process. A second chance was not handled by the application and made its way to the operating system and terminates the process.

The first thing to do is determine if you want to take a memory dump of a single instance or any instance of the process that generates an exception. If you want to capture the specific instances look in task manager for the Process ID (PID). If this is not displayed you will have to add the column. If you want to capture a dump of any instance of the process that generates an exception use the process name.

Use one of the following commands at an elevated command prompt:

Any instance of the process:

adplus –quiet -crash –pn {Process Name} –FullOnFirst –o “{Output Directory}”

A specific instance of the process:

adplus –quiet -crash –p {Process ID} –FullOnFirst –o “{Output Directory}”

If you receive an error about cscript not being the default scripting engine either make it the default or prepend the command with cscript.

At this point there should be a window displayed in the task bar that when opened is sitting at a debugger prompt with the g command (go) as the final line (or it may already be generating a dump). Adplus will continue to monitor the process for exceptions until it terminates or adplus is shutdown.

The output of the dump file will be placed in a subdirectory labeled with the process name and time. The contents of the output directory will be used in the next posting.

An important thing to understand with adplus is that although there is no install if the process shuts down will not reattach. DebugDiag that is the focus of the remainder of this posting can monitor for the process and reattach when it starts up again. DebugDiag requires an install to add the service. DebugDiag is also limited in that it only supports 32-bit or WOW processes.

DebugDiag can be installed from the msi downloaded from https://www.microsoft.com/downloads and search on DebugDiag

Run DebugDiag.exe and the select rule type dialog is displayed. Choose Crash which is used for both first and second chance exceptions. Select the process from the list of currently executing processes. If only the selected instance is to be monitored check the box labeled “This process instance only”. Select “Full User Dump” as the “Action type for unconfigured first chance exceptions”. Click next and enter a meaningful name for the rule. Clicking next will show the “Activate the rule now” option.

At this point watching the user dump count on the DebugDiag screen will let you know when it is complete. The dump files will be found in the log directory under the DebugDiag install location.

These files will be used in the next posting.