Troubleshooting Pool Leaks Part 4 – Debugging Multiple Users for a Tag

In our previous articles we discussed various techniques for identifying a pool memory leak and narrowing the scope of the leak to an individual pool tag.  Knowing the leaking pool tag is often sufficient to identify the cause of the problem and find a solution.  However, there may be a scenario where multiple drivers use the same pool tag (such as DDK) or when one driver uses the same tag in multiple places.  In this scenario you will need more information to identify the source of the leak.  In our next several articles we will present techniques to get this information.

 

This article will present a basic technique where we modify each pool tag to identify what code in which driver is allocating the memory that gets leaked.

 

This technique requires a live debug of the problematic system.  There are many resources with steps for how to configure a system for a live debug.  The debugging tools have instructions in the debugger.chm help file, under Debugging Tools for Windows\Debuggers\Installation and Setup\Kernel-Mode Setup.

 

 

Using the same technique as in Part 3, identify where the tag in question is used.

 

0: kd> !for_each_module s -a @#Base @#End "Leak"

fffff880`0496e3aa  4c 65 61 6b 3b c1 0f 42-c1 41 8d 49 fd 8b d0 ff  Leak;..B.A.I....

fffff880`0496e621  4c 65 61 6b 3b c1 0f 42-c1 33 c9 8b d0 ff 15 cc  Leak;..B.3......

 

Next, edit each instance so that they are unique.  The ASCII code for numeral 1 is 0x31, and the codes for each numeral increase sequentially.  Using this information edit each tag to be Lea1, Lea2, etc.

 

0: kd> eb fffff880`0496e3aa+3 31

0: kd> eb fffff880`0496e621+3 32

 

Confirm your edits resulted in the expected tags using the dc command.

 

0: kd> dc fffff880`0496e3aa l1

fffff880`0496e3aa  3161654c                             Lea1

0: kd> dc fffff880`0496e621 l1

fffff880`0496e621  3261654c                             Lea2

 

Now wait for the leak to happen and repeat the steps from Part 3 to identify which of the tags is leaked.  This tells you what code allocates the memory that gets leaked.  Below we can see that Lea2 is the tag being leaked.

 

0: kd> !poolused /t5 2

..

Sorting by NonPaged Pool Consumed

 

               NonPaged                  Paged

Tag     Allocs         Used     Allocs         Used

 

Lea2 257 263168000          0            0  UNKNOWN pooltag 'Lea2', please update pooltag.txt

nVsC       664      1531552          0            0  UNKNOWN pooltag 'nVsC', please update pooltag.txt

netv      4369      1172224          1          144  UNKNOWN pooltag 'netv', please update pooltag.txt

Leak 1 1024000          0            0  UNKNOWN pooltag 'Leak', please update pooltag.txt

EtwB        94       945136          4       163840  Etw Buffer, Binary: nt!etw

 

TOTAL     41296    281814544      44077     68102368

 

Knowing what code allocates the leaked pool may be very valuable to a driver developer who needs to narrow the scope of the problem.  Often this information is sufficient for a developer to code review the use of this memory and identify why it would be leaked.

 

There are times when more information is needed to determine the cause of the leak.  A developer may need the call stacks of memory being allocated and freed.  We will capture this information in Part 5.