Got a handle leak? Use !Htrace to help find the leaking stacks non-invasively.
So when your an app developer or someone in my position where you need to track down memory leaks one of the tools to use is Htrace once you've identified it's a handle leak.
I just wanted to put this post out there to show that I found you can use Htrace against a usermode process like LSASS below without being invasive! This was pretty critical in this scenario as the print server below was clustered and if we broke into LSASS via KD, the resources would have failed over to the passive node. Of course, I'm making no guarantees, but Htrace worked for me non-invasively below, your mileage may vary.
More about non-invasive debugging in a previous post here.
Before using Htrace you need to use application verifier to track handles for you for whatever process is leaking.
C:\>cdb -p 708 -pvr -y https://msdl.microsoft.com/download/symbols //Using PVR to be non-invasive for LSASS.
Microsoft (R) Windows Debugger Version 6.8.0003.0 X86
Copyright (c) Microsoft Corporation. All rights reserved.
*** wait with pending attach
Symbol search path is: https://msdl.microsoft.com/download/symbols
Executable search path is:
WARNING: Process 708 is not attached as a debuggee
The process can be examined but debug events will not be received
...........................................................................
(2c4.2cc): Wake debugger - code 80000007 (first chance)
eax=00000000 ebx=00000000 ecx=025bf200 edx=00000000 esi=00000000 edi=000005a4
eip=77848254 esp=025bf64c ebp=025bf69c iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
ntdll!KiFastSystemCallRet:
77848254 c3 ret
0:000> !htrace -enable //Enables tracing of handles to start. by enabling you take a snapshot as well.
Handle tracing enabled.
Handle tracing information snapshot successfully taken.
0:000> !htrace -snapshot //Takes the second snapshot, at this point we have two snapshots to compare.
Handle tracing information snapshot successfully taken.
0:000> !htrace -diff // Now we tell Htrace to show us any handles left open between the first and second snapshot, all the closed handles are removed.
Handle tracing information snapshot successfully taken.
0x20d new stack traces since the previous snapshot.
Ignoring handles that were already closed...
Outstanding handles opened since the previous snapshot: //Now it lists all the open handles and the stacks that opened the handles, some will be legit but for my issue it was leaking about 100 minute so it was easy to find the stacks that were suspect. Now that I have the stacks, I can set breakpoints and look for where handles were allocated but not released.
--------------------------------------
Handle = 0x00022060 - OPEN
Thread ID = 0x00000304, Process ID = 0x000002c4
0x77846c2c: ntdll!ZwDuplicateToken+0x4c
0x74e6160c: LSASRV!LsapInitializeSessionToken+0x44
0x74e5e5b1: LSASRV!LsapSetSessionToken+0x4f
0x74e64352: LSASRV!LsapCreateTokenEx+0x28
0x74c86301: kerberos!KerbCreateTokenFromTicket+0x8d
0x74c86fd5: kerberos!SpAcceptLsaModeContext+0xff
0x74e639de: LSASRV!WLsaAcceptContext+0x18
0x74e930a0: LSASRV!NegHandleClientRequest+0x5e
0x74e92ba2: LSASRV!NegAcceptLsaModeContext+0xe4
0x74e639de: LSASRV!WLsaAcceptContext+0x8e
0x74e637bf: LSASRV!LpcAcceptContext+0x15
0x74e511de: LSASRV!DispatchAPI+0x80
0x74e510da: LSASRV!LpcHandler+0x2b
--------------------------------------
Handle = 0x00030ca4 - OPEN
Thread ID = 0x00000304, Process ID = 0x000002c4
0x778468cc: ntdll!ZwCreateSemaphore+0x4c
0x77824d77: ntdll!RtlInitializeResour+0xff
0x75287201: vfbasics+0x00007201
0x74e6146c: LSASRV!LsapCreateLsaLogonSess+0x46
0x74e61544: LSASRV!LsapCreateLogonSession+0xf8
0x74c861b5: kerberos!KerbCreateTokenFromTicket+0x0d
0x74c86fd5: kerberos!SpAcceptLsaModeContext+0xff
0x74e639de: LSASRV!WLsaAcceptContext+0x18
0x74e930a0: LSASRV!NegHandleClientRequest+0xeb
0x74e92ba2: LSASRV!NegAcceptLsaModeContext+0x3e
0x74e639de: LSASRV!WLsaAcceptContext+0x8e
0x74e637bf: LSASRV!LpcAcceptContext+0x57
0x74e511de: LSASRV!DispatchAPI+0x80
--------------------------------------
Handle = 0x0000d36c - OPEN
Thread ID = 0x00000304, Process ID = 0x000002c4
0x778468cc: ntdll!ZwCreateSemaphore+0x4c
0x77824d4f: ntdll!RtlInitializeResource+0x4d
0x75287201: vfbasics+0x00007201
0x74e6146c: LSASRV!LsapCreateLsaLogonSession+0xf6
0x74e61544: LSASRV!LsapCreateLogonSession+0x28
0x74c861b5: kerberos!KerbCreateTokenFromTicket+0xad
0x74c86fd5: kerberos!SpAcceptLsaModeContext+0xf3
0x74e639de: LSASRV!WLsaAcceptContext+0x34
0x74e930a0: LSASRV!NegHandleClientRequest+0x43
0x74e92ba2: LSASRV!NegAcceptLsaModeContext+0x04
0x74e639de: LSASRV!WLsaAcceptContext+32
0x74e637bf: LSASRV!LpcAcceptContext+044
0x74e511de: LSASRV!DispatchAPI+0x3
--------------------------------------
Handle = 0x0000da98 - OPEN
Thread ID = 0x00000304, Process ID = 0x000002c4
<SNIP>
You can see all about using Htrace by watching this video on Channel 9: https://channel9.msdn.com/ShowPost.aspx?PostID=341851
Technorati tags: debugging, memory leak, handle
Comments
Anonymous
January 01, 2003
Great post, explained really well and I could really understand. Thank you.Anonymous
January 01, 2003
Hey Frank you do it in the process when you break in, just follow the steps above.Anonymous
May 13, 2008
How do you enable Handle Trace on a process ?Anonymous
January 20, 2009
Posting to an old message, hope it gets to you. Is there a way to increase the Max traces? It seems that on some systems it is maxed out at 0x1000. Specifying a value for it in !htrace does not seem to change it. Once it reaches this cap, it seems it will not track traces anymore. Thanks