Debugging User-Mode Processes Using a Kernel-Mode Debugger
In this post I'll try to clarify some small details, that are related to debugging a user-mode process (focusing on a UMDF driver) using a kernel-mode debugger. So, the setup is that we have a test computer, where the UMDF echo driver is running and another computer, where windbg is running and we're using it as a kernel-mode debugger. A first thing to do in our case would be to see what modules are loaded:
kd> lm
start end module name
81800000 81b95000 nt (export symbols) ntkrnlmp.exe
Unloaded modules:
85dac000 85db4000 drmkaud.sys
85ce8000 85cf5000 crashdmp.sys
82a05000 82a10000 dump_ataport.sys
85d94000 85d9c000 dump_atapi.sys
85c27000 85c38000 dump_dumpfve.sys
8d618000 8d632000 serial.sys
88020000 88029000 kbdhid.sys
88e52000 88e65000 i8042prt.sys
88e63000 88e78000 WUDFRd.sys
880aa000 880b0000 nothing.sys
Ok, that's interesting. Just with a first a look it seems that only the kernel is loaded and that there are a few unloaded modules. This view is deceiving, though. Let's reload symbols and try again:
kd> .reload /f
Connected to Windows Vista 6000 x86 compatible target, ptr64 FALSE
Loading Kernel Symbols
........................
Loading User Symbols
Loading unloaded module list
..........
kd> lm
start end module name
81800000 81b95000 nt (pdb symbols) c:\Debuggers\sym\ntkrnlmp.pdb\E556D3F077BB42BB83B132247BE9C4942\ntkrnlmp.pdb
81b95000 81bc9000 hal (pdb symbols) c:\Debuggers\sym\halmacpi.pdb\AE84FF5D9CEE4D64927E629F756036841\halmacpi.pdb
82004000 82012000 PCIIDEX (pdb symbols) c:\Debuggers\sym\pciidex.pdb\0A98C6B81AB842C483351BCA042A9B1A1\pciidex.pdb
82012000 82019000 intelide (pdb symbols) c:\Debuggers\sym\intelide.pdb\BFCA935B0A6B47C2AA4B9F25100409F11\intelide.pdb
82019000 82029000 mountmgr (pdb symbols) c:\Debuggers\sym\mountmgr.pdb\6F08CCFAE97F4F139853B1769DAB0CF31\mountmgr.pdb
82029000 82038000 volmgr (pdb symbols) c:\Debuggers\sym\volmgr.pdb\3C43C06A961143719A6DF9F0B2A9699C1\volmgr.pdb
82038000 8205d000 pci (pdb symbols) c:\Debuggers\sym\pci.pdb\A5E895C861984D7393087EB0459E7FE01\pci.pdb
... [output has been truncated] ...
94b78000 94b8c680 WUDFRd (pdb symbols) c:\Debuggers\sym\WUDFRd.pdb\D92A3D77AEBE4FFE8EE42628096819371\WUDFRd.pdb
Ok, this seems more promising. The symbols have been loaded and we now see many more modules. We also see that the reflector (WUDFRd) is loaded. However, where is our UMDF driver? Now it's time to go back to some theory. The UMDF driver is actually a user-mode process. When you break into a kernel-mode debugger, you just break into an arbitrary process. Which one? The !process command will show us.
kd> !process
PROCESS 818eff00 SessionId: none Cid: 0000 Peb: 00000000 ParentCid: 0000
DirBase: 00122000 ObjectTable: 85800178 HandleCount: 593.
Image: Idle
VadRoot 00000000 Vads 0 Clone 0 Private 0. Modified 215. Locked 0.
DeviceMap 00000000
Token 858037d8
ElapsedTime 00:00:00.000
UserTime 00:00:00.000
KernelTime 00:00:00.000
QuotaPoolUsage[PagedPool] 0
QuotaPoolUsage[NonPagedPool] 0
Working Set Sizes (now,min,max) (4, 50, 450) (16KB, 200KB, 1800KB)
PeakWorkingSetSize 0
VirtualSize 0 Mb
PeakVirtualSize 0 Mb
PageFaultCount 0
MemoryPriority BACKGROUND
BasePriority 0
CommitCharge 0
THREAD 818efac0 Cid 0000.0000 Teb: 00000000 Win32Thread: 00000000 RUNNING on processor 0
So, I'm debugging the idle thread. Now it's time to switch to the echo driver. In order to do that, we need to find all the process and search for wudfhost.exe.
kd> !process 0 0 WUDFHost.exe
PROCESS 85709738 SessionId: 0 Cid: 0624 Peb: 7ffde000 ParentCid: 03d8
DirBase: 1ba3d000 ObjectTable: 93a5cd40 HandleCount: 161.
Image: WUDFHost.exe
Ok, now that I've found the process that I need to connect to, I need to switch to it. However, it's not enough just to do a
.process 85709738
This would just tell windbg to change just the debugger context, i.e. show what's going on in that process (e.g. the stack), however remain in the context of the idle process. If we put breakpoints, then they will be set in the idle process and NOT in the process that we want to debug! We need to do a context switch to that process.
So, one way to do this is:
kd> .process /i 85709738
You need to continue execution (press 'g' <enter>) for the context to be switched. When the debugger breaks in again, you will be in the new process context.
kd> g
Break instruction exception - code 80000003 (first chance)
nt!RtlpBreakWithStatusInstruction:
818356e8 cc int 3
However, as my friend Patrick (the debugging guru) told me: with the /i switch you have to think about the
possible traps of releasing the system for the context switch to happen. After you press "g", you have to wait for the windows scheduler to do the context switch. And while the
process is being brought in to context, you also have to think you’re releasing
the system. On a single hardware thread execution machine, switching in
to the process context won’t cause any issues (hopefully), but what about when
you get in to an SMP / Multicore environment? The context switch happens
and the flagged process is locked, so nothing is scheduled from that process, but
what about all those other threads in queue?
So, the best option here is to do
kd> .process /r /p 85709738
In this case, the debugger does the context switch immediately (as opposed to the windows scheduler) and there is no waiting time.
Anyway, now we're in the correct context. Let's see what's loaded:
kd> lm
start end module name
00f60000 00f86000 WUDFHost (pdb symbols) c:\Debuggers\sym\WUDFHost.pdb\AEC9577097EE46CAAE1D12CDA87064E61\WUDFHost.pdb
73c80000 73c87000 WUDFEchoDriver (export symbols) WUDFEchoDriver.dll
73d00000 73d50000 WUDFx (pdb symbols) c:\Debuggers\sym\WUDFx.pdb\A416B8E3114F4EA4BE1EAC879505D2771\WUDFx.pdb
74460000 74491000 WUDFPlatform (pdb symbols) c:\Debuggers\sym\WUDFPlatform.pdb\2BA7585C1C874EBBB9B082D649DFD80F1\WUDFPlatform.pdb
[... snip ...]
8270d000 8271f000 WUDFPf (pdb symbols) c:\Debuggers\sym\WUDFPf.pdb\98F0D4EDE42944CF8C87B11DD7985EC41\WUDFPf.pdb
94b78000 94b8c680 WUDFRd (pdb symbols) c:\Debuggers\sym\WUDFRd.pdb\D92A3D77AEBE4FFE8EE42628096819371\WUDFRd.pdb
Ok, now I can see my echo driver (WUDFEchoDriver.dll). If the symbols are not loaded, then a .reload /f would fix everything. Of course, if there are many WudfHost.exe processes that are running (i.e. you are using many UMDF drivers), then you might have to search for your driver (i.e. WUDFEchoDriver.dll) in the list of loaded modules and if it's not there, then you'll have to switch to another instance of WudfHost.exe.
Now it's time to do some debugging. The UMDF extension is called wudfext. Let's try and see, if it's loaded:
kd> !wudfext.help
The call to LoadLibrary(wudfext) failed, Win32 error 0n2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
No it's not... Ok, let's see what's loaded:
kd> .chain
Extension DLL search Path:
c:\Debuggers\WINXP;c:\Debuggers\winext;c:\Debuggers\winext\arcade;c:\Debuggers\pri;c:\Debuggers;c:\Debuggers\winext\arcade;
Extension DLL chain:
c:\Debuggers\winext\wdfkd: image 6.0.6001.16470, API 1.0.0, built Sun Feb 25 04:47:28 2007
[path: c:\Debuggers\winext\wdfkd.dll]
dbghelp: image 6.7.0004.0, API 6.0.6, built Thu Mar 15 10:39:24 2007
[path: c:\Debuggers\dbghelp.dll]
ext: image 6.7.0004.0, API 1.0.0, built Thu Mar 15 10:39:14 2007
[path: c:\Debuggers\winext\ext.dll]
exts: image 6.7.0004.0, API 1.0.0, built Thu Mar 15 10:38:51 2007
[path: c:\Debuggers\WINXP\exts.dll]
kext: image 6.7.0004.0, API 1.0.0, built Thu Mar 15 10:38:58 2007
[path: c:\Debuggers\winext\kext.dll]
kdexts: image 6.0.6001.16470, API 1.0.0, built Thu Mar 15 10:58:28 2007
[path: c:\Debuggers\WINXP\kdexts.dll]
I know that my extension is in the WDK at, C:\WinDDK\6001\bin\x86\wudfext.dll. Let's load it:
kd> !load C:\WinDDK\6001\bin\x86\wudfext.dll
kd> .chain
Extension DLL search Path:
c:\Debuggers\WINXP;c:\Debuggers\winext;c:\Debuggers\winext\arcade;c:\Debuggers\pri;c:\Debuggers;c:\Debuggers\winext\arcade
Extension DLL chain:
C:\WinDDK\6001\bin\x86\wudfext.dll: image 6.0.6001.17138, API 1.0.0, built Thu Jan 17 23:39:16 2008
[path: C:\WinDDK\6001\bin\x86\wudfext.dll]
c:\Debuggers\winext\wdfkd: image 6.0.6001.16470, API 1.0.0, built Sun Feb 25 04:47:28 2007
[path: c:\Debuggers\winext\wdfkd.dll]
dbghelp: image 6.7.0004.0, API 6.0.6, built Thu Mar 15 10:39:24 2007
[path: c:\Debuggers\dbghelp.dll]
ext: image 6.7.0004.0, API 1.0.0, built Thu Mar 15 10:39:14 2007
[path: c:\Debuggers\winext\ext.dll]
exts: image 6.7.0004.0, API 1.0.0, built Thu Mar 15 10:38:51 2007
[path: c:\Debuggers\WINXP\exts.dll]
kext: image 6.7.0004.0, API 1.0.0, built Thu Mar 15 10:38:58 2007
[path: c:\Debuggers\winext\kext.dll]
kdexts: image 6.0.6001.16470, API 1.0.0, built Thu Mar 15 10:58:28 2007
[path: c:\Debuggers\WINXP\kdexts.dll]
Ok, it seems to be loaded. Let's try and use it:
kd> !wudfext.help
The call to LoadLibrary(wudfext) failed, Win32 error 0n2
"The system cannot find the file specified."
Please check your debugger configuration and/or network access.
It's not working! Why's that? Notice that .chain mentions wudfext.dll with the full path, whereas everything else just uses the filename without the extension. Let's try the full name:
kd> !C:\WinDDK\6001\bin\x86\wudfext.dll.help
Help for host process debug extension:
umdevstacks [flags] - This dumps all the device stacks in the host process.
umdevstack <address> [flags] - This dumps detailed info on each device stack.
umirps - This dumps UM Irps in the host process.
[... the rst of the output is truncated ...]
Ok, so it's loaded indeed. If I try and use an extension directly, then it works:
kd> !umirps
Number of pending IRPS: 0x0
#### CWudfIrp Current Type UniqueId KernelIrp
---- ---------------- ----------------------------------- ---------------- ---------
However, if there are extensions with the same name (e.g. !help) in other files, this is a problem. Let's make it easier to use:
kd> !unload C:\WinDDK\6001\bin\x86\wudfext.dll
Unloading C:\WinDDK\6001\bin\x86\wudfext.dll extension DLL
kd> .extpath+ C:\WinDDK\6001\bin\x86
Extension search path is: c:\Debuggers\WINXP;c:\Debuggers\winext;c:\Debuggers\winext\arcade;c:\Debuggers\pri;c:\Debuggers;c:\Debuggers\winext\arcade;C:\WinDDK\6001\bin\x86
kd> !load wudfext
kd> .chain
Extension DLL search Path:
c:\Debuggers\WINXP;c:\Debuggers\winext;c:\Debuggers\winext\arcade;c:\Debuggers\pri;c:\Debuggers;c:\Debuggers\winext\arcade;C:\WinDDK\6001\bin\x86
Extension DLL chain:
wudfext: image 6.0.6001, API 1.0.0, built Thu Jan 17 23:39:16 2008
[path: C:\WinDDK\6001\bin\x86\wudfext.dll]
kd> !wudfext.help
Help for host process debug extension:
umdevstacks [flags] - This dumps all the device stacks in the host process.
umdevstack <address> [flags] - This dumps detailed info on each device stack.
umirps - This dumps UM Irps in the host process.
[... the rest of the output is truncated ...]
Ok, so now it works. As a final note, just remember that the UMDF extension should be from the same WDK as the UMDF framework, e.g. if you're using UMDF 1.7 RC1, then wudfext should be 1.7 RC1. The driver can be compiled with any framework version (as long as the major version of UMDF on the machine is the same as the one that the driver was compiled with and the minor version of the running UMDF is greater or equal to the driver's). So, the UMDF extension doesn't have to be the same as the version of the framework that the driver was compiled with.
Comments
Anonymous
February 01, 2008
Hardly, now Doron on the other hand... Although I can debug w/o symbols, where you seem to think you need the WinDBG w/ sources and all those parameter expnasions, colors and local watch lists all in front of you. It's just a big crutch man...a fuzzy blue blanket crutch. Whimp. ;)Anonymous
February 07, 2008
Great work Ilias, its brief and informative, looking ahead more from you.