Compartir a través de

LPC part 2 Kernel Debugger Extensions

  • Artículo

Hello my name is Roy, I’m an EE on the Microsoft global escalation services / CPR team. This blog is a follow on to my first LPC blog. We will be discussing debugger extensions that allow you to look at LPC related issues.

Disclaimer: The purpose of this blog is to illustrate debugging techniques with LPC. Please do not rely on this information as documentation for the APIs when writing your own code. As always, this is subject to change in the future.

LPC Kernel Debugger Extensions

Command

Description

!lpc

Display the list and description of all the !lpc commands

!lpc message [MessageId]

Display the message with a given ID and all related information by attempting to match the given Message ID to the EHTREAD->LpcReceivedMessageId and to the ETHREAD->LpcReplyMessageId of all threads in the system.

If the MessageId parameter is omitted then it attempts to display all the outstanding messages in the system by searching for the tag ‘LpcM’ in the pools.

!lpc port [PortAddress]

Displays port information. If a server connection port address is specified then only information about that port is displayed. If either a client or server communication port is specified it prints information about the specified communication port, the peer communication port and the server connection port.

If the PortAddress parameter is omitted then it attempts to walk the list of all objects of type “Port” and “WaitablePort” and display them. Note that for this feature to work the GFlags option “+otl” i.e. “Maintain a list of objects for each type” must be enabled.

!lpc scan PortAddress

Displays port information. It attempts to walk the list of all objects of type “Port” and “WaitablePort” and display the one matching the specified port address. Note that for this feature to work the GFlags option “+otl” i.e. “Maintain a list of objects for each type” must be enabled.

!lpc thread [ThreadAddr]

If ThreadAddr is specified it walks the list of threads in the ETHREAD-> LpcReplyChain to locate the list head i.e. a “Port” or “WaitablePort” object on which the thread is waiting for a reply.

If the ThreadAddr parameter is omitted then it attempts to find all LPC server threads by looking for threads with a non-NULL EHTREAD->LpcReceivedMessageId and all client threads by looking for threads with a non-NULL ETHREAD->LpcReplyMessageId and displays them.

!lpc PoolSearch

Toggles a setting that controls whether the “lpc message” command will search for LPC message tag (‘LpcM’) in the kernel pools or not.

LPC Kernel Debugger Extension Usage

LPC Connection Port information from call stack

On the call stack of any client or server threads blocked on LPC data transfer or LPC connection there will be a frame containing either one of the functions NtRequestWaitReplyPort() or NtReplyWaitReceivePortEx() . The first parameter to either one of these functions is the handle to the port they are blocked on.

kd> !thread 810de2a8

THREAD 810de2a8 Cid 01dc.01f4 Teb: 7ffde000 Win32Thread: 00000000 WAIT: (WrLpcReceive) UserMode Non-Alertable

    81131188 Semaphore Limit 0x7fffffff

    810de398 NotificationTimer

Not impersonating

DeviceMap e196c460

Owning Process 810ddda0 Image: rpclpcs.exe

Wait Start TickCount 64555 Ticks: 402 (0:00:00:04.025)

Context Switch Count 2

UserTime 00:00:00.000

KernelTime 00:00:00.000

Win32 Start Address 0x77e76bf0

Start Address 0x7c810856

Stack Init f8e28000 Current f8e27c4c Base f8e28000 Limit f8e25000 Call 0

Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 0

ChildEBP RetAddr Args to Child

f8e27c64 804dc6a6 810de318 810de2a8 804dc6f2 nt!KiSwapContext+0x2e (FPO: [Uses EBP] [0,0,4])

f8e27c70 804dc6f2 e1084108 8055a540 e1084108 nt!KiSwapThread+0x46 (FPO: [0,0,0])

f8e27c98 8056a50a 00000001 00000010 00000001 nt!KeWaitForSingleObject+0x1c2 (FPO: [Non-Fpo])

f8e27d48 804df06b 000007c4 002bff70 00000000 nt!NtReplyWaitReceivePortEx+0x3dc (FPO: [Non-Fpo])

f8e27d48 7c90eb94 000007c4 002bff70 00000000 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ f8e27d64)

002bff80 00000000 00000000 00000000 00000000 ntdll!KiFastSystemCallRet (FPO: [0,0,0])

kd> !handle 7c4 3 810ddda0

processor number 0, process 810ddda0

PROCESS 810ddda0 SessionId: 0 Cid: 01dc Peb: 7ffd9000 ParentCid: 01b4

    DirBase: 058cd000 ObjectTable: e1a13278 HandleCount: 18.

    Image: rpclpcs.exe

Handle table at e107d000 with 18 Entries in use

07c4: Object: e1084108 GrantedAccess: 001f0001 Entry: e107df88

Object: e1084108 Type: (812b5c80) Port

    ObjectHeader: e10840f0 (old version)

        HandleCount: 1 PointerCount: 4

        Directory Object: e14c72c8 Name: rpclpc

kd> !lpc port e1084108

Server connection port e1084108 Name: rpclpc

    Handles: 1 References: 4

    Server process : 810ddda0 (rpclpcs.exe)

    Queue semaphore : 81131188

    Semaphore state 0 (0x0)

    The message queue is empty

    The LpcDataInfoChainHead queue is empty

LPC Messages that are waiting to be picked up by the server thread

When a message is queued to the server connection port and the server thread has not yet not been signaled to pull out the message the ‘!lpc port <Port>” displays the following output that includes the message.

kd> !lpc port e10d8388

Client communication port 0xe10d8388

    Handles: 1 References: 2

    The LpcDataInfoChainHead queue is empty

        Connected port: 0xe106dc00 Server connection port: 0xe15512e0

Server communication port 0xe106dc00

    Handles: 1 References: 1

    The LpcDataInfoChainHead queue is empty

Server connection port e15512e0 Name: rpclpc

    Handles: 1 References: 9

    Server process : ffbbebe8 (rpclpcs.exe)

    Queue semaphore : 81250848

    Semaphore state 0 (0x0)

        Messages in queue:

        0000 e10513f8 - Busy Id=000048d8 From: 05f4.0108 Context=80020000 [e15512f0 . e15512f0]

                   Length=0054003c Type=00000001 (LPC_REQUEST)

                   Data: 00000001 00000241 00000000 00000000 f877bc04 804ec10e

    The message queue contains 1 messages

    The LpcDataInfoChainHead queue is empty

LPC Messages being processed by the server

When an LPC message has been queued by the client, dequeued by the sever and being worked upon by the server both the client and the server thread have the message ID associated with them

kd> !lpc message 16fa

Searching message 16fa in threads ...

    Server thread 810de2a8 is working on message 16fa

Client thread 810dc930 waiting a reply from 16fa

Searching thread 810dc930 in port rundown queues ...

Server connection port e1084108 Name: rpclpc

    Handles: 1 References: 3

    Server process : 810ddda0 (rpclpcs.exe)

    Queue semaphore : 81131188

    Semaphore state 0 (0x0)

    The message queue is empty

    The LpcDataInfoChainHead queue is empty

Done.

kd> !thread 810de2a8

THREAD 810de2a8 Cid 01dc.01f4 Teb: 7ffde000 Win32Thread: 00000000 RUNNING on processor 0

Not impersonating

DeviceMap e196c460

Owning Process 810ddda0 Image: rpclpcs.exe

Wait Start TickCount 65732 Ticks: 10 (0:00:00:00.100)

Context Switch Count 4

UserTime 00:00:00.000

KernelTime 00:00:00.010

Win32 Start Address 0x000016fa

LPC Server thread working on message Id 16fa

Start Address kernel32!BaseThreadStartThunk (0x7c810856)

Stack Init f8e28000 Current f8e27728 Base f8e28000 Limit f8e25000 Call 0

Priority 9 BasePriority 8 PriorityDecrement 0 DecrementCount 0

ChildEBP RetAddr Args to Child

f8e27d64 7c90eb94 badb0d00 002bfe24 00000000 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ f8e27d64)

002bfe18 7c90e399 77e76703 000007c4 002bff70 ntdll!KiFastSystemCallRet (FPO: [0,0,0])

002bfe1c 77e76703 000007c4 002bff70 00000000 ntdll!NtReplyWaitReceivePortEx+0xc (FPO: [5,0,0])

002bff80 77e76c22 002bffa8 77e76a3b 00084540 RPCRT4!LRPC_ADDRESS::ReceiveLotsaCalls+0xf4 (FPO: [Non-Fpo])

002bff88 77e76a3b 00084540 7c90ee18 0006fb10 RPCRT4!RecvLotsaCallsWrapper+0xd (FPO: [Non-Fpo])

002bffa8 77e76c0a 00083f20 002bffec 7c80b50b RPCRT4!BaseCachedThreadRoutine+0x79 (FPO: [Non-Fpo])

002bffb4 7c80b50b 00084718 7c90ee18 0006fb10 RPCRT4!ThreadStartRoutine+0x1a (FPO: [Non-Fpo])

002bffec 00000000 77e76bf0 00084718 00000000 kernel32!BaseThreadStart+0x37 (FPO: [Non-Fpo])

kd> !thread 810dc930

THREAD 810dc930 Cid 01e4.01ec Teb: 7ffde000 Win32Thread: 00000000 READY

Waiting for reply to LPC MessageId 000016fa:

Pending LPC Reply Message:

    e16b8538: [e16b8538,e16b8538]

Not impersonating

DeviceMap e196c460

Owning Process 810e9860 Image: rpclpcc.exe

Wait Start TickCount 65731 Ticks: 11 (0:00:00:00.110)

Context Switch Count 26

UserTime 00:00:00.000

KernelTime 00:00:00.020

Win32 Start Address rpclpcs!pre_c_init (0x01001a91)

Start Address kernel32!BaseProcessStartThunk (0x7c810867)

Stack Init fbe0e000 Current fbe0dc28 Base fbe0e000 Limit fbe0b000 Call 0

Priority 8 BasePriority 8 PriorityDecrement 0 DecrementCount 16

ChildEBP RetAddr Args to Child

fbe0dc40 804ea3a4 00000000 810dc930 e107f388 nt!KiUnlockDispatcherDatabase+0x77 (FPO: [Uses EBP] [0,0,4])

fbe0dc58 80586255 81131188 00000001 00000001 nt!KeReleaseSemaphore+0x70 (FPO: [Non-Fpo])

fbe0dd10 80598c58 000847b0 0006f8c0 0006f8c8 nt!NtSecureConnectPort+0x635 (FPO: [Non-Fpo])

fbe0dd3c 804df06b 000847b0 0006f8c0 0006f8c8 nt!NtConnectPort+0x24 (FPO: [Non-Fpo])

fbe0dd3c 7c90eb94 000847b0 0006f8c0 0006f8c8 nt!KiFastCallEntry+0xf8 (FPO: [0,0] TrapFrame @ fbe0dd64)

0006f8d0 7c93040b ffffffff 00000000 7c859e48 ntdll!KiFastSystemCallRet (FPO: [0,0,0])

0006f9b0 7c8107fd 00000000 00084718 7c90e642 ntdll!DbgPrint+0x1b (FPO: [Non-Fpo])

0006fe68 77ea11a8 00000022 00084540 00000000 kernel32!CreateRemoteThread+0x284 (FPO: [Non-Fpo])

000847a8 00000000 00000001 00000000 00000000 RPCRT4!LRPC_ADDRESS::CompleteListen+0x84 (FPO: [Non-Fpo])

Finding all threads in the system involved in LPC communication

To get a consolidated list of all client threads in the system waiting on LPC replies and a list of server threads working on received LPC requests use the “!lpc threads” command without any parameters

0: kd> !lpc thread

Searching message 0 in threads ...

Client thread 89a4cdb0 waiting a reply from 15f6

    Server thread 898c6a50 is working on message 63f09

    Server thread 89671020 is working on message 63f75

    Server thread 89904a48 is working on message 63f10

    Server thread 88d6bdb0 is working on message 3d6b1

    Server thread 89973db0 is working on message 63f32

    Server thread 896d32b8 is working on message 3454

    Server thread 8995d020 is working on message 63f76

    Server thread 8960f020 is working on message 63f77

    Server thread 898cd350 is working on message 61bd9

    Server thread 8900edb0 is working on message 3832f

    Server thread 8900fbf0 is working on message 33c2e

    Server thread 88d539a8 is working on message 3343f

Client thread 89be4020 waiting a reply from 6077e

Client thread 89012990 waiting a reply from 39239

    Server thread 89012990 is working on message 39232

Client thread 89531020 waiting a reply from 3923d

    Server thread 89531020 is working on message 39236

Client thread 88d13b40 waiting a reply from 3c96e

Quick Tips

8 No thread in the system is doing a NtListenPort( ) does that mean no LPC connection port is open for connections?

!stacks 2 NtListenPort does not return any thread because NtListenPort( ) calls NtRequestWaitReplyPort( ) to wait for an incoming LPC_CONNECTION_REQUEST message. Instead of calling NtListenPort( ) for listening for incoming connections most LPC connection threads directly call NtRequestWaitReplyPort( ).

8 !lpc message does not show any message?

!poolfind LpcM 1 should do the trick.

0: kd> !lpc message

Scanning large pool allocation table for Tag: LpcM (8974e000 : 8976e000)

Searching Paged pool (e1000000 : f7000000) for Tag: LpcM

0: kd> !poolfind LpcM 1

Scanning large pool allocation table for Tag: LpcM (8974e000 : 8976e000)

Searching Paged pool (e1000000 : f7000000) for Tag: LpcM

e1061000 size: 168 previous size: 0 (Allocated) LpcM

e1090758 size: 168 previous size: 8 (Allocated) LpcM

e10a7a40 size: 168 previous size: 8 (Allocated) LpcM

e1188468 size: 8 previous size: 20 (Free) LpcM

e1261000 size: 168 previous size: 0 (Allocated) LpcM

e1297428 size: 168 previous size: 8 (Allocated) LpcM

Once you have the pool blocks listed with the “LpcM” tag dump the ones that are “Allocated”. On a 32-bit machine add 8bytes to the pool address listed on the left most column. On a 64-bit machine add 16bytes i.e. 0x10. Dump the message as below to retrieve the message id.

0: kd> dt nt!_LPCP_MESSAGE e1061000+8

   +0x000 Entry : _LIST_ENTRY [ 0xe295fa80 - 0xe29279e8 ]

   +0x000 FreeEntry : _SINGLE_LIST_ENTRY

   +0x004 Reserved0 : 0xe29279e8

   +0x008 SenderPort : 0xe2327020

   +0x00c RepliedToThread : (null)

   +0x010 PortContext : 0x8021000d

   +0x018 Request : _PORT_MESSAGE

0: kd> dt nt!_LPCP_MESSAGE e1061000+8 Request.

   +0x018 Request :

      +0x000 u1 : __unnamed

      +0x004 u2 : __unnamed

      +0x008 ClientId : _CLIENT_ID

      +0x008 DoNotUseThisField : 9.8460604703041575e-311

      +0x010 MessageId : 0x4617f

      +0x014 ClientViewSize : 0

      +0x014 CallbackId : 0

0: kd> !lpc message 0x4617f

Searching message 4617f in threads ...

    Server thread 88f91c08 is working on message 4617f

Client thread 88f0f5f8 waiting a reply from 4617f

Searching thread 88f0f5f8 in port rundown queues ...

Server communication port 0xe2300a38

    Handles: 1 References: 1

    The LpcDataInfoChainHead queue is empty

        Connected port: 0xe2327020 Server connection port: 0xe13734b0

Client communication port 0xe2327020

    Handles: 1 References: 9

    The LpcDataInfoChainHead queue is empty

Server connection port e13734b0 Name: epmapper

    Handles: 1 References: 93

  Server process : 896e1450 (svchost1.exe)

    Queue semaphore : 8995ac10

    Semaphore state 0 (0x0)

    The message queue is empty

8 The number of server threads shown by !lpc thread processing requests is much more than the number of client threads waiting for reply.

!lpc thread interprets stale data in ETHREAD structure as valid LPC information. An indication of the stale data is the ETHREAD.LpcExitThreadCalled will be set. This field is only set when the thread is exiting.

0: kd> !lpc thread

Searching message 0 in threads ...

Client thread 89a4cdb0 waiting a reply from 15f6

    Server thread 898c6a50 is working on message 63f09

    Server thread 89671020 is working on message 63f75

    Server thread 89904a48 is working on message 63f10

    Server thread 88d6bdb0 is working on message 3d6b1

    Server thread 89973db0 is working on message 63f32

    Server thread 896d32b8 is working on message 3454

    Server thread 8995d020 is working on message 63f76

    Server thread 8960f020 is working on message 63f77

    Server thread 898cd350 is working on message 61bd9

    Server thread 8900edb0 is working on message 3832f

    Server thread 8900fbf0 is working on message 33c2e

    Server thread 88d539a8 is working on message 3343f

0: kd> !lpc message 3454

Searching message 3454 in threads ...

    Server thread 896d32b8 is working on message 3454

Done.

0: kd> dt nt!_ETHREAD 896d32b8 Lpc.

   +0x1c0 LpcReplyChain : [ 0x896d3478 - 0x896d3478 ]

      +0x000 Flink : 0x896d3478 _LIST_ENTRY [ 0x896d3478 - 0x896d3478 ]

      +0x004 Blink : 0x896d3478 _LIST_ENTRY [ 0x896d3478 - 0x896d3478 ]

   +0x1ec LpcReplySemaphore :

      +0x000 Header : _DISPATCHER_HEADER

      +0x010 Limit : 1

   +0x200 LpcReplyMessage :

   +0x200 LpcWaitingOnPort :

   +0x220 LpcReceivedMessageId : 0x3454

   +0x234 LpcReplyMessageId : 0

   +0x248 LpcReceivedMsgIdValid : 0x1 ''

   +0x248 LpcExitThreadCalled : 0x1 ''

8 Finding all client threads in the system waiting for LPC reply.

!lpc thread

Comments