Freigeben über

[RESOLVED] Win7SP1 x86: STOP 0x7F in cng!SHATransformP3+2c

  • Artikel

Status: Resolved

Update 110930: The KB article for this will be 2592339. Issue resolved, case closed. ;)

Update 110811: Customer confirmed the workaround is working! :) Now we wait for the hotfix... scheduled for HTP11-10.

Update 110808: As a workaround, you can set the encryption level for RDP to Low. This way only client ->server data will be encrypted, not the server->client data. This will prevent the current stack causing the overflow from getting hit. I've sent this workaround to our current customer and I'll update this post with the results.

Currently I'm working on an issue where intermittently Win7SP1 VDI clients crash with a stack overfow. This looks to be caused by the win32k stack consumption. The stack in this case looks like this:

kd> knL
# ChildEBP RetAddr
00 00000000 87663f8c nt!KiTrap08+0x75
01 ab526134 8765d7cb cng!SHATransformP3+0x2c
02 ab526158 8765d9e7 cng!A_SHAUpdate+0xdb
03 ab526204 87646822 cng!HMACSHAInit+0x137
04 ab526220 8762a48e cng!MSCryptCreateHash+0x125
05 ab526274 8764ace9 cng!BCryptCreateHash+0x2d4
06 ab5262c8 8764b11d cng!Tls1ComputeMac+0x9b
07 ab52631c 8764a1b4 cng!TlsEncryptPacket+0x1aa
08 ab52634c 8762aee7 cng!SPSslEncryptPacket+0x8c
09 ab526380 87794413 cng!SslEncryptPacket+0x4d
0a ab5263e8 877a6226 ksecpkg!SslSealMessageStream+0x195
0b ab526404 875d41d0 ksecpkg!SslSealMessage+0x34
0c ab52641c 9cf4f15d ksecdd!EncryptMessage+0x34
0d ab526434 9cf4a352 tssecsrv!SpEncryptMessage+0x25
0e ab526494 9cf4a77f tssecsrv!CSecurityFilter::EncryptData+0xda
0f ab5264a8 9cf48145 tssecsrv!CSecurityFilter::FilterOutgoingData+0x22
10 ab5264d0 9cf4798f tssecsrv!CFilter::FilterOutgoingData+0x8d
11 ab5264fc 945b16f5 tssecsrv!ScrRawWrite+0x49
12 ab526518 945b17d6 termdd!_IcaCallSd+0x37
13 ab526534 9cf76653 termdd!IcaCallNextDriver+0x4a
14 ab526548 9cf5d589 RDPWD!FinalSendOutBuf+0x12
15 ab52657c 9cf5cc84 RDPWD!NM_SendData+0xd9
16 ab5265b4 9cf5e246 RDPWD!SM_SendData+0x8f
17 ab5265dc 9cf5ee9f RDPWD!ShareClass::SC_SendFastPathData+0x2a
18 ab526600 9cf672bf RDPWD!ShareClass::SC_FlushAndAllocPackageOfSize+0x2c
19 ab52663c 9cf675a6 RDPWD!ShareClass::UPSendOrdersBackFilling+0xfe
1a ab52664c 9cf615bf RDPWD!ShareClass::UPSendOrders+0x20
1b ab526694 9cf602d1 RDPWD!ShareClass::UP_SendUpdates+0xc6
1c ab5266e8 9cf58190 RDPWD!ShareClass::DCS_TimeToDoStuff+0xd5
1d ab526710 9cf56376 RDPWD!WDLIB_DDOutputAvailable+0x194
1e ab526728 945b16f5 RDPWD!WDSYS_Ioctl+0x20
1f ab526744 945b1bad termdd!_IcaCallSd+0x37
20 ab526764 945b254a termdd!_IcaCallStack+0x57
21 ab52678c 945b30d1 termdd!IcaCallDriver+0x11e
22 ab5267c8 945af2d8 termdd!IcaDeviceControlVirtual+0x265
23 ab5267f0 945aff9f termdd!IcaDeviceControlChannel+0x222
24 ab526820 945b0173 termdd!IcaDeviceControl+0x59
25 ab526838 82671593 termdd!IcaDispatch+0x13f
26 ab526850 8214b034 nt!IofCallDriver+0x63
27 ab52687c 8210cb28 win32k!CtxDeviceIoControlFile+0xa7
28 ab5268b8 81e711fa win32k!EngFileIoControl+0x31
29 ab526944 81e712c9 RDPDD!SCH_DDOutputAvailable+0x160
2a ab52695c 81e7e70c RDPDD!SCH_DDOutputAvailable+0x2f
2b ab526980 81e80fa1 RDPDD!OA_AllocOrderMem+0x42
2c ab5269dc 81e815a1 RDPDD!SBCCacheBits+0x125
2d ab526a78 81e7a516 RDPDD!SBC_CacheBitmapTile+0x1cb
2e ab526e04 81e7a768 RDPDD!OETileBitBltOrder+0x22c
2f ab526e2c 81e758d7 RDPDD!OEEncodeMemBlt+0x100
30 ab526eec 81e75f3f RDPDD!DrvBitBlt+0x425
31 ab526f28 82045a3b RDPDD!DrvCopyBits+0x41
32 ab526f70 82035b2a win32k!OffCopyBits+0x7d
33 ab527214 82045a86 win32k!SpBitBlt+0x252
34 ab527248 82048ab8 win32k!SpCopyBits+0x27
35 ab5274dc 8204aac0 win32k!EngTextOut+0x6f0
36 ab527528 8204ad2d win32k!OffTextOut+0x71
37 ab5277ac 8204a89d win32k!SpTextOut+0x1a2
38 ab527ad8 81fd9fbb win32k!GreExtTextOutWLocked+0x1040
39 ab527b54 8202c647 win32k!GreBatchTextOut+0x1e6
3a ab527cc4 828b2022 win32k!NtGdiFlushUserBatch+0x123
3b ab527d14 820ab756 nt!KeUserModeCallback+0x176
3c ab527fd8 82073878 win32k!xxxClientExtTextOutW+0x160
3d ab528230 82073684 win32k!xxxDrawMenuItemText+0xc9
3e ab528320 82073b6d win32k!xxxRealDrawMenuItem+0x808
3f ab5283e8 820c984e win32k!xxxDrawState+0x1f4
40 ab528454 820c32b7 win32k!xxxDrawMenuItem+0x3c4
41 ab5284cc 820df198 win32k!xxxMenuDraw+0x23a
42 ab528540 820ae63a win32k!xxxRealMenuWindowProc+0xe46
43 ab528574 820293b3 win32k!xxxMenuWindowProc+0xe1
44 ab5285b4 82029485 win32k!xxxSendMessageTimeout+0x1ac
45 ab5285dc 820ac4a4 win32k!xxxSendMessage+0x28
46 ab528624 8201e5d3 win32k!xxxDWPPrint+0x1cd
47 ab5286a0 81ff773a win32k!xxxRealDefWindowProc+0x13a8
48 ab5286c4 820df29f win32k!xxxDefWindowProc+0x10f
49 ab52873c 820ae63a win32k!xxxRealMenuWindowProc+0xf4d
4a ab528770 820293b3 win32k!xxxMenuWindowProc+0xe1
4b ab5287b0 82029485 win32k!xxxSendMessageTimeout+0x1ac
4c ab5287d8 820de947 win32k!xxxSendMessage+0x28
4d ab528850 820ae63a win32k!xxxRealMenuWindowProc+0x5f5
4e ab528884 820293b3 win32k!xxxMenuWindowProc+0xe1
4f ab5288c4 82029485 win32k!xxxSendMessageTimeout+0x1ac
50 ab5288ec 81ff948c win32k!xxxSendMessage+0x28
51 ab5289b0 8200e283 win32k!xxxCalcValidRects+0xf7
52 ab528a0c 8200e6ae win32k!xxxEndDeferWindowPosEx+0x104
53 ab528a2c 8208a358 win32k!xxxSetWindowPos+0xf6
54 ab528b30 82087972 win32k!xxxMNOpenHierarchy+0x60b
55 ab528b4c 820a93ef win32k!xxxMNButtonDown+0x4e
56 ab528bac 820a8b6e win32k!xxxHandleMenuMessages+0x57a
57 ab528bf8 820bb333 win32k!xxxMNLoop+0x12b
58 ab528c28 8201de2b win32k!xxxSysCommand+0x4a4
59 ab528ca4 8202b4d1 win32k!xxxRealDefWindowProc+0xc00
5a ab528cbc 81ff640a win32k!xxxWrapRealDefWindowProc+0x2b
5b ab528cd8 8202b38d win32k!NtUserfnNCDESTROY+0x27
5c ab528d10 826781ea win32k!NtUserMessageCall+0xc9
5d ab528d10 77cc70b4 nt!KiFastCallEntry+0x12a
5e 0012eadc 76ed4f51 ntdll!KiFastSystemCallRet

I'm very interested to find out if more people are experiencing this. If you see this on your machines, please notify me!

Comments

  • Anonymous
    January 06, 2012
    I have just had the same problem after analyzing a dump file on a Win2k08 server. What is causing it?