Want to check an HTTP web request without the sniffer?
Today I had to debug the contents of a web service call called thru a web reference proxy in a Visual Studio 2005 C# project. I wanted to check the exact contents of the HTTP request's body and I couldn't use a sniffer, because it was running on the local box (I'm sure that there are plenty of great tool for this task, but I'm too dumb to find them). Instead of looking for the magic tool, I started Debugging Tools for Windows, started the .net EXE from it, waited until the core .net components get loaded, then before the application attempted to send out the request, I paused the process in the debugger and ran the following commands:
First of all, loading SOS (it's in the C:\Windows\Microsoft.NET\Framework\v2.0.50727 folder. If you copy it under C:\Program Files\Debugging Tools for Windows, you make your life easier):
0:005> .load sos
Then, I want to set a breakpoint on the GetResponse() call, because I know that this is the method that should be called inside the web proxy to send the webrequest. In order to do this, I have to find out the MethodTable of the HttpWebRequest class implemeted in System.dll:
0:005> !name2ee *!System.Net.HttpWebRequest
Module: 790c2000 (mscorlib.dll)
--------------------------------------
Module: 009723cc (sortkey.nlp)
--------------------------------------
Module: 0097205c (sorttbls.nlp)
--------------------------------------
Module: 00952c14 (ConsoleApplication1.exe)
--------------------------------------
Module: 69e2c000 (System.Xml.dll)
--------------------------------------
Module: 7a714000 (System.dll)
Token: 0x020003c6
MethodTable: 7a770bcc
EEClass: 7a7ccdc8
Name: System.Net.HttpWebRequest
--------------------------------------
Module: 699aa000 (System.Web.Services.dll)
--------------------------------------
Module: 648e8000 (System.Configuration.dll)
--------------------------------------
Module: 0095792c (6oyrllpt, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null)
Then I have to find out the address of the GetResponse() method:
0:005> !dumpmt -md 7a770bcc
EEClass: 7a7ccdc8
Module: 7a714000
Name: System.Net.HttpWebRequest
mdToken: 020003c6 (C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
BaseSize: 0xf4
ComponentSize: 0x0
Number of IFaces in IFaceMap: 1
Slots in VTable: 216
--------------------------------------
MethodDesc Table
Entry MethodDesc JIT Name
79354bec 7913bd48 PreJIT System.Object.ToString()
793539c0 7913bd50 PreJIT System.Object.Equals(System.Object)
793539b0 7913bd68 PreJIT System.Object.GetHashCode()
7934a4c0 7913bd70 PreJIT System.Object.Finalize()
79361e50 7913dbd8 PreJIT System.MarshalByRefObject.GetLifetimeService()
79360770 7913dbe0 PreJIT
(loads of methods ...)
7a57bb64 7a819130 PreJIT System.Net.HttpWebRequest.GetRequestStream()
7a57c7d8 7a819180 PreJIT System.Net.HttpWebRequest.GetResponse()
7a57c1a8 7a819168 PreJIT System.Net.HttpWebRequest.BeginGetResponse(System.AsyncCallback, System.Object)
(loads of methods ...)
After having the method's address, I can set a managed breakpoint on it:
0:005> !bpmd -md 7a819180
MethodDesc = 7a819180
Setting breakpoint: bp 7A57C7D8 [System.Net.HttpWebRequest.GetResponse()]
*** WARNING: Unable to verify checksum for C:\WINDOWS\assembly\NativeImages_v2.0.50727_32\System\fb4a8dfd84f82f46ac369d69bf0f1888\System.ni.dll
*** ERROR: Module load completed but symbols could not be loaded for C:\WINDOWS\assembly\NativeImages_v2.0.50727_32\System\fb4a8dfd84f82f46ac369d69bf0f1888\System.ni.dll
Ignore the error messages, we don't need those symbols for this task. Then, I let it go, we'll break when the GetResponse() method is called:
0:005> g
Just a few seconds later, I got my breakpoint hit:
Breakpoint 0 hit
eax=7a770bcc ebx=012a1a0c ecx=0136833c edx=0136833c esi=012a1a0c edi=01365690
eip=7a57c7d8 esp=0012f380 ebp=0012f3ac iopl=0 nv up ei pl zr na pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000246
System_ni!System.Net.HttpWebRequest.GetResponse():
7a57c7d8 55 push ebp
OK, let's check the managed call stack together with the local variables:
0:000> !clrstack -l
OS Thread Id: 0x144 (0)
ESP EIP
0012f380 7a57c7d8 System.Net.HttpWebRequest.GetResponse()
LOCALS:
<no data>
<no data>
<no data>
<no data>
<no data>
<no data>
0012f384 6990bf7d System.Web.Services.Protocols.WebClientProtocol.GetWebResponse(System.Net.WebRequest)
LOCALS:
0x0012f384 = 0x00000000
<no data>
0012f3b4 6990c915 System.Web.Services.Protocols.HttpWebClientProtocol.GetWebResponse(System.Net.WebRequest)
LOCALS:
<no data>
0012f3b8 69919eb1 System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(System.String, System.Object[])
LOCALS:
<no data>
0x0012f3c8 = 0x0136833c
0x0012f3c4 = 0x0137c074
<no data>
<no data>
<no data>
0x0012f3b8 = 0x00000000
0012f3fc 00ce1738 ConsoleApplication1.demo1.Lists.GetListItems(System.String, System.String, System.Xml.XmlNode, System.Xml.XmlNode, System.String, System.Xml.XmlNode, System.String)
LOCALS:
0x0012f400 = 0x00000000
<CLR reg> = 0x00000000
<CLR reg> = 0x013656bc
0012f430 00ce01b9 ConsoleApplication1.Program.Main(System.String[])
LOCALS:
0x0012f44c = 0x012a1a0c
0x0012f448 = 0x01362a04
0x0012f444 = 0x01362dd0
0x0012f440 = 0x01362e24
0x0012f43c = 0x01362e78
0x0012f438 = 0x00000000
0x0012f434 = 0x00000000
0x0012f430 = 0x00000000
0012f69c 79e88f63 [GCFrame: 0012f69c]
After a few minutes of research (looking at the local variable types one by one), I found that the System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke method on the call stack has a local variable of type HttpWebRequest, at the address of 0x0136833c. Let's start with this one:
0:000> !do 0x0136833c
Name: System.Net.HttpWebRequest
MethodTable: 7a770bcc
EEClass: 7a7ccdc8
Size: 244(0xf4) bytes
(C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
MT Field Offset Type VT Attr Value Name
790f9c18 4000184 4 System.Object 0 instance 00000000 __identity
7a77c648 4001cef 14 System.Int32 0 instance 1 m_AuthenticationLevel
79172e58 4001cf0 18 System.Int32 0 instance 4 m_ImpersonationLevel
7a77b418 4001cf1 8 ...equestCachePolicy 0 instance 0137ad34 m_CachePolicy
(loads of methods...)
7a775ad0 4001edd 78 ...thenticationState 0 instance 0137dcd4 _ServerAuthenticationState
7a76e3dc 4001ede 7c ....Net.ICredentials 0 instance 013629e8 _AuthInfo
7a772290 4001edf 80 ...HttpAbortDelegate 0 instance 0137f118 _AbortDelegate
7a776e38 4001ee0 84 ...Net.ConnectStream 0 instance 01380488 _SubmitWriteStream
7a776e38 4001ee1 88 ...Net.ConnectStream 0 instance 00000000 _OldSubmitWriteStream
790fed1c 4001ee2 b8 System.Int32 0 instance 50 _MaximumAllowedRedirections
790fed1c 4001ee3 bc System.Int32 0 instance 0 _AutoRedirects
790fed1c 4001ee4 c0 System.Int32 0 instance 0 _RerequestCount
790fed1c 4001ee5 c4 System.Int32 0 instance 100000 _Timeout
7a77a50c 4001ee6 8c ...TimerThread+Timer 0 instance 0137dbec _Timer
(loads of methods...)
The _SubmitWriteStream method seems to be a stream holding the contents of the request stream. Let's check this one:
0:000> !do 01380488
Name: System.Net.ConnectStream
MethodTable: 7a776e38
EEClass: 7a7d888c
Size: 116(0x74) bytes
(C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
MT Field Offset Type VT Attr Value Name
790f9c18 4000184 4 System.Object 0 instance 00000000 __identity
79177788 4001b1d 8 ...ream+ReadDelegate 0 instance 00000000 _readDelegate
79177818 4001b1e c ...eam+WriteDelegate 0 instance 00000000 _writeDelegate
7910ccf4 4001b1f 10 ...ng.AutoResetEvent 0 instance 00000000 _asyncActiveEvent
790fed1c 4001b20 14 System.Int32 0 instance 1 _asyncActiveCount
790fe3c8 4001b1c 55c System.IO.Stream 0 shared static Null
>> Domain:Value 0014cef0:013645cc <<
790fed1c 4002462 40 System.Int32 0 instance 3 m_CallNesting
7a779a14 4002463 28 ...tterGatherBuffers 0 instance 013804fc m_BufferedData
79104f64 4002464 68 System.Boolean 0 instance 0 m_SuppressWrite
79104f64 4002465 69 System.Boolean 0 instance 1 m_BufferOnly
790fcb80 4002466 18 System.Int64 0 instance 764 m_BytesLeftToWrite
790fed1c 4002467 44 System.Int32 0 instance 0 m_BytesAlreadyTransferred
(loads of methods...)
Hmmm ... the one highlighted seems to be the one holding the data. Let's check it:
0:000> !do 013804fc
Name: System.Net.ScatterGatherBuffers
MethodTable: 7a779a14
EEClass: 7a7dc1d4
Size: 28(0x1c) bytes
(C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
MT Field Offset Type VT Attr Value Name
7a779a74 400260a 4 ...ffers+MemoryChunk 0 instance 01382cc4 headChunk
7a779a74 400260b 8 ...ffers+MemoryChunk 0 instance 01382cc4 currentChunk
790fed1c 400260c c System.Int32 0 instance 2048 nextChunkLength
790fed1c 400260d 10 System.Int32 0 instance 764 totalLength
790fed1c 400260e 14 System.Int32 0 instance 1 chunkCount
Hmmm ... we have to dive even deeper. Let's check the headChunk member
0:000> !do 01382cc4
Name: System.Net.ScatterGatherBuffers+MemoryChunk
MethodTable: 7a779a74
EEClass: 7a7dc274
Size: 20(0x14) bytes
(C:\WINDOWS\assembly\GAC_MSIL\System\2.0.0.0__b77a5c561934e089\System.dll)
Fields:
MT Field Offset Type VT Attr Value Name
79124418 400260f 4 System.Byte[] 0 instance 01382cd8 Buffer
790fed1c 4002610 c System.Int32 0 instance 764 FreeOffset
7a779a74 4002611 8 ...ffers+MemoryChunk 0 instance 00000000 Next
Hey, it seems that we are almost there! The Byte[] array holds the buffer that's the body of the request! Let's dump it out with the unmanaged da (Dump Ascii) command:
0:000> da 01382cd8
01382cd8 ".D.y"
0:000> da
01382cdd "."
0:000> da
01382cdf ""
0:000> da
01382ce0 "<?xml version="1.0" encoding="ut"
01382d00 "f-8"?><soap:Envelope xmlns:soap="
01382d20 ""schemas.xmlsoap.org/soap"
01382d40 "/envelope/" xmlns:xsi="ww"
01382d60 "w.w3.org/2001/XMLSchema-instance"
01382d80 "" xmlns:xsd="www.w3.org/2"
01382da0 "001/XMLSchema"><soap:Body><GetLi"
01382dc0 "stItems xmlns="schemas.mi"
01382de0 "crosoft.com/sharepoint/soap/"><l"
01382e00 "istName>CCECC48C-8DFC-4B8E-AA44-"
01382e20 "AF7F4D7D4C96</listName><query><Q"
01382e40 "uery xmlns=""><Where><Eq><FieldR"
0:000> da
01382e60 "ef Name="Title" /><Value Type="T"
01382e80 "ext">asd</Value></Eq></Where></Q"
01382ea0 "uery></query><viewFields><ViewFi"
01382ec0 "elds xmlns=""><FieldRef Name="Ti"
01382ee0 "tle" /><FieldRef Name="Amount" /"
01382f00 "></ViewFields></viewFields><quer"
01382f20 "yOptions><QueryOptions xmlns="">"
01382f40 "<IncludeMandatoryColumns>FALSE</"
01382f60 "IncludeMandatoryColumns><DateInU"
01382f80 "tc>TRUE</DateInUtc></QueryOption"
01382fa0 "s></queryOptions></GetListItems>"
01382fc0 "</soap:Body></soap:Envelope>"
Here we are! That wasn't easy but at least we learned something! :)
Comments
Anonymous
April 13, 2007
This is disgusting in an extraordinary way ;) What if stream content is not in a simple ascii format?Anonymous
April 14, 2007
The comment has been removedAnonymous
July 06, 2009
How about a more reasonable approach and use reflection in a partial class: Namespace SameNamespaceAsService Partial Public Class MyService Dim r As System.Xml.XmlWriter Dim msg As System.Web.Services.Protocols.SoapClientMessage Protected Overrides Function GetWebResponse(ByVal request As System.Net.WebRequest) As System.Net.WebResponse r.Flush() Dim members As System.Reflection.FieldInfo() members = msg.Stream.GetType().GetFields(Reflection.BindingFlags.Instance Or Reflection.BindingFlags.Public Or Reflection.BindingFlags.NonPublic) Dim obj As Object obj = members(1).GetValue(msg.Stream) members = obj.GetType().GetFields(Reflection.BindingFlags.Instance Or Reflection.BindingFlags.Public Or Reflection.BindingFlags.NonPublic) obj = members(0).GetValue(obj) members = obj.GetType().GetFields(Reflection.BindingFlags.Instance Or Reflection.BindingFlags.Public Or Reflection.BindingFlags.NonPublic) Dim content() As Byte = members(0).GetValue(obj) System.IO.File.WriteAllBytes("c:DevelopmentSoapEnvelope.xml", content) Return MyBase.GetWebResponse(request) End Function Protected Overrides Function GetWriterForMessage(ByVal message As System.Web.Services.Protocols.SoapClientMessage, ByVal bufferSize As Integer) As System.Xml.XmlWriter r = MyBase.GetWriterForMessage(message, bufferSize) msg = message Return r End Function End Class End NamespaceAnonymous
December 17, 2009
Hi, works fine :) But: This only logs the Request, and not the response. Any idea how to log the Response? Thanks :)