Troubleshooting: 413: Request size too large with WCF Service
Troubleshooting: 413: Request size too large with WCF Service
Problem Statement:
A WCF service configured with BasicHpptBinding throws 413: Request size too large error at the client application.
This gives an indication that there is some bottleneck at IIS is occurring while WCF service is configured for all the properties to Max.Int32.
maxBufferSize="2147483647"
maxBufferPoolSize="2147483647"
maxReceivedMessageSize="2147483647"
ReadersQuoata values are also set for max:
maxStringContentLength="2147483647"
maxArrayLength="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647"
From the initial point, this error is misguiding and will indicate towards IIS bottleneck but later on I discovered that it is WCF pipeline choking it.
Investigation:
From the IIS logs you will see that the status of the request is 413. Generally, status 4XX and 5XX correspond to IIS errors, that is why I am naming this error as misguiding error. You will see that WCF config is set for maximum while you are getting the 413: Request size too large. This exception will also get validated from System.Net traces.
System.Net Verbose: 0 : [7340] Exiting ConnectStream#55642016::Read() -> Int32#0
DateTime=2015-09-15T19:18:17.4572956Z
System.Net Error: 0 : [7340] Exception in HttpWebRequest#14015468::GetResponse - The remote server returned an error: (413) Request Entity Too Large..
As a next step, we will increase the IIS limit to handle the bigger requests by increasing the following settings:
<add header="Content-type" sizeLimit="100" />
uploadReadAheadSize to
2147483647
but still we are getting the same exception. Now what to do!!!
As a next step, we will collect the dump of the w3wp process under which our WCF service is running and will see if there is any http request which is returning 413. Use DebugDiag to collect the 2 full memory dumps.
1- As soon as the application starts.
2- Just after the you get the exception at the client side.
Or you can configure DebugDia to collect the dump at following exception type:
Type: System.ServiceModel.ProtocolException
Now we will analyze the dump with WinDbg. I am using the extension netext as WCF services are involved so we can dump all the WCF related objects as well.
First of all, I will dump all the http request in the memory dump to see the status of all the http request objects in the dump.
1c414674 -- 00:00:00 Finished 413 POST
Here I got a request which is throwing 413 and this indicates that request is passing through the IIS and reaching till WCF layer and WCF service is throwing it back saying request size too large. So it is not a bottleneck at IIS instead WCF service is blocking this request due to its size.
So, the question arises, why WCF service is throwing this exception when it is configured to have Readers Quota values as maximum.
In the second step I will dump the binding details of the WCF service to check the settings.
Service Info
================================
Address :13A922E4
Configuration Name : Test.GlobalInformation
State :Opened
EndPoints : 0n1
Base Addresses : 0n2
Behaviors : 0n9
Runtime Type : Test.GlobalInformation
Is Throttled? : True
Calls/Max Calls : 0n0/0n128
Sessions/Max : 0n0/0n800
Events Raised : No
Event raised
Handles Called : OnOpeningHandle OnOpenedHandle
Session Mode : False
Extensions : 1daac844
Service Behaviors
================================
Concurrency Mode : Single
Instance Mode : PerSession
Add Error in Faults: false
Max Items Obj Graph: 0n2147483647
Isolation Level : Unspecified
Session Shutdown : Automatic
ASP.NET Compatib : Allowed
Http Get Enabled : true
Https Get Enabled : false
Mex Enabled : true
All Service Behav : 17ab94ec
Service Base Addresses
================================
localhost/CustomerGlobal/GlobalInformation.svc
https://localhost/CustomerGlobal/GlobalInformation.svc
Channels
================================
Address : 17ABDCE4
Listener URI : localhost /CustomerGlobal/GlobalInformation.svc
Binding Name : <tempuri.org/:BasicHttpBinding>
Aborted : No
State : Opened
Transaction Type : No transaction
Listener State : Opened
Timeout settings : Open [00:01:00] Close [00:01:00] Receive: [00:10:00] Send: [00:01:00]
Server Capabilities: SupportsServerAuth [No ] SupportsClientAuth [No ] SupportsClientWinIdent [No ]
Request Prot Level : None
Response Prot Level: None
Events Raised : No Event raised
Handles Called : OnOpeningHandle OnOpenedHandle
Session Mode : False
Address : 17AC5998
Listener URI : localhost /CustomerGlobal/GlobalInformation.svc
Binding Name : ServiceMetadataBehaviorHttpGetBinding
Aborted : No
State : Opened
Transaction Type : No transaction
Listener State : Opened
Timeout settings : Open [00:01:00] Close [00:01:00] Receive: [00:10:00] Send: [00:01:00]
Server Capabilities: SupportsServerAuth [No ] SupportsClientAuth [No ] SupportsClientWinIdent [No ]
Request Prot Level : None
Response Prot Level: None
Events Raised : No Event raised
Handles Called : OnOpeningHandle OnOpenedHandle
Session Mode : False
Address : 17AC77AC
Listener URI : https:// localhost /CustomerGlobal/GlobalInformation.svc
Binding Name : ServiceMetadataBehaviorHttpGetBinding
Aborted : No
State : Opened
Transaction Type : No transaction
Listener State : Opened
Timeout settings : Open [00:01:00] Close [00:01:00] Receive: [00:10:00] Send: [00:01:00]
Server Capabilities: SupportsServerAuth [Yes] SupportsClientAuth [No ] SupportsClientWinIdent [No ]
Request Prot Level : EncryptAndSign
Response Prot Level: EncryptAndSign
Events Raised : No Event raised
Handles Called : OnOpeningHandle OnOpenedHandle
Session Mode : False
Endpoints
================================
Address : 17AB85C8
URI : localhost /CustomerGlobal/GlobalInformation.svc
Is Anonymous : False
Configuration Name : Test.IGlobalInformation
Type Name : Test.IGlobalInformation
Listening Mode : Explicit
Class Definition : 04113e30 Test.IGlobalInformation
Behaviors : 17ab94b8
Binding : 17ab6b34
0:000> !wselect * from 17ab6b34
[System.ServiceModel.BasicHttpBinding]
(string)System.String name = BasicHttpBinding
(string)System.String namespaceIdentifier = tempuri.org/
System.TimeSpan closeTimeout = -mt 737F7E8C 00000000 00:01:00
System.TimeSpan openTimeout = -mt 737F7E8C 00000000 00:01:00
System.TimeSpan receiveTimeout = -mt 737F7E8C 00000000 00:10:00
System.TimeSpan sendTimeout = -mt 737F7E8C 00000000 00:01:00
System.ServiceModel.Channels.HttpTransportBindingElement httpTransport = 17AB6B7C
System.ServiceModel.Channels.HttpsTransportBindingElement httpsTransport = 17AB6C28
System.ServiceModel.Channels.TextMessageEncodingBindingElement textEncoding = 17AB6C94
System.ServiceModel.Channels.MtomMessageEncodingBindingElement mtomEncoding = 17AB8250
System.ServiceModel.BasicHttpSecurity basicHttpSecurity = 17AB8294
(int32)System.ServiceModel.WSMessageEncoding messageEncoding = 0 (0n0) Text
0:000> !wselect * from 17ab6c28
[System.ServiceModel.Channels.HttpsTransportBindingElement]
(int64)System.Int64 maxBufferPoolSize = 80000 (0n524288)
(int64)System.Int64 maxReceivedMessageSize = 10000 (0n65536)
(bool)System.Boolean manualAddressing = 0 (False)
(bool)System.Boolean allowCookies = 0 (False)
(bool)System.Boolean bypassProxyOnLocal = 0 (False)
(bool)System.Boolean decompressionEnabled = 1 (True)
(string)System.String method = System.Uri proxyAddress = 00000000
(string)System.String realm = System.ServiceModel.Channels.WebSocketTransportSettings webSocketSettings = 17AB6BE0
System.Net.IWebProxy webProxy = 00000000
System.Security.Authentication.ExtendedProtection.ExtendedProtectionPolicy extendedProtectionPolicy = 17AB6C10
System.ServiceModel.Channels.HttpAnonymousUriPrefixMatcher anonymousUriPrefixMatcher = 00000000
System.ServiceModel.Channels.HttpMessageHandlerFactory httpMessageHandlerFactory = 00000000
(int32)System.Net.AuthenticationSchemes authenticationScheme = 8000 (0n32768) Anonymous
(int32)System.ServiceModel.HostNameComparisonMode hostNameComparisonMode = 0 (0n0) StrongWildcard
(int32)System.Int32 maxBufferSize = 10000 (0n65536)
(int32)System.Net.AuthenticationSchemes proxyAuthenticationScheme = 8000 (0n32768) Anonymous
(int32)System.ServiceModel.TransferMode transferMode = 0 (0n0) Buffered
(int32)System.Int32 maxPendingAccepts = 0 (0n0)
(bool)System.Boolean keepAliveEnabled = 1 (True)
(bool)System.Boolean inheritBaseAddressSettings = 0 (False)
(bool)System.Boolean maxBufferSizeInitialized = 0 (False)
(bool)System.Boolean unsafeConnectionNtlmAuthentication = 0 (False)
(bool)System.Boolean useDefaultWebProxy = 1 (True)
System.TimeSpan requestInitializationTimeout = -mt 737F7E8C 00000000 00:00:00
System.ServiceModel.MessageSecurityVersion messageSecurityVersion = 00000000
(bool)System.Boolean requireClientCertificate = 0 (False)
0:000> !wselect * from 17ab6c94
[System.ServiceModel.Channels.TextMessageEncodingBindingElement]
System.Xml.XmlDictionaryReaderQuotas readerQuotas = 17AB81C0
System.ServiceModel.Channels.MessageVersion messageVersion = 17AB8160
System.Text.Encoding writeEncoding = 1DAA6344
(int32)System.Int32 maxReadPoolSize = 40 (0n64)
(int32)System.Int32 maxWritePoolSize = 10 (0n16)
0:000> !wselect * from 17ab81c0
[System.Xml.XmlDictionaryReaderQuotas]
(int32)System.Int32 maxStringContentLength = 2000 (0n8192)
(int32)System.Int32 maxArrayLength = 4000 (0n16384)
(int32)System.Int32 maxDepth = 20 (0n32)
(int32)System.Int32 maxNameTableCharCount = 4000 (0n16384)
(int32)System.Int32 maxBytesPerRead = 1000 (0n4096)
(int32)System.Xml.XmlDictionaryReaderQuotaTypes modifiedQuotas = 0 (0n0)
(bool)System.Boolean readOnly = 0 (False)
static System.Xml.XmlDictionaryReaderQuotas defaultQuota = 17AB81E4
static System.Xml.XmlDictionaryReaderQuotas maxQuota = 17AB8208
From the highlighted you can see that WCF binding is not taking the values that is configured in the web.config, instead it is taking the default values available with default BasicHttpBinding.
Outcome:
The binding configured for WCF service is incorrect. Now we will write a new binding itself and will check if that works.
After writing new binding, everything worked as expected when we configured following settings as shown below:
maxBufferSize="2147483647"
maxBufferPoolSize="2147483647"
maxReceivedMessageSize="2147483647"
ReadersQuoata values are also set for max:
maxStringContentLength="2147483647"
maxArrayLength="2147483647"
maxBytesPerRead="2147483647"
maxNameTableCharCount="2147483647"
So what was the problem with the previous binding!!! After thorough checking, I found that the name of the WCF service was incorrect and this lead the WCF to use the default BasicHttpBinding that will be available even if you do not declare one.
So if you are getting a 413 with WCF service, following data must be collected at a very first step:
1- FREB traces.
2- WCF traces.
FREB will highlight the module setting up the 413 and from here we will get the indication if 413 is coming from WCF.
WCF traces will show you if the service is using default binding.
Finally, dump will give you all the values in the binding so you can conclude that WCF service is somehow not reading the configured binding and this is leading to this (413) Request Entity Too Large…
Hope this helps!!
Ashutosh Tripathi
MSFT(India)