WCF: Exposing metadata over NetTcp Binding issue

WCF: Exposing metadata over NetTcp Binding issue

 

Problem Description:

You will find your WCF service unresponsive and when you will collect the traces it will show you following exception:

<ExceptionString>System.Net.Sockets.SocketException (0x80004005): An existing connection was forcibly closed by the remote host

   at System.ServiceModel.Channels.SocketConnection.HandleReceiveAsyncCompleted()

   at System.ServiceModel.Channels.SocketConnection.OnReceiveAsync(Object sender, SocketAsyncEventArgs eventArgs)

</ExceptionString>

<NativeErrorCode>2746</NativeErrorCode>

 

When you will analyze it little more and you will observe that before throwing the above exception you will see following warning thrown by the service. This indicates a throttling is being hit.  

<TraceRecord xmlns="schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Warning">

<TraceIdentifier>msdn.microsoft.com/en-US/library/System.ServiceModel.Channels.ServerMaxPooledConnectionsQuotaReached.aspx</TraceIdentifier>

<Description>MaxOutboundConnectionsPerEndpoint quota (10) has been reached, so the connection was closed and not reused by the listener . </Description>

 

Solution:

Service is hosted over IIS and is exposed at netTcpBinding and security is transport security with client credential type as “Windows”.

So what is happening internally, there may be 2 reasons for this failure:

1-      Due to heavy load in the server this MaxOutboundConnectionsPerEndpoint value which is default 10 for netTcpBinding.

2-      Load is not heavy but your client proxies are not releasing the connections.

Let’s believe that client side programming is good and client connections are properly disposed. So we are considering the first scenario and we will write a custom binding to address this issue.

Here is the custom binding that will address this issue:

 

 

Now there are 2 ways to expose the metadata of the service:

1-      Expose it with HTTP mex address with mexHttpBinding

2-      Expose this with netTcp mex address as netTcpBinding

Now, if you expose the metadata over HTTP using system generated binding, it will work fine with IIS hosted environment. As soon as you expose the metadata at netTcpBinding, you will start running into the following issue.

 

 

 

Oops!! What happened?

Let’s check the client side binding and we see we have not updated the service reference after adding new binding. So let’s update it and try again!

First we need to expose the metadata at mexTcpBinding binding and then we need to update the service reference. Our endpoint configuration will look like this.

 

As soon as you will update your service reference you will find the below exception:

 

What is the problem?

Why this failing though I have exposed the service metadata and binding is available. Let’s trace it again:

Here is the exception:

“There is no compatible TransportManager found for URI 'net.tcp://ashu-16.fareast.corp.microsoft.com/Timeout_Demo_Time_Consumer/Service1.svc/Tcp'. This may be because you have used an absolute address that points outside of the virtual application, or the binding settings of the endpoint do not match those that have been set by other services or endpoints. Note that all bindings for the same protocol should have the same settings in the same application”

 

This is happening as we are using the custom binding to expose the service while metadata is still communicating at system provided mexTcpBinding. Though we are using the netTcp transport underneath but we have made little amendments in the basic settings (default settings) of Tcp transport as the value of MaxOutboundConnectionsPerEndpoint .

So we need to use the Custom Binding itself to expose the metadata. So our end point configuration should look like as below:

 

So your client side  app.config will look like as below:

 

 

Now everything works as expected.

Now let’s try another thing. We will try to create another service reference at client app with the net.tcp address url i.e. at

net.tcp://ashu-16.fareast.corp.microsoft.com/Timeout_Demo_Time_Consumer/Service1.svc/Tcp

As soon as you try to add new service reference from your visual studio and give above address to svcuiti.exe you will encounter with following error:

“The URI prefix is not recognized.

Metadata contains a reference that cannot be resolved: 'net.tcp://ashu-16.fareast.corp.microsoft.com/Timeout_Demo_Time_Consumer/Service1.svc/Tcp'.

The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '00:04:59.9440395'.

An existing connection was forcibly closed by the remote host

If the service is defined in the current solution, try building the solution and adding the service reference again.”

 

Let’s trace the server side again:

 

 

Which says “Stream Security is required at www.w3.org/2005/08/addressing/anonymous, but no security context was negotiated. This is likely caused by the remote endpoint missing a StreamSecurityBindingElement from its binding. ”

This exception is appearing as we have used “WindowsStreamSecurity” in custom binding as shown below:

 

 

This security is conflicting with IIS security as SvcUtil.exe runs with no security and IIS asks for a security.

This can be fixed in two ways:

1-      Change the security settings at IIS (highly not recommended as it will start impacting all the applications at IIS)

2-      Expose the metadata at HTTP and create the reference.

So, if you expose the metadata over http and create the reference, it will give you the reference to you. You can force your client to communicate over netTcp once you have metadata available with you.

 

Note: If you host your WCF service as windows service, the net.tcp url will work fine without any issue.

 

Ashutosh Tripathi

Microsoft India

Comments

• Anonymous
  November 25, 2016
  Hi,Thanks for the detailed article. I followed your steps but I had an error. I'm unable to update the service reference.Please help me in resolving the issue.System.ServiceModel.ServiceHostingEnvironment+HostingManager/43123581 System.ServiceModel.ServiceActivationException: The service cannot be activated due to an exception during compilation. The exception message is: Unrecognized attribute 'maxOutboundConnectionsPerEndPoint'. Note that attribute names are case-sensitive. System.Configuration.ConfigurationErrorsException: Unrecognized attribute 'maxOutboundConnectionsPerEndPoint'.