The first WCF call always timed out?

This is a scenario where both the WCF client and service are running in the same AppDomain inside a MVC application. The WC F call originates from MVC controller. In addition, the WCF service host was created in the same function as well. The code looks like: 

public ActionResult Index() {             //Create the service host             ServiceHost host = new ServiceHost(typeof(HelloWorld));             host.Open();             //Create the client             EndpointAddress address = new EndpointAddress("net.pipe://localhost/WCFSelfHostInMVC/HelloWorld");             NetNamedPipeBinding binding = new NetNamedPipeBinding();             binding.Security.Mode = NetNamedPipeSecurityMode.None;             ChannelFactory<IHelloWorld> factory = new                                 ChannelFactory<IHelloWorld>(binding, address);             IHelloWorld channel = factory.CreateChannel();            // Calling the service             ViewBag.Message = channel.DoWork();             return View(); }

We captured a dump, and found it is a deadlock scenario. The WCF client was owning a lock(AspNetSynchronizationContext) and waiting the response of the service. However, the service tried to lock AspNetSynchronizationContext object as well to handle the request.

Here is the stack of client, which was waiting the response from service. This thread owns the lock of AspNetSynchronizationContext object.

0:025:x86> ~22kL ChildEBP RetAddr  04b1df34 74f90bdd ntdll_77320000!ZwWaitForMultipleObjects+0x15 04b1dfd0 76681a2c KERNELBASE!WaitForMultipleObjectsEx+0x100 04b1e018 6dcff2ca kernel32!WaitForMultipleObjectsExImplementation+0xe0 04b1e07c 6dcff0f8 clr!WaitForMultipleObjectsEx_SO_TOLERANT+0x56 04b1e09c 6dcfef38 clr!Thread::DoAppropriateAptStateWait+0x4d 04b1e130 6dcfeff9 clr!Thread::DoAppropriateWaitWorker+0x17d 04b1e19c 6ddd9d46 clr!Thread::DoAppropriateWait+0x60 04b1e29c 025cb5ef clr!WaitHandleNative::CorWaitOneNative+0x196 04b1e2b4 025ab1ee mscorlib_ni!System.Threading.WaitHandle.InternalWaitOne(System.Runtime.InteropServices.SafeHandle, Int64, Boolean, Boolean)+0x2b 04b1e2d8 500cf2fc mscorlib_ni!System.Threading.WaitHandle.WaitOne(System.TimeSpan, Boolean)+0x6e 04b1e2f0 016042f0 System_Runtime_DurableInstancing_ni!System.Runtime.TimeoutHelper.WaitOne(System.Threading.WaitHandle, System.TimeSpan)+0x6c 04b1e308 517cbef8 System_ServiceModel_ni!System.ServiceModel.Channels.OverlappedContext.WaitForSyncOperation(System.TimeSpan, System.Object ByRef)+0x40 04b1e32c 517cb1bf System_ServiceModel_ni!System.ServiceModel.Channels.PipeConnection.WaitForSyncRead(System.TimeSpan, Boolean)+0x38 ……. 04b1e6f8 5179fb05 System_ServiceModel_ni!System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(System.Runtime.Remoting.Messaging.IMethodCallMessage, System.ServiceModel.Dispatcher.ProxyOperationRuntime)+0x59 04b1e740 0258a25e System_ServiceModel_ni!System.ServiceModel.Channels.ServiceChannelProxy.Invoke(System.Runtime.Remoting.Messaging.IMessage)+0x65 04b1e774 6dcc2336 mscorlib_ni!System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(System.Runtime.Remoting.Proxies.MessageData ByRef, Int32)+0xee

This thread is the service thread. However, this thread was trying to lock the AspNetSynchronizationContext object owned by thread 22. This results in the deadlock.

0:025:x86> kL ChildEBP RetAddr  053beba0 74f90bdd ntdll_77320000!ZwWaitForMultipleObjects+0x15 053bec3c 76681a2c KERNELBASE!WaitForMultipleObjectsEx+0x100 053bec84 6dcff2ca kernel32!WaitForMultipleObjectsExImplementation+0xe0 053bece8 6dcff0f8 clr!WaitForMultipleObjectsEx_SO_TOLERANT+0x56 …….. 053bef00 6ddd25a6 clr!AwareLock::EnterEpilogHelper+0xa8 053bef40 6ddd25eb clr!AwareLock::EnterEpilog+0x42 053bef60 6ddf78d1 clr!AwareLock::Enter+0x5f 053bf01c 025ce0c4 clr!JIT_MonReliableEnter_Portable+0x104 053bf02c53ef5477 mscorlib_ni!System.Threading.Monitor.Enter(System.Object, Boolean ByRef)+0x14 053bf05c 53ef55e9 System_Web_ni!System.Web.AspNetSynchronizationContext.CallCallback(System.Threading.SendOrPostCallback, System.Object)+0x53 053bf084 517bec09 System_Web_ni!System.Web.AspNetSynchronizationContext.Post(System.Threading.SendOrPostCallback, System.Object)+0x9 ……. 053bf164 517bd65b System_ServiceModel_ni!System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean)+0xff …………………………..

In short, the service thread was trying to lock the object(AspNetSynchronizationContext) owned by the WCF client thread, but client thread was blocked and waiting the service call back.

1. The WCF client thread (MVC thread) was holding the AspNetSynchronizationContext lock as it is an ASP.NET request and it was waiting the response of WCF service call.

2. The WCF service thread was waiting for the AspNetSynchronizationContext lock to process the WCF request. 

The reasons WCF is using the AspNetSynchronizationContext are:

- The ServiceBehavior – UseSynchronizationContext is set to true (by default).

- The WCF service host is created under ASP.NET context (As the code demoed, it was created inside the MCV controller.) 

There are several workarounds:

1. Setting UseSynchronizationContext to false.

2. Creating the service host in the Application_Start

3. Using the Service.svc file. 

In addition, only .Net 4.0 has this problem, .Net 3.5 doesn’t have this issue.

Wei from APGC DSI Team