Scale WCF Application Better with Asynchronous Programming
WCF is a framework that is asynchronous inside out. Internally, asynchronous patterns are used to achieve non-blocking calls between different layers inside WCF. Externally WCF exposes the asynchronous patterns for both client and service programming.
Why Asynchronous Pattern?
From performance perspective, it’s not always ideal to use asynchronous programming if you don’t have to. Any asynchronous pattern involves at least some thread switches and thus causes delays to requests. So you should not abuse it if you don’t have to.
However, for blocking calls such as I/O operations or remote calls, you should always thinking about using asynchronous patterns. In those cases, you would be able to reduce number of threads used in the system while waiting for operations to complete. Here are a few cases where you would like to use asynchronous patterns:
1) A WCF service operation is waiting for a database operation to complete.
2) A WCF service operation calls into another WCF client proxy of a backend WCF service.
3) A UI thread waits for data from WCF client calls.
4) Any other case when the calling thread is blocked to wait for a slow operation but non-CPU bound operation to complete.
If you are using synchronous calls for the above cases, you would clearly see the threads are blocked from a dump of the process. This is wasting resources and you are preventing more requests from being processed if the system is capable of. This becomes more serious if you are holding hundreds or thousands of threads. So whenever you are seeing a thread blocked waiting for something, you would have to think about why that is the case? Why can’t the thread go away and come back with a different thread when there is real work to do?
Of course in reality, there are a small number of threads doing bookkeeping work, for example, they are doing event wait handling, finalizing, garbage collection, AppDomain unloading, ThreadPool gating, timer handling, etc. The rest of threads are worker threads or I/O threads that do the real work and they are what you need to pay attention to.
Basic Asynchronous Pattern with IAsyncResult
There are a few common asynchronous programming patterns with .NET Framework. Here is a rough list:
1) Begin/End pattern with IAsyncResult
2) Raw Callback model using a delegate as in C++ world
3) Implicit callback model with synchronous APIs
WCF advocates the first approach from the high level and uses it in most of its implementation. For performance critical parts, WCF uses 3) to avoid extra GC costs dealing with IAsyncResult objects. Here I just want to describe the basic idea of how to use 1).
The basic pattern of using IAsyncResult is to expose a pair of Begin/End operations from your type as following:
public class IAsyncWork
{
public IAsyncResult BeginDoWork(MyPropertyBag propertyBag, AsyncCallback callback, object state);
public WorkResult EndDoWork(IAsyncResult result);
}
When a caller invokes the methods of this interface, it supplies some custom arguments (here it’s “propertyBag”), a callback delegate, and the state object that will be consumed by the callback. The callback delegate wraps a callback method of the caller with the following type:
public delegate void AsyncCallback(IAsyncResult ar);
For best performance, here are two principles when you call/implement the above asynchronous pattern:
· Principle 1: Do not do heavy-weighted work inside the Begin method (which is BeginDoWork above).
The reason for this is that you should return the calling thread as soon as possible so that the caller can schedule other work. If it’s a UI thread, the application needs to use the thread to respond to user inputs. You should always put heavy operations in a different thread if possible.
· Principle 2: Avoid calling End method (which is EndDoWork above) on the same thread of the Begin method.
The End method is normally blocking. It waits for the operation to complete. If you implement the End method, you would see that it actually calls IAsyncResult.WaitHandle.WaitOne(). On the other hand, as a normal implementation (for example, the sample attached in this Blog entry), this WaitHandle is a delay allocated ManualResetEvent. As long as you don’t call it, it would be not allocated at all. For fast operations, this is pretty cheap. However, once End is called, you would have to allocate it. The right place to call End is from the callback of the operation. When the callback is invoked, it means that the blocking work is really completed. At this point, you can call End to get data retrieved without sacrificing performance.
In online WCF samples, you would find a typical implementation IAsyncResult. I also included one in the sample attached.
Service Asynchronous Pattern
WCF provides asynchronous service operation contract support. An asynchronous service operation can be defined as following:
[ServiceContract
public interface IAsyncOrderService
{
[OperationContract(AsyncPattern=true)]
IAsyncResult BeginGetOrders(int numOrders, AsyncCallback callback, object state);
Order[] EndGetOrders(IAsyncResult result);
}
As you can see, other than the ServiceContract/OperationContract attributes, the interface is exactly the same as standard asynchronous patterns. One thing to note is that it is not your code but WCF that calls into these methods. So the operation names have to follow the Begin/End convention.
Another minor thing to note is that, if you also have a synchronous operation for the service that matches the signature of the asynchronous operation as following, WCF would call this synchronous operation instead of the asynchronous operation:
Order[] GetOrders(int numOrder);
You may have control of which one to call in future releases.
Asynchronous Database Access
Now it’s time for you to think about your own service implementation. In order to leverage the asynchronous advantage of WCF, you need to implement your service to be purely asynchronous so that no threads are blocking.
In the sample that I attached to this blog, I used SQL database access with System.Data API as an example. For simplicity, I used SQL Express which is available for free. To get asynchronous support, you need to use the “Asynchronous Processing” property of the connection string as following:
const string ConnectionString = @"Server=LOCALHOST\SQLEXPRESS;Database=MyOrderDatabase;
Integrated Security=True;
Asynchronous Processing=True";
Once you have the asynchronous support from the SqlConnection, you can invoke the asynchronous API SqlCommand.BeginExecuteReader as following:
IAsyncResult sqlAsyncResult = this.command.BeginExecuteReader(sqlGetOrdersCallback, this);
With this, you would have a good asynchrony for your service side.
Client Asynchrony
On the client side, you can also asynchronous pattern to access the service. When you invoke the service operations, you would call into the asynchronous methods of the service contract. For example, the sample calls it in the following way:
IAsyncResult result = client.BeginGetOrders(driver.numOrders, onGetOrders, this);
It supplies a callback delegate that handles the response when the operation is completed.
One important thing to note is that WCF is a fully decoupled platform. This means that the client-side asynchrony has nothing to do with the service-side asynchrony. So an asynchronous client can basically invoke either a synchronous or an asynchronous service. The decoupling happened at the transport layer when data is serialized or deserialized as byte arrays and being transmitted through the transport.
If you are building N-tier WCF services, you would have WCF service operations invoking WCF client proxies for other backend services. In this case, you would need to make sure that the middle-tier (routing layer) has asynchronous service operation invoking asynchronous WCF proxy operations. In this way, your middle-tier won’t run out of threads when processing many slow operations.
Exception Handling
The logic of exception handling for asynchronous programming is more complicated than synchronous programming. Basically when an exception is thrown from a callback thread, you should not let it bubble up to the caller, which is almost the top of the callstack of a worker thread or an I/O thread. If you do so, your application would blow up as no other code would catch the exception and handle it.
To best handle an exception happened from the callback thread, you would need to record the exception as a state somewhere, for example, in the AsyncResult as showing in the sample. At some point, when the End method is invoked, this exception will be thrown on that thread and thus causes the exception to be thrown on the caller’s thread.
In the sample attached, we showed this once in SqlGetOrdersAsyncResult. In its callback “OnSqlGetOrdersCallback”, it catches the exception and passes to the AsyncResult.Complete method which then stores the exception there. When the SqlGetOrdersAsyncResult.End is invoked from WCF, this exception would be rethrown from AsyncResult.End. In this way, WCF would be able to catch the exception internally and convert it into a fault and send it to the client side.
Sample Setup
The attached sample implements a simple performance test that demonstrates the 3-tier model as showing above: WCF client, WCF service, and SQL database. The code is pretty self-explanatory and I copy the readme file here:
Prerequisite
1) It’s better to use Windows 2008 so that you can leverage the asynchronous WCF HttpModule/HttpHandler that were released in .NET 3.5 SP1. Check here.
2) Visual Studio 2008 or above installed.
3) SQL Express or SQL server installed
4) IIS, IIS Management tools, and WCF HTTP Activation components installed.
Steps to Run the Test
1) Open the solution file "WCFOrderService.sln" with VS 2008 and compile the whole solution with "Release" type.
2) Run "SQL\SetupSQL.bat" to create database "MyOrderDatabase".
3) Run "OrderCreator\bin\Release\OrderCreator.exe 1000" to insert 1000 distinct orders into database.
4) Open IIS Manager, create a virtual application that points to the full path of "OrderService".
5) Use IIS Manager to configure the default ApplicationPool to have identity "OrderPerfUser". This user was created in step 2) above.
6) Browse the OrderService.svc file in the browser. Make sure that you can see the WCF service correctly responds to the request from browser.
7) Run the client "OrderClient\bin\Release\OrderClient.exe"
8) Tune different parameters in the OrderService\web.config and OrderClient\bin\Release\OrderClient.exe.config
9) Use the tool "Tools\WcfAsyncWebUtil.exe" to configure WCF to use the async HttpModule and increase the ASP.NET registry "MaxConcurrentRequestsPerCpu". Run the test again to see the difference.
10) Open PerfMon to monitor requests such as "Calls Outstanding" counter.
Comments
Anonymous
February 08, 2009
PingBack from http://www.anith.com/?p=7748Anonymous
February 12, 2009
Good idea to use async calls, but won't this render your source code nearly unreadable, and there unmaintainable, since program logic will be really hard to follow if functionaly is spread over a lot of methods?Anonymous
May 07, 2009
I download wcfasyncService codes,and deploy it follow the manual,but when i start client program,the consol window display the start test but never end and return. IAsyncResult result = client.BeginGetOrders(driver.numOrders, onGetOrders, this); if (result.CompletedSynchronously) { HandleResult(result); } result.CompletedSynchronously always return false. can you help me solve the problem?thxAnonymous
December 03, 2009
This sounds like a good performance boost. Is it possible to call an Asynchronous WCF service from javascript(Microsoft ajax)? Does the current WCF implementation support it or we have to write our own WcfScriptHandler.Anonymous
January 21, 2010
How can I deploy OrderService? When I start OrderClient it throws an exception about listening http://localhost/OrderService/OrderService.svcAnonymous
February 13, 2011
The comment has been removedAnonymous
December 23, 2012
Though you have provided a working sample, but its impossible to get the hang of what's happening from your sample, because there are too many chained method calls in your sample which make it result in more and more confusion. I could not really understand your sample. It would be better if you just had non-chained method calls to show how begin and end methods are used in WCF async pattern.