D3: Implementing the WCF Service

D3 release 0.1322 is now available.  The work that went into this release is all about building out the WCF service which D3 will use as the interface between our mid-tier and the client.  I encountered a grab bag of issues along the way, so here’s a list of tips & tricks as well as random thoughts on the topic of building a WCF service with the EF.

Authentication

Somewhere along the way between the last D3 release and this one I had knee surgery.  My recovery was actually pretty quick, but I did end up spending most of a week sitting at home with my knee in the air and a computer on my lap.  I spent a frighteningly large amount of time trying to read up on ways to do authentication using WCF for the D3 project.  One of the best blog posts I found was this:

<blogs.infosupport.com/blogs/alexb/archive/2009/10/02/silverlight-3-securing-your-wcf-service-with-a-custom-username-and-password-authentication-mechanism.aspx>

The overall approach really seemed to match my general goals in the sense that I want the service to be essentially secure and WCF to handle that security in a relatively transparent way, but I also want to have my own username and password tracking in my EF model (I don’t want to create windows accounts on a domain controller for every user of D3 or something like that).  The issue that drove me nuts is the fact that this approach absolutely requires SSL and at the time I was working on it there really wasn’t a good way to get SSL running (even with just a test cert) inside of VS because Cassini (the mini-webserver built into VS) doesn’t support it. 

Since then the IISExpress project seems to have gotten off the ground, and it may well be the right long-term answer, but after much banging of my head against this particular wall, I decided that for the current interim release I would completely fake all of this in a hacky way that would leave us setup for the right thing but be clear to implement now.  What I did was to add a username and password parameter to every method on the service.  We’ll return to this in a later release and make things more secure and nicer to use.

Service Contract Design

As I have written in other places (such as this short series of MSDN Magazine Articles: Anti-Patterns To Avoid In N-Tier Applications, N-Tier Application Patterns & N-Tier Apps and the Entity Framework: Building N-Tier Apps with EF4), there are lots of patterns/approaches that can and should be considered when designing a service.  While especially typical intranet line-of-business applications will benefit from the developer productivity and simplicity of an approach like self-tracking entities or RIA services, in this case my goal is to create a game that will run over the internet which means that I’d like to take more control over the contract. 

That led me to the decision that D3’s service should be fairly strictly SOA—at least for the most commonly used methods (issuing commands from the client and retrieving events from the server).  For these two core commands, I decided that the service should neither accept entities as parameters nor return them as results.  We just want the simplest, most basic possible interface, and the result is the following two method signatures:

 [OperationContract, FaultContract(typeof(CommandFault)), ApplyProxyResolver]
void ExecuteCommand(string userName, string password, int playerId, string command);

[OperationContract]
Tuple<IEnumerable<string>, int> GetEventMessages(string userName, string password, 
                                                 int playerId, int lastSeenEventId);

Ignoring the authentication parameters (which as discussed above should go away in a later release), the first method takes a player ID and a string representing the command that the player wants to execute—typically this will be something like “go north”, “get fancy sword” or “attack purple snorklewacker”.  You’ll notice that this method does not return any results—the only interesting result is if the command is invalid in which case the service will return a CommandFault with the error message.

The way the service returns the results of commands as well as the observable actions of others in the game is through the GetEventMessages method.  It also takes a player ID to determine the perspective of the player from which the events are observed, and it takes the ID of the last event which was returned by a previous call to GetEventMessages.  This method returns a tuple which contains an enumerable of strings which are messages to display on the client plus the ID of the last event which produced a member of that last event of messages.

The idea is that the client will have two asynchronous UI interaction streams going on.  First, at any time the player can type a command and send it to the server, and assuming the command was valid the server will process the command using its business logic and create events representing the result of that command.  The second thing going on is that the client will periodically poll the server for new event messages observed by the player.  The events which produce these messages are both those created as a part of processing this player’s commands as well as events caused by everyone else in the game.  Each time the client calls the GetEventMessages method, it will get back any messages that it hasn’t yet seen plus an ID used in the next call to the method to indicate the “high-water mark” of what it has already seen.

Obviously the GetEventMessages method will be the most frequently called top-level method in the entire application (every client will poll it frequently in order to maintain its UI).  So it will be very perf sensitive.  In subsequent blog posts, I’ll take a look at the topic of tracking and tuning the perf of this method.

Unit Testing vs. Integration Testing

If you’ve been reading my blog, you probably know by now that I’m a huge fan of unit testing, and I’ve been putting a lot of work into appropriately unit testing D3 as we go.  This has led to finding and fixing a lot of bugs early on, increasing my confidence in my ability to refactor the project frequently without much risk of introducing new subtle bugs and generally a better overall structure for the project.  Unit testing, though, isn’t everything.  When unit testing we try to isolate small parts of the application so we can focus what we test on a very small piece to make it fast and make sure we know what is responsible when a test fails (so it’s easy to fix).  Sometimes, though, you’ve got to move to a coarser granularity and make sure that you have some tests to verify that all the units go together properly. 

This lesson came home to me in a big way when I decided to write a small integration test that would simulate the most common client / service interactions.  I had been unit testing various parts including directly calling the service method implementations, etc., but I hadn’t yet put it all together and run it through WCF.  The code I wanted to get working wasn’t exactly huge or hugely complicated.  It looked like this:

 [TestMethod]
public void KickoffScenario_Works()
{
    using (var svc = new PlayerServiceClient(
        new BasicHttpBinding(), 
        new EndpointAddress("localhost:9999/PlayerService")))
    {
        svc.CreateUser("simmdan", "password");

        var id = svc.CreatePlayer("simmdan", "password", "joe");
        svc.GetPlayers("simmdan", "password").Single(p => p.Id == id);
        var logOnEventId = svc.LogOn("simmdan", "password", id);

        svc.ExecuteCommand("simmdan", "password", id, "get item1");
        var eventMessages = svc.GetEventMessages("simmdan", "password", id, logOnEventId);

        var message = eventMessages.Item1.Single();
        Assert.AreEqual("You pick up the Item1.\n", message);
    }
}

As I discovered, though, there were a surprising number of issues that came up including some things that were quick and easy to find and some others that were pretty subtle and took a fair amount of hunting.  Along the way, one thing I found very helpful was this post about turning on WCF tracing: www.c-sharpcorner.com/UploadFile/shivprasadk/6876788978904302009095012AM/68767889789.aspx  Following the directions on that site I added the following XML to the app.config file in D3.Web.Tests:

 <system.diagnostics>
  <sources>
    <source name="System.ServiceModel" switchValue="Information, ActivityTracing">
      <listeners>
        <add name="log" type="System.Diagnostics.XmlWriterTraceListener" initializeData="c:\Traces.svclog" />
      </listeners>
    </source>
  </sources>
</system.diagnostics>

Common Issues When Using the EF & WCF Together

So all of that leads me to the following short list of the most common issues I encounter in my own code or when talking with others specific to using the EF and WCF together.

1. Not understanding the basic design patterns involved.   This is by far and away the most common issue.  For this I recommend the magazine articles I referred to in the Service Contract Design section above. 

   (Side Note: It used to be that at least as common as this one was the issue of trying to figure out how EF’s disconnected APIs work—even if you did understand the basic patterns, it could be very hard to implement them correctly with the EF. Happily, since EF4 shipped the APIs are much easier to use, so this is not so common any more.)

    

2. Disposing the context before enumerating a query which is returned from the service method.   This one will get you almost every time.  The general recommended pattern is that each service method should create a context instance in a using block so that it (and its underlying connection) will be disposed properly as soon as possible to free up precious server resources.  If you have a method that returns an enumerable of some type which comes from a query to the database, though, you have to make sure that you fully enumerate that query before the context is disposed.  If you just return the query itself, in a regular CLR method call where the context lifetime is managed some other way, that query would be cast to IEnumerable and the CLR would lazily evaluate it when the results were actually needed.  This is simple and efficient, but in a case where the context is being disposed before the method returns, though, the query results won’t be enumerated until the WCF service’s serializer attempts to process things after the method returns, and in that case the context will first be disposed and then later the query will be enumerated, and it will throw an exception because its connection to the database will be gone.  The fix?  Just call .ToList() on your query before returning.

    

3. Using POCO without thinking through WCF’s requirements for objects it serializes.   If you have a WCF service which accepts entities as parameters or returns them as results and you use POCO entities (maybe by using the POCO T4 template or maybe just writing them by hand), then you have to think about WCF’s requirements for its objects because your POCO objects can be almost anything and they may well not be setup properly to work with WCF.  The three ways this comes up most commonly for me are if you have dynamic proxies (either just for lazy loading or for full change tracking), if you have inheritance in your model which the DataContract serializer is unaware of, or if you want to return entire graphs of entities which have cycles.  You can read about the first two issues (and how to address them) here: blogs.msdn.com/b/adonet/archive/2010/01/05/poco-proxies-part-2-serializing-poco-proxies.aspx the key concepts are the DataContractResolver and KnownTypes.  For the last issue, the answer is in the IsReference=true parameter to the DataContract attribute.  This is a feature that was added in .Net 3.5sp1 which makes the DataContract serializer smarter about graphs of related objects.  You can read more about that here: zamd.net/2008/05/20/datacontract-serializer-and-isreference-property/ 

   One thing to watch out for is that if you have POCO entities with no DataContract attributes, the DataContract serializer will handle them fine up to a point, but if you start having more advanced scenarios like graphs of related entities or dynamic proxies, then the default conventions which the DataContract serializer uses will no longer work, and in that case the answer is to add the attributes.  If you are using the POCO template to generate your entities, for instance, it’s pretty easy to hack these into the T4 template which is what I did for D3.  If you are using the default codegen, then all of these things are taken care of for you automatically, but of course there are other advantages to POCO—it’s all a matter of tradeoffs.

OK.  I suppose that’s enough for now.  Happy WCF-service building.

- Danny