Distributed Cache in Azure: Part II

In my last post, I talked about creating a simple distributed cache in Azure.   In reality, we aren’t creating a true distributed cache – what we’re going to do is allow each server manage its own cache, but we’ll use WCF and inter-role communication to keep them in sync.   The downside of this approach is that we’re wasting n* more RAM because each server has to maintain its own copy of the cached item.   The upside is:  this is easy to do.

So let’s get the obvious thing out the way.  Using the built in ASP.NET Cache, you can add something to the cache like so that inserts an object that expires in 30 minutes:

 1 Cache.Insert("some key",
2   someObj,
3   null,
4   DateTime.Now.AddMinutes(30),
5   System.Web.Caching.Cache.NoSlidingExpiration); 

With this project, you certainly could use the ASP.NET Cache, but I decided to you use the Patterns and Practices Caching Block.  The reason for this:  it works in Azure worker roles.  Even though we’re only looking at the web roles in this example, it’s flexible enough to go into worker roles, too.  To get started with the caching block, you can download it here on CodePlex. 

The documentation is pretty straightforward, but what I did was just set up the caching configuration in the web.config file.  For worker roles, you’d use an app.config:

  1   <cachingConfiguration defaultCacheManager="Default Cache Manager">
 2     <backingStores>
 3       <add name="inMemory"
 4     type="Microsoft.Practices.EnterpriseLibrary.Caching.BackingStoreImplementations.NullBackingStore, Microsoft.Practices.EnterpriseLibrary.Caching" />
 5     </backingStores>
 6 
 7     <cacheManagers>
 8       <add name="Default Cache Manager"
 9     type="Microsoft.Practices.EnterpriseLibrary.Caching.CacheManager, Microsoft.Practices.EnterpriseLibrary.Caching"
10     expirationPollFrequencyInSeconds="60"
11     maximumElementsInCacheBeforeScavenging="250"
12     numberToRemoveWhenScavenging="10"
13     backingStoreName="inMemory" />
14     </cacheManagers>
15   </cachingConfiguration>

The next step was creating a cache wrapper in the application.  Essentially, it’s a simple static class that wraps all the insert/deletes/etc. from the underlying cache.  It doesn’t really matter what the underlying cache is.  The wrapper is also responsible for notifying other roles about a cache change.   If you’re a purist, you’re going to see that this shouldn’t be a wrapper, but instead be implemented as a full fledged cache provider since it isn’t just wrapping the functionality.   That’s true, but again, I’m going for simplicity here – as in, I want this up and running _today_, not in a week.

Remember that the specific web app dealing with this request knows whether or not to flush the cache.  For example, it could be a customer updates their profile or other action that only this server knows about.  So when adding or removing items from the cache, we send a notify flag that instructs all other services to get notified…

  1 public static void Add(string key, object value, CacheItemPriority priority,
 2     DateTime expirationDate, bool notifyRoles)
 3 {           
 4     _Cache.Add(key,
 5         value,
 6         priority,
 7         null,
 8         new AbsoluteTime(expirationDate)
 9         );
10 
11     if (notifyRoles)
12     {
13         NotificationService.BroadcastCacheRemove(key);
14     }
15 }        
16 
17 public static void Remove(string key, bool notifyRoles)
18 {
19     _Cache.Remove(key);
20 
21     if (notifyRoles)
22     {
23         Trace.TraceWarning(string.Format("Removed key '{0}'.", key));
24         NotificationService.BroadcastCacheRemove(key);                               
25     }
26 }

The Notification Service is surprisingly simple, and this is the cool part about the Windows Azure platform.  Within the ServiceDefinition file (or through the properties page) we can simply define an internal endpoint:

This allows all of our instances to communicate with one another.  Even better, this all maintained by the static RoleEnvironment class.  So, as we add/remove instances to our app, everything magically works.  A simple WCF contract to test this prototype looked like so:

  1 [ServiceContract]
 2 public interface INotificationService
 3 {       
 4     [OperationContract(IsOneWay = true)]
 5     void RemoveFromCache(string key);
 6 
 7     [OperationContract(IsOneWay = true)]
 8     void FlushCache();
 9   
10     [OperationContract(IsOneWay = false)]
11     int GetCacheItemCount();
12 
13     [OperationContract(IsOneWay = false)]
14     DateTime GetSettingsDate(); 
15 }

In this case, I want to be able to tell another service to remove an item from its cache, to flush everything in its cache, to give me the number of items in its cache as well as the ‘settings date’ which is the last time the settings were updated.  This is largely for prototyping to make sure everything is in sync.

We’ll complete this in the next post where I’ll attach the project you can run yourself, but the next steps are creating the service and a test app to use it.    Check back soon!