Caching ASP.NET Content and Leveraging VM Local Storage with Windows Azure
Very recently I had a customer who approached me with a concern, and their problem resembles some common architectural guidance we have been talking with other customers about. I am hopeful this post fills a void in one or two areas for many of you using Windows Azure.
The scenarios was that this customer has content that that cache from a third-party CDN service. The customer is not using the Windows Azure Storage CDN capabilities yet. The content cached in the CDN is used by their ASP.NET web application on Windows Azure, and is held within the role’s ASP.NET Cache via the System.Web.Caching.CacheDependency class, which usually picks up static content from the root of the website. Periodically, the customer would like to replace the file, invalidating the current ASP.NET cache content, and having it pickup the new (non-stale) content. Remember, caching works great for content that is static, and does not change often, but there are times you might need to change the content.
Normally, with a standard ASP.NET server, you would just perform a file replacement at the root website file system location and be done, as the CacheDependency class would detect the changes and perform the needful cache update. However, with Azure, you don’t have write access to the directory, and you would need to re-deploy your service to get the same effect, which could mean service degradation even with the proper upgrade domain configuration. Remember by default for a subscription, you are limited to 20 weighted role instances throughout all your subscriptions services, unless you have called support to increase your subscription quota. You also by default have an upgradeDomainCount of 5, unless you modify this attribute on your ServiceDefinition element in your ServiceDefinition.csdef file. So if you had 20 instances of small VM’s running, across 5 UpgradeDomains, doing a in-place upgrade of a service will leave you with 4 small VMs unavailable while the upgrade process (Fabric Controller) does it thing.
So to solve the issue, the idea is two-fold. First, move the file that the CacheDependency class depends upon to provisioned LocalStorage on the VM itself, which will give you the ability to update the file without performing a redeployment or upgrade. (LocalStorage is writeable, and makes for an excellent location for scratch files, temporary caches, configuration files and other non-durable (transient) storage needs. Second, provision the file that is in that LocalStorage from a durable storage location, in our case, Windows Azure Blob Storage, and implement a timer that periodically replaces that file. This has the benefit of allowing you to update one location, and have multiple instances pickup the changes, without having to change files manually on every instance, or redeploy your application packages. There is some “buyer beware” here, in that you do pay for storage of the file, just like you would for deployment packages stored in Azure Blob Storage, but the cost is minimal compared with the benefits. Additionally, you should ensure your storage account and roles are within the same datacenter geographic locations, or add calculations to include the bandwidth costs across different datacenters for the file updates. Again, this is generally a minimal amount compared to the costs to geo-replicate files on your own.
Since I just returned from a (much needed and enjoyed) vacation to Canada, I didn’t have time to write a sample for this. However, I turned to my good buddy and co-worker at Microsoft, Balakrishna Mishro (Wipro), who was able to drum up a quick code sample while I was in yet another meeting.
The code sample, which you can download (LocalStorageSample.zip - 166kb) and use at your own risk, was created by simply creating a new ASP.NET web role, with a new BackgroundWorker class that is instantiated on Application_Start, via the Global.asax codebehind, passing the Cache property of the HTTPApplication Context in.
The BackgroundWorker class does all the “heavy” lifting, and we use the StorageManagement API to retrieve the file via the CloudBlob.DownloadToFile method on a simple timer. (Note: Added some line breaks here for readability.) :
1: using System;
2: using System.Collections.Generic;
3: using System.Linq;
4: using System.Web;
5: using System.Threading;
6: using System.Timers;
7: using System.Configuration;
8: using Microsoft.WindowsAzure;
9: using Microsoft.WindowsAzure.StorageClient;
10: using Microsoft.WindowsAzure.ServiceRuntime;
11: using System.Web.Caching;
12: using System.IO;
13:
14: namespace LocalStorageWebRole
15: {
16: public class BackgroundWorker
17: {
18: System.Timers.Timer timer = null;
19: Cache cache = null;
20:
21: public BackgroundWorker(Cache application)
22: {
23: timer = new System.Timers.Timer(1);
24: timer.Interval = 1;
25: timer.Elapsed += new ElapsedEventHandler(timer_Elapsed);
26: timer.Enabled = true;
27: this.cache = application;
28: }
29:
30: void timer_Elapsed(object sender, ElapsedEventArgs e)
31: {
32: timer.Enabled = false;
33:
34: timer.Interval = double.Parse(
RoleEnvironment.GetConfigurationSettingValue("TimeInterval"));
35:
36: string blobUrl = ConfigurationManager.AppSettings["BlobUrl"];
37:
38:
39: CloudStorageAccount cloudStorageAccount =
CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
40: CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
41:
42: try
43: {
44: CloudBlockBlob blob = new CloudBlockBlob(blobUrl, cloudBlobClient);
45:
46:
47: int startIndex = blobUrl.LastIndexOf("/".ToCharArray()[0]);
48: string fileName = string.Format("{0}{1}",
RoleEnvironment.GetLocalResource("InstanceDriveCache").RootPath, blobUrl.Substring(startIndex));
49: string content = string.Empty;
50:
51: if (cache.Get("ETag") == null)
52: {
53: blob.FetchAttributes();
54: blob.DownloadToFile(fileName);
55: cache["ETag"] = blob.Properties.ETag;
56:
57: using (TextReader tr = new StreamReader(fileName))
58: {
59: content = tr.ReadToEnd();
60: }
61:
62: //cache.Insert("Content",
content,
new System.Web.Caching.CacheDependency(fileName));
63: cache.Add("Content",
content,
new System.Web.Caching.CacheDependency(fileName),
Cache.NoAbsoluteExpiration,
Cache.NoSlidingExpiration,
CacheItemPriority.High,
OnRemove);
65: }
66: else
67: {
68: blob.FetchAttributes();
69: if (!cache["ETag"].ToString().Equals(blob.Properties.ETag))
70: {
71: blob.DownloadToFile(string.Format("{0}{1}",
RoleEnvironment.GetLocalResource("InstanceDriveCache").RootPath, blobUrl.Substring(startIndex)));
72: cache["ETag"] = blob.Properties.ETag;
73: }
74: }
75: }
76: catch (StorageClientException ex)
77: {
78: //Log the error
79: }
80: timer.Enabled = true;
81: }
82:
83: private void OnRemove(string key, object value, CacheItemRemovedReason reason)
84: {
85: string blobUrl = ConfigurationManager.AppSettings["BlobUrl"];
86: int startIndex = blobUrl.LastIndexOf("/".ToCharArray()[0]);
87: string fileName = string.Format("{0}{1}",
RoleEnvironment.GetLocalResource("InstanceDriveCache").RootPath, blobUrl.Substring(startIndex));
88:
89: using (TextReader tr = new StreamReader(fileName))
90: {
91: string content = tr.ReadToEnd();
92: cache.Add("Content",
content,
new System.Web.Caching.CacheDependency(fileName),
Cache.NoAbsoluteExpiration,
Cache.NoSlidingExpiration,
CacheItemPriority.High,
OnRemove);
94: }
95:
96:
97: }
98:
99: }
100: }
This code can be leveraged as a starting point to accomplish many scenarios with your Windows Azure worker roles, for example dynamically replacing PHP configuration files to change PHP server configurations dynamically, reconfigure Tomcat Apache servers, add/remove website directories, etc. Your creativity is the limit.
Let us know via comments how you have approached such situations. I’d love to hear some more successful scenarios.
-Eric