Pre-Populate the AppFabric Cache
When I start talking to people about the caching functionality that is part of Windows Server AppFabric I am usually asked "What is the AppFabric Cache?" The MSDN page at https://msdn.microsoft.com/en-us/library/ee790954.aspx provides a great overview (below) as well as additional information. The Cache is defined as:
"Windows Server AppFabric caching features use a cluster of servers that communicate with each other to form a single, unified application cache system. As a distributed cache system, all cache operations are abstracted to single point of reference, referred to as the cache cluster. In other words, your client applications can work with a single logical unit of cache in the cluster regardless of how many computers make up the cache cluster. "
When you develop against the AppFabric Cache you utilize the cache-aside pattern. This pattern outlines the steps in which the application will first check to see if the data is in the cache. If not, then query a database (or other data source), load the cache, then return the value. The idea is that the cache will be populated over time as instances of the application call for data. The AppFabric Cache comes with a number of options around how long data should remain in the cache and provides you the flexibility to tune the performance to your needs.
One thing that the cache-aside pattern doesn't provide for is the pre-population of data in the cache. This entry will walk through how we can provide this functionality.
The first thing we need to do is to create the cache. I outline how to create a cache in .NET code in my previous entry. You can check that out but I will also include the code in the sample below. As you go about creating the cache you also need to decide if you want eviction to occur and if so how long the data will stay in the cache until it is evicted. For this example I am going to setup the cache so that eviction is turned off (Expirable is set to false). Since I am preloading the cache I want to the data to remain in the cache. If I read your mind, the next question will be how will the code handle when new data is entered into the database. Through the use of the SQL Dependency functionality we can setup an event handler to respond when an event is raised when there is a change at the database layer. Once this event is caught we can empty the cache and reload it. We could also add code to take advantage of the cache-aside pattern and if we don't the data in the cache then query the database and populate the cache on demand.
As we look at the code example below lets jump directly to the PopulateLookUpCache method. The first part of this method sets up the database connection, the SQL Dependency and loads the data into a dataset. The second part of the method focuses on the loading of the cache. This is the part that we will focus on.
Before we can use the cache we need to create an instance of the DataCacheFactory. This object requires configuration information. This can either be through entries in a config file as shown below:
<
dataCacheClient>
<hosts>
<host name="AppFabricBeta2" cachePort="22233" />
</hosts>
<localCache isEnabled="true" sync="TTLBased" objectCount="100000" ttlValue="300" />
</dataCacheClient>
or through code by creating a DataCacheFactoryConfiguration object and passing it to the DataCacheFactory as shown below:
var config = new DataCacheFactoryConfiguration();
config.Servers = new List<DataCacheServerEndpoint>
{
new DataCacheServerEndpoint(Environment.MachineName, 22233)
};
DataCacheFactory dcf = new DataCacheFactory(config);
We can now call the GetCache method which will return a cache object based on the name of the cache that is passed into the method. The Cache object has a number of methods which allow us to interact with the cache. I use the Put method as the Put will add or replace an object in the cache whereas the Add method only adds an entry. If there is already an object in the cache it will return an exception.
In the code below, I loop through the DataSet and call the Put method to populate the cache. The Put method takes two parameters. The first is the key that will be used and the second is the value. In this case, since I am using the AdventureWorks sample database, the Products table contains the product id and the product which works very nicely for this example.
Once the loop has been executed and the data is now in the cache we can double check that the cache has been populated through the use of a PowerShell cmdlet. Switch to a PowerShell command window and type in (without the quotes) "Get-CacheStatistics -CacheName <your cache name>" and hit enter. This will return a list of attributes related to the cache and will show you how many items are currently in the cache.
We now have a populated cache and a now can use the GetLookUpCacheData method to read data out of the cache for our application. Take a look at the code below to see how all of this was done as well as how to setup the SQL Dependency code to get events. As always this code is provided as is and there are a number of places that should be refactored - such as the repeating DataCacheFactory objects.
using
System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Data.SqlClient;
using System.Data;
using Microsoft.ApplicationServer.Caching;
using System.Management.Automation;
using System.Management.Automation.Runspaces;
namespace AppFabricCacheWrapper
{
public class CacheHelper : IDisposable
{
private static DataSet lookUpDataset = null;
private string connString = string.Empty;
private string cacheName = "PreLoadSampleCache";
public CacheHelper()
{
connString = "Data Source=(local);Initial Catalog=AdventureWorksLT;Integrated Security=SSPI;";
SqlDependency.Start(connString);
CreateCache(cacheName);
PopulateLookUpCache(cacheName);
}
~CacheHelper()
{
SqlDependency.Stop(connString);
Dispose(false);
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
}
public string GetLookUpCacheData(string keyValue)
{
var config = new DataCacheFactoryConfiguration();
config.Servers = new List<DataCacheServerEndpoint>
{
new DataCacheServerEndpoint(Environment.MachineName, 22233)
};
DataCacheFactory dcf = new DataCacheFactory(config);
var cache = dcf.GetCache(cacheName);
string data = cache.Get(keyValue) as string;
if (data == null)
{
//Determine if you are going to query the database
}
return data;
}
/// <summary> /// Retrieves look up data for the given key and type from database /// </summary>
private void PopulateLookUpCache(string CacheName)
{
SqlConnection conn = null;
SqlCommand comm = null;
SqlCommand commDependency = null;
SqlDataAdapter sqlAdapter = null;
//populate DataSet
try
{
//Connect to look up database and retrieve the names of the products.
conn = new SqlConnection(connString);
conn.Open();
comm = new SqlCommand();
comm.Connection = conn;
comm.CommandText = "select ProductID, Name from SalesLT.Product";
comm.CommandType = CommandType.Text;
if (lookUpDataset == null)
{
lookUpDataset = new DataSet();
}
else
{
lookUpDataset.Clear();
}
sqlAdapter = new SqlDataAdapter(comm);
sqlAdapter.Fill(lookUpDataset);
//Command object used for subscribing to notifications
commDependency = new SqlCommand();
commDependency.Connection = conn;
commDependency.CommandText = "select ProductID, Name from SalesLT.Product";
commDependency.Notification = null;
SqlDependency dependency = new SqlDependency(commDependency);
dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);
commDependency.ExecuteNonQuery();
}
catch (Exception e)
{
throw new Exception(e.Message + e.StackTrace);
}
finally
{
sqlAdapter.Dispose();
comm.Dispose();
commDependency.Dispose();
conn.Close();
conn.Dispose();
}
//populate cache
try
{
//This can also be kept in a config file
var config = new DataCacheFactoryConfiguration();
config.Servers = new List<DataCacheServerEndpoint>
{
new DataCacheServerEndpoint(Environment.MachineName, 22233)
};
DataCacheFactory dcf = new DataCacheFactory(config);
if (dcf != null)
{
var cache = dcf.GetCache(CacheName);
foreach (DataRow product in lookUpDataset.Tables[0].Rows)
{
cache.Put(product["ProductID"].ToString(), product["Name"].ToString());
}
}
}
catch (Exception e)
{
throw new Exception
}
}
}
/// <summary>
/// Event which will be fired when there are any database changes done to the dependency query set
/// </summary>
private void dependency_OnChange(object sender, SqlNotificationEventArgs e)
{
if (e.Info != SqlNotificationInfo.Invalid)
{
PopulateLookUpCache(cacheName);
}
}
private void(string CacheName)
{
//This can also be kept in a config file
var config = new DataCacheFactoryConfiguration();
config.Servers = new List<DataCacheServerEndpoint>
{
new DataCacheServerEndpoint(Environment.MachineName, 22233)
};
DataCacheFactory dcf = new DataCacheFactory(config);
if (dcf != null)
{
var state = InitialSessionState.CreateDefault();
state.ImportPSModule(new string[] { "DistributedCacheAdministration", "DistributedCacheConfiguration" });
state.ThrowOnRunspaceOpenError = true;
var rs = RunspaceFactory.CreateRunspace(state);
rs.Open();
var pipe = rs.CreatePipeline();
pipe.Commands.Add(new Command("Use-CacheCluster"));
var cmd = new Command("New-Cache");
cmd.Parameters.Add(new CommandParameter("Name", CacheName));
cmd.Parameters.Add(new CommandParameter("Expirable", false));
pipe.Commands.Add(cmd);
var output = pipe.Invoke();
}
}
}