WCF Data Service; Server Driven Paging (SDP); Silverlight
Hi,
Today is January 5th 2010 and I am writing this post to explain the mechanism of the Server Driven Paging feature of WCF Data Service and also a code guide on how to use this from Silverlight 4.0.
The Pattern:
Server Driven paging is one of the new features that have been added in WCF Data Service Version 1.5 CTP2 (formerly called as ADO.NET Data Service). The feature in simple terms means transmitting data in pages. This feature is important from performance point of view and also an important pattern used in applications where UI layer and data access layer are separated by cloud. I view this as a pattern used many places; the most common of being “chunking” approach used in large data transfer. Take example for WCF data transfer of a large file. We typically implement and recommend to use “chunking” pattern to transmit large data from one place to another because of many reasons. One reason could be to support “resume” functionality. So if data transfer is interrupted at 50%, we don’t need to start all over again next time. We can resume from 50%. Another reason could be to allow large data transfers where network policies put an upper bound on data transfers. One more possibility I can think of is unreliable network (which internet is) where packet loss is very common and would otherwise lead to long time to get a perfect data flow in one go. Server driven paging is the same pattern/algorithm applied to transfer large set of records between WCF Data Service and client so that it doesn’t put pressure on client (e.g, to render millions of records with small scroller in a single data grid, or storing large records in client memory after he waited for good 10 minutes or so). In short, paging makes the overall process faster on average. Server driven paging gives more performance since we save time in transmitting very large number of records over the wire and improve the end user experience.
The Mechanism:
The mechanism is simple and worth understanding to make decision about whether to use this feature or not:
Step 1: Client sends request to fetch data by firing query.
Step 2: Server Fetches result from Data Base for a given page. This step needs a little elaboration since here comes the important factor that might decide how effective SDP of WCF Data service is for your scenario:
When a user fires a query from client, the data service fetches the result based on page size and the page on which currently the user is. If we want to implement paging in our application from scratch, we usually follow one of the two approaches to achieve this:
- Session based approach: We store the page on which a user currently is in the session. So we always know which page the user wants when he clicks next/previous. Pros: Sometimes in order to improve speed we also cache result set in session and simply return the required page and in fact for such cases, this is the only approach we use. We cannot afford to go to DB every time the request comes.
Another benefit of this approach is that the client code is very simple, we simply have to tell whether we need next or previous page. On server side, we fetch records from DB that should be present on a particular page.
Cons: Drawback is that the moment session expires, the page tracking is lost and it also requires us to scale server. Not a preferred approach for most of business scenarios that I have encountered.
- No session approach: Here we establish some unique key that can be used to calculate which records should be present on a particular page. An example would be a combination of “page number” and “page size” would give the records that should be present on a given page. Every time a new page is requested, we go to database to fetch the records for that page and return them. This approach is used where scalability is more important than speed. We don’t put pressure of storing calculated results in memory at server.
WCF Data Service uses “no session” approach and goes to DB every time a new page is fetched. It uses primary key as the counter to reach to a particular page. So when a next page request comes, it contains the primary key of the last record of the last page. WCF Data Service goes to Database, orders the query by primary key, fetches records of page size starting after the provided argument by client and returns result. The benefit of this approach is that you don’t depend on session for paging. This approach is also ideal for clients (especially thick clients like Silverlight) that might not talk to server for long time. Another benefit is that we don’t need to scale the server by not storing large set of data in session. The drawback of this approach is that it is not performance effective since it goes to DB for each request. Not only that, it uses orderby in query fired to DB which is even slower than an equivalent query not using “orderby”.
Step 3: WCF Data Service returns the data for the requested page. Along with the Data it also returns a URL that contains the URL to fetch next page.
Sample Code with Silverlight:
The following Sample Code has been developed using Visual Studio 2010 and Silverlight 4, but it will work fine with SL 3.0 as well since Data Service is all based on REST. Just that right client APIs need to be used and correct version of Data Service is needed at server side to enable Server Driven Paging. Contact me if you need a working sample code and if anything below did not worked.
- Create a Silverlight 4.0 Application
- Add Database to Server. Add Employee Table with FirstName, LastName and eMail (Primary Key). This is the table that we are going to page via server upon client request. Fill it with some data… maybe 5 records.
- Add DataLayer at server side using Entity Framework.
- Add ADO.NET Data Service at server side. Attach it to DataLayer, set its Access Rule, turn on ServerDriven Paging with Page size and Protocol version to be used by adding following statements in InitializeService() method of ADO.NET Data Service:
config.SetEntitySetAccessRule("*", EntitySetRights.All);
//enable server driven paging with PageSize
config.SetEntitySetPageSize("Employees", 2);
//This is required for server driven paging to work and maintain backword compatibility since SDP doesn't work with version 1 of data service
config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
- Add DataGrid and next button on Silverlight client. Remember to autogenerate columns to true for datagrid.
- Fix port address for website and add service reference to WCF data service in SL client.
- On Silverlight client code behind, add proxy to data service and client type that will hold link to next URI on every request fired to data service. The code is similar to below:
//add proxy to wcf data service where we can fire queries
public DatabaseEntities myDB = new DatabaseEntities(new
Uri (@"https://localhost:53951/WebDataService1.svc/", UriKind.RelativeOrAbsolute));
//add the client type that will hold the link to next uri (next page of records) after every request
public DataServiceQueryContinuation<Employee> token = null;
- On PageLoad, execute query to get all records from server. Code is same as any WCF Data Service code and shown below:
public MainPage()
{
InitializeComponent();
//execute query to get all records from table
var q = from e in myDB.Employees
select e;
var eq = q as DataServiceQuery<Employee>;
eq.BeginExecute(new AsyncCallback(MyCallback), eq);
}
- On Callback, write code to collect response for two different conditions. The entire code behind is shown below. I have not included XAML since its trivial having a blank Datagrid with autogenerate columns to true and a button for next page:
public partial class MainPage : UserControl
{
//add proxy to wcf data service where we can fire queries
public DatabaseEntities myDB = new DatabaseEntities(new
Uri(@"https://localhost:53951/WebDataService1.svc/", UriKind.RelativeOrAbsolute));
//add the client type that will hold the link to next uri (next page of records) after every request
public DataServiceQueryContinuation<Employee> token = null;
public MainPage()
{
InitializeComponent();
//execute query to get all records from table
var q = from e in myDB.Employees
select e;
var eq = q as DataServiceQuery<Employee>;
//this is called once and the caller is DataServiceQuery<Employee>;
//so in callback it has to be collected in the same datatype which is what is written
//in when next querytoken is null (since we are still to collect it).
eq.BeginExecute(new AsyncCallback(MyCallback), eq);
}
public void MyCallback(IAsyncResult result)
{
//slot to hold returned result
IEnumerable<Employee> response = null;
if (token != null) //this is executed everytime we have clicked on next button
{
//since in next button click we used DatabaseEntities to fire query , we need to typecast it to the same type
var eq = result.AsyncState as DatabaseEntities;
//collect the response
response = eq.EndExecute<Employee>(result);
}
else//this "else" is executed only for the first call to WCF data service; we get token only after first call
{
//since the first call was made by DataServiceQuery<Employee>, we are typecasting it to the same type on first callback
var eq = result.AsyncState as DataServiceQuery<Employee>;
//collect the response
response = eq.EndExecute(result);
}
//bind the data to datagrid (but do it before typecasting response to get token
//or collect data first in separate instance and then typecast the returned result to get next token)
dataGrid1.ItemsSource = response.ToList();
//collect next page uri by typecasting the returned response
var qor = response as QueryOperationResponse<Employee>;
token = qor.GetContinuation();
}
private void button1_Click(object sender, RoutedEventArgs e)
{
//if there are more pages
if (token != null)
//fire the next page query to proxy of WCF Data Service
myDB.BeginExecute(token, new AsyncCallback(MyCallback), myDB);
}
}
Comments/error always welcome. Contact me if you need a working sample code and if anything below did not worked.