Data Services Expressions – Part 9 – Expansions
Series: This post is the ninth part of the Data Services Expressions Series which describes expressions generated by WCF Data Services.
In the last post we looked at projections, which leaves us with just one last big feature, expansions.
The $expand query option
Query which targets a certain entity by default returns just primitive and complex properties of that entity. Navigation properties are only represented as links in the response. Expansions allow clients to ask the server to return the entity or entities referred to by a specified navigation property along with the base entity of the query. For a sample let’s take a look at this query:
https://host/service.svc/Categories
Such query returns category entities. Each category will contain its ID, Name and other primitive or complex properties in the payload. But the Products navigation property will only be represented in the payload as a link. If the client needs the products for each category as well, it would have to download the feed this link points to for each of the categories. Or it can use expansions and ask the server to include the Products in the response as well, like this:
https://host/service.svc/Categories?$expand=Products
This query will return the same Categories as the previous one, but the Products navigation property will now also include all the product entities for each category in the response. This feature can be very useful for clients to minimize the number of requests issued against the server.
Expressing expansions in expressions
Now let’s take a look at how such query is executed on the server. The simplest approach would be to simply assume that the server can ask for the value of the Products property and it would get back the product entities it needs. But this approach would be inefficient and rather complicated to implement for many providers.
As an example let’s consider a provider which reads its data from a database. In the database navigation properties are usually stored as foreign keys. A query which returns categories would simply return all the rows from the table with categories. At the time the service would ask for all products for a given category the provider would have to issue a new query against the database to get such data (from different tables). And this would be repeated for each category in the results. This could quickly become very expensive, not counting that your favorite DBA would not like your application at all.
Much better solution would be to ask the provider to get categories and their products in one query so that it could issue a join query. But how to tell the provider that Products are needed as well? We could invent some new way (special method or so), but that would go against the idea that all queries are passed to the provider as LINQ queries.
The way this is resolved is by explicitly projecting the navigation property which needs to be expanded. So a simple example of the query above could look something like this:
categories.Select(category => new
{
Category = category,
Products = category.Products
});
You can try this with any existing LINQ provider and it should work as expected, that is retrieve the categories as well as all their products from the data store.
The above approach can’t be used directly by WCF Data Services. The main problem is that it requires generation of a new type for each set of expanded properties (the return type of the query). So instead a special type is used, which is called ExpandedWrapper. It’s rather similar to the ProjectedWrapper which we discussed in the previous post and it serves very similar purpose.
ExpandedWrapper is a generic type (in fact there are 12 such types) where the generic parameters specify the type of the entity being expanded and then types of each of the expanded properties. Each ExpandedWrapper then has a property ExpandedElement which will store the entity being expanded (the category in our sample) and a certain number of properties called ProjectedProperty0, ProjectedProperty1 and so on to store the expanded navigation properties. So in our sample above using ExpandedWrapper the query would look like:
categories.Select(category => new ExpandedWrapper<Category, IEnumerable<Product>> {
ExpandedElement = category,
Description = "Products",
ProjectedProperty0 = category.Products
});
I didn’t mention the property Description, which servers the same purpose as ProjectedWrapper.PropertyNameList and stores a comma delimited list of names of navigation properties expanded.
The real expression
To be able to look at the real query generated by WCF Data Services our simple reflection based service we used so far won’t do. The reflection provider (which is used if you just provide classes as the service definition) assumes the exact thing we want to avoid. That is being able to access a navigation property any time without any consequences. For in-memory data, this is a very valid assumption as all the data is readily available without a need to issue expensive queries. So the expression generated by the reflection provider for a simple expansion like the one in our sample will not try to tell the LINQ provider to get the expanded properties, since there’s no need. For a custom provider WCF Data Services can’t make such assumption, and so it will generate the expansion expression like shown above.
So instead of our simple provider we used so far we will use a sample from the OData Provider Toolkit, which you can download here: https://www.odata.org/developers/odata-sdk. From the downloaded archive extract the Typed\RWNavProp folder and open the solution in it. Add the InterceptingProvider toolkit (as mentioned in our second post) to the solution and open the DSPResourceQueryProvider.cs file from the DataServiceProvider project. In it find the method GetTypedQueryRootForResourceSet<TElement> and wrap the call returned IQueryable into the intercepting query, something like this:
private System.Linq.IQueryable GetTypedQueryRootForResourceSet<TElement>(ResourceSet resourceSet)
{
return Toolkit.InterceptingProvider.Intercept(
this.dataSource.GetResourceSetEntities(resourceSet.Name).Cast<TElement>().AsQueryable(),
(expression) =>
{
System.Diagnostics.Trace.WriteLine(expression.ToString());
return expression;
});
}
Now simply start the ODataDemoService and issue our sample query. You should see an expression like this:
System.Linq.Enumerable+<CastIterator>d__b1`1[ODataDemo.CategoryEntity].Select(p =>
new ExpandedWrapper`2() {
ExpandedElement = p,
Description = "Products",
ProjectedProperty0 = p.Products})
Which is exactly like the one we constructed above, so it works as it should. The above text doesn’t actually show the real type of the ExpandedWrapper, it just shows that it’s a generic type with two parameters (that’s what the `2 means in CLR). The real type if you look at it in the debugger is ExpandedWrapper<CategoryEntity, IEnumerable<ProductEntity>>.
Note that unlike the ProjectedWrapper expressions, there’s no need for Convert operators in this one since all the properties are strongly typed due to the fact that the ExpandedWrapper is a generic class.
The p.Products expression above is yet another standard property access expression as described in this post, and as such might differ based on the property definition you use in your provider.
During serialization WCF Data Services will now recognize the results as being ExpandedWrapper objects, and it will access the ExpandedElement property and use its value as the entity instance. Then when it needs to get the value of the navigation property to expand it will access the right ProjecteProperty property (based on the comma delimited names in the Description property).
That’s it for a simple expansion. It gets much more complicated if several features are combined together. When both expansions and projections are combined the query will use a combination of ExpandedWrapper and ProjectedWrapper objects and it will get considerably more complex. Also using server driven paging will influence the expression tree a lot. But those are all topics for some future posts.
Comments
Anonymous
July 15, 2010
Vitkaras, I want to give you a big thank you for you continues effort guiding the community to know more about WCF Data Services, you are one of rare people who have 'Continues commitment to one thing' please keep up the good workAnonymous
August 09, 2010
Hi Vitek. Thanks, for the blog articles. I still have one question though. On my class that implements IQueryable<T> it calls "IEnumerable<ExpandedWrapper<MyEntityClass,MyEntityClass>> GetEnumerable()" How do I create an IEnumerable<ExpandedWrapper... if ExpandedWrapper is internal? I don't know how I'm suppose to return the info?Anonymous
August 09, 2010
Hi, Sorry, i forgot they aren't internal, just in the internal namespace :) My badAnonymous
August 09, 2010
Hi Marius, Great you foudn it yourself. Anyway - they are internal (and hidden from intellisense) so that most users won't be confused by them. As seen in these blog posts, they are complex and teh vast majority of users don't need to know about them. Those who do need to know, will find out anyway :-). Just like you did. Thanks