Data Services Expressions – Part 8 – Projections
Series: This post is the eighth part of the Data Services Expressions Series which describes expressions generated by WCF Data Services.
Projections feature is visible in the URL through the $select query option. It allows the client to ask the server to only return a subset of properties on the returned entities. In this post we’ll look into how simple projections are handled by the URL parsing and how are these translated into expression executed against the provider.
Properties projected
A simple projection sample in the URL:
https://host/service.svc/Products?$select=Name
This asks the server to return all product entities, but only include the Name property in the payload. Internally the server will construct a list of properties to request from the data source. So at first the server adds all the properties listed in the $select query option to the list of properties to be projected. For the sample above we have { Name } so far.
The client still expects to get back the usual feed of entries (in case of ATOM payload), and for this to work correctly each entity must have a unique identifier in the payload. The WCF Data Services server uses the key properties to construct the identification. So the server will include all key properties in the list of properties to be projected. In our sample above the Product has just one key property called ID, so we have { Name, ID } as the list of properties to project.
Note: If ETag is defined for the projected entity, all the ETag properties will be added into the list as well. Also, if there is any EntityPropertyMappingAttribute defined for the projected entity, all the properties mapped by these attributes will be included in the list as well.
ProjectedWrapper
Once we have the complete list of properties we will need for each entity from the data source, we need to define the type of each entity to be returned. This is determined by the LINQ expression we will use to project the properties. Projections are usually expressed through the Select operator like this:
products.Select(product => new Product()
{
Name = product.Name,
ID = product.ID,
});
But this approach doesn’t work if the provider doesn’t use CLR types which reflect the shape of the entity as defined by the provider’s metadata. Some providers use generic CLR types for their entity representation (for example a Dictionary or some other custom object). Accessing properties on such entities uses method calls as described in this post. So instead WCF Data Services uses a special types to project to, the ProjectedWrapper.
There are in fact ten ProjectedWrapper types. ProjectedWrapper0 to ProjectedWrapper8 are used when 0 to 8 properties need to be projected. ProjectedWrapperMany is used when more then 8 properties are needed, in which case the ProjectedWrapperMany has a Next link to the next ProjectedWrapperMany instance forming a linked list, where each item stores up to eight properties.
Each ProjectedWrapper type has a property ResourceTypeName which stores the full type name of the entity this projected wrapper represents. This is necessary as it’s not possible to tell the type of the entity based on the few properties projected. We will discuss in detail how type hierarchies are handled in projections in some later post. There’s also another property PropertyNameList on each ProjectedWrapper. This property stores a comma delimited list of names of properties projected into this wrapper.
The projected properties are then stored in ProjectedWrapper members ProjectedProperty0 to ProjectedProperty7. The first name in the PropertyNameList specifies the name of the property projected into the ProjectedProperty0, the second specifies the name of the ProjectedProperty1 and so on.
So in our sample above, where we are to project properties Name and ID, using a ProjectedWrapper would look like this:
products.Select(product => new ProjectedWrapper2() {
ResourceTypeName = “TestNamespace.Product”,
PropertyNameList = “Name,ID”,
ProjectedProperty0 = p.Name,
ProjectedProperty1 = p.ID
});
As a result the type of the query is now IQueryable<ProjectedWrapper2> (instead of IQueryable<Product> which would be the case without projections). The WCF Data Services will recognize this and will not assume each result to be of the entity type, but instead will use the ProjectedWrapper and the information in it to write the results to the response.
Note: The reason such a generic looking type like ProjectedWrapper is used instead of something more friendly is performance. Generating new type for each shape of the projection result would be rather expensive.
Note 2: Currently the data source must return instances of the ProjectedWrapper public type if such type was requested by the query. It is not supported for the provider to change the type of query even if it decides to ignore projections and return the original entity for example. In the future the product might support some looser contract in this place, for example an interface similar to the IExpandedResult.
Projection expression
Knowing about ProjectedWrappers we can finally look at the expression generated by WCF Data Services. Let’s take our sample query from above, the expression generated would look like this:
System.Collections.Generic.List`1[DataServiceExpression.Product]
.Select(p => IIF((p == null), null, new ProjectedWrapper2() {
ResourceTypeName = "TestNamespace.Product",
PropertyNameList = "Name,ID",
ProjectedProperty0 = Convert((p As Product).Name),
ProjectedProperty1 = Convert(Convert((p As Product).ID))}))
This is pretty close to what we came up with above. The first difference is the null check (the IIF((p == null), null, …)). This is so called “null propagation” and in detail I will discuss it at some later time. In short it makes sure that if the result being projected is null, the expression would still work and would project a null and wouldn’t crash. This part of the query can be influenced by modifying the value returned from IDataServiceQueryProvider.IsNullPropagationRequired, assuming you implement a custom provider. I will describe this in more detail in some later post.
The second difference is the type cast operation “p As Product”. This is needed when there’s type inheritance involved. In the simple case where there’s just one type of entity they are pretty much redundant as the parameter p is already of type Product anyway.
The third difference is the Convert operator. There are actually two types of these. The outer ones which are present on each property regardless of its type are conversions to System.Object. This is necessary since the ProjectedProperty0 is defined as having System.Object type, and thus the value assigned to it needs to be converted to such type. Note that in most programming languages the compiler will generated this conversion for you automatically, so you don’t usually see this in your code. The second type of Convert operators, the inner one, is only needed on value types and it converts the value to a nullable version of such type. So in the case above it converts the System.Int32 type of the ID property to a Nullable<System.Int32> type. This is again necessary when there’s type inheritance involved, in this simple case the value of the property will never be null.
So to makes all of the above more clear, here’s the same expression as above rewritten to what it would look like in C#:
products.Select(p => (p == null) ? null : new ProjectedWrapper2() {
ResourceTypeName = “TestNamespace.Product”,
ProductNameList = “Name,ID”,
ProjectedProperty0 = (object)((p as Product).Name),
ProjectedProperty1 = (object)(int?)((p as Product).ID)
});
And that’s all. We covered all the basics you might need to understand simple projection queries. As noted above it gets more complicated when type inheritance is involved. Also null propagation has some effects on the expressions generated in certain cases. Yet another interesting topic is expansions and their interaction with projections. But I will get to those in some later post.
Comments
- Anonymous
November 03, 2010
The comment has been removed - Anonymous
November 08, 2010
The ProjectedWrapper is unfortunately necessary. WCF Data Services is a server solution and since the $select comes from a client it is not feasible to generate a new type (anonymous types are just normal types, but with a weird name) based on the client request. Since types are not collectible in .NET (at least the onces used here are not), it would be a very easy way for the client to DoS the server easily by simply creating all kinds of variations in the $select query option. Your one option might be to rewrite the query to use some predefined types if you know what they are (I think NHibernate requires strongly types classes, so you have the types already). You can take a look at the prototype code in OData Provider Tookit the part under Experimental/AstoriaOverAstoria. That code rewrites the projections to use the strongly typed classes instead (gets rid of the ProjectedWrapper). It's not easy, but doable. The download link for the toolkit is here: www.odata.org/.../odata-sdk