POCO Proxies Part 2: Serializing POCO Proxies
The information in this post is out of date.
Visit msdn.com/data/ef for the latest information on current and past releases of EF.
This is the second of a two part blog series on using POCO proxies in the Entity Framework 4.0. The first post described the capabilities of proxies and details on how they work. Using proxies means your types aren’t exactly as your system expects. This can pose challenges for operations such as serialization where having the right type is critical. This post demonstrates the best practices for using proxies in scenarios where serialization is involved, either through WCF DataContract serialization or binary serialization.
Lazy Loading and Serialization
Lazy loading and serialization don’t mix well, and if you aren’t careful you can end up querying for your entire database just because lazy loading is enabled. Most serializers work by accessing each property on an instance of a type. Property access triggers lazy loading, so more entities get serialized. On those entities properties are accessed, and even more entities are loaded. It’s a good practice to turn lazy loading off before you serialize an entity. You can do this by setting the LazyLoadingEnabled property on the context to false:
context.ContextOptions.LazyLoadingEnabled = false;
WCF DataContract Serialization
It’s common to find a data access layer such as the Entity Framework in use as part of a WCF service. Services can query for entities, make updates to entities, run validation or other business logic. The entities that the service uses can be entity framework proxies, but there are some things to watch out for.
In the following example, a WCF service exposes an operation method that returns a Customer type:
[
ServiceContract
]
public interface
INorthwindService
{
[
OperationContract
]
Customer
GetCustomer(string id);
}
The implementation of this service method creates an ObjectContext and performs a query to retrieve a Customer by key. This Customer is then returned from the service:
public
Customer
GetCustomer(string id)
{
using (
NorthwindEntities
ctx = new
NorthwindEntities
())
{
Customer
c = ctx.Customers.SingleOrDefault(x => x.CustomerID == id);
return c;
}
}
To clients of this service, the operation is going to return a type “Customer”. The WSDL for this service knows about the type “Customer”. Clients will thus know about the type “Customer” but will not know about proxy types and will not be using any of the proxy behaviors. This is a good thing because your clients do not have a direct line of sight to the database to do things like lazy loading. So what happens when you try to execute the service operation and it returns a “CustomerProxy” instead of a “Customer”? If you guessed “you get an exception”, you were right.
The DataContractSerializer can only serialize and deserialize known types. There are plenty of ways to specify the list of known types to a serializer, but for these services we don’t want the client to have to know about proxy types (remember, no line of sight). This means proxies should not be part of the set of known types for the serializer.
Instead what we want to happen is for the proxy instance to serialize as if it was just a plain “Customer” type and not a “CustomerProxy” type. Fortunately, there is a new .NET 4.0 feature called DataContractResolver which can help. A DataContractResolver can map one type to another one during serialization; in the case of proxies, it can help map a “CustomerProxy” type to a “Customer” type. As part of the Entity Framework in .NET 4.0, we included the ProxyDataContractResolver which is an implementation of a DataContractResolver to map proxy types to their regular POCO types.
.NET 4.0 Beta 2 Note: The ProxyDataContractResolver class in .NET 4.0 Beta 2 is not fully functional. Instead, please use the implementation described below.
The ProxyDataContractResolver
A DataContractResolver has two methods that map types and names during the serialization process. During serialization, .NET types are mapped to type names that are part of the serialization payload. This is done in the “TryResolveType” method. The code below includes the implementation of the ProxyDataContractResolver. The TryResolveType method determines if the entity’s type is a proxy or not, and if it is a proxy uses the non-proxy type name in the payload:
public class
ProxyDataContractResolver
: DataContractResolver
{
private XsdDataContractExporter _exporter = new XsdDataContractExporter();
public override
Type
ResolveName(
string typeName,
string typeNamespace,
Type
declaredType,
DataContractResolver knownTypeResolver)
{
return knownTypeResolver.ResolveName(
typeName, typeNamespace, declaredType ,null);
}
public override bool TryResolveType(
Type
dataContractType,
Type
declaredType,
DataContractResolver knownTypeResolver,
out XmlDictionaryString typeName,
out XmlDictionaryString typeNamespace)
{
Type
nonProxyType = ObjectContext.GetObjectType(dataContractType);
if (nonProxyType != dataContractType)
{
// Type was a proxy type, so map the name to the non-proxy name
XmlQualifiedName qualifiedName = _exporter.GetSchemaTypeName(nonProxyType);
XmlDictionary dictionary = new XmlDictionary(2);
typeName = new XmlDictionaryString(dictionary,
qualifiedName.Name, 0);
typeNamespace = new XmlDictionaryString(dictionary,
qualifiedName.Namespace, 1);
return true;
}
else
{
// Type was not a proxy type, so do the default
return knownTypeResolver.TryResolveType(
dataContractType,
declaredType,
null,
out typeName,
out typeNamespace);
}
}
}
The ApplyProxyDataContractResolver
Once you have a ProxyDataContractResolver, the next step is to instruct the DataContractSerializer to use it in service operations. This can be done by defining an operation behavior attribute that you can use on your service methods. The attribute class below shows how to do this:
public class
ApplyProxyDataContractResolverAttribute : Attribute, IOperationBehavior
{
public void AddBindingParameters(
OperationDescription
description,
BindingParameterCollection
parameters)
{
}
public void ApplyClientBehavior(
OperationDescription
description,
ClientOperation
proxy)
{
DataContractSerializerOperationBehavior
dataContractSerializerOperationBehavior =
description.Behaviors.Find<
DataContractSerializerOperationBehavior
>();
dataContractSerializerOperationBehavior.DataContractResolver =
new
ProxyDataContractResolver
();
}
public void ApplyDispatchBehavior(
OperationDescription
description,
DispatchOperation
dispatch)
{
DataContractSerializerOperationBehavior
dataContractSerializerOperationBehavior =
description.Behaviors.Find<
DataContractSerializerOperationBehavior
>();
dataContractSerializerOperationBehavior.DataContractResolver =
new
ProxyDataContractResolver
();
}
public void Validate(
OperationDescription
description)
{
}
}
The final step to use the ProxyDataContractResolver is to modify your service methods to include this attribute:
[
ServiceContract
]
public interface
INorthwindService
{
[
OperationContract] [ApplyProxyDataContractResolver
]
Customer
GetCustomer(string id);
}
Serializing back to proxies
Another frequent question is how to send POCO instances on the client tier back to a service tier and have them show up as proxies. This problem is more difficult and unfortunately, doesn’t have a good solution even in .NET 4.0. When the Entity Framework creates proxies, two things happen:
1. A proxy instance is created
2. All of the collection navigation properties are set with the appropriate EntityCollection<T>
The ProxyDataContractResolver doesn’t help in this case because the method ‘ResolveName’ used in deserialization only can return a type, not an instance of a proxy. This means that the resolver can only help with step 1. One possible option is to write your own custom DataContract surrogate. This does give you the option to instantiate your own proxy types using context.CreateObject<T>(), but it also requires you to implement a fair bit of your own logic to do circular reference detection and identity resolution. If you really need a proxy instance for a non-proxy POCO entity that was sent from a client to a service, the simplest solution is to copy the values sent from the client into proxy instances you create inside your service method.
Binary Serialization
Binary serialization works by serializing the memory an instance or graph of instances is using. It does not call property getters/setters like DataContract serialization does (if DataMembers are on properties). This has both advantages and disadvantages, but for proxies the main advantage is that proxy instances are not recreated by setting properties, the memory is just rehydrated. This allows proxies to be serialized and deserialized as proxy types. With binary serialization the proxy types need to be available. To do this, you can use the CreateProxyTypes method to ensure that all proxy types are available for a given MetadataWorksapce (it checks the proxy type cache, and create the proxy types that aren’t yet there):
context.CreateProxyTypes(new
Type
[] { typeof(
Customer
), typeof(
Order
) });
The one qualification with deserialization is that the instances come back as proxy types, but will not have any of the proxy behaviors.
Here is an example of using the techniques above to write and read an entity to a file using binary serialization:
class
Program
{
static void Main(string[] args)
{
if (args[0] == "Write")
{
using (
NorthwindEntities
ctx = new
NorthwindEntities
())
{
Customer
c = ctx.Customers.SingleOrDefault(x => x.CustomerID == "ALFKI");
WriteToFile(c, "Customer.txt");
}
}
else if (args[0] == "Read")
{
using (
NorthwindEntities
ctx = new
NorthwindEntities
())
{
ctx.CreateProxyTypes(new
Type
[] { typeof(
Customer
) });
Customer
c = ReadFromFile<
Customer
>("Customer.txt");
// Once attached, the customer can be used again
ctx.Customers.Attach(c);
c.Name = “New Name”;
ctx.SaveChanges();
}
}
}
internal static void WriteToFile<T>(T obj, string filePath)
{
using (
FileStream
file = new
FileStream
(filePath,
FileMode
.Create))
{
(new
BinaryFormatter
()).Serialize(file, obj);
}
}
internal static T ReadFromFile<T>(string filePath)
{
using (
FileStream
file = new
FileStream
(filePath,
FileMode
.Open))
{
return (T)(new
BinaryFormatter
()).Deserialize(file);
}
}
}
In Summary
Proxies can be a great way to get additional functionality in your entities simply. You have to be willing to make concessions about the accessibility of your class and properties, as well as be willing to work with ICollection<T> if you are using change tracking proxies. Serialization can still be done with proxies, but it is a bit more work. Members of the EF team are thinking about how to improve the proxy experience even more by making it more flexible and customizable, as well as having better integration with other parts of .NET. Let us know your thoughts on what kinds of capabilities you’d like to see in extensible proxies for the Entity Framework.
Jeff Derstadt & Diego Vega
Entity Framework Team
Comments
Anonymous
January 05, 2010
What about keeping a gradually unwound json representation of the entities graph in the proxy, where that graph would match the real POCO and then some contract serializer that serializes back to the POCO could work by convention. I would prefer less moving parts generally speaking, I find this a little too verbose. The second part of the problem, having POCOs come back as Proxies is for me a non issue largely due to the way I would work. Don't send Entities send messages. Most task based systems are really only concerned with subsets of data that would find their way into what woud become 'eventual' entities server (or more accurately service) side.Anonymous
January 05, 2010
Would ApplyProxyDataContractResolverAttribute be part of RTM or is it a separate class we are writing.Anonymous
January 05, 2010
@Zeeshan: ApplyProxyDataContractResolverAttribute won't be part of the product in 4.0. You need to add it to your code.Anonymous
January 06, 2010
Offtopic Are you going to do something with the efficiency of LINQ provider which emmits SQL commands? I don't know if you are aware of the discussion here: http://ayende.com/Blog/archive/2010/01/05/nhibernate-vs.-entity-framework-4.0.aspx Where according to article, EF will load entire collection to memory when doing paging, whereas Linq To SQL or Nhibernate do not have this (bad) behaviour. Ayende Rahien (aka Oren Eini) wrote there: <i> The following code: using (var db = new Entities()) { db.Blogs.First().Posts.Skip(10).Take(5).ToList(); } Will generate the following SQL: -- statement #1 SELECT TOP ( 1 ) [c].[Id] AS [Id], [c].[Title] AS [Title], [c].[Subtitle] AS [Subtitle], [c].[AllowsComments] AS [AllowsComments], [c].[CreatedAt] AS [CreatedAt] FROM [dbo].[Blogs] AS [c] -- statement #2 SELECT [Extent1].[Id] AS [Id], [Extent1].[Title] AS [Title], [Extent1].[Text] AS [Text], [Extent1].[PostedAt] AS [PostedAt], [Extent1].[BlogId] AS [BlogId], [Extent1].[UserId] AS [UserId] FROM [dbo].[Posts] AS [Extent1] WHERE [Extent1].[BlogId] = 1 /* @EntityKeyValue1 */ eof I am surprised that years of working on EF still have such a rough edges, hope this will be addressed in final version.Anonymous
January 06, 2010
Well, when I have used proxys I make a SILwith DTOs. Cleaner, without risks, but with more code.Anonymous
January 11, 2010
@EF newbie db.Blogs.First().Posts is actually an IEnumerable<T> (as all EntityCollection<T> are), not an IQueryable<T> so this is why a query for the entire collection is performed. This isn't ideal in all situations, particularly where you want to do paging or filtering around large related collections. There is a way in EF to expose an IQueryable<T> form of a collection (or reference) and that is to do something like this: db.Blogs.First().Posts.CreateSourceQuery() Thanks, JeffAnonymous
January 12, 2010
Many thanks Jeff for the answer. So you mean that when I define (by following your examples) all my collections (1:n relations) on entities as EntityCollection<T>, they are no longer queryable? That means that working with that entity I am loading whole graph all the time? Or maybe I can get lazy load, but I can not alter generated SQL anymore, as it has already been evaluated to memory (IEnumerable). Or I am missing something... Well it works for L2S, care to elaborate> What is behind this decision and so on... I'll look on CreateSourceQuery(), thanks for pointing that out! Wishing you all the best to New Year 2010!Anonymous
January 13, 2010
Can this be applied to a WCF Data Service somehow? I'm having trouble with EF proxy classes not being serialized by my Data Service and this looks like the answer. The problem is, I'm not sure how to apply this behavior. I can't apply the attribute to System.Data.Services.IRequestHandler.ProcessRequestForMessage directly and I'm not sure where to do it programmatically in a WCF Data Service. Any ideas would be greatly appreciated!Anonymous
January 17, 2010
@EF Newbie Correct, an EntityCollection<T> is no longer queryable. Any LINQ operators you do on it will be doing LINQ to Objects over the data already in the collection. However, this is different than bringing down an entire graph. LINQ to SQL has a "DataLoadOptions" capability that allows you to bring down a customer and all of its orders in one query. The Entity Framework also supports this through the Include() method on the ObjectQuery<T> class. you can do something like this: ctx.Customers.Include("Orders").Where(...) This will bring down the whole graph. We are looking to add capabilities to this in the next release as well, including making EntityCollection<T> more queryable. JeffAnonymous
January 17, 2010
@Joe I'll follow up with members of the WCF Data Services team and post a response shortly. Thanks for the great question. JeffAnonymous
February 16, 2010
Hi All, Anyone tried to serialize a proxied poco in a asp.net web app? for instance serialize a user to the session state manager?Anonymous
February 16, 2010
Hi All, Anyone tried to serialize a proxied poco in a asp.net web app? for instance serialize a user to the session state manager? (sorry for the duplicated message)Anonymous
March 02, 2010
Hi, This solution work for me but there is a problem serializing related POCO. Any Idea? Thank's for help, ZiedAnonymous
March 18, 2010
I tried the implementation of the ApplyProxyDataContractResolverAttribute as described here, and also tried using the attribute code with the default ProxyDataContractResolver shipped in the RC. Neither seemed to work. So long as the entity has had a navigation property initialized I can't seem to get it to serialize via WCF. The problem seems to come when an EntityCollection is initialized, and unfortunately it doesn't get set to null even if empty and detached from the context. In the meantime I basically create a new Entity in the WCF layer with only the scalar properties and I can serialize that no problem, kind of a pain though!Anonymous
March 19, 2010
This worked for me: http://msdn.microsoft.com/en-us/library/ee705457(VS.100).aspxAnonymous
May 25, 2010
I have tried this implementation and I am running into a NetDispatcherFaultException on deserialization in my client. Anyone seen this before?Anonymous
May 26, 2010
The comment has been removedAnonymous
August 14, 2010
I'm also having the same problem with circular references when trying to return related entities. I've tried all solutions and walkthroughs mentioned already. I have not yet been able to find a solution and will resort to creating some kind of work around. Has anyone actually managed to serialize poco instances (proxy or not) that have relationships to other entities?Anonymous
January 14, 2011
@Shaun ... You can follow the MSDN article on serializing POCO proxies by applying the Attribute solution. This seemed to work for me, but only for XML. I'd like to use JSON for REST services, and I've yet to find a working solution. Since JSON doesn't support (DataContract.IsReference or cyclic references) WCF throws an error when attempting serialization. This is a problem for POCO proxies and straight POCO objects. Anyone have a solution for using JSON with EF4 POCO objects?Anonymous
May 12, 2011
if we change Customer GetCustomer(string id) to IEnumerable<Customer> GetCustomer(string id), will this cause a problem? The reason why I asked this question is that I got an error "The underlying connection was closed: A connection that was expected to be kept alive was closed by the server." on the client end. thanks.