Self-Tracking Entities: ApplyChanges and duplicate entities

Some customers using the Self-Tracking Entities template we included in Visual Studio 2010 have ran into scenarios in which they call ApplyChanges passing a graph of entities that they put together in the client tier of their app, and then they get an exception with the following message:

AcceptChanges cannot continue because the object’s key values conflict with another object in the ObjectStateManager.

This seems to be the most common unexpected issue our customers run against when using Self-Tracking Entities. I have responded multiple times to this on email and I have meant to blog about it for some time. Somehow I was able to do it today :)

We believe that most people finding this exception are either calling ApplyChanges to the same context with multiple unrelated graphs or they are merging graphs obtained in multiple requests so that they now have duplicate entities in the graph they pass to ApplyChanges.

By duplicate entities, what I mean is that they have more than one instance of the same entity, in other words, two or more objects with the same entity key values.

The current version of Self-Tracking Entities was specifically designed to not handle duplications. In fact, when we were designing this, our understanding was that a situation like this was most likely the result of a programming error and therefore it was more helpful to our customers to throw an exception.

The problem with the exception is that avoiding introducing duplicate entities can be hard. As an example, let’s say that you have service that exposes three service operations for a car catalog:

• GetModelsWithMakes: returns a list of car Models with their respective associated Makes
• GetMakes: returns the full list of car Makes
• UpdateModels: takes a list of car Models and uses ApplyChanges and SaveChanges to save changes in the database

And the typical operation of the application goes likes this:

1. Your client application invokes GetModelsWithMakes and uses it to populate a grid in the UI.
2. Then, the app invokes GetMakes and uses the results to populate items in a drop down field in the grid.
3. When a Make “A” is selected for a car Model, there is some piece of code that assigns the instance of Make “A” to the Model.Make navigation property.
4. When changes are saved, the UpdateModels operation is called on the server with the graph resulting from the steps above.

This is going to be a problem if there was another Model in the list that was already associated with the same Make “A”: since you brought some Makes with the graph of Models and some Makes from a separate call, you now have two completely different instances of “A” in the graph. The call to ApplyChanges will fail on the server with the exception describing a key conflict.

There are changes we have considered doing in the future to the code in ApplyChanges in order to avoid the exception but in the general case there might be inconsistencies between the states of the two Make “A” instances, and they can be associated with a different Models, making it very difficult for ApplyChanges to decide how to proceed.

In general, the best way to handle duplicates in the object graph seems to be to avoid introducing them in the first place!

Here are a few patterns that you can use to avoid them:

1. Only use Foreign Key values to manipulate associations:

You can use foreign key properties to set associations between objects without really connecting the two graphs. Every time you would do something like this:

model.Make = make;

… replace it with this:

model.MakeId = make.Id;

This is the simplest solution I can think of and should work well unless you have many-to-many associations or other “independent associations” in your graph, which don’t expose foreign key properties in the entities.

2. Use a “graph container” object and have a single “Get” service operation for each “Update” operation:

If we combine the operations used to obtain car Models and Makes into a single service operation, we can use Entity Framework to perform “identity resolution” on the entities obtained, so that we get a single instance for each make and model from the beginning.

This is a simplified version of “GetCarsCatalog” that brings together the data of both Models and Makes.

// type shared between client and server

public class CarsCatalog
{
public Model[] Models {get; set;}
public Make[] Makes {get; set;}
}

// server side code

public CarsCatalog GetCarsCatalog()
{
using (var db = new AutoEntities())
{
return new CarsCatalog
{
Models = context.Models.ToArray(),
Makes = context.Makes.ToArray()
};
}
}

// client side code

var catalog = service.GetCarsCatalog();
var model = catalog.Models.First();
var make = catalog.Makes.First();
model.Make = make;

This approach should work well even if you have associations without FKs. If you have many-to-many associations, it will be necessary to use the Include method in some queries, so that the data about the association itself is loaded from the database.

3. Perform identity resolution on the client:

If the simple solutions above don’t work for you, you can still make sure you don’t add duplicate objects in your graph while on the client.The basic idea for this approach is that every time you are going to assign a Make to a Model, you pass the Make through a process that will help you find whether there is already another instance that represents the same Make in the graph of the Model, so that you can avoid the duplication.

This is a really complicated way of doing it compared with the two solutions above, but using Jeff’s graph iterator template, it doesn’t really take a lot of extra code to do it:

// returns an instance from the graph with the same key or the original entity
public static class Extensions
{
public static TEntity MergeWith<TEntity, TGraph>(this TEntity entity, TGraph graph,
Func<TEntity, TEntity, bool> keyComparer)
where TEntity : class, IObjectWithChangeTracker
where TGraph: class, IObjectWithChangeTracker
{
return AutoEntitiesIterator.Create(graph).OfType<TEntity>()
            .SingleOrDefault(e => keyComparer(entity,e)) ?? entity;
}
}

// usage
model.Make = make.MergeWith(model, (j1, j2) => j1.Id == j2.Id);

Notice that the last argument of the MergeWith method is a delegate that is used to compare key values on instances of the TEntity type. When using EF, you can normally take for granted that EF will know what properties are the keys and that identity resolution will just happen automatically, but since on the client-side you only have a graph of Self-Tracking Entities, you need to provide this additional information.

Summary

Some customers using Self-Tracking Entities are running into exceptions in cases in which duplicate entities are introduced, typically when they have entities retrieved in multiple service operations merged into a single graph. ApplyChanges wasn’t designed to handle duplicates and the best practice is to avoid introducing the duplicates in the first place. I have showed a few patterns that can help with that.

Personally, I believe the best compromise between simplicity and flexibility is provided by a combination of the first and second patterns. For instance, you can use only foreign keys properties to associate entities with reference/read-only data (e.g. associate an OrderLine with a Product in an application used to process Orders), and use graph containers to transfer data that can be modified in the same transaction, i.e. entities that belong in the same aggregate (e.g, Order and its associated OrderLines).

Hope this helps,
Diego

Comments

• Anonymous
  October 09, 2010
  Hi Diego,  I´d like to use DDD aproach on ASP.NET WebForms App,  so I have  my Poco Classes  in separate assembli  of  the .edmx  Model,  I´m serializing the My POCO classes  in  ViewState  and it  works fine  to Insert  and Retrive  data  from my Repository,  also  the  LazyLoading  with  the  Virtual  in  my List<T> properties  works fine.  But,  to persist  in Data Base  the updated  object graph   is  another  long story. I´ve been looking  for one easy  answer  to that  but  I did´t  find  yet,  belive me I´ve been reading  a  lot  about EF.  This Aproach  with  STE  looks to be  more apropriate  to  use  with WCF  stuff  I´m  not  using services  on my App.  I was  trying   to  use  the  STE  generator  but  my first problem  was  It´s  generated  in the same assemblie  of  .edmx Model   and  if  I move  that to my Domain assemblie  to try to  use partial Classes in My  domain  I  lost  the  Interfaces  IChangeble......  ITracklebel....    Can you  give me any  tip  just  to  Save  back  my  object Graph  with  the modifications,  the  Funny part  is  The  modifications  are  there  in  the object  But  the  modifications  (add  or delete ) of my lists  are  ignored  by the  EF4. Thanks, Edmilson

• Anonymous
  November 05, 2010
  Thank's Diego for writing this. When is the new version of STE, which solves this issue, going to be available??

• Anonymous
  November 12, 2010
  Hi Diego, I am running into the problem described in the post and was hoping to come up with a pure server side solution because it is very difficult to control the all the use cases on the client.   The solution I had at first was to override Equals and GetHashCode to compare on entity keys, and while it worked great when one entity was persisted at any given time, the solution broke down when ApplyChanges was called on multiple entities sequentially. So, is there a server side solution that you would recommend?   And is there a fix on the horizon that we can expect? Thanks, Alex. abesidski@hotmail.com

• Anonymous
  November 30, 2010
  Very nice article. This made things a lot clearer. But I have still issues with many-to-many relations. Option 1 will work in most scenarios, but as you said, not in many-to-many relations. Option 2 will work, but will work, but then we need to fetch a lot of data we do not need, to add just one object. Option 3, we would rather not use :-) Is it any other options when working with many-to-many relations? Regards Magnus Rekkedal

• Anonymous
  December 08, 2010
  Hello, I'm facing this problem and never received any response to my numerous posts on the web. I do not understand why having 2 entities with the same EntityKey is a problem if those entities are in an unchanged state. STE is unusable in a real world app. I've a lot of many-to-many tables and thus, cannot bind to ID as stated in your first point. The second one is a performance-killer, i do not want to download a complete list of Make with each Model, just to be able to select one. Concerning the third point, it's not an example for a many-to-many table. Waht about a merge for TrackableCollection<T>? I'm completely stuck on a project using Silverlight / WCF and EF 4 STE. Can you please tell me how to proceed to save the entity back to the database? Thanks in advance David

• Anonymous
  December 22, 2010
  The comment has been removed

• Anonymous
  February 02, 2011
  Thanks, this article provides some good insights. We use kind of the same technique as in work-around 3. Except we do it on the server-side, when a graph arrives at the server, right before calling, AcceptChanges. At this point, we search for duplicates and throw duplicates out. This only works when the duplicates don't have changes them selves. But this works out fine in our scenarios, where like in this example, the duplicates usually come from a selection to link them to a Parent entity. This took us quite some time to figure out though. Some guidelines on what (and especially what not) is possibly with STE's would be nice. Without such documentation, we keep running into "unexpected" issues like this, which take a lot of time to figure out. Still struggling with other issues, like re-linking detached entities. (see social.msdn.microsoft.com/.../4fadd41f-3157-43cb-b5e1-7def59aacdb5) We feel like STE's were/are not mature enough to go live with. Thanks for your time, Koen

• Anonymous
  February 07, 2011
  Hi, I have found an issue with the Iterator solution to this problem.  In almost all cases it works but in this one case its still an issue and the iterator solution is the only solution we can use. Lets say I have two Person properties and a find person screen. I find a person and set it to person 1 using the merge.  I find a person and set it to person 2 using the merge and save.  This works correctly. I go back into it and i find the same person set on person 2 and set it to person 1, I find the same person that was previously in person 1 and set it to person 2. What this causes is two copies of the person that was originally in person 1.  The reason for this is when i set person 1 to person 2 it properly finds the merge and uses it, but then what was in person 1 originally is now in OriginalValues.  When i set person 2 to what was originally in person 1 the iterator does not find it because it exists in OriginalValues as an old value. When apply changes is called it still hits this issue because two copies exist of the same object, one directly in a navigation property and another in original values. Is there any way to fix this?  This issue overall is causing some major headaches.  I dont see why it is so hard to have the concept of finding a record and bringing it into an object graph to save the key.  We have to have the actual objects for modification purposes and display purposes.  there are other ways we can solve this in the UI layer but we really want to solve this in the model layer. Thanks

• Anonymous
  April 19, 2011
  This is a great fault by Microsoft in something that must be an easy task. Java allows works without problem, doing that i'm trying to do with STE... without the need to do any workaround.  

• Anonymous
  January 21, 2012
  I ran into this problem a lot and found that it could also be resolved by overriding the Equals method of each entity class to compare the key values. Reading the above, are you saying that this should not have worked?

• Anonymous
  February 21, 2012
  Hi @Martin, Could you show how you override the Equals method? Thank you.

• Anonymous
  April 26, 2012
  Just an additional note on only using the foreign key ID: I needed to be able to attach the same Product to PurchaseDetailLines and was getting the error, but my grid view uses the Product.Name property. Since everthing related to products was read-only I was able to do a foreach loop through the detail lines and set the Product entity equal to null. The foreign key kept and I was still able to use the referenced objects before the save.

• Anonymous
  November 27, 2013
  Hi guys This is very interesting... However, I'm getting that exception too but in a different scenario I guess.. I have already a recond In DB/DataContext. I deleted the record from DB and detach it from EntityContext. Then a second user comes, executes same logic, detects that the record no longer exist but needs it, so it is created again.. All this happens in two separated threads and both start at the same time (A user clicking OK to same Form for example). When I accept all changes for the record that is being created I get that Exception. Seems like the one I already deleted somehow still remains in memory/cache.. so, when the second thread tries to created again that record and acept the changes it conflicts with the other one.