Self-Tracking Entities: Original Values and Update Customization
I’ve been looking through the Entity Framework MSDN forums and one useful tip that I wanted to blog about was how to customize which original values are tracked so that you can optimize the service payload and to customize the number of things in the database UPDATE SQL. Self-tracking entities track at least some original values to support optimistic concurrency.
However, there are scenarios where you may want to customize whether your self-tracking entities to track original values for all properties, just the concurrency tokens (such as a rowversion), or not to no original values at all. Additionally, the Entity Framework has the capability to adjust the UPDATE SQL statement that is issued to the database to include all properties or only those that actually changed. In this post I’ll describe the default self-tracking entity behavior and how to customize the template to use some of these other options.
The Model
For simplicity, in this post I am using a single table called “Category” which I defined in SQL Server with an “Id” primary key, strings for “Name” and “Description”, and a timestamp column called “LastChanged” which I use as a concurrency token:
Self-Tracking Entity Defaults
Out of the box, the self-tracking entity template tracks original values for “required original values” inside the ChangeTracker.OriginalValues dictionary. There really aren’t “required original values”, but there are properties that give you more fidelity and a stronger guarantee that your update is going to do what you expected it to do. When the self-tracking entity template points at an edmx file, the following values are considered “required original values” by default:
· Primary key properties
· Foreign key properties
· Properties marked with a ConcurrencyModeof “Fixed”
· If the entity is mapped to a stored procedure for an update, then all properties are “required original values” (because the sproc allows passing in all current values and all original values)
When one of these “required original value” properties is changed, the original value is stored in the ChangeTracker.OriginalValues dictionary and the entity is marked as “Modified”. The ApplyChanges extension method that is part of the context self-tracking entity template is responsible for taking this information and setting it properly in the ObjectContext’s state manager so that the change can be saved to the database.
ApplyChanges will first see that the entity is marked as Modified and will call the ObjectStateManager.ChangeObjectState to make the modification. ChangeObjectState will mark the entity as Modified and will mark every property as being modified. The reason for this is that the self-tracking entity does not know which non-“required original values” changed, so it just sends them all to be safe. As a result, even if only the Category’s “Name” property was changed, the UPDATE SQL statement that is sent to the database looks like this with both “Name” and “Description” in the set clause:
update [dbo].[Category]
set [Name] = @0, [Description] = @1
where (([Id] = @2) and ([LastChanged] = @3))
So by default, self-tracking entities trade a reduced client-service communication size (by sending fewer original values) for a more complicated UPDATE SQL statement.
Tracking All Original Values
The first common customization request is about how to record original values for all properties. This can be done with a small customization to the Model.tt self-tracking entity template: modify the Model.tt file on line 15 to pass “false” into the constructor for OriginalValueMembers:
From:
15: OriginalValueMembers originalValueMembers = new
OriginalValueMembers(allMetadataLoaded, metadataWorkspace, ef);
To:
15: OriginalValueMembers originalValueMembers = new
OriginalValueMembers(false, metadataWorkspace, ef);
The OriginalValueMembers class determines which properties are “required original values”. By passing in “false”, you are telling the class that all properties should be “required original values”. Now when you modify the “Name” property of a Category, you will see its original value stored in the ChangeTracker.OriginalValues dictionary:
Category c;
using (var ctx = new PlaygroundEntities())
{
c = ctx.Categories.First();
c.StartTracking();
c.Name = "NewName";
Debug.Assert(ObjectState.Modified == c.ChangeTracker.State);
Debug.Assert(c.ChangeTracker.OriginalValues.ContainsKey("Name"));
}
Because all properties will now record an original value when they are changed, the size of the message between a client and service will increase over the default. However, storing all the original values gives you the opportunity to do some pretty interesting things such as to update only modified properties (see below), or possibly implement a “RejectChanges” method that reverts an entity back to its original values (let me know if you want to know more about this and I can do another post).
Update Only Modified Properties
The next change we’ll make to the self-tracking entity template is how to simplify the UPDATE SQL that is sent to the database to include only properties that were actually modified. There are two good ways to do this:
1. Modify ApplyChanges to use the fact that original values are tracked for every property to change the entity to the “Modified” state rather than using ChangeObjectState which marks all properties as modified. This is the simplest approach.
2. Modify the ObjectChangeTracker to record only the fact that a property changed, and modify ApplyChanges to mark only those properties as modified. This is more complicated than the first option, but as the additional advantage of not requiring original values for every property and so reduces the client to service message size. There's an updated Context Template and Model template attached at the bottom of this post if you want to skip the details.
The first option only requires another single line update on top of the “Tracking All Original Values” change that was made in the above section. In the Model.Context.tt file, change the ChangeEntityStateBasedOnObjectState method on line 927 that uses ChangeObjectState to simply put the entity into the Unchanged state rather than the Modified state:
From
927: context.ObjectStateManager.ChangeObjectState(entity, EntityState.Modified);
To:
927: context.ObjectStateManager.ChangeObjectState(entity, EntityState.Unchanged);
ApplyChanges initially puts all entities in the Added state. By putting the entity in the Unchanged state rather than the Modified state, this allows the transition from Unchanged to Modified to happen when original values are set later in ApplyChanges. As a result, if only the category’s “Name” is modified, this will be the only property in the set clause of the UPDATE SQL:
update [dbo].[Category]
set [Name] = @0
where (([Id] = @1) and ([LastChanged] = @2))
This approach is nice because it is fairly simple to implement (only two one-line template changes), but it does come with the overhead of sending around original values for all modified properties, which as I mentioned before causes the message size between client and server to be larger.
If you want the best of both worlds with a small message size and a small update size, you can use Option 2 above where the ObjectChangeTracker is modified to record the fact that specific properties are modified, but the original value isn’t stored.
The first step to do this is to add a collection of the changed property names to the ObjectChangeTracker class (in this example I’ll just use a collection of strings to record this, but you could further optimize this by coming up with a scheme where you used a BitArray to record changed properties). First we’ll add a public field called ChangedProperties to the ObjectChangeTracker class that is in the Model.tt file:
1436: [DataMember]
1437: public HashSet<string> ChangedProperties = new HashSet<string>();
Let’s also add a method to the ObjectChangeTracker called RecordPropertyChange to make it easy to use this field:
1572: internal void RecordPropertyChange(string propertyName)
1573: {
1574: if (_changeTrackingEnabled && _objectState != ObjectState.Added)
1575: {
1576: if (!ChangedProperties.Contains(propertyName))
1577: {
1578: ChangedProperties.Add(propertyName);
1579: }
1580: }
1590: }
Next, we’ll need to change the entity classes to use our new method to record that a scalar non-“required original value” property has changed:
85: else
86: {
87: #>
88: ChangeTracker.RecordPropertyChange("<#=edmProperty.Name#>");
89: <#
90: }
And we’ll need to do the same for complex types:
470: else
471: {
472: #>
473: ChangeTracker.RecordPropertyChange("<#=edmProperty.Name#>");
474: <#
475: }
We’ll also need to update the AcceptChanges method to clear this set of properties:
1562: public void AcceptChanges()
1563: {
1564: OnObjectStateChanging(ObjectState.Unchanged);
1565: OriginalValues.Clear();
1566: ObjectsAddedToCollectionProperties.Clear();
1567: ObjectsRemovedFromCollectionProperties.Clear();
1568: ChangeTrackingEnabled = true;
1569: _objectState = ObjectState.Unchanged;
1570: ChangedProperties.Clear();
1571: }
Now entities will record all specific property changes, even though an original value is not always stored. We can then use the ObjectChangeTracker.ChangedProperties collection in the Model.Context.tt file to update ApplyChanges to mark these specific properties as modified. The first change to ApplyChanges will be to make the change to stop using ChangeObjectState for Modified entities that is described in Option 1 in this section. Next, we’ll update the UpdateOriginalValues method in two places, first to make sure we do something when we see a ChangedProperty:
806: if (entity.ChangeTracker.State == ObjectState.Unchanged ||
807: entity.ChangeTracker.State == ObjectState.Added ||
808: (entity.ChangeTracker.OriginalValues == null &&
809: entity.ChangeTracker.ChangedProperties.Count == 0))
810: {
811: // nothing to do here
812: return;
813: }
And second, to actually use the values in ChangedProperties to mark a property as modified:
Replace:
830: if(property.TypeUsage.EdmType is PrimitiveType &&
entity.ChangeTracker.OriginalValues.TryGetValue(property.Name, out value))
831: {
832: originalValueRecord.SetValue(property, value);
833: }
With:
830: if (property.TypeUsage.EdmType is PrimitiveType)
832: {
833: if (entity.ChangeTracker.OriginalValues.TryGetValue(property.Name, out value))
834: {
835: originalValueRecord.SetValue(property, value);
836: }
837: else if(entity.ChangeTracker.ChangedProperties.Contains(property.Name))
838: {
839: entry.SetModifiedProperty(property.Name);
840: }
841: }
Now you are ready to use your template to minimize what is in the message between client and server, and to minimize what is included in your SQL UPDATE.
In Summary
Self-tracking entities can be customized to store either all original values, only the “required original values” (the default), or no original values with a few changes to the model and context templates. You can also further customize your templates to change the UPDATE SQL that is sent to the database to tailor how your application uses optimistic concurrency. If you have any questions about self-tracking entities or want to see explanations of additional patterns, just let me know.
Jeff Derstadt
Entity Framework Team
Comments
Anonymous
February 13, 2011
The comment has been removedAnonymous
February 14, 2011
I have two questions: 1). What about hierarchyID, geography and geometry data types? Any plans to support them? 2). Any plans to expose some API for intercepting the proxy creation, so we can inject our own customizations? Kind regards, Catalin PopAnonymous
February 17, 2011
@Catalin Both adding new data types to the Entity Framework (geography, geometery, etc.) and having an interception mechanism for proxy creation are work items on our roadmap. Stay tuned. JeffAnonymous
February 17, 2011
@Cheval I'll work on writing up a reject changes extension for STEs and about read-only/updatable patterns. Thanks for the topic suggestions!Anonymous
February 18, 2011
Great post! I'm agree with Cheval, an example on RejectChanges is appreciated :) I'll be waiting for that!Anonymous
February 19, 2011
Is it possible to post a 3-tier wpf application for a simple CRUD scenario using MVVM, EF with STEs?Anonymous
February 28, 2011
I second Mailman's suggestion. Also, I'd like to see a discussion of the proper way of getting the updated timestamp information (when implementing optimistic concurrency) into your client-side STE following a commit on the server. The examples/walkthroughs I've seen so far don't address the problem imposed if you implement concurrency: You can no longer use a client-side STE for additional updates after the initial submission.Anonymous
March 14, 2011
This is a good idea, but is very problematic for development. I've learned the hard way that it's best to leave the *.tt files as Visual Studio created them. What if you make these customizations to the Model.tt file, then you need to add a field to one of your database tables? I'll tell you what happens. You update your .edmx data model from the database to get the new field in the model. Then you delete your .tt files, and "Add Code Generation Item..." from the edmx. Now all your customizations are gone, and you have to go through this process again. Modifying the Model.tt is fine, unless you are working in a group environment with a large project where code is being developed in parallel with database changes (like "real world"). Then modifying Model.tt can become a painful practice. How about a better solution from the Entity Framework team: Expose a property for "OriginalValueMembers" so we can set them all to be tracked in the Model.tt Properties. Enable some sort of "Transform" functionality in Visual Studio, to make these code changes in a separate file (similar to what can be done with web.config in an ASP.NET application).Anonymous
April 11, 2011
@Jeff I think I have found a problem with this code. I'm using these modification together with the template that came along with SP1, but if I have a nullable column it won't update a value on an existing record if the original value is NULL. It works if I use a string type, but not with eg. an integer. After a bit digging I found that it is the change in ChangeEntityStateBasedOnObjectState from .Modified to .Unchanged that does this, but I can't find out why. Do you have any ideas/solutions for this? For now, I'm reverting it to modified. It's more important for me the load between the client/server than between the server/dbms. --RuneAnonymous
April 11, 2011
I found the problem after a bit debugging and it seems to be a bug in the template itself, not in this code. I have filed a bug on it on Connect. connect.microsoft.com/.../self-tracking-entities-in-entity-framework-fails-to-update-if-orignial-value-is-null --RuneAnonymous
April 16, 2011
I've got some problem with this template (using EF 4.1) Generated classes for object are corrupted, it seem that Fixup<#=np.Name#> is the problematic instruction => it give function name like this : internal void FixupTEST.Data.Model_DesignerTime_DesignerKeys(Designer value, bool forceRemove) Any correction ?Anonymous
July 20, 2011
I have a problem with EF 4.0 when I use complex properties. I have a complex property for ContactInformation with two strings (ZipCode, Street) when I change the ZipCode nothings happen to the database when I say ApplyChanges and SaveChanges. Any help would be nice! Kind Regards, DanielAnonymous
July 27, 2011
I found the problem with the ComplexTypes. When you change the t4 from context.ObjectStateManager.ChangeObjectState(entity, EntityState.Modified); to context.ObjectStateManager.ChangeObjectState(entity, EntityState.Unchanged); then the changes for the ComplexTypes are not tracked. So the Result is that all fields are updated in the database. Has anyone a solution for the complexTypes. I used the T4 from this site! Kind Regards, DanielAnonymous
June 12, 2012
When I'm using EF4.3.1, adding only one entity to the edmx file and using STE template to generate the code, to my surprise, I only changed one property, and traced the t-sql from sql profiler, the t-sql only contains set statement of the changed property, I didn't modify anything of the t4 template.Anonymous
June 20, 2012
@Allen - Are you fetching the entities in one context, serializing them and then using ApplyChanges, or are you just working with them using a single context. The STE functionality only kicks in for NTier scenarios, so if you are just working directly against the context you'll get the standard EF behavior, which is to only update the properties that actually change.Anonymous
June 27, 2014
Good job. I am now into hour 2 of the insane task of figuring out why the entity framework will no longer ' ADD' a record with giving a 'OriginalValues cannot be used for entities in the Added state'. Truly an amazing CF