Demo VIII: Batching Changes in Sync Services
I finally got to sit down and prepare my next demo application in the OfflineApplication series. This one shows you some neat feature that we added to the runtime a little bit after Beta 2 release. Your feedback was the driving force for including batching support to the product. The original plan was to punt it until the next release. All in all, it worked out very well and I am glad, thanks to you, that we did include batching in V1.
Sync Services is all about empowering the developer to control the sync operation. The same principal applies to batching support as you will see in the demo. The runtime gives three new session parameters that you can put to work when implementing batching logic. These parameters are:
- The number of batches you think you need to finish downloading changes
- Maximum anchor value that the runtime can store for you [optional]
- Batch size in terms of number of rows [optional]
To clarify the idea, let’s consider the following example: Your client has synchronized to an anchor of 100. The server side current anchor is 10,000. Now, say that you decided to batch the changes and you estimated that you will need 10 batches to complete the job. That said, you need to set the batch count parameter to 10, and the maximum anchor to 10,000. Recall that the sync runtime needs a sync_new_received_anchor value to enumerate changes for each batch. To do that you would set it for the first batch to 1000, then 2000 in the next batch ….etc until you hit the maximum anchor value. You got the idea.
Well, that was simple batching which seems to divide the anchor range and is purely math. But that is just one sample implementation. I bet you are thinking of a scenario where the same row was updated 10,000 times. This approach will make ten round trips to finally get the row at the last batch. That not good!? Of course not, but we invested very little in the previous example in building batching to ask for more than that. So, let’s invest some more and change the logic to count the number of changed rows before it sets the number of batches as well as the new anchor for each batch. The idea is simple and requires a UNION operation for all the timestamp columns in tables participating in sync. A little bit more of code as shown on this MSDN ‘How to’ document.
As you can see, you have the control to decide what goes when. I am sure your scenario might have some QoS requirements or load balancing needs that makes you come with more creative ideas to implement your batching\throttling logic.
Update: Just to let you know, I left Microsoft to start a new company, Raveable Hotel Reviews . See Examples: Romantic Hotels in Myrtle Beach , Top 10 Hotels in Seattle ,Kid Friendly Hotels in Miami, Best Hotels in San Francisco , Hotels with in-room jacuzzi and Best Hotels in Chicago. Your feedback is welcome on twitter.com/raveable,raveable blog.
I am not actively blogging about Sync Technologies. Please see Sync Team Blog for more updated content.
Comments
Anonymous
December 06, 2007
PingBack from http://msdnrss.thecoderblogs.com/2007/12/06/demo-viii-batching-changes-in-sync-services/Anonymous
May 29, 2011
Dear All, SYNC IS VERY VERY SLOW (With batching) using bidirectional by terminal I have implemented batching from syncguru website, now it is working fine, but it takes a long time to complete the process. "Demo VIII: Offline Application - Batching" the link is www.syncguru.com/.../SyncServicesDemoBatching.aspx Total records in my table : 60,000 records BatchSize : 3000 Total Duration : For updating 1000 records, it takes more than 90 seconds i need to reduce the "Total Duration" upto 30 seconds atleast. Please provide your valuable suggestions. Thanks in advance. regards yesu yesudassmca@gmail.com -- -- *********************************************** -- NewAnchor proc with batching support -- *********************************************** -- create procedure [dbo].[sp_new_batch_anchor] ( @sync_last_received_anchor bigint , @sync_batch_size int, @sync_max_received_anchor bigint output, @sync_new_received_anchor bigint output, @sync_batch_count int output) as if @sync_batch_size <= 0 or @sync_batch_size is null set @sync_batch_size = 3000 if @sync_max_received_anchor is null set @sync_max_received_anchor = min_active_rowversion()-1 -- @@DBTS --min_active_rowversion()-1 -- simplest form of batching if @sync_last_received_anchor is null or @sync_last_received_anchor = 0 begin set @sync_new_received_anchor = @sync_batch_size if @sync_batch_count <= 0 or @sync_batch_count is null set @sync_batch_count = (@sync_max_received_anchor / @sync_batch_size) + 1 end else begin set @sync_new_received_anchor = @sync_last_received_anchor + @sync_batch_size if @sync_batch_count <= 0 or @sync_batch_count is null set @sync_batch_count = (@sync_max_received_anchor / @sync_batch_size) - (@sync_new_received_anchor / @sync_batch_size) + 1 end -- check if this is the last batch if @sync_new_received_anchor >= @sync_max_received_anchor begin set @sync_new_received_anchor = @sync_max_received_anchor if @sync_batch_count <= 0 or @sync_batch_count is null set @sync_batch_count = 1 end