Share via

Understanding Persistence Points in Biztalk Orchestration

  • Article

Often ignored, but a really important point when designing orchestrations is of course the persistence points. Understanding of the persistence points becomes really an important factor if you are dealing with orchestration performance. Lets take a glance at what they are and how we can take care of them.  

The SQL Server does bulk of the job at the backend to keep the BizTalk server running.  BizTalk has this inherent behavior of saving the orchestration state to the SQL Server so as to quickly and easily recover from any failures. Persistence points are those points in the orchestration that save the orchestration state to the database.

However, this behavior induces latency in the execution of the orchestration because of several database trips that are needed when executing the them. Although its quite difficult to avoid the persistence points when designing the orchestration, understanding them and using appropriate shapes and patterns can ensure that you keep them to minimum. For example, the orchestration persists its state after end of every transaction, at the end of orchestration, after every send shape etc. However, if the Send shapes are enclosed within atomic transactions, they do not persist the state. Instead, the state is then persisted at the end of the atomic transaction.

In a usual scenario, the persistence in the orchestrations occur at the following points -

  • End of a transactional scope (atomic or long running)
  • At the execution of other orchestrations through the Start Orchestration shape
  • At the Send shape (except in an atomic transaction)
  • At the end of the orchestration

Apart from that, the engine might decide to persist the orchestration in the following cases -

  • When an Orchestration Instance is suspended
  • When the system shutdowns in a controlled manner
  • When the engine determines it wants to dehydrate
  • When an orchestration instance is finished

Persistence can also occur at debugging breakpoints when the orchestration debugger is being run.

Lets just illustrate with a simple example how persistence points can increase the latency during the execution of the orchestration. 

Consider orchestration scenario that has 2 send shapes sending messages to two different systems.

This can be implemented in the following ways -

A. A parallel branch that has two Send shapes.

B. Two send shapes being executed sequentially in the orchestration.

C. Two send shapes being executed sequentially in the atomic shape.

Question: Now which of these do you think would work the fastest?
Of course the (B) scenario is out of question. The two Send shapes will have their persistence points. If the Send shape is the last shape in the orchestration, then the persistence point of the Send shape and the end of the orchestration will be batched together to result in a single persistence point instead of two.

With scenario (A) even we have the parallel branches in place, Biztalk by nature executes them in a sequential manner. Parallel branches do not necessarily mean they are executed on separate threads. Hence, even in this case, it results in sequential execution with two persistence points. Infact, in this case, the persistence point will not be batched with end of the orchestration and hence it results in the slowest of the three.

The scenario (C) takes the fastest position of the three. The send shapes are enclosed in the atomic shape and hence the state is persisted only at the end of the atomic shape. Again, if it is at the end of the orchestration, the persistence point will be batched with the persistence point at the end of the orchestration. Hence resulting in a single persistence point.

Hence, when designing the orchestration from the performance perspective, it is imperative to take a look at what points the orchestration will be persisting the data. Reducing the database trips helps increase the performance of the orchestration.

Comments

  • Anonymous
    November 15, 2006
    Excellent explanation!

  • Anonymous
    January 18, 2007
    Thanks for the explanation!  Very well stated.

  • Anonymous
    April 30, 2007
    I finally got this! Thanks for this great post.

  • Anonymous
    November 13, 2007
    explanation is so clear, thank you for this great lesson.

  • Anonymous
    January 08, 2008
    thanks a lot !! this is something which is not visible to the developer. but after reading, it is very much visible.

  • Anonymous
    January 14, 2008
    Thanks a Lot for this excellent explanation.

  • Anonymous
    February 27, 2008
    ¿How can I avoid any persistence in may orquestrations? I have this behaviour in my deployed solution: Sometimes, the sql server machine has "memory problems" like this: The following stored procedure call failed: " { call [dbo].bts_FindSubscriptions}". SQL Server returned error string: "Could not perform the requested operation because the minimum query memory is not available. Decrease the configured value for the 'min memory per query' server configuration option.". And , if an orquestration is running, in that moment, the engine repeats all the process, so I have other problems due to this behaviour, I should prefer that the process would terminate... Is that possible? Thanks in advance! Ana

  • Anonymous
    March 25, 2008
    The comment has been removed

  • Anonymous
    April 20, 2008
    The comment has been removed

  • Anonymous
    May 12, 2008
    Bhavik, Thanks a lot for your comments and a really good description for dehydration. --Sanket

  • Anonymous
    October 01, 2008
    Well explained and expect more on near future

  • Anonymous
    June 22, 2010
    The explanation Sanket gave and the one by Bhavik are somewhat contradictory. According to Sanket, the point at which the orchestration state is saved to the database is Persistence point and going by Bhavik's waty, the point at which the data is taken out of the database is persistence point. So which of the two is the right one?

  • Anonymous
    July 11, 2010
    Abhishek, As I see it, both of us are saying the same thing - at the persistence point, the state is saved (persisted) in the database. When the orchestration needs to be reconstituted in the memory, thats when it starts again from the last persistence point that was saved from the database. Hope this makes it clear.

  • Anonymous
    January 15, 2015
    Thanks a lot for this clear cut explanation ! It helped me a lot

  • Anonymous
    July 28, 2016
    Timeless!

  • Anonymous
    November 11, 2016
    Thanks for this. Cleared a lot of doubts.