Share via

BizTalk patterns – the parallel shape

  • Article

The parallel shape in BizTalk Server (v2004 to v2009) is widely misinterpreted to be multi-threaded.

image

What the parallel shape does well is that it ensures that all branches complete their execution before the next process takes place. The parallel shape does a context switch on a thread wait (busy waiting, such as waiting for a message to come in from a receive port) between branches.
The reason why the parallel shape is single-threaded, is because the orchestration workflow itself, is single-threaded. However, it is still possible to conduct “multi-threaded” execution within BizTalk Server. image   image*Image courtesy of Werner Van Huffel A common practice to “multi-thread” the parallel shape is to use the “Start Orchestration” shape, which will instantiate a separate orchestration workflow, with its own thread, and resources.image Hence, all the logic would reside in the child orchestrations that would be called from the parent orchestration.

However – suppose that the number of branches of executions are not known at design time, then the parallel shape would not be suitable to use. A different approach will be required.

A classic example of parallel execution is de-batching a large message file into smaller files, and then aggregating all the results into a single large result. Each subset (small file) represents an independent unit of work that does not rely or relate to other subsets, except that it belongs to the same batch (large message). Eventually each child process (the independent unit of work) will return a result message that will be collected with other results, into a collated response message to be leveraged for further processing.

image

image *Image courtesy of Werner Van Huffel 

0. The initial step consists of de-batching the large message in the receive pipeline into smaller child messages.

1. The receive pipeline publishes the child messages to the message box database.

2. Child business process orchestrations (or 2-way send ports) subscribe against the child messages to accommodate business requirements. Note that individual business processing does not explicitly require orchestrations.

3. Responses from Line-of-Business (LOB) applications are submitted back to the child orchestrations (or to 2-way send ports).

4. These responses are aggregated back by an aggregator (can be accommodated by an orchestration as depicted, or by a SQL stored procedure).

5. The aggregator will submit the final aggregated result.

6. The next process (which may or may not use the final aggregated result) will proceed.

De-batching is conducted within the receive pipeline. To learn how to split a large file into smaller files, please refer to the BizTalk Server 2006 samples – Split File Pipeline example. Even though the sample is based on BizTalk Server 2006 (R1) – it remains relevant for BizTalk Server 2006 R2, and BizTalk Server 2009.

  • Note: There are several examples on how to de-batch messages, such as de-batching from SQL Server, as communicated by Rahul Garg’s blog on de-batching.

The next step is a bit trickier. The smaller messages need to be correlated somehow, so that their results can be aggregated into a singular large response message (for steps 4 and 5 to be successful). Correlation is typically done in orchestration, but the proposed method here avoids the use of orchestration altogether. In order to have correlation in messaging, please refer to Paolo Salvatori’s blog on Synchronous To Asynchronous Flows Without An Orchestration.

In it, he describes a method that requires a 2-way receive port (such as a web service) , because he leverages a couple of message context properties that are unique to a 2-way receive. These are

  • ReqRespTransmitPipelineID
  • IsRequestResponse

In addition to

  • EpmRRCorrelationToken
  • Correlation Token

It is strongly suggested to learn how correlation can be achieved in messaging (without orchestration) as described by Paolo Salvatori before proceeding to the next step.

Once that method has been mastered, a slight adjustment is needed. Since the de-batching method is leveraged in the pipeline, correlation in messaging can be achieved with a 1-way receive port (such as picking up the large message from a file directory), by using message context properties that are unique to the de-batch method.

These are

  • InterchangeID
  • InterchangeSequenceNumber
  • LastInterchangeMessage

Each child message (which is now in the messagebox database) should instantiate a business process (either with an orchestration workflow or a 2-way send port) that will eventually receive a result message.

Finally an aggregator is required to collect and collate the results from the child business processes. This aggregator can be an orchestration, or the preferred way would be from a SQL stored procedure.

image *Image courtesy of Paolo Salvatori

  • A message is received by a FILE Receive Port
  • The message is routed to a SQL Send Port where the original message is wrapped into a sort of envelope by a map which uses a custom XSLT. The outbound message is an Updategram which calls a stored procedure to insert the message into the table.
  • A SQL receive port calls a stored procedure to retrieve the records of a certain type from the custom table. This operation generates a single document.
  • This document is routed to a File Send Port where the message is eventually transformed into another format.

This method was also communicated in https://geekswithblogs.net/asmith/archive/2005/06/06/42281.aspx

Note that the XSLT can be eventually replaced by a custom pipeline component or directly by an envelope (in this case the send pipeline needs to contain the Xml Assembler component).

Sample code is provided courtesy of Ji Young Lee, of Microsoft (South) Korea. Uncompress the ZIP file with the directory structure intact on the C:\ root directory (i.e.: c:\QuocProj).

 

Please note the standard disclaimer that abdicates any form of responsibility:

The sample code is the property of Microsoft Corporation ®.

THIS CODE AND INFORMATION IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE.

The sample code is not supported under any Microsoft standard support program or service. These samples are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the samples and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the samples, code, and documentation, be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the code, samples, or documentation, even if Microsoft has been advised of the possibility of such damages.

For more information concerning the parallel shape, see “How to Configure the Parallel Actions Shape” topic in the BizTalk Server documentation.

Here are some related blogs regarding the parallel shape:

 

Technorati Tags: BizTalk Server,parallel,de-batching,aggregate,aggregator,debatch,large message processing