BizTalk Server: Ordered Delivery

Article
1/17/2024

Introduction

It is one more description of the Ordered Delivery (OD) in BizTalk.

The main article about it is in MSDN. There is also a blog article Improved performance for ordered send ports in BizTalk

Here I am discussing the BizTalk Ordered Delivery “implementation details”.

OD Considerations

Ordered Delivery (sequential) mode is opposite of the “Parallel Delivery” mode. Parallel Delivery is the most productive mode; the Ordered Delivery is less productive mode.
Transports such as MSMQ AMQP, and protocols, supporting the WS-ReliableMessaging, support OD. Other protocols as FTP, File, SQL, SMTP etc. do not have notion of the “order”.
BizTalk application usually is a part of the message workflow which includes other systems.
There are two approaches in the OD implementation:
- All steps of the message workflow independently support OD.
- A Destination System implements the re-sequencing and manages lost and duplicate messages.
Order is relative. Sequences can be ordered regards one or several parameters. For example, OD for the Company documents or for documents belongs to one Department inside a Company.

OD and the BizTalk Architecture

The BizTalk MessageBox database is an implementation of the Message Queue. OD is an intrinsic feature of the MessageBox.
The BizTalk Server works in the Parallel Delivery mode by default.
There are three parts in the BizTalk message exchange outside of the MessageBox in relation to OD: Receive Locations; Orchestrations; Send ports.
- Receive Locations support OD on the Transport level (say, MSMQ, Azure ServiceBus and some WCF adapters).
- OD in Orchestrations is implemented by the sequential convoy pattern.
- Send ports support OD for all static adapters.
The BizTalk Pipelines (part of Receive and Send Ports) always process messages in order using streams.

OD and Ports

To force Send Ports work in order we flag the “Ordered Delivery” in Send Ports, then the BizTalk MessageBox takes care of implementing OD.

To force Receive Locations work in order we flag the “Ordered Delivery” option in the Receive Location Transports, whenever is possible. Not all Transports have such option. With this flag set up, the BizTalk Transport Adapter takes care of implementing OD.

Note: Ordered Delivery Send Port instance works as a singleton service. Since start it stays in Running state. It will not recycle if we restart its Host Instance. We could manually terminate it if we want.

OD and Orchestrations

MessageBox implements the convoy message exchange pattern [See Using Correlations in Orchestrations]. See the detail convoy description in the BizTalk Server 2004 Convoy Deep Dive article.

It is not just a design pattern that developer should follow. There are special mechanisms inside MessageBox responsible for implementing OD.

OD and Orchestration: Sample

Imagine four Orchestrations which implement four approaches to the sequencing.

The first is the AllInParallel Orchestration. It processes all messages without any order. One AllInParallel Orchestration instance will be created for each inbound message.

The AllInOrder Orchestration processes all messages in one sequence. Only one AllInOrder Orchestration instance will be created.

The OrderByCompanyAndDate Orchestration processes messages in sequences correlated by the Company value (A, B, C, D, etc.) and by date. The separate queue is created for each new value of the Company for each new day. Messages inside queues are processed in order. Queues for different Companies+day are independent. A separate OrderByCompanyAndDate Orchestration instance will be created for each new Company value for each new day.

The OrderByProductAndDate Orchestration works exactly as the OrderByCompanyAndDate Orchestration but correlated by the Product value (xx, yy, etc.) instead of Company.
The number of companies is bigger than the number of products, so the number of orchestration instances for the OrderByCompanyAndDate orchestrations potentially bigger than number of instances of OrderByProductAndDate and OrderByCompanyAndDate would be more performant.** **
In way by choosing the right parameters to create an order, we could control the OD performance in orchestrations.

Discussions

Performance

By default, all Orchestration and Messaging Service instances work in the Parallel Delivery mode with maximum performance.

If we check up the Ordered Delivery option in Send Port, BizTalk will initiate the Send Port instance as a singleton service. It is always a single instance. We don’t have the flexibility of the Orchestration where we could tune up “the proportion of the sequencing” and could control the recycling of the service instances.

Send Port OD could be in two states, on and off:

OD is off: a service instance per message, one message per queue, maximum performance.
OD is on: one infinite service instance, all messages in one queue, minimum performance.

Orchestration OD could be in two states also, on and off. It is not just change in the orchestration run-time parameters (as it is in Ports), but a deep redesign of the Orchestration workflow.

OD is off: a service instance per one activating message, one activating message per queue, maximum performance.
OD is on: one or several service instances, one per new correlation set value; all correlated messages per queue; performance is from min to max, depends on the correlation set.

Carefully designing the correlation sets we could tune up the performance of the Orchestration. For example, if we only care of the document order for each separate Company, we include the Company to the Correlation set. If we had thousand documents related to hundreds of companies, the performance will be near maximum. If there are only two companies, the performance will be near minimum, and we should consider improving the correlation with one more parameter.

Orchestrations and Zombies

Zombies are big problem of Convoy Orchestrations. See BizTalk: Instance Subscription and Convoys: Details article with description of this problem. This problem could be mitigated but could not be completely removed. We are waiting a new version of the BizTalk Server where this issue will be targeted.

BizTalk Server version Next, Ordered Delivery and Zombies

It is possible the BizTalk Server version Next will implement the automatic Ordered Delivery for Orchestrations, with a pattern similar to the Ordered Delivery in Send Ports.

Three new Orchestration parameters are shown up there: Ordered Delivery, Stop on Exception, and Recycle Interval (in seconds).

Ordered Delivery parameter works in the same way as the Ordered Delivery parameter of the Send Port. Now we don’t have to build the Convoy Orchestration manually. No more Loop inside Orchestration.

It would be great if those parameters are accessible at the run-time.
If the Ordered Delivery parameter set up to True, the Orchestration is working as a Singleton. The first Receive shape receives all correlated messages in sequence. Correlation set is created implicitly regards of the Activation Subscription expression.

There are several limitations for this kind of Orchestration. The most important is: only one Receive shape is permitted here.

There are two big advantages of this new feature:

Simplified Orchestration design for the Ordered Delivery.
No more Zombies. The Orchestration instance is recycled in a controllable way, when no messages, matched the Orchestration Subscription, are placed in the MessageBox.

Conclusion

We discussed the Ordered Delivery implemented in the BizTalk Server and ways to improve it.

Share via