BizTalk Server 2013 R2 for Low-Latency Scenarios

Article
1/17/2024

Introduction

If you have a transactional system with a service surface, and you need to give that services platform some capabilities like resiliency, high-availability, agnostic connectivity, message persistence, tracing, dynamic transformation, dynamic routing and applying business rules, you better have a product that provides you with those capabilities. BizTalk Server 2013 R2 is a mature product with 15 years of development and nine versions. It not only support the above mentioned capabilities, but it cuts development time with their productivity tools for Visual Studio and simplifies management with its integrated administration tools and management packs.

Some facts about BizTalk Server:

More than 12,500 customers all over the world
81% of the "Top 100" Fortune Magazine
12 of the 15 largest Retailers in the World run Microsoft BizTalk
5 of 10 largest Hotel Chains in the World with over 2 Million rooms use Microsoft BizTalk
9 of 10 largest U.S. Telecommunications Companies use Microsoft BizTalk
6 of the 8 largest U.S. Pharmaceutical Companies use Microsoft BizTalk
4 of the 5 largest U.S. Electronics Parts Manufacturers use Microsoft BizTalk
9 of the 10 largest Aerospace and Defense Companies in the U.S. run Microsoft BizTalk
5 of the 8 largest U.S. Chemical Companies run Microsoft BizTalk
4 of the 5 largest Railroads in the U.S. run Microsoft BizTalk
9 of the 10 largest Insurance Companies in the World run Microsoft BizTalk
23 of 27 EU member governments use Microsoft BizTalk to provide more efficient government services

But it has been always a concern that BizTalk Server Publish/Subscribe architecture prevents its usage for these scenarios, as it uses a MessageBox database for persistence and subscribers evaluation. We have seen the image in Figure 1 many times when we talk about BizTalk architecture. If you think that on a typical orchestration based solution that routes an external request to an internal service, we have at least four writes to the database with a default polling interval of 500 milliseconds, you easily come to the conclusion that any low-latency scenario can’t be fulfilled with BizTalk Server.

http://i292.photobucket.com/albums/mm3/gabrielcor42/image2_zpsyikf8afl.png

Figure 1 - BizTalk Runtime Architecture

Well, the good news is that while the above picture is correct from a general architectural overview, it is not the only and it’s the least recommended design for a service bus implementation. In fact, from BizTalk Server 2009, we have had the BizTalk ESB Toolkit, that allows us to use all of the BizTalk tools and capabilities, and bring to life an ESB that gives us new and improved capabilities.

What is low-latency anyway?

One of our largest customers came to us with the concern of using BizTalk Server as their middleware solution because of their uncertainty that it can’t handle the load. As “low-latency” is a very loose term, we asked what their needs were. As they are providing financial services for banks (transaction validation), they had to be able to handle 25 TPS with less than 500ms of response time.

This middleware has to be able to route, aggregate, and transform messages which can be received over different protocols. As we shall see, we were more than able to fulfill their needs.

How the ESB Toolkit helps our purposes

The lifecycle of a message that originates from an external system and traverses through the ESB, is as follows:

An on-ramp receives the message: Which is a BizTalk Receive Location with special pipelines having components that understand how to process the message via an itinerary.
The itinerary is incorporated into the payload: An itinerary is a list of “services” (components that implement a certain interface) that will be executed using the message and its payload.
The itinerary is processed: While still in the receive pipeline, the itinerary is evaluated and the different services are executed.
The message is published to the MessageBox: for message delivery via an Off-ramp (Send Port with ESB aware pipeline).
If it is a request-response pattern: the message will be retrieved from the Send Port and routed back to the original caller (via itinerary processing).

http://i292.photobucket.com/albums/mm3/gabrielcor42/image3_zps9ygrscjh.png

Figure 2 - ESB Toolkit capabilities & Message flow diagram

As you can see, it turns out we can do all the above operations including routing and transformation with just two MessageBox persistence points (providing we do not use orchestration, which is not necessary for most requirements). In fact most of the processing occurs in the receive pipeline even before the first persistence point.

If you want to know more about BizTalk ESB Toolkit you can find more information here

Note: we previously had the option of using BizTalk Server with AppFabric hosting WF for these kinds of scenarios, but as there is no new Microsoft releases of AppFabric, it is no longer an option for new developments.

Proof of Concept

Our proof of concept had to fulfill the following requirements:

Resemble a real world solution: It is not only important to prove that we can handle the load and maintain a certain amount of latency, but to prove that this can be done with a transaction that is similar to what we will find in the real solution.
Implement VETRO (Validate, Enrich, Transform, Route, Operate) pattern: This should guarantee that any functional requirement that we can throw in, can be implemented without changing our solution design.
Use a service façade for consumers and providers: We should be able to change our consumers/providers and the underlying protocols without affecting the whole implementation.

Design

The solution exposes a web service on an ESB ramp via a BizTalk WCF Adapter. A static itinerary is attached to the receive location which executes a transformation and routes the message to a service provider. What the solution effectively does is expose an internal service to external systems with a different service contract, so it has to normalize the messages and route the request and responses to and from the internal service provider.

http://i292.photobucket.com/albums/mm3/gabrielcor42/image4_zpsarmsmg3j.png

Figure 3 - Solution Design

The message flow is as follows:

The client initiates the request with a message corresponding to the operation desired. It is received by the ESB Onramp and the pipeline, which in turn retrieves the itinerary and starts its processing.
The itinerary executes different steps retrieving and transforming the message
Also Inside the itinerary, the route to the destination is resolved.
Only in this moment the message is persisted to the MessageBox, so it can be delivered
Via a special ESB Send Pipeline
And then via a WCF adapter to the service provider which gives us the flexibility we stated in our requirements
The response message is retrieved and processed with a transformation (in the itinerary)
And routed to the MessageBox where it is processed for
Message delivery
To the original caller

BizTalk Artifacts

One of the key concepts behind the usage of a BizTalk based ESB is the simplicity and fast deployment of the solution. This solution has only three new developed components to perform the operation, the whole solution contains:

BizTalk Receive Port and Location: using a standard ESB ramp, which is only a configuration task. This contains a Validation stage.
BizTalk itinerary: The sequence of steps that Enrich, Transform, Route and Operate on the message. This had to be designed in Visual Studio, but it is only a visual design, no code is involved*.*
BizTalk Maps: two maps, one for mapping the original request to the provider request contract and one to map the provider response to the response of the public contract (exposed to the client). These had to be designed in Visual Studio, but it is only a visual design, no code is involved.
BizTalk Send Port: use a standard ESB off Ramp associated with a WCF adapter*.*

As we can see, most of the solution is configuration, and the rest of it is made on visual designers without code involved.

Note: this is an important consideration, as we did not need to use custom code to achieve maximum performance.

Lab Design

Our goal is to create an infrastructure that is reasonably good for our testing but that is not too far away from what an average customer is likely to find adequate for the requirements.

We designed the lab depicted in the figure below, where we have two BizTalk Servers in the same group, with an extra server dedicated to BizTalk Server databases. This is what we called the “ESB Tier” and its purpose is to host all the BizTalk application runtime and our artifacts. Then we have our “Internal Services Tier” which hosts two type of “services”: one is a typical Web Service and the other is implemented as a SQL Server stored procedure. Finally, we have our test agents where we use Visual Studio Team Test for this lab.

http://i292.photobucket.com/albums/mm3/gabrielcor42/image5_zpsrwvovfom.png

Figure 4 - Lab Design

The Azure Advantage and some Warnings

Trying to assemble this lab with our own infrastructure would have been prohibitive for this whitepaper, so we created all this infrastructure in Azure.

One of the advantages that gave us, was that we could use some of the pre-built machines and then reconfigure the platform. Eventually we re-created from scratch, but the first iterations were very smooth. In fact, if you have not tried yet, is very simple to create a BizTalk environment directly from the Azure Gallery.

Nevertheless we need to warn you that there are some caveats that you need to address:

All machines must be on the same VNET
Domain controller an DNS must have static IP
DNS must be added to VNET configuration
Storage must be configured to be locally redundant.

Hardware / Software Catalog

In the end, it doesn’t matter where you host it, to us it was a no-brainer as Azure was the only alternative that enabled us to make this proof of concept at a reasonable cost. Nevertheless, we documented the processing power and resources of our lab for your reference.

The table below contains the main characteristics of the hardware and software involved.

ID	Software	Azure Type	Memory	Processor	Storage
BTS01	BizTalk Server 2013 R2 Ent	G1	28GB	Xeon E5-2698B v3 @ 2.00GHz	127GB
BTS02	BizTalk Server 2013 R2 Ent	G1	28GB	Xeon E5-2698B v3 @ 2.00GHz	127GB
SQLBT01	SQL Server 2014 BizTalk Databases	G1	28GB	Xeon E5-2698B v3 @ 2.00GHz	127GB
SQLSR01	SQL Server 2014 Solution Databases	A2	3.5GB	Xeon E5-2660 @ 2.20 GHz	127GB
WS01	Web Services Host	A2	3.5GB	Xeon E5-2660 @ 2.20 GHz	127GB

Optimization Highlights and possible improvements

To make this throughput and latency possible, we optimized different areas of the operating system and the product.

Platform Optimization

Adjusted Windows Server 2012 for best performance. More information here
Change Pagefile to a fixed size the same size of the available RAM.
Stopped and disabled unused services in BizTalk Server and SQL Server nodes.
Disabled debug flag in web.config files.
Increase the Maxconnection settings from 2 to 48 on BizTalk Server and Web Service web server.
Split TempDB into multiple equal sized files. More information here

BizTalk Server Optimization

Adjusted Windows Server 2012 for best performance. More information here
Change Pagefile to a fixed size the same size of the available RAM.
Stopped and disabled unused services in BizTalk Server and SQL Server nodes.
Disabled debug flag in web.config files.
Increase the Maxconnection settings from 2 to 48 on BizTalk Server and Web Service web server.
Split TempDB into multiple equal sized files. More information here

Possible Improvements

The following are a series of improvements that should lead to better performance. Due to time and resource constraints, we did not include them in our tests.

SQL Optimizations.
Moved Data and Log files to separate LUNs.
Output all Log/Full backups to dedicated LUN.
Spread tables and indexes of the BizTalkMsgBoxDb across multiple filegroups / multiple LUNs.
Created dedicated SQL Server instance for BizTalkMsgBoxDb.
Moved BizTalkMsgBoxDb to a dedicated Server.
Use a static send port instead of a dynamic send port in ESB Off-ramp.

The results

We tested different loads, with 4, 10 and 50 concurrent users. The following sections include the results.

Tracking disabled

In this tests, all BizTalk tracking was disabled, as it is recommended for production environments.

As we can see, in all our tests the messages per second are higher than expected and the average response time is below the 500ms time that met our customer expectations.

Users	Time	Messages Per Second	Avg. Response Time	% < 500ms	Messages Per Day
4	5 min	35.1	0.11	100%	~ 3,032,640
10	5 min	64.4	0.15	100%	~ 5,564,160
50	5 min	156	0.3	100%	~ 13,478,400

In all tests, memory footprint was in normal limits and processor usage was below 10%.

http://i292.photobucket.com/albums/mm3/gabrielcor42/image6_zpsrvscmuww.png

Figure 5 - Memory & Processor Utilization

Tracking Enabled

Results with tracking enabled were similar to what we obtained in the disabled tracking scenario. We enabled full message body and property tracking.

Users	Time	Messages Per Second	Avg. Response Time	% < 500 ms	Messages Per Day
4	5 min	26.8	0.14	100%	~ 2,315,520
10	5 min	44	0.22	100%	~ 3,801,600
50	5 min	121	0.41	100%	~ 10,454,400

We have lost some performance, but we had 4 additional persistence points per message which enables us to review detailed message delivery and all message data.

Conclusions

The truth is out: BizTalk Server can be used for low-latency scenarios as the ones we depicted without any issues.

The factors that we need to address to achieve the desired throughput are:

Hardware and networking: provide the necessary hardware and network connections to ensure speed and minimize latency.
Optimization: Invest in platform and BizTalk Server optimization to avoid pitfalls and bottlenecks that can have an impact on performance or latency.
Design: Use ESB Toolkit itinerary based processing to minimize persistence and enable all BizTalk components and characteristics on-demand.

Using these three simple principles, we can use a robust, well-designed, time-proven and feature-full platform for our transaction processing.

Download

You can get a copy of the whitepaper here.

Share via