BizTalk: Improve Deployment and Tracking by Always Creating Custom Pipelines

아티클
1/17/2024

Introduction

Depending on what features you have installed, BizTalk Server ship with a number of default pipelines you can use in your application. Most developers will be familiar with the common ones:

Receive Pipelines	Send Pipelines
PassThruReceive	PassThruTransmit
XMLReceive	XMLTransmit
EDIReceive	EdiSend
EDIReceive	AS2Send
AS2EdiReceive	AS2EdiSend

While these Pipelines are functional and perfectly fine for testing or experimental purposes, we should always create and deploy custom Pipelines for all Ports used in our apps, even if they are same composition as the default Pipelines.

There are a number of reasons to take this minimal extra step.

First, and relatively minor is that the Tracking settings for the default Pipeline are turned off by default. This has led to a number of questions on the Forums about “missing messages” or “BizTalk not working”. The reality is that the app is running normally, but no tracking data is being written. The solution is simple, enable Tracking, but it’s an all or nothing scenario with the default pipelines.

Next, using custom Pipelines makes Tracking readable and understandable.

Finally, this is only* way to eliminate any current and future Schema resolution errors raised by the XmlDisassembler or XmlAssembler. I have rule, all XmlDisassembler and XmlAssembler component must be configured specifically with the Envelopes and Documents they are expected to process. Relying on the default global resolution is forbidden.

Takeaway Summary:

Always create a unique Custom Pipeline for every Port in your app.
Always explicitly set the Message Types on the XmlDisassembler and XmlAssembler Pipeline Components.

The sample Visual Studio Solution can be downloaded from: BizTalk: Improve Deployment and Tracking by Always Creating Custom Pipelines

*Technically, it’s not the only way, but it is the correct way.

Tracking Views With and Without Custom Pipelines

Using Default Pipelines

BizTalk Tracking is one of the best and most useful features in the product. It is a tremendous diagnostic, analysis and archival tool I’m sure most of us use every day, often to identify problems with other applications .

If you’re using the default Pipelines and go to Tracking to look for a particular message that came in or went out, you’d select either Completed or Suspended and be greeted with results that look like this:

Which is which? Adding extra columns doesn’t help since Port Name, what we’re always looking for, is not available. To find a message on a specific Port, we’d have to view the Message Flow for every Service Instance until we eventually find the one we’re looking for.

Using Custom Pipelines

Adding a new Pipeline to a project is pretty easy. Right-click, Add New Item, then goose Send Pipeline or Receive Pipeline. The focus of this article not the creation of Pipelines, but I will mention one thing, there is nothing wrong with creating and deploying Pipelines that are the exact same composition as any default Pipeline. For example, this Pipeline on the left is the exact same as XmlReceive while the one on the right is empty, no components, and is the same as PassThruTransmit:

The benefit of these custom Pipelines is that we set our own Type name, which is surfaced as Service Name in BizTalk Administrator.

When when we have custom Pipelines set on the App’s Ports, Tracking presents a whole new dimension of data:

Instead of indiscriminate operations, we can see immediately what ports are running and with basic knowledge of the system, correlate them to quickly narrow the scope of any diagnostic investigation. For instance, it is reasonable that the Message received and processed by TradingPartnerBReceive was then sent to LOBOneTransmit and LOBTwoTransmit.

To get the most usability from Tracking by using this pattern, you would and should create and deploy a custom Pipeline for every Port and Direction used in your app, even if they are functionally the same. There is no ‘performance’ consideration with this pattern at all.

Custom Pipelines to Eliminate Deployment Dependencies

BizTalk Server has a very strict dependency policy in that no resources can be removed from the system if any other resource depends on it. This has caused deployment pain in the past where developers have referenced existing artifacts, particularly Schemas form other projects. The pain is that in order to update any Assembly with a referenced Artifact, all dependent Assemblies must be undeployed first.

There are two ways to address this, Assembly Versioning or eliminating all dependencies. My preference, and another unalterable rule, is that dependencies between any separate deployment unit are not allowed. Meaning, you must never require any other app to be undeployed to update another.

The only notable consequence of this rule is that multiple copies of the same Schema are sometimes deployed. In the example above, there are two copies of LOBAppMessage deployed, one with ProjectA, the other with ProjectB. This is not a problem and there is nothing wrong with it. Having multiple Schemas with the same signature, namespace#root, is fully supported. The only point where this causes a problem for unprepared apps is when using the automatic Schema resolution in the XmlDisassembler and XmlAssembler. The solution is a custom Pipeline.

If you’re wondering why the Target Namespace column is empty, see my previous article: BizTalk: Simplify BizTalk Dev by Using the Empty Namespace

Configuring the XmlDisassembler and XmlAssembler

Configuring an app to use only local Schemas involves setting the Document schemas, and when necessary the Envelope schemas, property of the XmlDisassembler and XmlAssembler component.

With the schema Type explicitly set on the XmlDisassembler, there can be any number of LOBAppMessage deployed, but only the local copy, BizTalkProjectB.LOBAppMessage, will be used. For clarity, the schema can be in a different Assembly than the Pipeline, but they both should be in the same Solution and always Deployed together.

One point that’s often missed, automatic Schema resolution is one of only two places in BizTalk where the the combination of namespace#root is used. Everywhere else, the .Net Type (BizTalkProjectB.LOBAppMessage) is used. The other place is a default Subscription predicate on Orchestrations Ports when a Typed Message is used. There are other issues associated with this so we should never rely on BTS.MessageType only for Orchestration Subscriptions.

Side Deploying to Fix Older Apps

Even though we will all correctly develop and deploy our apps with custom Pipelines and properly configured Xml components, many older apps may still rely on default pipelines and Schema resolution. The creates problems when we deploy newer applications that follow the rules. Most often, the result is a break in the older app because of now duplicate Schema names:

“Reason: Cannot locate document specification because multiple schemas matched the message type”

Fortunately, it is very easy to patch these older apps by simply side-deploying an Assembly with custom Pipelines and configured Xml components.

Create a new BizTalk Project. It can be part of the legacy solution if possible, but this is not required.
Add a references to the Project with the Schemas to be resolved by this Pipeline and any other required Pipeline Components.
Develop a Pipeline with the exact same composition and configuration as the currently used Pipeline but explicitly set any Message Types on the XmlDisassembler and XmlAssembler.
Build the Project.
In BizTalk Administrator, add the new Pipeline Assembly directly to the legacy Application or an Application that it References.
Set the new Pipelines on any Port where they are required.

Working With the Sample Project