How to Develop BizTalk Custom Pipeline Components - Part1
Receive and send pipelines in BizTalk are used to perform a range of processing and operations on messages. They can –
1. Encrypt and decrypt messages
2. Sign and verify digitally signed messages
3. Validate message against schema
4. Deal with promoted properties used for content based routing
5. Disassemble or break single message into multiple
6. Wrap messages with header and footer
Pipelines contain stages and each stage can hold more than one pipeline component. Pipelines and pipeline components present out of box can do most of the tasks for you. But sometime specific message processing or massaging requirements encourage developers to develop custom pipeline components. Current series is focused on describing development of custom pipeline components.
There are multiple categories of pipeline components each doing specific job when employed on their predefined stages. These categories are –
Pipeline Components |
Stages Employed |
Tasks |
General Component |
Decode, Encode, Pre-assemble, Resolve Party or Validate |
Take one message process message and produce zero or one message |
Disassemble Component |
Disassemble |
Split message, promote custom properties |
Assemble Component |
Assemble |
Used to wrap message with head or trailer or both |
Probe Component |
This is not an independent component. Any pipeline component can implement the IProbeMessage interface if it must support message probing functionality. |
Enables the component to check the beginning part of the message |
During development, each of these components implements particular interfaces and execute custom written job (code) on messages. I will walk through development of each of these components with sample code.
Developing General Pipeline Component
As stated earlier, pipeline components implement certain interfaces. General pipeline component implements following interfaces –
d. IPersistPropertyBag (Optional. Required when pipeline design time properties are to be defined)
All pipelines implement this interface. Interface provides members which can be implemented to provide information about pipeline.
Members |
Usage |
Description |
Property. Used to specify small description about pipeline component. Description is visible on pipeline properties page at design time. |
Name |
Property used to specify name of pipeline component. Name is visible on pipeline properties page at design time. |
Version |
Property used to specify version (example 1.0.0.0) of pipeline component. Visible on pipeline properties page at design time. |
All pipeline components must implement this interface. Members of this interface provide design time support.
Members |
Usage |
Icon |
Property used to provide icon associated with pipeline component. |
Validate |
Method. This is called by pipeline designer before pipeline compilation to verify that all configuration properties are correctly set. |
This is core interface. Member of this interface is implemented for specific message processing/massaging.
Members |
Usage |
Execute |
Method. Does specific processing/massaging in inbound message and produces output message to be forwarded to next stages of pipeline or message box. |
IPersistPropertyBag
Interface provides members which can be used to save and load pipeline design time properties with property bag. This interface is implemented only when pipeline’s design time properties are to be defined.
Members |
Usage |
GetClassID |
Method. Retrieves the component's globally unique identifying value. |
InitNew |
Method. Initializes any objects needed by the component to use the persisted properties. |
Load |
Method. Used to load property from property bag. |
Save |
Method. Used to save property to property bag. |
Enough theory. This is time for some coding.
Code Walk Though with Sample
I am going to create a sample general pipeline component. Scenario is very simple. Whenever, I receive a message, I want to change namespace of message in pipeline component. Namespace to be changed can be configured in design time property of pipeline.
If I get a message like following –
<Input xmlns:ns0="https://MyNamespace">
<FirstName>Peter</FirstName>
<LastName>Decosta</LastName>
</Input>
And if changed namespace property is set to “https://MyNewNamespace” , then output message (which goes to next stages of pipeline and message box) would become –
<Input xmlns:ns0="https://MyNewNamespace">
<FirstName>Peter</FirstName>
<LastName>Decosta</LastName>
</Input>
I am using VS 2005 for development.
1. Create a class library project say SampleGenPipelineComponent.proj. When project is created, rename Class1.cs to SampleGenComponent.cs.
2. Configure strong name key file in project property so that assembly can be signed.
3. Add following reference
Microsoft.BizTalk.Pipeline.dll
References can be found in folder “C:\Program Files\Microsoft BizTalk Server 2006” depending on location of BTS installation.
4. Add following namespaces.
using System.Xml;
using System.IO;
using Microsoft.BizTalk.Message.Interop;
using Microsoft.BizTalk.Component.Interop;
5. Implement following interfaces-
IBaseComponent, IComponentUI, IComponent, IPersistPropertyBag
6. Add following attributes to pipeline component class
[ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
[ComponentCategory(CategoryTypes.CATID_Decoder )]
[System.Runtime.InteropServices.Guid("9d0e4103-4cce-4536-83fa-4a5040674ad6")]
public class SampleGenComponent :
IBaseComponent,
IComponentUI,
IComponent,
IPersistPropertyBag
General components can be used in various stages – Decode, Encode, Validate etc. Component Category attribute is used to limit use of general pipeline to restricted stages. In my example, I am restricting my pipeline to be used in Decode stage only. This concept is called “Stage Affinity”. Please read following MSDN link to learn more about this –
https://msdn2.microsoft.com/en-us/library/ms964541.aspx
7. Implement members of IBaseComponent interface.
public string Description
{
get
{
return "Pipeline component used to change namespace of message";
}
}
public string Name
{
get
{
return "SampleGenPipelineComponent";
}
}
public string Version
{
get
{
return "1.0.0.0";
}
}
Member properties are overriden to provide description, name and version of pipeline component.
8. Implement members of IComponentUI interface.
public IntPtr Icon
{
get
{
return new System.IntPtr();
}
}
public System.Collections.IEnumerator Validate(object projectSystem)
{
return null;
}
I have kept things simple and have not provided any pipeline design time icon and validation logic. In actuall implementation, you can add an icon in piepline resouce file and point that in impelemtation. Similarily, you can also verify design time configuration/properties in Validate method and can return all inconsistancies in collection of errors.
9. Implement members of IPersistPropertyBag interface.
private string _NewNameSpace;
public string NewNameSpace
{
get { return _NewNameSpace; }
set { _NewNameSpace = value; }
}
public void GetClassID(out Guid classID)
{
classID = new Guid("655B591F-8994-4e52-8ECD-2D7E8E78B25C");
}
public void InitNew()
{
}
public void Load(IPropertyBag propertyBag, int errorLog)
{
object val = null;
try
{
propertyBag.Read("NewNameSpace", out val, 0);
}
catch (Exception ex)
{
throw new ApplicationException("Error reading propertybag: " + ex.Message);
}
if (val != null)
_NewNameSpace = (string)val;
else
_NewNameSpace = "https://AnonymusURL";
}
public void Save(IPropertyBag propertyBag, bool clearDirty, bool saveAllProperties)
{
object val = (object)_NewNameSpace;
propertyBag.Write("NewNameSpace", ref val);
}
First of all, we created “NewNameSpace” property which is used to configure new namespace name at design time.
“Load” method loads value from property bag to pipeline property. “Save” method saves value to property bag from pipeline property. “GetClassID” returns component's unique identifying value in terms of GUID. “InitNew” is used to initialize object to be persisted in component properties. I have not implemented it because I don’t require.
10. Implement members of IComponent interface.
public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)
{
IBaseMessagePart bodyPart = pInMsg.BodyPart;
StringBuilder outputMessageText=null;
string systemPropertiesNamespace = @"https://schemas.microsoft.com/BizTalk/2003/system-properties";
string messageType = "";
if (bodyPart != null)
{
Stream originalStream = bodyPart.GetOriginalDataStream();
if (originalStream != null)
{
XmlDocument xdoc = new XmlDocument();
xdoc.Load(originalStream);
XmlElement root = xdoc.DocumentElement;
messageType = this.NewNameSpace + "#" + root.Name;
outputMessageText = new StringBuilder();
outputMessageText.Append("<" + root.Name + " xmlns:ns0='" + this.NewNameSpace + "'>");
outputMessageText.Append(root.InnerXml);
outputMessageText.Append("</" + root.Name + ">");
byte[] outBytes = System.Text.Encoding.ASCII.GetBytes(outputMessageText.ToString());
MemoryStream memStream = new MemoryStream();
memStream.Write(outBytes, 0, outBytes.Length);
memStream.Position = 0;
bodyPart.Data = memStream;
pContext.ResourceTracker.AddResource(memStream);
}
}
pInMsg.Context.Promote("MessageType", systemPropertiesNamespace, messageType);
return pInMsg;
}
This is core of implementation. Code flow is –
- First read body part of message “ … = pInMsg.BodyPart;”
- Then read message content (XML) in stream “… = bodyPart.GetOriginalDataStream();”
- Change namespace of XML content “outputMessageText”
- Create a stream of new XML (with namespace)
- Assign stream back to message body part “bodyPart.Data = memStream;“
- Since namespace has changed, we need to update message type context property otherwise there is a fair chance of message routing failure. “pInMsg.Context.Promote("MessageType",…”. To understand importance of MessageType, please refer “Message Processing” section in link - https://msdn2.microsoft.com/en-us/library/ms935116.aspx
- Return message so that it can be consumed by next stages of pipeline and/or message box itself.
11. Coding is complete. Build the project.
Deploy Component and Use
To deploy pipeline component, Copy SampleGenPipelineComponent.dll to “C:\Program Files\Microsoft BizTalk Server 2006\Pipeline Components”.
Pipeline component is now ready to use. In BTS receive pipeline project, add this pipeline component dll in toolbar and then use in decode stage. You will find that you can set value of “NewNameSpace” property at pipeline design time.
Summary
This was a sample general pipeline component which can be deployed to decode stage. Similarly, you can create other general pipeline components for requirement specific processing and data massaging. Hope this entry was useful. I am also attaching code zip file (SampleGenPipelineComponent.zip) for any further reference.
SampleGenPipelineComponent.zip
Comments
- Anonymous
December 26, 2006
This solution works well for small and moderate sized messages. Do you have any alternative suggestions when the message size is large (i.e., many MBs in size)? - Anonymous
December 26, 2006
With large message, it is not advisable to use XMLDocument. You can use other XML APIs, XmlReader, XmlWritter or any other stream based Xml modification API. I will do search on this and get back to you. - Anonymous
January 22, 2007
Hello,How would I get the guid of the class to be used in GetClassID ?thanksAK - Anonymous
January 24, 2007
to AK...You may create him with "Create Guid" tool in Visual Studio...For this in VS -> Tools -> Create GUID-> New GUID(in Registry Format) and copy him to your project.or System.Guid myGuid = new Guid(); in C# code....Vadim - Anonymous
April 02, 2007
The comment has been removed - Anonymous
April 07, 2007
Hi RSIf you check sample given with blog, you can get message body content as stream -Stream originalStream = bodyPart.GetOriginalDataStream();if this stream is XML based data then you can use .net DOM APIs or XMLReader to get required xml elements/fields.if stream is non-xml data then you need to do looping of records and record parsing to get required field data. - Anonymous
July 15, 2007
It's recommended to use VirtualStream to manipulate message streams in pipeline processing. HIH.Bruno Spinelli. - Anonymous
January 11, 2008
The comment has been removed - Anonymous
March 11, 2008
The comment has been removed - Anonymous
June 04, 2008
How should i dispose/close the memory stream used in this sample. if i do so, i don't get the changed message when debugging the pipeline? - Anonymous
July 10, 2008
The comment has been removed - Anonymous
April 18, 2010
I like it! thanks very much for writing this article - I will be referenceing it as soon as I understand how to create pipelines in BTS - regards, Van (wvhuffel)