How To Boost Message Transformations Using the XslCompiledTransform class

Introduction

The BizTalk Runtime still makes an extensive use of the System.Xml.Xsl.XslTransform.  When you create and build a BizTalk project, a separate .NET class is generated. for each transformation map. Each of these classes inherits from the Microsoft.XLANGs.BaseTypes.TransformBase class. For convenience, I used Reflector to retrieve and report its code in the table below. As you can easily note, the get accessor of the Transform property returns a XslTransform object.

TransformBase class

[Serializable]public abstract class TransformBase{ // Methods protected TransformBase() { } // Properties public virtual string[] SourceSchemas { get { return null; } } public BTSXslTransform StreamingTransform { get { StringReader input = new StringReader(this.XmlContent); XmlTextReader stylesheet = new XmlTextReader(input); BTSXslTransform transform = new BTSXslTransform(); transform.Load(stylesheet, null, base.GetType().Assembly.Evidence); return transform; } } public virtual string[] TargetSchemas { get { return null; } } public XslTransform Transform { get { StringReader input = new StringReader(this.XmlContent); XmlTextReader stylesheet = new XmlTextReader(input); XslTransform transform = new XslTransform(); transform.Load(stylesheet, null, base.GetType().Assembly.Evidence); return transform; } } public XsltArgumentList TransformArgs { get { XmlDocument document = new XmlDocument(); document.PreserveWhitespace = true; document.LoadXml(this.XsltArgumentListContent); XsltArgumentList list = new XsltArgumentList(); foreach (XmlNode node in document.SelectNodes("//ExtensionObjects/ExtensionObject")) { XmlAttributeCollection attributes = node.Attributes; XmlNode namedItem = attributes.GetNamedItem("Namespace"); XmlNode node3 = attributes.GetNamedItem("AssemblyName"); XmlNode node4 = attributes.GetNamedItem("ClassName"); object extension = Assembly.Load(node3.Value).CreateInstance(node4.Value); list.AddExtensionObject(namedItem.Value, extension); } return list; } } public abstract string XmlContent { get; } public abstract string XsltArgumentListContent { get; }}

 

When BizTalk Server 2004 was built, the  XslTransform was the only class provided by the Microsoft .NET Framework 1.1 to apply an XSLT to an inbound XML document. When the Microsoft .NET Framework version 2.0. was released, the  XslTransform was declared obsolete and  thus deprecated. As clearly stated on MSDN, the System.Xml.Xsl.XslCompiledTransform should be used instead. This class is used to compile and execute XSLT transformations. In most cases, the XslCompiledTransform class significantly outperforms the XslTransform class in terms of time need to execute the same XSLT against the same inbound XML document. The article Migrating From the XslTransform Class on MSDN reports as follows:

“The XslCompiledTransform class includes many performance improvements. The new XSLT processor compiles the XSLT style sheet down to a common intermediate format, similar to what the common language runtime (CLR) does for other programming languages. Once the style sheet is compiled, it can be cached and reused.”

The caveat is that because the XSLT is compiled to MSIL, the first time the transform is run there is a performance hit, but subsequent executions are much faster. To avoid paying the extra cost of initial compilation every time a map is executed, this latter could be cached in a static structure (e.g. Dictionary). I’ll show you how to implement this pattern in the second part of the article. For a detailed look at the performance differences between the  XslTransform and XslCompiledTransform classes (plus comparisons with other XSLT processors) have a look at following posts.

• XslCompiledTransform Performance: Beating MSXML 4.0
• XslCompiledTransform Slower than XslTransform?

Although the overall performance of the XslCompiledTransform class is better than the XslTransform class, the Load method of the XslCompiledTransform class might perform more slowly than the Load method of the XslTransform class the first time it is called on a transformation. This is because the XSLT file must be compiled before it is loaded. However, if you cache an XslCompiledTransform object for subsequent calls, its Transform method is incredibly faster than the equivalent Transform method of the XslTransform class. Therefore, from a performance perspective:

• The XslTransform class is the best choice in a "Load once, Transform once" scenario as it doesn't require the initial map-compilation.

• The XslCompiledTransform class is the best choice in a "Load once, Cache and Transform many times" scenario as it implies the initial cost for the map-compilation, but then this overhead is highly compensated by the fact that subsequent calls are much faster.

As BizTalk is a server application (or, if you prefer an application server), the second scenario is more likely than the first.  The only way to take advantage of this class (given that BizTalk does not currently make use of the XslCompiledTransform class) is to write custom components. If this seems a little strange to you, remember that all BizTalk versions since BizTalk Server 2004 have inherited that core engine, based on .NET Framework 1.1.  Since the XslCompiledTransform class wasn’t added until .NET Framework 2.0, it wasn’t leveraged in that version of BizTalk. While I’m currently working with the product group to see how best to take advantage of this class in the next version of BizTalk, let’s go ahead and walk through creating a helper class to boost the performance of message transformations in your current BizTalk implementation using the XslCompiledTransform class and let’s compare its performance with another helper component that makes use the old XslTransform class.

BizTalk Application

In order to compare the performance of the XslTransform and XslCompiledTransform classes I created an easy BizTalk application composed of the following projects:

Helpers

This library contains 2 helpers classes called, respectively, XslTransformHelper and XslCompiledTransformHelper. These components share most of the code and expose the same static methods. I minimized the differences between the 2 classes as the final scope was to compare the performance of the XslTransform and XslCompiledTransform classes. As their name suggests, the first helper class uses the XslTransform class, while the second makes use of the XslCompiledTransform class.  The Transform static method of both helper classes provides multiple overloads/variants/signatures. This allows the components to be invoked by any orchestration, pipeline component or .NET class in general. Either classes use a static Dictionary to cache maps in-process for later calls. The fully qualified name (FQDN) of a BizTalk map is used as key to retrieve the value of the corresponding instance within the Dictionary. The fully qualified name (FQDN) of a BizTalk map can be easily determined as follows:

• Open the BizTalk Administration Console and navigate to the Maps folder within your BizTalk application.
• Double click the map in question.
• Copy the content of the Name label (see the picture below) and paste it in a text editor.
• Append a comma followed by a space (“, “).
• Copy the content of the Assembly label (see the picture below) and paste it in a text editor.

Pretty easy, don’t you think?

XslTransformHelper class

#region Copyright//-------------------------------------------------// Author: Paolo Salvatori// Email: paolos@microsoft.com// History: 2010-01-26 Created//-------------------------------------------------#endregion#region Using Referencesusing System;using System.IO;using System.Text;using System.Collections.Generic;using System.Configuration;using System.Xml;using System.Xml.XPath;using System.Xml.Xsl;using System.Diagnostics;using Microsoft.XLANGs.BaseTypes;using Microsoft.XLANGs.Core;using Microsoft.BizTalk.Streaming;using Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers.Properties;#endregionnamespace Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers{ public class XslTransformHelper { #region Private Constants private const int DefaultBufferSize = 10240; //10 KB private const int DefaultThresholdSize = 1048576; //1 MB private const string DefaultPartName = "Body"; #endregion #region Private Static Fields private static Dictionary<string, TransformBase> mapDictionary; #endregion #region Static Constructor static XslTransformHelper() { mapDictionary = new Dictionary<string, TransformBase>(); } #endregion #region Public Static Methods public static XLANGMessage Transform(XLANGMessage message, string mapFullyQualifiedName, string messageName) { return Transform(message, 0, mapFullyQualifiedName, messageName, DefaultPartName, false, DefaultBufferSize, DefaultThresholdSize); } public static XLANGMessage Transform(XLANGMessage message, string mapFullyQualifiedName, string messageName, bool debug) { return Transform(message, 0, mapFullyQualifiedName, messageName, DefaultPartName, debug, DefaultBufferSize, DefaultThresholdSize); } public static XLANGMessage Transform(XLANGMessage message, int partIndex, string mapFullyQualifiedName, string messageName, string partName, bool debug, int bufferSize, int thresholdSize) { try { using (Stream stream = message[partIndex].RetrieveAs(typeof(Stream)) as Stream) { Stream response = Transform(stream, mapFullyQualifiedName, debug, bufferSize, thresholdSize); CustomBTXMessage customBTXMessage = null; customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext); customBTXMessage.AddPart(string.Empty, partName); customBTXMessage[0].LoadFrom(response); return customBTXMessage.GetMessageWrapperForUserCode(); } } catch (Exception ex) { ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex); TraceHelper.WriteLineIf(debug, null, ex.Message, EventLogEntryType.Error); throw; } finally { if (message != null) { message.Dispose(); } } } public static XLANGMessage Transform(XLANGMessage[] messageArray, int[] partIndexArray, string mapFullyQualifiedName, string messageName, string partName, bool debug, int bufferSize, int thresholdSize) { try { if (messageArray != null && messageArray.Length > 0) { Stream[] streamArray = new Stream[messageArray.Length]; for (int i = 0; i < messageArray.Length; i++) { streamArray[i] = messageArray[i][partIndexArray[i]].RetrieveAs(typeof(Stream)) as Stream; } Stream response = Transform(streamArray, mapFullyQualifiedName, debug, bufferSize, thresholdSize); CustomBTXMessage customBTXMessage = null; customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext); customBTXMessage.AddPart(string.Empty, partName); customBTXMessage[0].LoadFrom(response); return customBTXMessage.GetMessageWrapperForUserCode(); } } catch (Exception ex) { ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex); TraceHelper.WriteLineIf(debug, null, ex.Message, EventLogEntryType.Error); throw; } finally { if (messageArray != null && messageArray.Length > 0) { for (int i = 0; i < messageArray.Length; i++) { if (messageArray[i] != null) { messageArray[i].Dispose(); } } } } return null; } public static Stream Transform(Stream stream, string mapFullyQualifiedName) { return Transform(stream, mapFullyQualifiedName, false, DefaultBufferSize, DefaultThresholdSize); } public static Stream Transform(Stream stream, string mapFullyQualifiedName, bool debug) { return Transform(stream, mapFullyQualifiedName, debug, DefaultBufferSize, DefaultThresholdSize); } public static Stream Transform(Stream stream, string mapFullyQualifiedName, bool debug, int bufferSize, int thresholdSize) { try { TransformBase transformBase = GetTransformBase(mapFullyQualifiedName); if (transformBase != null) { VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize); XPathDocument xpathDocument = new XPathDocument(stream); transformBase.Transform.Transform(xpathDocument, transformBase.TransformArgs, virtualStream); virtualStream.Seek(0, SeekOrigin.Begin); return virtualStream; } } catch (Exception ex) { ExceptionHelper.HandleException(Resources.DynamicTransformsHelper, ex); TraceHelper.WriteLineIf(debug, null, ex.Message, EventLogEntryType.Error); throw; } return null; } public static Stream Transform(Stream[] streamArray, string mapFullyQualifiedName) { return Transform(streamArray, mapFullyQualifiedName, false, DefaultBufferSize, DefaultThresholdSize); } public static Stream Transform(Stream[] streamArray, string mapFullyQualifiedName, bool debug) { return Transform(streamArray, mapFullyQualifiedName, debug, DefaultBufferSize, DefaultThresholdSize); } public static Stream Transform(Stream[] streamArray, string mapFullyQualifiedName, bool debug, int bufferSize, int thresholdSize) { try { TransformBase transformBase = GetTransformBase(mapFullyQualifiedName); if (transformBase != null) { CompositeStream compositeStream = null; try { VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize); compositeStream = new CompositeStream(streamArray); XPathDocument xpathDocument = new XPathDocument(compositeStream); transformBase.Transform.Transform(xpathDocument, transformBase.TransformArgs, virtualStream); virtualStream.Seek(0, SeekOrigin.Begin); return virtualStream; } finally { if (compositeStream != null) { compositeStream.Close(); } } } } catch (Exception ex) { ExceptionHelper.HandleException(Resources.DynamicTransformsHelper, ex); TraceHelper.WriteLineIf(debug, null, ex.Message, EventLogEntryType.Error); throw; } return null; } #endregion #region Private Static Methods private static TransformBase GetTransformBase(string mapFullyQualifiedName) { TransformBase transformBase = null; lock (mapDictionary) { if (!mapDictionary.ContainsKey(mapFullyQualifiedName)) { Type type = Type.GetType(mapFullyQualifiedName); transformBase = Activator.CreateInstance(type) as TransformBase; if (transformBase != null) { mapDictionary[mapFullyQualifiedName] = transformBase; } } else { transformBase = mapDictionary[mapFullyQualifiedName]; } } return transformBase; } #endregion }}

XslCompiledTransformHelper class

#region Copyright//-------------------------------------------------// Author: Paolo Salvatori// Email: paolos@microsoft.com// History: 2010-01-26 Created//-------------------------------------------------#endregion#region Using Referencesusing System;using System.IO;using System.Text;using System.Collections.Generic;using System.Configuration;using System.Xml;using System.Xml.Xsl;using System.Xml.XPath;using System.Diagnostics;using Microsoft.XLANGs.BaseTypes;using Microsoft.XLANGs.Core;using Microsoft.BizTalk.Streaming;using Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers.Properties;#endregionnamespace Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers{ public class XslCompiledTransformHelper { #region Private Constants private const int DefaultBufferSize = 10240; //10 KB private const int DefaultThresholdSize = 1048576; //1 MB private const string DefaultPartName = "Body"; #endregion #region Private Static Fields private static Dictionary<string, MapInfo> mapDictionary; #endregion #region Static Constructor static XslCompiledTransformHelper() { mapDictionary = new Dictionary<string, MapInfo>(); } #endregion #region Public Static Methods public static XLANGMessage Transform(XLANGMessage message, string mapFullyQualifiedName, string messageName) { return Transform(message, 0, mapFullyQualifiedName, messageName, DefaultPartName, false, DefaultBufferSize, DefaultThresholdSize); } public static XLANGMessage Transform(XLANGMessage message, string mapFullyQualifiedName, string messageName, bool debug) { return Transform(message, 0, mapFullyQualifiedName, messageName, DefaultPartName, debug, DefaultBufferSize, DefaultThresholdSize); } public static XLANGMessage Transform(XLANGMessage message, int partIndex, string mapFullyQualifiedName, string messageName, string partName, bool debug, int bufferSize, int thresholdSize) { try { using (Stream stream = message[partIndex].RetrieveAs(typeof(Stream)) as Stream) { Stream response = Transform(stream, mapFullyQualifiedName, debug, bufferSize, thresholdSize); CustomBTXMessage customBTXMessage = null; customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext); customBTXMessage.AddPart(string.Empty, partName); customBTXMessage[0].LoadFrom(response); return customBTXMessage.GetMessageWrapperForUserCode(); } } catch (Exception ex) { ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex); TraceHelper.WriteLineIf(debug, null, ex.Message, EventLogEntryType.Error); throw; } finally { if (message != null) { message.Dispose(); } } } public static XLANGMessage Transform(XLANGMessage[] messageArray, int[] partIndexArray, string mapFullyQualifiedName, string messageName, string partName, bool debug, int bufferSize, int thresholdSize) { try { if (messageArray != null && messageArray.Length > 0) { Stream[] streamArray = new Stream[messageArray.Length]; for (int i = 0; i < messageArray.Length; i++) { streamArray[i] = messageArray[i][partIndexArray[i]].RetrieveAs(typeof(Stream)) as Stream; } Stream response = Transform(streamArray, mapFullyQualifiedName, debug, bufferSize, thresholdSize); CustomBTXMessage customBTXMessage = null; customBTXMessage = new CustomBTXMessage(messageName, Service.RootService.XlangStore.OwningContext); customBTXMessage.AddPart(string.Empty, partName); customBTXMessage[0].LoadFrom(response); return customBTXMessage.GetMessageWrapperForUserCode(); } } catch (Exception ex) { ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex); TraceHelper.WriteLineIf(debug, null, ex.Message, EventLogEntryType.Error); throw; } finally { if (messageArray != null && messageArray.Length > 0) { for (int i = 0; i < messageArray.Length; i++) { if (messageArray[i] != null) { messageArray[i].Dispose(); } } } } return null; } public static Stream Transform(Stream stream, string mapFullyQualifiedName) { return Transform(stream, mapFullyQualifiedName, false, DefaultBufferSize, DefaultThresholdSize); } public static Stream Transform(Stream stream, string mapFullyQualifiedName, bool debug) { return Transform(stream, mapFullyQualifiedName, debug, DefaultBufferSize, DefaultThresholdSize); } public static Stream Transform(Stream stream, string mapFullyQualifiedName, bool debug, int bufferSize, int thresholdSize) { try { MapInfo mapInfo = GetMapInfo(mapFullyQualifiedName, debug); if (mapInfo != null) { XmlTextReader xmlTextReader = null; try { VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize); xmlTextReader = new XmlTextReader(stream); mapInfo.Xsl.Transform(xmlTextReader, mapInfo.Arguments, virtualStream); virtualStream.Seek(0, SeekOrigin.Begin); return virtualStream; } finally { if (xmlTextReader != null) { xmlTextReader.Close(); } } } } catch (Exception ex) { ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex); TraceHelper.WriteLineIf(debug, null, ex.Message, EventLogEntryType.Error); throw; } return null; } public static Stream Transform(Stream[] streamArray, string mapFullyQualifiedName) { return Transform(streamArray, mapFullyQualifiedName, false, DefaultBufferSize, DefaultThresholdSize); } public static Stream Transform(Stream[] streamArray, string mapFullyQualifiedName, bool debug) { return Transform(streamArray, mapFullyQualifiedName, debug, DefaultBufferSize, DefaultThresholdSize); } public static Stream Transform(Stream[] streamArray, string mapFullyQualifiedName, bool debug, int bufferSize, int thresholdSize) { try { MapInfo mapInfo = GetMapInfo(mapFullyQualifiedName, debug); if (mapInfo != null) { CompositeStream compositeStream = null; try { VirtualStream virtualStream = new VirtualStream(bufferSize, thresholdSize); compositeStream = new CompositeStream(streamArray); XmlTextReader reader = new XmlTextReader(compositeStream); mapInfo.Xsl.Transform(reader, mapInfo.Arguments, virtualStream); virtualStream.Seek(0, SeekOrigin.Begin); return virtualStream; } finally { if (compositeStream != null) { compositeStream.Close(); } } } } catch (Exception ex) { ExceptionHelper.HandleException(Resources.XslCompiledTransformHelper, ex); TraceHelper.WriteLineIf(debug, null, ex.Message, EventLogEntryType.Error); throw; } return null; } #endregion #region Private Static Methods private static MapInfo GetMapInfo(string mapFullyQualifiedName, bool debug) { MapInfo mapInfo = null; lock (mapDictionary) { if (!mapDictionary.ContainsKey(mapFullyQualifiedName)) { Type type = Type.GetType(mapFullyQualifiedName); TransformBase transformBase = Activator.CreateInstance(type) as TransformBase; if (transformBase != null) { XslCompiledTransform map = new XslCompiledTransform(debug); using (StringReader stringReader = new StringReader(transformBase.XmlContent)) { XmlTextReader xmlTextReader = null; try { xmlTextReader = new XmlTextReader(stringReader); XsltSettings settings = new XsltSettings(true, true); map.Load(xmlTextReader, settings, new XmlUrlResolver()); mapInfo = new MapInfo(map, transformBase.TransformArgs); mapDictionary[mapFullyQualifiedName] = mapInfo; } finally { if (xmlTextReader != null) { xmlTextReader.Close(); } } } } } else { mapInfo = mapDictionary[mapFullyQualifiedName]; } } return mapInfo; } #endregion } public class MapInfo { #region Private Fields private XslCompiledTransform xsl; private XsltArgumentList arguments; #endregion #region Public Constructors public MapInfo() { this.xsl = null; this.arguments = null; } public MapInfo(XslCompiledTransform xsl, XsltArgumentList arguments) { this.xsl = xsl; this.arguments = arguments; } #endregion #region Public Properties public XslCompiledTransform Xsl { get { return this.xsl; } set { this.xsl = value; } } public XsltArgumentList Arguments { get { return this.arguments; } set { this.arguments = value; } } #endregion }}

Note: Support for embedded scripts is an optional XSLT setting on the XslCompiledTransform class. Script support is disabled by default. Therefore, to enable script support, it’s necessary to create an XsltSettings object with the EnableScript property set to true and pass the object to the Load method. That’s what I did in my code above.

Schemas

This project contains 2 Xml Schemas, CalculatorRequest and CalculatorResponse, which define, respectively, the request and response message and a PropertySchema that defines the Method promoted property. A CalculatorRequest message can contain zero or multiple Operation elements, as shown in the following picture:

CalculatorRequest message

<CalculatorRequest xmlns="https://microsoft.biztalk.cat/10/dynamictransforms/calculatorrequest"> <Method>UnitTest</Method> <Operations> <Operation> <Operator>+</Operator> <Operand1>82</Operand1> <Operand2>18</Operand2> </Operation> <Operation> <Operator>-</Operator> <Operand1>30</Operand1> <Operand2>12</Operand2> </Operation> <Operation> <Operator>*</Operator> <Operand1>25</Operand1> <Operand2>8</Operand2> </Operation> <Operation> <Operator>\</Operator> <Operand1>100</Operand1> <Operand2>25</Operand2> </Operation> </Operations></CalculatorRequest>

 

A CalculatorResponse message contains a Result element for each Operation element within the corresponding CalculatorRequest message, as shown in the following picture:

CalculatorResponse message

<CalculatorResponse xmlns="https://microsoft.biztalk.cat/10/dynamictransforms/calculatorresponse"> <Status>Ok</Status> <Results> <Result> <Value>100</Value> <Error>None</Error> </Result> <Result> <Value>18</Value> <Error>None</Error> </Result> <Result> <Value>200</Value> <Error>None</Error> </Result> <Result> <Value>4</Value> <Error>None</Error> </Result> </Results></CalculatorResponse>

Maps

This project contains the CalculatorRequestToCalculatorResponse map (see the picture below) that transforms an inbound request message into the corresponding response message.

Orchestrations

This project contains the 4 orchestrations.

SingleDynamicTransform Test Case

This flow had been created just to test the XslCompiledTransformHelper class within an orchestration.

The following picture depicts the architecture of the SingleDynamicTransform test case.

Message Flow:

1. A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the IN folder.
2. The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
3. The inbound request starts a new instance of the SingleDynamicTransform. This latter uses a Direct Bound Port and a Filter to receive only the CalculatorRequest messages with the Method promoted property = “SingleDynamicTransform”.
4. The SingleDynamicTransform invokes the Transform static method exposed by the XslCompiledTransformHelper class to apply the CalculatorRequestToCalculatorResponse  map to the inbound CalculatorRequest  message and generate the corresponding CalculatorResponse document.
5. The CalculatorRequestToCalculatorResponse  publishes the CalculatorResponse message to the MessageBox (BizTalkMsgBoxDb).
6. The response message is retrieved by a One-Way FILE Send Port.
7. The response message is written to an OUT folder by the One-Way FILE Send Port.

 

DefaultStaticLoop Test Case

As shown in the picture below, this orchestration receives a CalculatorRequest xml document (80KB)  and executes a loop (1000 iterations) in which it uses a Transform Shape to apply the CalculatorRequestToCalculatorResponse  map to the inbound message. The orchestration does not produce any response message. The code within the StartStepTrace and EndStepTrace Expression Shapes keeps track of the time spent to execute the map at each iteration, while the code contained in the final Trace Expression Shape writes the total elapsed time on the standard output. The objective of this test case is to measure the time spent by the orchestration to apply the map to the inbound document 1000 times using the Transform Shape.

The following picture depicts the architecture of the DefaultStaticLoop test case.

Message Flow:

1. A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the IN folder.
2. The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
3. The inbound request starts a new instance of the DefaultStaticLoop . This latter uses a Direct Bound Port and a Filter to receive only the CalculatorRequest messages with the Method promoted property = “DefaultStaticLoop”.
4. The DefaultStaticLoop executes a loop (1000 iterations) in which it uses a Transform Shape to apply the CalculatorRequestToCalculatorResponse  map to the inbound CalculatorRequest message (80KB).

 

DefaultDynamicLoop Test Case

This component is a variation of the DefaultStaticLoop orchestration. As this latter, it receives a CalculatorRequest xml document (80KB)  and executes a loop (1000 iterations), but it doesn’t use a Transform shape to execute the CalculatorRequestToCalculatorResponse  map against the inbound message, it rather uses a Message Assignment Shape that contain the following code. See How to Use Expressions to Dynamic Transform Messages for more information on this topic. The objective of this test case is to measure the time spent by the orchestration to apply the map to the inbound document 1000 times using the transform statement provided by the XLANG Runtime.

 

startTime = System.DateTime.Now;type = System.Type.GetType("<Map FQDN>");transform(CalculatorResponse) = type(CalculatorRequest);stopTime = System.DateTime.Now;elapsedTime = stopTime.Subtract(startTime);total = total + elapsedTime.TotalMilliseconds;i = i + 1;

As the DefaultStaticLoop, the orchestration does not produce any response. The code within the CreateResponse Shape keeps track of the time spent to execute the map at each iteration, while the code contained in the final Trace Expression Shape writes the total elapsed time on the standard output.

The following picture depicts the architecture of the DefaultDynamicLoop test case.

Message Flow:

1. A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the IN folder.
2. The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
3. The inbound request starts a new instance of the DefaultDynamicLoop . This latter uses a Direct Bound Port and a Filter to receive only the CalculatorRequest messages with the Method promoted property = “DefaultDynamicLoop”.
4. The DefaultDynamicLoop executes a loop (1000 iterations) in which it uses a Message Assignment Shape to execute the CalculatorRequestToCalculatorResponse  map against the inbound CalculatorRequest message (80KB).

CustomDynamicLoop Test Case

As the previous orchestrations, the CustomDynamicLoop receives a CalculatorRequest xml document (80KB)  and executes a loop (1000 iterations). However, instead of using a Transform shape or the Dynamic Transformation mechanism provided by BizTalk to apply the map to the inbound document, it uses an Expression Shape (see the code below) to invoke the Transform method exposed by my XslCompiledTransformHelper component. The objective of this test case is to measure the time spent by the orchestration to apply the map to the inbound document 1000 times using the XslCompiledTransformHelper class.

 

startTime = System.DateTime.Now;CalculatorResponse = Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Helpers.XslCompiledTransformHelper.Transform(CalculatorRequest, "<Map FQDN>");stopTime = System.DateTime.Now;elapsedTime = stopTime.Subtract(startTime);total = total + elapsedTime.TotalMilliseconds;i = i + 1;

As the previous orchestrations, the CustomDynamicLoop does not produce any response. The code within the final Trace Expression Shape writes the total elapsed time on the standard output.

The following picture depicts the architecture of the CustomDynamicLoop test case.

Message Flow:

1. A One-Way FILE Receive Location receives a new CalculatorRequest xml document from the IN folder.
2. The XML disassembler component within the XMLTransmit pipeline promotes the Method element inside the CalculatorRequest xml document. The Message Agent submits the incoming message to the MessageBox (BizTalkMsgBoxDb).
3. The inbound request starts a new instance of the CustomDynamicLoop. This latter uses a Direct Bound Port and a Filter to receive only the CalculatorRequest messages with the Method promoted property = “CustomDynamicLoop”.
4. The CustomDynamicLoop executes a loop (1000 iterations) in which it uses a XslCompiledTransformHelper class to execute the CalculatorRequestToCalculatorResponse  map against the inbound CalculatorRequest message (80KB).

Pipeline Components

This project contains 2 custom pipeline components called, respectively, TransformPipelineComponent and LoopbackPipelineComponent.

TransformPipelineComponent

This component can be used within a Receive or a Send custom pipeline to transform the inbound message using the XslCompiledTransformHelper class. For the sake of brevity, we just report the code of the  Execute method of in the picture below. Note that the if the loopback property exposed by the component equals true, this latter promotes the RouteDirectToTp context property to true. This way, when the TransformPipelineComponent is used by a Receive Pipeline within a Request-Response Receive Location, when the Message Agent posts the transformed message to the MessageBox, this latter is immediately returned as a response to the Receive Location (Loopback pattern).

Execute method

public IBaseMessage Execute(IPipelineContext context, IBaseMessage message){ try { if (componentEnabled) { if (context == null) { throw new ArgumentException("The pipeline context parameter cannot be null."); } if (message != null) { IBaseMessagePart bodyPart = message.BodyPart; Stream inboundStream = bodyPart.GetOriginalDataStream(); Stream outboundStream = XslCompiledTransformHelper.Transform(inboundStream, mapFQDN, traceEnabled, bufferSize, thresholdSize); bodyPart.Data = outboundStream; context.ResourceTracker.AddResource(inboundStream); context.ResourceTracker.AddResource(outboundStream); if (loopback) { message.Context.Promote("RouteDirectToTP", "https://schemas.microsoft.com/BizTalk/2003/system-properties", true); } } } } catch (Exception ex) { ExceptionHelper.HandleException("TransformPipelineComponent", ex); TraceHelper.WriteLineIf(traceEnabled, context, ex.Message, EventLogEntryType.Error); } return message;}

LoopbackPipelineComponent

This component can be used to set the RouteDirectToTp context property to true to implement the Loopback pattern. When used within a Receive Pipeline, the component allows to promote the MessageType property without the need to use an Xml Disassembler. At runtime, the MessageType is mandatory to determine the map to apply to a given message on a Request or Send Port.

Execute method

public IBaseMessage Execute(IPipelineContext context, IBaseMessage message){ try { if (loopback) { message.Context.Promote("RouteDirectToTP", "https://schemas.microsoft.com/BizTalk/2003/system-properties", true); if (messageType != null) { message.Context.Promote("MessageType", "https://schemas.microsoft.com/BizTalk/2003/system-properties", messageType); } } } catch (Exception ex) { ExceptionHelper.HandleException("LoopbackPipelineComponent", ex); TraceHelper.WriteLineIf(traceEnabled, context, ex.Message, EventLogEntryType.Error); } return message;}

Pipelines

This project contains 2 custom pipelines:

• TransformReceivePipeline:this pipeline contains only an instance of the TransformPipelineComponent.
• LoopbackReceivePipeline: this pipeline contains only an instance of the LoopbackPipelineComponent.

Then, I created 2 use cases to compare the performance of the default message transformation provided by BizTalk Messaging Engine and the message transformation accomplished using my XslCompiledTransformHelper class.

TransformStaticallyDefined Test Case

The following picture depicts the architecture of the TransformStaticallyDefined test case.

Message Flow:

1. The DT.TransformStaticallyDefined.WCF-NetTcp.RL WCF-NetTcp Request-Response Receive Location receives a CalculatorRequest xml document submitted running the InvokeStaticMap Unit Test within Visual Studio.
2. The LoopbackReceivePipeline promotes the RouteDirectToTp property to true and the MessageType property. I could have used the Xml Disassembler component within the Receive Pipeline to find and promote the MessageType, but I preferred to specify the MessageType of the inbound message as part of the configuration of the Receive Location (see the picture below). This way I can avoid the overhead introduced by the Xml Disassembler component and measure just the time spent by the Messaging Engine to apply the CalculatorRequestToCalculatorResponse  map statically defined on the Receive Port. Once transformed the CalculatorRequest message into a CalculatorResponse document, the Message Agent posts this latter to the MessageBox.
3. The transformed message is immediately returned to the Receive Location.
4. The response message is returned to the InvokeStaticMap Unit Test.

DT.TransformStaticallyDefined.RP Configuration

The screen below shows that the use of the CalculatorRequestToCalculatorResponse  map has been statically configured on the DT.TransformStaticallyDefined.RP Receive Port.

DT.TransformStaticallyDefined.WCF-NetTcp.RL Configuration

The following picture shows the configuration of the LoopbackReceivePipeline on the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location.

TransformReceivePipeline Test Case

The following picture depicts the architecture of the TransformReceivePipeline test case.

Message Flow:

1. The DT.TransformReceivePipeline.WCF-NetTcp.RL WCF-NetTcp Request-Response Receive Location receives a CalculatorRequest xml document submitted running the InvokeDynamicMap Unit Test within Visual Studio.
2. The TransformPipelineComponent (the following picture shows its configuration) promotes the RouteDirectToTp property to true and transforms the inbound message using the XslCompiledTransformHelper class and the CalculatorRequestToCalculatorResponse  map. Then the Message Agent posts the transformed message to the MessageBox.
3. The transformed message is immediately returned to the Receive Location.
4. The response message is returned to the InvokeDynamicMap Unit Test.

 DT.TransformReceivePipeline.WCF-NetTcp.RL Configuration

The following picture shows the configuration of the TransformReceivePipeline on the DT.TransformReceivePipeline.WCF-NetTcp.RL Receive Location.

UnitAndLoadTests

Finally, I created a Test Project called UnitAndLoadTests that contains a small set of unit and load tests described below:

• TestXslTransformHelper: this unit test can be used to measure the time spent to execute loops transformations using the XslTransformHelper class, where loops is defined in the configuration file. The following picture reports the code of the TestXslTransformHelper unit test.

TestXslTransformHelper method

[TestMethod]public void TestXslTransformHelper(){ Assert.AreNotEqual<string>(null, inputFile, "The inpuFile key in the configuration file cannot be null."); Assert.AreNotEqual<string>(String.Empty, inputFile, "The inpuFile key in the configuration file cannot be empty."); Assert.AreEqual<bool>(true, File.Exists(inputFile), string.Format(CultureInfo.CurrentCulture, "The {0} file does not exist.", inputFile)); Assert.AreNotEqual<string>(null, mapFullyQualifiedName, "The mapFullyQualifiedName key in the configuration file cannot be null."); Assert.AreNotEqual<string>(String.Empty, mapFullyQualifiedName, "The mapFullyQualifiedName key in the configuration file cannot be empty."); if (traceResponses) { Assert.AreEqual<bool>(true, Directory.Exists(outputFolder), string.Format(CultureInfo.CurrentCulture, "The {0} folder does not exist.", outputFolder)); } Type type = null; try { type = Type.GetType(mapFullyQualifiedName); } catch (Exception ex) { Assert.Fail(ex.Message); } MemoryStream stream = null; string message; using (StreamReader reader = new StreamReader(File.Open(inputFile, FileMode.Open, FileAccess.Read, FileShare.Read))) { message = reader.ReadToEnd(); } byte[] buffer = Encoding.UTF8.GetBytes(message); Stopwatch stopwatch = new Stopwatch(); Stream output = null; TestContext.BeginTimer("TestXslTransformHelper"); for (int i = 0; i < loops; i++) { stream = new MemoryStream(buffer); stopwatch.Start(); output = XslTransformHelper.Transform(stream, mapFullyQualifiedName); stopwatch.Stop(); if (output != null && traceResponses) { using (StreamReader reader = new StreamReader(output)) { message = reader.ReadToEnd(); } using (StreamWriter writer = new StreamWriter(File.OpenWrite( Path.Combine(outputFolder, string.Format(CultureInfo.CurrentCulture, "{{{0}}}.xml", Guid.NewGuid().ToString()))))) { writer.Write(message); writer.Flush(); } } } TestContext.EndTimer("TestXslTransformHelper"); Trace.WriteLine(String.Format(CultureInfo.CurrentCulture, "[TestXslTransformHelper] Loops: {0} Elapsed Time (milliseconds): {1}", loops, stopwatch.ElapsedMilliseconds));}

• TestXslCompiledTransformHelper: this unit test can be used to measure the time spent to execute loops transformations using the XslCompiledTransformHelper class, where loops is defined in the configuration file. For the sake of brevity, I omitted to include the code of the TestXslCompiledTransformHelper unit test as this latter is very similar to one of the previous unit test.
• InvokeStaticMap: this unit test can be used to send a single CalculatorRequest xml document to the DT.TransformStaticallyDefined.WCF-NetTcp.RL  Receive Location used by the TransformStaticallyDefined Test case.
• InvokeDynamicMap: this unit test can be used to send a single CalculatorRequest xml document to the DT.TransformReceivePipeline.WCF-NetTcp.RL Receive Location used by the TransformReceivePipeline Test case.
• StaticMapLoadTest: this load test is based on the InvokeStaticMap unit test and can be used to generate traffic against the TransformStaticallyDefined Use Case.
• DynamicMapLoadTest: this load test is based on the InvokeDynamicMap unit test and can be used to generate traffic against the TransformReceivePipeline Use Case.

All these tests share the same configuration contained in the App.config configuration file. In particular this latter contains the following information:

• The WCF Endpoint used to invoke the DT.TransformStaticallyDefined.WCF-NetTcp.RL and DT.TransformReceivePipeline.WCF-NetTcp.RL Receive Locations.
• The appSettings section defines multiple keys that allows to control the runtime behavior of unit and load tests:
  • mapFullyQualifiedName: contains the name of the map used by TestXslTransformHelper  and TestXslCompiledTransformHelper unit tests.
  • inputFile: defines the path of the inbound document used by all unit tests (TestXslTransformHelper , TestXslCompiledTransformHelper, InvokeStaticMap, InvokeDynamicMap).
  • outputFolder: indicates the path where to save response messages.
  • traceResponses: indicates whether to save response messages.
  • loops: allows to control the number of loop iterations performed by the TestXslTransformHelper  and TestXslCompiledTransformHelper unit tests.

For the sake of completeness, I include below the App.config I used for my tests.

App.config file

<?xml version="1.0" encoding="utf-8" ?><configuration> <system.serviceModel> <!-- Bindings used by client endpoints --> <bindings> <netTcpBinding> <binding name="netTcpBinding" closeTimeout="01:10:00" openTimeout="01:10:00" receiveTimeout="01:10:00" sendTimeout="01:10:00" transactionFlow="false" transferMode="Buffered" transactionProtocol="OleTransactions" hostNameComparisonMode="StrongWildcard" listenBacklog="100" maxBufferPoolSize="1048576" maxBufferSize="10485760" maxConnections="200" maxReceivedMessageSize="10485760"> <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384" maxBytesPerRead="4096" maxNameTableCharCount="16384" /> <reliableSession ordered="true" inactivityTimeout="00:10:00" enabled="false" /> <security mode="None"> <transport clientCredentialType="Windows" protectionLevel="EncryptAndSign" /> <message clientCredentialType="Windows" /> </security> </binding> </netTcpBinding> </bindings> <client> <!-- Client endpoints used by client excahnge messages with the WCF Receive Locations --> <endpoint address="net.tcp://localhost:3816/dynamictransforms" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="StaticMapEndpoint" /> <endpoint address="net.tcp://localhost:3817/dynamictransforms" binding="netTcpBinding" bindingConfiguration="netTcpBinding" contract="System.ServiceModel.Channels.IRequestChannel" name="DynamicMapEndpoint" /> </client> </system.serviceModel> <appSettings> <add key="mapFullyQualifiedName" value="Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Maps.CalculatorRequestToCalculatorResponse, Microsoft.BizTalk.CAT.Samples.DynamicTransforms.Maps, Version=1.0.0.0, Culture=neutral, PublicKeyToken=8c83cae5bc47edb0"/> <add key="inputFile" value="C:\Projects\DynamicTransforms\Test\UnitTest.xml"/> <add key="outputFolder" value="C:\Projects\DynamicTransforms\Test\Out"/> <add key="traceResponses" value="false"/> <add key="loops" value="1000"/> </appSettings></configuration>

 

Results

Let’s start running some of the test cases and unit tests I created. Take into account that the unit tests and that you can find in the code attached to the article are parametric and they can be executed using any xml message and map. Therefore, I strongly encourage you to repeat my tests using your own messages and maps.

TestXslTransformHelper vs TestXslCompiledTransformHelper

I configured both the unit tests to execute the CalculatorRequestToCalculatorResponse  map against the the UnitTest.xml file (80KB) 1000 times. Each test method uses an instance of the Stopwatch class to measure the time spent to executing all calls and finally traces a message containing the total elapsed time. The screens below were taken within Visual Studio at the end of the 2 tests.

TestXslTransformHelper

TestXslCompiledTransformHelper

The difference in terms of performance between the 2 unit tests is simply astonishing:

• TestXslTransformHelper Unit Test: Total Elapsed Time = ~144 seconds, Average Elapsed Time/Transformation = ~144 milliseconds
• TestXslCompiledTransformHelper Unit Test: Total Elapsed Time = ~3.5 seconds, Average Elapsed Time/Transformation = ~3.5 milliseconds

Obviously, I conducted several test runs and they all confirmed that the XslCompiledTransformHelper is class incredibly faster than the XslTransformHelper class and this clearly demonstrates that the XslCompiledTransform class is absolutely much better than the XslTransform class in a Load once, Cache and Transform many times” scenario.

DefaultStaticLoop Test Case vs DefaultDynamicLoop Test Case vs CustomDynamicLoop Test Case

All the orchestrations used in the 3 test cases share the same structure and implement the same behavior using a different technique:

• DefaultStaticLoop orchestration: uses a Transform shape to execute the CalculatorRequestToCalculatorResponse  against the inbound document.
• DefaultDynamicLoop orchestration: uses the transform method within a Message Assignment Shape to accomplish the same task.
• CustomDynamicLoop orchestration: uses the XslCompiledTransformHelper.Transform method to invoke the map against against the request message.

Each orchestration contains a loop that executes the message transformation exactly 1000 times and finally reports the total elapsed time. For the test I created 3 separate xml files (they can be found in the Test folder), one for each orchestration. As I explained in the first part of the article, each orchestration receives the request message through a Direct Bound Port. In particular, the  following Filter Expression has been defined on the Activate Receive Shape of each orchestration:

• https://microsoft.biztalk.cat/10/dynamictransforms/propertyschema.Method == <OrchestrationName>

Therefore, the following files are identical:

• DefaultStaticLoop.xml
• DefaultDynamicLoop.xml
• CustomDynamicLoop.xml

with the exception of the Method element that contains the name of the related orchestration. To execute each test case is sufficient to copy the corresponding file to the Test\IN folder: the DT.FILE.RL FILE Receive Location will than receive the message and activate the intended test case. I used DebugView to keep track of the elapsed time reported by each of the test cases:

Results are quite eloquent and don’t give room to doubts:

• DefaultStaticLoop Test Case: Total Elapsed Time = ~57.3 seconds, Average Elapsed Time/Transformation = ~57 milliseconds
• DefaultDynamicLoop Test Case: Total Elapsed Time = ~56.2 seconds, Average Elapsed Time/Transformation = ~56 milliseconds
• CustomDynamicLoop Test Case: Total Elapsed Time = ~3.6 seconds, Average Elapsed Time/Transformation = ~3.6 milliseconds

Once again, I conducted several test runs to confirm the results obtained and reported above. These latter clearly demonstrated that the XslCompiledTransformHelper class is an order of magnitude faster than the default mechanisms provided by BizTalk for transforming messages.

TransformStaticallyDefined vs TransformReceivePipeline

The objective of this test is to compare the performance of the following test cases:

• TransformStaticallyDefined Test Case: as explained in the first part of the article, the inbound CalculatorRequest message is transformed using the CalculatorRequestToCalculatorResponse  map declaratively configured on the DT.TransformStaticallyDefined.RP Request Port. Once posted to the MessageBox, the CalculatorResponse transformed message is immediately returned to the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location (Loopback pattern).
• TransformReceivePipeline Test Case: the inbound CalculatorRequest message is transformed by the TransformReceivePipeline  hosted by the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location. In particular, the TransformPipelineComponent invokes the XslCompiledTransformHelper.Transform static method to apply the CalculatorRequestToCalculatorResponse map to the inbound xml document. The FQDN of the map is declaratively specified in the pipeline configuration. Once posted to the MessageBox, the CalculatorResponse transformed message is immediately returned to the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location (Loopback pattern).

To generate traffic against the 2 test cases and measure performance I used the following Load Tests defined in the UnitAndLoadTests Test Project:

• StaticMapLoadTest: this load test is based on the InvokeStaticMap unit test and can be used to generate traffic against the TransformStaticallyDefined Use Case. The test is configured to send 1000 CalculatorRequest  messages to the DT.TransformStaticallyDefined.WCF-NetTcp.RL  Receive Location using 25 different worker threads.
• DynamicMapLoadTest: this load test is based on the InvokeDynamicMap unit test and can be used to generate traffic against the TransformReceivePipeline Use Case. The test is configured to send 1000 CalculatorRequest  messages to the DT.TransformStaticallyDefined.WCF-NetTcp.RL Receive Location using 25 different worker threads.

In particular, as shown in the picture below, I created a custom Counter Set called BizTalk composed of the following performance counters:

• Inbound Latency (sec): measures the average latency in milliseconds from when the Messaging Engine receives a document from the adapter until the time it is published to Message Box.
• Request-Response Latency (sec): measures the average latency in milliseconds from when the Messaging Engine receives a request document from the adapter until the time a response document is given back to the adapter.

Specifically, the average latency measured by the Inbound Latency (sec) counter includes the time spent for transforming the message in both use cases. Obviously it counts also the time spent running other activities like posting the message to the MessageBox, but still it represents a good mechanism to measure to compare the time spent by the 2 test cases for transforming the inbound message.

I conducted several test runs to confirm results obtained. The screens below were taken, respectively, at the end of StaticMapLoadTest and DynamicMapLoadTest:

StaticMapLoadTest Graphs & Summary

 

DynamicMapLoadTest Graphs & Summary

The following table reports for convenience the results highlighted in the screens above:

Test Case	Inbound Latency (sec)	Request Response Latency (sec)	Avg Test Time (sec)	Tests/sec (Throughput)	Duration (sec)	% CPU Time
StaticMapLoadTest	0.41	0.90	2.55	8.77	114	65.2
DynamicMapLoadTest	0.11	0.49	1.29	17.6	56	43

 

The difference in terms of latency and throughput between the 2 test cases is quite dramatic and this clearly confirms once again that the XslCompiledTransform  class is much faster than the XslTransform  class natively used by BizTalk. In our case, the adoption of the custom XslCompiledTransformHelper  class  allowed to double the throughput and halve the latency. Obviously, the performance gain can vary from case to case as it depends on many factors (inbound message size, map complexity, etc.), but it’s quite evident that the overall performance of a BizTalk application that makes an extensive use of message transformations can greatly be improved using a helper component like the XslCompiledTransformHelper  class  that exploits the XslCompiledTransform  class to compile, invoke and cache maps for later calls.

Conclusions

As I said in the first part of the article, I started to work with the product group to see how best to take advantage of the XslCompiledTransform  class in the next version of BizTalk. Nevertheless, you can immediately exploits this class in your custom components to boost the execution of you message transformations. Therefore, I encourage you to download my code here and repeat the tests described in this article using your own messages and maps.

Follow-Up

I wrote another article on this subject and extended my code to support multi-source-document-maps. You can find my post here.

Code

Here you can download the code. Any feedback is highly appreciated. ;-)