A Multi-Type Flat File Assembler for Multi-Part Messages

Article
1/17/2024

The standard Flat File Assembler pipeline component can assemble multi-part messages, but requires that all parts have the same type (schema). This article describes a custom pipeline component that can assemble a multi-part message where each part have its own type (schema). A second feature of the described component is that it can select which message parts to process.
This article is a slight rework of my original blog article A Conditional Flat File Assembler in BizTalk for Multi-Part Messages. The reader should have familiarity with creating custom pipeline components in BizTalk solutions.

Problem Description

The recurring case of creating a mail with a dynamic mail body (with data from the message), and one or more attachments with flat file(s) is generally considered as a case with the “dirty” remedy of running pipeline(s) in an orchestration and then building the multi-part message. The send port then uses the pass-through pipeline.
This article describes a cleaner solution, without cluttering the orchestration logic with lots of transport details.

Solution

The solution consists of an orchestration, still the only way to construct a multi-part message, and a custom pipeline component in your send pipeline. The orchestration implements the business requirements of creating and mapping the messages (such as the mail body part and mail attachment parts), and the pipeline component does the transport details of assembling the flat files. The difference here is that the pipeline component can handle all or only some of the message parts, where each message part can have its own schema.

Orchestration Logic

Create and map your message parts as you would normally do. Then, when you compose each sub-message into its multi-part container message, in the message assignment shape, you also give it an appropriate MIME type, because the custom pipeline component needs to know which message parts it should run through a flat file assembler. Also, it’s beneficial for your e-mail (or, rather, your e-mail’s receiver) to have the correct MIME types:

msgMailWithAttachment.Bodypart = msgBodypart;
msgMailWithAttachment.Attachment = msgAttachment;
msgMailWithAttachment(*) = msgNotifyStatisticsIntrastat(*);
msgMailWithAttachment.Bodypart(Microsoft.XLANGs.BaseTypes.ContentType) = "text/html";
msgMailWithAttachment.Attachment(Microsoft.XLANGs.BaseTypes.ContentType) = "application/octet-stream";

The Custom Pipeline Component

The basics behind the component is simple: For each message part, if the MIME type is the correct one, run it through a dedicated flat file assembler. However, there are two tricks needed to lure the standard flat file assembler component into doing its business for us:

AddDocument() Method

Creating The Document to Add

The first trick is to create a message and a message context, including to copy the original message’s context, but not all values. The CreateNewContextFrom()function copies context properties except those filtered out by the function IsExcludedProperty():

private IBaseMessageContext CreateNewContextFrom(IBaseMessageFactory usingFactory, IBaseMessageContext msgContextSource)
{
    IBaseMessageContext newContext = usingFactory.CreateMessageContext();
    for (int i = 0; i < msgContextSource.CountProperties; i++)
    {
        string propName, propNamespace;
        object propValue;
        propValue = msgContextSource.ReadAt(i, out propName, out propNamespace);
        if (!IsExcludedProperty(propName, propNamespace))
        {
            newContext.Write(propName, propNamespace, propValue);
            if (msgContextSource.IsPromoted(propName, propNamespace))
            {
                newContext.Promote(propName, propNamespace, propValue);
            }
        }
    }
    return newContext;
}

Here’s IsExcludedProperty():

private bool IsExcludedProperty(string propName, string propNamespace)
{
    return
        (
          (propName == "MessageType" && propNamespace == "http://schemas.microsoft.com/BizTalk/2003/system-properties")
        || (propName == "DocumentSpecName" && propNamespace == "http://schemas.microsoft.com/BizTalk/2003/xmlnorm-properties")
        );
}

The trick is completed by setting the message type property ourselves to its correct value, by peeking a little into the message stream. This handy function, GetDocType(), is already written for us in the Microsoft.BizTalk.Streaming.Utils class:

private string GetMessageTypeFromMessageStream(IBaseMessagePart msgPart)
{
    Stream originalDataStream = msgPart.GetOriginalDataStream();
    MarkableForwardOnlyEventingReadStream markableForwardOnlyEventingReadStream = originalDataStream as MarkableForwardOnlyEventingReadStream;
    if (markableForwardOnlyEventingReadStream == null)
    {
        markableForwardOnlyEventingReadStream = new MarkableForwardOnlyEventingReadStream(originalDataStream);
        msgPart.Data = markableForwardOnlyEventingReadStream;
    }
    return Utils.GetDocType(markableForwardOnlyEventingReadStream);
}

Creating The Flat File Assembler Instances

The second trick is how to create the flat file assembler component instances. Since the Flat File Assembler can only handle one type/schema, the component must create one instance of a flat file assembler for each message part to be assembled. This is the key to have an arbitrary flat file schema for each message part.

The component creates a list of MessagePartAssemblers:

private struct  MessagePartAssembler
{
    public string  messageType;
    public string  partName;
    public Microsoft.BizTalk.Component.FFAsmComp instance;
};
private List<MessagePartAssembler> flatfileAssemblerList;

so the central part of the AddDocument() method becomes:

if (listContentType.Contains(partMessageType))
{
    // Create an assembler for this message part:
    MessagePartAssembler ffa = new  MessagePartAssembler();
    ffa.instance = new  Microsoft.BizTalk.Component.FFAsmComp();
 
    // Create a new message for this flat file assembler to assemble/convert to flat file:
    IBaseMessage ffaInput = messageFactory.CreateMessage();
    ffaInput.AddPart(partName, msgPart, true); // Make this part a body part in the new message
    ffaInput.Context = CreateNewContextFrom(messageFactory, pInMsg.Context); // copy all context properties (with some exceptions)
    // Get the correct MessageType and promote:
    string docType = GetMessageTypeFromMessageStream(msgPart);
    ffaInput.Context.Write(propertyNameMessageType, propertyNamespaceMessageType, docType);
    ffaInput.Context.Promote(propertyNameMessageType, propertyNamespaceMessageType, docType);
 
    ffa.messageType = docType;
    ffa.partName = partName;
    ffa.instance.AddDocument(pContext, ffaInput); // Add the document to this assembler's documents
    this.flatfileAssemblerList.Add(ffa);
}
// Remove those parts who shall be flat file parts:
foreach (MessagePartAssembler ffa in flatfileAssemblerList)
{
    pInMsg.RemovePart(ffa.partName);
}

Note that at the end, the message part to be replaced by its flat file transformed message part, is deleted. Warning: Removing the body part causes the host instance to crash (literally, in MSVCR80.dll) and when it restarts it tries to pick up the message again, which causes it to crash again, and the cycle repeats. Make sure you have a body part which not will be removed for transformation.

MIME Content Type Checking

Almost done with AddDocument() method, all that is left is to know which message parts to process. Recall that the message parts' MIME content type was set in the orchestration. The MIME content types the component shall match is stored as a pipeline component parameter. To be able to specify several MIME types to match, the parameter is stored as a comma-separated string, which is then split into a list.
Here’s the body of that code, and the last line is the if clause from the snippet above:

// Create a nice list of the message types/content types we must match:
List<string> listContentType = SplitAndClean(this.contentTypes);
 
if (listContentType != null && listContentType.Count > 0)
{
    IBaseMessageFactory messageFactory = pContext.GetMessageFactory();
 
    for (int i = 0; i < pInMsg.PartCount; i++)
    {
        IBaseMessagePart msgPart = pInMsg.GetPartByIndex(i, out  partName);
        // Get ContentType, exists in two namespaces where btsPartContext has precedence:
        partMessageType = (string)msgPart.PartProperties.Read("ContentType", "http://schemas.microsoft.com/BizTalk/2003/btsPartContext");
        if (String.IsNullOrEmpty(partMessageType)) partMessageType = (string)msgPart.PartProperties.Read("ContentType", "http://schemas.microsoft.com/BizTalk/2003/messageagent-properties");
 
        if (listContentType.Contains(partMessageType))

That’s about it for the AddDocument() method. The helper function SplitAndClean() is simple, and does a little trimming on the strings:

private List<string> SplitAndClean(string messageTypes)
{
    List<string> listMessageType = null;
    if (!String.IsNullOrEmpty(messageTypes))
    {
        listMessageType = new  List(this.contentTypes.Split(new string[] { "," }, StringSplitOptions.RemoveEmptyEntries));
        for (int i = 0; i < listMessageType.Count; i++)
        {
            listMessageType[i] = listMessageType[i].Trim();
        }
    }
    return listMessageType;
}

Assemble() Method

The Assemble() method is simple, in contrast: Just loop through the list of assemblers, call each assembler’s Assemble() and add the returned document to the output message as a new message part:

foreach (MessagePartAssembler ffa in this.flatfileAssemblerList)
{
    IBaseMessage flatfileMsg = ffa.instance.Assemble(pContext);
    // Add back the flat file part(s) as part(s) in the original message:
    string partName;
    for (int i = 0; i < flatfileMsg.PartCount; i++)
    {
        IBaseMessagePart flatfilePart = flatfileMsg.GetPartByIndex(i, out  partName);
        this.allMessages[0].AddPart(partName, flatfilePart, false);
    }
    return this.allMessages[0];
}

Conclusion

Using this method of instantiating Flat File Assembler components in a custom pipeline component, it should be possible to run any kind of assembler component conditionally and each with different message schemas.

Share via