BizTalk Server: How to Extract Email Attachments By Pipeline

Article
1/17/2024

Introduction

In this article I will demonstrate step by step instructions of how to extract email attachments from POP3 adapter by using custom receive pipeline.

Problem Statement

You can have some customers that need to integrate with your integration middle-ware using POP3. So, They will send emails with multiple attachments and these attachments represent the messages that you need to process.

Solution

There are 2 BizTalk solutions that we can implement either in Message Level or Orchestration level. As performance perspective, a message level is much better because orchestration level has much persistence points which can reduce the performance of our integration solution.
If you would like to see the orchestration solution you can check this link.

To process email attachments through message level, then we need to process them using pipeline.
So, we need to implement custom receive pipeline and add it to disassemble part of receive pipeline
For now let's start our walk-though demonstration.

Create Custom Receive Pipeline

Create a new solution as shown in figure 1

Figure 1- Creating a new Solution
Create a new class library projects as shown in figure 2

Figure 2- Creating a new Class library project
Remove Class1.cs file
Create a new class as shown in figure 3

Figure 3- Creating a new class
Add Microsoft.BizTalk.Pipeline.dll from this path %BTSINSTALLPATH%
Add Microsoft.BizTalk.Pipeline.Components.dll as a reference from this path %BTSINSTALLPATH%\Pipeline Components

Implement the IBaseComponent, IComponentUI interfaces as shown in figure 4

namespace TechNetWiki.AttachmentsExtractor.PipelineComponent
{
    using System;
    using System.IO;
    using System.Text;
    using System.Reflection;
    using System.ComponentModel;
    using Microsoft.BizTalk.Message.Interop;
    using Microsoft.BizTalk.Component.Interop;
    using System.Runtime.InteropServices;
  
    [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
    [ComponentCategory(CategoryTypes.CATID_DisassemblingParser)]
    [Guid("01F04E58-7AD8-42EF-A3CB-D939B960E8F9")]
    public class  AttachmentsExtractor : IBaseComponent, IComponentUI
    {
        #region IBaseComponent Members
        /// <summary>
        /// Gets Description of the component
        /// </summary>  
        public string  Description
        {
            get
            {
                return "Email Attachments Extractor";
            }
        }
  
        /// <summary>
        /// Gets Name of the component
        /// </summary>  
        public string  Name
        {
            get
            {
                return "EmailAttachmentsExtractor";
            }
        }
  
        /// <summary>
        /// Gets Version of the component
        /// </summary>
        public string  Version
        {
            get
            {
                return "1.0";
            }
        }
  
        #endregion
  
        #region IComponentUI members
        /// <summary>
        /// Component icon to use in BizTalk Editor
        /// </summary>     
        public IntPtr Icon
        {
            get
            {
                return System.IntPtr.Zero;
            }
        }
  
        /// <summary>
        /// The Validate method is called by the BizTalk Editor during the build
        /// of a BizTalk project.
        /// </summary>
        /// <param name="obj">An Object containing the configuration properties.</param>
        /// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns>
        public System.Collections.IEnumerator Validate(object obj)
        {
            return null;
        }
        #endregion
    }
}

Figure 4-Implementing IBaseComponent, IComponentUI

Implement IDisassemblerComponent interface which is mandatory to implement GetNext and Disassemble methods
- GetNext method returns messages resulting from the disassemble method execution
- Disassemble method used to split or process the incoming message document

When we receive an email we will receive it as a message associated with multi-parts messages so In Disassemble method we need to loop in each Part of the message and create a new message.We will will start looping from index 1 because index zero is the body content which we are not interested as shown in figure 5

#region IDisassemblerComponent members
 
        /// <summary>
        /// Returns messages resulting from the disassemble method execution
        /// </summary>
        /// <param name="pc">the pipeline context</param>
        /// <returns></returns>
        public IBaseMessage
            GetNext(IPipelineContext pc)
        {
            // get the next message from the Queue and return it
            IBaseMessage msg = null;
            if ((_msgs.Count > 0))
            {
                msg = ((IBaseMessage)(_msgs.Dequeue()));
            }
            return msg;
        }
 
        /// <summary>
        /// called by the messaging engine when a new message arrives
        /// </summary>
        /// <param name="pc">the pipeline context</param>
        /// <param name="inmsg">the actual message</param>
        public void  Disassemble(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
        {
            var partName = string.Empty;
            // we start from index 1 because index zero contain the body of the message
            // which we are not interested
            for (int i = 1; i < inmsg.PartCount; i++)
            {
                IBaseMessagePart currentPart = inmsg.GetPartByIndex(i, out  partName);
                Stream currentPartStream = currentPart.GetOriginalDataStream();
                var ms = new  MemoryStream();
                IBaseMessage outMsg;
                outMsg = pc.GetMessageFactory().CreateMessage();
                outMsg.AddPart("Body", pc.GetMessageFactory().CreateMessagePart(), true);
                outMsg.Context = inmsg.Context;
                currentPartStream.CopyTo(ms);
                string attachmentContent = Encoding.UTF8.GetString(ms.ToArray());
                MemoryStream attachmentStream = new  System.IO.MemoryStream(
                System.Text.Encoding.UTF8.GetBytes(attachmentContent));
                outMsg.GetPart("Body").Data = attachmentStream;
                outMsg.Context.Write("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", partName);
                _msgs.Enqueue(outMsg);
            }
        }
 
        #endregion

Figure 5- Implementing IDisassemblerComponent interface

the Full code of custom pipeline as show in figure 6
**
**

namespace TechNetWiki.AttachmentsExtractor.PipelineComponent
{
    using System;
    using System.IO;
    using System.Text;
    using System.Reflection;
    using System.ComponentModel;
    using Microsoft.BizTalk.Message.Interop;
    using Microsoft.BizTalk.Component.Interop;
    using System.Runtime.InteropServices;
 
    [ComponentCategory(CategoryTypes.CATID_PipelineComponent)]
    [ComponentCategory(CategoryTypes.CATID_DisassemblingParser)]
    [Guid("01F04E58-7AD8-42EF-A3CB-D939B960E8F9")]
    public class  AttachmentsExtractor : IBaseComponent, IComponentUI, IDisassemblerComponent
    {
        #region IBaseComponent Members
        /// <summary>
        /// Gets Description of the component
        /// </summary>   
        public string  Description
        {
            get
            {
                return "Email Attachments Extractor";
            }
        }
 
        /// <summary>
        /// Gets Name of the component
        /// </summary>   
        public string  Name
        {
            get
            {
                return "EmailAttachmentsExtractor";
            }
        }
 
        /// <summary>
        /// Gets Version of the component
        /// </summary>
        public string  Version
        {
            get
            {
                return "1.0";
            }
        }
 
        #endregion
 
        #region IComponentUI members
        /// <summary>
        /// Component icon to use in BizTalk Editor
        /// </summary>      
        public IntPtr Icon
        {
            get
            {
                return System.IntPtr.Zero;
            }
        }
 
        /// <summary>
        /// The Validate method is called by the BizTalk Editor during the build 
        /// of a BizTalk project.
        /// </summary>
        /// <param name="obj">An Object containing the configuration properties.</param>
        /// <returns>The IEnumerator enables the caller to enumerate through a collection of strings containing error messages. These error messages appear as compiler error messages. To report successful property validation, the method should return an empty enumerator.</returns>
        public System.Collections.IEnumerator Validate(object obj)
        {
            return null;
        }
        #endregion
 
        /// <summary>
        /// this variable will contain any message generated by the Disassemble method
        /// </summary>
        private System.Collections.Queue _msgs = new System.Collections.Queue();
 
        #region IDisassemblerComponent members
 
        /// <summary>
        /// Returns messages resulting from the disassemble method execution
        /// </summary>
        /// <param name="pc">the pipeline context</param>
        /// <returns></returns>
        public IBaseMessage
            GetNext(IPipelineContext pc)
        {
            // get the next message from the Queue and return it
            IBaseMessage msg = null;
            if ((_msgs.Count > 0))
            {
                msg = ((IBaseMessage)(_msgs.Dequeue()));
            }
            return msg;
        }
 
        /// <summary>
        /// called by the messaging engine when a new message arrives
        /// </summary>
        /// <param name="pc">the pipeline context</param>
        /// <param name="inmsg">the actual message</param>
        public void  Disassemble(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
        {
            var partName = string.Empty;
            // we start from index 1 because index zero contains the body of the message
            // which we are not interested
            for (int i = 1; i < inmsg.PartCount; i++)
            {
                IBaseMessagePart currentPart = inmsg.GetPartByIndex(i, out  partName);
                Stream currentPartStream = currentPart.GetOriginalDataStream();
                var ms = new  MemoryStream();
                IBaseMessage outMsg;
                outMsg = pc.GetMessageFactory().CreateMessage();
                outMsg.AddPart("Body", pc.GetMessageFactory().CreateMessagePart(), true);
                outMsg.Context = inmsg.Context;
                currentPartStream.CopyTo(ms);
                string attachmentContent = Encoding.UTF8.GetString(ms.ToArray());
                MemoryStream attachmentStream = new  System.IO.MemoryStream(
                System.Text.Encoding.UTF8.GetBytes(attachmentContent));
                outMsg.GetPart("Body").Data = attachmentStream;
                outMsg.Context.Write("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", partName);
                _msgs.Enqueue(outMsg);
            }
        }
 
        #endregion
    }
}

Figure 6- Custom pipeline full code

Build the project then copy the generated TechNetWiki.AttachmentsExtractor.PipelineComponent.dll to %BTSINSTALLPATH%\Pipeline Components

Create Receive Pipeline

Create a new BizTalk Project by right click solution and select new project then select Empty BizTalk Server Project and name it TechNetWiki.AttachmentsExtractor.Pipelines as shown in figure 7

Figure 7- Create Pipeline BizTalk Project
Right click TechNetWiki.AttachmentsExtractor.Pipelines project -> Select Add-> New Item -> Select Receive Pipeline-> Name it AttachmentsExtractorPipeline as shown in figure 8

Figure 8- Create a new receive pipeline
Right click to ToolBox window **->**select Choose Items... -> select BizTalk Pipeline Components -> check EmailAttachmentsExtractor as shown in figure 9

Figure 9- Add pipeline component to ToolBox
Drag drop EmailAttachmentsExtractor to Disassemble part as shown in figure 10

Figure 10- Setting pipeline component in Disassemble part
Right click project TechNetWiki.AttachmentsExtractor.Pipelines -> select Properties -> select Sign tab-> check Sign the assembly -> choose a strong name key file -> select New and rename it to key.snk -> select Deployment tab -> set Application Name to TechNetWiki.AttachmentsExtractor
Right click TechNetWiki.AttachmentsExtractor.Pipelines -> select Deploy

Configure POP3 Adapter

Open BizTalk Server Administration Console
Right click TechNetWiki.AttachmentsExtractor - Receive Port -> select New -> One-way Receive Port... -> change name to Rcv_EmailTest -> select Receive Location tab -> change name to RcvLoc_EmailTest
Select POP3 adapter-> click Configure button -> set POP3 adapter configure as shown in figure 11 In this demo I am using my hotmail account to configure POP3 adapter

Figure 11 - Configuring POP3 adapter
Create a new send port and name it Snd_EmailTest -> select File Adapter and configure it as shown in figure 12

figure 12- Creating send port
Select Filters tab then set filter to our receive port name as shown in figure 13

Figure 13- Setting filter

Test the Solution

Send email with sample attachment message to the configured email that we configured on POP3 adapter which is my demo YouUserName@hotmail.com
Check send port location

Sample Code

You can find the sample source code in the following link

Conclusion

In BizTalk Server there are two ways to extract email attachments either using custom receive pipeline or orchestration

In this article, I walked-through in step by step how to create a custom receive pipeline to extract email attachments.

Share via