Поделиться через

FTP Adapter Enhancements in BizTalk Server 2010 : Video Tutorial

  • Статья

I have just finished creating and uploading a video that demonstrates the new features introduced in the FTP adapter in BizTalk Server 2010. Here are some related links:

For the sake of better search engine hits, I am also pasting here the transcript I used for the video.

************************************************************************************************************************

In the Enterprise Application Integration (EAI) tutorial, we developed a BizTalk Server solution that takes an inventory replenishment request in XML format, and processes the request. The tutorial demonstrates how BizTalk Server transports and transforms the messages, and manages the business process. In the EAITutorial the transport adapter used is a FILE adapter. This presentation builds on the EAI tutorial to demonstrate the enhancements introduced in BizTalk Server 2010 using another transport adapter, the FTP adapter. Microsoft recommends you go through the EAI tutorial before watching this video. You can find the tutorial at https://go.microsoft.com/fwlink/?LinkId=195163. You can find the video for the tutorial at https://go.microsoft.com/fwlink/?LinkId=196966. You can download the files used for the tutorial from https://go.microsoft.com/fwlink/?LinkID=207706.

In this presentation we will change the scenario in the EAITutorial so that the warehouse system sends the replenishment request from an FTP server using an FTP receive location. If the request is approved, the message is sent to the ERP system using an FTP send port. If the request is declined, the message is sent back to the warehouse using another FTP send port.

Through this presentation, we will demonstrate the following enhancements introduced in the FTP adapter for BizTalk Server 2010:

  • Using secure data transfer
  • Transferring files atomically to an FTP server
  • Transferring files from an FTP read-only location

To demonstrate these features, we will not make any changes to the orchestration already created in the EAITutorial. We will only make changes to the port binding using the BizTalk Server Administration console. We will create new receive location and send ports that use the FTP adapter and will bind them to the logical ports in the orchestration.

Set up the FTP Server

This presentation assumes that you already have an FTP Server setup. The name of the FTP server used in this presentation is FTP-SERVER. The FTP server…

  • …. must be FTP version 7.5... Support for SSL over FTP is available with this version.
  • …. must be configured with basic authentication
  • …..must provide read and write permissions on the FTP site content directory.
  • …. must allow SSL connections
  • … must have its content directory set to C:\inetpub\ftproot. Within this, you must have created a folder called “Request”, “Response”, and “Temp”. In this presentation, we will use these folders for sending and receiving messages.
  • … must have its server certificate added to the Trusted Root Certification Authorities store on the computer where you will configure the FTP adapter. For this presentation, we will be using a self-signed certificate.

For instructions on how to set up an FTP site using SSL, see https://go.microsoft.com/fwlink/?LinkId=207600.

Deploy and Test the Existing EAISolution

This presentation is based on the EAI tutorial scenario. Let’s first deploy and test the EAISolution that we created in the EAI tutorial.

The files needed for this presentation can be downloaded from https://go.microsoft.com/fwlink/?LinkID=207706. After you extract the files to the C: root directory on the computer running BizTalk Server, you will have the following folders/files:

  • Completed folder contains the completed solution.
  • EAISolution folder contains the solution we developed in the EAI tutorial; this is also the base solution for this presentation.
  • ERP folder is a simulation of the ERP system.
  • WareHouse folder is a simulation of the Warehouse system. This folder also contains the request messages that we will be using to test the solution.
  • BTStutorial.snk is the strong name key needed to be compiled the projects.
  • EAISolution.msi is the installer for the base EAI tutorial solution.

Open BizTalk Server administration console … expand BizTalk Server Administration … expand BizTalk Group … expand Applications … import the EAISolution.msi … under the C drive … BTSTutorials … folder … use the default settings… When import succeeds, make sure you select the check box “Run the Application Installation Wizard to install the application on the local computer”. Importing the MSI only adds the application to the BizTalk management database.. to have the application function properly… you must also install it to install the assemblies….

In the BizTalk Server administration console… the installer created an EAISolution application … there are two assemblies deployed to this application, EAIOrchestration and EAISchemas.

Let’s start the application and run a quick test … copy the two files with the .xml extension… RequestInstace.xml and RequestInstace(Over Limit).xml… to the request folder … wait for them to disappear… we shall see one file under the Warehouse … RequestDecline folder … the date and time match the current system time … and one file under ERP … Request folder.

So far, we have deployed the EAISolution and tested the solution….. In this presentation, we will modify the solution so that it can use an FTP adapter receive location and send ports instead of the FILE adapter.

Secure Data Transfer

The FTP adapter in BizTalk Server 2010 introduces binding properties using which you can specify whether the data transferred to or from an FTP server will happen over a secure channel. The binding properties introduced are:

  • Use SSL – There are two channels for FTP communication…. Command or control channel… and data channel… All the FTP-related commands and user credentials are transferred on the command channel.. and any data transfer happens over the data channel… By setting the Use SSL property to Yes… you specify that the all communication happening over the command channel is encrypted and secure…. In effect, this is the property that specifies whether the FTP receive location is configured for secure data transfer. All other binding properties we will discuss now…. are ineffective, if this property is set to No.
  • Use Data Protection – You can set this property to specify whether the file transfer over the data channel is encrypted….
  • Client Certificate Hash – You can set this property to specify the thumbprint of the client certificate if you want the FTP client to be authenticated by the FTP server. In the context of this presentation… the FTP client is the computer running the FTP adapter.
  • FTPS Connection Mode – There are two connection modes, Explicit and Implicit….. In an Explicit connection mode, the FTP client connects over the control or command channel with an FTP server and then negotiates an SSL handshake with the FTP server…… In an Implicit connection mode, the SSL handshake must be negotiated before any FTP commands are sent by the FTP client…... While the Explicit connection mode enables the client to decide whether and when to upgrade to an SSL connection, the Implicit connection mode mandates that the entire FTP session is encrypted. A point to note is that the Implicit connection mode is deprecated by most FTP servers so in this presentation we will use the Explicit connection mode.

In the EAITutorial scenario, the receive location for picking the request message is configured using the FILE adapter……. The send ports for sending the request approved or request declined messages are also configured using the FILE adapter……. To demonstrate the secure data transfer feature, we will add a new receive location that uses the FTP adapter……. The new FTP receive location will pick the request messages from the “Request” folder that you must have already created on the FTP server under the FTP content directory……… The FTP receive location will be configured for a secure data transfer ….. The request approved and request declined messages will still be dropped to the same file locations as configured in the original EAITutorial presentation….. Once the FTP receive location is configured, we will disable the FILE receive location.

Configure Secure Data Transfer

In the BizTalk Server Administration Console, expand the EAISolution node….. click Receive Ports …. and add a new receive location to the EAISolutionReceiveRequestPort….. name the receive location as EAISolutionReceiveRequestLocation_FTP…. Specify the transport type as FTP and configure the adapter properties….

Under the FTP category…..

  • Specify the folder under the FTP content directory where you will be dropping the request message. For this presentation, you already created a folder “Request” under the FTP content directory C:\inetpub\ftproot on the computer running the FTP server.
  • Specify the password to connect to the computer running the FTP server.
  • Specify the fully qualified domain name of the FTP Server. When using SSL, you must specify the fully qualified domain name of the FTP server.
  • Specify the username to connect to the FTP server

Under the SSL category, specify these binding properties.

  • Set the FTPS connection mode to Explicit.
  • Set Use SSL and Use Data Protection to Yes because we want both the control channel and data channel to be secure.

Because we will still be dropping the same XML request files to the FTP server receive location, set the receive pipeline to XMLReceive…

Click the Receive Locations node….. disable the existing EAISolutionReceiveRequestLocation…. and enable EAISolutionReceiveRequestLocation_FTP that you just created.

Test the Solution

Let’s run a quick test to verify secure data transfer…. Share the C:\BTSTutorials folder where you extracted the tutorial files so that you can copy the .xml files from the computer running the FTP server… In this presentation, we have copied over the .xml files locally to the FTP server …. Log on to the computer running the FTP server and copy the two files with the .xml extension… RequestInstace.xml and RequestInstace(Over Limit).xml… to C:\inetput\ftproot\Request folder … wait for them to disappear… Now go back to the computer running BizTalk Server …. we shall see one file under the Warehouse … RequestDecline folder … the date and time match the current system time … and one file under ERP … Request folder.

If you want to verify that the data transfer is indeed secure… you can use the Microsoft Network Monitor … also called NetMon to verify that the communication between the FTP server and BizTalk Server happens in a secure manner…..

In these screenshots, both Use SSL and Use Data Protection are set to No… You can see that commands sent to the FTP server are in plain text... The password is hidden for security reasons….. you can also see the contents of the file being transferred….

In this screenshot, both Use SSL and Use Data Protection are set to Yes and the FTPS connection mode is Explicit.. Notice that the connection is upgraded to a secure connection using the AUTH command… and after that.. all the commands sent over the command channel are encrypted….

Atomic File Transfer

With previous versions of BizTalk Server, atomic file transfer was only supported for binary mode….. With BizTalk Server 2010, the FTP adapter is enhanced to also support atomic file transfer in ASCII mode…. Before we start talking about how the FTP adapter supports atomic file transfer, let us first understand why we need atomic file transfer at all…… Think of a scenario where the response sent from BizTalk Server to an FTP server is of a considerably large size, say 1GB….. Transferring 1GB to a relevant location on the FTP server will take some time….. However, on the location where the file is being transferred on the FTP server, you can start seeing the file even before it is completely transferred…… There can be scenarios where another application starts processing the file the moment it is available on the FTP server……. But.. because the file is large and is still being transferred, the application might start processing the file and eventually fail….. The FTP adapter rules out any such possibilities by supporting atomic file transfer…. which means the file is available in the relevant FTP location only when it is completely transferred…

To demonstrate secure data transfer to an FTP server, we modified the scenario in the EAITutorial to use an FTP adapter receive location….. Now to demonstrate sending a file to an FTP server atomically, we will further modify the scenario to now use FTP adapter send ports….. We will replace the two file ports with two FTP adapter send ports, one each for approved and declined requests….. However, both the send ports would eventually drop the response to the same physical folder on the FTP server….. This is the “Response” folder that you must have already created on the FTP server under the FTP content directory.

The FTP adapter provides atomic transfer by using a temporary folder on the FTP server…... You must have already created a “Temp” folder on the FTP server under the FTP content directory…… To begin with, BizTalk Server starts writing the response file to the temporary folder on the FTP server. When the response is completely written to the temporary folder, it is moved to the actual response folder….. The assumption here is that moving the response from the temporary folder to the actual response folder is atomic because it happens locally on the same computer.

Once the FTP send ports are configured, we will disable the FILE adapter send ports that were already part of the original EAI tutorial.

In this presentation, we will perform the following steps to configure atomic file transfer.

  • Configure an FTP send port for approved requests. If BizTalk Server approves the request message, it will send the response back to the FTP server using this send port.
  • Configure an FTP send port for declined requests. If BizTalk Server declines the request message, it will send the response back to the FTP server using this send port.
  • Configure send port groups for approved and declined requests. A send port group can have one or more send ports of different kinds such as FILE, FTP, etc. We will create a send port group for the approved requests, which will contain both the FILE send port that was already part of the EAI solution…. and the FTP send port that you just created for processing approved request messages…. Similarly, we will create a send port group for declined request messages to include the FILE and FTP adapter send ports for processing declined request messages.

Why do we need to configure send port groups? Well…. once we create send port groups, we will bind them to the logical send ports created for the EAISolution application…... Once we have done that, we can add or remove send ports from the send port groups without modifying the port binding in the BizTalk application. Let us take an example…. say after finishing this tutorial…. you want to include an SMTP adapter to send the response from BizTalk Server as an e-mail….. For that you will create another send port…. and then change the EAI solution port binding to now use the new SMTP send port… and every time you change the port binding.. you will have to restart the application… which essentially means downtime for your application…… Instead, if you have the logical ports bound to send port groups instead of send ports…… you can add the new SMTP send port to the send port group without changing the port binding…. When the application is bound to a send port group, the message is sent to all the send ports in the group, provided they are running….

In the context of this presentation, once we add the FTP send ports to the send port groups, we will disable the FILE send port so that the message is only routed to the FTP locations…. However, if you want to play around with the BizTalk application, you can simply enable the FILE port as well to have the messages sent to the FILE folder as well as an FTP location.

  • Bind the send port groups to the EAISolution. Once both the send port groups are created, we will change the EAI solution port binding to have the logical ports bind to the send port groups instead of the send ports.

Configure an FTP Send Port for Approved Requests

In the BizTalk Server Administration Console, expand the EAISolution node….. and create a new static one-way send port… name the port as EAISolutionSendToERPPort_FTP…Specify the transport type as FTP and configure the adapter properties….

Under the FTP category:

  • Specify the folder under the FTP content directory where BizTalk Server will drop the response message…… For this presentation, you already created a folder “Response” under the FTP content directory C:\inetpub\ftproot on the computer running the FTP server.
  • Specify the password to connect to the computer running the FTP server.
  • Because we have to achieve atomic file transfer for ASCII mode, set Representation to ASCII.
  • Specify the name of the FTP Server. If you want even this data transfer from BizTalk Server to FTP server to be secure, you must specify the fully qualified domain name of the FTP server….. We will also configure SSL-related properties on the port.
  • Specify the name with which the response file will be created on the FTP Server. By default BizTalk Server uses the %MessageID% macro…… However, in this presentation, because both the request approved and declined messages will be dropped to the same “Response” folder….., let us include a differentiator for both the files….. Let us call the response message for approved requests “Request_%MessageID%.xml”.
  • Specify the username to connect to the FTP server

Under the SSL category:

  • Set the FTPS connection mode to Explicit.
  • Set Use SSL and Use Data Protection to Yes.

Under the Tuning Parameters category, specify the temporary folder on the FTP server where BizTalk Server will first start copying the response message in case the response message is of a considerable large size….. For this presentation, you already created a folder “Temp” under the FTP content directory C:\inetpub\ftproot on the computer running the FTP server.

Because BizTalk Server will be sending XML response messages to the FTP server, set the send pipeline to XMLTransmit.

Start the send port.

Configure an FTP Send Port for Declined Requests

In the BizTalk Server Administration Console, expand the EAISolution node….. and create a new static one-way send port… name the port as EAISolutionSendDeclinePort_FTP…Specify the transport type as FTP and configure the adapter properties….

Under the FTP category:

  • Set the folder to “Response”…. because both the request approved and declined messages will be dropped to the same folder
  • Specify the password to connect to the computer running the FTP server.
  • Set Representation to ASCII.
  • Specify the fully qualified domain name of the FTP Server.
  • Specify the name with which the response file will be created on the FTP Server. For request declined messages let us specify the file name as “RequestDeclined_%MessageID%.xml”.
  • Specify the username to connect to the FTP server

Under the SSL category:

  • Set the FTPS connection mode to Explicit.
  • Set Use SSL and Use Data Protection to Yes.

Under the Tuning Parameters category, specify the temporary folder on the FTP server …. For this presentation, we will use the “Temp” folder under the FTP content directory.

Set the send pipeline to XMLTransmit….Start the two send ports.

Configure Send Port Groups for Approved and Declined Requests

In the BizTalk Server Administration Console, expand the EAISolution node….. create a new send port group… and name it RequestApproved_SendPortGroup…. From the list of send ports, add the FILE and FTP send ports that are created to process request messages that are approved…

Similarly, create another send port group…. Name it RequestDecline_SendPortGroup…. and add the FILE and FTP send ports that are created to process request messages that are declined….

Bind the Send Port Groups to the EAISolution

We will now bind the logical ports in the orchestration to the send port groups we just created….. Before changing the port binding you must stop the EAISolution application…

In the BizTalk Server Administration Console, expand the EAISolution node….. open the properties dialog box for the orchestration… from the Bindings tab, change the port bindings for the outbound logical ports… The Outbound Logical Ports column shows the name of the logical ports that were created in the orchestration. These are mapped to the physical ports on the right….. Map them to send port groups instead….. Map the SendDeclinePort logical port to RequestDecline_SendPortGroup… similarly, map the SendToERPPort to RequestApproved_SendPortGroup.

Start the EAISolution application… this starts all the send ports and receive locations in the application. Because we will only test receiving request messages from an FTP location and send response back to another FTP location… disable the FILE receive location and unenlist the FILE send ports…..

Test the Solution

Before we test atomic file transfer.. . let’s have a basic test to verify that the response message is sent back the FTP server… log on to the FTP server… copy the two files with the .xml extension… RequestInstace.xml and RequestInstace(Over Limit).xml… to the C:\inetput\ftproot\Request folder … wait for them to disappear… go to C:\inetpub\ftproot\Response folder …. we shall see one file with the “Request” prefix.. and another file with the “RequestDecline” prefix…. the date and time match the current system time …

Let’s now run a test to verify atomic file transfer …. But before that.. we will need to do a small tweak to the FTP receive location… To test atomic file transfer we need a large request message…. this request message will be picked from the FTP receive location…. The message we will use for this test is RequestInstanceLarge.xml which is approximately 300MB in size…. so for the FTP receive location to pick this request message, we will have to configure the FTP receive location to pick large files because by default it will only pick messages within 100MB in size…. So, open the FTP receive location properties…. and set the MaxFileSize property to a large value… for this presentation let’s set it to 1000MB… After setting this property, you must restart the receive location… Now copy the RequestInstanceLarge.xml file to the C:\inetput\ftproot\Request folder … wait for it to disappear… this might take a while because of the huge file size… after the file disappears.. go to the C:\inetpub\ftproot\Temp folder… you will notice that BizTalk Server has started writing the response file to this location.. the name of the file is a GUID…. refresh the folder to see the file size growing.. go to the C:\inetpub\ftproot\Response folder.. it won’t contain any response message yet.. wait for the message to appear… once the message appears, go back to the Temp folder.. you will notice that the folder is now empty.. So, we saw that for an ASCII file representation…. BizTalk Server used the Temp folder to temporarily write the response message while it was still being transferred.. once the transfer was complete.. the response message was moved to the desired folder…..

Transfer files from a Read-only FTP Location

As we tested the other two feature enhancements for the FTP adapter, we noticed that when BizTalk Server receive location picks the request message from the FTP server, it also deletes the message from the folder. However, if the FTP content directory does not provide write permissions to the clients, BizTalk Server will not be able to delete the request message and you will see a warning logged in the event viewer.

In BizTalk Server 2010, FTP adapter is enhanced to read the request message without deleting the file from the FTP server. To enable this support, FTP adapter introduces the following properties for the FTP receive location:

  • Delete After Download – This property specifies whether BizTalk Server deletes the request message from the FTP server after consuming the message….. By default this property is set to Yes. However, if the FTP server does not provide write permissions, we can set this property to No so that the receive location does not attempt to delete the request message.

Also, if this property is set to No, all other binding properties we are about to discuss will be ineffective.

  • Enable Timestamp Comparison – Consider a scenario where a request message abc.xml is not deleted after being processed by the FTP receive location, which means the file abc.xml continues to exist in the same folder….. Now if abc.xml is updated to include some more recent information and this updated file is copied over to the FTP server, there has to be a way for the FTP adapter to identify that an updated file is now available to be processed…. If you set the Enable Timestamp Comparison property to Yes, the FTP adapter processes the most recent file because the timestamp would have changed. So, in a nutshell, the FTP receive location is able to process the most recent file, even though the file never gets deleted from the FTP server….

One important point to note is that not all FTP servers support timestamp comparison. So, before setting this property on the FTP receive location, you must verify whether the target FTP server supports this.

  • Redownload Interval – If an FTP server does not support timestamp comparison, how do we configure the FTP adapter to pick the most recent file? Well…. we make a safe bet to pick up the available file at a given interval, whether or not the file has been updated…. This interval is defined using the Redownload Interval property. This property is used only if both Delete After Download and Enable Timestamp Comparison are set to No.

Configure File Transfer from a Read-Only FTP Location

In the context of this presentation, to configure file transfer from a read-only FTP location, we will perform the following tasks:

  • Revoke write permissions on the FTP server content directory – when you initially configured the FTP server it had both read and write permissions….For a read-only FTP location.. you must revoke the write permissions on the FTP server …
  • Configure the FTP adapter receive location – In this step, we will configure the receive location properties so that it can process request messages without deleting the file from the FTP server..
  • Start the FILE adapter send ports – Once we have revoked write permissions on the FTP server, BizTalk Server will not be able to write the response message back to an FTP location using the FTP send port… Instead, we will use the FILE adapter send ports that were originally part of the EAI solution…. We will also unenlist the FTP send ports we created earlier in this presentation.

Revoke Write Permissions on the FTP server

Log on to the FTP server… open IIS Manager… click the FTP site running on the server…. Open FTP Authorization Rules… and revoke the write permissions on the FTP site.

Configure the properties on the FTP receive location

In the BizTalk Server Administration Console, expand the EAISolution node….. and open the properties for the FTP receive location.. Under the Polling category..

  • Set Delete After Download to No so that the request message is not deleted from the FTP server after the FTP receive location consumes it.
  • Set Enable Timestamp Comparison to Yes.

Apply the changes, disable the receive location, and then enable it.

Start the FILE send ports

Because we have revoked the write permissions on the FTP server, we will now send the response message to the same location as it was originally sent in the EAI tutorial. For this, unenlist the FTP adapter send ports and start the FILE adapter send ports.

By the way, do you now see the advantage of configuring the send port groups? You could easily switch from the FTP send port to FILE send port without changing the application settings..

Test the Solution

To test the solution, log on to the computer running the FTP server and copy RequestInstance.xml to the C:\inetpub\ftproot\Request folder.…. Notice that the file does not disappear from the folder because the the FTP server does not provide write permissions….. Now switch back to the computer running BizTalk Server and look for the response message under C:\BTSTutorials\ERP\Request… the date and time match the current system time….

Now.. log back to the FTP server.. and navigate to the folder where you copied the request messages locally…. open the RequestInstance.xml and make some changes to the request message….. Copy the edited request message and paste it back to the C:\inetpub\ftproot\Request folder…. Switch back to the computer running BizTalk server and look for a response message under C:\BTSTutorials\ERP\Request….. You will notice another response message ... the date and time match the current system time… open the message and notice that the response message contains the changes you made to the request message….

Let us now test how the request messages will be picked up if Enable Timestamp Comparison is set to No. Open the FTP receive location properties dialog box... Set Enable Timestamp Comparison to No…. and set Redownload Interval to.. say 300.. and set Units to seconds.. this means that the FTP receive location will pick the request message from the FTP server…. every 5 minutes.. irrespective of whether the request message is updated or not.. If you are wondering why we need to specify 5 minutes as 300 seconds when we can just change the units to Minutes and set redownload interval to 5.. Well, here’s the reason.. the Units you specify here will be applicable to both Interval and Redownload Interval property.. So, if you set Units to Minutes.. the FTP adapter will poll the receive location every 60 minutes.. and then redownload the same file 5 minutes from then.. which is not what we want.. we want the file to be downloaded every 5 minutes.. after the polling interval.. so, we set the units to seconds…. And redownload interval to 300.

Apply the changes, disable the receive location.. and enable it again.. Go to C:\BTSTutorials\ERP\Request…. you will notice that the response message keeps appearing after every 5 or 6 minutes…

This concludes the presentation. If you have comments and suggestions regarding this video, please send them to BTSdf@microsoft.com, and include the name of this video “FTP adapter enhancements video” in the subject. Thank you for watching.