BizTalk Server 2013 R2: JSON support and integration with Cloud API's
Introduction
BizTalk Server 2013 introduced a couple of new adapters. One of them was the WCF-WebHttp adapter which offers REST Support. The WCF-WebHttp adapter gives you the ability to send messages to Restful services or receive messages through an exposed endpoint. One of the limitations with the adapter (binding) was the lack of Json support. You had to write your own custom pipeline components to serialize the Json format to XML (you can read about it in this blog post: BizTalk Server support for restful services). BizTalk Server 2013 R2 has out-of the box support for sending and receiving JSON messages with the following features:
- A wizard to generate an XSD schema from a JSON instance,
- and an Encoder and Decoder component to use with custom pipelines.
By creating a custom pipeline and dragging either a JsonEncoder or JsonDecoder you can serialize Json into xml or vice versa. With an instance of a Json message you can use the wizard to create a Json XSD schema. No need to write your own custom components any more.
Scenario's
There are many web services present that have a REST interface and talk JSON over the wire. The WCF-WebHttp adapter in BizTalk Server 2013 R2 provides a means to communicate with these services. There are various scenario’s you can think of to how to demonstrate the functionality of the WCF- WebHttp. This article demonstrates how to consume a relatively simple Restful Endpoint that you can choose from the Restful Service endpoints of the US Federal Aviation Administration. The Airport Service is used as an example. The Airport service provides the airport status and delay information from the Air Traffic Control System Command Center (ATCSCC) for every US Airport. It has one endpoint that only supports the GET operation.
The GET request is the fundamental, widely used operation in the REST world. You can simply visit a URL in a browser (or programmatically) and for instance in the case of the Airport Service type the following URL:
http://services.faa.gov/airport/status/SEA?format=xml
The browser will return a machine understandable structured data like below:
Figure 1. Response Airport Service status.
In this article we will not specify the xml format, but the Json format i.e. format=json. There will be scenarios in the real world that services do not support xml format and only communicate through the REST protocol and json format. Let’s assume the airport service only supports json. The URL would look like:
The following more advanced scenario describes how the airport service is consumed through BizTalk Server 2013 R2 that receives a request from a client application:
From a client application a request for the status of an airport will be sent as a soap/xml message. BizTalk will map this request to a GET operation to the Restful service endpoint. That’s is the incoming message contains the airport code that is marked as the property for its schema (xsd). That property will mapped to the outgoing request URL to call the Restful endpoint. The endpoint on its turn will process the request and hopefully will provide the status of a given airport based upon the airport code provided within the request URL. The result will be mapped to response message that will be routed back to the client application, where it will be rendered in a Windows Form.
Figure 2. Scenario 1 Consuming a RESTful endpoint.
The communication with the internal endpoint in this scenario will be SOAP/XML as explained. The endpoint is hosted in a two way receive port (request/response). The message will subsequently be mapped to a REST call to the service indicating that the expected response format is json. BizTalk will decode the response into XML, so that it is published as an XML message in the MessageBox. The receive port will subscribe to that message so it will be routed back as a response to the client that on his turn renders it in the UI (Form). This scenario shows that BizTalk acts as a broker and protocol mediator (SOAP/XML --> REST/JSON --> SOAP/XML) between internal client and the external airport service.
I have second scenario I will showcase in this article, where I have built a client application. This application has the following functionality:
- Get information about an artist.
- Get the top albums from the artist.
Information and top albums can be obtained through calling the Last.fm API artist methods. The client will via BizTalk call these API methods. Similar as in my previous scenario calling the Airport Service to retrieve its status. Below you find an overview of the scenario.
Figure 3. Integrating with Cloud API Last.fm.
The communication with the internal endpoint in this scenario will be SOAP/XML. The endpoint is hosted in a two way receive port (request/response). It exposes schemas for two operations: GetArtistInfo and GetTopAlbums. The request message will subsequently be mapped to a REST call to the service indicating that the expected response format is Json or default xml. BizTalk will decode the response into XML, so that it is published as an XML message in the MessageBox in case the response message is Json (GetArtistInfo) otherwise it will be just received by the adapter (GetTopAlbums). The receive port will subscribe to that message so it will be routed back as a response to the client that on his turn renders it in the UI (Form). This scenario shows that BizTalk acts as a broker and protocol mediator (SOAP/XML --> REST/JSON –> SOAP/XML or SOAP/XML –> REST/XML –> SOAP/XML) between internal client and the external Last.fm API.
Building the BizTalk solution
The solution of the described scenario's consists of the following parts that will be discussed in the subsequent paragraphs:
- Exposing schema’s for internal service exposing an operation to the client application that will consume the service.
- Creating a custom receive pipeline to enable decoding of the Json message to xml.
- Configuration of a Send Port with the Web-Http adapter (or binding if you like), send and receive.
- Demonstrating the solution.
Exposing schema’s as service
The request schema for the first scenario will contain one element, AirportCode, which is promoted as property. Later I will explain, the reason why the AirportCode is promoted property. The other schema is based on the Json response of the Airport Service.
http://i208.photobucket.com/albums/bb152/Steef-Jan/Json%20Biztalk%20Server%202013%20R2/Schema2_zpsabbd6548.png
Figure 4. Request schema for internal service
By calling the service in the browser (Chrome with Postman application) using the following http://services.faa.gov/airport/status/SEA?format=json you can obtain the Json.
Figure 5. Obtaining a Json instance of the Airport Service response using Postman.
Save this file as json. Subsequently, you follow the following steps:
- In the Solution Explorer, right-click the project name > Add > New Item > JSON Schema Wizard. Provide a name for the schema (JSONSchemaAirportStatus.xsd****), and then click Add.
http://i208.photobucket.com/albums/bb152/Steef-Jan/Json%20Biztalk%20Server%202013%20R2/JSONSchemaWizard2_zpsd8dd4d29.png
Figure 6. JSON Schema Wizard in Visual Studio 2013.
- In the JSON Schema Wizard, on the welcome page, click Next.
http://i208.photobucket.com/albums/bb152/Steef-Jan/Json%20Biztalk%20Server%202013%20R2/JSONSchemaWizard3_zps92266bfb.png
Figure 7. JSON Schema Wizard Welcome screen.
- In the JSON Schema Information page, provide the location of the JSON purchase order file that is sent to the BizTalk Server application. Provide a root node name, a target namespace, and then click Finish.
http://i208.photobucket.com/albums/bb152/Steef-Jan/Json%20Biztalk%20Server%202013%20R2/JSONSchemaWizard4_zps8833bf11.png
Figure 8. Specify the root node and target namespace for JSON Schema.
- Now you will have a schema like below:
http://i208.photobucket.com/albums/bb152/Steef-Jan/Json%20Biztalk%20Server%202013%20R2/JSONSchemaWizard5_zps07eb5359.png
Figure 9. Created JSON Schema for Airport Status.
First schema will be the request of the service endpoint and the second the response the generated schema based on Json. Both schemas are with the same BizTalk project. You must compile this BizTalk project as you will need the assembly later.
The following steps will lead to the creation of a WCF Service based on the earlier two created schemas:
- Launch the BizTalk Web Services Publishing Wizard and follow the steps described in the MSDN page How to Use the BizTalk Web Services Publishing Wizard to Publish Schemas as a Web Service. Basically you launch the wizard, pass the welcome screen. Specify adapter (WCF binding) to communicate with the service, whether or not you want the service expose Meta data, in which application (receive location) you like tie the service too.
http://i208.photobucket.com/albums/bb152/Steef-Jan/Json%20Biztalk%20Server%202013%20R2/WCFWizard1_zps28813b15.png
Figure 10. Specify transport type, meta data, and location in BizTalk application.
- Click Next and select Publish schemas as WCF Service.
Figure 11. Choose a method of creating the WCF-service.
- Again, click Next and start specifying the service operations, assign schema’s you created earlier and that are within the BizTalk assembly to request and response of the Service Method.
Figure 12. Creating Service, operations and assigning the schemas.
- Click Next and specify a target namespace.
Figure 13. Specify the target namespace.
- Again click Next and to specify the location of the service in IIS. BizTalk delegates the hosting of service to IIS, yet to start the service it relies on the receive location to be enabled.
Figure 14. Specify the IIS location, and if anonymous access desired or not.
- Last time to click Next to see the summary of what you have specified for the service.
Figure 15. The summary of the WCF-Service to be created.
- Click Create to provision the WCF-service.
Figure 16. End screen of the Wizard.
- Click Finish to end the Wizard. The provisioned service will appear in IIS.
Figure 17. Created service will be available in IIS.
- When you enable the receive location in BizTalk that has the Uri of the service you can browse to it.
Figure 18. Receive location in the specified BizTalk application.
- The service is up and running the WSDL can be imported as a service reference in the client.
Figure 19. Service description in the browser.
The above steps describe the approach for the first scenario. The same steps apply for the second scenario.
Create a custom receive pipeline
BizTalk Server 2013 R2 provides pipeline components that can be used to process JSON messages within a BizTalk Server application, i.e. JSON decoder and JSON encoder. For the custom receive pipeline, we will use the JSON decoder pipeline component (this applies for both scenario's). To create the custom receive pipeline you can follow the steps below:
- In Visual Studio within your Solution Explorer of your project, you right-click and point to Add > New Item > Receive Pipeline. Specify a name for the pipeline name like JsonReceive.btp, and then click Add. Within the Decode stage you can add the new JSON decoder. In the other stages and other pipeline components as shown in the screenshot, and save changes.
Figure 20. Add the Json decoder in the receive pipeline.
- In the properties of the JSON decoder you specify the Root Node and Root Node Namespace.
Picture 21. Specify Root Node and Root Node Namespace at design time.
- You can do this at design time like above or in run-time.
- Next you add the XML Disassembler pipeline component in the Dissemble stage. Save and your custom receive pipeline for serializing JSON to XML is ready.
Creating and configuring the Send Port with the Web-Http adapter
To be able to consume the Airport Service with BizTalk you will need to have a send port configured with the WCF-WebHttp adapter. This can be done by configuring the WCF-WebHttp adapter or binding if you like in case you choose WCF-Custom. The configuration is pretty straight forward. The Airport Service is a public service that requires no authentication for its usage. In the general tab of the WCF-WebHttp Transport properties the address of the service can be specified (URI). Besides the address you specify here the HTTP Method (GET) and perform a URL mapping. To be able to communicate with the Last.fm API and call both methods I needed to have two send ports configured with the WCF-WebHttp adapter. The Last.fm API doesn’t require authentication other than supplying the api_key as a parameter in a call to any of the API methods. The general tab for the WCF-WebHttp for Airport Service is like below:
Picture 22. Specify the address of the Airport service, HTTP method and URL mapping.
In the HTTP Method and URL Mapping section you specify the method (operations) you are going to perform. In the case of the Airport Service this is going to be only the GET. In case you use an orchestration that the Name has to be specified, which is the name of the operation of the logical port. The URL mapping you define what is going to added after the specified URI. To make it more dynamic instead of hard-coding in general you make use of the variable mapping configuration feature. So what's between the brackets is a variable that can be mapped to the promoted property. The HTTP Method and URL Mapping looks in this case like:
<BtsHttpUrlMapping>
<Operation Method="GET" Url=”status/{airportcode}?”/format=json">
</BtsHttpUrlMapping>
Variable mapping is a powerful technique to define any custom variable (or place holder) in your URL, in this scenario case {airportcode} and map that variable to any context property with the property name and namespace. The Variable mapping is specified by click the Edit… button.
Picture 23. Assign Variable to Property.
Variable is mapped to the property namespace that defines the AirportCode.
The general tab is important for specifying the address, method and URL mapping. The other tabs are:
- The Binding tab provides you the ability to configure, the time-out and encoding-related properties.
- The Security tab provide you the ability to define the security capabilities of the WCF-WebHttp send port.
- The Behaviour tab provides you the ability to configure the endpoint behavior for the send port.
- The Proxy tab provides you the ability to configure the proxy setting for the WCF-WebHttp send port.
- The Messages tab provides you the ability to specify how the message is sent to the REST interface.
Note: In this scenario we only use GET operation of the Airport service. Based on the verb you have to specify in the Suppress Body for Verbs the GET, because a payload is not required for this operation. Since BizTalk sends out messages with a payload this needs to suppress!
Figure 24. Suppress body for Verb GET.
The general tab for the WCF-WebHttp for Last.fm operations is a bit different. More variables need to be assigned here and the url mapping format is different.
Picture 25. Specify the address of the Last.fm service, HTTP method and URL mapping.
For the URL mapping I needed to add a few parameters to my REST call.
- http://ws.audioscrobbler.com/2.0/?**method**=*artist.getinfo*&**artist**=*Metallica*&**api_key***=\<your last fm api_key>*&format=json
My HTTP Method and URL Mapping will look like:
<BtsHttpUrlMapping>
<Operation Method="GET" Url="/?method={method}&artist={artist}&api_key={api_key}&format=json"/>
</BtsHttpUrlMapping>
Interesting thing in this URL mapping is that & is and &. If you try to just use the & you will run into an error like depicted below:
Figure 26. Error when applying wrong format in URL mapping.
(System.ArgumentException) Invalid configuration for the HTTP Method, or the mapping from Operation to HTTP method and Url.
Parameter name: mapping
(System.Xml.XmlException) '=' is an unexpected token. The expected token is ';'. Line 2, position 54.
For further details on configuration settings for these tabs see MSDN article How to Configure a WCF-WebHttp Send Port.
Test the solutions
Now the exciting part is the testing of the created solutions. The client for both scenarios is a Windows Forms application. The application for the Airport Sertice has a simple UI and a reference to the created WCF-Service host in a BizTalk receive location. The client will send the chosen airport code to the service. The client has a combo box with airport names and corresponding codes of all the US Airports. When I select SEA as the code for Seattle/Tacoma Airport then the airport code SEA will be sent as a request message to the WCF-Service. Eventually the response will be rendered on the client.
Figure 27. Client application to request the status of a given airport.
If you want to monitor the network traffic between BizTalk and the Restful service you could for instance, use Netmonitor 3.4. In this I monitored the call to the FAA Airport RestFul service.
Figure 28. Examining the communication with Airport Service using Netmon.
In case you enable tracking for the WCF-WebHttp configured send port you can examine the tracked messages in BizTalk. Particular interesting the response from the Airport Service arriving at the Send port as Json.
Figure 29. Examining the response in BizTalk Tracked Messages.
For the second scenario the client application will make a call to the Last.fm API methods indirectly via BizTalk. The responses will be nicely rendered in a Windows Form. When I enter an artist name, and click GetInfo then a request will be send to BizTalk routed to the send port that communicates with Lastfm API and request info of the band Metallica.
Figure 30. UI for Last.fm retrieving the information on a given band (Metallica).
When I click TopAlbums another request is sent to a different send port that send to a different Last.fm API method.
Figure 31. UI for Last.fm retrieving the top albums on a given band (Metallica).
If we look at the traffic between BizTalk Server and Last.fm using Netmon I can examine what goes over the wire.
Figure 32. Examining the communication with Last.fm API methods using Netmon.
Wrap up
This article has demonstrated how easy it is to consume a Json message with BizTalk Server 2013 R2 after invoking a Restful service. Now custom coding is required to serialize a Json format into XML the internal format of the BizTalk messaging engine. Currently REST/Json has taken over the XML/Soap world, at least in the cloud that is. Numerous services available in the cloud support REST let alone only support REST. Therefore, BizTalk Server has adapted to that shift in cloud and supports REST through the WCF-WebHttp adapter and support for JSON. I also showed how I was able to leverage an API from Last.fm. The cool thing is that BizTalk Server 2013 R2 is capable to communicate with tons of REST API’s out there in the cloud with the WCF-WebHttp adapter. And with JSON support things get less complex. I haven't tried communicating with an API that requires Basic- or OAuth authentication. I probably will have to do some custom coding using behaviours like explained in the blog post from Quicklearn.
Source Code
For the code sample of the first scenario see MSDN Code gallery:
See Also
Another important place to find an extensive amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.