BizTalk Top 10 Tips From an Admin to Developers

Article
1/17/2024

Introduction

BizTalk administration and development have some overlapping areas but are still two different competence areas. Most consultants focus on one area, although a very few master both disciplines. Having brilliant developers is not worth much if the operations side is neglected, and vice versa.

Having done some development myself, but mainly administration and operations for 10 years now, here are my top 10 tips to BizTalk developers. We have chosen not to number them from 1 to 10, as several tips may be equally significant. Each tip will have a real-world example we have encountered in at least one case.

Note! This article will mention a lot of technologies, features and tools. It's not supposed to be a comprehensive guide to all of them, as that would require a book itself. Use TechNet and other resources to look further into my clues.

Top 10 Tips

Consider the performance of your solution

"If it works on my development box, deploy it!" The performance aspect is often forgotten. The solution may run on a dev box during development, but will often run for years in Production. The solution will probably share resources with several other deployed solutions. One bad solution can affect the performance and stability of others. For these reasons, consider the performance of your solution. To be more specific: Keep the number of persistence points to a minimum, reduce hops to the MessageBox (each hop adds a minimum of 500ms delay by default), use XPath only when necessary (it loads the entire message into memory), use streaming when applicable, use PassThru pipelines if possible, and don't rely on extensive use of tracking. These are just some examples. The main point is to not choose easy solutions that will negatively affect runtime performance.

Real-world example: One solution was deployed to Production, and shortly after being started consumed so many resources the entire environment went down. It had to be stopped, and later completely redesigned.

Use the features provided by BizTalk

This tip is meant for the code monkeys out there, but also some proficient BizTalk developers. BizTalk comes with a vast collection of features. Use them. A lot of developers like to write code, and overuse expression shapes, code libraries etc. Chances are high the built-in features of BizTalk will perform and scale better than custom code. There are also several features which are rarely used, like Backup Transport and Compensation. When creating error handling, use Routing for Failed Messages. Include BAM so your business users can track messages. The Business Rules Engine is very powerful, and your business users can even create and update the rule-set for you. On the flipside of this tip, some features should be avoided. One example is deprecated features (SOAP adapter, old SQL adapter etc.). And as always, do not spam the event log!

Real-world example: One C# developer included all the functionality of a workflow in one or more expression shapes. The expression shapes were used to call various code libraries. In fact, most of his solutions looked like the illustration below.

Use your BizTalk administrator

A dedicated and experienced BizTalk administrator has a lot of knowledge about integration, architecture, servers, performance tuning and more. Use that knowledge. Sometimes your solution will have specific requirements that differ from others. A classic example is low-latency, which BizTalk (out of the box) is not tuned for. When developing synchronous solutions with such requirements, it may be beneficial to create a dedicated host (and host instances) for this purpose.

Do you know what these settings can be used for? Your BizTalk administrator does (or should).

Is your solution not performing as it should during QA/Pre-Prod testing? Use your BizTalk administrator to identify bottlenecks and possible solutions.

Not sure what to include in deployment instructions? Ask what your administrator what is required, and to what detail. This will help ensure a smooth deployment to environments where developers do not have access. A best-case scenario is a fully automated deployment process.

Ask your administrator what Service Pack and Cumulative Update level is deployed on the servers, and make sure you use the same on your development box. Maybe you are struggling with a bug that has already been fixed.

Real-world example: A team of developers worked on a project with several response time requirements. When being tested in QA, the response times were not even close to the requirements. The developers, architects and project manager had no idea what to do. We were asked to look into the case. After performing a detailed analysis, We created a list of improvement suggestions. After implementing some of them, the response times were reduced dramatically. None of the implemented suggestions involved rewriting/-deploying the code.

Architect your solution the right way

This is a huge topic, so it will be limited to a few concepts - mainly choosing the right tools for the task. When starting a new project, one question should be "which technologies should we use to solve this?". Some BizTalk developers seem to be thinking "BizTalk is the solution, what is the problem?" Some Microsoft products can be used for many things. We've seen Excel spreadsheets with thousands of lines of code, acting as business critical solutions. It can be done, but it is hardly the best solution. Sometimes it helps to take a step back, and consider other tools. Other times some BizTalk features should be avoided. A couple of examples: If you have low latency requirements, don't rely on the FTP adapter. If you will be monitoring Twitter and put tweets in an Azure SQL database, consider using Logic Apps.

Real-world example: A BizTalk solution was developed to process large files. These files would vary in size and frequency, but more often than not be above 100MB. The processing took a long time and consumed a lot of resources. It was eventually rewritten to SSIS, and processing finished in seconds.

Perform peer reviews

Developers should have reviews of their own, but that will not be elaborated more here. During the development cycle, there should be reviews, and possible problems need to be corrected before proceeding. At a minimum, a review must be performed before deployment to Production. We recommend having the following review intervals:

Solution Design Approval
Pre-QA
Pre-Prod

Include sections for BizTalk administrators in such reviews. More details about this can be read here. Several checklist items can be automated, and should not take very long. This time is a useful and necessary investment to reduce issues and errors.

Real-world example: One client had maintenance hours late in the evening when very few people were at work. All BizTalk deployments had to be done in the evenings. The deployment instructions from developers were not adequate, binding files were not updated, and port passwords not available. Developers had to be called in to solve these issues, and this caused delays and added man-hours. A regime with reviews was established, and deployment automated as much as possible, making this a trivial task.

Consider the security aspect of your solution

This is very important, and sometimes hard. Increasing security of an integration solution often increases the development time, cost and complexity. Some developers are happy to avoid all this, to ease their job. After all, some developers do not care about a solution after it is deployed. In addition, business users often do not understand this aspect and do not want to pay for it. This is one area where administrators should have a strict regime. This often depends on the data being transferred, but here is my (and several others') opinion on when to encrypt data: When it can't be published on the front page in the paper. BizTalk is often used to transfer sensitive data, but security is not always good enough. If encryption, certificates, digital signatures, SSO and SSL are unfamiliar terms, improvements must be made. Instead of using FTP, use SFTP. Instead of using HTTP, use HTTPS.

Real-world example: After a comprehensive internal security review of several systems, it was discovered that several BizTalk solutions transferred sensitive data via the Internet without any added security. This company was a popular target for hackers. No one (to this day) knows if outsiders obtained some of this unprotected data. Security requirements were established, and exceptions had to be approved on a case by case basis.

Stay collocated with your operations team

This is more of a practical than a technical tip. If you have development and operations teams, let them sit in the same landscape. This enables communication on a new level, allowing lots of positive synergies. Overhearing each other creates interesting discussions that would not happen if teams are separated. A BizTalk developer may not need all the details about server architecture, load balancing and clustering, but I firmly believe it adds value. In addition, when developers plan a solution, input from an eavesdropping administrator can sometimes be more useful than you can imagine. Try out "A day with Operations" for your developers, and see what they learn.

Real-world example: BizTalk developers and the administrator were placed in different buildings. There was no real communication, and requests had to be ordered by e-mail. The competency areas had no overlap whatsoever. This lead to misunderstandings added man-hours and little understanding of roles. They were later collocated, which almost revolutionized the team spirit and collaboration.

Automate error handling

Error handling is an area often neglected, especially by junior developers. Although the built-in error handling in BizTalk may be adequate, often it is not. Several patterns exist for various scenarios. Some keywords are Failed Message Routing, Delivery Notification, One/Two Way Send Failures and ESB Exception Management Framework. When to apply what technique should be in a developer's repertoire. In general, it is my opinion that administrators should not be troubled with content error handling. If business users can solve such issues themselves, let them. This can be solved in many ways. SharePoint is one example. Messages with invalid data are sent to a SharePoint library, the business users get notified, update the data and resubmit. In addition, in BizTalk360 you can easily let business users handle their own applications.

Real-world example: Some applications generated a lot of suspended messages daily, and action needed to be taken by administrators. Business users had to be involved every time. This affected more important tasks, caused frustration and delays for the business. A SharePoint solution as mentioned above was established, automating the entire process. Now, business users could handle such errors themselves at their own pace, and administrators could focus on other tasks

Consistent naming standard and versioning

This is more important than some developers realize, for several reasons. Consistent naming makes handovers easier, solutions are easier to understand, and solving support incidents using (orchestration) debugger is faster. Implement and use a naming standard on all artifacts, including Visual Studio artifacts, BizTalk artifacts and XML namespaces. Here is one example worth looking into.

The next bit is versioning. This article has a lot of details and strategies for BizTalk versioning, and will not be repeated here. The only thing that should be added is to keep a record of version history in the documentation. Version number, dates and short change log should be readily available.

An orchestration without a sound naming standard:

Real-world example: An old application failed once in a while, and it was not clear why. This was a multi-server BizTalk environment. Documentation lacked, artifacts were not named sensibly, and newer versions were never reflected in the version number. It turned out different versions were deployed on the servers, making troubleshooting hard. All of the assemblies used version number 1.0. The 'new 1.0' worked, but the 'old 1.0' did not. This could easily be avoided using the techniques described above.

Stay updated on Azure

This actually applies to everyone in the Microsoft realm, including BizTalk developers. Some may argue that the developments in this area apply mostly to infrastructure people. Surely, news in BizTalk Server 2016 like AlwaysOn and production workload support is most relevant for administrators. Cloud computing will, however, affect developers beyond having support for new versions of Visual Studio and a resizable Schema dialog window. This is directly related to my tip about solution architecture and design. A traditional BizTalk developer staying in the on-premises world will surely miss out on some of the amazing possibilities in Azure. This thinking will also negatively affect customers. One example: An on-premise HA BizTalk Server environment is a huge up-front investment for companies, with respect to server cost, licensing and complexity. If the customer just wants simple integrations between some cloud services, a Logic App, PowerApp or Flow may solve this at much lower cost, time and complexity. Several companies are using a Cloud First strategy, and Cloud Only strategies are already emerging.

Looking forward, we believe BizTalk developers should consider themselves Integration Developers, having BizTalk as one of many tools available in their skillset.

Real-world example: One consultant was using some features available in Azure in a project, but created custom code for some missing functionality. In a status meeting, he was asked why he did not use a built-in feature available in Azure. It turned out this feature was not available when the project started, and he was not aware of it.

Share via