Message Bus

 

Retired Content
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.

Integration Patterns

Contents

Context

Problem

Forces

Solution

Example

Resulting Context

Security Considerations

Operational Considerations

Related Patterns

Acknowledgments

Context

You have an integration solution that consists of applications that are provided by different vendors. These applications run on a variety of platforms. Some of these applications generate messages and many other applications consume the messages.

For example, consider a financial application that integrates trading tools, portfolio management applications, modeling and risk analysis tools, trend indicators, and tickers. Market activity causes interaction between these systems. For example, a trading system communicates the completion of a sell transaction by sending a message to all other trading applications. The trading system could have individual connections to each trading application. This works well for a few applications, but becomes a burden as the number of applications increases. Managing the addition or removal of trading applications should not interfere with processing trades.

Problem

As an integration solution grows, how can you lower the cost of adding or removing applications?

Forces

Adding an application to an integration solution or removing an application from an integration solution entails balancing the following forces:

• Communication between applications usually creates dependencies between the applications. The sender must communicate with the receivers. The receiver must recognize the messages from all the senders. These dependencies translate into coupling between the participants.
• In a configuration where point-to-point connectivity exists, the coupling has a quadratic (or O[n2]) growth with the number of applications [Chandra03, Levine03]. For example, three fully connected applications need three connections, but 10 applications need 45 connections. This quadratic growth hampers maintainability, modifiability, and integrability.
• Usually, the applications of an integration solution have different interfaces. Changing the interfaces of proprietary applications is difficult. Even if you change the interface of one application, it is not feasible to change the interface for all the applications of your integration solution.
• Some integration solutions consist of a fixed set of applications. An integration solution that has low extensibility and modifiability requirements typically does not need to accommodate new applications.

Solution

Connect all applications through a logical component known as a message bus. A message bus specializes in transporting messages between applications. A message bus contains three key elements:

• A set of agreed-upon message schemas
• A set of common command messages [Hohpe04]
• A shared infrastructure for sending bus messages to recipients

When you use a message bus, an application that sends a message no longer has individual connections to all the applications that must receive the message. Instead, the application merely passes the message to the message bus, and the message bus transports the message to all the other applications that are listening for bus messages through a shared infrastructure. Likewise, an application that receives a message no longer obtains it directly from the sender. Instead, it takes the message from the message bus. In effect, the message bus reduces the fan-out of each application from many to one.

Usually, the bus does not preserve message ordering. Internal optimizations, routing, buffering, or even the underlying transport mechanism might affect how the messages travel to the receiving applications. Therefore, the order in which messages reach each receiver is nondeterministic. Preserving the order of messages requires additional logic. This additional logic can be provided by the participating applications. For example, the sending application could insert sequence numbers into the outgoing messages, and the receivers could use those numbers to reorder the incoming messages. The logic could also be provided by the bus, and the logic could therefore be transparent for the participating applications. However, this additional logic is not required.

Figure 1 shows an integration solution that uses a message bus. An application that sends messages through the bus must prepare the messages so that the messages comply with the type of messages the bus expects. Similarly, an application that receives messages must be able to understand (syntactically, although not necessarily semantically) the message types. If all applications in the integration solution implement the bus interface, adding applications or removing applications from the bus incurs no changes.

Figure 1. Applications communicating through a message bus

The shared infrastructure between a message bus and the listening applications can be achieved by using a Message Router [Hohpe04] or by using a Publish/Subscribe mechanism. This book focuses on message buses that use Publish/Subscribe mechanisms. For details on how to design a message bus that uses message-oriented middleware and a Message Router, see Enterprise Integration Patterns [Hohpe04].

Figure 2 shows the Message Bus pattern associated with the Publish/Subscribe pattern.

Figure 2. The Message Bus pattern associated with the Publish/Subscribe pattern

The kind of Publish/Subscribe implementation that you decide to use with a particular message bus has a profound impact on the message bus architecture. There are three types of Publish/Subscribe implementations: List-Based Publish/Subscribe, Broadcast-Based Publish/Subscribe, and Content-Based Publish/Subscribe.

Note   Although the Publish/Subscribe pattern is an important part of a message bus, Publish/Subscribe implementations are also used independently of message buses. For example, Publish/Subscribe mechanisms are used with the Point-to-Point Connection and Message Broker patterns. See the Publish/Subscribe pattern for more details.

Message Bus with List-Based Publish/Subscribe

Maintaining lists of published topics (subjects) and subscribers (observers) and then notifying each one individually as events occur is the essence of List-Based Publish/Subscribe implementations.

To use a message bus that contains a List-Based Publish/Subscribe implementation, a system sends a command message to the message bus. The message bus then looks up all interested message bus subscribers and sends each message bus subscriber a copy of the original message. Any data that is associated with the message is in a common format so that all systems can interpret the command message and the data, and then respond appropriately.

Message Bus with Broadcast-Based Publish/Subscribe

To use a Message Bus implementation that contains a Broadcast-Based Publish/Subscribe implementation, a system sends a command message to the message bus. The message bus broadcasts the message to all the nodes that are listening to the bus. The message bus makes no attempt to determine the appropriate subscribers. Instead, each listening node filters any unwanted messages and processes only the messages that it is interested in. Any data that is associated with the message is in a common format so that all systems can interpret the command message and the data, and then respond appropriately.

Message Bus with Content-Based Publish/Subscribe

To use a Message Bus pattern that contains a Content-Based Publish/Subscribe implementation, a system sends a command message to the message bus. After the message bus receives the message, it is responsible for matching the message against a set of subscribers and then forwarding the message to each of the appropriate subscribers. To match the message against a set of subscribers, the message bus must determine if there are any subscribers interested in this particular message. If an interested subscriber exists, the subscriber matches a particular message field or fields and a set of values. If a match exists between the message content and a subscriber, the message is then forwarded to each matching subscriber.

After a subscribing system receives a bus message, the subscribing system is able to process the message because the message contains the common command message and the agreed-upon message schemas.

While the Message Bus using List-Based Publish/Subscribe and the Message Bus using Content-Based Publish/Subscribe patterns both check for subscriptions before forwarding messages, there are key differences. The list-based approach matches subscriptions on subjects. However, the content-based approach is much more flexible. Because the content-based approach allows you to identify one or more fields in the message and a set of acceptable values for each field, you can create very intelligent routing capabilities.

To make Content-BasedPublish/Subscribe work, you need a high-performance infrastructure component that can read each message, query a set of potential subscribers, and efficiently route the message to each subscriber. In practice, you can think of this as a message switch. This is exactly how Microsoft BizTalk Server 2004 is designed.

Figure 3 shows the Message Bus pattern using a Publish/Subscribe pattern or one of the Publish/Subscribe variants.

Figure 3. Message Bus pattern using a Publish/Subscribe pattern or one of the Publish/Subscribe variants

Table 1 shows the responsibilities and collaborations that are associated with the message bus.

Table 1: Message Bus Responsibilities and Collaborations

Responsibilities	Collaborations
–Provides a common set of message formats to the participating applications. –Transports messages from the sender to the other applications that are connected to the bus.	–Senders tag outgoing messages and pass them to the bus. –Receivers inspect the incoming messages and discard the messages that are of no interest to any application.

Choosing a message bus for communication between the components of an integration solution lowers the coupling, but it introduces other problems. The following are some questions you should ask when considering a message bus for an integration solution:

• Bus latency. How long does it take the message bus to deliver a message to all the applications that are connected to it? What happens if a sender tries to pass a message to the bus before the bus completes message delivery?
• Bus contention. How do you prevent some applications from monopolizing the message bus? If some applications monopolize the message bus, other applications cannot use it.
• Bus arbitration. How do you deal with multiple applications that want to pass messages to the message bus at the same time?

Example

Consider an integration solution that integrates two trading systems, a portfolio manager, a risk analysis application, a modeling application, a trend indicator, and a stock ticker. The trading systems communicate with each other whenever they process a transaction. They also send updates to the other applications.

A point-to-point configuration requires individual connections between each trading system and all six applications. In other words, integrating the participating applications involves 11 connections between the participating applications. Figure 4 shows this topology (top) and the connections that are required to extend the solution to include an additional trading system (bottom). The dotted lines represent the new connections.

Figure 4. Trading applications that use point-to-point connectivity

A message bus reduces the number of connections between the trading applications; Figure 5 (top) shows this topology. As you can see from the figure, each trading application has a single connection to the bus. Each trading system is unaware of how many applications are interested in its transactions. With this topology (bottom), adding a new trading system requires a single connection and does not affect the existing applications.

Figure 5. Trading systems communicating through a message bus

Resulting Context

When considering integration through a message bus, you should weigh the following benefits and liabilities that are associated with it:

Benefits

• Improved modifiability. Each application has a single connection to the message bus instead of multiple dedicated connections to each of the other applications. Adding or removing an application has no impact on the existing applications. In addition, upgrading the bus interface does not require changing any applications as long as the messages remain compatible with existing ones. For example, this is the case when you add new message types.
• Reduced application complexity. The message bus transports messages between senders and receivers. Senders no longer interact with all the receivers that they need to send messages to.
• Improved performance. There are no intermediaries between the communicating applications. Communication latency depends only on how fast the message bus can move messages.
• Improved scalability. Adding applications has a constant low cost. In addition, you can add applications to the message bus until the message bus is saturated. A message bus is saturated when it cannot keep up with the data that it has to move between applications.

Liabilities

• Increased complexity. Integrating through a message bus increases the complexity of the integration solution for the following reasons:
  • Architectural mismatch. The applications of an integration solution typically make conflicting architectural assumptions [Garlan95]. Designing the message bus interface and solving the mismatch around the data model is a difficult endeavor.
  • Message bus access policies. Communication through a shared resource such as a message bus requires you to implement policies that ensure fair access and that resolve concurrency conflicts.
• Lowered modifiability when the bus interface breaks compatibility. Changing the message bus interface in a way that breaks compatibility typically affects all the applications that use the bus. In other words, the bus interface represents an extreme example of a published interface. Designing it requires foresight.
• Lowered integrability. All the applications that are hooked to the message bus must have the same message bus interface. Applications that have incompatible interfaces cannot use the message bus. Because the message bus interface includes a common set of command messages, message schemas, and shared infrastructure, these elements together define a common subset that may somewhat restrict the operation of the participating applications.
• Lowered security. A message bus that uses the Broadcast-Based Publish/Subscribe pattern reaches all the applications that are connected to the bus, regardless of the applications that the message is intended for. Broadcasting to all participants may not be acceptable if the messages contain sensitive data.
• Low tolerance for application unavailability. The receiver must be able to process messages when the sender passes the messages to the bus. This solution does not tolerate receiver downtime. In addition, it does not provide direct support for disconnected operation.

Security Considerations

Before you use a message bus that uses Broadcast-Based Publish/Subscribe, you should consider whether this configuration meets your security requirements. The applications that are connected to the bus receive every message that goes through the message bus. Participants that require a private conversation must encrypt their communication. Also, applications that communicate through the message bus do not have intermediate components between them. In other words, no physical component exists for mapping between different security contexts. Consequently, this configuration is appropriate when the security context is managed through impersonation.

Operational Considerations

When a message bus becomes saturated, message delivery may take a long time or even fail. Saturation could occur after you add new applications or after you make changes to the communication profile of existing applications. For example, changes to the communication profile include changes in the message size and rate. Because both situations are common in bus-centered integration solutions, it is important to prevent saturation. This translates into monitoring the operation of the message bus and proactively keeping the message volume below the maximum capacity of the message bus.

Related Patterns

For more information, see the following related patterns:

• Publish/Subscribe. This pattern helps keep cooperating systems synchronized by one-way propagation of messages; one publisher sends a message to any number of intended subscribers.
• Message Bus Architecture [Hohpe04]. This pattern revolves around a messaging infrastructure, and it relies on a canonical data model and a common command set that is mentioned earlier in "Liabilities."
• Blackboard [Buschmann96]. The Blackboard pattern describes a shared storage area that the components of the pattern use to communicate. Consumer components monitor the blackboard and grab the data that matches their interest. Producers put their output on the blackboard so that it becomes available to the others. Typically, integration applications such as rule engines and expert systems use this pattern.

Acknowledgments

[Buschmann96] Buschmann, Frank; Regine Meunier, Hans Rohnert, Peter Sommerland, and Michael Stal. Pattern-Oriented Software Architecture, Volume 1: A System of Patterns. John Wiley & Sons Ltd, 1996.

[Chandra03] Chandra, David; Anna Liu, Ulrich Roxburgh, Andrew Mason, E.G.Nadhan, Paul Slater. Guidelines for Application Integration, Microsoft Patterns & Practices, December 2003. Available on MSDN at: https://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag/html/eappint.asp.

[Garlan95] Garlan, David; Robert Allen, and John Ockerbloom. "Architectural Mismatch: Why Reuse Is So Hard," in IEEE Software, Volume 12, Issue 6, November 1995: 17-26.

[Hohpe04] Hohpe, Gregor and Bobby Woolf, Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions. Addison-Wesley, 2003.

[Levine03] Levine, Russell. "The Myth of the Disappearing Interfaces," in Business Integration Journal, November 2003.

Start | Previous | Next

Retired Content
This content is outdated and is no longer being maintained. It is provided as a courtesy for individuals who are still using these technologies. This page may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist.

© Microsoft Corporation. All rights reserved.