Component Tales: Safe and Reliable Communication using Message Queuing
[The following article is authored by one of the Windows Embedded MVPs (Most Valuable Professionals). Our MVPs have a heavy background in Embedded systems and are a great repository of information on Windows Embedded products. We’re providing this space on our team blog as a service to our readers by allowing MVPs to share some of their knowledge with the rest of the community.]
Embedded devices must be reliable and robust! While this can be achieved with reasonable effort on the device itself, e.g. using the Embedded Enabling Features (EEFs), such as Enhanced Write Filter, on Windows Embedded Standard, a weak spot generally is the communication of systems, for example with a backend infrastructure. This is especially true if the devices are distributed in the field or are mobile, and experience unreliable network connections. Some devices are equipped with different network access technologies such as Wi-Fi, Bluetooth, WiMax, UMTS, GPRS, etc. and switch to the available network transport as required, but this does not solve the problem from an application perspective.
Get robust - asynchronous communication
Most of the communication scenarios we use in our daily life are synchronous, because this is easy to handle. Think of a browser accessing an Internet site, asking for the page and getting its contents right after the request. This works perfectly as long as the connection to the site is reliable, but fails if the line is even for a few seconds. This problem can be solved using asynchronous requests, which get stored on both endpoints of the communication and delivered whenever possible. It does not work for all scenarios, such as the user sitting in front of a browser, but in a lot of application scenarios it is sufficient to be sure that the communication will be successful and an immediate response is not needed.
The classic use case for this is an order process. A customer places an order, but does not expect it to get to the warehouse in the very second it is placed. It is fine for him to wait one or two days to get the delivery. This means that orders can get stored, e.g. on an embedded device, as the first step and then be transferred in a second step as soon as an adequate transport mechanism is available.Orders and acknowledgements are stored in message queues on all endpoints of the communication scenario and delivered “asynchronously”. Naming of the “message queues” is quite descriptive- because the technique is very similar to sending letters via mail you could think of them as letter boxes, as well.
Microsoft Message Queuing (MSMQ)
As a software platform vendor Microsoft is offering a development framework for this kind of communication called MSMQ, which is part of all Windows operating systems and of course Windows Embedded Standard, as well. It offers an API and object model for developers to target. The benefit of using this API is that they do not have to create their own mechanisms to create the queues and message handling, which are quite complex. MSMQ is also available for Windows CE-based devices, providing the means to leverage this capable technology even in more advanced, heterogeneous embedded environments.
MSMQ Components
As part of infrastructure, MSMQ feature components can be found below the Software\System\Management\Infrastructure node in the Component Catalog of Windows Embedded Standard.
There is no global MSMQ macro component, therefore the related feature components need to be included into the image as needed. In Target Designer there are no settings to configure. This means that the MSMQ configuration needs to be done on the target device either manually or programmatically.
Created as an enterprise infrastructure component, similar to COM+ or ADSI, MSMQ is not a lightweight feature. It expects a lot of APIs and functionalities that are usually available in larger OS images only. The footprint of these images starts with 200 MB minimum, but one can expect to get image sizes of 400-600 MB in the end to satisfy all project requirements.This is not small, but CF card sizes more than double every 12 months, making this problem much less significant than some years ago.
Using MSMQ
MSMQ queues are easy to work with, because the encapsulation of the complex transport mechanisms has been done very well. For the developer it all boils down to sending and receiving messages and the most complex thing is setting up the queue.
There are public and private queues, which can be non-transactional or transactional. Public queues are discoverable via directory services such as Active Directory and therefore require some interaction and management effort. Private queues are not discoverable and therefore need to be well-known to all involved communication endpoints. If one requires maximum reliability, the use of transactional queues is a must. Local transactions are handled in the MSMQ system. For external transactions with other resource managers (e.g. SQL Server) MSMQ leverages the proven services of MSDTC (the Microsoft Distributed Transaction Coordinator).
MSMQ and managed code
For use with managed code, the native MSMQ service is wrapped in the System.Messaging namespace. Great documentation as well as samples showing how simple it is to work with message queues can be found on MSDN. In addition there is a highly recommended article by Duncan Mackenzi to get you started on using MSMQ in managed code. If you experience problems using MSMQ in combination with EWF, this blog post might help.
MSMQ is a cool and relatively unheard-of technology, worth to explore especially from an embedded perspective. Just check it out!
Alexander Wechsler
Wechsler Consulting