Freigeben über

Microsoft Message Queue - An Introduction

  • Artikel

In this series I would like to introduce you to MSMQ history, key concepts, dwell into painpoints, suggest workarounds, and give an insigth into to future.

What is MSMQ?

MSMQ enables Asychronous communicaiton between two different applications. Application A(Sender) sends a message to a queue hosted in MSMQ, Application B(receiver) receives and processes the message from the queue MSMQ is usually used in the following scenarios a. Where there is an impedence mismatch between the sending and receiving applicaiton b. Load-levelling scenarios c. Sending and Receiving applications need to be running at different times d. Sending and Receiving applications need to be running across network or even across the internet.

MSMQ enables you to send messages over a variety of protocols such as TCP, IPX (if you care anymore :-) ) , HTTP, PGM (Pragmatic Multicasting) MSMQ provides out of the box security and routing of messages.

The History

MSMQ was designed and developed by a Microsoft developement team in Haifa, Israel.

MSMQ version 1.0 was shipped with the NT Option pack. In this version MSMQ was integrated with SQL Server. The programming model was easy but the Installation and management was cumbersome.

MSMQ version 2.0 shipped as an optional windows component with Windows 2000. Active Directory was born, MSMQ was quick to grab the opportunity to replace SQl-server dependency with AD.

MSMQ vesrion 3.0 shipped as an optional windows component with WinXP and Windows 2003. Major new features like HTTP protocol support, Multicasting support, Distribution lists, etc were introduced. Security and reliability of the product was greatly enhanced in version 3.0

Key Conepts

MSMQ can be installed in two modes

  • Domain Mode - Integrated with Active Directory For MSMQ to be installed in domain mode the computer on which MSMQ is installed should be joined to a domain. The benefits of AD integratio are the following.
    • You can create and use both public and private queues: Public queues are one whose properties(metadeta) are published in the Active directory. Public queues can be discovered by querying the AD. All publics in an enterpirse can be managed through the AD Management console. This acts as a single point of management Note: The messages in the public queues are not stored in AD, only the properties are stored in AD. The messages are stored on the computer on which the queue is create. Private queues are one whose properties(metadata) and messages(data) are published on the computer on which the queue was created. The name "Private" is rather misleading. You can send and receive message to a private queue from anywhere on the network using the Direct format name addressing scheme. Infact this is the only addressing scheme available for pivate queues.
    • Integrated securit: MSMQ should be installed in domain mode if you require out of the box authentication and encryption of messages. This is because for authentication MSMQ signs the message using the user certificate registered in AD. For encryption the message is signed using the machine certificate registered in AD.
    • Routin: If you need to route messages between different sites then you need MSMQ to be installed in domain mode. All routing configurations are stored in AD.
  • Workgroup Mode - Does not require Active Directory If the computer is joined to a domain MSMQ will be installed in workgroup mode. In this mode you have the capability to create and use only private queues. You don't have the out of the box authentication and encryption features. Howevere MSMQ allows you to attach any standard X.509 certificate to the message, but the receiver application need to authenticate the message itself using crypto APIs. The routing feature is also not available if MSMQ is installed in workgroup mode. If you dont require out of the box security or routing workgroup mode is your friend because it is easy to install and manage and doesn't have any external dependencies (AD).

APIs

You can program MSMQ using 4 APIs

  • C++ API: Coding is a really tedious and cumbersome, but obviously the most performant b.
  • COM API: COM wrapper over the C++ API. Better coding experience. Can use VB or C++ to program.
  • System.Messaging: .NET wrapper over the C++ API (uses PInvoke to call into the C++ APIs). Considerably imporved programming experience over C++ and COM APIs. Great increase in productivity. Performance is equal to or better than the COM APIs.
  • Indigo Queuing programming model: The next generation distributed programming model. Built on top of C++ APIs. Service oriented programming model. Performance is equal to or better than System.Messaging. Best programming experience. Microsoft will make all its investments in Indigo going forward.

Queues

Queues are nothing but First-In-First-Out(FIFO) datastructures.Queues can be created using the MSMQ management tool inside Computer management console MMC or programatically. Queues can be created and used dynamically. However you need to be aware of issues that might be caued by replication delays in the case of public queues. Private queues does not have such issues There is no physical limit on the number of queues that you can create on a given computer

Messages

Queues hold messages. Messages contain several properties and body. Some properties such as priority, correlationID, messageID etc are used to specify the contract of the message. Some properties like Label, body etc can contain the payload or data. The total size of the message including properties and body is limited to 4MB. The body of the message can contain pretty much anyting. You can ship COM objects that supports IPersistsStream interface, .NET Data types that can be serialized through binary or XML serializer. You need to be aware of the fact that message occupy both virtual memory and page pool. A fixed number of page pool bytes are consumed per message. If you have a lot of messages (like 1 million) on the disk you will start putting pressure on the page pool. In normal situaltions where sender and receiver applications are working smoothly you don't end up with a lot of messages in the queue. However in scenarios where there is a huge impedence mismatch between the sender and receiver or the receiver applicatio is down you might end up queuing a lot of messages.

MSMQ supports specifying storage quotas at the machine level and queue level. This ensures that you get warnings in the form of Negative acknowledgements when you hit the quota. This can help you prevent situations you have too many messages in the queue which will put pressure on system memory and page pool.

MSMQ support two types of messages

  • Express: Express messages are not stored on the disk and hence will be lost if the computer restarts/crashes. Express messages as the name suggests are very fast. They are used in scenarios where you dont really care about message loss. For example a stock applications sends stock quotes every second and it doesnt care if one of the quote was lost, it can catch up int the next second. However the application wants the message to reach the destination as fast as possible.
  • Recoverable: Recoverable messages are stored on the disk and hence can withstand computer crashes/restarts. Sending and receiving messages

You can send messages to both local and remote queues. You can send messages using several protocols such as TCP, IPX, HTTP, PGM (Multicasting). Receive is always done over RPC. You can receive messages from both local and remote queues.

Messages can be send and received as a part of a transaction.

MSMQ supports two types of transactions

  • Internal transaction: You can send and receive messages to MSMQ as a part of a transaction
  • MSDTC co-ordinated transaction You can send and receive messages as a part of a transaction where other resource managers such as SQL-Server can participate.

You can send and receive transactional messages to both local and remote queues.

High Availability and Scalability

MSMQ provides high availability by the use of Microsoft Clustering service. MSMQ supports both Active-Active and Active-Passive clusering. MSMQ provides scalability in two scenarios.

Load-balancing

In this scenario you can have a farm of MSMQ servers behind a hardware or software load balancer. Clients can send messages to the MSMQ servers through the load balancer.

If you are using the TCP protocol to send messages you are limited to only non-transactional sends.

If you are using the HTTP protocol you can send messages both transactionaly and non-transactionlly.

Load-levelling

In this scenario the queues are hosted on a central server (clustered for high availability). Client applications send messages to the queues. The reveiver applications are hosted on a farm of MSMQ servers. They receive and process messages from the central queues. Messages in the central queues are levelled across the server farm.

Security

MSMQ supports authentication and encryption of messages. When you send a message you set certain properties of the message to specify whether you want authentication and encryption or not. You can also select the has algorithm and encryption algorithm that msmq uses to sign and encrypt the message.

You can set the "requires authentication" property on the queue thereby resticting the applications to be able to send authenticated messages to the queue.

You can also set the encryption property of the queue to accept only encrypted messages.

Note: Messages are encrypted only on the wire, once they reach the destination queue messages are decrypted and stored as plain text.

-Anandraj

Comments

  • Anonymous
    August 05, 2005
    NICE Blog header dude.....this has got to be one of the funniest blog headers I've ever seen...I'm nominating you for my Best Blog Header Awards :-)

  • Anonymous
    August 10, 2005
    Very very nice post!

  • Anonymous
    August 24, 2005
    My 2 cents,
    Nice posting but lot of it are kind of repro from MSDN/help. It would be nice if you write more on real world scenario/integration issues...

  • Anonymous
    November 21, 2006
    Help !!! I'd like to receive one complete C# sample of WCF->MSMQ->PGM serv - multi/client Who knows , send me it Thank Andrey

  • Anonymous
    March 14, 2007
    Du musst ein Fachmann sein - wirklich guter Aufstellungsort, den du hast!

  • Anonymous
    March 18, 2007
    luogo grande:) nessun osservazioni!

  • Anonymous
    March 19, 2007
    Great site! Good luck to it's owner!

  • Anonymous
    May 15, 2007
    hi.. i have a query here can we CREATE Private Queue on remote Machine???

  • Anonymous
    June 22, 2007
    Hello: I'm extremely new to MSMQ and have a question regarding transactional versus non-transactional messages. Can you give good definition of both? Thanks Joe

  • Anonymous
    October 16, 2007
    How to create privatepublic queue on remote machine??? Pla help me.... Thanks in Advance!!!!

  • Anonymous
    January 04, 2008
    PingBack from http://actors.247blogging.info/?p=3664

  • Anonymous
    March 20, 2008
    PingBack from http://dinnermoviesblog.info/solutions-monkey-microsoft-message-queue-an-introduction/

  • Anonymous
    July 08, 2008
    I know absolutely nothing about MSMQ. However it's been suggested I try to see if there is a way to increase the size of the queue. I assume a farm of MSMQ servers is one way. Is there a way to optimaly increase the size on one server?

  • Anonymous
    August 10, 2008
    Receive transactional messages from a remote private queue using msmq 3.0

  • Anonymous
    October 24, 2008
    How I Can greate private queue automatic? Run Setup.exe or ....?

  • Anonymous
    June 19, 2009
    PingBack from http://mydebtconsolidator.info/story.php?id=17339

  • Anonymous
    June 19, 2009
    PingBack from http://debtsolutionsnow.info/story.php?id=6572