Partilhar via

Why is there a 4MB limit on MSMQ messages?

  • Artigo

Here's a request I've received from Dhaval Patel

**

"

I would really like to see some blog entries on how to handle messages greater than 4MB. I have heard some colleagues mention 'COM Wrappers' but I don't like the idea. To begin with, most of us were thinking that starting with Vista, the 4MB limitation would go away... it's a shame that this didn't happen! I was really looking forward to putting MSMQ to good use via WF and WCF, but seeing this limitation is still there, it seems folks like me will just need to stick to SOAP for eternity :)
Please feel free to enlighten me with the history of why MSMQ was forced to handle only 4 MB messages?"

OK, Dhavel, in this blog we will dip into the history behind this limitation.

The first place to start is the excellent MSMQ FAQ document:

3.5 Why is there a 4-MB message size limitation?

There are two main reasons:

  • The Message Queuing driver (Mqac.sys) uses memory-mapped files for message storage, and the present implementation does not break a large message into several message fragments. The driver maps the files to kernel memory address space when Message Queuing runtime sends or receives a message. The kernel memory address space is limited to 16 MB, shared by all drivers and fonts. This is one of the main reasons for the message size limitation. The 16-MB kernel address space limitation also means that a cluster node can host up to three Message Queuing resources, assuming that Message Queuing is not running on the node computer. This is because each resource can consume its own 4-MB range from the kernel address space.
  • The Message Queuing code that manages sessions is not designed to deliver large messages and handle session acknowledgements (ACKs) in the middle of message delivery. This can cause sessions to be closed while a large message is still being sent, because an internal session ACK is not received.

A bit more information on this 16 MB area can be found in the "Deploying Message Queuing (MSMQ) 3.0 in a Server Cluster" document:

Q. Are there any limitations for multiple MSMQ resources in a server cluster?

A. Yes. Each instance of MSMQ on a server node maps 4 MB of the system view space when handling message activity. This results in a default limit of three active, working instances of MSMQ in a cluster node. In a server cluster with three MSMQ resources, a node could have four concurrent MSMQ services running — the service running on the local node plus the three services associated with the MSMQ resources. In such a scenario, message activity might be limited, resulting in resource failures. To overcome this, you should increase the size of the system view space memory pool on each node of a server cluster that is running three or more MSMQ resources. We recommend that you do this even if your node is running less than three.

So we're talking about the system view space, the desktop heap for drivers. Although you can increase the size of this space with the SystemViewSize registry value, this will not affect the MSMQ message size in any way. The registry value is documented on the TechNet website:

Group and resource failure problems

  • The Message Queuing resource fails to handle message activity correctly which may result in resource failures

Although there is another registry value that does seem to set the MSMQ message size, you shouldn't use it. This isn't one of those "It's not supported but here it is for you to try anyway" references - I do actually mean "you shouldn't use it". The value description for MaxMessageSize does say:

Caution

  • Do not change the value of this entry. Increasing this value might prevent you from sending messages larger than 4 MB. Decreasing this value might block transports between Message Queuing computers.

So, in summary, the 4MB limit is there for good design reasons which are still valid in the 21st century.

In the next blog I'll go into ways to get around this limitation.

[[Edited 18th November, 2009]]

Here is an important note from the very useful Resource Management in MSMQ Applications article:

Message Size

MSMQ can support messages under 4 MB in size only, this includes Queued Components messages. Any attempt to send a message through the system that is larger than this will raise the insufficient resources error. Be aware that Unicode data takes up twice as much space as non-Unicode data, as two bytes are needed for each character.

Comments

  • Anonymous
    August 22, 2007
    I'm not sure that those sound like "good design reasons" -- more like "limitations when we chose an implementation". I'm definately interested in seeing how to get around it. I've run into situations where I had to take a less-than-ideal approach because of the 4MB limit. Most of my messages would have ended up around 1-2MB, but it was possible that sometimes we'd have a jumbo message of arbitrary size.

  • Anonymous
    August 22, 2007
    The comment has been removed

  • Anonymous
    August 22, 2007
    Thanks for the explanation, John. Definitely helps!!!

  • Anonymous
    May 19, 2009
    Hi, is the 4 MB limit for all messages stored in a queue, or is it for per single message that can be stored in a queue. If it is for per single message, then what is the total size of the message queue?

  • Anonymous
    May 19, 2009
    Hi Gobinda, The 4MB limit is per message. The total size of the message header + message body cannot exceed 4MB. The capacity of an individual queue is determined by three things: 1  the queue quota, if set in the queue properties 2  the computer quota, if set in the message queuing properties 3  the practical capacity of the machine to store MSMQ messages, which is variable. For discussions on machine capacity, check this post: http://blogs.msdn.com/johnbreakwell/archive/2008/02/29/what-are-msmq-s-limits-if-i-had-a-farthing-for-every-time.aspx Cheers John Breakwell (MSFT)

  • Anonymous
    March 28, 2010
    Excellent explanation. Has the limitation been overcome ? Does the Biztalk Server support message payloads greater than 4MB ? Regards, Franklin

  • Anonymous
    March 28, 2010
    Hi Franklin, No, the limit is unchanged. BizTalk used MSMQ/T for payloads over 4MB but I think that adapter is now deprecated. Cheers John Breakwell (MSFT)

  • Anonymous
    March 28, 2010
    Dear John,  Thanks for the clarity. If the MSMQ(T) has been depricated, what suffices as the Messaging Provider for Biztalk?

  • Anonymous
    March 29, 2010
    Hi Franklin, Normal MSMQ can still be used by BizTalk. There is no replacement that easily handles large MSMQ messages. Cheers John