The evading solution to large remoting messages

Thanks to rick for the title!

We are seeing more and more people encountering:

Q322975 BUG: Passing large quantities of data in .NET Remoting calls causes an unexpected exception to occur - https://support.microsoft.com/?id=322975.

As you can tell from the title of the KB article, this is indeed a bug. Unfortunately the behavior addressed by this knowledgebase article is more of a “by design” problem than a coding error. Remoting was never designed for large messages and to my knowledge no testing was done on the types of large messages that cause the errors mentioned in the KB article. From what I have seen with remoting a “typical” remoting capsule is in the neighborhood of 16k – 32k (This is a very generous estimate).

There’s a HUGE difference between 32k and > ~5MB. Taking the development stance you can see why this behavior is by design. However, taking the implementer’s stance, we can see that when using datasets and other complex objects real world scenarios can easily exceed 32k. Developers don’t like fixing code that has an “easy” work around, and implementers don’t want to compromise their design or already coded applications. What can be done?

Well, the last time I checked both you and I lived in the real world, so here are some real world things that can be done to mitigate seeing the SocketException from Q322975.

• The source of this exception is actually within the networking stack and is courtesy of the implementation of one or another socket library. Since I’m not familiar with that code I’ve had to speculate about a few things and can tell you this: The error is somewhat hardware bound and throwing hardware at it has mildly successful results. For example, I’d see this problem sooner and more often on a PIII 600 Mhz with 256MB Ram and lest often on a P4 3.6 Ghz with 2GB of Ram. I’m under the impression that the buffering of the incoming data is somewhat dependent on the system hardware. That means that this exception can be alleviated to some extent but not eliminated. This is kind of like taking some cold medication; the medicine suppresses the symptoms but you still have the cold…it just may or may not be there when the medicine wears off.

• If you’ve already maxed out the hardware and want the least impact to your existing codebase possible, you can create a new remoting channel that implements some level of data compression. Think of this as zipping up all of your remoting messages. Of course this requires the minor change of modifying your remoting configuration. It might cost you time and money depending on your ability/willingness to write your own channel implementation. You can certainly save both time and money by using a 3^rd party channel implementation such as the one provided by GenuineChannels.

• A more involved solution includes actually modifying your code to a slightly larger extent. The benefit of this option however is that it’s not as involved as writing your own channel and does not involve the acquisition of a 3^rd party product. Another type of compression that can be implemented is actually shrinking the remoting message by excluding that data which is not needed by the client. Please consider the following KB article: Q829740 Improving DataSet Serialization and Remoting Performance – https://support.microsoft.com/?id=829740. Simply put, you modify the serialization of types you pass via remoting in such a way that you are sending only data that is necessary on the other side of the remoting call.

• The last option is indicated in the original KB article: chunk your data into smaller pieces. This work around involves modifying those calls that generate huge messages into several smaller pieces. I recommend that messages probably shouldn’t exceed 2 MB as I’ve never seen this exception on messages smaller than twice that size.

Comments

• Anonymous
  May 09, 2005
  The comment has been removed
• Anonymous
  May 09, 2005
  In my case at least, it was always the client machine tossing WSAENOBUFS. This is in a production environment where the client and server machines have the same specs. I've never seen our server machines complain about big remote calls, up to at least 20-25MB's.
  
  I would be careful with the DataSetSurrogate solution in (3). It's a nice little class (the C# version anyway), but there are some gotchas. More info here: http://mcfunley.com/blog/archive/2005/05/06/242.aspx
• Anonymous
  May 11, 2005
  Hey Dan, in your scenario, do you have the client sending large messages, or getting them back in response to calling a remote method? Or both?
• Anonymous
  May 15, 2005
  The client is receiving large responses to comparatively tiny method call messages.
• Anonymous
  June 19, 2009
  PingBack from http://debtsolutionsnow.info/story.php?id=11060