Hangs and how to solve them - part 2 - Queuing

So in a previous post, we began discussing hangs, Hangs and how to solve them - part 1 - Deadlocks. So we will now move on to another common problem we see with ASP.NET...  Request Queuing.

How to tell Queuing is happening

So the first step is to see if this is really your problem.  One way to see this is to look at PerfMon.  There is a performance object for ASP.NET Applications (be sure to pick the one for the version of the framework you are using).  This is where you will find the Requests in Application Queue counter.  This one is what you want to use prior to 2.0 to monitor your queuing.

In 2.0 and later, you want to look until the ASP.NET v2.0.50727 object (again, best to pick the versioned one) and look at the Requests Queued counter.

Alternatively, you can look at a dump of the process to determine if queuing is happening.  In 2.0 you can just load the 2.0 sos and run !threadpool.

For earlier versions, you need to do a bit more, you first run !dumpheap -stat

And then !dumpheap -mt <address of System.Web.RequestQueue>

Then dump out the RequestQueue objects using !dumpobj <object>

This now gives you the _localQueue and _externQueue.  Dumping them out will show if anything is in them

You can see that the _head, _tail, _size are all 0.  If there was data in here, we would then dump out the _array and we could look at the items in the array.  Here we are using !do -v <queue> to print out the items in the array.

If you aren't getting this data from the source, maybe you should check out the original: https://blogs.msdn.com/tom

How to fix queuing

So if we have determined that queuing is happening, what should we do about it?  Well, obviously we don’t want that.  It can cause request times to be larger and can cause requests to time out.   The first thing to check is to make sure our threadpool settings are set according to what the recommended settings are.

Note: In 2.0 and later, if you leave the default of autoconfig=true, you will have these settings

The recommended settings are, from: https://msdn2.microsoft.com/en-us/library/ms998583.aspx

Table 17.7: Recommended Threading Settings for Reducing Contention

Configuration setting	Default (.NET 1.1)	Recommended value
maxconnection	2	12* #CPUs
maxIoThreads	20	100
maxWorkerThreads	20	100
minFreeThreads	8	88 * #CPUs
minLocalRequestFreeThreads	4	76 * #CPUs

Please note that the default and recommended values are shown, and that if hyperthreading is enabled, you want to treat a hyperthreaded CPU as two.  So if you look at the Performance tab of TaskManager, however many CPU’s it is showing is the number you want to use.  For example, if I have a dual-proc box that is hyperthreaded, I will use 4 CPU’s and the numbers become:

maxconnection	48
maxIoThreads	100
maxWorkerThreads	100
minFreeThreads	352
minLocalRequestFreeThreads	304

This will generally fix the problem if the settings haven’t already been adjusted in versions prior to 2.0.  If you have already made this change or are using 2.0 with the default settings, then you will have to do some more tuning.  You may want to see if there is a call to a database that is taking a long time.  Or a WebService call.  If so, you can try to cache the results if they are used by multiple users.  Another option is to balance your load across two webservers.

You can get more information about load balancing at:

• https://technet.microsoft.com/en-us/library/bb742459.aspx
• https://joeon.net/post/2008/01/Maarten-Balliauw-on-ASPNET-load-balancing-and-the-ASPNET-state-server.aspx
• https://msdn2.microsoft.com/en-us/library/aa746240.aspx

Comments

• Anonymous
  March 25, 2008
  You've been kicked (a good thing) - Trackback from DotNetKicks.com

• Anonymous
  March 26, 2008
  .NET: How to Build a REST app in .NET (with WCF) IIS7 - Mobile Admin What is wrong with the ASP.NET Community

• Anonymous
  March 26, 2008
  Insightfull and I imagine queuing would happen when there are a lot of concurrent requests. Thanks Josh Coswell http://riverasp.net

• Anonymous
  March 27, 2008
  The comment has been removed

• Anonymous
  March 28, 2008
  【原文地址】 March 28th Links: ASP.NET, ASP.NET AJAX, ASP.NET MVC, Visual Studio, Silverlight, .NET 【原文发表日期

• Anonymous
  April 12, 2008
  Collegamenti del 28 Marzo, ASP.NET, ASP.NET AJAX, ASP.NET MVC, Visual Studio, Silverlight, .NET

• Anonymous
  April 18, 2008
  Aqui está a última versão da minha lista de links . Verifique também minha página de Dicas, Truques e

• Anonymous
  July 22, 2008
  So I was hoping to get some more comments to this before posting the answer but I guess that isn’t going

• Anonymous
  July 24, 2013
  Probably the best writeup about troubleshooting queueing that I've seen! Unfortunately, these queues are no longer used in Integrated mode on IIS 7 and later.  If you are looking for more info on IIS & ASP.NET request queueing, this may also help: blog.leansentry.com/.../all-about-iis-asp-net-request-queues This also goes over how to get the requests stuck in the queue without debugging (not as precise as this post, but should be good enough in most cases). Best, Mike