Gardening (on the Web Server)

While some people are born with a green thumb, others of us have the brown thumb that comes from working to get a garden to grow. While we dream of sweet fruits and crisp, fresh vegetables, sometimes all we get for our troubles is dirt under our fingernails. There is a feature in IIS6 that can have similar results and oddly enough shares the name.

IIS6 Web Gardens is a nifty idea. With the simple click of the mouse (or a few key strokes using adsutil.vbs), you can magically make your application pool use many worker processes. While this seems to imply a "green thumb" approach to solving issues like scalability, with just a little clearer understanding, we can see the dirt (and compost) that appears under the glossy exterior.

What does enabling a Web Garden really do? As the configuration suggests, it causes the application pool (and therefore each application in that apppool) to be running in multiple processes. The key that people tend to not notice (or not care to think about) is the fact that each worker process (and the applications loaded) work independently and concurrently. What does this mean? It means, if you use a caching system, the cache will be duplicated in each application pool. It means, if you use a resource, that resource needs to be shared by each instance in the application. It means, that you now have doubled (or worse depending on how high you set the number) the number of threads vying for CPU time. It means, if you are using ASP.NET there will be that many more Garbage Collection threads - all running at high priority if using the default server GC - consuming the CPU time at an alarming rate. In short (yeah, too late for that), it means more resources used with worse performance, due to the decrease in resource reuse.

Note that many applications have problems dealing with Web Gardens. The systemic problem is usually the same that doesn't allow a given application to run in multiple application pools or enabling over-lapped recycling - namely that many applications assume dedicated access to some system level resource. In other words when this kind of application starts, it takes exclusive access to a resource; so when a second version of the application starts, it fails to get this exclusive access.

In much of the positive "literature" about this feature, one of the reasons people give for enabling Web Gardens is "to enhance performance" (IIS6 Resource Kit page 666 - just the page number alone should give us some hint about this feature and performance). While using multiple process can remove a bottleneck related to a highly latent back-end dependency, from a performance point of view, the cost is significant. Especially considering that as an administrator you can generally control the number of threads (and therefore connections) that are used by IIS. Thus, rather than scaling the number of processes to scale the number of connections or threads, why not just scale the number you want directly? In the (same) literature, there are claims of scaling improvements due to relieving contention. While it is possible for this to happen, in most applications in the wild, the resource in question is of the kind that requires exclusive access - as stated before this makes Web Gardens a non-option. Also, it is not pointed out that one of the ways contention on non-exclusively accessed resources is decreased is by reproducing that resource in each process - again consuming more system resources, memory in particular.

At this point you may be wondering "Why does the feature exist?" or at least "Why are you bothering to talk about this feature beyond saying: 'Don't Use it'?". Oddly enough, even though the feature shows up on the Performance tab of InetMgr, the one redeeming feature or positive use is as a reliability tool. For example, if we follow the performance statements from above and bump up the number of threads to increase the concurrency to a backend dependency, our one worker process would have an extremely large number of outstanding HTTP requests and customers. If something bad happened to the worker process (not that we ever have anything go wrong with any worker process at Microsoft.com), all of those requests are lost, the HTTP connections dropped. If it happens often enough, your availability numbers start to drop. In this scenario (highly latent back-end that is not capacity bound, just latent), if you suspect (i.e. know) you have unreliable code running in your worker processes (obviously the work of some third-party, since no one in your company would write such garbage ;^), you *might* be able to improve your availability by trading performance and resources for it using Web Gardens. It works because each worker process only has 1/nth of the customers (since HTTP.SYS uses a round-robin algorithm to assign requests to existing worker processes), so if a given worker process dies only 1/nth the customers get their connections reset.

A better solution to consider is not Web Gardens, but multiple application pools. It is often possible to split applications up into parts that are independent. If this is possible, then scaling up the number of application pools allows for the reliability wins of Web Gardens without the negativity of resource usage duplication - though some resource usage increase is expected, the Garbage Collection example from before still applies. While not as simple clicking the checkbox and inputting a number, breaking applications into multiple application pools has better potential from increasing both Performance and Reliability of your application. Your caches will run cleaner, and your applications will be bounded.

In the end, while not a performance feature, Web Gardens could be used as a stop-gap measure to help get you through a hard time while you try to improve the reliability of the applications running in IIS. This is similar to what recycling is supposed to do - in both cases, IIS6 helps you improve the availability of poor applications but neither should be considered "the right solution". If you have an unreliable application, you should be working through the issues, getting fixes from the people who own the code. If you have highly latent back-ends, you can work to reduce the dependencies or the amount of traffic that is communicated to the back-end. Obviously this kind of work takes time, so it is nice that IIS6 can help you out a bit - however it is important to understand if you sew Brussels sprouts don't expect to reap strawberries for your ice cream.

Comments

• Anonymous
  January 01, 2003
  Voici un article présentant comment installer IIS 6 et comment le configurer, ainsi que quelques recommandations

• Anonymous
  July 19, 2007
  Another reason for web gardens (honestly the only one I can truly stand behind, in addition to the above) is to break the 2GB x86 process barrier.  I worked with a customer that was reaching the 2GB mark every week or so, thus making the worker process to crash.  The app was not using session state, so we enabled web gardens and used 3 "flowers" (get it?  flowers, gardens...my own coined term, btw. quite nerdy I admit), which spread the load over 3 worker processes and prevented them from hitting the upper 2GB limit and crashing.

• Anonymous
  June 26, 2008
  The comment has been removed