How is the user load adjusted for a load test with a goal based load pattern?

This post describes how the user load is adjusted when a load test scenario uses a goal based load pattern. 

 

First of all, if there is a warm-up period specified, the goal based pattern does not apply during the warmup period. Instead, the load test creates a temporary step load pattern that is used during the warm-up period to increase the user load until it reaches the Inital User Count setting in the goal based load pattern's properties. (In VSTS 2008 SP1 the warmup is done at a rate calculated to reach the Initial User Count just as the warm-up period ends; in previous versions, the warmup load pattern increased the user load at a rate of 10 users per second on each agent until the Initial User Count was reached, and this was too quick a warm-up for some tests.)

 

Here's what happens with a goal based load pattern after the warm-up period completes:

1. The user load starts at the value specified by the Initial User Count property of the Goal Based Load Pattern.

2. At each sampling interval (which defaults to 5 seconds, but can be modified by the “Sample Rate” property in the load test run settings), the performance counter defined in the goal based load pattern is sampled. (If it can’t be sampled for some reason, an error is logged and the user load remains the same.)

3. The value sampled is compared with the “Low End” and “High End” properties of the “Target Range for Performance Counter”.

4. If the value is within the boundaries of the “Low End” and “High End”, the user load remains the same.

5. If the value is not within the boundaries of the “Low End” and “High End”, the user load is adjusted as follows:

o The midpoint of the target range for the goal is divided by the sample valued for the goal performance counter to calculate an “adjustment factor”.

o For example, if the goal is defined as “% Processor Time” between 50 and 70, the midpoint is 60. If the sampled value for % Processor Time is 40, then AdjustmentFactor = 60/40 = 1.5, or if the sampled value is 80, the AdjustmentFactor = 60/80 = 0.75.

o The AdjustmentFactor is multiplied by the current user load to get the new user load.

o However, if the difference between the new user load and the current user load is greater than the “Maximum User Count Increase/Decrease” property (whichever applies), then the user load is only adjusted by as much as max increase/decrease property. My experience has been that keeping these values fairly small is a good idea; otherwise the algorithm tends to cause too much fluctuation (the perf counter keeps going above and below the target range).

o The new user load can also not be larger than the value specified by the goal based pattern’s MaximumUserCount property or less than the Minimum User Count property.

o Two more considerations based on special properties of the goal based load pattern:

§ If the property “Lower Values Imply Higher Resource Use” is True (which you might use for example for a performance count such as Memory\Available Mbytes), then the user load is adjusted in the opposite direction: the user load is decreased when the sampled counter value is less than the Low End of the target range and increased when the user load is greater than the High End of the target range.

§ If the property “Stop Adjusting User Count When Goal Achieved” is True, then once the sampled goal performance counter is within the target range for 3 consecutive sampling intervals, then the user load is no longer adjusted and remains constant for the remainder of the load test.

o Lastly, as is true for all of the user load patterns, in a test rig with multiple agents, the new user load is distributed among the agents equally by default, or according to the “Agent Weightings” if these are specified in the agent properties.

Comments

• Anonymous
  September 12, 2008
  PingBack from http://www.easycoded.com/how-is-the-user-load-adjusted-for-a-load-test-with-a-goal-based-load-pattern/
• Anonymous
  September 16, 2008
  Eugene Zakehareyev on Work Item customization tidbits: customization process (part 10 of x) and A Rule...
• Anonymous
  November 24, 2008
  How is the user load adjusted for a load test with a goal based load pattern? I found a great article
• Anonymous
  February 28, 2009
  Visual Studio Team System for Testers Content Index for Web Tests and Load Tests Getting Started Online
• Anonymous
  July 18, 2009
  Greatly written indeed… I really enjoyed your article and found it to be very informative, keep up the good work, I’ll be coming back to read any of your future articles..
• Anonymous
  December 02, 2011
  Really nice article and explained to a great simplicity. Please keep publishing the latest features in VSTS 2010 about Load test framework.