Using a load test plugin to limit the number of tests run in a load test
Normally the number of iterations of the inner tests (usually web tests and unit tests) run during a Visual Studio Team System (VSTS) load test is determined by the duration of the load test. However, some users which to control the number of inner test iterations exactly. There is not direct support for this in VSTS, but it can be accomplished using a load test plugin. Below is an example of a load test plug-in that does this. Instructions for using it are included in the comments.
Note that if the load test is run on a test rig with more than one load test agent, the number of test iterations specified will be run on each of the agents. You can adjust for this by specifying a smaller number of tests for the value of the "MaxTestIterations" context parameter, or modify the load test plug-in code to divide that number by the number of agents (but I'll leave that as an excercise for the reader).
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using Microsoft.VisualStudio.TestTools.LoadTesting;
namespace Plugins
{
/*
This class is a sample load test plugin that allows you
to limit the number of test iterations that are run during a load test.
To use this class:
1. Add the class to a test project.
2. Build the test project.
3. Hook this load test plugin up to your load test:
* In the load test editor, select the top-most node in the load test tree.
This is the load test name in the tree.
* Go to the property sheet.
* Click on the symbol next to the value for "Load Test Plug-in".
* Select the MaxTestIterationsLoadTestPlugin class as your plugin.
4. Add a context parameter named "MaxTestIterations" to your load test:
* Right-click the "Active Run Settings" node in the tree
* Select "Add Context Parameter" - you will see the property sheet
* For the parameter name, type "MaxTestIterations"
* For the parameter value, specify the maximum number of tests to run
Note that the duration specified for the load test still applies; the
load test will stop when the time specified for the duration has expired
even if the number of tests executed is less than "MaxTestIterations".
5. (Optional) Add a context parameter named "AbortWhenMaxTestsCompleted" to your load test:
* Follow the same step as for #4 above, but type "AbortWhenMaxTestsCompleted" for the
context parameter name, and either "true" or "false" for the value:
- When true, the load test plug-in will abort the load test as soon as the number of
test specified by MaxTestIterations has been completed. The value for "Outcome"
in the Visual Studio Test Results window will be "Error" in this case.
- When false, the load test will continue to run until the duration specified for
the load test is complete even after there are no more tests running (so "Tests / Sec"
and "Requests / Sec" will drop to 0 for the remainder of the test. This is the default.
*/
public class MaxTestIterationsLoadTestPlugin : ILoadTestPlugin
{
#region ILoadTestPlugin Members
public void Initialize(LoadTest loadTest)
{
m_loadTest = loadTest;
if (m_loadTest.Context.ContainsKey("MaxTestIterations"))
{
try
{
m_maxTestIterations = int.Parse((string)m_loadTest.Context["MaxTestIterations"]);
}
catch (FormatException)
{
throw new ApplicationException("MaxTestIterations not in integer format");
}
m_loadTest.TestStarting += new EventHandler<TestStartingEventArgs>(m_loadTest_TestStarting);
if (m_loadTest.Context.ContainsKey("AbortWhenMaxTestsCompleted"))
{
try
{
if (bool.Parse((string)m_loadTest.Context["AbortWhenMaxTestsCompleted"]))
{
m_loadTest.TestFinished += new EventHandler<TestFinishedEventArgs>(m_loadTest_TestFinished);
}
}
catch (FormatException)
{
throw new ApplicationException("AbortWhenMaxTestsCompleted not in bool format");
}
}
}
}
void m_loadTest_TestStarting(object sender, TestStartingEventArgs e)
{
if (++m_testsStarted == m_maxTestIterations)
{
SetLoadForAllScenariosToZero();
}
}
void SetLoadForAllScenariosToZero()
{
foreach (LoadTestScenario scenario in m_loadTest.Scenarios)
{
scenario.CurrentLoad = 0;
}
}
void m_loadTest_TestFinished(object sender, TestFinishedEventArgs e)
{
if (++m_testsFinished == m_maxTestIterations)
{
m_loadTest.Abort();
}
}
private LoadTest m_loadTest;
private int m_maxTestIterations;
private int m_testsStarted;
private int m_testsFinished;
#endregion
}
}
Comments
- Anonymous
February 16, 2006
I've posted an article with an example load test plug-in that allows you to specify the maximum number... - Anonymous
November 13, 2006
You did not just say "I'll leave that as an excercise for the reader!! " :) Thanks for the info though. - Anonymous
December 11, 2007
Hi Bill,I have question regarding this how this will work in load test rig having more than 1 agent.m_loadTest.Abort() method will abort the test on all agents even if the test iterations are less than the MaxTestIterations on some of agents, because even if one agent completed MaxTestIterations it will abort the test.How to do sync across various agent or how to share a variable across agents?Thanks,Gagan - Anonymous
March 17, 2008
That's true, and there is no good way to share a variable across agents other than doing something like having the code on the agents all access a shared file, database, or web service. However, the good news is that this blog post really only applies to VSTS 2005, as VSTS 2008 has first class suppport for specifying the number of test iterations to run in a load test rather than specifying a time based duration. I highly recommend upgrading to VSTS 2008 for this and many other enhancements listed here: http://blogs.msdn.com/edglas/archive/2007/03/28/what-s-new-in-orcas.aspx.Bill - Anonymous
April 21, 2010
There are some very great sources here and thank you for being so kind to post them here. c - Anonymous
May 03, 2012
very useful samples thanks! - Anonymous
November 25, 2014
can you tell me what text mix model,duration,other load test settings for this plugin