Load Testing in cloud using Visual Studio

 

Update in July 2013: Visual studio 2013 preview now has the capability to execute load tests in cloud built in! Details about how to get started with cloud based load testing with Team foundation service is here.

Visual Studio provides an intuitive and straight forward way to do load testing. The integrated experience in IDE, ease with which different configurations of the test can be controlled and the ease with which results can be interpreted is excellent. It also provides scale out, if we want to drive load from multiple machines. For intranet applications, out of the box solution works perfectly well. Setup the test rig with necessary agent machines and just start the test.

For internet facing applications, like the ones deployed in windows azure, this is a little tricky due to following reasons

1. Usually traffic from all machines inside a company goes through proxy.
   1. The proxy would have defined set of limits for outgoing request – this might interfere with load test which by definition would do lots of simultaneous requests.
   2. Since the proxy is shared, we don’t get a consistent bandwidth across multiple test runs. Consistent test bed configuration is critical for any kind of performance testing.
2. All the cloud providers charge for outgoing and incoming traffic from datacenter.

So keeping the test rig – pool of machines that drive traffic and the service close together makes sense. When they are in the same datacenter, we get standard and consistent bandwidth between those machines and the service we are trying to test.

This blog has three posts that give detailed explanation of how a test rig with Controller and Agents can be created in Windows Azure. It has everything need to get going.

Basic Idea is that we use windows azure worker roles as a simple VM, the worker role code wouldn’t actually do anything. Agent / Controller is installed and configured at role startup, after that it runs as a local windows service in the azure worker role VM. We create couple of instances of this and boom we have a test bed.

One detail here is – Visual Studio load test started out as a solution for load testing applications inside intranet. It requires all the agent machines to be part of a local network. To overcome this we use a feature called “Windows Azure Connect” . After configuring the test VM instances with “Azure Connect “ , they all become part of a local network – they would behave as if they are in local intranet. Much of this process is automatable – so setting up the rig is really quick and easy. I would love to see an OOB solution in codeplexJ .

Couple of issues that we might face with this approach

1. Connectivity from developer desktop (or a machine part of test automation) to test rig would typically be slow – it will be over internet. If the size of all referenced dlls is more than a couple of MBs , we might hit test deployment timeout issues. Instruction for this is at an msdn forum.
2. Windows Azure updates . Windows azure reboots or even reimages the machines at least once in a month. This is to ensure that the machines have all the necessary patches and security updates installed.
   1. A reboot in the middle of a load test run would stop the run.
   2. Manual intervention would be needed after such a reboot to setup/fix the rig.

Comments

• Anonymous
  May 06, 2012
  Thank you for sharing this informative post on Load Testing in cloud using Visual Studio. This articulation will certainly help the load testing experts. Look forward to your next post.