Running tests in mstest without installing the VS IDE

Yes, you CAN run your unit tests or any other VS test type on a machine without having Visual Studio IDE installed on that machine. You need the Visual Studio Team Agent to be installed on your test machine to run your tests. This is a 15 min lightweight install and is pretty easy to configure.

Those of you familiar with remote testing from our previous versions, will already know that you can trigger a test run on the client to run tests on a remote machine with the test agent installed. Just use the test settings to point to a test agent where your tests can run remotely. With the VS Team Agent installed on machine A and the client (VS Ultimate or Pro or Test Elements) installed on machine B, you can run tests remotely on A by triggering the run from client B.

Another alternate scenario where you would want to run tests without having the Visual Studio IDE installed is during builds. If you use Team Build to do builds and run tests as part of your build process, you can setup the build agent on a machine with no VS IDE installed. You can now go ahead and run all your tests without any additional installs on the build machine in VS 2010.

What if you want to run mstest from the command line though? You can choose to run mstest.exe from either the test agent or the build agent to run tests, but note that you can run only the unit test and ordered test types.

Comments

• Anonymous
  December 22, 2009
  Can the Team Agent collect and publish code coverage data or do you still need a VS install for this?

• Anonymous
  December 22, 2009
  Good q. When you run tests remotely  kicking off a run from a client on a team agent, the team agent can collect and publish code coverage data if you are using CC as a collector to run your automated tests.However,if you want to run mstest on the agent macine,you cannot collect CC data - only run unit and ordered tests.

• Anonymous
  January 17, 2010
  The comment has been removed

• Anonymous
  January 20, 2010
  Sure - just use mstest to run your load test and pass the load test file as an argument. If you want to run as part of team build, you can just configure that in the build definition

• Anonymous
  February 24, 2010
  Question: Do the Team Agents require TFS 2010 as well, or could this work with TFS 2008 running VS 2010 Test Projects? [Anu] Team agents dont really need TFS to run mstest - it can work standalone. You cannot get TFS 2008 to connect to a 2010 controller or a 2008 controller to connect to a 2010 agent.

• Anonymous
  November 16, 2010
  We are using VS2010 Premium to run automated tests (lists of generic tests) for the purpose of collecting test impact data.  When running these tests in the VS IDE a test impact xml file is created for each test in the list and the signature data file (testimpactdata.sdf) is updated.  When running the same lists of tests using mstest, test impact xml file is created for each test but the signature data file is not updated.  Why is this?[Anu] As a result, are you not seeing impacted tests?

• Anonymous
  November 17, 2010
  In this trial, only subsets of our automated tests were run (the entire test set takes 6 hours).  Running one subset (list) of tests in the VS IDE generates test impact xml files for that set and the signature data file is updated.  Running another different subset using mstest generates test impact files for that set but the signature data file is not updated.  Making changes to methods that are known to be called by tests in the first subset of tests (run using the VS IDE) and rebuilding shows the impacted tests.  Making changes to methods that are known to be called by tests in the second subset of tests (run using mstest) and rebuilding shows no tests are known to call these methods and thus no impacted tests.  It is my understanding that the signature data file contains the actual test impact data and that every time tests are run, the test impact data is updated.

• Anonymous
  November 17, 2010
  In this trial, only subsets of our automated tests were run (the entire test set takes 6 hours).  Running one subset (list) of tests in the VS IDE generates test impact xml files for that set and the signature data file is updated.  Running another different subset using mstest generates test impact files for that set but the signature data file is not updated.  After running the tests changes were made to methods that are known to be called by tests in the first subset of tests (run using the VS IDE).  After making the changes and rebuilding the solution, the test impact view shows the tests that are known to call those methods and which ones are impacted.  Changes were also made to methods that are known to be called by tests in the second subset of tests (run using mstest).  After making those changes and rebuilding, the test impact view shows no tests are known to call these methods and thus no impacted tests.  It is my understanding that the signature data file contains the actual test impact data and that every time tests are run, the test impact data is updated.  Because our tests take so long to run we prefer to run them at night.  Our builds and tests are run from scripts.  The test script executes mstest and specifies which lists to run.

• Anonymous
  December 08, 2010
  The comment has been removed

• Anonymous
  June 27, 2011
  The comment has been removed

• Anonymous
  August 01, 2011
  We have problem to run web.webtest  in the LabCenter. LabCenter environment  has installed: test agent , lab agent (test controller defined on other host); VS2010 is not installed.If I run "mstest.exe  /testconteiner:C:qaproduction.webtest"  the next error appeared:File extension specified ‘.webtest’ is not a valid test extensionPlease help.[Anu] Irina - can you please try running this via mstest on your local client first? You might have a typo in your switches

• Anonymous
  August 04, 2011
  The comment has been removed

• Anonymous
  October 02, 2011
  The comment has been removed

• Anonymous
  November 17, 2011
  The link to the Visual Studio Test Agent doesn't seem to point to the correct place.  Any chance you could update it?[Anu] Thanks for reporting this, Justin. The link is now fixed.

• Anonymous
  December 01, 2011
  The thread in the link below talks about a way to run tests from mstest without installing Visual Studio AND without installing Test Agent:social.msdn.microsoft.com/.../4c80703b-db52-495c-b28d-31a52bc36894The thread describes how to extract mstest.exe (and all the DLLs and registry entries associated with it), copy them to another machine, and run tests.Can you comment on whether that solution is:valid recommended supported by Microsoft