Udostępnij za pośrednictwem

Running Tests With Team Build Orcas

  • Artykuł

The Orcas Beta 2 ReadMe contains (or should contain - let me know if it doesn't!) the following text:

On build machines with Visual Studio 2005 installed, running tests may fail with the following message:  “Failed to load tests from '<assembly>': Microsoft.VisualStudio.TestTools.Exceptions.EqtDataException: UTA059: The test DLL '<assembly>' was built using Visual Studio 2005, and cannot be run. To resolve this issue please rebuild the test DLL using the current version of Visual Studio.”  To work around this issue, it will be necessary to use the Visual Studio 2005 version of the TestToolsTask for the affected build definitions.  This can be done by setting two properties within TfsBuild.proj as follows:

  <PropertyGroup>

    <V8TestToolsTask>true</V8TestToolsTask>

    <MSTestRefPath>$(ProgramFiles)\Microsoft Visual Studio 8\Common7\IDE\PrivateAssemblies</MSTestRefPath>

  </PropertyGroup>

The value for MSTestRefPath may need to be adjusted, depending on the install location of Visual Studio 2005.  This property group can come at any point in TfsBuild.proj after the import of Microsoft.TeamFoundation.Build.targets.

It should be noted that this issue has been resolved for Orcas RTM, at which point the Visual Studio 2008 TestToolsTask will be able to run unit tests compiled using Visual Studio 2005 or Visual Studio 2008 and this workaround will no longer be necessary.  What exactly is going on here, you might ask?

The basic issue is that the Visual Studio 2005 TestToolsTask and earlier versions (up to Beta 2) of the Visual Studio 2008 TestToolsTask were built on the same framework used for MSTest.exe.  As such, using the VS 2005 TestToolsTask was essentially equivalent to using VS 2005 MSTest.exe, and using the VS 2008 TestToolsTask was essentially equivalent to running VS 2008 MSTest.exe.  As a result, the VS 2005 TestToolsTask can only run unit tests compiled against the VS 2005 unit test framework, and the VS 2008 TestToolsTask can only run unit tests compiled against the VS 2008 unit test framework.

To fix the issue for Orcas RTM, the TestToolsTask was redesigned from the ground up as an inheritor of Microsoft.Build.Utilities.ToolTask, which "...provides functionality for a task that wraps a command line tool".  Internally, the task will attempt to detect the version of the unit test framework that the unit tests it is running were compiled against and will locate and invoke the appropriate version of MSTest.exe. 

Of course, as a side effect of this change, the properties of ToolTask are available to users of TestToolsTask.  Interesting properties include:

  • ExitCode.  This output property gives the exit code returned by MSTest.exe.
  • StandardErrorImportance / StandardOutputImportance.  These properties control the importance with which standard error and standard output are redirected to the attached MSBuild loggers.  In combination with the logger verbosity used for the various loggers, this will control whether these messages show up in your build logs.
  • ToolPath.  This property allows you to explicitly control the path where MSTest.exe is executed from.  If the default logic of locating the appropriate version of MSTest.exe doesn't work for you, this path will give you a workaround to explicitly specify the location of MSTest.exe.
  • ToolExe.  This property allows to to explicitly control the name of the executed tool.  You probably won't ever need to set this, but if for some reason you have a tool that uses the same command-line syntax as MSTest.exe but is called something else, this property would give you the ability to use it...

-Aaron

Comments

  • Anonymous
    September 12, 2007
    PingBack from http://msdnrss.thecoderblogs.com/2007/09/12/running-tests-with-team-build-orcas/
  • Anonymous
    June 16, 2008
    Cool stuff. However, we have a problem. Years of using CCNet with complex MSBuild projects has led to a slow adoption of Team Build. Anyway, we're now trying to use the TestToolsTask to replace a <Exec Command="mstest ..."/>. It work brilliantly except from the fact that I cannot find a way to control the /detail swith. Specifically I would like to get the duration output. Any advice?
  • Anonymous
    June 30, 2008
    The comment has been removed