Share via

Running Unit Tests for Individual Configurations with Team Build

  • Article

The default behavior of Team Build when running unit tests is to run them for each configuration defined for the build.  That is, when you specify configurations for compilation (e.g. Debug|Any CPU and Release|Any CPU) you are also specifying them for tests.  This is true of both VS TFS Build 2005 and VS TFS Build 2008 (Orcas).

This may not always be desirable - for example, you may want to compile both Debug and Release configurations, but only run unit tests for Release.  This sort of behavior is possible within both versions of Team Build, but requires some slightly different modifications in each case.

VS TFS Build 2005

In 2005, you will need to do something we don't typically recommend - override one of the CoreXX targets.  The reason we don't typically recommend this is that it is likely to break when you upgrade to newer versions of Team Build.  And indeed - the change I will be outlining here will break if and when you upgrade to VS TFS Build 2008.  Unfortunately, it's the only method I've come up with for doing this sort of thing, so when you're ready to upgrade just come back to this post and follow the method given for VS TFS Build 2008 instead...

This is the declaration of the CoreTest target in Microsoft.TeamFoundation.Build.targets:

   <Target Name="CoreTest"
          Condition=" '$(RunTest)'=='true' "
          DependsOnTargets="$(CoreTestDependsOn)" >

    <MakeDir Directories="$(TestResultsRoot)"
             Condition="!Exists('$(TestResultsRoot)')" />

    <MSBuild Projects="$(MSBuildProjectFile)"
             Targets="RunTestWithConfiguration"
             Properties="BuildNumber=$(BuildNumber);Platform=%(ConfigurationToBuild.PlatformToBuild);Flavor=%(ConfigurationToBuild.FlavorToBuild);IsDesktopBuild=$(IsDesktopBuild)" />

  </Target>

The important bit is the MSBuild task invocation - it batches over the ConfigurationToBuild item group and calls back into TfsBuild.proj (on the RunTestWithConfiguration target) for each unique combination of PlatformToBuild and FlavorToBuild.  That is, it runs tests for each configuration!

To modify this behavior, you'll need to modify the target so that it only calls back into TfsBuild.proj for the desired configurations.  For example, if you only want to run tests for Release|Any CPU, you could do:

   <Target Name="CoreTest"
          Condition=" '$(RunTest)'=='true' "
          DependsOnTargets="$(CoreTestDependsOn)" >

    <MakeDir Directories="$(TestResultsRoot)"
             Condition="!Exists('$(TestResultsRoot)')" />

    <MSBuild Projects="$(MSBuildProjectFile)"
             Targets="RunTestWithConfiguration"
             Properties="BuildNumber=$(BuildNumber);Platform=Any CPU;Flavor=Release;IsDesktopBuild=$(IsDesktopBuild)" />

  </Target>

If you want to run tests for all Release configuration, you could do:

   <Target Name="CoreTest"
          Condition=" '$(RunTest)'=='true' "
          DependsOnTargets="$(CoreTestDependsOn)" >

    <MakeDir Directories="$(TestResultsRoot)"
             Condition="!Exists('$(TestResultsRoot)')" />

    <MSBuild Projects="$(MSBuildProjectFile)"
             Condition=" '%(ConfigurationToBuild.FlavorToBuild)' == 'Release' "
             Targets="RunTestWithConfiguration"
             Properties="BuildNumber=$(BuildNumber);Platform=%(ConfigurationToBuild.PlatformToBuild);Flavor=%(ConfigurationToBuild.FlavorToBuild);IsDesktopBuild=$(IsDesktopBuild)" />

  </Target>

And so forth.  Note that you should make these modifications in your TfsBuild.proj file after the <Import> of Microsoft.TeamFoundation.Build.targets rather than modifying the targets file itself - this way the changes will only impact your build definition and not all build definitions that execute on the build machine.

VS TFS Build 2008

In 2008, more extensibility points (targets designed to be overridden) have been added to the build script, including the Before- and AfterTestConfiguration targets, which execute before and after tests are executed for a particular configuration.  The simplest thing to do, then, in VS TFS Build 2008, is just to specify a different set of tests to be run for each configuration.  In particular, we can empty out the list of metadata files (and/or test containers) for each configuration where unit tests should not be run.

 <Target Name="BeforeTestConfiguration">

    <ItemGroup Condition=" '$(Configuration)' != 'Release' ">
        <MetadataFile Remove="@(MetadataFile)" />
        <TestContainer Remove="@(TestContainer)" />
    </ItemGroup>

</Target>

Note that this is only possible because MSBuild added support for removing items from item groups in MSBuild 3.5 - in MSBuild 2.0 you could only add to item groups, you could never take anything out.  Note also that we no longer need to use batching in the condition - the Before- and AfterTestConfiguration targets (and the Before- and AfterCompileConfiguration targets) are executed once per combination of Platform and Flavor (Configuration), and the properties $(Platform) and $(Configuration) can be used to retrieve the current values.

Comments

  • Anonymous
    October 19, 2007
    PingBack from http://www.artofbam.com/wordpress/?p=10662
  • Anonymous
    October 23, 2007
    Brian Harry on TFS 2008 System Recommendations. Martin Hinshelwood on Falling off the TFS Rehabilitation...