Share via

How to running multiple tests in team build?

  • Article

 

In beta3 you have better control on the way tests are executed. Some common scenarios are:-

  • To run tests of different categories:-
    • You need to modify tfsbuild.proj file to introduce additional test category. For example if your metadata item is defined as

<MetaDataFile Include = "$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi">

  <TestList> BVT1 </TestList>

     </MetaDataFile>

and you want to execute another category of test (i.e. BVT2). You can easily add the additional test category.

<MetaDataFile Include="$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi">

<TestList> BVT1; BVT2 </TestList>

</MetaDataFile>

    • Do note that build will fail if one of the following things hold true
      • Test category BVT2 is not defined in HelloWorld.vsmdi
      • BVT2 contain tests that evaluates to inconclusive
  • Using multiple vsmdi files to running tests:-
    • This is not working on beta3 bits. There is a bug in file Microsoft.TeamFoundation.Build.targets. You need to make the following change at two locations in above mentioned targets file.

From:-

<TestToolsTask

Condition = " '$(IsDesktopBuild)' ! = 'true' "

…

To:-

<TestToolsTask

Condition =" '$(IsDesktopBuild)' ! = 'true' and ' %(MetaDataFile.Identity)' ! = '' "

…

Do note that this is fixed in post beta3 sources.

    • If want to run additional tests defined in different vsmdi file using team build. You need to define new item for the corresponding vsmdi file. For example you want to run test of the category “BVT2” defined in HelloWorld2.vsmdi. You need to do the following steps:-

      • Modify tfsbuild.proj file and add new MetaDataFile item for HelloWorld2.vsmdi file

      <ItemGroup>

 

<MetaDataFile Include = "$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi">

        <TestList> BVT1 </TestList>

</MetaDataFile>

 

<MetaDataFile Include = "$(SolutionRoot)\HelloWorld\HelloWorld2.vsmdi">

        <TestList> BVT2 </TestList>

</MetaDataFile>

 

      </ItemGroup>

  • Running tests selectively for particular “platform|configuration” combination:-

     

    • Make sure you have the above mentioned fix of “Microsoft.teamfoundation.build.targets” file.
    • Modify tfsbuild.proj file. For example if you want to run the test for only “Debug|Any CPU” configuration, you need to modify the corresponding MetaDataFile item and add the condition (evaluate this item value only when "platform|configuration" is "Any CPU|Debug")

<MetaDataFile

            Condition = ” ’$(Platform)’ = = ’Any CPU’ and ‘$(Flavor)’ = = ’Debug’ ”

Include = "$(SolutionRoot)\HelloWorld\HelloWorld.vsmdi">

      <TestList>BVT1</TestList>

</MetaDataFile>

Comments

  • Anonymous
    October 13, 2006
    The comment has been removed

  • Anonymous
    December 28, 2009
    Hi Above sample Condition property work Please use below code <MetaDataFile Condition=" '$(Configuration)|$(Platform)' == 'Release|Any CPU' " Include = "$(SolutionRoot)HelloWorldHelloWorld.vsmdi">                       <TestList>BVT1</TestList> </MetaDataFile>

  • Anonymous
    January 25, 2010
    Hi, I set up the following in my .proj file: <MetaDataFile Condition="'$(Configuration)|$(Platform)' == 'Release|Any CPU'"                  Include="$(BuildProjectFolderPath)/NewConnectionPrj.vsmdi">                  <TestList>BVT1;BVT2;</TestList>    </MetaDataFile> i have also <RunTest>true</RunTest> but i receive the following error from msbuild: C:Program FilesMSBuildMicrosoftVisualStudioTeamBuildMicrosoft.TeamFoundation.Build.targets(1264,5): error MSB4062: The "TestToolsTask" task could not be loaded from the assembly PrivateAssembliesMicrosoft.VisualStudio.QualityTools.MSBuildTasks.dll. Could not load file or assembly 'file:///C:PrivateAssembliesMicrosoft.VisualStudio.QualityTools.MSBuildTasks.dll' or one of its dependencies. The system cannot find the file specified. Confirm that the <UsingTask> declaration is correct, and that the assembly and all its dependencies are available. Do you have a clue?