次の方法で共有

Automated Testing in VSTS / TFS 2010

  • [アーティクル]

I thought I’d make a brief post giving an overview of what automated testing looks like in our 2010 release.  While it’s true that in this release we focused primarily on nailing the experience for the manual tester there is quite a bit of carryover from those features that also apply to automated testing, and as we do quite a bit of automated testing ourselves and dogfood our own product we did end up working on the automated testing experience as well despite that heavy focus on the manual test side of things.

Since the 2005 release we’ve had several types of tests that you could create in a Visual Studio Test Project, they included Unit Tests, Web Tests, Load Tests, etc.  In those versions you could run your tests locally through Visual Studio or MSTest, or you could run them against a remote controller and agent from either of those clients.  When your tests were complete you could publish them against a TFS Server / Project / Build and then use the various reporting mechanisms in the product to view the test results.  They would also show up on the Build Summary Report. 

In the 2010 release we’ve added a ton of new test features.  We now have first class support for Test Case work items, and tooling for creating Test Plans, organizing your test cases into test suites, tracking which test configurations you’ve run against, etc.  There are tons of blog posts covering these new features so I won’t go into a lot of detail here on them, instead I’ll focus on how those automated test types fit into this new landscape of test objects.

Each Test Case can have associated automation, this creates a reference from the Test Case work item to the actual automated test, be it a Unit Test, Web Performance Test, Coded UI Test, or whatever.  When you add that Test Case to one of the Test Suites in your Test Plan and run it, it will look for the referenced test on the build share for whatever build you’re running against.  When you create this association between your Test Case work items and your VSTT tests you can then track your test progress within the context of your Test Plan and/or Test Suites and publishing / saving the results is all handled for you without the need for a separate publish operation.

If you aren’t using Team Build to generate your builds yet then you’ll need to use “Fake Builds” that your test plan can reference and use another process to copy your test assemblies / files to the drop location you specify on the fake build, that way when you run your automated tests through MTM it will know where to find the associated test files.  See my earlier post Creating Fake Builds in TFS 2010 for more information on that.  Alternatively you can use tcm.exe to run your tests and point to any share containing your test assemblies / files.

In the most basic scenario you would create an Environment (physical in this case, but there are also virtual environments) that specifies a Test Controller to run your tests against.  When you kick off an automated test run from MTM you’ll have specified the environment to run tests on as well as the build you are running against.  The test assemblies / files from the build you specified will be sent over to the controller & agent much like they were in the previous versions, they’ll be run there, and the results will be posted back to the new system.  From within MTM you can then drill into failed tests and even open up the corresponding trx’s into Visual Studio.

All of this allows you to use all of the new test planning, organization, execution, and tracking features that we’ve added in 2010 with the automated test types that you’ve been producing throughout our earlier releases.  TCM.exe provides some features that allow easier importing of these tests so that you don’t have to generate Test Case work items for each individual test case manually one by one.  It also allows you to easily keep a Test Suite in sync with a unit test assembly for example without having to manually account for added or removed unit tests as you move through the product cycle.

For more details on this refer to this MSDN article.

Comments

  • Anonymous
    September 20, 2010
    Hi,  Is there any way to link web scripts to the test cases? Thanks,

  • Anonymous
    January 31, 2011
    The comment has been removed

  • Anonymous
    March 31, 2011
    Is there any way to import test scripts from test partner to TFS 2010

  • Anonymous
    April 06, 2011
    Please guide me to do step by step procedure to perform Automation testing in TFS 2010