Share via

Extending the Visual Studio Unit Test Type, part 2

  • Article

In part 1, we introduced Unit Test Type Extensibility, highlighting the main parts of the interface.  In this round, we will discuss three additional pieces:

  1. Adding a user interface
  2. Parameterizing the test method
  3. Aggregating test results
  4. UI extension

Due to the amount of code in this sample, I will paste some of it at the end of the blog.  The rest I will try to upload as a file to the blog.

The Finished Product

Lets begin again at the end.  “What do these three additional pieces give me?”

A snapshot seems appropriate here

image

Yes, I know, unit test should already have this built into the standard implementation, but since we have not done this, one is able to build a very quick implementation of it for your own testing.

Lets go through this code.

  1. The RowTestClass attribute tells the framework to load my test type extension to run these tests
    image
  2. We have a new custom attribute, called Row, that will allow me to pass in the data that I want to send to my parameterized test method
    image
  3. Now, with our test method being parameterized, we can use those variables directly in our test method instead of having to pull them off the DataRow in the TestContext
    image

Lets blaze on ahead with the different files.  I will quickly show the References, TestClassAttribute, TestExtensionExecution, and parts of the Invoker.  We will dive deeper into the invoker, aggregation and UI code.

Row Test Test Type Extension : References

First we need the references.  The Microsoft.VisualStudio.QualityTools.UnitTestFramework assembly can be found in the Global Assembly Cache, and the other three Microsoft assemblies can be found in PrivateAssemblies.  For more information, see Extending the Visual Studio Unit Test Type, part 1: Run As Extension : References section

 image

Row Test Test Type Extension : RowTestClassAttribute.cs file (quick view)

File number 1, RowTestClassAttribute.cs, is similar to the RunAsTestClassAttribute with the following differences:

image

  1. The Uri is unique to this attribute.
  2. In our GetClientSide implementation, we return a RowTestExtensionClientSide object. (see Row Test Test Type Extension : RowTestExtensionClientSide.cs file)

For more information, see Extending the Visual Studio Unit Test Type, part 1 for more information regarding the TestClassExtensionAttribute.

Row Test Test Type Extension : RowTestExtension.cs file (quick view)

File number 2, RowTestExtension.cs, is similar to the RunAsTestExtensionExecution.cs file and there only difference is the object type returned, which is a RowTestInvoker class (see Row Test Test Type Extension : RowTestInvoker.cs file (quick view) )

image

Row Test Test Type Extension : RowTestInvoker.cs file (quick view)

File number 3, RowTestInvoker.cs, is similar to the RunAsTestMethodInvoker.cs, with the difference being in the Invoke method.

image

The Invoke method is where it gets interesting.

  1. First we initialize our RowTestResults class.  This class is used to aggregate our results when we execute them.
    image 
  2. We then gather all the RowAttributes off the test method using reflection.image
  3. We then check the count of the RowAttributes we have.  If we don’t have any, we will invoke the base method passing a null as a parameter.
    image
  4. If we do find a list of RowAttributes, we will iterate through them and call the Invoke method, passing the row information.  We also aggregate them into our the RowTestResults instance that we created in step #1.
    image
  5. Finally we return the single aggregated results
    image

Row Test Test Type Extension : RowTestExtensionClientSide.cs file (quick view)

This file allows us to return the UserControl which will be displayed in the UI when the results details is requested.  It also has the icon we would like to use for our Test Type Extension.image

The Icon property returns an embedded Icon that I placed in the Resources for the project.

The ResultExtensionViewer returns the user control that we have created to display our results.

Row Test Test Type Extension : RowTestExtensionViewer.cs file (quick view)

This is a user control, so it also has a .Designer.cs and a .resx file with it.  However, the most important is the implementation of the User control.  It must implement ITestTypeExtensionResultViewer

image

Of note, the Initialize function is what makes this work.

  1. The initialize takes a TestResult argument.  This is will contain quite a bit of information regarding outcome, error, stack trace and other items you may need for that result.  But keep in mind, this is the overall result.  You can call it the result container.
    image 
  2. For our purpose we need to get a little piece of information off it regarding the ITestResultExtension.  So we cast it, check that it isn’t null and dump the results to it (since we returned a string earlier (see Row Test Test Type Extension : RowTestInvoker.cs file – Step #5).
    image
    This line also assumes that we have placed a label on our UserControl.

And that is it.  A little more work but now we are ready to deploy

Row Test Test Type Extension : Registering your test type extension

To make it all work you now need to place your test type extension assembly in the correct location and register your test type in the registry.

Location:  The location of your test type, and your PDB if you want, needs to be in the PrivateAssemblies location; the same one that contains the assemblies that we referenced in the beginning

clip_image001

Registry: The test type needs to be in the following key:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\VisualStudio\10.0\EnterpriseTools\QualityTools\TestTypes\{13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b}\TestTypeExtensions\RowTestClassAttribute]

The registry entry is a string called “AttributeProvider” and will contain your attribute and the name of your assembly.
"AttributeProvider"="DemoExtension.RowTest.RowTestClassAttribute, DemoExtension"

image

There is a second location that is usually populated when you start Visual Studio.  It is located under the following key: (NOTE the bolded text for the differences in the two keys)

[HKEY_CURRENT_USER\SOFTWARE\Microsoft\VisualStudio\10.0_Config
\EnterpriseTools\QualityTools\TestTypes\{13cdc9d9-ddb5-4fa4-a97d-d965ccfc6d4b}\TestTypeExtensions\RowTestClassAttribute]
"AttributeProvider"="DemoExtension.RowTest.RowTestClassAttribute, DemoExtension"

I place the second key in manually because sometimes it doesn’t do it for me straight away and I get impatient and cannot wait for the key to be updated.  The full registry file can be found at the end of the blog as RowTestExtension.reg

FINAL REGISTRY NOTE: If you are on an x64 machine, the registry key will be under the Wow6432Node as in HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft…

FINAL LOCATION NOTE: If you are on an x64 machine, the location key will be under the Program Files(x86) folder.

WORKING WITH VS NOTE: Because the test type is loaded within Visual Studio, developing the test type in addition to testing it poses problems when you need to update the test type extension assembly.  To make this work so you can develop in one instance of Visual Studio and test in a separate instance of Visual Studio, you can use the Visual Studio Development Experimental Model found here

Summary

Good Luck!  Let us know if there are any questions or thoughts around this.  Hopefully extending the unit test type will be far easier than doing it from scratch like in Visual Studio 2008.  Doing a test type from scratch is still supported and sometimes needed, but if you can get along by using the test type extension, things will be much easier.

Bruce Taimana
Program Manager
Visual Studio Team Test

Full Code

We now have our SDK available and code can be found here when it goes live. 

The download is found on code gallery here.

Comments

  • Anonymous
    April 08, 2010
    it is good to see MSTest following the lead of other frameworks: http://xunit.codeplex.com/wikipage?title=Comparisons I'm surprised you did not follow Microsoft's own other unit testing framework, xUnit.net, by Jim Newkirk and Brad Wilson, and call your "Row" "InlineData" which imo is more meaningful. http://www.codinginstinct.com/2008/10/funky-xunit-theory.html http://xunit.codeplex.com/Thread/View.aspx?ThreadId=204257 Also, I hope you've got your test counting more accurate (see the link immediately above this one). Your SDK link does not work:  http://go.microsoft.com/fwlink/?LinkId=185591. Regards ~~ Gerry (Lowry)

  • Anonymous
    March 13, 2012
    Will this functionality be implemented in the future version of Visual Studio. Does Visual Studio 11 beta improve on this functionality in any way? Our team doesn’t want to really use an external testing framework like NUnit just so we can get this functionality.

  • Anonymous
    October 07, 2012
    your sample does not work in vs2012. Please give an approach for implementing your extension for visual studio unit test type in the vs 2012.

  • Anonymous
    December 06, 2012
    I got this to work in VS 2010 but only with c#.  In VB I  get an message UTA007: Method AddParams defined in class SampleVBTestProject.SampleVBTest does not have correct signature. Test method marked with the [TestMethod] attribute must be non-static, public, does not return a value and should not take any parameter. for example: public void Test.Class1.Test(). It compiles but it's not runnable as a unit test method.  Is there a way around this?

  • Anonymous
    May 05, 2013
    Got part of it working in VS 2012. However the UI Extensibility part seems to be deprecated along with the Test Results. What is the new counterpart to this ?

  • Anonymous
    July 24, 2013
    +1 to "UI Extensibility part seems to be deprecated along with the Test Results. What is the new counterpart to this ?"

  • Anonymous
    September 19, 2013
    +1 to "UI Extensibility part seems to be deprecated along with the Test Results. What is the new counterpart to this ?"

  • Anonymous
    April 09, 2014
    Is there a link (that works) that points to how to download the actual code for this? Not a blog, or a help reference, an actual link to download this actual code. I'm astounded that this STILL hasn't made it into VS even as of VS 2013 Update 2. Well, there is the same functionality, but it only applies to Windows 8 apps. Adding a reference to the Microsoft.VisualStudio.TestPlatform.UnitTestFramework assembly SEEMS like it's going to work, the test is even discovered by the test runner. But running the test results in a blue exclamation icon with absolutely NO information about what's wrong.

  • Anonymous
    March 01, 2016
    Hi The links for code do not seem to be working anylonger. Is it possible to get code for this?