Freigeben über

Extending Unit Tests in Visual Studio

  • Artikel

It’s funny how you walk around with a question on your mind for a couple weeks and then one day it just suddenly feels like the ‘right time’ to go look for the answer.

Question: Does VS 2010 allow you to extend the (nice) integrated unit testing to run tests in custom ways?
Answer: Yes! Actually it was easy to find a guide with a sample [RunAsUnitTest] extension on the VS Team Test blog. [RunAs = ‘‘run as user’)]

The scheme is: define your own custom [TestClass] attribute, and VS will load tests using whatever execution engine the attribute provides. The article looks like it dates back to Beta versions of VS, and maybe something has changed but there were a few issues I had with the article.

Here are a few minor and major issues:

  1. You probably only actually need to add one assembly reference: Microsoft.VisualStudio.QualityTools.UnitTestFramework – which your unit tests will reference anyway

  2. It tells you to install your assembly to Visual Studio’s ‘PrivateAssemblies’ folder. However, this did not work for me. I ended up using ‘PublicAssemblies’ folder instead. Without this, I would get the message

    Failed to initialize the unit test extension 'urn:microsoft.adp.unittesting.stackoverflow.asdf': A unit test extension is not registered for the following attribute: System.Active.Test.StackOverflowTest.StackOverflowUnitTestAttribute.

    Maybe this was just because the UnitTestFramework assembly it depends is also in the PublicAssemblies folder, and we need dependent assemblies in the same folder?

  3. The HKEY_LOCAL_MACHINE registry key for registering custom type seemed to have no effect

  4. The HKEY_CURRENT_USER registry key for registering custom type seemed to be required

  5. When you get errors that the attribute type ‘T’ cannot be loaded from the Assembly ‘X’ you are on the right track: it did find your assembly. But you have either put the wrong type name in the registry key, or you are missing some dependency.

OK, on to practical applications. Why did I even think of customizing unit tests in the first place?

The original trigger of thought that unit tests which fail by encountering Stack Overflow (note: you should never have a passing test case involving catching StackOverflow, because StackOverflow means your process state just got corrupted) are not handled so well by Visual Studio’s default test host: the Windows error reporting dialog will appear. Obviously I wouldn’t care if all our test cases passed, but we are not that far along in our new project.

So: the idea is creating a test extension that can handle the stackoverflow exception more elegantly (by e.g. exiting, instead of crashing)?

Writing up the classes following the example was easy. But the experience of registering the type hurt, as you can guess from the above. I would have liked having my test extension be allowed as part of my test assembly, instead of registered. The article mentions that this is necessary because the test type is being loaded by Visual Studio itself, but that seems rather unnecessary considering that my extension doesn’t provide any custom Client-Side functionality. Perhaps in a future version there could be a ‘light’ run-time-only extension which VS doesn’t need to load?

Also unfortunately - in practice seems it is nearly impossible to catch StackOverflowException no matter what you do, but that one probably deserves its own blog entry or further research…

Comments

  • Anonymous
    January 21, 2013
    It seems there is a way to avoid the registration issue... All I wanted to do was add some logging after finishing a UI test. So what I did, was:
  • Create my own TestClassExtension
  • Instanciate a CodedUITestAttribute, and forward its ExtensionId
  • Create my own TestExtensionExecution, and forward Initialize and Dispose to the CodedUITestAttribute
  • Implement the ITestMethodInvoker, call Invoke on the Invoker of the CodedUITestAttribute, and before returning the result, I write my own log messages So, basically, all I did was write a little wrapper for an existing TestClass attribute. My guess is: as long as the ID is recognized as something familiar, and the new attribute behaves at least similar as the element it wraps, it should work fine.

  • Anonymous
    November 11, 2013
    goppeltm, can you please past the extension code that works without the need to register the extension? thanks

  • Anonymous
    February 08, 2015
    Dear tilovell09 Almost 5 years have passed since you first published my bible "Extending the Visual Studio Unit Test Type, part 1" which I've found recently. I'm trying to make my own unit test extension in accordance with your guideline, but it does not work fine. My developing environment is Visual Studio Professional 2013 on Windows 8.1. So far, my VS seems to notice my test class attribute extension, as VS steps into the class to find ExtensionID property, but the test fails with the message "Failed to initialize the unit test extension 'urn:xxxxxxxxxxxx: A unit test extension is not registered for the following attribute: xxxxxxxxxxx" I checked registry hierarchy for a several times and found no errors in it. (Node structure is slightly different, as my environment is VS 2013 and OS is 64bit architecture. So the first half is as follows. [HKEY_LOCAL_MACHINESOFTWAREWow6432NodeMicrosoftVisualStudio12.0････] This theme seems have been issued not so many times since VS 2013 is release. I would be very much appreciated if you tell us if this guideline still effective in VS 2013 Professional, and also a next VS release 2015. thanks.

  • Anonymous
    February 16, 2015
    Sorry TrailRunner-MF, but this was written back in days of VS 2010 (or earlier?) and I haven't touched the space in years. When it comes to VS 2013 I have no idea if any of this still applies. I do have another idea though. There do seem to be a lot of extensions on the visual studio gallery these days, and maybe some of these are open source in ways which will help you figure this out. Good luck! Tim