Getting started with Unit Test Generation
Welcome
Hi I’m Michael Brisset a Quality Assurance Engineer for Visual Studio Team System. Welcome to CodeGen University! While we don’t have diplomas or faculty (or exams) we do have a pretty good football team. Over the next few months I am going to blog about the basics of generating unit tests, discuss some features to help maximize the value of these tests, and eventually discuss some of the advanced technologies used in this feature like the code model and reflection.
CGEN 101: Creating & Verifying simple unit tests
To begin we need an application. We have several options as Unit Test Generation supports C#, Visual Basic, and C++ /CLR:safe projects. Since my boss loves VB, let’s get started with a simple class written in Visual Basic:
1. Create a new Visual Basic Class Library.
2. Add a Reference to System.Drawing.
3. Paste the following code into Class1.vb:
Imports System.Drawing
Public Class MyFavoriteTeam
Dim Result As String
Public Function NextGame() As String
Return "at Pittsburgh"
End Function
Public Function Colors() As System.Drawing.Color()
Dim SchoolColors(2) As System.Drawing.Color
SchoolColors(1) = Color.Blue
SchoolColors(2) = Color.Gold
Return SchoolColors
End Function
Public Function Ranking() As Integer
Return 1
End Function
Public Sub LastGame(ByVal bowlGame As Boolean)
' Set to true if the caller wants the last bowl game reported.
If bowlGame = True Then
Result = "Loss in the Insight Bowl"
Else
Result = "Loss to USC"
End If
Console.WriteLine(Result)
End Sub
End Class
Our application has four important methods and we need to validate their correctness. Rather than write these tests by hand we will use Unit Test Generation to help us get started.
4. Place the cursor on the class declaration.
5. Right-click and choose Create Unit Tests...
6. A tree view will be presented with all of our classes and methods. By default items are checked relative to the position of the cursor in the editor. In this case, since the cursor was at the class level, all methods are checked. Hit the + box next to the Members node to display all of the methods in our class.
7. There is lots of functionality packed into this dialog. We’re going to leave all of that goodness alone for now and focus on the Output Project control. From here we can choose to generate tests into any language supported by test projects (C#, Visual Basic, and C++). If your solution already contains tests projects they will be listed here as well. For now let’s continue to use VB, select new Visual Basic test project if it isn’t already and hit OK.
8. Keep the default test project name and hit Create.
9. Look over the generated file, Class1Test.vb. We now have four new unit tests and we are well on our way to validating our application! Let’s walk-through each test method, examine what was generated and change them slightly to produce more meaningful tests.
10. Let’s start with the RankingTest. Our class is instantiated, variables representing our expected and actual values are declared and two Assert methods are generated. The first Assert method validates that the expected value matches the actual value. Essentially verifying that the function did its job. The final Assert method simply forces an Inconclusive result, this is a safety-valve that can be removed after the editing of the test method is complete.
We understand this function should return MyFavoriteTeam’s ranking. We had better check espn.com to see what the correct value is.
Hmm… there are three polls listed here. Which one should this function reference? Perhaps the spec is incomplete (or non-existent). I don’t see MyFavoriteTeam listed here. What is the proper behavior in this case? Should the function return 0 or -1? Or should it throw an exception?
Early unit testing will help flush out assumptions like these and can contribute to the definition of feature specifications early in the product cycle. We’ll expect a value of 0 in this case and remove the Inconclusive method. This unit test is now complete.
11. Next we’ll take a look at ColorsTest. Since this function returns an array, a simple Assert method is no longer sufficient. We need to validate the entire array, not just a single value! To support this, a different Assert method, a CollectionAssert was generated for us.
Everyone knows MyFavoriteTeam’s colors are blue and gold. We’ll modify the test method to expect these values:
12. Let’s move now to the LastGameTest. This method takes a parameter, which unit test generation will declare for us. There is also a TODO comment generated, reminding us to declare the variable appropriate for the test we want to author.
Notice the new text in the generated Inconclusive call. Unit Test Generation is reminding us that correctness for this method cannot be verified through a return value. Methods with no return value will still fail if an exception is thrown.
Only two small changes are necessary for this test. We’ll initialize bowlGame to True and modify the TODO comment to remind us to test the opposite condition in the future as well.
We will also keep the generated Inconclusive statement in this case since we are validating the function didn’t fail rather than explicitly validating correctness. In a later course we’ll validate the correctness of this method using unit test generation’s accessor model and remove the Inconclusive statement.
13. The final method to update is the NextGameTest.
Unit Test generation gave us a very good start. All we need to do is modify the expected value and remove the Inconclusive statement.
14. Now to the really fun stuff, let’s run our tests and see how our application (and unit tests) are performing.
Test -> Window -> Test View.
Select all the tests and hit the run button.
15. The Test Results window will come to focus and report the results of the tests. 1 Passed and 1 ended with Inconclusive as expected. But 2 tests have failed. Looking closer at the error message column gives a pretty good indication of what went wrong.
16. Troubleshoot. The Ranking function is returning 1 in cases when MyFavoriteTeam is unranked. That looks like a bug, and is confirmed by looking back at our implementation. Perhaps it was too optimistic to always assume MyFavoriteTeam is ranked #1? J
The results from the NextGameTest is more subtle. The data looks correct, but is simply in the wrong format. Perhaps we can change the Assert method to fix this…
Summary
Wow, we've covered a lot! To recap we have:
created our application.
generated unit tests for all of our application’s methods.
created a new test project.
discovered a design issue and application behavior that needs to be better defined.
used different Assert methods to validate application correctness.
modified parameters to functions and expected values to create meaningful tests.
executed tests through Test View.
analyzed results in the Test Results window.
Comments
Anonymous
June 29, 2005
Okay, I know I've been a slacker lately about aggregating Team System content (there's a lot coming,...Anonymous
June 29, 2005
Okay, I know I've been a slacker lately about aggregating Team System content (there's a lot coming,...Anonymous
June 30, 2005
Okay, I know I've been a slacker lately about aggregating Team System content (there's a lot coming,...Anonymous
July 04, 2005
The comment has been removedAnonymous
July 09, 2005
Articles
MSDN Chat: Team Edition for Software Developer and Software Testers, June 2005
Getting...Anonymous
June 09, 2009
PingBack from http://toenailfungusite.info/story.php?id=3580Anonymous
June 09, 2010
I have a situation where the class to be tested has a function that returns a list of objects. Like in the example, if there was another function GetAllTeams() that returns a generic list of "MyFavoriteTeam" objects, how could we instantiate MyFavoriteTeam object in the test? I'm having issues with instantiating the class or even just calling the function GetAllTeams(). It gives me an error as follows, "The type 'App_Code.MyFavoriteTeam' is defined in an assembly that is not referenced. You must add a reference to assembly 'App_Code, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'." Any help will be great. Thanks, Ashwanth.