Test Types

The recent discussion spawned by my former post has prompted me to write this entry about test types. There's not a single clear definition of what a unit test is, so sometimes people approach me with unit testing challenges that don't really relate to unit testing per se. The fact that you can use unit testing frameworks (such as Visual Studio 2005 Team System or NUnit) to drive not only unit tests, but also other types of tests, confounds the issue.

Although some people find such categorization semantic (or even pedantic), I distinguish between the following test types:

• Unit tests subject a single unit to various stimuli in isolation from its volatile dependencies. In my terminology, when talking about .NET, a unit corresponds to an assembly, although other people view a unit as corresponding to a specific type. Viewing a unit as a single type is, in my opinion, too specific, as it effectively prohibits scenario-driven (or agile) development methods, since you will not be able to test the interaction between several types in the same assembly.
  Isolation from volatile dependencies is essential, because it enables you to develop your unit without having to rely on its dependencies.
• Integration tests cover a wide variety of scenarios where you connect two or more units into an integrated subsystem. This covers the simple case where two units are used together in the same process; e.g. if you want to test the interaction between a UI controller unit and a domain logic unit.
  Integration tests can also span process or even machine boundaries, which is the case if you test a data access component against a relational database system, or if you test a web service by making requests against it using a proxy class.
  Sometimes I wish the terminology would allow us to distinguish between the in-proc and out-of-process scenarios, but they are all integration tests.
  Since integration tests involve the interaction of several moving parts, they are typically more difficult to set up to run as fully automated tests, although unit testing frameworks such as Visual Studio 2005 Team System are well suited also for this task.
• System tests are by some also called acceptance tests. These test a system in its entirety, by using its intended interfaces - typically its user interface.
  Since the entire application, including user interfaces, is being tested, it is typically very difficult to run system tests in a fully automated fashion, because there is often much deployment and configuration to be done.

As you can see from the comments to my earlier post, people sometimes feel they have problems with unit tests, when in fact they have problems with integration tests. Integration tests are more difficult to configure for full automation, which is why I recommend that you try to test as much of your code as possible with unit tests.

For example, instead of testing a web service with an integration test where you make requests against the service using a proxy class, you should impement the service logic in a testable unit (i.e. a library), and then unit test that logic. Exposing the service as a web service will then be a simple case of implementing the Remote Façade design pattern, and you probably will not need to test the web service at all, because its implementation will be very simple.

That is not to say that I don't recommend integration or system tests - these can provide valuable insights into the interaction of multiple components that would be difficult to reproduce otherwise. Because integration and system tests are more difficult to automate than unit tests, you can get more value for your effort early in the development process with unit tests, but as you approach project completion, you need to begin testing the complex interactions.

Comments

• Anonymous
  October 06, 2006
  The week after speaking at TechEd , I'll be giving a talk at .NET Days 2006 in Århus. In this talk ,

• Anonymous
  November 15, 2006
  As I've written before , integration tests are different from unit tests, but that does not mean that

• Anonymous
  November 17, 2006
  In a typical n-layer enterprise application, there's almost always a data access layer with one or more

• Anonymous
  November 17, 2006
  In a typical n-layer enterprise application, there's almost always a data access layer with one or more

• Anonymous
  December 03, 2006
  A few years ago, when ASP.NET web services were the only (or at least most common) implementation of

• Anonymous
  December 04, 2006
  In my previous post about unit testing WCF services , I hinted at the need to perform integration testing

• Anonymous
  January 19, 2007
  PingBack from http://www.desertpenguins.com/PermaLink,guid,6300e7d7-c4fd-4bd2-8659-3cf04f684f07.aspx

• Anonymous
  September 15, 2007
  If at all applicable, a well-written application should include one or more performance counters, which

• Anonymous
  September 15, 2007
  If at all applicable, a well-written application should include one or more performance counters, which

• Anonymous
  September 15, 2007
  PingBack from http://msdnrss.thecoderblogs.com/2007/09/15/performance-counter-testing/

• Anonymous
  January 20, 2009
  PingBack from http://nirajrules.wordpress.com/2009/01/20/unit-testing-links/