Freigeben über

Assert.Inconclusive

  • Artikel

I'm currently using MSTest in my day to day work and I'm not always writing true unit tests but occasionally I'm also writing what I call integration tests. That is tests that uses a real database and/or web service. As such these integration tests are fragile and might fail because of the configuration on the machine running the tests or just because a service is temporarily down. Getting a test failure because of this is annoying and you really want to make sure that whoever sees the failure knows an occasional failure may be "OK". So in order to signal that the test didn't really fail but was unable to run as expected and intended I use Assert.Inconclusive. Basically if I'm relying on a web service I ping it if there is a failure. if the ping also fails I'll use Assert.Inconclusive to signal that the test can be ignored in the test run.

Comments

  • Anonymous
    January 21, 2011
    I think this is a way Assert.Inconclusive is meant to be used. I think including an informative Message, so the Tester knows which Problem to Fix to get the Test running, is very important.

  • Anonymous
    January 21, 2011
    I don't feel very comfortable with this idea. It feels like a bit of a cop-out for when we don't have the time to make sure our integration tests run in a reliable/repeatable manner.

  • Anonymous
    January 22, 2011
    @Matthew: Yes I agree it's a cop-out. And frequent inconclusive tests should be addressed and fixed. But when yo only get occasional failures due to environment glitches I think it's better to signal it as a known glitch than just failing. Naturally you can fail with a good message but my experience is that differentiating known glitches from actual failures leads to less stress in the team since it's easy to identify the "glitch failures".

  • Anonymous
    January 24, 2011
    Unfortunately, Assert.Inconclusive marks the build as partially failed when using TFS build and automated tests. In this case, there are two options:

  • make the tests more reliable (for example retry twice your ping).
  • remove them from release build (in my job, I run the tests locally in debug configuration and TFS build runs them in release configuration, so a few differences can be made)