Test Cases at Microsoft

Brian Lutz is a lucky contracting Tester in the building with my favorite cafeteria on campus, 110. He brings up a great comment about the role of Test Cases at Microsoft:

One thing I’d like to see is some discussion about test cases, and how they are written and executed. In talking with some colleagues outside of MS who have done ad hoc testing on their products, when I presented the idea of a test case and provided examples, it was virtually a new concept for them. It seems that in particular, people who develop software alone or with small groups where there are no designated testers may not really know much on the subject.

When I worked on a small MIS (Management of Information Services) team, I noticed the same thing as Brian. Many small (and even medium size) shops feel forced to rely on ad-hoc testing, instead of the more structured approach that we take at Microsoft. While ad-hoc testing can be a great way to have fun and find *some* bugs, it's not going to help you ship a world-class piece of software.

Here's a handy definition of a Test Case:

“A test case is a document specifying inputs, predicts results, and a set of execution conditions for a test item”.

That definition is short... sweet... and boring! Here's three reasons Test Cases can actually help you:

• It gives you a great starting point for writing automation. Writing test cases is a crucial first step before writing test code.
• Adding headcount before shipping a product? New members of your team will hit the ground running if they have test cases to start from! Mentoring a new hire quickly becomes easier.
• If you have a moment of ingenuity (like, “What happens when I launch my XP application in Win95 compatability mode?“), then you can ensure that the case is remembered later on when you might need it.

Now, a question for the people at home: Anyone have any test case horror stories? (I've got one that I'll add later : )

-Greg

Comments

• Anonymous
  March 25, 2004
  How about to "reveal" a few test cases as examples?
• Anonymous
  March 26, 2004
  Good idea- I'll do this in a future post.
• Anonymous
  March 26, 2004
  Not sure that I completely share your assessment of the B110 cafeteria (I tend to like the ones in building 9 and down at Sammamish better,) but thanks for posting on the topic. I find that especially in situations where you may be the only one working on something, you may not get much testing in beyond the standard "It doesn't crash, ship it!" phase. You're usually too busy programming things at the time, and don't want to spend a whole bunch of time checking it. If you're taking this approach, you almost guarantee that you will find bugs that you should have known about at the most inopportune times, usually after you think you've completed the project.
  
  One of the big problems that people have with writing test cases for their projects is that there needs to be some way to organize it. Here at MS, we have well-developed test case database tools, but a lot of smaller developers may not have the functionality for this available to them. I'm not familiar with any third party tools that would handle that, but if anyone does know anything about the topic, it may be another good idea for an entry?
  
  I'll post more later, when the topic comes up again.
• Anonymous
  April 09, 2004
  Test cases in my company are usually only written up at the client's request. That is, if we are releasing a fix or a new piece of software it will only be tested on the fly and only for the issue it is supposed to address. If the client wants a written test case so that they replicate the test then it gets done.
  
  So, a hint to anyone having to replicate or validate tests, if you're in the postition to do so, demand written test cases. Not only does it help a company internally, but it's benificial for anyone needing to repeate the test. More work but MUCH better code as a result.
• Anonymous
  June 14, 2004
  hjj
• Anonymous
  July 14, 2004
  There is growing concern about Test cases yeild - no of bugs that they find. what are your views on that. Have you heard of pesticide paradox for test cases?
  
  Copy your reply and views to shrinik@microsoft.com. I love to hear from you on this topic.
  
  Shrini K Test lead LCAIT India Hyderabd