Model Based Testing – An Introduction
In this article I will attempt to give a brief overview of Model Based Testing (MBT) and provide links to help you dig in for more details. I have been increasingly getting more and more fascinated (obsessed?) with this paradigm, which I believe holds great promise for the future of the software testing discipline. Hope to rub some of that passion for MBT on to you.
MBT is an approach in which you define the behavior of a system in terms of actions that change the state of the system. Such a model of the system results in a well-defined Finite State Machine (FSM) which helps us to understand and predict the system’s behavior. For e.g. when working with a model of a car, the action of “press accelerator” has a different result depending on whether the state of the car is “neutral” or “drive”.
The biggest MBT take-away from a testing perspective is that any test case that you define is really just a traversal of the model that you have of the system – either an explicitly defined one or the one in your head. Generating test cases thus boils down to traversing this state machine. MBT is therefore especially useful for automatic generation of test sequences from the defined model. Various graph theory algorithms can be employed to walk this graph (e.g. shortest path, N-states, all-states, all-transitions etc.). The best part is that when you are using a good MBT tool, you do not need to know or dig into all this graph theory all the time. Simply define the states, actions, transitions etc. for your model and click a button to start generating tons of interesting test cases.
Model based testing offers a lot of benefits, some of which I list here. I shall expound on some of these in later posts.
1. Forces detailed understanding of the system behavior
2. Early bug detection (which is much cheaper)
3. Test suite grows with the product
4. Manage the model instead of the cases (useful when features changing constantly).
5. Can generate endless tests (since test cases are machine generated)
6. Resistant to pesticide paradox
7. Find crashing and non-crashing bugs
8. Automation is cheaper and more effective
9. One implementation per model, then all cases free (test code is better enabled for changes).
10. Gain automated exploratory testing
11. Testers can address bigger test issues
Any other benefits that you think it brings to the table? I would love to hear from anyone actually using model based testing outside of Microsoft, and what their experience has been using MBT.
If I had to recommend just one and only one short introduction to Model Based Testing in order to sell the idea and get you hooked to it, I would point to this paper, Intelligent Test Automation, by Harry Robinson, which appeared a few years ago in the STQE magazine (which I believe has morphed into the Better Software magazine). Interestingly on this page, Harry has put up the 2-file state model of the example in the paper and also a Chinese Postman traversal for the model.
Here is an extensive list of technical papers related to model based testing that can help you dig into the details and the theory behind this super powerful testing paradigm.
One great publicly available tool you can use for Model Based Testing is Abstract State Machine Language(ASML) produced by the Foundations of Software Engineering(FSE) group Microsoft Research(MSR). You can actually have executable specifications with ASML, so when your Program Manager generates the functional specs, you can simply execute that spec and generate test cases directly from the spec. Buckle up - The future is here (almost)!
Comments
Anonymous
May 30, 2004
Outstanding. I am also in love with these ideas, and even have a few to contribute. You are one of my newest favorite people. Subscribed! -JB.Anonymous
July 25, 2004
Moreover, with .Net reflection, you can easily explore an application and build the model "on the heap"...Anonymous
July 31, 2005
I am in love with this concept since 2001.
Would like to share thoughts with this interest group.Anonymous
February 06, 2006
Its great! my research area is MBT.It is quite interestingAnonymous
February 06, 2006
My research topic model based testing and I will delighted to get more information about itAnonymous
May 06, 2006
In a past post I talked about the basic functionality of a test harness.  That is, it should be...Anonymous
June 12, 2006
Nihit,
Good article for giving a insight into the exciting area of MBTAnonymous
July 28, 2006
The future was here – now it’s the past. I am, however, wiser about the ways-of-the-model-based world....Anonymous
August 11, 2006
In this article I will be attempting to explain one of the biggest (or what I feel is the biggest) drawback...Anonymous
January 03, 2007
You know your team members are really catching onto this whole model-based way of thinking when theyAnonymous
January 19, 2007
You know your team members are really catching onto this whole model-based way of thinking when theyAnonymous
March 23, 2007
In this series of "Elements of MBT" posts, I will be covering some basic concepts used when trying toAnonymous
May 22, 2007
pleae reade the book Practical Model based testing from prof Bruno Legeard and Mark utting isbn-10:0-12-372501-1Anonymous
May 22, 2007
Hi Josef, I am going through this currently - very interesting. Thanks, NihitAnonymous
September 10, 2007
HEY I WOULD LIKE TO HEAR MORE ABOUT IT, I WAS JUST LOOKING FUTURE IN MODEL BASED TESTING.Anonymous
December 02, 2007
just outstanding. helped me alot writing my assignment about model-based testing. nice work buddyAnonymous
April 23, 2008
Hi, For information, All4Tec MaTeLo is a tool for MBT. MaTeLo, new tool for validation using the statistical test usage. MaTeLo test your software under the real conditions, just like will do it your customer. Thanks,Anonymous
May 22, 2008
I also agree that this is the future of testing, and even i can say that modeling and MDE is the future. the main problem with these approaches is that the tools and languages for modeling are still insufficient in complex examples. the modeling part is the only negative part of this, and by negative i mean hard to achieve. other than that this model based approaches in SE seem only natural.Anonymous
June 07, 2008
In this article I will attempt to give a brief overview of Model Based Testing (MBT) and provide links to help you dig in for more details. I have been increasingly getting more and more fascinated (obsessed?) with this paradigm, which I believe holdAnonymous
March 27, 2009
I have to agree with one of the comments: The future is here! TestOptimal offers 4 test case generators, supports both java and xml scripting, cross browsers, app modeling, load / stress testing, debug, statistical analysis, lots of other features. Check it out at http://TestOptimal.com/.Anonymous
December 01, 2009
You might want to take a look at Spec Explorer, a Model-Based Testing tool from Microsoft, which you can download free of charge: http://msdn.microsoft.com/en-us/devlabs/ee692301.aspx. There are some general MBT articles in the Spec Explorer Team Blog too: http://blogs.msdn.com/specexplorer/.Anonymous
June 01, 2011
If anyone is interested, I'm running a blog on Model-Based Testing from an applied perspective. Besides sharing my experience with MBT through examples, I explore some of the pros and cons of Model-Based Testing. You can find it here: http://appmbt.blogspot.com/