Test Cases and Test Suites
In traditional testing, tests are structured into test suites that serve as containers of test cases. A test case is a unit of partial testing derived from the exploration with the selected traversal criteria. Each test case should be considered an incidental side-effect of the test-generation process. Only the entire test suite has meaning for testing a scenario. Traditionally, for manually written test cases, each test case is the smallest meaningful unit; that is, each test case is its own scenario.
A successful test case must always end in an accepting state which reflects that enough actions have taken place to demonstrate that a test purpose has been achieved. However, a test case may choose to continue from an accepting state so long as it ultimately ends in one. In model programs, all states are accepting by default unless an accepting condition is specified, in which case the accepting states are only those for which the accepting condition is true or the mode list for the AcceptingModeAttribute is a subset of the state's mode. In a scenario, only the last state of the behavioral expression is considered accepting.
Spec Explorer supports three test modes: test code generation (TCG) mode, dynamic traversal (DT) mode, and on-the-fly (OTF) mode.
In TCG mode, the smallest meaningful unit is a test suite. This approach is one level higher than in traditional, manually created testing. A test suite is a set of test cases derived from one action machine created by translating the exploration results into test cases according to the selected traversal algorithm with test generation criteria. Each action machine represents one test purpose, which is a slice of the model that has been selected for testing. This action machine in turn drives the generation of the corresponding test suite.
DT mode is a special case of TCG mode. A full test suite is not provided; rather the transition system, delegates, and so forth are provided to a class that implements the IDynamicTraversal interface. The user can then either use the default dynamic traversal class or create their own. With DT, a test suite will provide the same coverage as the test suites generated with TCG, but DT allows re-execution of test cases once the initial coverage is complete.
In OTF mode, an experiment is a single trace of interactions with the system under test (SUT), roughly similar to a TCG test case, although the details are quite different. A test run is a collection of experiments generated against the same model, roughly similar to a test suite, although, again, the details are quite different. Experiments and test runs are still derived from one action machine, and thus a test run satisfies a test purpose.
When you make changes to a model and generate new test cases, you might want to be able to tell whether some of these test cases are the same as some of the test cases generated from a previous version of the model. To assist you in this task, Spec Explorer can compute a hash code for each test case from the sequence of action invocations that make up the test case. For more information, see Test Case Identifiers.
See Also
Reference
AcceptingModeAttribute
IDynamicTraversal