Test Execution Switches

The following switches apply during test execution. Note that some switches are for test code generation (TCG) only, some for on-the-fly (OTF) testing only, and some can be used during either but may have different impacts.

Test Execution Switches – TCG and OTF

These switches are valid in both TCG and OTF. Note that the TestClassBase switch has different behavior in TCG and OTF.

Switch	Possible Values (Default Value Bold)	Description
LogProbes	none, string	A comma-separated list of probe names. For each probe name in the list, the corresponding probe value of each state will be inserted into logging statements in the generated test code. Probe names may be short (local scope) method/property names or fully qualified names. The word "none" here is a keyword.
ProceedControlTimeout	100, number	Time, in milliseconds, the test engine waits for observations before proceeding with a control action, if control actions are enabled.
QuiescenceTimeout	30000, number	Time, in milliseconds, until the implementation is assumed to be unresponsive at which point the test case will be marked as Failed.
TestClassBase	vs, string	This switch behaves differently in TCG and OTF. Class from which the generated test class or on-the-fly experiment will derive. TCG: When the value is "vs", the VsTestClassBase class in the Microsoft.SpecExplorer.Runtime.Testing namespace will be used as the base class, and all test-related attributes are VSTT test framework attributes unless users set other attribute switches to the desired value. Unlike this switch's operation under OTF, user-defined class bases need not be specified using a fully qualified name, although it is recommended that a fully qualified name be used. If a user-defined test class base is used, the test suite project must contain a reference to the assembly that contains the custom test class. OTF: When the value is "vs", the VsTestClassBase class in the Microsoft.SpecExplorer.Runtime.Testing namespace will be used. All test attributes for offline testing are ignored. If a user-defined class base is used, it must be specified using a fully qualified name. If a user-defined test class base is used, the modeling project must contain a reference to the assembly that contains the custom test class.

Test Execution Switches – TCG only

These switches are ignored during OTF testing.

Switch	Possible Values (Default Value Bolded)	Description
ClassCleanupAttribute	Microsoft.VisualStudio.TestTools.UnitTesting.ClassCleanupAttribute, string	Attribute used to annotate generated test code. This attribute is applied to the generated test class cleanup method. A fully qualified class attribute name must be provided.
ClassInitializeAttribute	Microsoft.VisualStudio.TestTools.UnitTesting.ClassInitializeAttribute, string	Attribute used to annotate generated test code. This attribute is applied to the generated test class initialize method. A fully qualified class attribute name must be provided.
ReRuns	1, number	Number of times a testing experiment is repeated. This number is relevant for on-the-fly testing or for testing of non-deterministic implementations.
SuppressGeneratedTestLogging	true, false	Whether to suppress the insertion of logging statements in generated test code.
TestAssemblyAttribute	none, string	Attribute used to annotate the test code assembly's namespace declaration.
TestClassAttribute	Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute, string	Attribute used to annotate program elements in generated test code. A fully qualified class attribute name must be provided.
TestCleanupAttribute	Microsoft.VisualStudio.TestTools.UnitTesting.TestCleanupAttribute, string	Attribute used to annotate program elements in generated test code. A fully qualified class attribute name must be provided.
TestInitializeAttribute	Microsoft.VisualStudio.TestTools.UnitTesting.TestInitializeAttribute, string	Attribute used to annotate the generated test initialization method. A fully qualified class attribute name must be provided.
TestMethodAttribute	Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute, string	Attribute that will be used to annotate each generated test method. This switch overrides the TestIDAttribute, TestIDPropertyName, and TestDescriptionAttribute switches. A fully qualified class attribute name must be provided.

Test Execution Switches – OTF only

These switches are ignored during TCG testing.

Switch	Possible Values (Default Value Bolded)	Description
OnTheFlyAllowEndingAtEventState	true, false	Controls whether Spec Explorer may end an OTF test in a state where one or more following events are expected. Setting this switch to true will allow tests to end at an accepting state without waiting for outgoing event steps.
OnTheFlyExperimentInconclusiveException	Microsoft.VisualStudio.TestTools.UnitTesting.AssertInconclusiveException, string	Defines the exception type that a user-defined OTF test framework may throw to indicate an inconclusive test result.
OnTheFlyMinimumPathDepth	none, number	Defines the attempted minimum path length before a nonterminal accepting state will be eligible to be considered as a candidate to terminate an experiment. Prior terminal accepting states will be allowed to end the experiment successfully.
OnTheFlyMinimumRequirementCount	none, number	Defines the required minimum number of requirements covered by all successful experiments in one OTF test run, before it will be eligible to terminate. The value of "none" means that there is no minimum requirement count.
OnTheFlyReplayStrategy	ApproximateReplay, Exact	Specifies the strategy to generate new experiments by OTF, when using a previous exploration result as guidance. ApproximateReplay indicates a heuristic to attempt to get close to the original with no guarantee of exactly reproducing the original experiment. Exact means to try to exactly match the original result.
OnTheFlySaveExperimentTrace	All, none, string	Specifies whether to save an experiment's exploration result in a .seexpl file. This is required for the experiment to be eligible for replay, or to be viewable by the Spec Explorer Exploration Viewer. All saves the exploration result for all experiments regardless of outcome. The word none, which is a keyword, saves no exploration results. Any comma-separated combination of one or more of Passed, Failed, or Inconclusive will save the exploration result for the corresponding type of test result.
OnTheFlySaveState	true, false	Defines whether to save state content to experiment's exploration result (.seexpl) during on-the-fly testing. Exploration result without state content can also be used for replay.

See Also

Reference

SwitchClause

Concepts

Cord Syntax Definition