Runtime Fault Injection using TestAPI
“Fault injection is a technique for improving the coverage of a test by introducing faults to test code paths, in particular error handling code paths that might otherwise rarely be followed” - wikipedia
TestAPI in its current release supports runtime fault injection. The API is pretty easy to use. It comprises of the following steps
Part\Step# |
API |
Signatureof the function where the fault is injected |
This is just a string |
Fault condition – specifies when to trigger the fault |
(static methods in BuiltInConditions) TriggerEveryOnNthCall(int n); TriggerIfCalledBy(string caller); TriggerIfStackContains(string method); TriggerOnNthCall(int n); TriggerOnNthCallBy(int n, string caller); |
Fault– determines what needs to be done when the condition is satisfied |
(static methods in BuiltInFaults) ReturnFault(); ReturnValueFault(object returnValue); ReturnValueRuntimeFault(string returnValueExpression); ThrowExceptionFault(Exception exceptionValue); ThrowExceptionRuntimeFault(string exceptionExpression); |
Compose theFaultRule |
FaultRule(string method); FaultRule(string method, ICondition condition, IFault fault); |
FaultSession– this is a collection of rules to be applied to the application that is being tested |
FaultSession(params FaultRule[] rules); |
Setup the FaultSession |
(method in FaultSession) public ProcessStartInfo GetProcessStartInfo(string file); //The file param is the path to the binary under test |
Start the test application |
Process.Start(ProcessStartInfo); |
In code, the above looks something like
string method = "TestApplication.DoSomething()";
ICondition condition = BuiltInConditions.TriggerOnNthCall(2);
IFault fault = BuiltInFaults.ThrowExceptionFault(new ArgumentNullException());
FaultRule rule = new FaultRule(method, condition, fault);
FaultSession session = new FaultSession(rule);
ProcessStartInfo processInfo = session.GetProcessStartInfo(@".\TestApplication.exe");
Process p = Process.Start(processInfo);
You can look at more usage scenarios in the docs that come along with the download from Codeplex.
While writing this post, I came across this interesting term Bebugging which involves knowingly adding bugs into code and then using # of bugs unidentified as an indicator of real bugs remaining.