A Lap Around TestApi v0.5
A few days ago we released the new version of TestApi – the API library for testing. My previous post from 6/18 summarized the additions and modifications to the library. This post goes a little bit deeper.
Documentation
In the TestApi package, we release both MSDN-style documentation (as CHM file and as a set of HTML files) as well as “conceptual documentation”. In TestApi 0.5, we spent a lot of effort cleaning up, updating and extending all of our documentation in an attempt to improve the usability of the library.
Theme Configuration API
TestApi v0.5 has a simple API for OS theme management. For example, you can easily list all available themes on the system and dump various attributes:
using Microsoft.Test.Theming;
...
Console.WriteLine("Current Theme:");
Console.WriteLine("\tName: {0}\n", Theme.GetCurrent().Name);
Console.WriteLine("Available Themes:");
foreach (Theme t in Theme.GetAvailableSystemThemes())
{
Console.WriteLine("\tName: {0}", t.Name);
Console.WriteLine("\tPath: {0}", t.Path);
Console.WriteLine("\tEnabled: {0}", t.IsEnabled);
Console.WriteLine("\tStyle: {0}\n", t.Style);
}
You can also switch to a desired theme programmatically, which is valuable when you need to verify appearance across themes:
Theme ot = Theme.GetCurrent();
try
{
Theme[] availableThemes = Theme.GetAvailableSystemThemes();
foreach (Theme t in availableThemes)
{
Console.WriteLine("Switching to theme \"{0}\"...", t.Name);
Theme.SetCurrent(t);
}
}
finally
{
Theme.SetCurrent(ot);
}
Revamped Input Injection API
In previous releases of the library, our input injection API had the peculiarity of being dependent on WPF types (specifically the Key and MouseButton enums). Apart from introducing an unnecessary dependency for tests that target WinForms, this had the annoying side effect of making the use of the API too verbose, i.e. in TestApi 0.4 you had to explicitly qualify Mouse and Keyboard, because you had to have dependencies on WPF namespaces which happen to also have similarly named classed in them:
using System;
using System.Windows.Input;
using Microsoft.Test.Input;
...
Microsoft.Test.Input.Mouse.MoveTo(new System.Drawing.Point(...));
Microsoft.Test.Input.Mouse.Click(MouseButton.Left);
...
Microsoft.Test.Input.Keyboard.Type("Hello, world!");
...
TestApi 0.5 fixes all of that. Now you can simply do:
using System;
using System.Drawing;
using Microsoft.Test.Input;
...
Mouse.MoveTo(new Point(...));
Mouse.Click(MouseButton.Left);
...
Keyboard.Type("Hello, world!");
...
Visual Verification API Additions
By popular demand, we have added support for searching for sub-images to Snapshot, so now you can do things like:
using Microsoft.Test.VisualVerification;
Snapshot s1 = Snapshot.FromFile("BigImage.png");
Snapshot s2 = Snapshot.FromFile("SubImage.png");
foreach (Rectangle r in s1.Find(s2))
{
Console.WriteLine(
"Found s2... Coords: ({0}, {1}); Size: ({2}, {3})",
r.Left, r.Top, r.Width, r.Height);
}
Snapshot.Find also has an overload which takes as a second parameter a SnapshotVerifer, so that you can do “fuzzy” searches.
Text String Generation API Extensions
TestApi 0.4 introduced the new text string generation API. However, what we had in TestApi 0.4 had an important limitation – you could only generate strings in one UnicodeRange at a time, i.e. you used to write the following code:
using Microsoft.Test.Text;
...
StringProperties sp = new StringProperties();
sp.MinNumberOfCodePoints = 10;
sp.MaxNumberOfCodePoints = 20;
sp.UnicodeRange = new UnicodeRange(UnicodeChart.BraillePatterns);
string s = StringFactory.GenerateRandomString(sp, 1234);
TestApi 0.5 lifts that restriction. Now you can generate strings that contain code points from multiple ranges:
using Microsoft.Test.Text;
...
StringProperties sp = new StringProperties();
sp.MinNumberOfCodePoints = 10;
sp.MaxNumberOfCodePoints = 20;
sp.UnicodeRanges.Add(new UnicodeRange(UnicodeChart.BraillePatterns));
sp.UnicodeRanges.Add(new UnicodeRange(UnicodeChart.Cyrillic));
string s = StringFactory.GenerateRandomString(sp, 1234);
Revamped Combinatorial Variation Generation API
The Combinatorial Variation Generation API underwent a major revamp in TestApi 0.5. We introduced support for generics, support for declarative models and moved to a LINQ-based syntax for constraints.
The example below demonstrates the basic usage:
using System;
using System.Collections.Generic;
using Microsoft.Test.VariationGeneration;
...
// Specify the parameters and parameter values
var os = new Parameter<string>("OS") { "WinXP", "Win2k3", "Vista", "Win7" };
var memory = new Parameter<int>("Memory") { 512, 1024, 2048, 4096 };
var graphics = new Parameter<string>("Graphics") { "Nvidia", "ATI", "Intel" };
var lang = new Parameter<string>("Lang") { "enu", "jpn", "deu", "chs", "ptb" };
var parameters = new List<ParameterBase> { os, memory, graphics, lang };
var model = new Model(parameters);
// The model is complete; now generate the configurations and print out
foreach (var config in model.GenerateVariations(2))
{
Console.WriteLine("{0} {1} {2} {3}",
config["OS"],
config["Memory"],
config["Graphics"],
config["Lang"]);
}
Often, one needs to apply constraints when generating variations. TestApi 0.5 constraints are now LINQ-based as shown below:
var de = new Parameter<string>("Destination") { "Whistler", "Hawaii", "Las Vegas" };
var ho = new Parameter<int>("Hotel Quality") { 5, 4, 3, 2, 1 };
var ac = new Parameter<string>("Activity") { "gambling", "swimming", "shopping", "skiing" };
var parameters = new List<ParameterBase> { de, ho, ac };
var constraints = new List<Constraint<Variation>>
{
// If going to Whistler or Hawaii, then no gambling
Constraint<Variation>
.If(v => de.GetValue(v) == "Whistler" || de.GetValue(v) == "Hawaii")
.Then(v => ac.GetValue(v) != "gambling"),
// If going to Las Vegas or Hawaii, then no skiing
Constraint<Variation>
.If(v => de.GetValue(v) == "Las Vegas" || de.GetValue(v) == "Hawaii")
.Then(v => ac.GetValue(v) != "skiing"),
// If going to Whistler, then no swimming
Constraint<Variation>
.If(v => de.GetValue(v) == "Whistler")
.Then(v => ac.GetValue(v) != "swimming"),
};
var model = new Model(parameters, constraints);
foreach (var vacationOption in model.GenerateVariations(2))
{
Console.WriteLine("{0}, {1} stars -- {2}",
vacationOption["Destination"],
vacationOption["Hotel Quality"],
vacationOption["Activity"]);
}
Another major addition in TestApi 0.5 is the ability to use declarative models, which simplifies the model definition syntax significantly. Users of the API apply attributes to a regular class to declare parameters and possible parameter values:
class OsConfiguration
{
[Parameter(512, 1024, 2048)]
public int Memory { get; set; }
[Parameter("WinXP", "Vista", "Win2K8", "Win7")]
public string OS { get; set; }
[Parameter("enu", "jpn", "deu", "chs", "ptb")]
public string Language { get; set; }
[Parameter("NVidia", "ATI", "Intel")]
public string Graphics { get; set; }
}
Then the model is formed and used as follows:
var model = new Model<OsConfiguration>();
foreach (OsConfiguration c in model.GenerateVariations(2))
{
Console.WriteLine("{0} {1} {2} {3}",
c.Memory,
c.OS,
c.Language,
c.Graphics);
}
The declarative syntax supports also equivalence classes, as well as parameter value weights. In the example below for example”Vista” and “Win7” are treated as values in same equivalence class. “Vista” or “Win7” will collectively appear more often than “WinXP” because of their larger weight value:
class OsConfiguration
{
[Parameter("WinXP", Weight = .3F)]
[Parameter("Vista", "Win7", Weight = .5F)]
public string OS { get; set; }
[Parameter("EN-US")]
[Parameter("JP-JP", "CN-CH")]
[Parameter("HE-IL", "AR-SA")]
public string Language { get; set; }
}
Fault Injection API Enabled for .NET 4 Binaries
The fault injection facilities in TestApi 0.4 did not work for .NET 4 binaries. TestApi 0.5 fixes that problem. TestApi 0.5 also ships all binaries necessary for fault injection as part of the package, to reduce dependency on the .NET installation on the target machine.
We have also made a minor naming change, renaming BuiltInConditions.TriggerEveryOnNthCall to BuiltInConditions.TriggerOnEveryNthCall.
Fixes and Additions to the Command-Line Parsing API
We have made significant modifications and fixes to the documentation of the command-line parsing API, expanding and fixing the MSDN-style documentation and the conceptual documentation to achieve consistency and correctness and to streamline the use of the API.
xUnit 1.5
As part of TestApi 0.5 we have switched to using xUnit 1.5. xUnit continues to be a fantastic unit-testing framework – both simpler and more full features than other comparable frameworks. xUnit 1.5 in particular provides a significantly improved GUI tool, which makes execution of the TestApi acceptance tests significantly easier.
+++
Most of the updates in TestApi 0.5 are under the cover, but in their entirety represent a significant update compared to TestApi 0.4. Let us know what you’d like to see in the future revisions of the library.
Attached is a solution with the sample code above.