How To: Do image comparison in Coded UI Test

Article
04/08/2010

This is guest blog by my colleague Rajeev Kumar.

The Coded UI Test does not provide the API to compare images. There are cases when user wants to verify the changes in the UI by comparing the images of two controls. This can be easily achieved using Test API with Coded UI Test. The overview of Test API is described here and Test API visual verification is described here.

Getting the Test API

Download the Test API from here.
Add reference of TestAPICore.dll in the Coded UI Test project.
Add this using statement in your code- using Microsoft.Test.VisualVerification;

General Test API terms

Snapshot: A pixel buffer used for representing and evaluating screen image data.
Verifier: An object which determines whether a snapshot passes against specified inputs.
Actual: The snapshot being evaluated.
Master: The reference data (image) which is used to evaluate the actual snapshot.
Tolerance: The accepted bounding range based on which the actual snapshot will be accepted as valid.

The general Visual Verification workflow

Search the UI Test Control.
Capture the image of UI Test Control.
Generate an expected snapshot (e.g. load a master image from disk, etc.)
Compare the actual snapshot to the expected snapshot and generate the difference (diff) snapshot.
Verify the diff using a verifier.
Report test result.

Sample code to use Test API.

From the Test API copy these files to Coded UI Test:
- Samples\MSTest\Tests\bin\Debug\Master.png
- Samples\MSTest\Tests\bin\Debug\ToleranceMap.png
- Samples\MSTest\Tests\bin\Debug\SampleApp.exe
Add these files in the Coded UI Test project and make “Copy to Output Directory = Copy Always” in the property window for each files.
Now add this test method the Coded UI Test.

 [TestMethod]
[DeploymentItem("Master.png")]
[DeploymentItem("ToleranceMap.png")]
[DeploymentItem("SampleApp.exe")]
public void CompareImagesTest()
{
    // Launch the application.
    using (ApplicationUnderTest app = ApplicationUnderTest.Launch(
                Path.Combine(Directory.GetCurrentDirectory(),
                "SampleApp.exe")))
    {
        // Select the check box, so that the background will change.
        WpfCheckBox checkBox = new WpfCheckBox(app);
        checkBox.Checked = true;

        // Get the actual, master and toleracne snapshots.
        Snapshot toleranceMap =
                Snapshot.FromFile("ToleranceMap.png");
        Snapshot master = Snapshot.FromFile("Master.png");
        Snapshot actual = Snapshot.FromWindow(app.WindowHandle,
                WindowSnapshotMode.ExcludeWindowBorder);

        // Get the difference of two images.
        Snapshot difference = actual.CompareTo(master);

        // Save the file to for manual viewing.
        master.ToFile(@"Master-expected.png", ImageFormat.Png);
        actual.ToFile(@"Master-actual.png", ImageFormat.Png);
        difference.ToFile(@"Master-difference.png",
                            ImageFormat.Png);

        // Use SnapshotToleranceMapVerifier to verify the comparison.
        SnapshotVerifier verifier =
                new SnapshotToleranceMapVerifier(toleranceMap);
        Assert.AreEqual(VerificationResult.Pass,
                verifier.Verify(difference));
    }
}

Description of the code

Launch the sample application.
Check the check box on the sample allocation so that the app changes its background color.
Create the Snapshot object for master, actual and tolerance map.
Compares the actual image with the stored copy of master image.
Saves images files for manual diagnosis (in the test fails, then these images are useful)
Then verifies the differences of two images are within tolerance as given by tolerance map image.

This was guest blog by my colleague Rajeev Kumar.

Comments

Anonymous
April 08, 2010
Wow seems remarkably like my Mix demo...
Anonymous
April 08, 2010
Yes, that was an inspiration for this. We just built integrated scenario with CUIT on this.
Anonymous
June 09, 2010
Thanks for the post! One question, what is the process to generate a tolerance map?
Anonymous
July 12, 2010
Sorry for late reply. For some reason, my blog settings got messed up and I had trouble login in. In general, the Coded UI Test forum (social.msdn.microsoft.com/.../threads) is better place to post your questions as that is monitored by many folks & you will get faster response. Today you have to generate the tolerance map by hand. Please refer the TestApi documentation for this. Thanks, Gautam
Anonymous
July 27, 2012
Gautam I really appreciate your effort,help,dedication,understanding,time management.................I really really thankful to you and your understanding!
Anonymous
August 20, 2012
Hi Gautam, Can you please help in assisting how the ToleranceMap.Png gets created by hand. Thanks, Suman
Anonymous
August 22, 2012
Check the Test API documentation - http://testapi.codeplex.com/

Share via