Test report steps are out of sync

Important

Visual Studio App Center is scheduled for retirement on March 31, 2025. While you can continue to use Visual Studio App Center until it is fully retired, there are several recommended alternatives that you may consider migrating to.

Learn more about support timelines and alternatives.

App Center Test creates test reports that synchronize each test and test step across devices used. To organize the reports, Test relies on the filenames and order of the screenshots it takes.

If your test suite runs on multiple devices, and allows devices to take different paths through the test code, test steps might appear out of order, skipped, or duplicated in the report. The report summary doesn't count tests with these symptoms as failures.

The following pseudocode examples demonstrate this issue and a workaround.

Pseudocode to reproduce the issue

This example creates discrepancies in the test report if both of the following conditions apply:

• The test is running on multiple devices.
• At least one device takes each code path.

if(bool)
{
    // code to run if true
    app.Screenshot("True");
} else 
{
    // code to run if false
    app.Screenshot("False");
};

Pseudocode workaround

In the following pseudocode, since both code paths create the same screenshot names and sequence, the test report can reconcile both paths to the same test step.

if(bool)
{
    // code to run if true
    app.Screenshot("Result");
} else 
{
    // code to run if false
    app.Screenshot("Result");
};