Windows Phone Store Test Kit for Windows Phone 8
[ This article is for Windows Phone 8 developers. If you’re developing for Windows 10, see the latest documentation. ]
This topic contains the following sections.
What the Store Test Kit tells you
Running the Store Test Kit on your app will help you determine whether or not the app will pass Store certification. If a test fails, the Store Test Kit provides details about what you need to fix in the app before you submit it for Store evaluation. The following lists some of the tests in the Store Test Kit.
Whether the XAP file meets size requirements and whether the app manifest file is valid.
Whether a Direct3D app that targets Windows Phone 8 uses APIs that are not allowed on the phone.
Whether a background agent app uses APIs that are not allowed with background agents.
What capabilities the app uses (for apps that target Windows Phone OS 7.1 only).
Whether the specified images and screenshots meet certification requirements.
Whether the app icon and background image used in the app meet certification requirements.
Opening the Store Test Kit
The Store Test Kit is integrated within Visual Studio. Use the following steps to open the Store Test Kit.
In Visual Studio, open a Windows Phone app solution that targets Windows Phone OS 8.0 or Windows Phone OS 7.1.
In Solution Explorer, select the project you want to test.
On the Project menu, choose the Open Store Test Kit option.
-or-
Right-click the project to show its context menu and select the Open Store Test Kit option.
The Store Test Kit opens in a new Visual Studio tab named Store Test Kit. To see the options available in the Store Test Kit, see the illustrations later in this topic.
At the bottom of the Store Test Kit tab, you may see an alert indicating that the test cases have been updated. The following image shows the update prompt.
.png)
If you see the update prompt at the bottom of the Store Test Kit tab, click the Update button. After you update the Store Test Kit, close and restart it before you continue.
Using the Store Test Kit
The Store Test Kit has multiple pages that contain app details and test categories. Each test category lists a series of tests with a name and description. The tests are categorized by whether they can be run automatically or whether they need partial or full participation from a developer. The Store Test Kit has the follow pages:
Application Details
Automated Tests
Manual Tests
The following sections describe these pages and how to run the tests.
Because the Store Test Kit opens in a new Visual Studio tab, you can easily switch between the Store Test Kit and the files in your app.
Note
All tests will be run on the project that was selected when the Store Test Kit was opened. You can run multiple instances of the Store Test Kit that target different projects at the same time. If you want to open the Store Test Kit for a different project, in Solution Explorer, right-click the project to show its context menu, and then select the Open Store Test Kit option. Otherwise, to change the target project, you must close the Store Test Kit, select the other project, and then reopen the Store Test Kit.
App details
The Applications Details page lists the app package and lets you specify images that will be evaluated later in the testing procedure. The following image shows the Application Details page in the Store Test Kit.
.png)
The Application package box lists the path to the app package (XAP file) for your app, which by default is the Bin\Release folder of the currently selected project. This means you must use a release build to pass the automated tests.
The Applications Details page also lets you specify the images that will represent your app in the Store. These images will be tested as part of the Store Test Kit. If you do not specify the required images, your app will not pass the tests in the Store Test Kit. The following table lists the images that you can specify.
Note
Be aware that the image requirements can change. Be sure to test your app with an updated version of the Store Test Kit, and check the certification requirements on MSDN. For more information about image requirements, see App submission requirements for Windows Phone.
Image |
Size (pixels) |
Format |
Required |
|---|---|---|---|
Store Tile |
300 x 300 |
PNG |
Yes |
App screenshots |
|
PNG |
Yes |
Up to 7 additional app screenshots for each screen resolution |
|
PNG |
No |
For more information about the artwork required to submit an app, see Upload and describe your package(s).
You can use the screenshot tool provided by the emulator to easily obtain screenshots of your app that are the correct size and resolution. For more information, see How to create screenshots for the Store for Windows Phone 8.
Automated tests
The Automated Tests page provides tests that evaluate the basic criteria of your app, such as app size, and the presence and size of app images.
The Capability Validation test is available only for apps that target Windows Phone OS 7.1.
The Automated Tests page also provides a link to the App Monitoring feature. With app monitoring you can capture all the key metrics about the quality and responsiveness of your app. For more information, see App monitoring for Windows Phone 8.
The following image shows the Automated Test page in the Store Test Kit.
.png)
Note
Be aware that certification requirements can change. Be sure to test your app with an updated version of the Store Test Kit, and check the certification requirements on MSDN. For more information about the certification requirements for the automated tests, see App submission requirements for Windows Phone.
To run the automated tests
In the Windows Phone Store Test Kit, on the Application Details page, make sure the additional resources for your app are specified.
Create a release build. For more information, see How to build an app for Windows Phone 8.
You must use a release build to pass the automated tests.
Click the Automated Tests page.
The automated tests display.
Click the Run Tests button.
The automated tests start. When the tests complete, you will receive a pass or fail result for each test. You will also receive a description of the test and any error messages that are relevant.
If a test fails, use the provided information to fix the issue and run the tests again. You should repeat this process until your app passes all the tests in this section.
Manual tests
We recommend that you complete the series of test cases on the Manual Tests page with your app running on a Windows Phone device. The manual tests require you to navigate through your app and observe its behavior in several different conditions to make sure it meets the app certification requirements. The following image shows some of the Manual Tests page in the Store Test Kit.
.png)
Note
Be aware that the certification requirements can change. Be sure to test your app with an updated version of the Store Test Kit, and check the certification requirements on MSDN. For more information about the certification requirements for the manual tests, see Technical certification requirements for Windows Phone.
To run the manual tests
In Visual Studio, create a release build. For optimal results you should also specify Windows Phone Device as the deployment target. For more information, see How to build an app for Windows Phone 8.
On the Standard toolbar, set the target device. For optimal results you should specify Windows Phone Device.
In the Windows Phone Store Test Kit, click the Manual Tests page.
The manual tests appear.
For each test, follow the steps provided in the Test Description column.
Observe your app’s behavior and change the drop-down in the Result column to Passed or Failed depending on the results you receive.
If a test fails, use the provided information to fix the issue. You can get more information about the certification requirement for the test by clicking the More info… link in the Test Description column.
When you have fixed the issues with your app, redeploy it to a device and walk through the test steps again. You should repeat this process until your app passes all the tests in this section.