UITest framework in Visual Studio 2010 - Part 1

[アーティクル]
04/13/2010

Introduction

Visual Studio Team System (VSTS) 2010 has lot of exciting features which caters to the needs of customers in the ALM space. Visual Studio Team Test (VSTT) is part of VSTS offering and in this release lot of new cool features has been added like integrated test case management story with TFS, Microsoft Test & Lab Manager (MTM) tool with rich UI to manage and execute manual test cases with fast forward feature and functional test automation solution using CodedUITest (CUIT). Microsoft Test & Lab Manager (MTM) is a WPF based UI tool which is a one stop solution to manage QA deliverables in a Life-Cycle of an Application. This tool can be used to create test case artifacts in TFS and manage Test plan and test pass for multiple releases of the application.

One of the main feature of this tool is to help manual testers to minimize repetitive tasks in a test pass recording UI actions on the Application Under Test (AuT) so that they can be played back when they execute the same test case on a later build using Fast Forward feature.. Once the test case has been recorded it can be used to generate test automation code in C#/VB automatically. A new test type has been introduced in this release called CodedUITest (CUIT) which primarily contains test automation API’s that can be used to drive the Application under Test (AuT). Both these features uses UITest framework to enable users to minimize the manual test effort.

This document gives you an overview of UITest framework and how it is used to help manual and automation tester to test their UI Application with ease.

UITest Architecture

The above diagram is mostly self-explanatory –

The lowest layer is of technology (specific) adapters. A technology adapter is a module that understands the corresponding UI technology and provides the UI technology specific services to rest of the modules.
Next up is the abstraction layer which helps abstract rest of the code from various technologies.
After this are the recorder and playback modules.
- The recorder first records the raw steps and based on the filter\aggregation rules, these raw steps are converted into filtered steps or user-intention.
- The playback has the public API for the users to use. Apart from this, it also has property provider to give information on properties supported by each control and browser service for browser specific operations like navigate to URL.
The top layer is the two clients that we have today for this feature.
- There is Test Runner which uses this functionality to do Fast Forwarding for manual tests. This “interprets” the recording on the fly and calls appropriate API on the playback side.
- The Coded UI Test (or VS client) generates code out of the recording. To do this, it uses information provided to it by property provider for each control. The users can alternatively also hand-code using the API.

UITest framework Engine

For the tool to record user action or do a playback, it needs to use accessibility API’s to get details about the control so that it can be searched and perform actions on them. UITest framework uses IE DOM for IE applications, MSAA for Win32 and Winforms application and UIA for WPF applications

During recording the tool stores the following information so that it can be played back and help user in troubleshooting any failures due to difference in recording and playback.

System Information
Window properties of the app
The technology of the control acted upon
Query Id of the control (so that it can be searched)
Control Type
Action Type
- Aggregation
- Password Encryption
- ThinkTime
- Continue On Error
Action Log

During playback the above information will be used to locate and perform the action on the control. Apart from this playback also has following features so that it is resilient when the action is played back multiple times in different situations

Search
WFR
Ensure Visible
Action
UISync

Let’s look at each of the above areas in detail.

Recorder

Store system Info

UITest framework first records the information on the system where the recorder was started. This will help the tester to identify the config where the test case was recorded.

Window properties of the app

UITest framework first records the information about top level window of the control the user acted on so that it can be searched during playback. This information is vital because without this the control has to be searched on each window opened in the user’s machine. It stores the class name and title of the window.

Technology of the control

This is the basic requirement of the recorder. The recorder first needs to determine what technology does the app belongs to so that it can use the appropriate accessibility API’s. UITest framework architecture enables this feature with a plug-in model where a plug-in can be registered for a specific technology. For example UIA plug-in is registered for WPF application where the tool identifies a WPF application by checking the window class name adhering to the regular expression ^HwndWrapper[.*;;.*]. Similarly IEDOM plugin will identify IE window using similar logic. And MSAA plug-in identifies a Winforms window based on the class name of the window containing “WindowsForms10.Window”. However UITest framework uses MSAA as a default plug-in if the window does not qualify search logic of any plug-in. So if there is a Win32 application window, MSAA plug-in will be used to record and playback the actions.

Query Id of the control

To put it in a simple term this is the address of a control. It stores all required information of the hierarchy of the control so that it can be traced back during playback. A typical winforms query id will look like this

“<Top Level Window’s Query Element>;<Immediate Parent’s Query Element>;<Control’s Query Element>”.

Control Type

UITest framework needs to know the control type so that it can store any specific data on the control which can be used during playback and to identify if the action associated to the control needs any more processing (more info given in the next section). This information is also used to let user know what control the user acted upon (Action Log).

Action Type

Once the control type is determined, the action can be stored as it is or it can be combined with other actions on the same control. Action can also be omitted if it is not relevant in UITest framework context. This feature is called aggregation. For example sendkey action of ‘a’, ‘b’, ‘c’ on a text box will be aggregated as SetValue of ‘abc’. At the same time if user did sendkey of ‘a’, ‘b’, ‘d’, ‘bkspace’ and ‘c’. This will also transform to setvalue of ‘abc’ as sendkeys of ‘d’ and ’bkspace’ will be ignored.

Recorded Actions in XML

System Info - System info contains the basic details about the machine where the recording was performed including the IE configs.

UITestAction - UITest framework has predefined sets of actions with different parameters to characterize the user's actions on the desktop in the form of xml. Following are the commonly used actions

MouseAction - Click, DragAndDropAction.

KeyboardAction - SendKeysAction, SetValueAction.

Others - LaunchApplication, SetStateAction, NaivigateToUrl.

Every action has its own parameters and a QueryID for the control where the action was executed.

The following is the example of one simple mouse action:

<IsGlobalHotkey>false</IsGlobalHotkey>

<ActionType>Click</ActionType>

</MouseAction>

UIMap - UIMap contains the information related to the controls on which the actions were performed. We generate QueryID from the properties of the controls and its UI hierarchy, to identify it uniquely among all the controls under the desktop. Typically a QueryID for a winform control would be like A.B.C where A would be the Top level window.

Playback

Search

Using the top level window and technology information, playback uses the right accessibility API to locate the control. Search has lot features to be successful even if the app is going through changes. Search algorithm for Winforms is complex as many applications do not have proper accessibility support.

WFR

WFR stands for Wait For Ready, where actions on a control is deferred till the application is ready for user actions. In winforms there is no proper ways to know if the application is busy or not, so we only depend on foreground thread on the application to determine if it is ready for actions

Ensure Visible

This feature will make sure the control is present in the visible area of the screen so that actions can be performed. This is accomplished in two ways, by doing setfocus on the control or scroll the window to make the control visible.

Action

Each action that was recorded has related playback logic. The logic depends on the combination of control type and the action type recorded.

UISync

Once the action is performed UISync component will ensure that the Application Under Test (AuT) received the action. This will ensure that if playback completed successfully then one can assume that all actions completed successfully.

Fast Forward using Test Runner

When the functional test cases are executed in Test Runnermanually, the actions performed on the AuT will get recorded automatically by the UITest feature. When the test run data is published to TFS, the recorded strip will also be saved into TFS. When a new test run is created with the same manual test cases the user executed in earlier run, Test Runner gives an option to playback the recorder strip instead of performing those actions manually again. Users can playback portions of the recording or the whole recording using markers in Test Runner UI. This feature in Test Runner is called Fast Forward which let’s manual tester to navigate to a desired state of the application.

MSDN Link on Recording and Playing back Manual tests: https://msdn.microsoft.com/en-us/library/dd286714(VS.100).aspx

How is UITest framework engine is used in Fast Forward scenario

Please take a look at the below blogs

Microsoft Test Runner series – part 4 – Fast Forwarding

Fast Forward Testing – Part 1 – The magic w(b)and

Customizing Fast Forward settings

Before doing a manual run in MTM, user can customize Fast Forward settings by creating of modifying TestSettings. User can go to

LabManager -> TestSettings -> New/Open Test Settings -> DataAndDiagnostics -> Action Recording and Action Log.

Fast Forward options in MTM	Comments
Action Log	User can turn off action log which will not start Recording while testing in MTM
Action Recording	Users can turn of action recording if they don’t want Fast Forward. But if Action Log is enabled, recorder will be started but it will only save the action log but will save any automation strip.
User Delay between action	If the user wants to capture time taken between actions as part of recording, this option can be turned on. If the user takes more than the time given in this option then the time taken will be stored in the strip. Playback will honor this based on the following option
User Delay Action Multiplier	This determined whether playback should honor User delay Between actions. If the value is 0, even if the delay is recorded playback will not honor that. If the value is 2 then playback will wait for twice the recorded time. This will be useful if the Fast Forward happens on slow connection
Constant Delay Between Action	This is the time taken between two actions during playback. If the user feels the playback is very fast to follow what is happening, he/she can change this setting
Search time for controls	Be default the value is 2 minutes.

Launch of Application Aggregator

Launching an application or a web page for UITesting is a frequent scenario and it can be achieved in many ways. UITest supports a special UITestAction type for this Scenario 'LaunchApplication' which will be recorded for the following ways of launching application.

1. Launch an App from StartMenu -> All Programs

2. Launch an App from QuickLaunch Toolbar

3. Launch an App from the shortcut provided on the desktop

Following are some of the ways that are not supported in this version.

1. Launch an App exe from some other location e.g., explorer

2. Launch the App from run window

Following are the different parameters used in LaunchApplication:

Filename	The full path to the executable without any substitution environment variables. E.g. “C:\Program Files\MyApplication\MyApp.exe”.
AlternateFilename	The full path to the executable with transform of environment variables. E.g. “%ProgramFiles%\MyApplication\MyApp.exe”.
Arguments	The arguments passed to the application during launch.

For web page launch we record NavigateToURL as the UITestAction with the two parameters:

URL: URL of the web page

NewInstance: True if web page has to be launched in new IE session, false otherwise.

Summary

In this blog UITest framework is introduced and we have seen what are the different components of UITest and how it is used in Microsoft Test Runner. In Part 2 we shall discuss how it is used in CodedUITest, how to get resilient playback and how to troubleshoot common failures.

Author – Venkatesh Sargunesan

Test Lead - CodedUITest

Comments

Anonymous
May 31, 2010
Hi Venkatesh, I have a scenario where I have different environments supporting the same code. The environments are development, QA and production having different URLs. Now i wish to my tests recorded on dev environment against QA and production. But the coded ui tests fail? How would i go about doing this?
Anonymous
March 29, 2012
The link needs to be provided as a parameter in UIDesigner.cs file and hence depending upon the URL mentioned as Inpout would be the target app for testing

次の方法で共有