Compartilhar via

UITest Framework – WPF plugin – Part 1

  • Artigo

Introduction

UITest uses Windows Automation API 2.0/3.0 also known as UI Automation (UIA) to drive the automation on WPF controls. UITest supports WPF applications developed on top of .net 3.5/4.0 framework. However the level of support for UITest Applications depends on the combination of the 2 variable ie UIA version and .Net version. This is explained in detail later.

UITest provides inbuilt support for WPF Applications supporting Virtualization. Virtualization is a technique by which the DataItems are created only when the User wants to interact with it. If a List has 100 elements Virtualization ensures that only 10-15 elements (Depending on the view-port size) are loaded at the beginning and then if User Scrolls down the other elements are created. This improves performance considerably. However this introduces lot of complexities for identifying the control not loaded and is a challenge to record and play on Virtualized Control. UITest has good support for Virtualization but has some pre-requisites. This is explained in details here.

UIA is a improvement on the legacy Microsoft Active Accessibility (MSAA). UIA offers a richer set of properties, as well as a set of extended interfaces called control patterns to manipulate accessible objects in ways Microsoft Active Accessibility cannot. The Application HAS TO implement UIA for UITEST to work on the application. This is especially true for the custom controls. The default WPF controls implement UIA. The following section describes the accessibility BestPractices for making UITest work on WPF.

 

 

Accessibility Best Practices for WPF

The following bestPractices will go a long way in ensuring good support for WPF in UITest.

· Have a unique AutomationID for the Control. AutomationId is always non localized.

· Override the ToString of the dataObject to which the control is bound. If the ToString is not overridden user can also Set the Property AutomationProperties.Name. Sample XAML would be<Setter Property="AutomationProperties.Name" Value="{Binding XPath=@Name}"/>

· For a Datagrid have unique Name for the Row. If Databound it has to follow the above steps.

· For VirtualizedControls ensure that the ItemContainer is within 2 ParentLevel of the control.

 

 

Recording

During recording following the following information would be stored in the recording strip which would enable the strip to be played back

· QueryID of the Recorded WPF Control

· Aggregated Action

 

QueryID of the Recorded WPF Control

Technology of the control

This is the basic requirement of the recorder. The recorder first needs to determine what technology does the app belongs to so that it can use the appropriate accessibility API’s. UITest tool architecture enables this feature with a plug-in model where a plug-in can be registered for a specific technology. For example UIA plug-in is registered for WPF application where the tool identifies a WPF application by checking the window class name adhering to the regular expression ^HwndWrapper[.*;;.*].

Query Id of the control

To put it in a simple term this is the address of a control. It stores all required information of the hierarchy of the control so that it can be traced back during playback. A typical query id will look like this

“<Top Level Window’s Query Element>;<Special Intermediate Parent/ItemContainer>;<Immediate Parent’s Query Element>;<Control’s Query Element>”. e.g.,

Following are the brief description of each of the component

TopLevelWindow

It contains the info related to TLW directly rooted under desktop. In most of the cases the Title of the window is recorded as the name of the window, control type would be ‘Window’. Technology would be UIA for WPF Window.

ItemContainer/Special Intermediate Parent

We generate a ItemContainer for the Virtualized Items for which we recursively search upto 2 levels and keep generating the Container. We also generate Special Intermediate Parent like Frames incase they exist within 15 Levels of the parent and provided the Parent is not a ItemContainer.

Immediate Parent

The Immediate Parent is generated as a part of the QueryID of the element. The Control is searched within the Parent.

Control

The control would be identified using AutomationID,Name, HelpText,LabelledBy and controltype property. Instance is also generated for the control to identify it uniquely among other similar control under the same recorded parent.

Additional Search Properties

Apart from the control specific properties, some other properties may be required to identify the control, e.g., if the control is nameless or the parent has to be expended before looking for a control e.g, tree item. Following are some of those.

Search configuration of the control

We have some predefined search configuration which helps to narrow down the search space (search only in visible controls) or to do perform some prerequisite before actually starting the search (expand parent tree node before searching for the child node). Following are the different search configs:

SearchConfig Parameter

Description

AlwaysSearch

UITest uses a cache while doing actions on an Application by adding Always Search in the search config user can force UITest to not to use the cached value for the control.

DisambiguateSearch

If the parent and the controls properties are same there are chances that UITest would start doing action on the parent itself. This config can be used to ask the playback to act on its child rather than the parent itself.

ExpandWhileSearching

Expand thecontrol before looking for the other control inside it. E.g., TreeView

NextSibling

Search in the siblings inside the container. Sometime if the control is nameless we may iterate from the named control inside the same container to reach the control.

VisibleOnly

Search only in the visible control. It helps to reduce the search space.
NextSibling

This is generated for the nameless control. It contains a integer value and another control info. Offset is basically the distance from the named control among its sibling. So during playback the actual control is found using named control and with the offset value.

Instance

NextTo is generated for a limited no of distance i.e., if there is no named control within a specific range from the nameless control, the instance property is generated which is its position among all similar controls having same search properties.

 

Aggregated Action

Once the control type is determined, the action can be stored as it is or it can be combined with other actions on the same control. Action can also be omitted if it is not relevant in UITest context. This feature is called aggregation. For example sendkey action of ‘a’, ‘b’, ‘c’ on a text box will be aggregated as SetValue of ‘abc’. At the same time if user did sendkey of ‘a’, ‘b’, ‘d’, ‘bkspace’ and ‘c’. This will also transform to setvalue of ‘abc’ as sendkeys of ‘d’ and ’bkspace’ will be ignored.

WPF Aggregators

UITest uses some predefined set of rules to filter out/combine a group of actions to a well-defined action on the control. For example if a user

types a string say 'xyz' in the edit box

deletes 'z' using backspace

Types 'abc' again.

The simplest raw recording would have been

SendKeys('xyz')

SendKeys("BackSpace")

SendKeys("abc")

Though this will work in normal way but this sequence of action is not very resilient. Instead of having raw actions in the recording strip UITest will record single SetValue('xyabc") action on the edit box. The advantage is the state of the textbox will always be correct after the action irrespective of the initial state of edit box.

Another common instance is selecting a value in the combo box which compromises following steps

Click on the expand button

Select a list item

UITest would record this as SetValue on the combobox.

Following are some of the common WPF controls and aggregated actions which will get recorded.

Control Name

Possible user action

RecordedAction

TextBox, RichTextBox(except PasswordBox)

Type xyz in the control

SetValue(xyz)

ComboBox

Select some item say Item1 in the control

Incase the Element inside ComboBox is a TreeView

SetValue(item1)

Click ComboBox

Click on the TreeNode.

ListBox

Select Items a/b/c/d

SetValue(a,b,c,d)

CheckBox/Radiobutton/ToggleButton

Selection/DeSelection

SetState(Checked)/SetState(Clear)/SetState(Intermediate)

DateTimePicker/MonthCalendar

Selection of a date

SetValue(SelectedDateRange)

MenuItem

Click on TopMenuItem->Click On IntermediateMenuItem->Click on LeafMenuItem

MouseClick(LeafMenuItem)

The MenuItems touched in the process will all be part of the QueryID and during Playback each of them will be expanded and action will happen on the LeafMenuItem

TreeView

Selecting a TreeItem When the TreeItems are not other contained Controls

We will record Click on the LeafTreeItem with the full QueryID of the treeitem stored uptil the TopLevel TreeNode

Selecting a TreeItem when the TreeItems are contained Control like CheckBox

In this scenario we will record an Expand Action on the TreeItem and subsequent action on the contained Control. In the current example we will record Expand(treeItem),SetState(CheckBox).

DataGridView

SetValue(data)

SetState(checked)/SetState(clear)

SetValue(Item)

SendKeys(keyboard_Key)

Slider

Click on the Slider/Drag the Slider Pointer

SetValue(Value)

 

 

Playback of WPF Application

The Playback in WPF uses the recorded information and plays back the action first by finding the Control using the Control QueryID properties and then playing back the aggregated Actions.

Search for a WPF Control goes via 2 plugins. The topLevel Window Search is powered by MSAA and then the UIA Plugin takes up. UIA Search follows different route for Virtualized and Non-Virtualized Controls. The UIA Plugin searches for the control using a BFS Search. All the properties specified in the QueryID are matched completely against the elements in the tree and the first element where it matches completely is returned .

WPF Plugin supports 2 Enumerators one each for the Virtualized and Non-Virtualized Elements. The non-virtualized Enumerator iterates through the Children matching the properties with the QID properties. The virtualized Enumerator uses the FindItemByProperty of UIA to search for the Virtualized Childrens.

If the Search for any of the Intermediate parent Fails the Search will commence from the TopLevelElement and proceed further.

The search has some limitations when

If the WPF application has very deep nesting such that an ItemContainer exists at a Level Greater than 2 Levels from the parent of the control Our recorder generated QueryID leads to SearchFailure as we arenot able to capture the Itemcontainer. This is because we generate parents only upto 2 Levels from the parent of the Control. This is to done as to get better Resillience and to get better performance during recording.

In the following Example We don’t generate the DataGrid and the DataItem as the parent of the Button as they exist at a level more than 2 Levels from the parent.

WPF_TreeView

In such scenarios workaround is available in CUIT wherein user has to add all the ItemContainers in the hierarchy.

When the playback starts first it tries to find top level window with the following logic:

1. Pass 1: Search only in all visible windows inside desktop (no timeout) with SmartMatchOption turned off

2. Pass 2: Search in all visible + minimized windows with a timeout of 15% of total search time out value (e.g, 2 min by default) . In this pass also the SmartMatch option is set to false

3. Pass 3: Both options minimized windows and SmartMatch are enabled in this and with a timeout value of (SearchTimeout - 15%ofSearchTimeout)

SmartMatch

SmartMatch feature is used to search for a TopLevelWindow. Some time it is possible that the title of the window is not same as it was during recording. For example when an App has multiple pages inside it the title would have the page name along with the App name on it. So there are chances that during playback the title is not same as it was during recording. e.g., Notepad. Smartmatch helps in these cases by enabling the search to pass by selecting a window with matching title based on predefined heuristics.

By default SmartMatch is turned ON. User can turn if off by

Playback.PlaybackSettings.SmartMatchOptions =SmartMatchOptions.None;

Search Timeout

The searchTimeout is configurable by the User and default is Set to 2mins. Default Search will take much less time and come out.

For MTM User can configure it from the TestSettings. For CUIT he can specify it by

Playback.PlaybackSettings.SearchTimeout=100;

 

WFR

Sometime in an App there might be some controls which take time either to load or to get enabled. The WFR Feature in UITest ensures that the Search for the control doesn’t start until the control is ready. WaitForReady feature helps UITest to be resilient and eliminates the usage of any hardcoded TimeOuts in the TestCode. This will make the playback more resilient and speed it up as well.

UITest Supports WFR both through the native WFR mechanisms provided by the underlying accessibility Technology or through a Thread based Wait Mechanism. UIA doesnot provide any WFR functionality and so in WPF the WFR is only the ThreadBased WFR. ThreadBased WFR has the following kind of waiting mechanisms:

· Foreground thread

· Background thread

· No thread

Following are the two properties related to WFR which can be set by the user :

WFR Level

FastForward a test in MTM

In MTM scenario WFR for the foreground is set by default. This is not configurable.

CUIT

A user can change the WFR settings using following APIs in CodedUITest.

Playback.Settings.WaitForReadyLevel defined under (Microsoft.VisualStudio.TestTools.UITesting) can be used to get/set the WFR options.

The possible WFR options are defined in an Enum named WaitForReadyLevel in Microsoft.VisualStudio.TestTools.UITest.Extension.

public enum WaitForReadyLevel

{

AllThreads,

Disabled,

UIThreadOnly,

}

However since the Plugin doesn’t provide any support for WFR lot of times we will end up in scenarios where WFR will fail . Like Scenarios like waiting for a WebServiceCall/Database call. In such Scenarios users can use THINKTIME.

Apart from the ThreadBased WFR user can also use WaitForControlEnabled method to wait till the Control is enabled.

WFR Timeout

The default timeout for WFR is 1 min in both MTM and CUIT scenario. One can change this value as follows:

FastForward a test in MTM

This is not exposed through test settings but can be re-configured in MTM.exe.config file.

CUIT

This can be get/set using the following property :

Playback.PlaybackSettings.WaitForReadyTimeout
 
 

Ensure Visible

UITest has very good support to playback the action on the control even if the control is not in viewport. There are 2 important scenarios here

· Application is visible but the control on which Action is to be performed is not viewable.

· Application is outside the ScreenArea

· Application is obscured by SomeOther Application

In the above Scenarios UITest will ensure that the control is made visible and then action is performed on it. This is absolutely essential as UITest does Actual Mouse/Keyboard actions and the control must be visible for that. Because of the advanced support for Ensurevisible All actions done on the Scrollbar are aggregated out getting us a very clean recording.

Since UIA supports very good support for scrolling using ScrollPatterns the Scrolling in WPF for controls which supports ScrollPatterns are done using ScrollPattern else the Scrolling is done in the following order by using

· ScrollIntoView

· SCROLL_BAR (Programmatic)

· Mouse WHEEL

· KEYBOARD(UP/DOWN/LEFT/RIGHT/PageUP/Down)

· SCROLL_BAR_CLICK

UITest also supports EnsureVisible across multiple Plugins and in that case both the Scrolling happens across multiple technologies to get the element into view. An example may be scrolling for a WPF control hosted inside a Winforms control.

 

Summary

In this part 1 of the blog we have seen how WPFplugin records and playback actions on WPF applications. In Part 2 of the blog we will the known issues and how the user can troubleshoot issues in record and Playback on WPF applications.

 

Author – Rituparna Paul

SDET II - CodedUITest