Configuring Playback in VSTT 2010
In a previous post, Abhishek had talked about the Record and Playback Engine in VSTT 2010. The Coded UI Test feature in VSTT 2010 also uses this engine to playback UI actions. In this post, i will describe the various settings for Playback and how you can modify them to improve the resilience of your Coded UI Tests. Playback is configured by modifying the fields in Playback.PlaybackSettings class.
1. Continue on Error
When you are recording a Coded UI test on a login page, the “Auto Complete passwords” popup may have appeared.
When you playback actions on the login page, this dialog will *not* come up. So the action that you performed will cause an error. To improve test resilience, the Record & Playback engine automatically marks this action as “Continue on Error”. When the test is being played back and the error occurs, the engine will continue on to the next action. Specific actions (IE popups, implicit Hovers) when performed by the user are tagged by the engine as “Continue on Error”.
In Coded UI Test, you have the flexibility to turn on “Continue on Error” for any action. This is done by the following code snippet.
Playback.PlaybackSettings.ContinueOnError = true;
Remember to turn this flag to false after the section where there are potential failures. Otherwise the engine will ignore all errors and your test will return incorrect results. By default this flag is set to false in Coded UI Test.
2. Match Exact hierarchy
The Record and Playback Engine uses a hierarchical structure to locate controls. The Yes button in the dialog above is identified as
To locate YesButton during playback, the engine will search for AutoCompletePasswordWindow, then for YesWindow in it, and finally YesButton in YesWindow.
Sometimes application may change so that the hierarchy gets modified. Record and Playback engine attempts to locate the control, in such cases, by skipping the intermediate elements in the hierarchy. Thus in the example, if it is unable to locate YesWindow, it will search for YesButton directly under AutoCompletePasswordWindow and all its children. This behavior can be controlled by the “Match Exact Hierarchy” setting. This is done by the following code snippet.
Playback.PlaybackSettings.MatchExactHierarchy = true;
If the flag is set to true, every control is the hierarchy will be searched in order. Any failure will result in failure to locate the control.
By default, this flag is set to false. Note that this may sometimes lead to false positives i.e an unintended control may be identified. In such cases, you will have to change this setting.
3. Search in minimized windows
The Record and Playback Engine searches for controls even inside minimized windows. The minimized window is restored and actions performed on the control inside it.
You have the ability to turn this feature off i.e. prevent the engine from looking at minimized windows. This is done by the following code snippet.
Playback.PlaybackSettings.SearchInMinimizedWindows = false;
4. Smart Match
The Record and Playback Engine identifies controls on the user interface by its search properties. e.g:- The standard calculator window is identified as below.
SearchProperties[WinProperties.Window.ControlType] = ControlType.Window.Name;
SearchProperties[WinProperties.Window.Name] = "Calculator";
SearchProperties[WinProperties.Window.ClassName] = "CalcFrame";
Some of the search properties may change over time. Record and Playback engine uses a Smart Match algorithm to identify controls if they cannot be located using the exact properties. The smart match algorithm uses heuristics and attempts to locate the control using variations on the search properties.
You can control when Smart Match should be applied. By default Smart Match is applied for Top Level Windows and all controls. You can turn off Smart match using the following code snippet.
Playback.PlaybackSettings.SmartMatchOptions = SmartMatchOptions.None;
The default settings (Smart match top level windows and controls) is optimal for a resilient coded UI test. But sometimes it may lead to false positives and then you will have to modify this setting.
5. Wait For Ready Level
The Record and Playback engine ensures that each UI control is ready to be acted upon before performing any action. The smart “Wait For Ready” algorithm significantly improves the resilience of your Coded UI Test. Now you don’t need to add the annoying Sleep statements whenever a UI Control is busy and not ready to receive input. By default, the engine checks the UI Thread (foreground thread) to determine if a control is ready.
You can turn off Wait For Ready completely by the following code snippet.
Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.Disabled;
Alternately you may want to wait for all threads including background threads. This may be required if the application is making asynchronous calls that results in UI changes. You can do this by the following code snippet.
Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.AllThreads;
NOTE that this setting will slow down the engine and should be carefully used.
6. Set Property Verification
After setting the property of any UI control, the record and playback engine performs a verification pass to ensure that the set succeeded and the UI control now has the value assigned to it.
e.g:- You type the text “MSFT” in the search edit box in Bing. After playing back this action, the engine reads the text property of the search edit box to ensure that this set has succeeded.
In certain applications, the set property may trigger off some business logic which then changes that property itself. In such cases, the verification fails. If your application exhibits such a behavior you can turn of the set property verification using the following code snippet.
Playback.PlaybackSettings.SkipSetPropertyVerification = true;
7. Fail Fast
The Record and playback engine’s search algorithm has a fail fast logic. i.e. If the engine is reasonably confident that the search will not succeed, it will return with a failure. This ensures that you are not waiting for a non-existent control for 2 mins (the default search timeout).
Sometimes you may want to disable fail fast. Typically, you do this along with increasing your search timeout. You are confidant that the UI control will become available within a specified duration (which is longer than the search timeout) and you want the engine to wait for that duration. You can do this by using the following code snippet.
Playback.PlaybackSettings.ShouldSearchFailFast = false;
8. Misc
Other playback settings which can be modified include
a. Delay Between Actions: How long should the engine wait before performing two successive actions? By default, 100 ms.
b. Search timeout – How long should the engine attempt to locate a control? By default, 2 mins.
c. Wait for Ready Timeout – How long should the engine continue waiting for a control to be ready? By Default, 60s
d. Think Time multiplier – For slow machines, assign a high multiple so that the recorded think time is scaled up to handle slower application responses.
e. Send Keys As ScanCode - By default, the engine sends keys to the application under test as Unicode. But in certain cases, it may need to send keys as scancode. Sending Unicode to such applications will result in failure
NOTE: The Test runner tool in VSTT 2010 also uses the Record and Playback Engine. You can configure the playback settings described above in Test runner tool from
a. mtlm.exe.config – located in %VSINSTALLDIR%\Common7\IDE
b. Microsoft Test & Lab Manager, navigate to Lab Center –> Test Settings , open the applicable Test Settings, select Data and Diagnostics section and click on the Configure button next to Action Log and Action Recording. In the Advanced section of the dialog, some of the Playback settings described above may be configured.
Comments
Anonymous
October 19, 2011
I tried both ReadyLevel options - >WaitForReadyLevel.AllThreads – Wait for all requests to finish in all currently running applications. >WaitForReadyLevel.UIThreadOnly – Wait for all requests to finish in the UI you are working on (your web application). but control does not wait and test case fails. Do I need any other change to make above work ?Anonymous
December 08, 2011
I have the same experience when setting to AllThreads. Any thoughts?Anonymous
January 04, 2012
Many a times due to network issues, the next page does not load. The above wait for ready levels dont seem to be working. Are there any other settings which specifically wait for the page load?Anonymous
July 03, 2012
I'm using a data source to run my tests. But half of the time, my test will fail because the playback won't enter some of the charters in the specified string. Now I understand but is there anything to do to avoid it? I'd like to avoid setting SkipSetPropertyVerification to false.Anonymous
November 01, 2012
I did not used any synchronized methods, even my code is slow while performing an actions on the controls even if readily available in same page..Anonymous
November 25, 2012
It seems coded ui perfornamce is slow when working with ASP.net controls than working with html controlsAnonymous
February 11, 2013
The comment has been removedAnonymous
November 28, 2013
Hello all, I am using following code-------- Playback.PlaybackSettings.MatchExactHierarchy = true; contract = new UIMap_DealPreparationContract(); try { results.startTime = DateTime.UtcNow; // br.Refresh(); //click on Left nav 'Deal preparation tab' Mouse.Click(contract.UICompassPremierPremieWindow.UICompassPremierPremieDocument21.UIDefaultDetailsMenuViPane.UIDealPreparationInProCustom); //enter GSX Id contract.UICompassPremierPremieWindow.UICompassPremierPremieDocument21.UICRMOpportunityIdEdit.Text = DealPreparation_ContractInputs.GSXID; contract.UICompassPremierPremieWindow.UICompassPremierPremieDocument21.UIOpportunityNameEdit.Text = "Test Opportunity"; //enter account id contract.UICompassPremierPremieWindow.UICompassPremierPremieDocument21.UIAccountIdEdit.Text = "1234";//DealPreparation_ContractInputs.AccountID; //enter TPId contract.UICompassPremierPremieWindow.UICompassPremierPremieDocument21.UITPIdEdit.Text = DealPreparation_ContractInputs.TPID; //select work order type contract.UICompassPremierPremieWindow.UICompassPremierPremieDocument21.UIWorkOrderTypesComboBox.SelectedItem = "Custom Work Order"; //click on save Mouse.Click(contract.UICompassPremierPremieWindow.UICompassPremierPremieDocument21.UISaveCustom); } there is nothing complicated in my code but when it fills values in textbox or for nay mouse click action taking too much time code is working fine but taking long long time........ please help me ...Thanks a lot in advance!Anonymous
July 10, 2014
Hi, Firstly, just like to say this is an awesome resource - thanks for the post! Secondly, for the comment headed "Coded UI Test for asp.net application ..running very very slow.. " is it possible that the problem lies in the fact that you refer to your controls by their entire hierarchy tree every time you use them? Would it be better to assign these to a new control variable at the start of your method and then refer to the variable for each use? I would also recommend creating a forum post for this kind of thing in the future because I rarely see answers posted in comments unless the questions are directed at the blog content. Hope that helps,