SearchConfiguration.NextSibling

In this post I will be describing what does NextSibling means in context of search. But before doing that I will mention the ways in which search can happen.

There are two ways (exclusive) in which search can happen:-

a)    Plugin search :- By plugin search I mean that the respective plugin for that Technology (as specified by the user) searches for the control.

b)    Playback Engine search :- In this case playback navigates through the UI tree of the Application in BFS manner to search for the control.

Today in case of MSAA (Winforms/Win32 controls) and UIA (WPF controls) plugins, playback searches for the control, where as in case of Web (IE controls) the search happens using plugin search.

Let us try to understand a simple search that happens using Playback engine. Consider the following UI tree.

 

 

Suppose user specifies following Search Criteria for Control “F”. (I will name the variable same as what is specified in the blocks to make things clear)

If you see in above case A is the container element for C and C is container element for F. Container elements are those element which act as a pivot for for searching another element. In above case playback engine will search C only inside C and will search F only inside C (and not B). As mentioned before playback search happens using BFS traversal of the UI tree. Lets try to mimic what happens when engine tries to search for Control F.

In the above case A is the Toplevel element. Top Level search happens using Window search. Lets assume that playback was able to find A using Window based search. Lets put A in the array. This A will now act as the Pivot.

Now engine gets all the children of A (it queries the plugin for all the child) and puts them in the array. Here is what the updated array looks like

Now engine sees that the C’s search properties matches the Search properties that user has specified in the code. Bingo! element is found!!

So till now engine is able to search for A and then inside A, C. Now it goes and tries to search for F. It puts C in the array. Updated array looks something like (just cleared the array for clarity)

As it did before it gets the child of C and tries to match the properties. In this case the array looks like

 

In this case it matches the properties specified by the user and properties of F, which eventually matches and hence engine is able to search F.

Above example is a very basic scenario in which search happens, just to get things setup.

 

Now let us analyze the UI tree drawn above and try to get relationship among the controls. As you can see D and E are siblings which has common parent B.

F does not have a real sibling. But it has siblings D and E with common ancestor being A.

Question: What if we are able to define a relation of controls (like what we do for ancestor elements what we say container elements) which are siblings of each other for in case of search.

Answer: Yes it is possible. This is why NextSibling SearchConfiguration is there.

 

Moving ahead with NextSibling SearchConfiguration. Today this is used in case of MSAA and UIA plugin which also means that this configuration is understood by the playback engine.

EXAMPLE 1

Lets take for example the below code

In short I have defined a relationship between E as a next sibling of D with common ancestor B.

Below diagram shows how search for E happens in this case. Let us assume that B is already searched by Playback engine. Hence the array will look something like :

Now the motive of Engine is to search D (because that is the next element it needs to search before searching E). So it gets the children of B one by one and puts it into the array. So it puts D (The first child of B) in the array and eventually marks it as element D because search properties matches. Here is the updated array

Now it goes and try to search E. Engine on seeing that NextSibling is the SearchConfiguration for E, it doesn’t gets the children of D (as it did in the original search example) rather it continues getting children of B, because E will be the next sibling of D. Doing so it finally hits E which matches the search properties specified by the user.

This was a simple scenario of a NextSibling match.

EXAMPLE 2

Let us discuss one complex example. Below code shows one of the complex scenarios :

In this above scenario I am trying to search F with NextSilbling configuration. If you notice A is the common ancestor of both D and F and hence I have written the code in this way (have not used B as the container element of D).

Let us see how does this work. Assume A is already searched and part of the array. So the array looks like

As the search progresses for element D engine pushes children of A inside the array, but none of them matches. The state of array will be like:

The pivot element (meaning the element of whose children was getting queued in the array) was initially A. Now when all the children of A is in the array the pivot element becomes B (next element in the array). Hence children of B is now queried for and queued into the array. In this case D is the element which matches the search properties specified for control D. Hence the state of array looks like

Now the engine tries to search for F. It sees a NextSibling Search configuration of F. Hence rather getting children of D it continues getting children of B which has only one element left that is E. Array looks something like :

The pivote element moves from B to C. The only child of C ie F gets queued into the array and also matches the SearchProperties specified by the user. Hence engine returns element F.

Uses of NextSibling SearchConfiguration

The codeduitest recorder (in case of UIA and MSAA plugin cases) generates NextSibling SearchConfiguration when the element does not have a good Searchable properties (ie they have just have ControlType property), but it’s previous sibling has good identification strings.

In example 1 E didn’t had any searchable property but D had. Hence recorder in this case will generate NextSibling for element E.

In example 2 F didn’t had searchable property and so did E. Hence it generated NextSibling for F with D being the previous sibling with good Searchable property.

User can always hand code and use this configuration as and when required.

Note: As mentioned before NextSibling today is applicable for MSAA/UIA plugin and not web.

Comments

• Anonymous
  May 19, 2011
  Very nice article. I have one additional question: Can you mix UITestControl-s found by MSAA and UIA? For example, can you search for a control using MSAA with container element found from UIA technology? And the other way around?

• Anonymous
  November 16, 2011
  In example 2, when the search engine sees that F has "NextSibling" configuration (counting from D) and continues getting children of B, why doesnt E match already? Or should we treat NextSibling search configuration as a function of two arguments: the pivot (D) and common parent (here, A, which can be taken from D.GetParent)? If we look for next sibling from D, this is E. But if we also look that D was created with A as parent, we can say "find next sibling of D such that they have common parent A", and this will be F.

• Anonymous
  January 21, 2012
  Brilliant Article Siddhartha!!! Can you please suggest some good readings on UIA and CUIT.