Custom UI Automation Providers in Depth: Part 3

In part 2, we created a very simple, “Hello World!” provider.  In this part, we’re going to talk in depth about UI Automation properties and add some to our sample provider.  Properties describe what a UI element is.  They are usually an answer to a question that someone might ask about your UI; understanding the question will help you determine the right answer.   The sample code for this section is here.

I am often asked, “Which properties do I have to implement for my custom control?”  In part 1, we talked about how UI Automation is really about creating a programming interface for your UI – so one answer to the question is, “Whatever properties you think your developers might need to use.”  But this feels a little circular.  I think it’s more commonsense to say, “What questions do you need to answer about your control?”  With that in mind, let’s walk through some of the basic UI Automation properties to see what questions they answer.

[There is a full MSDN reference to these properties, which is a good supplement to this informal walkthrough.]

Control Type: Control Type answers the question, “What is this element?”  Or, “What is its nature?”  The answer is a noun: it might be a button, or an image, or a document.  Now, we might want to know this so that we can tell the user what the element is – in that case, the answer would be a string.  This is the Localized Control Type.    Alternatively, we might want to know this to make a decision in our code, in which case we really can’t use a localized string – we need a locale-independent identifier.  This is simply the Control Type, which is an integer.  If you use an existing Control Type in UI Automation, the localized string is provided gratis, but if you can’t find one that fits, use the “Custom” Control Type and set the Localized Control Type yourself.

Name: Name answers the question, “What is this instance of the element called?”  You might have a UI surface with several similar edit boxes, say, and you’d need to tell them apart.  A screen reader would just read this out directly.  For this reason, it’s always localized.

Automation ID: Since Name is localized, it’s hard to write automated scripts that rely on it, since they’d break on any other language.  Automation ID answers the question, “What is a locale-independent way to refer to this element?”  It is a string.  Automation IDs are shallow: they identify an element uniquely among its siblings in the automation tree, but not uniquely relative to anything else.  For example, a password dialog might have controls with IDs “Name” and “Password”, even though other dialogs might have controls with the same IDs.  As long as there aren’t two sibling elements with the same Automation ID, all is well. 

Is Keyboard Focusable: “ Can this element have keyboard focus?”

Has Keyboard Focus: “ Does this element have keyboard focus?”  More precisely, “Is this the most derived element with keyboard focus?,” since in some UI frameworks you can imagine a whole chain of UI element ancestors that all have keyboard focus.  In UI Automation, only the lowest child should claim keyboard focus. 

Help Text: “ What more can you tell me about this element besides its name?”  This is a localized string, a longer description of the control.  If you imagine writing a tooltip for the control, that’s usually about the right length.  This would be also be a good place to describe how the user might use the element.

Accelerator Key / Access Key: “ What’s the keyboard command for this?”  This is also a string.  The difference between these two similar properties is that Access keys are primarily for menu items and are often a single letter: the letter that is underlined.  Accelerators are usually valid whenever the app has focus, not just when the menu is open.  So, for instance, in the File menu, the Open item often has an Accelerator Key of “Ctrl+O” and an Access Key of “O”.

Is Control Element / Content Element: “ How important is this element likely to be to the customer?”  These two properties are Booleans that control whether the element is part of the Control Tree or the Content Tree, two filtered views of a UI Automation accessibility tree that try to capture the higher-priority aspects of the tree.  There is a subset relationship here: all content elements should be control elements.  So, the Content Tree is the smallest, highest priority tree, followed by the Control Tree and then the Raw Tree (the largest, unfiltered view).  My simplified explanation for these properties (see MSDN for the full one) is that anything that the user can see and manipulate is a Control Element. Anything that you would want described to you if someone was reading the UI out loud to you is a Content element.  A scroll bar is certainly a Control element, but not a Content element.

Framework ID: “ Which UI framework created this control?”
Provider Description: “ Which UI Automation provider is responding for this control?”  Both of these strings are useful for debugging purposes.  They can tell you where the data for a given element is being generated.  A custom control in WPF might respond that its framework is “WPF” but its provider is “Custom-Drawn Element Provider.” 

I haven’t done all of them – you can play this game by yourself at this point.  When you think about the properties this way, it becomes obvious that certain properties really make sense for an element and others don’t.  For example, if you are doing a password-entry control, IsPassword is very relevant; otherwise, it isn’t.  Trying to decide whether you “have” to implement IsPassword is kind of silly.  If the question is relevant, your customer probably wants to know the answer, so go ahead and answer the question.

Reviewing this list of properties, it becomes clear that many of them never change.  If they don’t change, they can be easily represented as a property bag: a map from property ID to value.  If we implement this way, we can add a new property with a single line of code.  So, I added a property bag to BaseSimpleProvider.  GetPropertyValue() will check the property bag first:

        public virtual object GetPropertyValue(int propertyId)
        {
            // Check the static props list first
            if (this.staticProps.ContainsKey(propertyId))
            {
                return this.staticProps[propertyId];
            }

            // Switching construct to go get the right property from a virtual method.
            if (propertyId == AutomationElementIdentifiers.NameProperty.Id)
            { …

 

Now, I can add a set of new properties to TriColorProvider very easily in the constructor:

 

        public TriColorProvider(TriColorControl control)
        {
            this.control = control;

            // Populate static properties
            //
            AddStaticProperty(AutomationElementIdentifiers.ControlTypeProperty.Id, ControlType.Custom.Id);
            AddStaticProperty(AutomationElementIdentifiers.LocalizedControlTypeProperty.Id, "tri-color picker");
            AddStaticProperty(ProviderDescriptionId, "UIASamples: Tri-Color Provider");
            AddStaticProperty(AutomationElementIdentifiers.HelpTextProperty.Id,
"This is a color picker for a choice of three colors. Use Up and Down arrows to move the selection between the colors.");
            // The WinForm name for this control makes a good Automation ID.
            AddStaticProperty(AutomationElementIdentifiers.AutomationIdProperty.Id, this.control.Name);
            AddStaticProperty(AutomationElementIdentifiers.IsKeyboardFocusableProperty.Id, true);
            AddStaticProperty(AutomationElementIdentifiers.IsControlElementProperty.Id, true);
            AddStaticProperty(AutomationElementIdentifiers.IsContentElementProperty.Id, true);

            // Some properties are provided for me already by HWND provider
            // NativeWindowHandle, ProcessId, FrameworkId, IsEnabled, HasKeyboardFocus
        }

 

Echoing the last comment: do take a look at your control in Inspect before adding new properties, since you will get some properties for free from the HWND provider.  For a full HWND, some of the free properties you get are

NativeWindowHandle, ClassName, ProcessId, FrameworkId, IsEnabled, and HasKeyboardFocus. 

 

When I look at it now in Inspect, you can see all the new values!

 

So, we now have a provider that answers a bunch of useful questions about itself.  But we’ve skipped one important question: what can this control do?  The answer to that question will involve UI Automation Control Patterns, which will be the topic of the next part.

Next time: Adding a control pattern to our provider.

Comments

• Anonymous
  June 04, 2010
  excellent article, very informative netflix-vs-blockbuster.net/blockbuster

• Anonymous
  December 20, 2015
  Nice post. But i could not able to download the sample file. Can i get the sample?

• Anonymous
  December 20, 2015
  Is there any way to show the custom properties?