Share via


UI Automation and Microsoft Active Accessibility

Microsoft Active Accessibility was the earlier solution for making applications accessible. Microsoft UI Automation is the new accessibility model for Microsoft Windows and is intended to address the needs of assistive technology products and automated testing tools. UI Automation offers many improvements over Active Accessibility.

This topic includes the main features of UI Automation and explains how these features differ from Active Accessibility.

This topic contains the following sections.

  • Programming Languages
  • Support in Windows Presentation Foundation
  • Servers and Clients
  • UI Elements
  • Tree Views and Navigation
  • Roles and Control Types
  • States and Properties
  • Events
  • Security
  • Related Topics

Programming Languages

Active Accessibility is based on the Component Object Model (COM) with support for dual interfaces, and is therefore programmable in C/C++, Microsoft Visual Basic 6.0, and scripting languages. UI Automation (including the client-side provider library for standard controls) is written in managed code, and UI Automation client applications are most easily programmed using Microsoft Visual C# or Microsoft Visual Basic .NET. UI Automation providers, which are interface implementations, can be written in managed code or in C/C++.

Support in Windows Presentation Foundation

Windows Presentation Foundation (WPF) is the new model for creating user interfaces. WPF elements do not contain native support for Active Accessibility; however, they do support UI Automation, which includes bridging support for Active Accessibility clients. Only clients written specifically for UI Automation can take full advantage of the accessibility features of WPF, such as the rich support for text.

Servers and Clients

In Active Accessibility, servers and clients communicate directly, largely through the server's implementation of IAccessible.

In UI Automation, a core service lies between the server (called a provider) and the client. The core service makes calls to the interfaces implemented by providers and provides additional services such as generating unique runtime identifiers for elements. Client applications use library functions to call the UI Automation service.

UI Automation providers can provide information to Active Accessibility clients, and Active Accessibility servers can provide information to UI Automation client applications. However, because Active Accessibility does not expose as much information as UI Automation, the two models are not fully compatible.

UI Elements

Active Accessibility presents UI elements either as an IAccessible interface or as a child identifier. It is difficult to compare two IAccessible pointers to determine if they refer to the same element.

In UI Automation, every element is represented as an AutomationElement object. Comparison is done by using the equality operator or the Equals method, both of which compare the unique runtime identifiers of the elements.

Tree Views and Navigation

The user interface (UI) elements on the screen can be seen as a tree structure with the desktop as the root, application windows as immediate children, and elements within applications as further descendants.

In Active Accessibility, many automation elements that are irrelevant to end users are exposed in the tree. Client applications have to look at all the elements to determine which are meaningful.

UI Automation client applications see the UI through a filtered view. The view contains only elements of interest: those that give information to the user or enable interaction. Predefined views of only control elements and only content elements are available; in addition, applications can define custom views. UI Automation simplifies the task of describing the UI to the user and helping the user interact with the application.

Navigation between elements, in Active Accessibility, is either spatial (for example, moving to the element that lies to the left on the screen), logical (for example, moving to the next menu item, or the next item in the tab order within a dialog box), or hierarchical (for example, moving the first child in a container, or from the child to its parent). Hierarchical navigation is complicated by the fact that child elements are not always objects that implement IAccessible.

In UI Automation, all UI elements are AutomationElement objects that support the same basic functionality. (From the standpoint of the provider, they are objects that implement an interface inherited from IRawElementProviderSimple.) Navigation is mainly hierarchical: from parents to children, and from one sibling to the next. (Navigation between siblings has a logical element, as it may follow the tab order.) You can navigate from any starting-point, using any filtered view of the tree, by using the TreeWalker class. You can also navigate to particular children or descendants by using FindFirst and FindAll; for example, it is very easy to retrieve all elements within a dialog box that support a specified control pattern.

Navigation in UI Automation is more consistent than in Active Accessibility. Some elements such as drop-down lists and pop-up windows appear twice in the Active Accessibility tree, and navigation from them may have unexpected results. It is actually impossible to properly implement Active Accessibility for a rebar control. UI Automation enables reparenting and repositioning, so that an element can be placed anywhere in the tree despite the hierarchy imposed by ownership of windows.

Roles and Control Types

Active Accessibility uses the accRole property (IAccessible::get_actRole) to retrieve a description of the element's role in the UI, such as ROLE_SYSTEM_SLIDER or ROLE_SYSTEM_MENUITEM. The role of an element is the main clue to its available functionality. Interaction with a control is achieved by using fixed methods such as IAccessible::accSelect and IAccessible::accDoDefaultAction. The interaction between the client application and the UI is limited to what can be done through IAccessible.

In contrast, UI Automation largely decouples the control type of the element (described by the ControlType property) from its expected functionality. Functionality is determined by the control patterns that are supported by the provider through its implementation of specialized interfaces. Control patterns can be combined to describe the full set of functionality supported by a particular UI element. Some providers are required to support a particular control pattern; for example, the provider for a check box must support the Toggle control pattern. Other providers are required to support one or more of a set of control patterns; for example, a button must support either Toggle or Invoke. Still others support no control patterns at all; for example, a pane that cannot be moved, resized, or docked does not have any control patterns.

UI Automation supports custom controls, which are identified by the Custom property and can be described by the LocalizedControlTypeProperty property.

The following table shows the mapping of Active Accessibility roles to UI Automation control types.

Active Accessibility role

UI Automation control type

ROLE_SYSTEM_PUSHBUTTON

Button

ROLE_SYSTEM_CLIENT

Calendar

ROLE_SYSTEM_CHECKBUTTON

Check box

ROLE_SYSTEM_COMBOBOX

Combo box

ROLE_SYSTEM_CLIENT

Custom

ROLE_SYSTEM_LIST

Data grid

ROLE_SYSTEM_LISTITEM

Data item

ROLE_SYSTEM_DOCUMENT

Document

ROLE_SYSTEM_TEXT

Edit

ROLE_SYSTEM_GROUPING

Group

ROLE_SYSTEM_LIST

Header

ROLE_SYSTEM_COLUMNHEADER

Header item

ROLE_SYSTEM_LINK

Hyperlink

ROLE_SYSTEM_GRAPHIC

Image

ROLE_SYSTEM_LIST

List

ROLE_SYSTEM_LISTITEM

List item

ROLE_SYSTEM_MENUPOPUP

Menu

ROLE_SYSTEM_MENUBAR

Menu bar

ROLE_SYSTEM_MENUITEM

Menu item

ROLE_SYSTEM_PANE

Pane

ROLE_SYSTEM_PROGRESSBAR

Progress bar

ROLE_SYSTEM_RADIOBUTTON

Radio button

ROLE_SYSTEM_SCROLLBAR

Scroll bar

ROLE_SYSTEM_SEPARATOR

Separator

ROLE_SYSTEM_SLIDER

Slider

ROLE_SYSTEM_SPINBUTTON

Spinner

ROLE_SYSTEM_SPLITBUTTON

Split button

ROLE_SYSTEM_STATUSBAR

Status bar

ROLE_SYSTEM_PAGETABLIST

Tab

ROLE_SYSTEM_PAGETAB

Tab item

ROLE_SYSTEM_TABLE

Table

ROLE_SYSTEM_STATICTEXT

Text

ROLE_SYSTEM_INDICATOR

Thumb

ROLE_SYSTEM_TITLEBAR

Title bar

ROLE_SYSTEM_TOOLBAR

Tool bar

ROLE_SYSTEM_TOOLTIP

ToolTip

ROLE_SYSTEM_OUTLINE

Tree

ROLE_SYSTEM_OUTLINEITEM

Tree item

ROLE_SYSTEM_WINDOW

Window

For more information about the different control types, see UI Automation Control Types.

States and Properties

In Active Accessibility, elements support a common set of properties, and some properties (such as accState) must describe very different things, depending on the element's role. Servers must implement all methods of IAccessible that return a property, even those that are not relevant to the element.

UI Automation defines many more properties, some of which correspond to states in Active Accessibility. Some are common to all elements, but others are specific to control types and control patterns. Properties are distinguished by unique identifiers, and most properties can be retrieved by using a single method, GetCurrentPropertyValue or GetCachedPropertyValue. Many properties are also easily retrievable from the Current and Cached property accessors.

A UI Automation provider does not have to implement irrelevant properties, but can simply return a null value for any properties it does not support. Also, the UI Automation core service can obtain some properties from the default window provider, and these are amalgamated with properties explicitly implemented by the provider.

As well as supporting many more properties, UI Automation supplies better performance by allowing multiple properties to be retrieved with a single cross-process call.

The following table shows the correspondence between properties in the two models.

Active Accessibility property accessor

UI Automation property ID

Remarks

get_accKeyboardShortcut

AccessKeyProperty or AcceleratorKeyProperty

AccessKeyProperty takes precedence if both are present.

get_accName

NameProperty

get_accRole

ControlTypeProperty

See the previous table for mapping of roles to control types.

get_accValue

ValuePattern.ValueProperty

RangeValuePattern.ValueProperty

Valid only for control types that support ValuePattern or RangeValuePattern. RangeValue values are normalized to 0-100, to be consistent with MSAA behavior. Value items use a string.

get_accHelp

HelpTextProperty

accLocation

BoundingRectangleProperty

get_accDescription

Not supported in UI Automation

accDescription did not have a clear specification within MSAA, which resulted in providers placing different pieces of information in this property.

get_accHelpTopic

Not supported in UI Automation

The following table shows which UI Automation properties correspond to Active Accessibility state constants.

Active Accessibility state

UI Automation property

Triggers State Change?

STATE_SYSTEM_CHECKED

For check box, ToggleStateProperty

For radio button, IsSelectedProperty

Y

STATE_SYSTEM_COLLAPSED

ExpandCollapseState = Collapsed

Y

STATE_SYSTEM_EXPANDED

ExpandCollapseState = Expanded or PartiallyExpanded

Y

STATE_SYSTEM_FOCUSABLE

IsKeyboardFocusableProperty

N

STATE_SYSTEM_FOCUSED

HasKeyboardFocusProperty

N

STATE_SYSTEM_HASPOPUP

ExpandCollapsePattern for menu items

N

STATE_SYSTEM_INVISIBLE

IsOffscreenProperty = True and GetClickablePoint causes NoClickablePointException

N

STATE_SYSTEM_LINKED

ControlTypeProperty =

Hyperlink

N

STATE_SYSTEM_MIXED

ToggleState = Indeterminate

N

STATE_SYSTEM_MOVEABLE

CanMoveProperty

N

STATE_SYSTEM_MUTLISELECTABLE

CanSelectMultipleProperty

N

STATE_SYSTEM_OFFSCREEN

IsOffscreenProperty = True

N

STATE_SYSTEM_PROTECTED

IsPasswordProperty

N

STATE_SYSTEM_READONLY

RangeValuePattern.IsReadOnlyProperty and ValuePattern.IsReadOnlyProperty

N

STATE_SYSTEM_SELECTABLE

SelectionItemPattern is supported

N

STATE_SYSTEM_SELECTED

IsSelectedProperty

N

STATE_SYSTEM_SIZEABLE

CanResize

N

STATE_SYSTEM_UNAVAILABLE

IsEnabledProperty

Y

The following states either were not implemented by most Active Accessibility control servers or have no equivalent in UI Automation.

Active Accessibility state

Remarks

STATE_SYSTEM_BUSY

Not available in UI Automation

STATE_SYSTEM_DEFAULT

Not available in UI Automation

STATE_SYSTEM_ANIMATED

Not available in UI Automation

STATE_SYSTEM_EXTSELECTABLE

Not widely implemented by Active Accessibility servers

STATE_SYSTEM_MARQUEED

Not widely implemented by Active Accessibility servers

STATE_SYSTEM_SELFVOICING

Not widely implemented by Active Accessibility servers

STATE_SYSTEM_TRAVERSED

Not available in UI Automation

STATE_SYSTEM_ALERT_HIGH

Not widely implemented by Active Accessibility servers

STATE_SYSTEM_ALERT_MEDIUM

Not widely implemented by Active Accessibility servers

STATE_SYSTEM_ALERT_LOW

Not widely implemented by Active Accessibility servers

STATE_SYSTEM_FLOATING

Not widely implemented by Active Accessibility servers

STATE_SYSTEM_HOTTRACKED

Not available in UI Automation

STATE_SYSTEM_PRESSED

Not available in UI Automation

For a complete list of UI Automation property identifiers, see UI Automation Properties Overview.

Events

The event mechanism in UI Automation, unlike that in Active Accessibility, does not rely on Windows event routing (which is closely tied in with window handles) and does not require the client application to set up hooks. Subscriptions to events can be fine-tuned not just to particular events but to particular parts of the tree. Providers can also fine-tune their raising of events by keeping track of what events are being listened for.

It is also easier for clients to retrieve the elements that raise events, as these are passed directly to the event callback. Properties of the element are automatically prefetched if a cache request was active when the client subscribed to the event.

The following table shows the correspondence of Active Accessibility WinEvents and UI Automation events.

WinEvent

UI Automation event identifier

EVENT_OBJECT_ACCELERATORCHANGE

AcceleratorKeyProperty property change

EVENT_OBJECT_CONTENTSCROLLED

VerticalScrollPercentProperty or HorizontalScrollPercentProperty property change on the associated scroll bars

EVENT_OBJECT_CREATE

StructureChangedEvent

EVENT_OBJECT_DEFACTIONCHANGE

No equivalent

EVENT_OBJECT_DESCRIPTIONCHANGE

No exact equivalent; perhaps HelpTextProperty or LocalizedControlTypeProperty property change

EVENT_OBJECT_DESTROY

StructureChangedEvent

EVENT_OBJECT_FOCUS

AutomationFocusChangedEvent

EVENT_OBJECT_HELPCHANGE

HelpTextProperty change

EVENT_OBJECT_HIDE

StructureChangedEvent

EVENT_OBJECT_LOCATIONCHANGE

BoundingRectangleProperty property change

EVENT_OBJECT_NAMECHANGE

NameProperty property change

EVENT_OBJECT_PARENTCHANGE

StructureChangedEvent

EVENT_OBJECT_REORDER

Not consistently used in Active Accessibility. No directly corresponding event is defined in UI Automation.

EVENT_OBJECT_SELECTION

ElementSelectedEvent

EVENT_OBJECT_SELECTIONADD

ElementAddedToSelectionEvent

EVENT_OBJECT_SELECTIONREMOVE

ElementRemovedFromSelectionEvent

EVENT_OBJECT_SELECTIONWITHIN

No equivalent

EVENT_OBJECT_SHOW

StructureChangedEvent

EVENT_OBJECT_STATECHANGE

Various property-changed events

EVENT_OBJECT_VALUECHANGE

RangeValuePattern.ValueProperty and ValuePattern.ValueProperty changed

EVENT_SYSTEM_ALERT

No equivalent

EVENT_SYSTEM_CAPTUREEND

No equivalent

EVENT_SYSTEM_CAPTURESTART

No equivalent

EVENT_SYSTEM_CONTEXTHELPEND

No equivalent

EVENT_SYSTEM_CONTEXTHELPSTART

No equivalent

EVENT_SYSTEM_DIALOGEND

WindowClosedEvent

EVENT_SYSTEM_DIALOGSTART

WindowOpenedEvent

EVENT_SYSTEM_DRAGDROPEND

No equivalent

EVENT_SYSTEM_DRAGDROPSTART

No equivalent

EVENT_SYSTEM_FOREGROUND

AutomationFocusChangedEvent

EVENT_SYSTEM_MENUEND

MenuClosedEvent

EVENT_SYSTEM_MENUPOPUPEND

MenuClosedEvent

EVENT_SYSTEM_MENUPOPUPSTART

MenuOpenedEvent

EVENT_SYSTEM_MENUSTART

MenuOpenedEvent

EVENT_SYSTEM_MINIMIZEEND

WindowVisualStateProperty property change

EVENT_SYSTEM_MINIMIZESTART

WindowVisualStateProperty property change

EVENT_SYSTEM_MOVESIZEEND

BoundingRectangleProperty property change

EVENT_SYSTEM_MOVESIZESTART

BoundingRectangleProperty property change

EVENT_SYSTEM_SCROLLINGEND

VerticalScrollPercentProperty or HorizontalScrollPercentProperty property change

EVENT_SYSTEM_SCROLLINGSTART

VerticalScrollPercentProperty or HorizontalScrollPercentProperty property change

EVENT_SYSTEM_SOUND

No equivalent

EVENT_SYSTEM_SWITCHEND

No equivalent, but an AutomationFocusChangedEvent event signals that a new application has received the focus

EVENT_SYSTEM_SWITCHSTART

No equivalent

No equivalent

CurrentViewProperty property change

No equivalent

ValueProperty property change

No equivalent

HorizontallyScrollableProperty property change

No equivalent

VerticallyScrollableProperty property change

No equivalent

HorizontalScrollPercentProperty property change

No equivalent

VerticalScrollPercentProperty property change

No equivalent

HorizontalViewSizeProperty property change

No equivalent

VerticalViewSizeProperty property change

No equivalent

ToggleStateProperty property change

No equivalent

ValueProperty property change

No equivalent

WindowVisualStateProperty property change

No equivalent

AsyncContentLoadedEvent property change

No equivalent

ToolTipOpenedEvent

Security

Some IAccessible customization scenarios require wrapping a base IAccessible and calling through to it. This has security implications, since a partially trusted component should not be an intermediary on a code path.

The UI Automation model removes the need for providers to call through to other provider code. The UI Automation core service does all the necessary aggregation.

See Also

Other Resources

UI Automation Fundamentals