Partager via

Windows Automation API SDK Tools

  • Article

Windows Automation API 3.0

Some of you may be wondering why we refer to the Microsoft Accessibility programming platform as "Windows Automation API 3.0". We named it that way because the SDK isn’t really just about MSAA or UI Automation anymore. It's a combination of both old and new technologies. For instance, the Dynamic Annotation API is no longer unique to MSAA Servers – it can be used by these applications to add support or annotate UI Automation properties. The IAccessibleEx interface bridges two worlds together in a very organic way. Even from UI Automation, new LegacyIAccessible control pattern and the new Client API (i.e., ElementFromIAccessible) function enables jumping between MSAA accessible object tree and UI Automation element tree. We set the version of Windows Automation API to 3.0 in order to respect the previous MSAA version number as well as the generations of new UI Automation implementations in Windows.

When we talk about technologies specifics, we continue to use 'UI Automation' or 'MSAA'. Windows Automation API is used when we talk about collections of technologies or the Windows specific frameworks.

SDK Tools

The Windows 7 RC SDK features a new Inspect Object tool (Inspect) for the Windows Automation API SDK. It may look familiar to some of you who’ve been using the old Microsoft Active Accessibilty (MSAA) tool, however the bulk of the code is actually re-written from scratch. The new Inspect tool features the essence of UI Automation included in Windows 7.

Core Tools

Microsoft Windows SDK for Windows 7 (Release Candidate) includes all of the tools. The latest revisions of Windows Automation API SDK tools are:

  • Inspect Object (Inspect.exe) features accessible UI inspection by hit-testing or event tracking either by MSAA or by UI Automation. It uses the latest UI Automation Client COM API and it has most coverage of the ad-hoc UI testing by Windows Automation API. Important: New UIA mode of the Inspect.exe has dependency on new Windows Automation API framework – you have to have pre-release Windows 7 environment to use the new mode as of today.
  • Accessible Event Watcher (AccEvent.exe) tracks WinEvents for MSAA. The current version is limited to WinEvents, however we are planning to extend it to support UI Automation events in the future. Since WinEvent isn’t limited just for MSAA, you can also use the tool for general purpose of tracking WinEvents sent across the desktop. (FYI: You can actually see some UIA events that are based on IAccessibleEx or some (not all) native UIA implementations. Those events are actually visible by AccEvent by choosing "(unknown)" and "Event" in the "Events" and "Event Information" options of AccEvent. You can find those WinEvent based UI Automation event IDs from UIAutomationClient.h by searching "UIA_EventIds" or "UIA_PropertyIds" (for corresponding "property changed events" for UI Automation). )
  • UI Spy (UISpy.exe) is as if Inspect and AccEvent combined with neat object tree presentation for UI Automation. The tool is written in managed UI Automation client API – the tool doesn’t feature new UI Automation control patterns and properties introduced with Windows 7 because the managed API hasn’t updated with the OS release.

Inspect

It's worth highlighting some of the new tricks for UI Automation support using Inspect. You can now inspect brand new UI surfaces that support UI Automation natively. Following are a list of things you can try out with Windows 7 RC and Internet Explorer 8.

  • New UI Automation AriaRole and AriaProperties properties for web contents that supports ARIA and displayed by the Internet Explorer. You can find IE8 specifics of the implementation under their SDK. You can find the original specification from UI Automation Community Promise, too.
  • Most of UI Automation Control Pattern methods are accessible though the “Action” menu in the UIA mode. From simple Invoke.Invoke function to ItemContainer.FindItemByPropertycontrol; ad-hoc investigation can be achieved easily and quickly.

You can find all keyboard shortcuts and other key features of Inspect Object tool at https://msdn.microsoft.com/en-us/library/dd318521(VS.85).aspx.

The SDK and platform are still under development, therefore things may still change, but we hope you find the tools useful in the meantime. Keep in mind that the MSDN online Library is still being updated with new changes, but these should get you started in the right direction.

Old versions of these tools are located in the Microsoft Download Center as MSAA 2.0 SDK tools, however these are retiring with the release of new versions (with the exception of AccExplorer).

UIAVerify and AccChecker

While they are not technically SDK tools, there are also a couple of other tools that you can leverage (one called 'UI Accessibility Checker' and the other 'UI Automation Verify',) but it would take us an entire post to describe them! If you'd like to learn a bit more about what they do, check out https://www.codeplex.com/AccCheck and https://www.codeplex.com/UIAutomationVerify. In short, UI Automation Verify is designed for integrating your UI Automation Provider testing into your test pass and UI Accessibility Checker features a set of accessibility test scenarios and criteria for general UI accessibility testing.

Common tips and suggestions

There are few common tips and useful suggestions to keep in mind whenever any of these accessibility test tools are used.

  • Launch tools as elevated (with Windows Vista, Windows Server 2008, or Windows 7):
    With increased security of the desktop by Windows, accessibility tools should be either signed with UIAccess flag in the manifest (and installed on either under %programfiles% or under %windir%) or you could get blocked from certain types of UI surface. Quick workaround is to launch the tool elevated, by either right click (shift+f10) the executable and select “Run as administrator” or choose compatibility tab “Run this program as an administrator” in file properties of the executable (e.g., Inspect.exe).
  • Do not mix hit testing and focus tracking (or any other mix):
    While Inspect is set to follow both mouse pointer and focus events (for convenience of quickly locating UI on screen), doing mixed texting can be very confusing even with ad-hoc investigations. You should stick to either just mouse for keyboard to navigate the UI, and make sure you choose right combination for the object tracking; “focus tracking with keyboard navigation” OR “mouse cursor tracking with hit-testing.” You can use tools’ keyboard shortcut (e.g., CTRL+SHIFT+F6) to navigate the object structure without disturbing the focus or mouse pointer location if necessary.
  • Do mix tools per scenarios and purposes
    While many people confused Inspect as test tool for focus tracking – it simply is not. Inspect is certainly handy for doing sanity check of focus location on desktop, but timing issues of focus event(s) can fool you very badly. Simply, Inspect isn’t simply design to achieve the goal of “any event tracking” – AccEvent or UISpy’s UIA event logger is designed for the very purpose. So do mix or use right set of tools when your test scenario is looking for particular scenarios. Again, Inspect isn’t tool for focus tracking test.

Quick References and Useful Links

Supports MSAA?

Support UIA?

Distribution

Supported test scenarios

Inspect

Yes

Yes

Windows 7 SDK (RC)

Ad-hoc UI accessibility investigation by focus or by a point on screen

AccEvent

Yes

No

Windows 7 SDK (RC)

WinEvent logging

UISpy

No

Yes*

Windows 7 SDK (RC)

Ad-hoc UI accessibility investigation with UIA Event logging support

UIAVerify

No

Yes*

CodePlex

Automated verification of UIA Provider implementations (Ad-hoc investigation tool available)

AccExplorer

Yes

No

MSDN download

Ad-hoc MSAA Server testing with preset test criteria per MSAA specification (no longer supported or maintained)

AccChecker

Yes

No

CodePlex

General UI Accessibility test tool featuring some huristic based MSAA based testing.

*UISpy and UIAVerify are based on managed UI Automation Client API which has not been fully updated with the Windows 7 version of UI Automation implementation. The tool may not find properties, patterns or events that are not yet supported by the managed API.

Just a note - We are aware of the broken table-of-contents (TOC) with MSDN Library “UI Automation for Win32 Applications” section and apologize for any inconvenience. Once the TOC is fixed, we should have all the latest SDK contents featuring the new UI Automation Specifications implemented by Windows 7. In addition, the navigation is supposed to be replaced by “Windows Automation API: UI Automation” containing additional content.

Comments

  • Anonymous
    June 19, 2009
    PingBack from http://mydebtconsolidator.info/story.php?id=20295

  • Anonymous
    October 18, 2010
    Can you open the AccEvent source code for those who want learn UI Automation as a (about event's work threads problem) samples? Thank you very muck!

  • Anonymous
    May 05, 2011
    I want to use Insepct on Windows XP SP3. I installed Microsoft SDK 7.1 and tried to launch Inspect. I get an error when I launch that the program cannot query the UI Automation APIs.From where can I download these?

  • Anonymous
    May 05, 2011
    Maureen - This is the article that talks about downloading Windows Automation API for XP: support.microsoft.com/.../971513.

  • Anonymous
    December 14, 2011
    Nursing programs: www.nursingprogramtraining.com/schools.php

  • Anonymous
    March 15, 2012
    Where do i get Inspect.exe tool . I need it.Help

  • Anonymous
    January 10, 2015
    Hi. Can you add Automation Spy (http://automationspy.com/) to the list of Windows Automation tools? @Venkatesh K: Maybe my answer is a little late but if you want Inspect.exe you need to install Windows SDK (www.microsoft.com/.../details.aspx). If you have Windows 7 you will find it in "Start/All Programs/Microsoft Windows SDK v7.1/Tools/Inspect Objects".