RichEdit 8.0 Accessibility

An important part of a computer’s user interface is to provide for user interaction alternatives to the usual sight-oriented methods. Such capability requires programmatic access to what’s on the screen so that screen reader programs can express the content using voice or other means. Microsoft UI Automation (UIA) provides such access. A valuable side benefit is that test scripts can use UIA to test the content of a wide variety of programs without having to understand the individual program object models.

RichEdit 8 has an implementation of UIA that exposes most objects in a RichEdit instance. This is done via the UIA Text Pattern and includes basic character and paragraph formatting, images, OLE objects, math zones, tables, hyperlinks, and the text inside or associated with these objects. The implementation is showcased in WordPad on Windows 8 and provides a popular (inside Microsoft) example of how to implement UIA for text applications. All UIA value providers are read only. For example, you can read out a URL’s text but cannot change it. To change it programmatically, you can use RichEdit’s Text Object Model (TOM) or the Microsoft Text Services Framework.

If you implement a windowless RichEdit control, you hook up its UIA tree by calling ITextServices::QueryInterface() for an IRicheditWindowlessAccessibility interface and then call IRicheditWindowlessAccessibility::CreateProvider() with an IRawElementProviderWindowlessSite to get the UIA standard base interface IRawElementProviderSimple.

You can examine the complete UIA tree using the Microsoft SDK inspect.exe. The raw text for a RichEdit instance is given for the “RichEdit Control” document node. This raw text includes the structure characters defining any table-row start and end delimiters and cell markers, as well as the math structure characters for math objects like fractions. A math-zone object is treated as an image with a text string giving the math in the linear format. That format gives a pretty good idea of the math content especially if it’s the way the user enters math. Tables are shown as a tree that can be expanded to show each cell’s plain text.

The UIA text range is similar to the original TOM ITextRange and the TOM specification influenced some of the UIA text range choices. Unlike ITextRange2, the UIA text range doesn’t provide access to math text semantics. In this connection, it could be handy if UIA added an ITextMathProvider interface to aid in communicating math text verbally. Alternatively a UIA custom property could be defined that returns the MathML for math zones. MathML has become an important format in math accessibility. Only one property would be needed. Its UIAutomationPropertyInfo structure would contain the property GUID, the UIAutomationPropertyInfo::pProgrammaticName = L”MathML” and the UIAutomationPropertyInfo::type = UIAutomationType_String. One could be more specific and use L”Presentation MathML”, but since it’s part of a text pattern, a client would probably assume it’s Presentation MathML and not Content MathML.

An advantage of a built-in math provider is that the internal Office representation of math is a bit closer to what one would say in speaking the math. For example, the internal (and external OMML) representation uses an n-ary object for integrals, summations, and the like, so a reader immediately knows the kind of operation involved. This is simpler than MathML’s use of <msub>, <msup>, <msubsup>, <mover>, <munder>, and <munderover>, all of which represent n-ary expressions if their base is an n-ary operator. But appropriate programming can parse the MathML and there are readers that make MathML accessible, so a MathML UI custom property is an easy way to math-enable UIA. For MathML-unaware UIA clients, the RichEdit 8.0 image object with a math linear format string is a good fallback.

Thanks go to Rick Sailor, who implemented RichEdit 8.0's UIA support and answered many questions about it.

Comments

• Anonymous
  October 20, 2012
  As a blind user, I have to say that even though Narrator in Windows 8 allows you to read by word, paragraph, character, etc, in Rich Edit 8.0 and in IE, this functionality is not extended to other places in the OS. So, if I want for example to read an application's name word by word because I don't understand the way it is pronounced or to read a notification character by character, I can't. This is a big inconsistnecy (Rich Edit and IE HTML controls behave one way whilst other controls another) which does not exist for example on the iPhone, where Voiceover could, from its first version, read by character, word, paragraph, etc, in all places.