Accessibility Gotchas 1: Xaml ListView speaks in tongues
I keep having the same conversation with a blind colleague:
It always starts out well: Rob! Did you hear that Contoso has published their app to the Windows Store? It should be great! I’ve really been looking forward to this one!
But then goes down hill: But I can’t use it… it has the same problem Fabrikam’s app had! When I try to tab through its lists Narrator starts speaking in tongues!
======
Let’s examine what this problem is, why it occurs, and how to avoid it. As promised last time: we can quick fix it in one line of code.
First, what is the problem?
This is a symptom that can occur with any data bound items control, such as a ListView or a GridView. The ListView may look great on the visual display, but when read by a screen reader such as Narrator instead of correctly describing the list items it reads what sounds like gibberish to the non-technical user.
I’ve attached a sample project (based on the blog reader tutorial) that you can hear this with for yourself. Build the sample, run Narrator, start the sample and turn off your monitor. Once the initial descriptions settle down focus will be in the GridView. Use the arrow keys to examine the tiles.
You’ll hear something like:
WindowsBlogReader.FeedData (1 of 14)… WindowsBlogReader.FeedData (2 of 14)… WindowsBlogReader.FeedData (3 of 14)…
As a developer you may recognize what this is, but to the user it is opaque gibberish. Instead of reading a useful description of the tile, Narrator reads the class name of the bound data.
Why does this occur?
To answer this we need to know a bit more about what Narrator does, and for that lets take a very brief look at the Windows Accessibility system. You can find more information in the Windows Automation API documentation on MSDN (accessibility and automation go hand in hand: accessibility apps such as screen readers need the same information that automation apps such as Visual Studio’s Coded UI need).
Each app contains an accessibility tree which describes the hierarchy of elements that define window layout. It has containers such as panes and panels and objects such as text or buttons. Each element has standard properties such as the Name which is read to the user, the ID which can be used to programmatically identify the item, and the BoundingRectangle which describes where the control is on the screen. These properties apply to all items. Elements can also have patterns such as TextPattern or SelectionPattern which provide role specific behaviors.
Accessibility apps can walk this hierarchy and use these properties to identify which controls are where. Combined with accessibility events the app can identify and report on the controls the user is actively interacting with.
If we examine the blog reader’s Xaml we can see several controls which define AutomationProperties.Name and AutomationProperties.AutomationId values.
<GridView
x:Name="itemGridView"
AutomationProperties.AutomationId="ItemsGridView"
AutomationProperties.Name="Items"
…..
And others where these are left default.
The Windows SDK comes with several tools to examine the accessible information.
We started off with screen readers like Narrator, which are what real users use, and so it is the best way to get the real experience of what the app will sound like. You can put on your headphones, run Narrator, and then go through your app’s key scenarios and listen to what it reports. Think about the information that you hear: is it sufficient to successfully complete the scenario?
The Windows 8.1 SDK also has several developer oriented tools for examining accessibility. Inspect.exe displays the entire accessibility tree and all of the properties and patterns for each element. AccScope.exe visually displays the accessible objects as a wireframe with key properties listed. It can give a visual view of what Narrator will report.
You can find both of these in the SDK’s bin directory (by default in C:\Program Files (x86)\Windows Kits\8.0\bin\x64 ). You can use Iinspect to browse the app’s tree to make sure that all of the names and descriptions are reasonable.
If we look at the blog app in Inspect we see the Accessible Name is WindowsBlogReader.DataFeed, the same as Narrator reports:
AccScope shows the same for the container (item 1), and also shows that the internal information is available once we can get past the container to see it. Narrator will be able to browse in for the details once the user finds the right tile to select.
So where does this Name come from if we didn’t set it?
If we don’t say how to display an object Windows will fall back to the object’s ToString() value, and if you don’t override ToString then ToString will return the class name.
The quick fix
This leads to the one line fix: override ToString in the data objects to return useful data:
public override string ToString() { return Title + " " + Description; }
Try it out: add ToString overrides to FeedData and FeedItem then give the sample another run through Narrator.
Much better!
This is bare bones, but sufficient if we have simple data. It is how the Visual Studio’s Grid App and Hub App avoid this problem.
The full implementation
While this will let users navigate the list with Narrator, it provides the bare minimum information. It is similar to creating the list without a data template, and if we don’t set up the data template we’ll get the same ToString default behavior:
For a more thorough fix, we can set AutomationProperties on the container object. This is necessary to provide more information than just the Name to read. As we saw earlier, it’s easy to set AutomationProperties on a static control like the GridView itself. It is trickier for the generated containers, since they aren’t directly represented in Xaml.
If we have more complex data or need to set other properties then we can subclass the GridView and use the PrepareContainerForItemOverride method to bind the AutomationProperties to properties on the data-bound class. The XAML accessibility sample demonstrates this in Scenario 4:
public class MyGrid : GridView
{
protected override void PrepareContainerForItemOverride(DependencyObject element, object item)
{
base.PrepareContainerForItemOverride(element, item);
FrameworkElement source = element as FrameworkElement;
source.SetBinding(AutomationProperties.AutomationIdProperty, new Binding { Path = new PropertyPath("AutomationId") });
source.SetBinding(AutomationProperties.NameProperty, new Binding { Path = new PropertyPath("AutomationName") });
}
}
The takeaway
Either way, by deliberately setting the item’s Name our apps can provide useful information to screen readers instead of exposing internal class names. Take a few minutes to run your app scenarios through Narrator to make sure the app reports useful information.
I look forward to hearing about great apps being published that are great for everybody, not just for sighted users!
Previous entries in this series:
Accessibility Gotchas: Introduction
Comments
Anonymous
February 12, 2014
Surely you can do better than this. If the information is correctly visible on screen, you could certainly set the automation properties from the data temlate yourselves, instead of relying on the developer to do it. Most will simply not know or bother with this.Anonymous
February 19, 2014
@N, It would be lovely if this could be queried from the data template, but in the current implementation that isn't possible. Unfortunately you are right that many developers do not know about this. That is the purpose of this blog post! The problem will be obvious if you test with Narrator. The information used here isn't "correctly visible on screen": the visible rendering is the selection container which doesn't include text. What you can see is the child of the object we're looking at here, which doesn't currently include summary information.In the AccScope image we're talking about item 1, which has no visible text. The visibly named items are 2-5. Once the user tabs into the tile or touches over the visible information can be read, but before you get there you need to be able to choose a tile. It's at that choice that we need the summary information. For basic use setting ToString() to something descriptive is both straightforward and sufficient.