Updating the explorable map to add support for scrolling with a screen reader
I recently had some fun building an app which presents a map, and when a Narrator user touches the map, the name of the state, county or city beneath the finger can be spoken. Details of how I did this are at Using UI Automation to explore a map, and the app itself can be downloaded from the Windows Store at State Your Name Please.
After having done this, I got the great feedback that it really would be preferable for the app to support panning the map via Narrator’s scroll gestures, rather than having to interact with the four explicit panning buttons that I’d added.
It turned out this was really easy for me to implement. All I had to do was add support for the UI Automation Scroll Pattern to my custom AutomationPeer which provides the accessibility support for my custom control hosting the map control. By doing this, Narrator would be told that the UI element that it’s interacting with can be scrolled, and Narrator could ask the element to scroll in whatever direction that the user wants to pan.
There are three steps to adding support for the UIA Scroll Pattern to my custom AutomationPeer.
1. Have the AutomationPeer derive from IScrollProvider.
2. Implement the members of iScrollProvider.
3. Override my custom AutomationPeer's GetPatternCore() method to provide UIA clients with an object which supports IScrollProvider.
Note that for this experiment I only implemented IScrollProvider to a point where my app’s map could be scrolled via Narrator’s touch gestures. If this was a shipping app, I’d complete the IScrollProvider implementation.
The code snippets below show how I updated my custom AutomationPeer to allow Narrator to scroll the map.
// Add support for the UI Automation Scroll Pattern to the custom control
// which hosts the Map control.
class CustomAutomationPeer : FrameworkElementAutomationPeer, IScrollProvider
{
// When UIA asks this custom AutomationPeer for an object which implements
// the Scroll Pattern, return the peer itself.
protected override object GetPatternCore(PatternInterface patternInterface)
{
if (patternInterface == PatternInterface.Scroll)
{
return this;
}
return base.GetPatternCore(patternInterface);
}
// Now implement the Scroll Pattern's properties and methods.
// Say that the element can scroll both horizontally and vertically.
public bool VerticallyScrollable
{
get
{
return true;
}
}
public bool HorizontallyScrollable
{
get
{
return true;
}
}
// Vertical scrolling extends from the North Pole to the South Pole. (The user cannot
// pan the map beyond the poles.) So consider a scroll position of 0% to be the South
// Pole and 100% to be the North Pole.
public double VerticalScrollPercent
{
get
{
// Convert the map center's latitude into the appropriate scroll position.
// Future: Using the center of the map means that in practice the user
// nevers actually reaches 0% or 100%. (Instead the scroll is limited
// to 3% to 97%.) Consider changing this such that from the user's
// perspective, the scroll is between 0% and 100%.
return (((mapContainer.MapControl.Center.Latitude + 90.0) * 5.0) / 9.0);
}
}
// Horizontal scrolling is effectively infinite, because the map can be panned
// left and right forever. As such, we cannot return a meaningful percentage
// to UIA for the current horizontal scroll percentage. If we were to consider
// some particular point as both 0% and 100%, (eg the International Date Line,)
// then Narrator would not allow the user to scroll beyond that, and this app
// wants to allow infinite panning.
public double HorizontalScrollPercent
{
get
{
// Return a value such that Narrator will always allow left and right scrolling,
// (despite the value being meaningless to the user.)
return 1;
}
}
// A UIA client has programmatically requested that the element scroll.
public void Scroll(ScrollAmount horizontalAmount, ScrollAmount verticalAmount)
{
// The "PanType" here is an app-specific type used as part of this
// custom AutomationPeer communicating with the custom control
// which hosts the map control.
PanType panType = PanType.panNone;
// FUTURE: Requests for small and large scrolls result in the
// same pan distance on the map. Update this to have a large
// scroll pan further than a small scroll.
if ((horizontalAmount == ScrollAmount.LargeIncrement) ||
(horizontalAmount == ScrollAmount.SmallIncrement))
{
panType = PanType.panRight;
}
else if ((horizontalAmount == ScrollAmount.LargeDecrement) ||
(horizontalAmount == ScrollAmount.SmallDecrement))
{
panType = PanType.panLeft;
}
else if ((verticalAmount == ScrollAmount.LargeIncrement) ||
(verticalAmount == ScrollAmount.SmallIncrement))
{
panType = PanType.panDown;
}
else if ((verticalAmount == ScrollAmount.LargeDecrement) ||
(verticalAmount == ScrollAmount.SmallDecrement))
{
panType = PanType.panUp;
}
if (panType != PanType.panNone)
{
// Now ask the custom control hosting the map control to pan
// the map in the appropriate direction.
mapContainer.ScrollMap(panType);
}
}
// Future: Implement the following...
public double VerticalViewSize { get; set; }
public double HorizontalViewSize { get; set; }
public void SetScrollPercent(double horizontalPercent, double verticalPercent) { }
And this is the method that I added to another class in my app, which gets called by the custom AutomationPeer when the map is to be panned in response to a request to scroll via UIA.
public void ScrollMap(PanType type)
{
// Get the current lat/long of the center of the map.
Location center = new Location(_map.Center);
// Depending on the direction of the pan, move the center to
// be at the current bounds of the map view.
switch (type)
{
case PanType.panRight:
center.Longitude = _map.Bounds.East;
break;
case PanType.panLeft:
center.Longitude = _map.Bounds.West;
break;
case PanType.panUp:
center.Latitude = _map.Bounds.North;
break;
case PanType.panDown:
center.Latitude = _map.Bounds.South;
break;
}
// Now change the view as required, and request no animation of the view change.
_map.SetView(center, TimeSpan.Zero);
}
Once I’d added the above code, I could use the Inspect SDK tool to verify that the UI element did declare itself to support the UIA Scroll Pattern as expected.
Figure 1. Inspect showing that the UI element's IsScrollPatternAvailable property is true, and the Scroll Pattern's properties and methods are all exposed.
Now that Narrator's 2-finger scroll gestures can be used at the app to pan the map, I removed the four pan-related buttons that I'd added to the previous version of the app. And to tidy things up further, I also removed my own Zoom In and Zoom Out buttons too. I replaced the use of my own Zoom In and Zoom Out buttons with use of the buttons provided by the Map control itself. The map control has a ShowNavigationBar property, which when true, means that the map control will present Zoom In and Zoom Out buttons that the Narrator user can interact with.
As it happens, the visibility of the map control's Zoom In and Zoom Out buttons is tied with the visibility of a few other buttons which the user might not be interested in. (These being the Show Traffic and View Mode buttons.) Also, when a single-finger swipe gesture is used to reach the Zoom In and Zoom Out buttons, the user also reaches a nameless text element inside the Zoom In and Zoom Out buttons. So all these additional elements have to be ignored by the user. Using the map control's built-in Zoom In and Zoom Out buttons still beats supporting my own buttons for zooming.
One thing I'm keen on is that the app is useful to people who don't use Narrator and those who do, and the app does not have an explicit mode for Narrator use. While this requirement has been met, the options in the app are rather confusing now. For example, when Narrator's not running, the map control's Zoom In and Zoom out buttons appear when the "Show zoom buttons" is checked, but the buttons can't be interacted with unless the "Enable zoom and pan" option is also set. So at some point I'll probably try to clean up the options a bit to make it clearer as to exactly what to expect when certain options are checked.
Overall I'm really pleased how easy it was to update the app such that the map can be panned via Narrator's 2-finger scroll gesture.
Comments
- Anonymous
June 15, 2014
To see the above code in action, check out the video "Demo using the Narrator screen reader with touch", near the top of herbi.org/.../StateYourNamePlease.htm.