Let your customers know of important status changes in your WinForms app
This post describes an approach to having the Narrator screen reader announce important status changes occurring in a WinForms app. An introduction to the UI Automation (UIA) API mentioned below, and the related Inspect and AccEvent SDK tools can be found at Introduction to UIA: Microsoft's Accessibility API.
Introduction
It's not uncommon for an app to convey important status information through a static text string shown somewhere in the app. For example, a string might show a download status as complete, or a network status of disconnected. This text string is often shown visually at some distance from where the customer is working in the app at the time. If customers using a screen reader are not made aware of that information, then that can have a severe impact on their progress through a task.
By default, the Narrator screen reader will not announce changes occurring in an app which are happening at some location distant from where the customer's working. This is because to announce all changes occurring in an app could be a severe distraction. For example, as I type this in Word 2016, a word count is constantly being updated in a status bar at the bottom of the window. If a customer using Narrator was creating a document in Word, there's no way they'd want the word count to be announced every time it's updated in the status bar. As such, the app dev decides what status string is important enough for it to be announced by Narrator when the string gets updated, and they take action to make that happen. This is typically done through the use of "LiveRegions".
The concept of LiveRegions originated on the web, (involving the use of aria-live,) but has been adopted for some other types of UI. In the case of UWP XAML apps, it involves the use of the AutomationProperty.LiveSetting, and related code snippets can be found at Let your customers know what's going on by leveraging the XAML LiveSetting property. With a little more work, you can also leverage LiveRegions in Win32 UI, as described at How to have important changes in your Win32 UI announced by Narrator.
In all these cases, Narrator first requires the app to raise a UIA LiveRegionChanged event once the status string has changed in the app. When Narrator receives that event, it will go back to the app and determine whether the UIA element that raised the event has a UIA LiveSetting property of "Assertive" or "Polite". If the element does have a property of one of those two values, Narrator will announce the UIA Name of the element that raised the event.
Important: If Narrator receives another UIA event around the same time, (for example a FocusChanged event,) Narrator might choose to only announce that other event.
A traditional challenge for WinForms app devs has been how they can make their customers aware of important status changes being shown visually in their app. The WinForms framework does not natively support LiveRegions, and so app devs have had to consider other approaches for making their customers aware of the important information being exposed through the status text string.
The WinForms app
Given that LiveRegions aren't natively supported by the WinForms framework, one potential approach that might be considered is whether a MessageBox might be a reasonable way to make the customer aware of the status change. If a MessageBox popped up to let the customer know that a download was complete, Narrator would certainly announce something related to the MessageBox. So that might in some cases be a reasonable approach, but I do recognize that in many other cases, it will not be.
Technically, another approach might be to set a UIA string property on the UI element where Narrator is likely to be examining at the time of the status change, such that Narrator would announce that string. This is similar to how Narrator announces the new UIA Name property on a button, if it's interacting with the button at the time of the Name change. So say your customer invokes a button, and keyboard focus stays on that button while a status string appears visually elsewhere in the app. If a UIA string property were set on that button, it could have the same value as the status string shown visually, and Narrator might announce the property. As it happens, while this might technically be possible, I don't like the idea. I think it might take a few dev steps to get this working, and often there's no guarantee Narrator would actually be working with the affected button at the time of the property change, and care would need to be taken to update or remove the related property later. So while it's useful to be aware of what's technically possible, I'd not recommend this approach.
But as it happens, I have found a way to leverage LiveRegions in a WinForms app. The work involved is not trivial, but it's straightforward enough that I'd say it should be considered in some cases. The difference it could make to your customer experience is very significant.
The LiveLabel
For this discussion, I'm only going to consider how to add support for LiveRegions to the WinForms Label control. I expect the work to turn other types of WinForms controls into LiveRegions will be more significant, and it's often Labels that are of most interest in scenarios relating to making the customer aware of status changes.
I hinted earlier that the process of leveraging LiveRegions involves two steps. The first step is to have a UIA LiveRegionChanged event raised by the app. This step is pretty easy to do through interop in a WinForms app. It's only a case of calling the UIA's UiaRaiseAutomationEvent(), passing in the value of the UIA_LiveRegionChangedEventId. The code snippet below shows how to do this. In that snippet, the code explicitly raises the event after setting new text on the Label-related control, rather than adding a TextChanged event handler, and raising the LiveRegionChanged event inside that handler. I've not tried that latter approach, but I expect it would work.
The more interesting step in leveraging LiveRegions in a WinForms app, is how a control can declare itself to be an Assertive or Polite LiveRegion. If the control doesn't do that, then Narrator will ignore the LiveRegionChanged event that gets raised by the control. In order to achieve this, I'm going to create a new control which derives from Label, but which also supports UIA's IRawElementProviderSimple interface. Implementing that interface can sometimes be a lot of work, and usually this isn't the sort of thing that's done unless a significant custom control is being developed. But it turns out in this case, the work isn't as significant as I'd expected.
It is straightforward to declare the control as being an Assertive or Polite LiveRegion through IRawElementProviderSimple.GetPropertyValue(). But my concern was that by turning the control into a native UIA provider, (by implementing IRawElementProviderSimple,) I'd be losing some accessibility that was previously provided by the control by default. So in this situation, it's very important that I consider the UIA representation of a standard Label control, and compare it to the UIA representation of my new control. If the two representations were very different, I'd give up on this approach. But as it happens, while I think my concerns would be justified with many other types of control, in the case of the Label control, even with me adding my own IRawElementProviderSimple implementation, the two UIA representations were very similar. This was true despite my implementation really not doing very much at all.
There were some small differences between the two UIA representations, so I decided to update my IRawElementProviderSimple.GetPropertyValue() such that it returned properties that matched the default properties associated with a standard WinForms Label control. I made some assumptions in my implementation, but I expect they're fine in practice for the majority of cases where you'd want to turn a Label into a LiveRegion.
Regarding the IRawElementProviderSimple.GetPatternProvider() implementation, it turned out I didn't need to do anything there. A WinForms Label control only supports the UIA LegacyIAccessible pattern, and my control got that for free.
The only other work I need to do it make sure that when UIA calls the control, (through a window message,) to learn if there is an object available that supports IRawElementProviderSimple, I return the control's custom implementation.
The code
// STEP 1: Add these. Be sure to include .NET's UIAutomationProvider in your References list.
using System.Runtime.InteropServices;
using System.Windows.Automation.Provider;
// STEP 2: Add support for the UIA IRawElementProviderSimple interface to a standard WinForms Label control.
public class LiveLabel : Label, IRawElementProviderSimple
{
// Override WndProc to provide our own IRawElementProviderSimple provider when queried by UIA.
protected override void WndProc(ref Message m)
{
// Is UIA asking for a IRawElementProviderSimple provider?
if ((m.Msg == NativeMethods.WM_GETOBJECT) && (m.LParam == (IntPtr)NativeMethods.UiaRootObjectId))
{
// Return our custom implementation of IRawElementProviderSimple.
m.Result = AutomationInteropProvider.ReturnRawElementProvider(
this.Handle,
m.WParam,
m.LParam,
(IRawElementProviderSimple)this);
return;
}
base.WndProc(ref m);
}
// IRawElementProviderSimple implementation.
ProviderOptions IRawElementProviderSimple.ProviderOptions
{
get
{
// Assume the UIA provider is always running in the server process.
return ProviderOptions.ServerSideProvider | ProviderOptions.UseComThreading;
}
}
IRawElementProviderSimple IRawElementProviderSimple.HostRawElementProvider
{
get
{
return AutomationInteropProvider.HostProviderFromHandle(this.Handle);
}
}
public object GetPatternProvider(int patternId)
{
// The WinForms Label control only supports the LegacyIAccessible pattern,
// and this custom control gets that for free.
return null;
}
public object GetPropertyValue(int propertyId)
{
// With the exception of the UIA_LiveSettingPropertyId handling below,
// all properties returned here are done so in order to replicate the
// UIA representation of the standard WinForms Label control.
// Note that (with the exception of the LiveSetting property,) the only difference between
// the UIA properties of the LiveLabel and the standard Label is the ProviderDescription.
// The standard Label's property will include:
// "Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)",
// whereas the LiveLabel's will include (something like):
// "Unidentified Provider (managed:WinForms_LiveRegion.LiveLabel, WinForms_LiveRegion"
switch (propertyId)
{
case NativeMethods.UIA_ControlTypePropertyId:
{
return NativeMethods.UIA_TextControlTypeId;
}
case NativeMethods.UIA_AccessKeyPropertyId:
{
// This assumes the control has no access key. If it does have an access key,
// look for an '&' in the control's text, and return a string of the form
// "Alt+<the access key character>". It's pretty unlikely that the control
// would have an access key, as if it did, the LiveRegion-related announcement
// might unexpectedly include "and" in it.
return "";
}
case NativeMethods.UIA_IsKeyboardFocusablePropertyId:
{
return false;
}
case NativeMethods.UIA_IsPasswordPropertyId:
{
return false;
}
case NativeMethods.UIA_IsOffscreenPropertyId:
{
// Assume the control is always visible on the screen while it exists.
return false;
}
case NativeMethods.UIA_LiveSettingPropertyId:
{
// Return whichever of Polite or Assertive is most appropriate for your scenario.
// Note that a value of zero, (for "Off"), could also be returned, but typically if
// an element is ever a LiveRegion, it's always either Assertive or Polite.
return NativeMethods.Assertive;
}
default:
{
return null;
}
}
}
}
// STEP 3: Add whatever's required for interop.
public class NativeMethods
{
public const int WM_GETOBJECT = 0x003D;
public const int UiaRootObjectId = -25;
public const int UIA_LiveRegionChangedEventId = 20024;
public const int UIA_ControlTypePropertyId = 30003;
public const int UIA_AccessKeyPropertyId = 30007;
public const int UIA_IsKeyboardFocusablePropertyId = 30009;
public const int UIA_IsPasswordPropertyId = 30019;
public const int UIA_IsOffscreenPropertyId = 30022;
public const int UIA_LiveSettingPropertyId = 30135;
public const int UIA_TextControlTypeId = 50020;
public const int Polite = 1;
public const int Assertive = 2;
[DllImport("UIAutomationCore.dll")]
public static extern int UiaRaiseAutomationEvent(
IRawElementProviderSimple provider,
int id);
[DllImport("UIAutomationCore.dll")]
public static extern bool UiaClientsAreListening();
}
// STEP 4: Create an instance of the LiveLabel, rather than standard WinForms Label.
this.labelLiveStatus = new LiveLabel();
…
private LiveLabel labelLiveStatus;
// STEP 5: Update the status text shown on the LiveLabel control. The
// labelLiveStatus control here is an instance of the LiveLabel class.
labelLiveStatus.Text = "Download complete"; // <- Replace this placeholder text with your own localized status.
// If any UIA client app is running, (such as Narrator,) raise
// an event to make the app aware of the change in status.
if (NativeMethods.UiaClientsAreListening())
{
// Raise a LiveRegionChanged event from the UIA provider associated with the LiveLabel control.
NativeMethods.UiaRaiseAutomationEvent(labelLiveStatus, NativeMethods.UIA_LiveRegionChangedEventId);
}
So did it work?
After making a change like that shown above, there's no way I'd simply point Narrator at the UI and listen to its announcements. Instead I'd point the Inspect and AccEvent SDK tools to the UI, and verify that things seem to be working as expected from a UIA perspective. Only when I feel that things are working as expected using those tools, would I run Narrator against the UI. After all, if I pointed Narrator to the UI first, and Narrator said nothing when the status string changed, does that mean the LiveRegionChanged event's being raised and Narrator's ignoring it, or the event isn't being raised at all?
The screenshot below shows Inspect reporting the UIA properties of a LiveLabel control in a test app. It reports that the UIA LiveSettings property of the control is Assertive, and that's what I expected using the above code snippet.
Figure 1: The Inspect SDK tool reporting that the UIA LiveSetting property on the LiveLabel control has a value of Assertive.
Having done that, I could then point the AccEvent SDK tool at the test app, to verify that the LiveRegionChanged event is raised as expected after I change the text on the LiveLabel. And sure enough, the event is raised, and AccEvent reports that the UIA Name property of the element that raised the event is "Download complete", and its LiveSetting property is Assertive.
Figure 2: The AccEvent SDK tool reporting that a UIA LiveRegionChanged event is raised by the test app after the status string has changed.
So having done all the verification of the UIA representation of the app, I can try using it with Narrator. The text below is the announcement made by Narrator as I tab to the Close button, then to the Update status button, and then as I invoke the button which triggers the app to change the displayed status string from "Download in progress" to "Download complete".
LiveLabelTest window, Close button, Alt+ c,
Tab
Update status button, Alt+ u,
Space
Update status button, Alt+ u,
Download complete,
Summary
Sometimes critically important information is shown visually in the form of status strings in an app. If the customer is not made aware of that status information at the time it becomes relevant, that could severely impact whether the customer can complete their task. For example, they may be waiting for a transaction to complete, and the network connection is lost during the transaction. If the change in network status is not announced immediately, then the customer may continue to wait for the transaction to complete, and it never will complete. In some cases, a delay in responding to a transaction failing could have important consequences.
So if at all practical, you'll want to make sure your customer is aware of changes made to important status strings shown in your app, at the time the changes occur. In the case of a WinForms app, please consider whether the approach described above could in some cases help you deliver the experience that your customers need and deserve.
Guy