How to have important changes in your Win32 UI announced by Narrator

Artikkeli
04/16/2016

This post describes how you can have the Narrator screen reader announce critically important changes which are happening in your Win32 UI, even when Narrator isn’t looking directly at that UI.

Introduction

As your customer interacts with the UI in your super-helpful feature, the Narrator screen reader will announce information about the UI element which Narrator is interacting with. So as your customer moves keyboard focus through the UI, Narrator will announce details about the element that’s gained keyboard focus. Or if they touch the screen, Narrator will announce details about the element beneath your customer’s finger. And while Narrator is looking at a particular element, changes to such things as the accessible name of the element will be announced.

By default, Narrator will not announce changes that are happening at some other place on the screen where Narrator is not currently looking. This is intentional. If Narrator were to announce all changes that are happening on the screen, this could be a real distraction to your customer. To be frequently interrupted while trying to work is not what your customer wants.

There are some situations however, where you feel it really would be helpful for your customer to learn of a change to some particular UI, even though Narrator might not be looking directly at that UI at the time. For example, some static text element might have appeared somewhere on your page, and is reporting an error message which will probably affect what your customer does next. You may feel your customer must be informed.

For some apps, the UI framework being used can make this relatively straightforward through the use of “LiveRegions”. This post, Let your customers know what’s going on by leveraging the XAML LiveSetting property, details how XAML apps can leverage LiveRegions. By having a UI Automation (UIA) LiveRegionChanged event raised, the UI can request that Narrator makes your customer aware of a change in the UI.

But some UI frameworks were built before the UIA API existed. You may have some Win32 UI, and you feel it would be very helpful for a LiveRegionChanged event to be raised by the UI. So how do you do that? Technically you could add your own implementation of a UIA provider, but you don’t want to have to do all that. All you want is to raise an event. As it happens, some years back when support for LiveRegions was added to UIA, work was done at the same time to help Win32 apps which don’t natively support UIA, also leverage LiveRegions. This is great news, given that I got a question the other day from a developer, asking how important changes in their Win32 UI can be announced by Narrator, even when Narrator isn’t looking at that UI at the time of the change.

Hold on, is this really helpful to your customer?

Before going further, I really must stress that when considering the use of LiveRegions, please do consider how valuable it is to your customer, and whether in fact it could become a distraction. There are people who feel overuse of LiveRegions is causing more harm than good, and use of LiveRegions should be discouraged. For now, let’s assume that you’ve considered this, and you still feel it really is valuable for your customer to be informed of a change through the use of a LiveRegion.

Consider exactly what customer experience you want to ship

And still before going further, it’s worth considering exactly what you want your customer experience to be. In the case of error text appearing, the experience might be fairly straightforward. The text appears, and Narrator announces it.

But what about announcing changes in some progress UI that doesn’t display text visually? If the progress UI is appearing on a page where your customer will be working at other elements, your customer won’t want to be interrupted with progress updates while they’re working. (Instead, they could move Narrator over to the UI periodically to learn of the current progress if they want to.) But say in your case, the page is all about the progress, (perhaps there’s only the progress UI and a few static text labels shown on the page,) and all your customer will be doing at the page is waiting for the progress to complete. So you want them to hear periodic announcements relating to the progress, regardless of what text string Narrator happens to be looking at on the page.

So you’ll set a helpful, concise, localized accessible name on the UI element which contains the current progress value. When Narrator receives the LiveRegionChanged event relating to the element, it will simply announce the accessible name of the element. Now let’s consider what happens if the accessible name is a description of the progress operation followed by the current progress value, for example, “Downloading Multiverse, 4%”. Say the progress updates 1% each second, and you raise a LiveRegionChanged event with every update. Narrator will start announcing the name, and then receive another event, and start reading the updated name. So I expect all your customer will hear will be something like “downloading mu downloading mu downloading mu downloading mu” etc.

Now, it’s important to remember here that programmatic accessibility is not at all equivalent to supporting the Narrator experience. While I consider the Narrator experience to be very high priority, there are many other assistive technology (AT) tools being used today, including some very powerful screen readers other than Narrator. Narrator will not necessarily react to LiveRegionChanged events in the same way other screen readers do.

But for me, I try to build a UI which I feel is behaving in a way that makes sense, such that any AT could reasonably interact with it. If I expect the progress UI to update visually fairly rapidly, maybe I’d raise a LiveRegionChanged event every 10%, but if the progress is really slow, maybe I would raise the event at smaller percentage increments. I’d ship whatever seems reasonable, remembering that I don’t want my customer to be distracted by announcements when they’re trying to get on with some other work.

Raising an event to let Narrator know that important new information is available

Ok, so now we finally get on to the technical stuff.

You have Win32 UI, and you want Narrator, (which is a UIA client app,) to receive UIA LiveRegionChanged events as a result of your UI changing. While it’s not straightforward for you to raise UIA events directly from your code, it is easy for you to raise winevents. When UI raises a winevent, UIA itself can sometimes map the winevent to a related UIA event, so that UIA client apps are made aware of the event. And sure enough, when support for the UIA LiveRegionChanged event was added to UIA, an EVENT_OBJECT_LIVEREGIONCHANGED winevent was added to Windows at the same time, specifically to allow this mapping. So all your Win32 UI needs to do, is call…

NotifyWinEvent(EVENT_OBJECT_LIVEREGIONCHANGED, hwnd, OBJID_CLIENT, CHILDID_SELF);

Where hwnd is the window whose accessible name you want spoken in response to the event.

I tried a quick test of this by creating a new Win32 app from one of the templates in Visual Studio, and I raised the event off one of the static text labels in the About dialog box. I could then use the AccEvent SDK tool to confirm that the UIA LiveRegionChanged event was getting through to the UIA client app. (Both AccEvent and Narrator are UIA client apps.)

Figure 1: The AccEvent SDK tool reporting that the UIA LiveRegionChanged event is being generated in response to a Win32 app raising the EVENT_OBJECT_LIVEREGIONCHANGED winevent.

Declaring the element that raised the LiveRegionChanged event to be a LiveRegion

So with the addition of that line of code above, your UI can trigger UIA LiveRegionChanged events which can be consumed by UIA client apps. But in some cases, (and certainly in the case of Narrator,) you need to take one additional step before your customer is made aware of the change. And that is, you need to set up the source of the event such that it declares itself to be a LiveRegion.

By default, UI elements are not LiveRegions. But for those special cases where you feel it’s appropriate to use LiveRegionChanged events, you need to declare the source of the event to be a LiveRegion. (After all, a LiveRegionChanged event is all about announcing a change to a LiveRegion.) In Narrator’s case, once it receives a LiveRegionChanged event, it checks that that source of the event is really a LiveRegion. If your UI says it is not a LiveRegion, then Narrator will ignore the event.

When you declare your hwnd to be a LiveRegion, you can give it one of two values. It is either “assertive”, meaning that you’d like a screen reader to announce the change immediately, interrupting whatever the screen reader happened to be saying at the time, or “passive”, meaning that you’d like the screen reader to finish what it was saying before announcing the change to the LiveRegion. I tend to see “assertive” being used more often than passive, but you’d pick whatever you feel is most helpful in your situation.

So the question now is, how do you declare the UIA element that represents your hwnd to be a LiveRegion? Well, another handy feature of Windows accessibility, is that you can set specific accessibility properties on hwnds. And this includes the LiveRegion property. The related interface you’d use is detailed at IAccPropServices interface.

I updated my earlier test to set the LiveRegion property when the About dlg appeared, and to clear the property when the dlg is closed. I raised the LiveRegionChanged event every 5 seconds, simply to be able to test Narrator repeatedly announcing the accessible name of the hwnd which I’d now declared to be a LiveRegion.

The modified dlg proc for the SDK sample’s About dlg is shown below.

#include <initguid.h>
#include "objbase.h"
#include "uiautomation.h"

IAccPropServices* _pAccPropServices = NULL;

#define MYTESTTIMERID 1234

// Message handler for about box.
INT_PTR CALLBACK About(HWND hDlg, UINT message, WPARAM wParam, LPARAM lParam)
{
HRESULT hr = S_OK;

UNREFERENCED_PARAMETER(lParam);

    switch (message)
    {
    case WM_INITDIALOG:
    {
        // Get the hwnd of the control that we want to declare to be a LiveRegion.
        // (I added the IDC_LIVEREGIONCONTROL id to uniquely identify one of the
        // static text labels in the dlg.)

HWND hWndControl = GetDlgItem(hDlg, IDC_LIVEREGIONCONTROL);

        hr = CoCreateInstance(
            CLSID_AccPropServices,
            nullptr,
            CLSCTX_INPROC,
            IID_PPV_ARGS(&_pAccPropServices));
        if (SUCCEEDED(hr))
        {
            // Let's make the control an assertive LiveRegion.
            VARIANT var;
            var.vt = VT_I4;
            var.lVal = Assertive; // Picked up by including UIAutomation.h.

            hr = _pAccPropServices->SetHwndProp(
                hWndControl,
                OBJID_CLIENT,
                CHILDID_SELF,
                LiveSetting_Property_GUID, // Picked up by including UIAutomation.h.
                var);
        }

// For this test, have Narrator announce the name of the control every 5 seconds.
SetTimer(hDlg, MYTESTTIMERID, 5000, NULL);

        return (INT_PTR)TRUE;
    }
    case WM_DESTROY:
    {
        if (_pAccPropServices != nullptr)
        {
            // We only added the one property to the hwnd.
            MSAAPROPID props[] = { LiveSetting_Property_GUID };

            HWND hWndControl = GetDlgItem(hDlg, IDC_LIVEREGIONCONTROL);
            hr = _pAccPropServices->ClearHwndProps(
                hWndControl,
                OBJID_CLIENT,
                CHILDID_SELF,
                props,
                ARRAYSIZE(props));

            _pAccPropServices->Release();
            _pAccPropServices = NULL;
        }

KillTimer(hDlg, MYTESTTIMERID);

        break;
    }
    case WM_TIMER:
    {
        HWND hWndControl = GetDlgItem(hDlg, IDC_LIVEREGIONCONTROL);

        NotifyWinEvent(
            EVENT_OBJECT_LIVEREGIONCHANGED,
            hWndControl,
            OBJID_CLIENT,
            CHILDID_SELF);

        break;
    }
    case WM_COMMAND:

        if (LOWORD(wParam) == IDOK || LOWORD(wParam) == IDCANCEL)
        {
            EndDialog(hDlg, LOWORD(wParam));

return (INT_PTR)TRUE;
}

break;
}

return (INT_PTR)FALSE;
}

I can now run the test app again, and use AccEvent to verify that the source of the LiveRegionChanged event is an assertive LiveRegion.

Figure 2: The AccEvent SDK tool reporting that the source of a UIA LiveRegionChanged event is an assertive LiveRegion.

Summary

Periodically you may find that you want to enhance the default accessibility of your Win32 UI. In some cases, you may want to consider how functions such as SetHwndProp() or SetHwndPropStr() might be able to help.

If you feel strongly that it would help your customer to be informed of changes to the accessible name of a UIA element representing an hwnd in your Win32 UI, even when Narrator isn’t looking directly at the element at the time of the change, consider declaring it to be a LiveRegion, using code similar to the following code:

hr = CoCreateInstance(
    CLSID_AccPropServices,
    nullptr,
    CLSCTX_INPROC,
    IID_PPV_ARGS(&_pAccPropServices));
if (SUCCEEDED(hr))
{
    // Let's make the control an assertive LiveRegion.
    VARIANT var;
    var.vt = VT_I4;
    var.lVal = Assertive; // Picked up by including UIAutomation.h.

    hr = _pAccPropServices->SetHwndProp(
        hWndControl,
        OBJID_CLIENT,
        CHILDID_SELF,
        LiveSetting_Property_GUID, // Picked up by including UIAutomation.h.
        var);

// Don’t forget to call ClearHwndProps() later.
}

Then raise a LiveRegionChanged event to have Narrator announce the new accessible name, using the following code:

NotifyWinEvent(
    EVENT_OBJECT_LIVEREGIONCHANGED,
    hWndControl,
    OBJID_CLIENT,
    CHILDID_SELF);

I have to say, I’m really pleased that when LiveRegion support was added to UIA, additional work was done to enable apps built using the Win32 UI framework, (which doesn’t natively support UIA,) to also benefit from LiveRegions. This support has real potential to help your customers when critically important information appears in your Win32 UI.

Guy

Jaa