Too much fun at CSUN! Part 6: And not forgetting... Herbi Clicker - Efficiently invoking anything on the screen with the keyboard (Windows Desktop app, WinForms/C#)
After I’d demo’d four of my apps at CSUN, I was asked about how an app might allow someone who only uses the keyboard to click anything on the screen. Any high quality app will make it easy for the user to quickly access commonly used features through the keyboard, but not all apps are high quality. Instead sometimes apps force the user to press the tab key many times before they can click a button, or even worse, sometimes the user can’t reach a button through the keyboard at all.
This question made me think back to an app I’d build a few years ago, which explored how a switch device user could efficiently click anything on the screen using the Windows 7 On-Screen Keyboard, (OSK). My app used UI Automation (UIA) to find anything on the screen that said it can be invoked, and I then showed numbers by the elements. The user could then control the OSK through the switch device, and enter the number for the element of interest and click the element.
But a user of a physical keyboard could do the same thing. So I downloaded the Herbi Clicker app from my Herbi.org site, and tried running it for the first time in years. To my delight, it seemed to work fine. I loaded a news web page into IE, pressed the ` key to have numbers shown on all the invokable elements, typed a number, and pressed Escape to click a link of interest. I chose to use the ` and Escape key when building the app because they’re quick for a switch device user to reach in the Windows 7 OSK.
Figure 1: The Herbi Clicker app showing numbers for efficient clicking of links via the keyboard. The numbers are shown using colors from the active high contrast theme.
So this question coming from the CSUN attendee was a great reminder of how useful UIA can be as part of building assistive technology solutions. After eleven years of building simple tools, I’m more convinced than ever that Windows technologies have so much to offer in this space. Windows technologies are ready and waiting to be used in new and creative ways by developers who recognize how they could help someone near them.
I can’t wait to see what I get to build over the next eleven years! :)
Further posts in this series:
Too much fun at CSUN! Part 1: Demo’ing four assistive technology apps for Windows at CSUN 2015
Too much fun at CSUN! Part 2: Herbi WriteAbout - Handwriting development
Too much fun at CSUN! Part 4: My Way Speaker - Using a switch device to efficiently browse the web
Too much fun at CSUN! Part 5: Herbi Speaks - Creating boards for education or communication