using UIAutomation events

In my previous post I had written about how UIAutomation (UIA) can be used to drive user interfaces. UIA was designed to serve both assistive and test-automation requirements. Both of these require that user interfaces fire events to notify clients that some thing happened on them. Assistive tools like screen readers can notify the user (e.g. new dialog has come up) or automation tools can use this to wait on completion of some action. UIA supports these scenarios by the use of UIA events. User-interfaces fire these events and client applications can subscribe to them to "listen" to changes in the user interface.

UIA works across UI technologies and provides bridges (providers) for existing accessibility technologies like Microsoft Active Accessibility (MSAA) so that they work seamlessly through UIA. MSAA uses WinEvents to notify other applications of such change. UIA wraps these up and also exposes them as UIA Events.

Its easy to listen to these events. The following code subscribes to events so that we are able to listen to all invokes (as in button clicks) and all menu open/close events. It adds OnUIAEvent as the handler for the event which displays it

 UIAEventHandler = new AutomationEventHandler(OnUIAEvent);
Automation.AddAutomationEventHandler(InvokePattern.InvokedEvent,
                        AutomationElement.RootElement,
                        TreeScope.Descendants, UIAEventHandler);
Automation.AddAutomationEventHandler(AutomationElement.MenuOpenedEvent,
                        AutomationElement.RootElement,
                        TreeScope.Descendants, UIAEventHandler);
Automation.AddAutomationEventHandler(AutomationElement.MenuClosedEvent,
                        AutomationElement.RootElement,
                        TreeScope.Descendants,
                        UIAEventHandler);
public void OnUIAEvent(object src, AutomationEventArgs args)
{
    AutomationElement element;
    try
    {
        element = src as AutomationElement;
    }
    catch 
    {
        return;
    }
    string name = "";
    if (element == null)
        name = "null";
    else
    {
        name = element.GetCurrentPropertyValue(
                AutomationElement.NameProperty) as string;
    }
    if (name.Length == 0) name = "<NoName>";
    string str = string.Format("{0} : {1}", name, 
                                args.EventId.ProgrammaticName);
    Console.WriteLine(str);
}

Comments

• Anonymous
  November 06, 2006
  In on eof my projects, i used the event handler to handle structure changed event. I saw that many events were lost or didn't make it to the event handler. Is this a known problem in UIAutomation? Is there a work around?/Chet

• Anonymous
  July 31, 2007
  Hi, When I press a key of the keyboard or a click of mouse, how can I know the pressed key or the button pressed of the mouse? Thanks.

• Anonymous
  August 01, 2007
  luukas you have 3 options.

1. Listen to the various accessibility events fired by the control if that action had any effect on them. Search for WinEvents
2. spy on the window messages on the target app
3. install and use Low level os wide hooks.

• Anonymous
  June 15, 2008
  Hi, Please help me how to automate an application using c#, i am trying to automate the application using c# UIAutomation element, but in my application most of the object does not have id please reply me how to get the all child elements in the automation element? Thanks, Surya.

• Anonymous
  January 10, 2010
  Hi, I am Jignesh Patel. I am looking to hire someone to work on my C#.Net project for 2 to 3 hours a day. I am willing to pay 15 to 20 dollars an hour for the work. Please email me at jignesh4325@gmail.com if you are interested? Regards, Jignesh

• Anonymous
  October 15, 2015
  hi, iam to try listen for the Outlook send button event but not able to get. here is my code do i need to make any changes plz let me know. private void SendButtonInvoke()        {            Process[] processes = Process.GetProcessesByName("WINWORD");            Process[] pro = Process.GetProcesses();            AutomationElement aeOutLook = null;            Process proc1 = new Process();            foreach (var item in processes)            {                aeOutLook = AutomationElement.FromHandle(item.MainWindowHandle);            }            AutomationElement buttonAddInstance = aeOutLook.FindFirst(TreeScope.Descendants,                   new PropertyCondition(AutomationElement.NameProperty, "Send"));            if (buttonAddInstance == null)            {                MessageBox.Show("Add button instance not found");            }            else            {                AutomationEventHandler UIAEventHandler = new AutomationEventHandler(ButtonChecked_EventHandler);                Automation.AddAutomationEventHandler(InvokePattern.InvokedEvent, aeOutLook, TreeScope.Descendants, UIAEventHandler);            }        }        private void ButtonChecked_EventHandler(object sender, AutomationEventArgs e)        {            AutomationElement ar = sender as AutomationElement;            MessageBox.Show("Button Clicked Sucessfully.");        }