Share via

UI Automation in Silverlight - Part II (The Easy Way)

  • Article

So, I wrote a blog post a bit earlier on exposing custom controls for accessibility and automation in Silverlight. If you're only interested in the automation perspective and the Silverlight unit testing framework doesn't quite suit your needs, then there's an easier way than going out and implementing AutomationPeer types.

 

As of Silverlight Beta 2, you can utilize the AutomationProperties.AutomationId attribute in your XAML to make types you use unique and identifiable through automation:

 

   <TextBox AutomationProperties.AutomationId="SearchTextBox" x:Name="SearchText" KeyDown="CheckKey" />

 

AutomationProperties.AutomationId doesn't come up in Intellisense in Beta 2, and you'll get a false positive on the "The attachable property 'AutomationId' was not found in type 'AutomationProperties'." error message. Ignore it for now and spin up your Silverlight app :) Once this has been done, your element should look something like this when identified in UISpy (more):

 

   Identification

    ClassName: ""

    ControlType: "ControlType.Custom"

    Culture: "(null)"

    AutomationId: "SearchTextBox"

    LocalizedControlType: "custom"

    Name: ""

    ProcessId: "2808 (iexplore)"

    RuntimeId: "42 197822 3"

    IsPassword: "False"

    IsControlElement: "True"

    IsContentElement: "True"

 

  Visibility

    BoundingRectangle: "(240, 327, 300, 23)"

    ClickablePoint: "390,338"

    IsOffscreen: "False"

 

So, if we want to simulate a user at this point, we have three objectives:

 

   1. Move the mouse cursor over to the ClickablePoint.

   2. Click the mouse button.

   3. Start typing.

 

To move the cursor to the clickable point, we need to set the Cursor.Position property. To do this, we must do a conversion between System.Windows.Point and System.Drawing.Point. I chose to handle this conversion through an extension method that looks something like:

 

    public static class ExtensionMethods

    {

        public static System.Drawing.Point ToDrawingPoint(this System.Windows.Point windowsPoint)

        {

            return new System.Drawing.Point

                    {

                        X = Convert.ToInt32(windowsPoint.X),

                        Y = Convert.ToInt32(windowsPoint.Y)

                    };

        }

    }

 

Once we've moved the mouse, we also need to simulate a click. We're a DllImport away from making that happen. We can create a nice wrapper class to simplify our calling code:

 

    public static class Mouse

    {

        private const UInt32 MouseEventLeftDown = 0x0002;

        private const UInt32 MouseEventLeftUp = 0x0004;

        [DllImport("user32.dll")]

        private static extern void mouse_event(UInt32 dwFlags, UInt32 dx, UInt32 dy, UInt32 dwData, IntPtr dwExtraInfo);

 

        public static void Click()

        {

            mouse_event(MouseEventLeftDown, 0, 0, 0, IntPtr.Zero);

            mouse_event(MouseEventLeftUp, 0, 0, 0, IntPtr.Zero);

        }

    }

 

So, we set automation IDs on our elements in XAML, we can identify elements with unique automation IDs, we can move the mouse with an element's ClickablePoint property, and we can simulate a mouse click. How does this all come together?

 

        [TestMethod]

        public void TestMethod1()

        {

              // Assumes an existing Internet Explorer process is running and pointed at your Silverlight app

            Process process = System.Diagnostics.Process.GetProcessesByName("iexplore").First();

            AutomationElement browserInstance = System.Windows.Automation.AutomationElement.FromHandle(process.MainWindowHandle);

            Thread.Sleep(1000);

 

            TreeWalker tw = new TreeWalker(new PropertyCondition(AutomationElement.AutomationIdProperty, "SearchTextBox"));

            AutomationElement searchBox = tw.GetFirstChild(browserInstance);

            Thread.Sleep(1000);

 

            System.Windows.Point uiaPoint;

 

            if (searchBox.TryGetClickablePoint(out uiaPoint))

            {

                Cursor.Position = uiaPoint.ToDrawingPoint();

                Mouse.Click();

                SendKeys.SendWait("Hello, world!");

            }

            else

            {

                Assert.Fail();

            }

        }

But what if you're dynamically building out your controls in code? You can still set the AutomationId property of generated controls using the static methods in AutomationProperties, which would look something like this:

    ListBoxItem myDynamicListBoxItem = new ListBoxItem { Content = "Hello, world!" };

    AutomationProperties.SetAutomationId(myDynamicListBoxItem, "myDynamicListBoxAutomationId");

 

SendKeys has some new behaviors that were introduced in the .NET Framework 3.0. If that sort of thing interests you, you can read more about it here. If you find that those results give you some inconsistent behaviors, you can make the following addition to your app.config in your Test project:

 

    <appSettings>

        <add key="SendKeys" value="SendInput"/>

    </appSettings>

 

Hope this helps :)

Comments