GUI test automation is not child's play

There are many approaches to test automation from unit testing to system level testing through the GUI. Of course, the most often discussed approach is the automation approach that drives the GUI to perform some action; or GUI automation. This also happens to be the most controversial approach to test automation, and is perhaps the hardest type of automation to design and develop. One reason why an automated GUI test fails or doesn't achieve its potential is due to a lack of understanding of the "system" by the tester, which in turn leads to a poorly designed test from the outset.

This problem is especially obvious when people who may have specialized business knowledge but lack a in-depth understanding of the systems they are working on (non-technical testers) are asked to 'automate' something. The automation in this case is usually in the form of using record/playback tools or perhaps creating a rote script to drive a keyword-driven framework, and the 'test' is usually consists of nothing more than merely mimicking some contrived behavior by the 'tester.' In fact, this over-simplified view of test automation is sometimes perpetuated by tool vendors. A manager at one tool vendor said, "By automatically capturing the tester's process and documenting their keystrokes and mouse clicks, the tester will have more time for the interesting, value-add activities, such as testing complex scenarios that would be difficult or impossible to justify automating."

There are 2 fundamental problems with the above quote, and this approach to 'automation' (and I use that term loosely in this context). First, while I don't totally discount the value of record/playback, and in the right context it could very well be the best approach in a specific situation, the general consensus in the industry is that due to the limited capabilities of record/playback type tools this type of automation is simply one level above using a hoard of monkeys trained to repeat a set of sequences using the keyboard and mouse. In fact, some might suggest trained monkeys may be a better alternative because bananas are much cheaper than the costs of licensing a tool and then realizing that you have to hire someone to try to patch together some proprietary script in an often vain attempt to build an automation test suite that is beyond the reasonable capabilities of a record/playback automation approach. Unfortunately, in either case the organization is usually left with a horrible mess that nobody wants to clean up. Secondly, it makes a ridiculous assumption that testers are too stupid to automate complex scenarios or test automation is a brain-dead, non-interesting, zero-value-add activity.

If simply recording or 'documenting' rudimentary scripts that essentially repeat a sequence of 'hard-coded' steps over and over again is someone's idea of well-designed test automation then I would agree that automation is a brain-dead activity. When you automate poorly designed tests, you simply get poorly designed automated tests! And, since I don't mind calling the kettle black, I will say it…recording a set of actions, or documenting a sequence of steps with hard-coded values to feed into a keyword driven architecture is not test automation! Surely it automates tasks, but well designed automated tests are much more powerful than the production of some crude script that automates the actions performed by someone sitting in front of a computer.

Of course, well designed automated tests require highly skilled professional testers with in-depth knowledge of the systems they are working on, as well as proficiency in programming concepts and languages. Similar to how doctors study anatomy, physiology, pharmacology, immunology, biochemistry, etc., professional testers need to constantly study the various systems they are tasked to test. Developing a well designed automated test is very different than simply using a tool to automatically repeat a sequence of actions. Designing robust tests (automated or not) requires not only incredible creativity and problem-solving skills, but an in-depth knowledge of the system and an understanding of how to manipulate the system programmatically.

But, if you buy into the idea that automation is simple and merely rote recording or documenting some sequence of steps performed by some person then you get exactly that; simplistic repetitive automated actions. Simple automation is simply automated simplicity.

Comments

• Anonymous
  March 12, 2009
  PingBack from http://www.anith.com/?p=18288

• Anonymous
  March 16, 2009
  The comment has been removed

• Anonymous
  March 16, 2009
  The comment has been removed

• Anonymous
  March 17, 2009
  The comment has been removed

• Anonymous
  April 21, 2009
  The comment has been removed

• Anonymous
  April 22, 2009
  The comment has been removed

• Anonymous
  December 09, 2009
  Very interesting article JIT when I was looking for ways to get into automated testing. Could someone kindly tell me WHAT THE DIFFERENCE IS BETWEEN QTP AND UI TESTING E.G C# UI TESTING. I have looked a bit into each and depending on the blog I'm reading, opinions are very subjective. I hold a Bsc. in Computer Science and have experience in C# development therefore cost of learning programming skills is eliminated. I could write VBScript, Java and C# etc on an intermediate level therefore the programming language is not an issue. Especially, WHAT CAN BE DONE IN ONE THAT CANNOT BE DONE IN THE OTHER? Thanks in advance.