Jaa

Test Automation: Simple Automated Test Case Templates

  • Artikkeli

Templates can be useful tools to help increase individual efficiency and team consistency. Yet, I am sometimes quite surprised that some organizations lack standard templates for such daily routines such as bug reporting, status reporting, and even test cases. There are some in the industry who shun templates because they claim they are too restrictive and the person using the template is 'forced' to do things one way and not encouraged to think beyond the scope of the template. Of course, any person who has matured to the point of being able to think rationally for themselves and has the cognitive ability to for logical reasoning (as opposed to those who base decisions on pure emotion and volitional impulse) realize this narrow-minded viewpoint is pure nonsense!

A template is simply a model or reference that provides a starting point. Frankly, if a person does not understand that a template simply provides a starting framework from which he must rely on his knowledge, skill, and experience to customize to fit the specific context, and how to use a template to help him accomplish a task more efficiently then I would recommend that person not be allowed beyond the reach of their mother's apron strings because they are probably a danger to themselves and people around them and they are incapable of rational constructive thinking.

An example of where templates have added value for me is in the test development process. When developing test cases using Visual Studio I frequently find myself adding the references and using directives repeatedly to each new test case. For example, when developing a UI automated test case I need to include references for Windows.Forms, UIAutomationClient, UIAutomationType, UIAutomationClientsideProviders, and UIAutomationProvider. That's a lot of clicking for each new test case.

Creating a template in Visual Studio is easy and straight forward. In this case, I started with a standard console application project. Next, I added the following references to the project:

  • System.Windows.Forms
  • UIAutomationClient
  • UIAutomationClientsideProviders
  • UIAutomationProvider
  • UIAutomationTypes

Then I customize the template as illustrated in the code below by adding common namespace directives, including XML comments, and commonly used code snippets.

 using System;
using System.IO;                       // For file I/O operations
using System.Text;                     // For StringBuilder and encoder/decoder
using System.Diagnostics;              // Process, stopwatch, trace/debug 
using System.Globalization;            // For Unicode support
using System.Windows.Forms;            // For sendkeys and clipboard
using System.Windows.Automation;       // For UIAutomation

namespace CompanyName.ProjectName.FeatureArea
{
    class TestCase
    {
        /// <summary>TEST PURPOSE: (specific functional capability or end-2-end user scenario test is verifying)</summary>
        /// <status>TEST STATUS: (Active, NotInMilestone, Obsolete)</status>
        /// <category>TEST TYPE: (category or test suite bucket such as BVT/BAT, Regression, etc.)</category>
        /// <priority>TEST PRIORITY: (importance of test in relation to critical functionality or requirements)</priority>
        /// <time>TEST TIME: (approximate duration of test)</time>
        /// <steps>TEST STEPS: (outline high level steps or procedures)</steps>
        /// <expected>EXPECTED RESULTS: (specific verification criteria for oracle to determine pass, fail, or indeterminate)</expected>
        /// <dependencies>CONFIGURATION: (required expected settings or dependencies)</dependencies>
        /// <remarks>OTHER INFO:</remarks>

        [STAThread]
        static void Main(string[] args)
        {
            // Time the duration of test in order to better predict time to
            // run each test suite
            Stopwatch TestTimer = new Stopwatch();
            TestTimer.Start();

            // TODO: Design and develop the code to achieve the specific purpose
            //       of the test. 

            // Don't forget to add comments to the code!

            // Don't forget to clean up or restore the test bed to pre-test
            // conditions in cases other than end-to-end user scenarios.

            TestTimer.Stop();
            // Don't forget to log or record test time
        }
    }
}

I also like to include valuable information about the test case using XML comments to jog my memory or provide some background for anyone who views the code. For example it is a best practice to clearly define the specific purpose of each test so people don't have to guess what the test is proving/disproving (or at least what it is supposed to be doing). (If someone doesn't have a clearly defined purpose and goal of what the automated test is going to prove or disprove and they are simply writing automated test just to automate tests than that person is probably a mindless dolt!) The information that adds value to the organization may be subjective, so determine what information is important for the consumers of the completed template. Essentially, in this case, the comments captured in each test should provide value to the management and maintainability of the test and the testing process. If a template tries to collect too much information that is not being consumed or is not important to reduce long term costs of maintenance then that information should be suspect and is probably not adding value to the overall team. A big advantage of Visual Studio and C# is that it allows comments and important information to be easily captured via XML tags documented in a XML file at build time so I don't have to worry about performing a separate task to document each automated test.

The STAThread attribute is included above the Main method to support the Clipboard class members. Without this attribute set the test will throw an un-handled exception. Finally, I put in any code that I commonly reuse. That doesn't mean that I have to use it...I can easily remove it if I don't need it. (That's the beauty of being able to think logically for one's self!) But, if it is code that is commonly used then I don't have to retype it (meaning...it increases my efficiency even by a little bit).

To complete the process I simply select the File --> Export Template... menu item and complete the Export Template Wizards.

image

I can create several custom templates depending on the context of each type of test. For example, if the automated test case is not going to drive the UI then I don't need all those additional references in my template. Rather than remove them each time, I can simply create another template for non-UI automated test cases. Now, each time I start a new project I can simply select the appropriate template. The templates are saved in a ZIP file in the Visual Studio\My Exported Templates directory, and those ZIP files can be sent around to other members of the team.

Also, here are some important key concepts for creating standard templates (in general):

  • Don't screw the majority just to make the minority happy! Only include key elements that provide value to the majority of consumers of the template the majority of the time. A common problem with many templates is that they try to include everything for everybody. When templates are over-laden with things that only a handful of people use, or if people have to spend too much time trimming out things the majority of people don't use then the templates are a bottleneck. Remember, templates can be customized by those few who might need a few additional things occasionally; majority rules!
  • Don't over-document useless tidbits. Notes and additional comments must be useful, but brief. If someone has to spend more time learning how to fill out a template than it takes to fill out the template then that template provides no value in terms of making the process or the team more efficient. If the template is so complicated that it requires a user manual...then it is not a useful template.
  • Intuition is important. If the template is not easy to use, if the consumer of the template has to search around for items, or even worse, forces the user to use parts of the template because they are on the template, then the template is not intuitive and using the template becomes a chore rather than a tool to improve efficiency.
  • Get buy in! Don't develop templates in isolation. Talk to various consumers and users of the templates to better define the template and what is captured in the template. For example, if you are developing a template for reporting anomalies in software then developers should be included in that conversation as should program managers and any other consumers of the completed templates.
  • Templates are not commandments! Templates provide a starting point; they are not engraved in stone. Templates should be designed to include things that are commonly reused by various users and consumers of the completed template. But, sometimes they can include things that are unnecessary and do not add value in a specific context and in those cases the user of the template should be able to remove those parts of the template or at least ignore them. Conversely, other times additional things may be necessary in a template and the user of the template should be able to easily add those things that provide important value to the purpose of the completed template.

Bottom line, templates are tools to provide a common reference, or central starting point...they are not hand-cuffs designed to constrain rational and logical thought or constrict a person's creativity. In fact, good templates are tools that enable and support more efficient creativity!

Comments

  • Anonymous
    May 23, 2009
    Test Automation: Simple Automated Test Case Templates

  • Anonymous
    December 06, 2010
    I want to find [how to write good automation test case ?]