Share via

How to Schedule Automated Tests Without Using TFS

  • Article

Introduction

A programmed test can be executed through command line program MSTest.exe. Windows Task scheduler can be used to schedule the MSTest.exe to run at particular time or in particular period to allow scheduling the automated tests. This piece of writing provides overview of MSTest.exe which is offered as part of Visual Studio or Test Agent installations and Windows Task Scheduler which is component of various Windows operating systems.

MSTest.exe

MSTest.exe is the command-line command that is used to run tests. This command has a number of options which can be used to tailor the test run. MSTest.exe does neither interpret the options nor the values specified for them in a case-sensitive way.
MSTest.exe program can be used to run automated tests in a test assembly from a command line. It is also feasible to view the test results from these test runs, save the outcome to disk, and save results to Team Foundation Server. See How to: Run Automated Tests from the Command Line Using MSTest for additional information on the same.
Below is a snapshot displaying a small amount of of the command-line options that can be used to modify the run. Refer MSTest.exe Command-Line Options for entire list.

 

Windows Task Scheduler

Task scheduler is a Windows component that provides the facility to schedule the initiation of programs or scripts at pre-defined times or after specified time intervals. The Windows Event Log service must be running before the Task Scheduler starts up.
The Task Scheduler service works by managing tasks; Task refers to an action taken in response to trigger. A task is defined by associating a set of events, which can include launching an application, executing a command or taking some custom-defined action, to a set of triggers, which can either, be time-based or event-based. See How To Schedule Tasks in Windows XP for more information on the same. Similar information can be found for diverse operating systems.
The picture below gives summary of screens and available options to schedule a task using Windows Task Scheduler in Windows XP operating system.

Scheduling an Automated Test

From the information available in above sections, it is possible to schedule any automated test by creating a parameterized command of MSTest.exe to execute that test with required customization and scheduling that command with the help of Windows Task Scheduler. For the ease of managing the complex commands that have multiple lengthy parameters, it is good to use a batch file to execute that command. Batch file is type of a script, a text file that contains one or more commands to be executed. Thus we can manage the MSTest.exe command and its parameters separately from the task that is created in Windows Task Scheduler.

A batch file with MSTest.exe command

This batch file should have command that shall be executed to run an automated test. The command will be somewhat similar to shown below. It has placeholders for required and commonly used parameters to help user create the command. The command doesn’t contain actual parameters and hence they must be replaced with actual values. The parameters which are not used should be removed or MSTEST.EXE may throw invalid argument error! The output of this command will be saved in a .log file created with date time to see in case of issues.

 

"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe" /testcontainer:[.dll/.loadtest/.webtest] /testsettings:[.testsettings] [/test:[TestMethod names] multiple times] [/testmetadata:.vsmdi file] [/testlist:[testlists] multipletimes]>> %date:~-4,4%-%date:~-7,2%-%date:~-10,2%_%time:~0,2%.%time:~3,2%.%time:~6,2%.log

Below is short description of the parameters provided and the usage:

 

Parameter Name

Used to specify

Comments

Testsettings

To use the specified test settings file for test run. (provide .testsettings file)

 

Testcontainer

Load a file that contains tests. Provide .dll (for Coded UI), .webtest or .loadtest file.

Multiple files can be provided.

 

Test

The name of a test to be run.

Multiple tests can be provided.

Use with TestContainer to run only selected tests in case .dll is provided that contains multiple tests.

Testmetadata

Load a file that contains test metadata (for .vsmdi file)

Do not use this option along with TestContainer.

testlist

The name of the test list to be run (as specified in metadata file)

Multiple lists can be provided.

Use with TestMetadata to run selected tests using test list in case .dll is provided that contains multiple tests.

Since the resultfile parameter is not provided in the command, the results will be generated inside TestResults directory inside the path where batch file is executed (Start in path of the task). This is a better option compared to giving explicit result file because the result file may get overwritten if same test runs multiple times.

A Task that contains details of the batch file to execute (.job file)

This is a task configured to run at desired time or with available triggers. Below is a table explaining the parameters to be set or changed in the task:

 

Tab / Section

Field

Used to specify

Comments

Task

Run

Path of the file to be run as scheduled task

Give the full path of the batch file here

Task

Start in

Path of the folder where file/program is located

Give path of the folder that contains the batch file

Task

Run as

User account details that runs the scheduled tasks

Give valid user name password that has access to schedule tests

Schedule

Schedule Task

How often the task should run

Select Once/ Daily/ Weekly as required

Schedule

Start Time

Time when scheduled task should start

 

Schedule

Advanced Options

Additional Scheduling options such as dates

 

Schedule/
Advanced

Repeat Task

Specify to repeat task at interval specified

Use for executing task multiple times

 

Steps

  • Create a directory where you want to put the batch file. Let us say [WorkingDirectory]. Note that by default, results file and other logs will be generated in this directory.
  • Create a batch file MSTest.bat in [WorkingDirectory].
  • Create Batch File.
    • Open MSTest.bat with notepad.
    • Right Click the file and select Edit in context menu. OR
    • Open Notepad, and use Open File menu to browse the MSTest.bat file.
    • Avoid double clicking the batch file because that will directly execute the command.
    • Copy and paste the MSTEST command (available in above section) to this file.
    • Modify the parameter values accordingly:
      • Binary/WebTest/LoadTest file path
      • Other parameter values and removed unwanted parameters
    • Save the file.
  • Create task.
    • Open Task Scheduler.
    • Click Create New (The actual option will vary based on operating system).
    • Type a name for the task and an optional description.
    • Do one of the following:
      • To select a schedule based on the calendar, click Daily, Weekly, Monthly, or One time, click Next; specify the schedule you want to use, and then click Next.
      • To select a schedule based on common recurring events, click When the computer starts or When I log on, and then click Next.
      • To select a schedule based on specific events, click When a specific event is logged, click Next; specify the event log and other information using the drop-down lists, and Then click Next.
    • Click Browse to find and select the batch file (MSTEST.bat) created in above steps.
    • It’s good to specify the directory to start task in. Set it to [Workingdirectory].
    • Click OK/Finish.
  • Done. Your scheduled task will be triggered at specified time and it will execute the batch file to start execution of the automated test(s).

See Also