Integrating JavaScript Unit Tests with Continuous Integration

One of the main stories for the Web Client Developer Guidance project was to learn how to manage the JavaScript code that is required in modern web applications. This includes how to organize JavaScript, how to load it on a page (especially a problem when doing composite UI), and also how to test it. This blog post will concentrate on how we were able to write unit tests that validated the behavior of our custom JavaScript libraries and how we integrated those unit tests into our continuous integration builds.

JavaScript Unit Testing Frameworks

Initially we used the JsUnit unit testing framework. A big reason I liked JsUnit was that it supported CI integration with the help of a server component that ran on the Java VM. With a bit of work we were able to replace the server component with our own that runs on ASP.NET. We could run JavaScript unit tests on several browsers and get reports of success/failure in XUnit format.

However, we started running into issues with JsUnit tests. The test failure information was not detailed enough to get an idea why the tests failed. Also, it was hard to create tests in JsUnit that validated asynchronous behavior, typical in our AJAX calls.

We then migrated our JavaScript unit tests to QUnit. Two downsides to QUnit was that it 1) didn’t support a server component out of the box and 2) had only 3 assertion methods. These issues were easy to work around.

First we added functionality to the QUnit test runner that harvested the results of each test run and posted the results to the same ASP.NET server component we used for the JsUnit test runner.

Take a look at this code in our source folder: Source\RI\MusicStore.Web.Tests\Scripts\TestRunner\testRunner.js

Second, we added additional MSTest style assertions.

Check out the code here: Source\RI\MusicStore.Web.Tests\Scripts\TestRunner\qunitExtensions.js

Running the tests in MSBuild

Once we had the test runner configured with our unit tests, we then automated running the tests in a browser. We accomplished this using an MSBuild task similar to the following:

 <Exec Command="&quot;%(QUnit_BrowserPaths.Identity)&quot; &quot;file:///$(QUnit_TestRunnerFilePath)?&amp;submitResults&quot;" 
      ContinueOnError="true"/>

QUnit_BrowserPaths is an ItemGroup of paths to browser exe files that you want tests run on. QUnit_TestRunnerFilePath is a property defining the location of the test runner HTML file that hosts the suite of JavaScript unit tests:

 <QUnit_TestRunnerFilePath>
    $(SourceWorkingDir)RI\MusicStore.Web.Tests\Scripts\TestRunner\testRunner.html
</QUnit_TestRunnerFilePath>

Notice the ContinueOnError=”true”. This is necessary because the test runner automatically closes the browser instance after a test suite run. Internet Explorer raises an error when we force it to close, whether or not the tests ran successfully.

Server Component

The server component receives the test run results that was posted by the testRunner.html. This data is translated into XUnit style XML and saved to disk. If there is a failing test, the server component will also save a file to disk named FailingJSTest.xml. When our CI build is run, it will look for this file and raise an error if it is found.

   <Target Name="Qunit_CoreLookForFailingTests">
    <ItemGroup>
      <FailingJSTests Include="$(JavaScriptTestResultsFolder)FailingJSTest*.xml"/>
    </ItemGroup>
    <Error Condition="'@(FailingJSTests)' != ''" Text="Failing JavaScript unit test found"/>
  </Target>

Comments

• Anonymous
  May 03, 2011
  Hi, Great article- where about's is your source folder as I'd love to take a look at the code... Thanks, Shaw.

• Anonymous
  May 04, 2011
  Sorry, I forgot to include a link to our guidance: webclientguidance.codeplex.com Take a look at the CHM file. You'll find a topic titled "How to: Create and Run JavaScriptUnit Tests" as well as all our source code. Also, the Silk project extends what we did. Please take a look at their guidance. http://silk.codeplex.com/