Partilhar via

Using the SysTest unit testing framework in Dynamics AX – why bother?

  • Artigo

I’ve written about the Microsoft Dynamics AX unit testing framework, SysTest, in this blog in the past. The most notable was a post dedicated to reference materials for the framework here. I closed that post, dated February 2008, with “More on SysTest later…”. I doubt that I had over three years later in mind when I wrote that… nevertheless, here we are in 2011.

I’ve had a lot more opportunity to talk directly with members of the AX ecosystem about testing in the past year. While much of that time is spent discussing the coolness that is Visual Studio 2010 testing tools, the testing framework that will be part of its third AX release when Microsoft Dynamics AX 2012 arrives shortly is improving with each release and is still worth discussing. More often than not, it is me who is bringing it up because it’s a very important part of the overall AX testing story.

A recent e-mail discussion with a partner triggered this post. The partner is new to AX testing and was asking for some more resources, particularly about SysTest. After sending him the list of resources from my 2008 post and having him experiment with the framework, he sent me an e-mail that could be summarized as “why should I bother?” He understood the framework, but didn’t see the value proposition of applying the framework after working through the MDSN examples and other examples.

His response demonstrates a gap in what we’re proving to the ecosystem. We do explain the SysTest framework reasonably effectively (in my opinion, certainly open to other feedback). But in order to focus on the framework, our examples are trivial and don’t simulate typical usage. The gap to fill is providing more information on application of the framework to problems that AX ISVs, implementation partners, and customers are solving.

This post if the first step towards filling that gap. Coming back to the question in this post’s title… why bother? I’m going to answer this at a high level for this post and save details for the future.

  1. Many engineers in the ecosystem want to develop a set of automated regression tests. ISVs want the tests as they create new releases or test new releases from Microsoft. Implementation partners and customer developers want the tests as they are creating additional features for point solutions and move the features from development to test to production environments.
  2. While the initial tendency is to create automated tests through the user interface, that’s not the best starting point. The best starting point is to test the business logic below the user interface. These “headless” tests are better because they are:
    • More reliable – It is critical that tests consistently give the same answer. Headless tests are much more reliable than UI driven tests as they don’t have the control discovery and timing issues that frequently challenge UI tests.
    • Higher performing – It is critical that tests provide feedback quickly. Headless tests can run one or more orders of magnitude faster than UI tests.
    • More maintainable – It is critical that the cost of maintaining tests is manageable over time. This is the most important part of the return on investment equation for test automation and headless tests are much cheaper to maintain than UI tests over the long haul.

Notice that I’ve bucketed automated regression tests into two very broad categories – UI and headless. How you go about effective authoring of headless tests using SysTest in AX is coming… and I won’t wait for another 3 years to deliver on that!

Comments

  • Anonymous
    August 01, 2011
    Hi Dave, use Microsoft this framework also for developing Dynamics AX? If yes, why don´t Microsoft deliver these test cases? Peter

  • Anonymous
    August 19, 2011
    @Peter - Microsoft does use SysTest internally and has tens of thousands of test cases implemented.  The latter half of that statement is one of the challenges to delivering the tests.  Delivering all of the SysTest test cases would overwhelm any partner or customer.  Delivering a subset requires some internal engineering work.  Which tests?  What dependencies do these tests have on other internal functions?  How do we deliver them?  How do we support them?  The required engineering work was considered for AX 2012 but ultimately, after consulting with some partners, we decided to focus our investment on the VS10 functional testing with AX.