通过

SysTest part IV.: Test/Suite initialization and cleaning; Execution isolation

  • 项目

We already know how to create test suites. Today I would like to cover some additional reasons for creating test suites: initialization, cleaning and test execution isolation.

Initialization and Clean up

As long as our test class contains just one test method everything is fine. In that method we include all preparation steps as well as all cleaning steps (resource cleaning and stuff). But honestly how often do you write test classes with just one test method? Not that often, right? Once we have more than jus one test method in the class we should start refactoring the code. That's pretty easy by extracting setup and tearDown methods. In these two methods we do whatever we need to do. All our test methods then call these methods:

  private void initialize()
 {
    // initialization code...
 }
 private void clean(
 {
    // cleaning code...
 } 
 
 public void testMethod1()
 {
    ;
    this.initialize();
 
    // your test method code...
 
    this.clean();
 }

The situation gets little bit more complicated when exceptions are expected.

  public void testException()
 {
    ;
    this.initialize();
 
    try
    {
        // do something here...
    }
    catch()
    {
        // make sure you catch every possible exception because if an exception gets uncaught
        // then it will be caught by the framework and your cleaning method will not be called!
    }
    this.clean();
 }

It would be much easier if the framework itself would force calling of these two methods. That's really happening. The framework itself is calling setup before every test method and tearDown after every test method (regardless of the test result).

You can rewrite the previous simple sample to:

  public class InitAndCleanTest extends SysTestCase
 {
    public static void main(Args _params)
    {
        SysTestSuite suite = new SysTestSuite(classstr(InitAndCleanTest));
        SysTestResult result = new SysTestResult();
        ;
        result.addListener(new SysTestListenerPrint());
        suite.run(result);
        print result.getSummary();
    
        pause;
    }

    public void setup()
    {
        ;
        print 'Test initialization called';
    }
    public void tearDown()
    {
        ;
        print 'Test cleaning called...';
    }

    public void testMethod1()
    {
        ;
        print 'testMethod1';
    }
    public void testMethod2()
    {
        ;
        print 'testMethod2';
    }
 }

But why do I mention it when discussing test suites? Well that might be one of the reasons for you to create a test suite. As well as test method setup and teardown methods, SysTest has setup and teardown methods for test suites. That means that the framework would call setup method before calling the first method in the suite and tearDown after the last method in the suite.

Let's create a simple suite with setup and tearDown methods and let's add our previous test class in it:

  public class InitAndCleanSuite extends SysTestSuite
 { 
    public static void main(Args _params)
    {
        InitAndCleanSuite suite = new InitAndCleanSuite();
        SysTestResult result = new SysTestResult();
        ;
        result.addListener(new SysTestListenerPrint());
        suite.run(result);
        print result.getSummary();
    
        pause;
    }

    public void new()
    {
        ;
        super();
        this.add(new InitAndCleanTest());
    }
    public void setup()
    {
        ;
        print 'Suite initialization called';
    }
    public void tearDown()
    {
        ;
        print 'Suite cleaning called...';
    }
 }

Run the suite and notice that the setup and tearDown are called just once. So whenever you need to call a special initialization (data setup for example) routine before all your tests then creating a suite would be a way to achieve that.

Execution isolation

Another reason for a suite would be an "execution isolation". Sometimes we want to run the test but we don't necessarily want to update data in the database. SysTest offers several execution isolations.

Transaction

We can ask SysTest to place our code in a transaction that will be aborted at the end of our test. You might ask why after every test. Well in unit testing all unit tests should run independently and thus it doesn't make sense to keep data from the previous test to the next ones. That's why SysTest aborts open transactions after every test method.

How do we do that? It's super easy. Whenever you create a test suite then extend the suite not from SysTestSuite but from SysTestSuiteTTS.

  public class InitAndCleanSuite extends SysTestSuiteTTS
 { 
    ...
 }

Company account

The problem with transaction isolation is in forms. Forms usually require data to be committed to the database. The solution that SysTest offers is in a separate company account. SysTest can create a company account for your test and at the end delete that company account. It has one drawback though. Creation, switching and deletion of the company account takes some time and the test execution is then dramatically affected. That's why SysTest offers here two isolation types: method and class isolations.

If you derive your test suite from SysTestSuiteCompanyIsolateMethod then a company account is created before each test. The test then is executed within that company account and the account is deleted when the test is finished.

This is probably exactly what you would expect. To speed it up a little bit you can derive the suite from SysTestSuiteCompanyIsolateClass and SysTest will not delete the company account after every test but only after the last test. This goes a little against the idea of independent tests and you should treat this as a compromise.

Runtime test isolation

In the previous section I have described how to create a test suite serving as execution isolation for our test classes that are included in this suite. Do we have something similar without actually creating the test suite manually at design time? Yes we do. Do you still remember how we executed our tests using SysTest API (programmatically)? If not then here is the code:

     SysTestSuite suite = new SysTestSuite(classstr(InitAndCleanTest));
    SysTestResult result = new SysTestResult();
    ;
    result.addListener(new SysTestListenerPrint());
    suite.run(result);
    print result.getSummary();
 
    pause;

To run a test we create a default suite and wrap our test class in this suite. Then we can run the suite (and it will run all or test methods). Well, we are not forced to stay with the base suite class. We can use whatever SysTestSuite class. The following job would run the same tests but this time that would be isolated in a transaction:

     SysTestSuite suite = new SysTestSuiteTts(classstr(InitAndCleanTest));
    SysTestResult result = new SysTestResult();
    ;
    result.addListener(new SysTestListenerPrint());
    suite.run(result);
    print result.getSummary();
 
    pause;

Final note about transaction isolation

Assume we have the following test class (testing that an exception was thrown).

  public class TtsExceptionTest extends SysTestCase
 { 
    public static void main(Args _params)
    {
        SysTestSuite suite = new SysTestSuite(classstr(TtsExceptionTest));
        SysTestResult result = new SysTestResult();
        ;
        result.addListener(new SysTestListenerPrint());
        suite.run(result);
        print result.getSummary();
    
        pause;
    }

    public void testExpectedException()
    {
        ;
        try
        {
            // exception is being expected
            throw Exception::Error;
        }
        catch
        {
            // exception was caught
        }
    }
 }

When you run the test you will see the expected result:

  2 run, 0 failed

Now let's change the execution isolation to transaction isolation.

     public static void main(Args _params)
    {
        SysTestSuite suite = new SysTestSuiteTTS(classstr(TtsExceptionTest));
        SysTestResult result = new SysTestResult();
        ;
        result.addListener(new SysTestListenerPrint());
        suite.run(result);
        print result.getSummary();
    
        pause;
    }

Notice that the test is now wrapped in SysTestSuiteTTS and executed. Suprisingly the test result is now:

  2 run, 1 failed

What!!! Why is that? Well, that's due to a "unique" way of handling exceptions inside transactions in DynamicsAX X++ language. When you are inside a transaction (and we are here) and an exception is thrown then the transaction is aborted and the exception is handled at the level where the transaction is defined (in our case inside SysTest framework). Therefore our catch doesn't work here and the framework reports our test as failure.
For more information about this see Developer Help in DynamicsAX 4.0 (Developer.chm) and navigate to Microsoft Dynamics AX SDK > X++ Language Reference Guide > Statements and Loops > Exception Handling > Exceptions Inside Transactions.
You can also read about this topic in the blog post Axapta error handling and database transactions posted on SysDictCoder.

You can download these samples: Part04: Suites and Execution isolation.xpo

In the next post I will try to describe listeners.

Comments