WF4 Versioning Spike: How To Unit Test Activity Versioning

With the work I’ve been doing on versioning I’ve had to write unit tests that verify the behavior I expect from the helper classes in Microsoft.Activities.dll.  If you want to verify that your assembly versioning strategy is working correctly you may have to do similar testing.  This sort of testing is tricky… in this post I’ll share with you my solutions to some tough problems.

Test Problem: Multiple Versions of the Same Assembly in a Test Run

There is only one deployment directory for the test run and all deployment items are copied there.  I can’t deploy ActivityLibrary V1 and V2 to the same test directory but I need to deploy both for a test run (note: I did not tackle GAC deployment for this set of tests)

Solution

The solution comes in two parts.

1. How to create different versions of the assemblies in one build
2. How to deploy the different versions of the assemblies when testing

Creating Different Versions of the Assemblies in One Build

For my testing I need a variety of different assemblies in debug and release builds with different versions, signing options and references to other assemblies

• ActivityLibrary – Version 1 (signed and unsigned), Version 2 (signed and unsigned)
• Workflow – Version 1 (signed), Version 2 (signed, unsigned)

To do this I created a number of projects that produce the same assembly and write the output to the bin directory for all build configurations (rather than bin\debug or bin\release)

As you can see I have several ActivityLibrary projects with different names but they all produce ActivityLibrary.dll and the same is true for WorkflowLibrary

Deploy the Different Versions of the Assemblies When Testing

I deploy different versions of the assemblies into subdirectories of the test directory.  Then when I create the AppDomain for the test I set the ApplicationBase to the subdirectory for that test.  This ensures that the test directory contains the versions of the assemblies that I want.

To make the test code less error prone, I created constants to define the versions of assemblies and combinations of directories where I will deploy them and I pass these values to the [DeploymentItem] attribute 

    1: /// <summary>
    2: ///   The Activity Library(V1)
    3: /// </summary>
    4: private const string ActivityV1 = @"Tests\Test Projects\ActivityLibrary.V1\bin\ActivityLibrary.dll";
    5:  
    6: /// <summary>
    7: ///   Deploy Directory with Workflow (V1) Activity (V1)
    8: /// </summary>
    9: private const string WorkflowV1ActivityV1 = "WorkflowV1ActivityV1";
   10:  
   11: /// <summary>
   12: /// Given 
   13: ///   * The following components are deployed
   14: ///   * Workflow (compiled) V1
   15: ///   * ActivityLibrary.dll V1
   16: ///   When 
   17: ///   * Workflow (compiled) is constructed using reference to Activity V1
   18: ///   Then
   19: ///   * Workflow should load and return version 1.0.0.0
   20: /// </summary>
   21: [TestMethod]
   22: [DeploymentItem(TestAssembly, WorkflowV1ActivityV1)]
   23: [DeploymentItem(ActivityV1, WorkflowV1ActivityV1)]
   24: [DeploymentItem(WorkflowV1, WorkflowV1ActivityV1)]
   25: public void WorkflowV1RefActivityV1DeployedActivityV1ShouldLoad()
   26: { 
   27:    // ...
   28: }
   29:     

Test Problem: Assembly.Load and Cached Assemblies in the AppDomain

The biggest issue I ran into when writing my tests is caused by the behavior of Assembly.Load.  The issue is that when you call Assembly.Load it checks to see if it has already loaded an assembly that will satisfy the request and will use that assembly.  When you have a number of tests that need to verify if the correct assembly was loaded you find that suddenly you have a test order dependency.  What happens is that when you run a test by itself it passes but when you run all of your tests some of them fail. 

I want to be sure that when I run my tests that I always get the same results no matter how many I run or in what order.  To solve this problem I need to deal with the assemblies cached in the AppDomain and there is no way to unload an assembly once it has been loaded.

Solution

To solve this problem for each test I’m going to create a new AppDomain run the test code in the new AppDomain and then Unload it when I’m finished.  This ensures that I start with an empty AppDomain and I can verify that only the assemblies I want are loaded. 

My test class StaticXamlHelperTest has a matching worker class StaticXamlTestWorker which does the actual testing in the new AppDomain.

    1: [Serializable]
    2: public class StaticXamlTestWorker : MarshalByRefObject

Then I added two helper methods to the StaticXamlHelperTest class to create the AppDomain and create the Worker class in the AppDomain

    1: private static StaticXamlTestWorker CreateTestWorker(AppDomain domain)
    2: {
    3:     domain.Load(Assembly.GetExecutingAssembly().GetName().FullName);
    4:     var worker =
    5:         (StaticXamlTestWorker)
    6:         domain.CreateInstanceAndUnwrap(
    7:             Assembly.GetExecutingAssembly().GetName().FullName, 
    8:             "Microsoft.Activities.Tests.StaticXamlTestWorker");
    9:  
   10:     return worker;
   11: }
   12:  
   13: private AppDomain CreateWorkerDomain(string workerPath)
   14: {
   15:     return AppDomain.CreateDomain(
   16:         this.TestContext.TestName, 
   17:         null, 
   18:         new AppDomainSetup { ApplicationBase = Path.Combine(this.TestContext.DeploymentDirectory, workerPath) });
   19: }

Then for each test I follow a simple pattern

    1: [TestMethod]
    2: [DeploymentItem(TestAssembly, WorkflowV1ActivityV1)]
    3: [DeploymentItem(ActivityV1, WorkflowV1ActivityV1)]
    4: [DeploymentItem(WorkflowV1, WorkflowV1ActivityV1)]
    5: public void WorkflowV1RefActivityV1DeployedActivityV1ShouldLoad()
    6: {
    7:     var domain = this.CreateWorkerDomain(WorkflowV1ActivityV1);
    8:     try
    9:     {
   10:         CreateTestWorker(domain).WorkflowV1RefActivityV1DeployedActivityV1ShouldLoad();
   11:     }
   12:     finally
   13:     {
   14:         if (domain != null)
   15:         {
   16:             AppDomain.Unload(domain);
   17:         }
   18:     }
   19: }

• Create the AppDomain (line 7)
• Create the worker inside a try block and call the test method (line 10)
• In the finally block Unload the AppDomain (line 16)

Test Problem: How to Know Which Version of the Activity Library Was Actually Loaded

Problems with assembly loading generally result in exceptions being thrown but sometimes you might be surprised to find the workflow loading and happily running with a version of the activity that is something other than what you expected.

Since I am testing infrastructure I created Workflows with the sole purpose of loading an activity from an Activity Library and returning the version of the assembly that contained the activity.  My activity is named GetAssemblyVersion.

    1: public sealed class GetAssemblyVersion : CodeActivity<Version>
    2: {
    3:     protected override Version Execute(CodeActivityContext context)
    4:     {
    5:         return Assembly.GetExecutingAssembly().GetName().Version;
    6:     }
    7: }

I then create a Workflow that declares an OutArgument<Version> and uses the GetAssemblyVersion activity.

Because I’m testing Microsoft.Activities.StaticXamlHelper I create a partial class with an overloaded constructor that calls the method I really want to test  StaticXamlHelper.InitializeComponent

    1: public Workflow(XamlAssemblyResolutionOption xamlAssemblyResolutionOption, IList<string> referencedAssemblies)
    2: {
    3:     referencedAssemblies.Add(Assembly.GetExecutingAssembly().GetName().FullName);
    4:  
    5:     switch (xamlAssemblyResolutionOption)
    6:     {
    7:         case XamlAssemblyResolutionOption.FullName:
    8:             StrictXamlHelper.InitializeComponent(this, this.FindResource(), referencedAssemblies);
    9:             ShowAssemblies();
   10:             break;
   11:         case XamlAssemblyResolutionOption.VersionIndependent:
   12:             this.InitializeComponent();
   13:             break;
   14:         default:
   15:             throw new ArgumentOutOfRangeException("xamlAssemblyResolutionOption");
   16:     }
   17: }

Now I’m ready for some test code – remember this code will run in the new AppDomain and it makes use of Microsoft.Activities.UnitTesting

    1: public void WorkflowV1RefActivityV1DeployedActivityV1ShouldLoad()
    2: {
    3:     var activity = new Workflow(XamlAssemblyResolutionOption.FullName, GetListWithActivityLibraryVersion(1));
    4:     var host = new WorkflowInvokerTest(activity);
    5:     host.TestActivity();
    6:     host.AssertOutArgument.AreEqual("AssemblyVersion", new Version(1, 0, 0, 0));
    7: }

Here is how it works

• Line 3 - Create the activity using the overloaded constructor providing the list of assemblies you want to reference (provided by a helper method)
• Line 4 - Create a test host – Microsoft.Activities.UnitTesting.WorkflowInvokerTest
• Line 5 - Test the activity
• Line 6 – Assert the out argument “AssemblyVersion” is 1.0.0.0

Bottom Line

If this sounds complicated… that’s just because it is.  The complete source for all the unit tests is included in the Microsoft.Activities source so you can check out the details of how it works.

I know… you are thinking this sounds like way too much work for your project.  If you only knew how many bugs I discovered in my code and fixed before you ever saw them (including one very obscure bug that didn’t appear on VS2010 RTM but only on VS2010 SP1 beta) you would take the time to write some quality code.