Integrating Additional Tools in a SharePoint Continuous Integration Build

Update: This article is brought to you by Chris O’Brien, SharePoint MVP and consultant based in UK. You can read more from him @ https://www.sharepointnutsandbolts.com. Thanks for reading!

We have finally come to the last post in our series on Continuous Integration for SharePoint. Today we’ll first wrap up on the subject of running tests from a build, and then primarily we’ll be discussing code quality. In particular, we’ll look at how we can use the build process as the vehicle for performing several forms of analysis on our code, using tools such as SPDisposeCheck and Visual Studio Code Analysis.

Running unit/integration tests in a build

In the last post, Running Tests as Part of a Build, I showed how Visual Studio Coded UI Tests can be integrated into a build to verify its quality. Since we didn’t get to cover it there, it’s worth mentioning some considerations for running unit tests within a SharePoint build; of course, unit tests may provide equal or greater value than Coded UI Tests, and ultimately a mix of several test types is recommended. There is a caveat here – because SharePoint 2010 runs on .NET 3.5 and Team Foundation Server 2010 Team Build runs on .NET 4, it is not possible to use unit tests which call the SharePoint API within an automated build. Such tests are typically referred to as integration tests.

The table below gives a summary:

Type of test	Can be used in TFS Build
Unit test	Yes
Integration test	No*
Unit test with mocking framework (e.g. Typemock)	Yes

* Integration tests may work on a build server which has both the TFS Build Agent and SharePoint 2010 installed, but this configuration is not recommended.

It’s worth remembering that this caveat applies to automated builds only. Unit and integration tests for SharePoint can be run manually from within Visual Studio 2010 of course, so long as you have Visual Studio 2010 Service Pack 1 and some Visual Studio configuration settings are changed. See SharePoint 2010 and unit tests for more information.

With that covered, let’s move on to SPDisposeCheck and Code Analysis.

Integrating SPDisposeCheck into the build

If you’re not yet familiar with SPDisposeCheck, it’s a tool built by Microsoft which scans custom SharePoint code for issues related to object disposal – such issues can cause severe performance problems, and use of the tool is considered a best practice. Integration of such tools into the build tends to work well, since it means that individual developers do not need to remember to run the tools manually. First, however, a note on running tests in a build. Since SPDisposeCheck is a command-line executable, it can be integrated into a build easily. As you may remember from Deploying WSPs as part of an automated build, we can use the InvokeProcess activity in a TFS build workflow to call out to any external process. In this case, all we really need to do is drag an InvokeProcess activity onto the workflow design surface, and then configure it with the install location of SPDisposeCheck.exe and the location where the assemblies which are output from the build can be found.

Integrating SPDisposeCheck – the process

Here are the detailed steps:

1. Install SPDisposeCheck on the build server (i.e. the server running the TFS Build Agent – this could be multiple if you have scaled out this role). We’ll need to take a note of the install location of the executable file – by default, the installer will use C:\Program Files (x86)\Microsoft\SharePoint Dispose Check\SPDisposeCheck.exe.

2. In TFS source control, open the XAML file which represents your build process from the BuildProcessTemplates folder. Double-click the file to open it in the Visual Studio workflow designer.

3. Identify a suitable place in the workflow to run SPDisposeCheck. This needs to be somewhere after the step which performs compilation, as we need the assemblies to have been created in the build output directory. For simplicity, consider implementing these modifications to the build template in the same location as any other custom steps you have, e.g. just before steps to deploy WSP packages.

4. Drag on a Sequence activity. Rename the display name to something like “Run SPDisposeCheck”.

5. Inside the Sequence, drag on an InvokeProcess activity. Rename the display name to something like “SPDisposeCheck”.

6. [OPTIONAL] Define an integer variable to collect the number of failures returned by SPDisposeCheck. This should be configured as below:

7. Configure the InvokeProcess activity. The FileName property should be the location of SPDisposeCheck.exe on the build agent, and the Arguments property should be the outputDirectory build variable, wrapped in a quotation marks. Additionally, if we defined the integer variable in step 6, the Result property should populate this:

8. Add logging activities to the InvokeProcess activity, so that any messages output from SPDisposeCheck appear in the build report. Drag a WriteBuildMessage activity to capture the stdOutput variable, and a WriteBuildError activity to capture errOutput.

Unconfigured	Configured

9. Edit the properties of the WriteBuildMessage activity - change the Importance property to BuildMessageImportance.High. Without this, messages from SPDisposeCheck will not be written to the build report by default.

10. [OPTIONAL] Write a warning message to the build summary if SPDisposeCheck finds issues.

a. Add an ‘If’ activity immediately after the InvokeProcess activity which calls SPDisposeCheck. Modify the Condition property to be an expression which tests if the integer variable (created in step 6) is greater than zero. This may look like “SPDisposeCheckResult > 0”.

b. Within the ‘If’ activity, drop a WriteBuildWarning activity. Edit the Message property to log the number of issues reported by SPDisposeCheck. This may look like String.Format(“SPDisposeCheck reported {0} issues – these should be investigated.”, SPDisposeCheckResult)

11. Save and check in the build process template file.

Integrating SPDisposeCheck – the result

When a build runs, the build report will now contain details from the analysis SPDisposeCheck performed across the assemblies which were built. Since SPDisposeCheck writes to the console, our WriteBuildMessage activities will take this output and add it to the build log:

If you performed the optional steps to write a warning message to the build summary, this can be seen in the summary view for the build (before going into the detailed log):

Using Code Analysis in a build

Like other tools which developers can run manually but may forget to, Code Analysis is also perfectly suited to Continuous Integration. Code Analysis is a Visual Studio 2010 (Premium or above) feature which checks code for adherence to good practices, including those documented in the .NET Framework Guidelines. Microsoft supplies several Code Analysis rule sets to check code for different issues, and it is also possible to create custom rule sets and suppress individual rules to have full control over the analysis process. My experience of the capability is that whilst some tuning is often beneficial, even adept developers can discover valid issues with their code which they had not previously noticed.

Enabling Code Analysis – the process

Here are the detailed steps:

1. Edit the build definition, and in the ‘Process’ tab, find the ‘Perform Code Analysis’ option. Set this to either ‘Always’ or ‘AsConfigured’ – using the latter means the settings on each individual Visual Studio project being built will be respected.

2. Queue a new build using this build definition. When the build finishes, the build report will most likely contain many CA0060 errors mainly relating to SharePoint assemblies. The description of each CA0060 error will be similar to “The indirectly-referenced assembly [assembly details] could not be found. This assembly is not required for analysis, however, analysis results could be incomplete.” as shown in the image below:

3. Add missing assemblies to build server. Whilst Code Analysis should in fact run successfully despite the CA0060 errors (and any issues should now appear in the build report), it is highly recommended that the additional ‘noise’ caused by the CA0060 errors is dealt with. To do this, each of the assemblies listed with a CA0060 error should be copied to the server(s) running the TFS Build Agent service. The assemblies can be copied from the GAC of a SharePoint 2010 server – these should be added to the folder on the build server which already contains other SharePoint assemblies. If you used the Microsoft-supplied SharePointProjectToTfsBuildServer.ps1 PowerShell script as documented in Creating your first TFS Build Process for SharePoint projects, this location will be ‘C:\Program Files (x86)\Reference Assemblies\Microsoft\SharePoint14’.

4. Restart the TFS Build service. Ensure no builds are currently running, and then open the Team Foundation Server Administration Console on the TFS Server. In the ‘Build Configuration’ view, use the ‘Restart’ link for the Build Service to do this:

5. Test the changes by queuing a new build. The build report should now only valid Code Analysis issues, without the noise of the CA0060 errors:

Summary

Once automated builds are implemented, they provide a great facility for automatically running processes ‘in the background’ which developers would otherwise have to run manually. In real-world conditions, this provides a significant advantage since individual developers do not have to periodically run such tools between coding. Additionally, the automated build never forgets to do these tasks.

In this article we showed how to include SPDisposeCheck and Visual Studio Code Analysis in a build process. Practically any tool which can be called from the command-line can be run from an automated build, though it’s worth noting that some Visual Studio capabilities such as Code Profiling are not supported in TFS 2010/VS 2010.

In broader terms, this series has hopefully shown that continuous integration can provide huge gains in the SharePoint development process. Bugs can be found more quickly, and productivity can be enhanced by automating regular tasks such as those related to code quality.

Comments

• Anonymous
  September 20, 2012
  Great post! It represents a great progression in SharePoint development to be able to automate the build process and add in additional tools (like SPDisposeCheck). I know it has been possible for a while, but this series goes a long way in showing us how to make it real. Regarding your comment: "it is not possible to use unit tests which call the SharePoint API within an automated build", in my opinion, it would be a big step forward to make that possible. I know this is conjecture, but would it be a good guess to say that we will be able to use unit tests that call the SharePoint API at some point in the future. That is...will TFS Team build and SharePoint ever be on the same framework.

• Anonymous
  September 25, 2012
  Such a NICE POST FOR sHAREPOINT DEVELOPER

• Anonymous
  September 26, 2012
  Yes, this is great! Don't forget to follow Chris on twitter (@ChrisO_Brien). Thanks for your comments Doug. I'll make sure to pass them to the VS-SharePoint team.

• Anonymous
  September 27, 2012
  Yeah, the approach Chris describes works really well. One additional point though - you might get tired of false positives from SPDisposeCheck (it does produce them sometimes) - so it can be worth using the SPDisposeCheckIgnoreAttribute, and that has some practical 'quirks' that I've blogged about previously: www.novolocus.com/.../practical-spdisposecheck-namespaces-delegates-and-rootwebs

• Anonymous
  October 02, 2012
  It is a very nice thing to see your excellent work and I like your article very much. With your rich knowledge, we can learn more from your wonderful post. Thanks so much. www.sharepointengine.com

• Anonymous
  September 06, 2013
  Love being able to customize and add additional tools. Thanks for this.