Set up a CI/CD pipeline with unit testing and code coverage for your VSTS extension

We receive a notification that there’s a pull request to review for our Team Services extensibility project. We discuss the proposed changes as a team, eventually merge the new feature(s) into the master branch, and close the pull request. Done … right?

Not quite. The merge triggers our automated CI/CD pipeline, which consistently builds, tests and delivers our package to the Visual Studio marketplace.

This post should be a valuable reference, if you’re interested in how we configured our Countdown Widget pipeline and how the team included testing and code coverage. Let’s walk through the configuration of our pipeline.

MODIFIED 2017.08.14 – Update to align with latest codebase from https://github.com/ALM-Rangers/Countdown-Widget-Extension. If you remember the previous pipeline, you’ll notice that the pipeline has been simplified by code generated by the the fastest path to a new VSTS extension generator.

---

Build

We start with the continuous integration build, made up of thirteen steps.

The magic around unit testing and code coverage happens during steps 8-11. See Testing a Team Services extension and tracking code coverage for more details.

#	BUILD STEP	BUILD SNIPPET	WHAT’S HAPPENING?
1	Install npm dependencies		Install all modules listed as dependencies in package.json.
2	npm custom		Run a custom npm task (install –g whitesource) to install the Whitesource npm plugin.
3	Replace tokens in Whitesource config		We replace the WhiteSourceAPIKey in the whitesource.config.json file with the value of the variable with the same name. Token Regex is “__(\w+)__”, which means that we’re looking for one or more letter, digit or underscore characters, surrounded by a double underscore “__name__”.
4	Replace tokens for TelemetryClient		We replace the INSTRUMENTATION key in the TelemetryClientSettings.ts file. Variables INSTRUMENTATIONKEY defines the Application Insights instrumentation key. We’re using a variable to ensure we do not leak the instrumentation key into the OSS repo. system.debug can be set to true for detailed and verbose logging during the build.
5	Run whitesource		Run the Whitesource tool to scan and detect security vulnerabilities, problematic OSS licenses and quality issues. See Manage your open source usage and security in your pipeline for details.
6	npm run build		Run the package build script, as defined under “scripts” in package.json.
7	nom run tests		Run the package test script, as defined under “scripts” in package.json.
8	publish test results		Publish the test results, located in the ./reportTests/ folder.
9	Publish Code Coverage Results	RESULTS	Publish the code coverage results from the formatted cobertura-coverage.xml file.
10	Package Extension		We package the extension using the VSTS Developer Tools Build Tasks extension and generate the output.vsix artifact. Extension ID is appended with Alpha to differentiate it from deployed DEV, BETA, and PROD packages.
11	Publish Artifact		We publish the artifact, output.vsix, to $(Extension.OutputPath) to be picked up by the release.

---

Release

Next we continue with continuous delivery (CD), made up of three environments. With the exception of configuration values, this part of the pipeline is identical (consistent) to the one we covered in up Set up a CI/CD pipeline for your Team Services extension.

Continuous Delivery is the ability to use the output from the CI to build and deploy the new known good build to one or more environments automatically ( bit.ly/28PWsfk ). There is a subtle difference between Continuous Delivery and Continuous Deployment. The latter is to a single environment. A small team might only implement Continuous Deployment because each change goes directly to production. Continuous Delivery is moving code through several environments, ultimately ending up in production, which may include automated UI, load and performance tests and approvals along the way. – Extract from DevOps - Applying DevOps to a Software Development Project

DEV

The environment with one task is triggered after a successful release creation. Both the pre-deployment and post-deployment approvals are automatic, ensuring that the development team is always working with the latest version. There’s no manual intervention.

#	RELEASE STEP	BUILD SNIPPET	WHAT’S HAPPENING?
1	Publish Extension		We publish the extension to the alm-rangers publisher, using the VSTS Developer Tools Build Tasks extension.  The published private extension is shared with our BETA sandbox-one and used for exploratory and user acceptance testing. The extension ID is suffixed with BETA to make it simple to identify this extension.

---

BETA

The environment with only one task is triggered after a successful deployment to the DEV environment. The BETA approvals mandate that one of the specified users approve the release, to ensure that a high quality bar can be met. Users typically include the project lead and program manager.

#	RELEASE STEP	BUILD SNIPPET	WHAT’S HAPPENING?
1	Publish Extension		We publish the extension to the alm-rangers publisher, using the VSTS Developer Tools Build Tasks extension.  The published private extension is shared with our BETA sandbox-one and used for exploratory and user acceptance testing. The extension ID is suffixed with BETA to make it simple to identify this extension.

---

PROD

The environment with one task is triggered after a successful deployment to the BETA environment. The PROD approvals mandate that all of the specified users approve the release, to validate that the aka.ms/vsarDoD (definition of done) is met. Users typically include the project lead, program manager, and product owner.

#	BUILD STEP	BUILD SNIPPET	WHAT’S HAPPENING?
1	Publish Extension		We publish the extension to the ms-devlabs publisher, using the VSTS Developer Tools Build Tasks extension.   The published extension is public and available on the marketplace.

---

That’s it!

Now that you have an understanding of how our pipeline works, you should explore how you can configure your continuous integration and delivery pipeline by #RubDevOpsOnIt.

Other pipelines examples

Reference Information

DevOps - Applying DevOps to a Software Development Project, lcov-to-cobertura-xml, Set up a CI/CD pipeline for your Team Services extension, Testing a Team Services extension and tracking code coverage, VSTS Developer Tools Build Tasks

Comments

• Anonymous
  May 08, 2018
  Are you aware of any working tutorials or examples of how to unit test a vss-sdk web extension that uses amd module and webpack and typescript? I've spent days trying various tutorials and I always end up hitting a wall with errors such as 'define is not defined'.