Our journey to WhiteSource and OSS security, licensing and quality checking
This is the third, and final, article in the WhiteSource series for managing open source usage and security in your CI/CD pipeline. If you haven't viewed the previous articles, I recommend you take a moment to review them now:
- Manage your open source usage and security in your pipeline
- Manage your open source usage and security as reported by your CI/CD pipeline
In the last article we discussed some of the common challenges that we were faced with when integrating WhiteSource scans with some of the ALM / DevOps Rangers projects. These challenges included dealing with myriad types of open source (OSS) licenses and npm "craziness". Since we last checked in we have made some pretty good strides in getting a grip on these issues. This article takes a look at what's been implemented to alleviate some of these issues.
OSS Licenses
The primary issue we were attempting to mitigate with OSS licenses was the plethora of copyleft licenses utilized by various OSS projects. As mentioned in the previous article, some OSS licenses are worded such that you might have to make your source code public under certain circumstances. If there is intellectual property embedded within the source code and/or you rely on the source code to generate revenue then you need to be careful about which OSS licenses you work with.
To mitigate this issue, we have created organizational policies within our WhiteSource account to reject certain groups of license types. When a project is scanned that contains one of these forbidden licenses the scan will fail. While this does not prevent the license from being used in the first place it does allow us to get timely feedback, via continuous integration (CI) builds, so we can address the situation early - before the product is released to the public.
The following screenshot shows a few policy groups that have been setup at the organization level within WhiteSource. Notice the last two have the Action set to Reject. You can ignore the first policy; This was a test policy I was working on a while back.
npm Craziness
In our last article I mentioned the new NPM plugin that WhiteSource had just released. Since then we have been able to integrate this plugin with several of our builds. The primary advantage of this plugin over the previous TFS plugin we were utilizing is that the new NPM plugin will scan only the dependencies directly referenced within your project's package.json file. Unlike the NPM plugin, the TFS plugin (that we were using previously) scans files based on a list of file extensions. If you configure it to scan .js files then it's very likely it will end up scanning thousands of files once the npm build has completed.
We also talked about separating npm dependencies between devDependencies and (production) dependencies. This is very useful because you can configure the new NPM plugin to ignore devDependencies. Refer back to the previous article for details on how to separate the dependencies.
Using the NPM Plugin
To help each team work through the process of integrating the new NPM plugin into their build processes, we created a step-by-step guide.
STEP 1:
Within your project, add a whitesource.config.json file at the same level as the package.json file. The contents of this file should be as follows:
{
"apiKey": "__WhiteSourceAPIKey__",
"checkPolicies": true,
"productName": "__ProductName__",
"productVer": "__ProductVersion__",
"projectName": "__ProjectName__",
"devDep": false
}
The items surrounded by double underscores (__) are placeholders that will be replaced with the values shown in step 2 below using the Replace Tokens task referenced in step 4.
STEP 2:
Add WhiteSource Configuration Variables to the build definition as follows:
| VARIABLE NAME | VALUE |
| WhiteSourceAPIKey | <insert your WhiteSource API key [GUID] here> |
| ProductName | <insert your product name here> |
| ProductVersion | $(Build.BuildNumber) |
| ProjectName | $(Build.Repository.Name) |
STEP 3:
Just after the npm install task add another npm install task configured as follows:
| PARAMETER NAME | VALUE |
| Working Folder | <relative path to package.json file> |
| npm command | Install |
| Arguments | - g whitesource |
The following screenshot is from the VSTS/TFS Countdown Widget build definition:
STEP 4:
After the previous task, add a Replace Tokens* task if none exists, configured as follows:
| PARAMETER NAME | VALUE |
| Source Path | <path to folder containing the whitesource.config.json file> |
| Target File Pattern | whitesource.config.json |
NOTE – We’re making use of Colin's ALM Corner Custom Build Tasks to replace the tokens within the whitesource.config.json file at build time with values stored in build variables. You do not have to follow this approach if you don't want to. There are also other build tasks that can replace tokens within files if you prefer to make use of one of them.
STEP 5:
After the previous task, add a Command Line task configured as follows:
| PARAMETER NAME | VALUE |
| Tool | whitesource |
| Arguments | Run |
| Working Folder | <path to folder containing the whitesource.config.json file> |
IMPORTANT
While the above steps will get you going, there are a couple of issues to watch out for:
The name element in the package.json file must be provided - e.g. the following package.json file has left it empty:
If you do not provide a name in the package.json file you will see an error message along these lines in the run WhiteSource task (step 5 above):
Node package -Name- and -Version- must be specified in module Package.json file
The run whitesource task sometimes seems to fail the first time the build is run. I’m not sure yet if this is because the path isn’t being picked up or if it’s something else. However, it seems to work fine on subsequent builds.
Interesting views from our dashboard
Here’s a dashboard snapshots from our WhiteSource dashboard for two our our extensions:
At a glance:
- We’re secure.
- We have four alerts and need to notify the extension teams to consider new library versions.
- There are a couple of tasks we need to investigate, mainly license assignments.
- We’re primarily using components with the MIT and a few ISC license types.
- We’re waiting for WhiteSource to act on three of our license assignment requests.
Summary
With the changes mentioned above we have been able to reduce a substantial amount of noise within our scan results. Where we were seeing hundreds of libraries showing up in our scans prior to these changes it's now not uncommon to see only one or two dozen libraries showing up in the scan. This makes for a much better review process as we move more projects to utilizing WhiteSource.
This has been an interesting journey getting to this point. While the journey is not yet over - there are lots of projects to be scanned (!) - we feel like we now have a good process to follow that yields a manageable amount of information on which we can act appropriately.
Lastly, if you have any questions, please do not hesitate to start a conversation in the comments section below!