Frequently asked questions | Azure

No. The extension isn't available for downloading and installation for Azure DevOps Server (formerly Visual Studio Team Foundation Server).

Maybe. It depends on the type of analysis tool. The source code might be the only thing that's required, or the build output might be required.

For example, Credential Scanner (CredScan) analyzes files within the folder structure of the code repository. Because of this analysis, you can run the CredScan and Publish Security Analysis Logs build tasks in a standalone build to get results.

For other tools like BinSkim that analyze post-build artifacts, the build is required first.

Yes. You can introduce a build break when any tool reports an issue or problem in its log file. Add the Post-Analysis build task, and select the checkbox for any tool for which you want to break the build.

In the UI of the Post-Analysis task, you can choose to break the build when any tool reports either errors only or both errors and warnings.

Usually, the Azure DevOps build tasks are direct wrappers around the command-line arguments of the security tools. You can pass as arguments to a build task anything you normally pass to a command-line tool.

Noticeable differences:

• Tools run from the source folder of the agent $(Build.SourcesDirectory) or from %BUILD_SOURCESDIRECTORY%. An example is C:\agent_work\1\s.
• Paths in the arguments can be relative to the root of the source directory previously listed. Paths can also be absolute. You get absolute paths either by using Azure DevOps Build Variables or by running an on-premises agent with known deployment locations of local resources.
• Tools automatically provide an output file path or folder. If you provide an output location for a build task, that location is replaced with a path to our well-known location of logs on the build agent
• Some other command-line arguments are changed for some tools. One example is the addition or removal of options that ensure no GUI is launched.

No. Running the secure development tools across multiple repositories in a single pipeline isn't supported.

The build tasks filter some user input. For this question specifically, they update the location of the generated output file to be a common location on the build agent. For more information on this location, see the following questions.

The build tasks automatically add output paths to this well-known location on the build agent: $(Agent.BuildDirectory)_sdt\logs. Because we standardize on this location, all teams that produce or consume code-analysis logs have access to the output.

Yes. All tasks and tools in the extension can be executed on a hosted build agent.

In most cases, yes.

However, Azure DevOps doesn't support running tasks within release pipelines when those tasks publish artifacts. This lack of support prevents the Publish Security Analysis Logs task from running successfully in a release pipeline. The task instead fails with a descriptive error message.

Build tasks can download the tools' NuGet packages from the Azure DevOps Package Management feed. Build tasks can also use Node Package Manager, which must be preinstalled on the build agent. An example of such installation is the command npm install tslint.

Upon their installation, the security build tasks provided by the extension becomes available to all users in your organization. When you create or edit an Azure Pipeline, these tasks are available from the build-task collection list. Otherwise, installing the extension in your Azure DevOps organization has no effect. The installation doesn't modify any account settings, project settings, or pipelines.

No. Installing the extension makes the security build tasks available for addition to your pipelines. You're still required to add or update build definitions, so that the tools can work with your build process.

Task-specific FAQ

Questions specific to build tasks are listed in this section.

Here are details of two of the most common suppression scenarios.

To suppress all occurrences of a given secret within the specified path

The hash key of the secret from the CredScan output file is required as shown in the following sample.

{
    "tool": "Credential Scanner",
    "suppressions": [
    {
        "hash": "CLgYxl2FcQE8XZgha9/UbKLTkJkUh3Vakkxh2CAdhtY=",
        "_justification": "Secret used by MSDN sample, it is fake."
    }
  ]
}

To suppress all secrets in a specified file or to suppress the secrets file itself

The file expression can be a file name. It can also be the basename part of a full file path or a file name. Wildcards are not supported.

The following examples show how to suppress the file <InputPath>\src\JS\lib\angular.js

Examples of valid suppression rules:

• <InputPath>\src\JS\lib\angular.js - suppresses the file in the specified path
• \src\JS\lib\angular.js
• \JS\lib\angular.js
• \lib\angular.js
• angular.js - suppresses any file with the same name

{
    "tool": "Credential Scanner",
    "suppressions": [
    {
        "file": "\\files\\AdditonalSearcher.xml", 
        "_justification": "Additional CredScan searcher specific to my team"
    },
    {
        "file": "\\files\\unittest.pfx", 
        "_justification": "Legitimate UT certificate file with private key"
    }
  ]
}

The following resources help you securely manage secrets and access sensitive information from within your applications:

• Azure Key Vault
• Azure Active Directory (Azure AD)
• Azure AD Managed Service Identity (MSI)
• Managed identities for Azure resources
• Managed identities in Azure App Service and Azure Functions
• AppAuthentication library

For more information, see the blog post Managing Secrets Securely in the Cloud.

Credential Scanner relies on a set of content searchers that are commonly defined in the buildsearchers.xml file. The file contains an array of XML serialized objects that represent a ContentSearcher object. The program is distributed with a set of well-tested searchers. But you can implement your own custom searchers too.

A content searcher is defined as follows:

• Name: The descriptive searcher name to be used in Credential Scanner output files. We recommended you use the camel-case naming convention for searcher names.

• RuleId: The stable opaque ID of the searcher:

  • A Credential Scanner default searcher is assigned a RuleId value like CSCAN0010, CSCAN0020, or CSCAN0030. The last digit is reserved for potentially merging or dividing searcher groups via regular expressions (regex).
  • The RuleId value for a customized searcher should have its own namespace. Examples include CSCAN-<Namespace>0010, CSCAN-<Namespace>0020, and CSCAN-<Namespace>0030.
  • A fully qualified searcher name is the combination of a RuleId value and a searcher name. Examples include CSCAN0010.KeyStoreFiles and CSCAN0020.Base64EncodedCertificate.

• ResourceMatchPattern: Regex of file extensions to check against the searcher.

• ContentSearchPatterns: An array of strings containing regex statements to match. If no search patterns are defined, all files matching the ResourceMatchPattern value are returned.

• ContentSearchFilters: An array of strings containing regex statements to filter searcher-specific false positives.

• MatchDetails: A descriptive message, mitigation instructions, or both to be added for each match of the searcher.

• Recommendation: The suggestions-field content for a match using the PREfast report format.

• Severity: An integer that reflects the severity level of an issue. The highest severity level has the value 1.

  

The project was restored using a wrong Microsoft.NETCore.App version

The full error message:

"Error: The project was restored using Microsoft.NETCore.App version x.x.x, but with current settings, version y.y.y would be used instead. To resolve this issue, make sure the same settings are used for restore and for subsequent operations such as build or publish. Typically this issue can occur if the RuntimeIdentifier property is set during build or publish but not during restore."

Because Roslyn Analyzers tasks run as part of compilation, the source tree on the build machine must be in a buildable state.

A step between your main build and Roslyn Analyzers steps might have put the source tree into a state that prevents building. This extra step is probably dotnet.exe publish. Try duplicating the step that does a NuGet restoration just before the Roslyn Analyzers step. This duplicated step might put the source tree back in a buildable state.

csc.exe can't create an analyzer instance

The full error message:

"'csc.exe' exited with error code 1 -- An instance of analyzer AAAA cannot be created from C:\BBBB.dll : Could not load file or assembly 'Microsoft.CodeAnalysis, Version=X.X.X.X, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified."

Ensure your compiler supports Roslyn Analyzers. Running the command csc.exe /version should report a version value of 2.6 or later.

Sometimes a .csproj file can override the build machine's Visual Studio installation by referencing a package from Microsoft.Net.Compilers. If you don't intend to use a specific version of the compiler, remove references to Microsoft.Net.Compilers. Otherwise, make sure that the version of the referenced package is also 2.6 or later.

Try to get the error-log path, which is specified in the csc.exe /errorlog option. The option and path appear in the log for the Roslyn Analyzers build task. They might look something like /errorlog:F:\ts-services-123_work\456\s\Some\Project\Code\Code.csproj.sarif

The C# compiler version isn't recent enough

To get the latest versions of the C# compiler, go to Microsoft.Net.Compilers. To get your installed version, run csc.exe /version at a command prompt. Ensure that you reference a Microsoft.Net.Compilers NuGet package that is version 2.6 or later.

MSBuild and VSBuild logs aren't found

The Roslyn Analyzers build task must query Azure DevOps for the MSBuild log from the MSBuild build task. If the analyzer task runs immediately after the MSBuild task, the log won't yet be available. Place other tasks between the MSBuild task and the Roslyn Analyzers task. Examples of other tasks include BinSkim and Anti-Malware Scanner.

Next steps

If you need additional assistance, Microsoft Security Code Analysis Support is available Monday to Friday from 9:00 AM to 5:00 PM Pacific Standard Time.

• Onboarding: Refer to our Onboarding documentation

• Support: Email our team at Microsoft Security Code Analysis Support