Configure enhanced security and compliance settings

Enhanced Security and Compliance is a platform add-on that provides enhanced security and controls for your compliance needs. See the pricing page. This article describes how to configure enhanced security and compliance settings on your Azure Databricks workspace. Your Azure Databricks workspace must be on the Premium plan.

Enable enhanced security and compliance features using the Azure portal

1. In the Azure portal, click the Security & compliance tab on an existing Azure Databricks workspace or on the Azure Databricks workspace creation page.

2. To enable the compliance security profile, select the checkbox next to Enable compliance security profile. In the dropdown, select one or more compliance standards or select None.

   

   If you enable the compliance security profile or add compliance standards, those selections are permanent for that workspace.

3. To enable enhanced security monitoring, select the checkbox Enable enhanced security monitoring.

4. To enable automatic cluster update, select the checkbox Enable automatic cluster update.

   To configure the maintenance window and its frequency, see Automatic cluster update

Enable enhanced security and compliance features using an ARM template

You can configure the Enhanced Security and Compliance add-on features with an ARM template that Databricks provides. It contains additional parameters that you can set to Enabled or Disabled. If you want to add them to an existing template to update the workspace, you can do so. You can set features independently except as indicated:

• complianceSecurityProfile: Enables the compliance security profile. Once enabled, this feature is permanently enabled on the workspace.
• complianceStandards: Configures an array of compliance standards to use with the compliance security profile.
  • If complianceSecurityProfile is set to Disabled, pass an empty array.
  • If complianceSecurityProfile is set to Enabled, you must pass an array of one or more strings that specify which (if any) compliance standards you want for your workspace. Possible selections are HIPAA, PCI_DSS, or NONE. Add the single array element NONE if you are using the compliance security profile only for its security benefits but not to process any regulated data.
• enhancedSecurityMonitoring — Enables enhanced security monitoring. If the compliance security profile is enabled, you must set this feature to Enabled explicitly in the template.
• automaticClusterUpdate — Enables automatic cluster update. If the compliance security profile is enabled, you must set this feature to Enabled explicitly in the template. To configure the maintenance window and its frequency, see Automatic cluster update.

To update a workspace with one or more of these features, follow the same instructions for deploying a custom template as you would for creating a new workspace with a template. However, check that you use your original template and then copy the fields from the provided example template into your existing workspace template.

Workspace template with enhanced security and compliance features

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "disablePublicIp": {
      "type": "bool",
      "defaultValue": false,
      "metadata": {
        "description": "Specifies whether to deploy Azure Databricks workspace with secure cluster connectivity (No Public IP) enabled."
      }
    },
    "workspaceName": {
      "type": "string",
      "metadata": {
        "description": "The name of the Azure Databricks workspace to create."
      }
    },
    "pricingTier": {
      "type": "string",
      "defaultValue": "premium",
      "allowedValues": [
        "standard",
        "premium"
      ],
      "metadata": {
        "description": "The pricing tier of workspace."
      }
    },
  "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    },
    "automaticClusterUpdate": {
      "type": "string",
      "defaultValue": "Disabled",
      "allowedValues": [
        "Disabled",
        "Enabled"
      ],
      "metadata": {
        "description": "Enable/Disable automatic cluster update"
      }
    },
  "enhancedSecurityMonitoring": {
      "type": "string",
      "defaultValue": "Disabled",
      "allowedValues": [
        "Disabled",
        "Enabled"
      ],
      "metadata": {
        "description": "Enable/Disable enhanced security monitoring"
      }
    },
  "complianceSecurityProfile": {
      "type": "string",
      "defaultValue": "Disabled",
      "allowedValues": [
        "Disabled",
        "Enabled"
      ],
      "metadata": {
        "description": "Enable/Disable the Compliance Security Profile"
      }
    },
  "complianceStandards": {
      "type": "array",
      "defaultValue": [],
      "allowedValues": [
        [],
        ["NONE"],
        ["HIPAA"],
        ["PCI_DSS"],
        ["HIPAA", "PCI_DSS"]
      ],
      "metadata": {
        "description": "Specify the desired compliance standards for your compliance security profile"
      }
    }
  },
  "variables": {
    "managedResourceGroupName": "[format('databricks-rg-{0}-{1}', parameters('workspaceName'), uniqueString(parameters('workspaceName'), resourceGroup().id))]",
    "trimmedMRGName": "[substring(variables('managedResourceGroupName'), 0, min(length(variables('managedResourceGroupName')), 90))]",
    "managedResourceGroupId": "[format('{0}/resourceGroups/{1}', subscription().id, variables('trimmedMRGName'))]"
  },
  "resources": [
    {
      "type": "Microsoft.Databricks/workspaces",
      "apiVersion": "2023-09-15-preview",
      "name": "[parameters('workspaceName')]",
      "location": "[parameters('location')]",
      "sku": {
        "name": "[parameters('pricingTier')]"
      },
      "properties": {
        "managedResourceGroupId": "[variables('managedResourceGroupId')]",
        "parameters": {
          "enableNoPublicIp": {
            "value": "[parameters('disablePublicIp')]"
          }
        },
        "enhancedSecurityCompliance": {
          "automaticClusterUpdate": {
            "value": "[parameters('automaticClusterUpdate')]"
          },
          "complianceSecurityProfile": {
            "value": "[parameters('complianceSecurityProfile')]",
            "complianceStandards": "[parameters('complianceStandards')]"
          },
          "enhancedSecurityMonitoring": {
            "value": "[parameters('enhancedSecurityMonitoring')]"
          }
        }
      }
    }
  ],
  "outputs": {
    "workspace": {
      "type": "object",
      "value": "[reference(resourceId('Microsoft.Databricks/workspaces', parameters('workspaceName')), '2023-09-15-preview', 'full')]"
    }
  }
}

Enable enhanced security and compliance features using Terraform

Enhanced security and compliance can also be enabled on a Azure Databricks workspace using the azurerm Terraform plugin for Databricks. For more information about the azurerm Terraform plugin, see azurerm_databricks_workspace.

For example, to create an Azure Databricks workspace with HIPAA and PCI-DSS compliance controls enabled, use the following:

resource "azurerm_databricks_workspace" "this" {
  name                        = "${local.prefix}-workspace"
  resource_group_name         = azurerm_resource_group.this.name
  location                    = azurerm_resource_group.this.location
  sku                         = "premium"
  managed_resource_group_name = "${local.prefix}-workspace-rg"
  tags                        = local.tags

  enhanced_security_compliance {
  automatic_cluster_update_enabled    = true
  compliance_security_profile_enabled   = true
  compliance_security_profile_standards = ["HIPAA", "PCI_DSS"]
  enhanced_security_monitoring_enabled  = true
  }
}