whitelist the serverless data plane subnets in the cloud region

Question

I am following below instructions from the documentation to whitelist the serverless data plane subnets in the cloud region of your Databricks workspace. But unable to find ARM resource ID of the serverless compute subnet details

 

https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewall

I was able to set it with subnet below. But now it's throwing error

subnet "/subscriptions/8453a5d5-9e9e-40c7-87a4-0ab4cc197f48/resourceGroups/prod-azure-westeurope-nephos3/providers/Microsoft.Network/virtualNetworks/kaas-vnet/subnets/worker-subnet"

Could you please help me to find/get it.

Answer 1

To whitelist the serverless data plane subnets for your Databricks workspace, you'll need to follow these steps to get the ARM resource ID of the serverless compute subnet:

1. Create a Network Connectivity Configuration (NCC):
   • Go to the Azure Databricks account console.
   • Navigate to Cloud Resources and click on Network Connectivity Configuration.
   • Click on Add Network Connectivity Configurations, provide a name, choose the region that matches your workspace, and add the NCC.
2. Copy Subnet IDs:
   • Once the NCC is created, go to Default Rules under Network identities and click on View all.
   • Click the Copy subnets button to copy the ARM resource IDs of the serverless compute subnets.
3. Attach NCC to Workspaces:
   • Go to the Workspaces section in the account console.
   • Select your workspace and click Update workspace.
   • In the Network Connectivity Config field, select your NCC. Ensure the regions match between the workspace and the NCC.
4. Update Firewall Rules:
   • In the Azure portal, navigate to your storage account.
   • Go to Networking and under Public network access, select Enabled from selected virtual networks and IP addresses.
   • Add network rules using the copied subnet IDs to allow the serverless compute subnets to access your storage account.

For more detailed instructions, refer to the official documentation on configuring firewall for serverless compute access in Azure Databricks:

https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewall