Vnet Injected Databricks Workspace with IP ACL gives 403 while executing notebook using compute cluster

Question

I have an IP access list enabled on my Vnet injected databricks workspace in Azure.

I can access my workspace as my VPN ip is enabled on the workspace IP list. However, when I try executing my notebook by attaching a cluster, it gives below error-

403: Source IP address: 4.221.228.47 is blocked by Databricks IP ACL for workspace: 56981763330245

I think my clusters are not getting the IP's from vnet, but the public ip's.

These are my current networking settings-

I thought the first option here should be enabled, so when I try to change the first option to 'Enabled' I keep getting this error-

Answer 1

Hi @Pratima Patil

Thank you for your question.

Based on the error and details you provided, it seems that your Databricks cluster is using public IPs rather than private IPs from the VNet, which causes the IP Access List (IP ACL) to block the traffic. Here’s how you can address this issue:

Cluster Networking and Private IP Configuration:

In a VNet-injected Databricks workspace, clusters should ideally use private IPs from the VNet. If clusters are using public IPs, it could mean that:

• The No Public IP for Clusters setting is not enabled.
• There might be an issue with the VNet subnet configuration.

Error When Enabling "No Public IP for Clusters":

The error Networking property update failed: Unexpected HTTP status code '412' indicates a precondition failure. This generally occurs when:

• Active clusters or jobs are running in the workspace. Stop all clusters and ensure no jobs are running before making the change.
• There are missing permissions or dependencies related to the VNet or subnets, such as inadequate IP range or misconfigured Network Security Groups (NSGs).

IP ACL Configuration:

Ensure that the private IP range of your VNet subnets is included in the IP Access List. If the range is missing, even private IPs assigned to the clusters will be blocked.

Steps to Resolve

1. Stop any running clusters in the workspace before attempting to enable the "No Public IP for Clusters" option.
2. Ensure that the subnets associated with your workspace have enough private IPs available.
3. Confirm that NSG rules allow communication between the workspace and clusters.
4. After stopping all clusters, try enabling the "No Public IP for Clusters" option again.
5. Ensure that the IP Access List includes the private IP range of your VNet subnets to prevent blocking traffic from clusters.

Reference: https://learn.microsoft.com/en-us/azure/databricks/security/network/classic/secure-cluster-connectivity

I hope this helps. Please let me know if you have any questions.