Connecting Databricks through Microsoft Purview by enabling "No Public Access"

Question

Hi,

  I was able to connect Databricks through Microsoft Purview using Public Network but now the requirement is to connect Databricks through Microsoft Purview by enabling "No Public Access" in Databricks. The Databricks account is created with appropriate configuration and when I try scanning that Databricks using Microsoft Purview , I am getting error like " Error: (3805) Connector Exception: Connection to the database (with URL jdbc:databricks://adb-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) failed: Can't connect to 'jdbc:databricks://adb-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'".


                   Also tried adding the IP Address of the system where Self-Hosted Integration Runtime is configured. But At present I have learnt that Private links cannot be created for Databricks and even I tried to add that IP Address , I am getting error as "No Supported Sub-Resources".


                   So could you please let me know is there any other action to be done for connecting Databricks through Microsoft Purview by enabling "No Public Access".

Note : 1)I have used 'x' in the error to mask the databricks account details.

Answer 1

Hello @Paarveandan Y K

Have you tried this :

Copy

Create a subnet for Databricks in your VNet.

Create a Private Link Service for Databricks. This can be done by going to the Databricks workspace and selecting "Networking" and then "Private Endpoints." From there, you can create a new Private Link Service.

Configure the Private Link Service for Databricks. This will involve specifying the VNet and subnet you created in step 1.

Create a Private Endpoint for Databricks. This can be done by going to the Azure portal and selecting "Private Endpoints." From there, you can create a new Private Endpoint for Databricks.

Configure your firewall and network security groups to allow traffic to flow between your VNet and Databricks.

Once you have set up a Private Endpoint for Databricks, you can use the private IP address to connect to Databricks through Microsoft Purview. You can do this by creating a connection in Purview and specifying the private IP address and the appropriate authentication credentials.

ALSO , Please read this : https://learn.microsoft.com/en-us/azure/databricks/administration-guide/cloud-configurations/azure/private-link

In case you find this helpful kindly mark the answer as Accepted or inform us accordingly to proceed with additional troubleshooting!

BR