How to connect from Azure Databricks to an prem MS SQL server

Question

I have an Azure Databricks instance deployed on a Virtual Network.

On the same (!) Virtual Network, I also have a Azure VM deployed, that is successfully able to connect to an on-prem MS SQL server. However, when I try to connect via the Azure Databricks instance to the MS SQL server, it gives a connection time-out error.

I have already added the Databricks private subnet IP range to the tunnel. Also, I allow outbound Databricks Traffic to the SQL Server in the NSG.

What could be the issue here? Why am I able to connect via the VM to the MS SQL server, but not via the Databricks instance?

Answer 1

@Nathalie

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.

Issue:

I have an Azure Databricks instance deployed on a Virtual Network.

On the same (!) Virtual Network, I also have a Azure VM deployed, that is successfully able to connect to an on-prem MS SQL server. However, when I try to connect via the Azure Databricks instance to the MS SQL server, it gives a connection time-out error.

I have already added the Databricks private subnet IP range to the tunnel. Also, I allow outbound Databricks Traffic to the SQL Server in the NSG.

What could be the issue here? Why am I able to connect via the VM to the MS SQL server, but not via the Databricks instance?

Solution:

To resolve the Databricks connection issue! Verify that the on-premises firewall allows incoming connections from the Databricks subnet IP range. Your explanation highlights a crucial point about how Azure Databricks networking works, especially when connecting to on-premises resources. It's valuable information for other users who might be struggling with similar Databricks-to-on-premises connectivity problems.

when configuring your on-premises firewall to allow connections from your Azure Databricks workspace, you need to use the public IP range of your Databricks workspace, not the private IP range of the subnets within your VNet.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

Hope this helps. Do let us know if you have any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Answer 2

Thanks for posting your question in the Microsoft Q&A forum.

You need to ensure proper network connectivity and configuration.

• Ensure that the Databricks VNet is properly peered with the transit VNet containing the Virtual Network Gateway. Enable Use Remote Gateways on the Databricks VNet side and Allow Gateway Transit on the transit VNet side
• Add UDRs to the Databricks subnet to route traffic destined for your on-premises network through the Virtual Network Gateway
• Ensure that the Databricks workspace can resolve the on-premises SQL Server hostname. You may need to configure custom DNS settings or use IP addresses
• Verify that the on-premises firewall allows incoming connections from the Databricks subnet IP range
• Ensure that the SQL Server is configured to accept remote connections and that the appropriate ports (usually 1433) are open
• When connecting from Databricks, use the appropriate JDBC connection string with the correct server name, port, and authentication details
• Review the NSGs associated with the Databricks subnet to ensure they allow outbound traffic to the SQL Server

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful