Del via

Install Databricks Connect for Scala

  • Artikel

Note

This article covers Databricks Connect for Databricks Runtime 13.3 LTS and above.

This article describes how to install Databricks Connect for Scala. See What is Databricks Connect?. For the Python version of this article, see Install Databricks Connect for Python.

Requirements

  • Your target Azure Databricks workspace and cluster must meet the requirements for Compute configuration for Databricks Connect.

  • You have the Java Development Kit (JDK) installed on your development machine. Databricks recommends that the version of your JDK installation matches the JDK version on your Azure Databricks cluster. To find the JDK version of the Databricks Runtime on your cluster, refer to the System environment section of the Databricks Runtime release notes or the version compatibility table.

    Note

    Choosing a JDK install that is below or above the JDK version on your cluster might produce unexpected results, or your code might not run at all.

  • You have Scala installed on your development machine. Databricks recommends that the version of your Scala installation matches the Scala version on your Azure Databricks cluster. To find the Scala version of the Databricks Runtime version of your cluster, refer to the System environment section of the Databricks Runtime release notes or the version compatibility table.

  • If you are using user-defined functions (UDFs), the local Scala and Java versions must match the Scala and Java versions of the Databricks Runtime version of the cluster. To find the Scala and Java versions of the Databricks Runtime version of your cluster, refer to the System environment section of the Databricks Runtime release notes or the version compatibility table.

  • A Scala build tool on your development machine, such as sbt.

Version support matrix

The following table shows compatible Databricks Connect, JDK, and Scala versions. Databricks Connect version numbers correspond to Databricks Runtime version numbers.

Compute type Databricks Connect version JDK version Scala version
Cluster 16.0 and above

16.0 ML and above		 JDK 17 2.12.18
Cluster 15.4 LTS

15.4 ML LTS		 JDK 8 2.12.18
Cluster 13.3 LTS to 14.3 LTS

13.3 ML LTS to 14.3 ML LTS		 JDK 8 2.12.15

Add a reference to the Databricks Connect client

To set up the Databricks Connect client, first add a reference to the client. In your Scala project’s build file such as build.sbt for sbt, pom.xml for Maven, or build.gradle for Gradle, add the following reference to the Databricks Connect client. Replace 14.0.0 with the version of the Databricks Connect library that matches the Databricks Runtime version on your cluster. You can find the Databricks Connect library version numbers in the Maven central repository.

Sbt

libraryDependencies += "com.databricks" % "databricks-connect" % "14.0.0"

Maven

<dependency>
  <groupId>com.databricks</groupId>
  <artifactId>databricks-connect</artifactId>
  <version>14.0.0</version>
</dependency>

Gradle

implementation 'com.databricks.databricks-connect:14.0.0'

Configure connection properties

Next, configure properties to establish a connection between Databricks Connect and your remote Azure Databricks cluster. These properties include settings to authenticate Databricks Connect with your cluster. See Compute configuration for Databricks Connect.

For Databricks Connect for Databricks Runtime 13.3 LTS and above, for Scala, Databricks Connect includes the Databricks SDK for Java. This SDK implements the Databricks client unified authentication standard, a consolidated and consistent architectural and programmatic approach to authentication. This approach makes setting up and automating authentication with Azure Databricks more centralized and predictable. It enables you to configure Azure Databricks authentication once and then use that configuration across multiple Azure Databricks tools and SDKs without further authentication configuration changes.

Note