Share via

Configuration profiles for the Databricks CLI

  • Article

Note

This information applies to Databricks CLI versions 0.205 and above. The Databricks CLI is in Public Preview.

Databricks CLI use is subject to the Databricks License and Databricks Privacy Notice, including any Usage Data provisions.

This article describes how to set up authentication between the Databricks CLI and your Azure Databricks accounts and workspaces. See What is the Databricks CLI?.

This article assumes that you have already installed the Databricks CLI and set up the CLI for authentication. See:

Get information about configuration profiles

Adding multiple configuration profiles to the .databrickscfg file enables you to quickly run commands across various workspaces by specifying the target configuration profile’s name in the command’s --profile or -p option, for those commands that support this option. If you do not specify the --profile or -p option in a command that supports this option, the command will use the DEFAULT configuration profile by default.

Tip

You can press Tab after --profile or -p to display a list of existing available configuration profiles to choose from, instead of entering the configuration profile name manually.

For example, you could have a configuration profile named DEV that references an Azure Databricks workspace that you use for development workloads and a separate configuration profile named PROD that references a different Azure Databricks workspace that you use for production workloads.

By default, the Databricks CLI looks for the .databrickscfg file in your ~ (your user home) folder on Unix, Linux, or macOS, or your %USERPROFILE% (your user home) folder on Windows. You can change the default path of the .databrickscfg file by setting the environment variable DATABRICKS_CONFIG_FILE. To learn how to set environment variables, see your operating system’s documentation.

To get information about an existing configuration profile, run the auth env command:

databricks auth env --profile <configuration-profile-name>

# Or:
databricks auth env --host <account-console-url>

# Or:
databricks auth env --host <workspace-url>

For example, here is the output for a profile that is configured with Azure Databricks personal access token authentication:

{
  "env": {
    "DATABRICKS_AUTH_TYPE": "pat",
    "DATABRICKS_CONFIG_PROFILE": "<configuration-profile-name>",
    "DATABRICKS_HOST": "<workspace-url>",
    "DATABRICKS_TOKEN": "<token-value>"
  }
}

To get information about all available profiles, run the auth profiles command:

databricks auth profiles

Output (the ellipses represent omitted content, for brevity):

{
  "profiles": [
    {
      "name": "<configuration-profile-name>",
      "host": "<workspace-url>",
      "cloud": "<cloud-id>",
      "auth_type": "<auth-type>",
      "valid": true
    },
    {
      "...": "..."
    }
  ]
}

The output of the auth profiles command does not display any access tokens. To display an access token, run the preceding auth env command.

Important

The Databricks CLI does not work with a .netrc file. You can have a .netrc file in your environment for other purposes, but the Databricks CLI will not use that .netrc file.

Test your DEFAULT configuration profile setup

To check whether you set up authentication correctly, you can run a command such as the following, which lists the available Databricks Runtime versions for the Azure Databricks workspace that is associated with your DEFAULT profile.

The following call assumes that you do not have any special environment variables set, which take precedence over the settings in your DEFAULT profile. For more information, see Authentication order of evaluation.

databricks clusters spark-versions

Test your configuration profiles

To check whether you set up any configuration profiles correctly, you can run a command such as the following with one of your workspace-level configuration profile names. This command lists the available Databricks Runtime versions for the Azure Databricks workspace that is associated with the specified configuration profile, represented here by the placeholder <configuration-profile-name>:

databricks clusters spark-versions -p <configuration-profile-name>

Tip

You can press Tab after --profile or -p to display a list of existing available configuration profiles to choose from, instead of entering the configuration profile name manually.

To list details for a specific profile, run the following command:

databricks auth env --profile <configuration-profile-name>

To list details for all of your available profiles, run the following command:

databricks auth profiles