Credentials

Applies to: check marked yes Databricks SQL check marked yes Databricks Runtime check marked yes Unity Catalog only

Unity Catalog and the built-in Azure Databricks Hive metastore use default locations for managed tables. Unity Catalog introduces several new securable objects to grant privileges to external cloud services and data in cloud object storage.

Credential

A credential is a securable object representing an Azure managed identity or Microsoft Entra ID service principal.

After a credential is created, access to it can be granted to principals (users and groups).

Storage credentials are primarily used to create external locations, which scope access to a specific storage path.

Credential names are unqualified and must be unique within the metastore.

Graphical Representation of relationships

The following diagram describes the relationship between:

  • storage credentials
  • external locations
  • external tables
  • storage paths
  • IAM entities
  • Azure service accounts

External location ER diagram

Examples

Using CLI create a storage credential my_azure_storage_cred for a Microsoft Entra ID service principal.

databricks storage-credentials create --json '{"name": "my_azure_storage_cred", "azure_service_principal": {"directory_id": "12345678-9abc-def0-1234-56789abcdef0", "application_id": "23456789-9abc-def0-1234-56789abcdef0", "client_secret": "Cli3nt5ecr3t"}}'

The rest of the commands can be run within SQL.

-- Grant access to the storage credential
> GRANT READ FILES ON STORAGE CREDENTIAL my_azure_storage_cred TO ceo;

-- ceo can directly read from any storage path using myazure_storage_cred
> SELECT count(1) FROM `delta`.`abfss://container@storageaccount.dfs.core.windows.net/depts/finance/forecast/somefile` WITH (CREDENTIAL my_azure_storage_cred);
  100
> SELECT count(1) FROM `delta`.`abfss://container@storageaccount.dfs.core.windows.net/depts/hr/employees` WITH (CREDENTIAL my_azure_storage_cred);
  2017

-- Create an external location on specific path to which `my_azure_storage_cred` has access
> CREATE EXTERNAL LOCATION finance_loc URL 'abfss://container@storageaccount.dfs.core.windows.net/depts/finance'
    WITH (CREDENTIAL my_azure_storage_cred)
    COMMENT 'finance';