Using Azure Databricks Clusters with Unity Catalog and Managed Identity Support.

Question

We are leveraging Azure Databricks for our Data Engineering project and intend to utilize Databricks clusters that support both Unity Catalog and Managed Identity. However, according to the Databricks documentation (https://docs.databricks.com/en/compute/access-mode-limitations.html#create-clusters--sql-warehouses-with-unity-catalog-access), it is noted that shared clusters support Unity Catalog, whereas clusters with no-isolation do not.

1. What type of clusters should we use to ensure compatibility with both Unity Catalog and Managed Identity?
2. Is this compatibility truly supported? If so, what should be the specific configuration of these clusters?"
3. We need to understand possible solutions, so that we can plan our prod deployments

Answer 1

Hello @Hemanth Kumar

Thanks for the question and using MS Q&A platform.

According to this documentation: Access modes

To ensure compatibility with both Unity Catalog and Managed Identity in Azure Databricks, you should use shared clusters. These clusters support Unity Catalog, whereas clusters with no isolation do not.

Please check the below image for reference:

Refer to the below image for the specific cluster configurations:

For more details refer to these MS documentations: Use Azure managed identities in Unity Catalog to access storage

Access modes

Hope this helps. Do let us know if you have any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.