Hello Mitasha Dutta (LTIMINDTREE LIMITED),
Welcome to the Microsoft Q&A and thank you for posting your questions here.
Regarding your question. Yes, a Databricks workspace can use an access connector to retrieve secrets from Azure Key Vault, but there are specific permissions and configurations required to avoid errors like the one you encountered.
About the error message you received, "Invalid permission on specified Key Vault, and status code 403," indicates that the access connector does not have the necessary permissions to access the secrets in the Key Vault. To resolve the issue of accessing secrets from Azure Key Vault using a Databricks workspace, ensure the access connector object ID has "Get" and "List" permissions in the Key Vault's access policies. Verify that the access connector is assigned the "Key Vault Secrets User" role in Azure. Check network settings to confirm that the Databricks workspace can communicate with the Key Vault, including any firewall or virtual network configurations. If using a managed identity, ensure it is properly configured with the necessary permissions. Finally, double-check the Databricks configuration to ensure the Key Vault URL and other parameters are correctly specified.
For more detailed guidance, you can refer to the Azure Databricks documentation on secret management - https://learn.microsoft.com/en-us/azure/databricks/security/secrets and the Azure Key Vault documentation as listed in the additional resource.
If you continue to face issues, it will be helpful to review the specific error message and logs to identify any additional details that could point to the root cause
- https://community.databricks.com/t5/data-engineering/azure-key-vault-keys-client-library-for-python-keys-list/td-p/5115
- https://community.databricks.com/t5/data-engineering/can-we-use-quot-access-connector-for-azure-databricks-quot-to/td-p/18356 and
- https://learn.microsoft.com/en-us/azure/databricks/security/secrets
I hope this is helpful! Do not hesitate to let me know if you have any other questions.
Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.