ADF failing CopyData

Question

I have a copy data action that reads from kusto db with a query and has a cosmos db sink. I have validated that the cosmos and kusto are successfully connected, yet while debugging the pipeline I run into this error:

Could this be rbac related? I have given the adf factory read and write permissions to the cosmos db

Answer 1

Hi Saad Khan ,

It sounds like you're encountering an issue with the Role-Based Access Control (RBAC) settings while trying to copy data from Kusto DB to Cosmos DB using Azure Data Factory (ADF).

Kindly checkout similar issue faced by other customer here: Azure Data Factory deployed via ARM Template

Here are some steps and considerations that might help you troubleshoot and resolve this issue:

1. Verify RBAC Permissions: Ensure that the ADF has the necessary RBAC permissions to access both the Kusto DB and Cosmos DB. You mentioned that you have given read and write permissions to the Cosmos DB, but it's also important to verify that the ADF has the appropriate permissions on the Kusto DB.
2. Check Managed Identity: If you are using a managed identity for ADF, make sure that the managed identity has been granted the necessary permissions on both the Kusto DB and Cosmos DB. This includes roles such as Reader and Contributor.
3. Network Configuration: Ensure that there are no network restrictions preventing ADF from accessing the Kusto DB and Cosmos DB. This includes checking any firewall rules, virtual network (VNet) configurations, and private endpoints.
4. Diagnostics and Logs: Check the diagnostics and logs for both the ADF pipeline and the Kusto DB/Cosmos DB to identify any specific errors or issues that might be causing the problem. This can provide insights into what might be going wrong.

In case you are still facing issue, kindly share configuration details of copy activity source and sink settings.

Hope it helps. Please let us know how it goes. Thankyou