How to set up capture to AzureStorage from EventHub Avro type files

Question

Hi, I am trying to set up Avro files capture for EventHub to be stored on Azure Storage. I have a container, granted access to it as a Storage Contributor to aforementioned EventHub. When saving, I am getting all the time

Not sure if anything is wrong or there is an issue in Azure.

Answer 1

Hello Mirko,

Based on the error you need to add the data plane access to Azure Storage resources. For accessing Blobs only contributor access isn't enough so you need specific Azure RBAC roles:

• Storage Blob Data Contributor: Grants full access to blob data (read, write, delete).

Steps to Resolve

• Assign the appropriate role via Azure Portal:
• Go to Storage Account > Access Control (IAM) > Add Role Assignment.
  • Assign the relevant role (e.g., Storage Blob Data Contributor) to your user.
  • Alternatively, assign via Azure CLI:BashCopy

        az role assignment create \
        --role "Storage Blob Data Contributor" \
        --assignee <YourUserPrincipalName> \
        --scope /subscriptions/<SubscriptionId>/resourceGroups/<ResourceGroup>/providers/Microsoft.Storage/storageAccounts/<StorageAccountName>
  

Reauthenticate in Azure Storage Explorer after role assignment to ensure access.

References

• https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles/storage
• https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/control-plane-and-data-plane

If this resolves your issue, please accept the answer.

Luis

Answer 2

@Mirko, Welcome to Microsoft Q&A Forum, Thank you for posting your query!

Adding more information to the above response!

Based on the error message: Please refer to the suggestion mentioned in this GitHub thread

To resolve the issue, you need to grant the user assigned managed identity with Storage Blob Data Owner permissions on the storage account. You can do this by adding a role assignment for the managed identity on the storage account with the required permissions. Specifically, you need to add the Microsoft.Storage/storageAccounts/blobServices/containers/write and Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write permissions to the role assignment.

For more detials, refer to Capture events through Azure Event Hubs in Azure Blob Storage or Azure Data Lake Storage - Azure Storage account as a destination

This article lists some examples of role assignment conditions for controlling access to Azure Blob Storage. Example Azure role assignment conditions for Blob Storage
Security considerations for Azure role assignment conditions in Azure Blob Storage
Azure Event Hubs enables you to automatically capture the streaming data in Event Hubs in an Azure Blob storage or Azure Data Lake Storage Gen 1 or Gen 2 account of your choice. Setting up Capture is fast, there are no administrative costs to run it, and it scales automatically with Event Hubs throughput units. The files produced by Event Hubs Capture have the Avro schema. You can process these Avro files with Azure Data Lake Analytics using U-SQL

If you encounter issues, ensure that the destination storage account is in the same subscription as the event hub when not using managed identity for authentication. Event Hubs doesn't support capturing events in a premium storage account