Data factory storage event trigger support rename event

Question

Is it possible to add storage event trigger to support blob rename?

Currently it only support create and delete activity. But we have another platform called SAP DataSphere, it's using BlobRenamed event type and RenameFile api to push data into storage account.

So the storage event trigger dosen't work in Data Factory.

Answer 1

Hello @Joe Zeng

Welcome to the Microsoft Q&A and thank you for posting your questions here.

In addition to solution above by @Pinaki Ghatak if you would like to have scalability and security solution, these are improvement solution and best practices:

1. To maintain state and handle retries more robustly if there are failures when triggering the ADF pipeline and if the rename events are frequent, consider using Azure Durable Functions to handle the webhook logic - https://learn.microsoft.com/azure/azure-functions/durable/durable-functions-overview
2. To minimize the overhead of processing unrelated events, ensure that your Event Grid subscription is set up with appropriate filters to capture only the relevant events (BlobRenamed). - https://learn.microsoft.com/azure/event-grid/event-schema-blob-storage
3. Secure your webhook endpoint using authentication methods like Azure Active Directory (AAD) or API Management, ensuring that only authorized services can trigger the webhook - https://learn.microsoft.com/azure/api-management/api-management-howto-protect-backend-with-aad
4. When triggering the Data Factory pipeline from Azure Functions or Logic Apps, you can use Managed Identity to authenticate without storing credentials - https://learn.microsoft.com/azure/data-factory/using-azure-managed-identities
5. Implement logging within the webhook (Logic App or Function) for tracing failures or bottlenecks and leverage Azure Monitor and Application Insights to track events and performance - https://learn.microsoft.com/azure/azure-monitor/overview

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.

Answer 2

Hello @Joe Zeng

Azure Data Factory supports BlobCreated and BlobDeleted events for storage event triggers.

However, it does not support BlobRenamed events. If you want to use BlobRenamed events to trigger your Data Factory pipelines, you can use Azure Event Grid to route BlobRenamed events to a webhook endpoint, which can then trigger your Data Factory pipeline.

Here are steps of how you can use Azure Event Grid to route BlobRenamed events to a webhook endpoint:

1. Create a webhook endpoint that triggers your Data Factory pipeline. You can use Azure Logic Apps or Azure Functions to create the webhook endpoint.
2. Create an Event Grid subscription that filters for BlobRenamed events and routes them to your webhook endpoint. You can use the Azure portal or Azure CLI to create the subscription.
3. Once you have set up the Event Grid subscription, any BlobRenamed events will be routed to your webhook endpoint, which can then trigger your Data Factory pipeline.

---

I hope this helps