DeliveryFailedEvent from Azure Event Grid to Synapse trigger with errorCode=RequestEntityTooLarge

Hobday, Colin 40 Reputation points
2024-11-08T08:39:37.6+00:00

I have an event-driven flow where

  1. changes in Comsos DB triggers an Azure Function
  2. the Azure Function pushes each changed document to a topic in Azure Event Grid
  3. a Synapse trigger is triggered and pushes the changed document to a Synapse Pipeline

Parts 1 and 2 seem to be working, but in Azure Event Grid, I am getting DeliveryFailedEvents.

Part 3 is not working when the documents is 150KB.

From the diagnostic logs in AegDeliveryFailureLogs table, it is clear that the reason is PayloadTooLarge, although the message is only about 150KB.

outcome=PayloadTooLarge, latencyInMs=86, id=8e10d942-42f5-435c-a416-49af181c936e, outputEventSystemId=e549e75e-dc5d-484f-a839-67ef9fd3210a, state=Filtered, deliveryTime=11/4/2024 7:51:11 AM, deliveryCount=0, probationCount=0, deliverySchema=EventGridEvent, trackedSystemTopicState=, eventSubscriptionDeliverySchema=EventGridEvent, outputEventFields=InputEvent| EventSubscriptionId| DeliveryTime| State| Id| LastHttpStatusCode| DeliverySchema| LastDeliveryAttemptTime| SystemId, outputEventFieldCount=, dedicatedQueueDeliveryQueueId=, requestExpiration=1/1/0001 12:00:00 AM, delivered=False id=eqpt-1111, inputEventSystemId=5d72a537-62ec-4bcd-b24a-3433a53cec68 publishTime=11/4/2024 7:51:11 AM, eventTime=11/4/2024 7:51:11 AM, eventType=update, deliveryTime=11/4/2024 7:51:11 AM, filteringState=FilteredWithRpc, inputSchema=EventGridEvent, publisher=XXX-EVENT-TOPIC.SOUTHEASTASIA-1.EVENTGRID.AZURE.NET, size=131852, subject=cosmos/update, inputEventFields=Id| PublishTime| SerializedBody| EventType| Topic| Subject| FilteringHashCode| SystemId| Publisher| FilteringTopic| TopicCategory| DataVersion| MetadataVersion| InputSchema| EventTime| FilteringPolicy, inputEventFieldCount=16, type=HttpPost, subType=AzureDataFactory, supportsBatching=False, aadIntegration=False, managedIdentityType=None, urlPath=https://xxx.svc.datafactory.azure.com:4443/triggerevent/XXTrigger/triggerCosmosDbChange, deliveryResponse=PayloadTooLarge, errorCode=RequestEntityTooLarge, HttpRequestMessage: httpVersion=1.1, HttpResponseMessage: HttpVersion=1.1, StatusCode=RequestEntityTooLarge(RequestEntityTooLarge), StatusDescription=Payload Too Large, IsRedirected=False, RedirectUrl=,


I have found that it works when the document is <32KB, i.e. the Synapse pipeline is trggered.


@app.event_grid_output(
    arg_name="outputEvent",
    topic_endpoint_uri="EventGridTopicEndpointUri",  # in application settings
    topic_key_setting="EventGridTopicKeySetting",  # in application settings
)                     
def cosmosdb_trigger1(
    docs: func.DocumentList, 
    outputEvent: func.Out[func.EventGridOutputEvent]
):
    logging.info('Python CosmosDB triggered.')
    if docs:
        for document in docs:
            now_time = datetime.datetime.utcnow() 
            event_data = {
                'id':  now_time.isoformat() + '-' + document['id'], 
                'data': document.to_json(),
                'subject':"cosmos/update",
                'event_type': "update",
                'event_time': now_time.isoformat(),
                'data_version': "1.0"                
            }
            event_data_str = json.dumps(event_data)
            event_data_size = len(event_data_str.encode('utf-8'))
            logging.info(f"Event data size: {event_data_size} bytes Content: {event_data_str[:1000]}")

            event_data.pop('event_time')
            outputEvent.set(func.EventGridOutputEvent(**event_data, event_time=now_time))
    else:
        logging.info("No documents found in the trigger.")

Why is the Synapse trigger not running when the payload is >32 KB?

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,124 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,002 questions
Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,675 questions
Azure Event Grid
Azure Event Grid
An Azure event routing service designed for high availability, consistent performance, and dynamic scale.
396 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.