Pull delivery with HTTP
This article builds on the What is Azure Event Grid? and on the Event Grid concepts article to provide essential information before you start using Event Grid’s pull delivery over HTTP. It covers fundamental concepts, resource models, and message delivery modes supported. At the end of this document, you find useful links to articles that guide you on how to use Event Grid and to articles that offer in-depth conceptual information.
Note
This document helps you get started with Event Grid capabilities that use the HTTP protocol. This article is suitable for users who need to integrate applications on the cloud. If you require to communicate IoT device data, see Overview of the MQTT Broker feature in Azure Event Grid.
CloudEvents
Event Grid namespace topics accepts events that comply with the Cloud Native Computing Foundation (CNCF)’s open standard CloudEvents 1.0 specification using the HTTP protocol binding with JSON format.
For more information, see CloudEvents support for more information.
CloudEvents content modes
The CloudEvents specification defines three content modes you can use: binary, structured, and batched.
Important
With any content mode you can exchange text (JSON, text/*, etc.) or binary encoded event data. The binary content mode is not exclusively used for sending binary data.
The content modes aren't about the encoding you use, binary or text, but about how the event data and its metadata are described and exchanged. The structured content mode uses a single structure, for example, a JSON object, where both the context attributes and event data are together in the HTTP payload. The binary content mode separates context attributes, which are mapped to HTTP headers, and event data, which is the HTTP payload encoded according to the media type value in Content-Type.
For more information, see CloudEvents content modes for more information.
Messages and events
A CloudEvent typically carries event data announcing an occurrence in a system, that is, a system state change. However, you could convey any kind of data when using CloudEvents. For example, you might want to use the CloudEvents exchange format to send a command message to request an action to a downstream application. Another example is when you're routing messages from Event Grid's MQTT broker to a topic. Under this scenario, you're routing an MQTT message wrapped up in a CloudEvents envelope.
Pull delivery
With pull delivery, your application connects to Event Grid to read CloudEvents using queue-like semantics.
Pull delivery offers these event consumption benefits:
Consume events at your own pace, at scale or at an ingress rate your application supports.
Consume events at a time of your own choosing. For example, given business requirements, messages are processed at night.
Consume events over a private link so that your data uses private IP space.
Note
- Namespaces provide a simpler resource model featuring a single kind of topic. Currently, Event Grid supports publishing your own application events through namespace topics. You cannot consume events from Azure services or partner SaaS systems using namespace topics. You also cannot create system topics, domain topics or partner topics in a namespace.
- Namespace topics support CloudEvents JSON format.
Queue event subscriptions
When receiving events or using operations that manage event state, an application specifies a namespace HTTP endpoint, a topic name, and the name of a queue event subscription. A queue event subscription has its deliveryMode set to "queue". Queue event subscriptions are used to consume events using the pull delivery API. For more information about how to create these resources, see create namespaces, topics, and event subscriptions.
You use an event subscription to define the filtering criteria for events and in doing so, you effectively define the set of events that are available for consumption through that event subscription. One or more subscriber (consumer) applications can connect to the same namespace endpoint and use the same topic and event subscription.
Pull delivery operations
Your application uses the following operations when working with pull delivery.
- A receive operation is used to read one or more events using a single request to Event Grid. By default, the broker waits for up to 60 seconds for events to become available. For example, events become available for delivery when they're first published. A successful receive request returns zero or more events. If events are available, it returns as many available events as possible up to the event count requested. Event Grid also returns a lock token for every event read.
- A lock token is a kind of handle that identifies an event that you can use to control its state.
- Once a consumer application receives an event and processes it, it acknowledges that event. This operation instructs Event Grid to delete the event so it isn't redelivered to another client. The consumer application acknowledges one or more tokens with a single request by specifying their lock tokens before they expire.
In some other occasions, your consumer application might want to release or reject events.
Your consumer application releases a received event to signal Event Grid that it isn't ready to process that event and to make it available for redelivery. It does so by calling the release operation with the lock tokens identifying the events to be returned back to Event Grid. Your application can control if the event should be released immediately or if a delay should be used before the event is available for redelivery.
You could opt to reject an event if there's a condition, possibly permanent, that prevents your consumer application to process the event. For example, a malformed message can be rejected as it can't be successfully parsed. Rejected events are dead-lettered, if a dead-letter destination is available. Otherwise, they're dropped.
Scope on which pull delivery operations run
When you invoke a receive, acknowledge, release, reject, or renew lock operation, those actions are performed in the context of the event subscription. For example, if you acknowledge an event, that event is no longer available through the event subscription used when calling the acknowledge action. Other event subscription could still have the "same" event available. That is because an event subscription gets a copy of the events published. Those event copies are effectively distinct from each other across event subscriptions. Each event has its own state independent of other events.
Data shape when receiving events using pull delivery
When delivering events using pull delivery, Event Grid includes an array of objects that in turn includes the event and brokerProperties objects. The value of the event property is the CloudEvent delivered in structured content mode. The brokerProperties object contains the lock token associated to the CloudEvent delivered. The following json object is a sample response from a receive operation that returns two events:
{
"value": [
{
"brokerProperties": {
"lockToken": "CiYKJDUwNjE4QTFFLUNDODQtNDZBQy1BN0Y4LUE5QkE3NjEwNzQxMxISChDXYS23Z+5Hq754VqQjxywE",
"deliveryCount": 2
},
"event": {
"specversion": "1.0",
"id": "A234-1234-1235",
"source": "/mycontext",
"time": "2018-04-05T17:31:00Z",
"type": "com.example.someeventtype",
"data": "some data"
}
},
{
"brokerProperties": {
"lockToken": "CiYKJDUwNjE4QTFFLUNDODQtNDZBQy1BN0Y4LUE5QkE3NjEwNzQxMxISChDLeaL+nRJLNq3/5NXd/T0b",
"deliveryCount": 1
},
"event": {
"specversion": "1.0",
"id": "B688-1234-1235",
"source": "/mycontext",
"type": "com.example.someeventtype",
"time": "2018-04-05T17:31:00Z",
"data": {
"somekey" : "value",
"someOtherKey" : 9
}
}
}
]
}
Push and pull delivery
Event Grid supports push and pull event delivery using HTTP. With push delivery, you define a destination in an event subscription, a webhook, or an Azure service, to which Event Grid sends events. With pull delivery, subscriber applications connect to Event Grid to consume events. Pull delivery is supported for topics in an Event Grid namespace.
Important
Event Hubs is supported as a destination for subscriptions to namespace topics. In coming releases, Event Grid Namespaces will support all destinations currently available in Event Grid Basic along with additional destinations.
When to use push delivery vs. pull delivery
The following are general guidelines to help you decide when to use pull or push delivery.
Pull delivery
- You need full control as to when to receive events. For example, your application might not be up all the time, not stable enough, or you process data at certain times.
- You need full control over event consumption. For example, a downstream service or layer in your consumer application has a problem that prevents you from processing events. In that case, the pull delivery API allows the consumer app to release an already read event back to the broker so that it can be delivered later.
- You want to use private links when receiving events, which is possible only with the pull delivery, not the push delivery.
- You don't have the ability to expose an endpoint and use push delivery, but you can connect to Event Grid to consume events.
Push delivery
- You want to avoid constant polling to determine that a system state change has occurred. You rather use Event Grid to send events to you at the time state changes happen.
- You have an application that can't make outbound calls. For example, your organization might be concerned about data exfiltration. However, your application can receive events through a public endpoint.
Next steps
The following articles provide you with information on how to use Event Grid or provide you with additional information on concepts.
Other useful links
- Control plane and data plane SDKs.
- Data plane SDKs announcement with a plethora of information, samples, and links.
- Quotas and limits.