Azure Event Grid - Making the Cloud Reactive

With serverless computing becoming more pervasive, the older models for communication based on long polling or hammer polling doesn't really gel well in a serverless application architecture. What we need is a service that will enable event-based reactive programming on the cloud. A service that can react to events and then execute the necessary programming logic. This is where Event Grid comes in.

What is Event Grid?

According to the online documentation - Azure Event Grid is a fully-managed intelligent event routing service that allows for uniform event consumption using a publish-subscribe model.

I have bolded the two important words in the above definition.

• Azure Event Grid is a fully-managed event routing service which greatly simplifies the development of event-based applications and simplifies the creation of serverless workflows. Using a single service, Azure Event Grid manages all routing of events from any source, to any destination, for any application.
• Azure Event Grid, is also intelligent, i.e. it enables direct event filtering using event type, prefix or suffix, so your application will only need to receive the events you care about. Whether you want to handle built-in Azure events, like a file being added to storage, or you want to produce your own custom events and event handlers, Event Grid enables this through the same underlying model. Thus, no matter the service or the use case, the intelligent routing and filtering capabilities apply to every event scenario and ensure that your apps can focus on the core business logic instead of worrying about routing events.

As shown above, Blob Storage, Resource Groups, Azure Subscriptions, Event Hubs, Custom Topics (custom apps) are a few event Publishers and Azure Functions, Logic Apps, Azure Automation, Webhooks are a few event Subscribers supported by Event Grid as of writing this blog. But with even more event sources and destinations expected later this year, including Azure Active Directory, API Management, IoT Hub, Service Bus, Azure Data Lake Store, Azure Cosmos DB, Azure Data Factory, and Storage Queues, it will be a formidable service to watch out for.

Wait.. Isn't this identical to Azure Service Bus ?

It can be confusing when there are multiple identical technologies doing similar stuff. Specifically, Event Grid's resemblance to Azure Service Bus Topics is hard to ignore, but there are significant differences and a reason for each of their existence. 

Service Bus is really an enterprise messaging product. If you're moving money or material you probably want transactions, dead-lettering, duplicate detection, sessions and other capabilities that Service Bus provides you. Event Grid is really about eventing, not enterprise messaging.  Service Bus is better for financial transactions and workflows, Event Grid is for reactive programming in the cloud.

Another significant difference lies in the fact that events are pushed to 'subscribers' in the Event Grid unlike Service Bus where the ‘subscribers’ have to poll the queue, get the message and process it. There are also differences in parameters like throughput and message/event sizes. Event Grid promises 10 million events per second, per region, which is far more than what Azure Service Bus can handle.

What about Azure Event Hubs?

There is no such thing as Event Hub vs Event Grid, rather, they complement each other. Azure Event Hub is used more for the telemetry scenarios. A great example is Azure Application Insight which under the hood uses Azure Event Hub to capture the telemetry information.

This excellent sample demonstrates how Event Hub and Event Grid complement each other - https://github.com/Azure/azure-event-hubs/tree/master/samples/e2e/EventHubsCaptureEventGridDemo

So is this like AWS Simple Notification Service?

Again, similar, but here the difference lies in the the way its integrated with core cloud services. In AWS, developers need to explicitly connect SNS infrastructure to services like S3 to receive bucket and object related notifications. Azure In Event Grid, developers can quickly consume the events generated by these services without any explicit configuration. Another minor difference lies in the way data is filtered. While SNS dumps every event at the consumer, Event Grid subscribers can choose a subset of events through filters.

What else do we get?

• Near-real-time event delivery
• Fan-out - Subscribe multiple endpoints to the same event to send copies of the event to as many places as needed.
• Massive scale - Supports high-volume workloads with support for millions of events per second. 
• Reliable Delivery - Event Grid will perform an exponential back off for all deliveries. Ex: If your WebHook does not return a HTTP 2xx, the retry begins immediately. When the back off interval reaches one hour, retries will be performed every hour. After 24 hours, the service will stop attempting to deliver events

What can we do with Event Grid?

Azure Event Grid provides several capabilities that vastly improve serverless, ops automation, and integration work:

Serverless application architectures

To connect various cloud services so as to react to changes. For example, use Event Grid to instantly trigger a serverless function to run image analysis each time a new photo is added to a blob storage container. Another example could be that of an ad-serving app built on Azure publishes millions of events to an application topic that has several event subscriptions connected to it. All events are pushed to SQL Data Warehouse and are also sent to one of several custom Node.js apps for processing, based on the origin of the event.

Ops Automation

Event Grid also allows to speed up automation and simplify policy enforcement. For example, Event Grid can notify Azure Automation when a virtual machine is created, or a SQL Database is spun up. These events can then be used to check that service configurations are compliant, tag virtual machines, or file work items, etc.

Custom Application integration

Event Grid connects can also connect a custom app with other services. For example, we can create a custom topic to send your app's event data to Event Grid, and take advantage of its reliable delivery, advanced routing, and direct integration with Azure and use it in an Azure Function that is created with Webhook Trigger.

In summary,  Azure Event Hub is a great new addition to the big basket of Azure offerings which makes the services reactive  and also apps integration easier. It is huge step forward in the serverless model. Event Grid is in preview currently and uses a pay-per-event pricing model, costing $0.30 per million operations during preview ($0.60 after GA) and the first 100,000 operation per month are free.

Images sourced from original documentation: /en-us/azure/event-grid/