Edit

Share via

.NET Aspire Azure Event Hubs integration

  • Article

Azure Event Hubs is a native data-streaming service in the cloud that can stream millions of events per second, with low latency, from any source to any destination. The .NET Aspire Azure Event Hubs integration enables you to connect to Azure Event Hubs instances from your .NET applications.

Hosting integration

The .NET Aspire Azure Event Hubs hosting integration models the various Event Hub resources as the following types:

To access these types and APIs for expressing them within your app host project, install the 📦 Aspire.Hosting.Azure.EventHubs NuGet package:

dotnet add package Aspire.Hosting.Azure.EventHubs

For more information, see dotnet add package or Manage package dependencies in .NET applications.

Add an Azure Event Hubs resource

To add an AzureEventHubsResource to your app host project, call the AddAzureEventHubs method providing a name, and then call AddHub:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs");
eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
       .WithReference(eventHubs);

// After adding all resources, run the app...

When you add an Azure Event Hubs resource to the app host, it exposes other useful APIs to add Event Hub resources, consumer groups, express explicit provisioning configuration, and enables the use of the Azure Event Hubs emulator. The preceding code adds an Azure Event Hubs resource named event-hubs and an Event Hub named messages to the app host project. The WithReference method passes the connection information to the ExampleService project.

Important

When you call AddAzureEventHubs, it implicitly calls AddAzureProvisioning(IDistributedApplicationBuilder)—which adds support for generating Azure resources dynamically during app startup. The app must configure the appropriate subscription and location. For more information, see Local provisioning: Configuration

Generated provisioning Bicep

If you're new to Bicep, it's a domain-specific language for defining Azure resources. With .NET Aspire, you don't need to write Bicep by-hand, instead the provisioning APIs generate Bicep for you. When you publish your app, the generated Bicep is output alongside the manifest file. When you add an Azure Event Hubs resource, the following Bicep is generated:

@description('The location for the resource(s) to be deployed.')
param location string = resourceGroup().location

param sku string = 'Standard'

param principalType string

param principalId string

resource event_hubs 'Microsoft.EventHub/namespaces@2024-01-01' = {
  name: take('event_hubs-${uniqueString(resourceGroup().id)}', 256)
  location: location
  sku: {
    name: sku
  }
  tags: {
    'aspire-resource-name': 'event-hubs'
  }
}

resource event_hubs_AzureEventHubsDataOwner 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(event_hubs.id, principalId, subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'f526a384-b230-433a-b45c-95f59c4a2dec'))
  properties: {
    principalId: principalId
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'f526a384-b230-433a-b45c-95f59c4a2dec')
    principalType: principalType
  }
  scope: event_hubs
}

resource messages 'Microsoft.EventHub/namespaces/eventhubs@2024-01-01' = {
  name: 'messages'
  parent: event_hubs
}

output eventHubsEndpoint string = event_hubs.properties.serviceBusEndpoint

The preceding Bicep is a module that provisions an Azure Event Hubs resource with the following defaults:

  • location: The location of the resource group.
  • sku: The SKU of the Event Hubs resource, defaults to Standard.
  • principalId: The principal ID of the Event Hubs resource.
  • principalType: The principal type of the Event Hubs resource.
  • event_hubs: The Event Hubs namespace resource.
  • event_hubs_AzureEventHubsDataOwner: The Event Hubs resource owner, based on the build-in Azure Event Hubs Data Owner role. For more information, see Azure Event Hubs Data Owner.
  • messages: The Event Hub resource.
  • eventHubsEndpoint: The endpoint of the Event Hubs resource.

The generated Bicep is a starting point and can be customized to meet your specific requirements.

Customize provisioning infrastructure

All .NET Aspire Azure resources are subclasses of the AzureProvisioningResource type. This type enables the customization of the generated Bicep by providing a fluent API to configure the Azure resources—using the ConfigureInfrastructure<T>(IResourceBuilder<T>, Action<AzureResourceInfrastructure>) API. For example, you can configure the kind, consistencyPolicy, locations, and more. The following example demonstrates how to customize the Azure Cosmos DB resource:

builder.AddAzureEventHubs("event-hubs")
    .ConfigureInfrastructure(infra =>
    {
        var eventHubs = infra.GetProvisionableResources()
                             .OfType<EventHubsNamespace>()
                             .Single();

        eventHubs.Sku = new EventHubsSku()
        {
            Name = EventHubsSkuName.Premium,
            Tier = EventHubsSkuTier.Premium,
            Capacity = 7,
        };
        eventHubs.PublicNetworkAccess = EventHubsPublicNetworkAccess.SecuredByPerimeter;
        eventHubs.Tags.Add("ExampleKey", "Example value");
    });

The preceding code:

There are many more configuration options available to customize the Event Hubs resource resource. For more information, see Azure.Provisioning.PostgreSql. For more information, see Azure.Provisioning customization.

Connect to an existing Azure Event Hubs namespace

You might have an existing Azure Event Hubs namespace that you want to connect to. Instead of representing a new Azure Event Hubs resource, you can add a connection string to the app host. To add a connection to an existing Azure Event Hubs namespace, call the AddConnectionString method:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddConnectionString("event-hubs");

builder.AddProject<Projects.WebApplication>("web")
       .WithReference(eventHubs);

// After adding all resources, run the app...

Note

Connection strings are used to represent a wide range of connection information, including database connections, message brokers, endpoint URIs, and other services. In .NET Aspire nomenclature, the term "connection string" is used to represent any kind of connection information.

The connection string is configured in the app host's configuration, typically under User Secrets, under the ConnectionStrings section. The app host injects this connection string as an environment variable into all dependent resources, for example:

{
  "ConnectionStrings": {
    "event-hubs": "{your_namespace}.servicebus.windows.net"
  }
}

The dependent resource can access the injected connection string by calling the GetConnectionString method, and passing the connection name as the parameter, in this case "event-hubs". The GetConnectionString API is shorthand for IConfiguration.GetSection("ConnectionStrings")[name].

Add Event Hub consumer group

To add a consumer group, chain a call on an IResourceBuilder<AzureEventHubsResource> to the AddConsumerGroup API:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs");
var messages = eventHubs.AddHub("messages");
messages.AddConsumerGroup("messagesConsumer");

builder.AddProject<Projects.ExampleService>()
       .WithReference(eventHubs);

// After adding all resources, run the app...

When you call AddConsumerGroup, it configures your messages Event Hub resource to have a consumer group named messagesConsumer. The consumer group is created in the Azure Event Hubs namespace that's represented by the AzureEventHubsResource that you added earlier. For more information, see Azure Event Hubs: Consumer groups.

Add Azure Event Hubs emulator resource

The .NET Aspire Azure Event Hubs hosting integration supports running the Event Hubs resource as an emulator locally, based on the mcr.microsoft.com/azure-messaging/eventhubs-emulator/latest container image. This is beneficial for situations where you want to run the Event Hubs resource locally for development and testing purposes, avoiding the need to provision an Azure resource or connect to an existing Azure Event Hubs server.

To run the Event Hubs resource as an emulator, call the RunAsEmulator method:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
                       .RunAsEmulator();

eventHubs.AddHub("messages");

var exampleProject = builder.AddProject<Projects.ExampleProject>()
                            .WithReference(eventHubs);

// After adding all resources, run the app...

The preceding code configures an Azure Event Hubs resource to run locally in a container. For more information, see Azure Event Hubs Emulator.

Configure Event Hubs emulator container

There are various configurations available for container resources, for example, you can configure the container's ports, data bind mounts, data volumes, or providing a wholistic JSON configuration which overrides everything.

Configure Event Hubs emulator container host port

By default, the Event Hubs emulator container when configured by .NET Aspire, exposes the following endpoints:

Endpoint Image Container port Host port
emulator mcr.microsoft.com/azure-messaging/eventhubs-emulator/latest 5672 dynamic

The port that it's listening on is dynamic by default. When the container starts, the port is mapped to a random port on the host machine. To configure the endpoint port, chain calls on the container resource builder provided by the RunAsEmulator method as shown in the following example:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
                       .RunAsEmulator(emulator =>
                       {
                           emulator.WithHostPort(7777);
                       });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
       .WithReference(eventHubs);

// After adding all resources, run the app...

The preceding code configures the Azure Event emulator container's existing emulator endpoint to listen on port 7777. The Azure Event emulator container's port is mapped to the host port as shown in the following table:

Endpoint name Port mapping (container:host)
emulator 5672:7777
Add Event Hubs emulator with data volume

To add a data volume to the Event Hubs emulator resource, call the WithDataVolume method on the Event Hubs emulator resource:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
                       .RunAsEmulator(emulator =>
                       {
                           emulator.WithDataVolume();
                       });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
       .WithReference(eventHubs);

// After adding all resources, run the app...

The data volume is used to persist the Event Hubs emulator data outside the lifecycle of its container. The data volume is mounted at the /data path in the container. A name is generated at random unless you provide a set the name parameter. For more information on data volumes and details on why they're preferred over bind mounts, see Docker docs: Volumes.

Add Event Hubs emulator with data bind mount

The add a bind mount to the Event Hubs emulator container, chain a call to the WithDataBindMount API, as shown in the following example:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
                       .RunAsEmulator(emulator =>
                       {
                           emulator.WithDataBindMount("/path/to/data");
                       });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
       .WithReference(eventHubs);

// After adding all resources, run the app...

Important

Data bind mounts have limited functionality compared to volumes, which offer better performance, portability, and security, making them more suitable for production environments. However, bind mounts allow direct access and modification of files on the host system, ideal for development and testing where real-time changes are needed.

Data bind mounts rely on the host machine's filesystem to persist the Azure Event Hubs emulator resource data across container restarts. The data bind mount is mounted at the /path/to/data path on the host machine in the container. For more information on data bind mounts, see Docker docs: Bind mounts.

Configure Event Hubs emulator container JSON configuration

The Event Hubs emulator container runs with a default config.json file. You can override this file entirely, or update the JSON configuration with a JsonNode representation of the configuration.

To provide a custom JSON configuration file, call the WithConfigurationFile method:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
                       .RunAsEmulator(emulator =>
                       {
                           emulator.WithConfigurationFile("./messaging/custom-config.json");
                       });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
       .WithReference(eventHubs);

// After adding all resources, run the app...

The preceding code configures the Event Hubs emulator container to use a custom JSON configuration file located at ./messaging/custom-config.json. This will be mounted at the /Eventhubs_Emulator/ConfigFiles/Config.json path on the container, as a read-only file. To instead override specific properties in the default configuration, call the WithConfiguration method:

var builder = DistributedApplication.CreateBuilder(args);

var eventHubs = builder.AddAzureEventHubs("event-hubs")
                       .RunAsEmulator(emulator =>
                       {
                           emulator.WithConfiguration(
                               (JsonNode configuration) =>
                               {
                                   var userConfig = configuration["UserConfig"];
                                   var ns = userConfig["NamespaceConfig"][0];
                                   var firstEntity = ns["Entities"][0];
                                   
                                   firstEntity["PartitionCount"] = 5;
                               });
                       });

eventHubs.AddHub("messages");

builder.AddProject<Projects.ExampleService>()
       .WithReference(eventHubs);

// After adding all resources, run the app...

The preceding code retrieves the UserConfig node from the default configuration. It then updates the first entity's PartitionCount to a 5.

Hosting integration health checks

The Azure Event Hubs hosting integration automatically adds a health check for the Event Hubs resource. The health check verifies that the Event Hubs is running and that a connection can be established to it.

The hosting integration relies on the 📦 AspNetCore.HealthChecks.Azure.Messaging.EventHubs NuGet package.

Client integration

To get started with the .NET Aspire Azure Event Hubs client integration, install the 📦 Aspire.Azure.Messaging.EventHubs NuGet package in the client-consuming project, that is, the project for the application that uses the Event Hubs client.

dotnet add package Aspire.Azure.Messaging.EventHubs

Supported Event Hubs client types

The following Event Hub clients are supported by the library, along with their corresponding options and settings classes:

Azure client type Azure options class .NET Aspire settings class
EventHubProducerClient EventHubProducerClientOptions AzureMessagingEventHubsProducerSettings
EventHubBufferedProducerClient EventHubBufferedProducerClientOptions AzureMessagingEventHubsBufferedProducerSettings
EventHubConsumerClient EventHubConsumerClientOptions AzureMessagingEventHubsConsumerSettings
EventProcessorClient EventProcessorClientOptions AzureMessagingEventHubsProcessorSettings
PartitionReceiver PartitionReceiverOptions AzureMessagingEventHubsPartitionReceiverSettings

The client types are from the Azure SDK for .NET, as are the corresponding options classes. The settings classes are provided by the .NET Aspire. The settings classes are used to configure the client instances.

Add an Event Hubs producer client

In the Program.cs file of your client-consuming project, call the AddAzureEventHubProducerClient extension method on any IHostApplicationBuilder to register an EventHubProducerClient for use via the dependency injection container. The method takes a connection name parameter.

builder.AddAzureEventHubProducerClient(connectionName: "event-hubs");

Tip

The connectionName parameter must match the name used when adding the Event Hubs resource in the app host project. For more information, see Add an Azure Event Hubs resource.

After adding the EventHubProducerClient, you can retrieve the client instance using dependency injection. For example, to retrieve your data source object from an example service define it as a constructor parameter and ensure the ExampleService class is registered with the dependency injection container:

public class ExampleService(EventHubProducerClient client)
{
    // Use client...
}

For more information, see:

Additional APIs to consider

The client integration provides additional APIs to configure client instances. When you need to register an Event Hubs client, consider the following APIs:

Azure client type Registration API
EventHubProducerClient AddAzureEventHubProducerClient
EventHubBufferedProducerClient AddAzureEventHubBufferedProducerClient
EventHubConsumerClient AddAzureEventHubConsumerClient
EventProcessorClient AddAzureEventProcessorClient
PartitionReceiver AddAzurePartitionReceiverClient

All of the aforementioned APIs include optional parameters to configure the client instances.

Add keyed Event Hubs producer client

There might be situations where you want to register multiple EventHubProducerClient instances with different connection names. To register keyed Event Hubs clients, call the AddKeyedAzureServiceBusClient method:

builder.AddKeyedAzureEventHubProducerClient(name: "messages");
builder.AddKeyedAzureEventHubProducerClient(name: "commands");

Important

When using keyed services, it's expected that your Event Hubs resource configured two named hubs, one for the messages and one for the commands.

Then you can retrieve the client instances using dependency injection. For example, to retrieve the clients from a service:

public class ExampleService(
    [KeyedService("messages")] EventHubProducerClient messagesClient,
    [KeyedService("commands")] EventHubProducerClient commandsClient)
{
    // Use clients...
}

For more information, see Keyed services in .NET.

Additional keyed APIs to consider

The client integration provides additional APIs to configure keyed client instances. When you need to register a keyed Event Hubs client, consider the following APIs:

Azure client type Registration API
EventHubProducerClient AddKeyedAzureEventHubProducerClient
EventHubBufferedProducerClient AddKeyedAzureEventHubBufferedProducerClient
EventHubConsumerClient AddKeyedAzureEventHubConsumerClient
EventProcessorClient AddKeyedAzureEventProcessorClient
PartitionReceiver AddKeyedAzurePartitionReceiverClient

All of the aforementioned APIs include optional parameters to configure the client instances.

Configuration

The .NET Aspire Azure Event Hubs library provides multiple options to configure the Azure Event Hubs connection based on the requirements and conventions of your project. Either a FullyQualifiedNamespace or a ConnectionString is a required to be supplied.

Use a connection string

When using a connection string from the ConnectionStrings configuration section, provide the name of the connection string when calling builder.AddAzureEventHubProducerClient() and other supported Event Hubs clients. In this example, the connection string does not include the EntityPath property, so the EventHubName property must be set in the settings callback:

builder.AddAzureEventHubProducerClient(
    "event-hubs",
    static settings =>
    {
        settings.EventHubName = "MyHub";
    });

And then the connection information will be retrieved from the ConnectionStrings configuration section. Two connection formats are supported:

Fully Qualified Namespace (FQN)

The recommended approach is to use a fully qualified namespace, which works with the AzureMessagingEventHubsSettings.Credential property to establish a connection. If no credential is configured, the DefaultAzureCredential is used.

{
  "ConnectionStrings": {
    "event-hubs": "{your_namespace}.servicebus.windows.net"
  }
}

Connection string

Alternatively, use a connection string:

{
  "ConnectionStrings": {
    "event-hubs": "Endpoint=sb://mynamespace.servicebus.windows.net/;SharedAccessKeyName=accesskeyname;SharedAccessKey=accesskey;EntityPath=MyHub"
  }
}

Use configuration providers

The .NET Aspire Azure Event Hubs library supports Microsoft.Extensions.Configuration. It loads the AzureMessagingEventHubsSettings and the associated Options, e.g. EventProcessorClientOptions, from configuration by using the Aspire:Azure:Messaging:EventHubs: key prefix, followed by the name of the specific client in use. For example, consider the appsettings.json that configures some of the options for an EventProcessorClient:

{
  "Aspire": {
    "Azure": {
      "Messaging": {
        "EventHubs": {
          "EventProcessorClient": {
            "EventHubName": "MyHub",
            "ClientOptions": {
              "Identifier": "PROCESSOR_ID"
            }
          }
        }
      }
    }
  }
}

You can also setup the Options type using the optional Action<IAzureClientBuilder<EventProcessorClient, EventProcessorClientOptions>> configureClientBuilder parameter of the AddAzureEventProcessorClient method. For example, to set the processor's client ID for this client:

builder.AddAzureEventProcessorClient(
    "event-hubs",
    configureClientBuilder: clientBuilder => clientBuilder.ConfigureOptions(
        options => options.Identifier = "PROCESSOR_ID"));

Observability and telemetry

.NET Aspire integrations automatically set up Logging, Tracing, and Metrics configurations, which are sometimes known as the pillars of observability. For more information about integration observability and telemetry, see .NET Aspire integrations overview. Depending on the backing service, some integrations may only support some of these features. For example, some integrations support logging and tracing, but not metrics. Telemetry features can also be disabled using the techniques presented in the Configuration section.

Logging

The .NET Aspire Azure Event Hubs integration uses the following log categories:

  • Azure.Core
  • Azure.Identity

Tracing

The .NET Aspire Azure Event Hubs integration will emit the following tracing activities using OpenTelemetry:

  • Azure.Messaging.EventHubs.*

Metrics

The .NET Aspire Azure Event Hubs integration currently doesn't support metrics by default due to limitations with the Azure SDK for .NET. If that changes in the future, this section will be updated to reflect those changes.

See also