Edit

Share via

Build resilient HTTP apps: Key development patterns

  • Article

Building robust HTTP apps that can recover from transient fault errors is a common requirement. This article assumes that you've already read Introduction to resilient app development, as this article extends the core concepts conveyed. To help build resilient HTTP apps, the Microsoft.Extensions.Http.Resilience NuGet package provides resilience mechanisms specifically for the HttpClient. This NuGet package relies on the Microsoft.Extensions.Resilience library and Polly, which is a popular open-source project. For more information, see Polly.

Get started

To use resilience-patterns in HTTP apps, install the Microsoft.Extensions.Http.Resilience NuGet package.

dotnet add package Microsoft.Extensions.Http.Resilience --version 8.0.0

For more information, see dotnet add package or Manage package dependencies in .NET applications.

Add resilience to an HTTP client

To add resilience to an HttpClient, you chain a call on the IHttpClientBuilder type that is returned from calling any of the available AddHttpClient methods. For more information, see IHttpClientFactory with .NET.

There are several resilience-centric extensions available. Some are standard, thus employing various industry best practices, and others are more customizable. When adding resilience, you should only add one resilience handler and avoid stacking handlers. If you need to add multiple resilience handlers, you should consider using the AddResilienceHandler extension method, which allows you to customize the resilience strategies.

Important

All of the examples within this article rely on the AddHttpClient API, from the Microsoft.Extensions.Http library, which returns an IHttpClientBuilder instance. The IHttpClientBuilder instance is used to configure the HttpClient and add the resilience handler.

Add standard resilience handler

The standard resilience handler uses multiple resilience strategies stacked atop one another, with default options to send the requests and handle any transient errors. The standard resilience handler is added by calling the AddStandardResilienceHandler extension method on an IHttpClientBuilder instance.

var services = new ServiceCollection();

var httpClientBuilder = services.AddHttpClient<ExampleClient>(
    configureClient: static client =>
    {
        client.BaseAddress = new("https://jsonplaceholder.typicode.com");
    });

The preceding code:

  • Creates a ServiceCollection instance.
  • Adds an HttpClient for the ExampleClient type to the service container.
  • Configures the HttpClient to use "https://jsonplaceholder.typicode.com" as the base address.
  • Creates the httpClientBuilder that's used throughout the other examples within this article.

A more real-world example would rely on hosting, such as that described in the .NET Generic Host article. Using the Microsoft.Extensions.Hosting NuGet package, consider the following updated example:

using Http.Resilience.Example;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

HostApplicationBuilder builder = Host.CreateApplicationBuilder(args);

IHttpClientBuilder httpClientBuilder = builder.Services.AddHttpClient<ExampleClient>(
    configureClient: static client =>
    {
        client.BaseAddress = new("https://jsonplaceholder.typicode.com");
    });

The preceding code is similar to the manual ServiceCollection creation approach, but instead relies on the Host.CreateApplicationBuilder() to build out a host that exposes the services.

The ExampleClient is defined as follows:

using System.Net.Http.Json;

namespace Http.Resilience.Example;

/// <summary>
/// An example client service, that relies on the <see cref="HttpClient"/> instance.
/// </summary>
/// <param name="client">The given <see cref="HttpClient"/> instance.</param>
internal sealed class ExampleClient(HttpClient client)
{
    /// <summary>
    /// Returns an <see cref="IAsyncEnumerable{T}"/> of <see cref="Comment"/>s.
    /// </summary>
    public IAsyncEnumerable<Comment?> GetCommentsAsync()
    {
        return client.GetFromJsonAsAsyncEnumerable<Comment>("/comments");
    }
}

The preceding code:

  • Defines an ExampleClient type that has a constructor that accepts an HttpClient.
  • Exposes a GetCommentsAsync method that sends a GET request to the /comments endpoint and returns the response.

The Comment type is defined as follows:

namespace Http.Resilience.Example;

public record class Comment(
    int PostId, int Id, string Name, string Email, string Body);

Given that you've created an IHttpClientBuilder (httpClientBuilder), and you now understand the ExampleClient implementation and corresponding Comment model, consider the following example:

httpClientBuilder.AddStandardResilienceHandler();

The preceding code adds the standard resilience handler to the HttpClient. Like most resilience APIs, there are overloads that allow you to customize the default options and applied resilience strategies.

Standard resilience handler defaults

The default configuration chains five resilience strategies in the following order (from the outermost to the innermost):

Order Strategy Description Defaults
1 Rate limiter The rate limiter pipeline limits the maximum number of concurrent requests being sent to the dependency. Queue: 0
Permit: 1_000
2 Total timeout The total request timeout pipeline applies an overall timeout to the execution, ensuring that the request, including retry attempts, doesn't exceed the configured limit. Total timeout: 30s
3 Retry The retry pipeline retries the request in case the dependency is slow or returns a transient error. Max retries: 3
Backoff: Exponential
Use jitter: true
Delay:2s
4 Circuit breaker The circuit breaker blocks the execution if too many direct failures or timeouts are detected. Failure ratio: 10%
Min throughput: 100
Sampling duration: 30s
Break duration: 5s
5 Attempt timeout The attempt timeout pipeline limits each request attempt duration and throws if it's exceeded. Attempt timeout: 10s

Retries and circuit breakers

The retry and circuit breaker strategies both handle a set of specific HTTP status codes and exceptions. Consider the following HTTP status codes:

  • HTTP 500 and above (Server errors)
  • HTTP 408 (Request timeout)
  • HTTP 429 (Too many requests)

Additionally, these strategies handle the following exceptions:

  • HttpRequestException
  • TimeoutRejectedException

Add standard hedging handler

The standard hedging handler wraps the execution of the request with a standard hedging mechanism. Hedging retries slow requests in parallel.

To use the standard hedging handler, call AddStandardHedgingHandler extension method. The following example configures the ExampleClient to use the standard hedging handler.

httpClientBuilder.AddStandardHedgingHandler();

The preceding code adds the standard hedging handler to the HttpClient.

Standard hedging handler defaults

The standard hedging uses a pool of circuit breakers to ensure that unhealthy endpoints aren't hedged against. By default, the selection from the pool is based on the URL authority (scheme + host + port).

Tip

It's recommended that you configure the way the strategies are selected by calling StandardHedgingHandlerBuilderExtensions.SelectPipelineByAuthority or StandardHedgingHandlerBuilderExtensions.SelectPipelineBy for more advanced scenarios.

The preceding code adds the standard hedging handler to the IHttpClientBuilder. The default configuration chains five resilience strategies in the following order (from the outermost to the innermost):

Order Strategy Description Defaults
1 Total request timeout The total request timeout pipeline applies an overall timeout to the execution, ensuring that the request, including hedging attempts, doesn't exceed the configured limit. Total timeout: 30s
2 Hedging The hedging strategy executes the requests against multiple endpoints in case the dependency is slow or returns a transient error. Routing is options, by default it just hedges the URL provided by the original HttpRequestMessage. Min attempts: 1
Max attempts: 10
Delay: 2s
3 Rate limiter (per endpoint) The rate limiter pipeline limits the maximum number of concurrent requests being sent to the dependency. Queue: 0
Permit: 1_000
4 Circuit breaker (per endpoint) The circuit breaker blocks the execution if too many direct failures or timeouts are detected. Failure ratio: 10%
Min throughput: 100
Sampling duration: 30s
Break duration: 5s
5 Attempt timeout (per endpoint) The attempt timeout pipeline limits each request attempt duration and throws if it's exceeded. Timeout: 10s

Customize hedging handler route selection

When using the standard hedging handler, you can customize the way the request endpoints are selected by calling various extensions on the IRoutingStrategyBuilder type. This can be useful for scenarios such as A/B testing, where you want to route a percentage of the requests to a different endpoint:

httpClientBuilder.AddStandardHedgingHandler(static (IRoutingStrategyBuilder builder) =>
{
    // Hedging allows sending multiple concurrent requests
    builder.ConfigureOrderedGroups(static options =>
    {
        options.Groups.Add(new UriEndpointGroup()
        {
            Endpoints =
            {
                // Imagine a scenario where 3% of the requests are 
                // sent to the experimental endpoint.
                new() { Uri = new("https://example.net/api/experimental"), Weight = 3 },
                new() { Uri = new("https://example.net/api/stable"), Weight = 97 }
            }
        });
    });
});

The preceding code:

  • Adds the hedging handler to the IHttpClientBuilder.
  • Configures the IRoutingStrategyBuilder to use the ConfigureOrderedGroups method to configure the ordered groups.
  • Adds an EndpointGroup to the orderedGroup that routes 3% of the requests to the https://example.net/api/experimental endpoint and 97% of the requests to the https://example.net/api/stable endpoint.
  • Configures the IRoutingStrategyBuilder to use the ConfigureWeightedGroups method to configure the

To configure a weighted group, call the ConfigureWeightedGroups method on the IRoutingStrategyBuilder type. The following example configures the IRoutingStrategyBuilder to use the ConfigureWeightedGroups method to configure the weighted groups.

httpClientBuilder.AddStandardHedgingHandler(static (IRoutingStrategyBuilder builder) =>
{
    // Hedging allows sending multiple concurrent requests
    builder.ConfigureWeightedGroups(static options =>
    {
        options.SelectionMode = WeightedGroupSelectionMode.EveryAttempt;

        options.Groups.Add(new WeightedUriEndpointGroup()
        {
            Endpoints =
            {
                // Imagine A/B testing
                new() { Uri = new("https://example.net/api/a"), Weight = 33 },
                new() { Uri = new("https://example.net/api/b"), Weight = 33 },
                new() { Uri = new("https://example.net/api/c"), Weight = 33 }
            }
        });
    });
});

The preceding code:

  • Adds the hedging handler to the IHttpClientBuilder.
  • Configures the IRoutingStrategyBuilder to use the ConfigureWeightedGroups method to configure the weighted groups.
  • Sets the SelectionMode to WeightedGroupSelectionMode.EveryAttempt.
  • Adds a WeightedEndpointGroup to the weightedGroup that routes 33% of the requests to the https://example.net/api/a endpoint, 33% of the requests to the https://example.net/api/b endpoint, and 33% of the requests to the https://example.net/api/c endpoint.

Tip

The maximum number of hedging attempts directly correlates to the number of configured groups. For example, if you have two groups, the maximum number of attempts is two.

For more information, see Polly docs: Hedging resilience strategy.

It's common to configure either an ordered group or weighted group, but it's valid to configure both. Using ordered and weighted groups is helpful in scenarios where you want to send a percentage of the requests to a different endpoint, such is the case with A/B testing.

Add custom resilience handlers

To have more control, you can customize the resilience handlers by using the AddResilienceHandler API. This method accepts a delegate that configures the ResiliencePipelineBuilder<HttpResponseMessage> instance that is used to create the resilience strategies.

To configure a named resilience handler, call the AddResilienceHandler extension method with the name of the handler. The following example configures a named resilience handler called "CustomPipeline".

httpClientBuilder.AddResilienceHandler(
    "CustomPipeline",
    static builder =>
{
    // See: https://www.pollydocs.org/strategies/retry.html
    builder.AddRetry(new HttpRetryStrategyOptions
    {
        // Customize and configure the retry logic.
        BackoffType = DelayBackoffType.Exponential,
        MaxRetryAttempts = 5,
        UseJitter = true
    });

    // See: https://www.pollydocs.org/strategies/circuit-breaker.html
    builder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
    {
        // Customize and configure the circuit breaker logic.
        SamplingDuration = TimeSpan.FromSeconds(10),
        FailureRatio = 0.2,
        MinimumThroughput = 3,
        ShouldHandle = static args =>
        {
            return ValueTask.FromResult(args is
            {
                Outcome.Result.StatusCode:
                    HttpStatusCode.RequestTimeout or
                        HttpStatusCode.TooManyRequests
            });
        }
    });

    // See: https://www.pollydocs.org/strategies/timeout.html
    builder.AddTimeout(TimeSpan.FromSeconds(5));
});

The preceding code:

  • Adds a resilience handler with the name "CustomPipeline" as the pipelineName to the service container.
  • Adds a retry strategy with exponential backoff, five retries, and jitter preference to the resilience builder.
  • Adds a circuit breaker strategy with a sampling duration of 10 seconds, a failure ratio of 0.2 (20%), a minimum throughput of three, and a predicate that handles RequestTimeout and TooManyRequests HTTP status codes to the resilience builder.
  • Adds a timeout strategy with a timeout of five seconds to the resilience builder.

There are many options available for each of the resilience strategies. For more information, see the Polly docs: Strategies. For more information about configuring ShouldHandle delegates, see Polly docs: Fault handling in reactive strategies.

Dynamic reload

Polly supports dynamic reloading of the configured resilience strategies. This means that you can change the configuration of the resilience strategies at run time. To enable dynamic reload, use the appropriate AddResilienceHandler overload that exposes the ResilienceHandlerContext. Given the context, call EnableReloads of the corresponding resilience strategy options:

httpClientBuilder.AddResilienceHandler(
    "AdvancedPipeline",
    static (ResiliencePipelineBuilder<HttpResponseMessage> builder,
        ResilienceHandlerContext context) =>
    {
        // Enable reloads whenever the named options change
        context.EnableReloads<HttpRetryStrategyOptions>("RetryOptions");

        // Retrieve the named options
        var retryOptions =
            context.GetOptions<HttpRetryStrategyOptions>("RetryOptions");

        // Add retries using the resolved options
        builder.AddRetry(retryOptions);
    });

The preceding code:

  • Adds a resilience handler with the name "AdvancedPipeline" as the pipelineName to the service container.
  • Enables the reloads of the "AdvancedPipeline" pipeline whenever the named RetryStrategyOptions options change.
  • Retrieves the named options from the IOptionsMonitor<TOptions> service.
  • Adds a retry strategy with the retrieved options to the resilience builder.

For more information, see Polly docs: Advanced dependency injection.

This example relies on an options section that is capable of change, such as an appsettings.json file. Consider the following appsettings.json file:

{
    "RetryOptions": {
        "Retry": {
            "BackoffType": "Linear",
            "UseJitter": false,
            "MaxRetryAttempts": 7
        }
    }
}

Now imagine that these options were bound to the app's configuration, binding the HttpRetryStrategyOptions to the "RetryOptions" section:

var section = builder.Configuration.GetSection("RetryOptions");

builder.Services.Configure<HttpStandardResilienceOptions>(section);

For more information, see Options pattern in .NET.

Example usage

Your app relies on dependency injection to resolve the ExampleClient and its corresponding HttpClient. The code builds the IServiceProvider and resolves the ExampleClient from it.

IHost host = builder.Build();

ExampleClient client = host.Services.GetRequiredService<ExampleClient>();

await foreach (Comment? comment in client.GetCommentsAsync())
{
    Console.WriteLine(comment);
}

The preceding code:

Imagine a situation where the network goes down or the server becomes unresponsive. The following diagram shows how the resilience strategies would handle the situation, given the ExampleClient and the GetCommentsAsync method:

Example HTTP GET work flow with resilience pipeline.

The preceding diagram depicts:

  • The ExampleClient sends an HTTP GET request to the /comments endpoint.
  • The HttpResponseMessage is evaluated:
    • If the response is successful (HTTP 200), the response is returned.
    • If the response is unsuccessful (HTTP non-200), the resilience pipeline employs the configured resilience strategies.

While this is a simple example, it demonstrates how the resilience strategies can be used to handle transient errors. For more information, see Polly docs: Strategies.

Known issues

The following sections detail various known issues.

Compatibility with the Grpc.Net.ClientFactory package

If you're using Grpc.Net.ClientFactory version 2.63.0 or earlier, then enabling the standard resilience or hedging handlers for a gRPC client could cause a runtime exception. Specifically, consider the following code sample:

services
    .AddGrpcClient<Greeter.GreeterClient>()
    .AddStandardResilienceHandler();

The preceding code results in the following exception:

System.InvalidOperationException: The ConfigureHttpClient method is not supported when creating gRPC clients. Unable to create client with name 'GreeterClient'.

To resolve this issue, we recommend upgrading to Grpc.Net.ClientFactory version 2.64.0 or later.

There's a build time check that verifies if you're using Grpc.Net.ClientFactory version 2.63.0 or earlier, and if you are the check produces a compilation warning. You can suppress the warning by setting the following property in your project file:

<PropertyGroup>
  <SuppressCheckGrpcNetClientFactoryVersion>true</SuppressCheckGrpcNetClientFactoryVersion>
</PropertyGroup>

Compatibility with .NET Application Insights

If you're using .NET Application Insights, then enabling resilience functionality in your application could cause all Application Insights telemetry to be missing. The issue occurs when resilience functionality is registered before Application Insights services. Consider the following sample causing the issue:

// At first, we register resilience functionality.
services.AddHttpClient().AddStandardResilienceHandler();

// And then we register Application Insights. As a result, Application Insights doesn't work.
services.AddApplicationInsightsTelemetry();

The issue is caused by the following bug in Application Insights and can be fixed by registering Application Insights services before resilience functionality, as shown below:

// We register Application Insights first, and now it will be working correctly.
services.AddApplicationInsightsTelemetry();
services.AddHttpClient().AddStandardResilienceHandler();