HttpWebRequest to HttpClient migration guide

This article aims to guide developers through the process of migrating from HttpWebRequest, ServicePoint, and ServicePointManager to HttpClient. The migration is necessary due to the obsolescence of the older APIs and the numerous benefits offered by HttpClient, including improved performance, better resource management, and a more modern and flexible API design. By following the steps outlined in this document, developers will be able to transition their codebases smoothly and take full advantage of the features provided by HttpClient.

Warning

Migrating from HttpWebRequest, ServicePoint, and ServicePointManager to HttpClient is not just a "nice to have" performance improvement. It's crucial to understand that the existing WebRequest logic's performance is likely to degrade significantly once you move to .NET (Core). That's because WebRequest is maintained as a minimal compatibility layer, which means it lacks many optimizations, such as connection reuse in numerous cases. Therefore, transitioning to HttpClient is essential to ensure your application's performance and resource management are up to modern standards.

Migrate from HttpWebRequest to HttpClient

Let's start with some examples:

Simple GET Request Using HttpWebRequest

Here's an example of how the code might look:

HttpWebRequest request = WebRequest.CreateHttp(uri);
using WebResponse response = await request.GetResponseAsync();

Simple GET Request Using HttpClient

Here's an example of how the code might look:

HttpClient client = new();
using HttpResponseMessage message = await client.GetAsync(uri);

Simple POST Request Using HttpWebRequest

Here's an example of how the code might look:

HttpWebRequest request = WebRequest.CreateHttp(uri);
request.Method = "POST";
request.ContentType = "text/plain";
await using Stream stream = await request.GetRequestStreamAsync();
await stream.WriteAsync("Hello World!"u8.ToArray());
using WebResponse response = await request.GetResponseAsync();

Simple POST Request Using HttpClient

Here's an example of how the code might look:

HttpClient client = new();
using HttpResponseMessage responseMessage = await client.PostAsync(uri, new StringContent("Hello World!"));

HttpWebRequest to HttpClient, SocketsHttpHandler migration guide

HttpWebRequest Old API New API Notes
Accept Accept Example: Set Request Headers.
Address RequestUri Example: Fetch Redirected URI.
AllowAutoRedirect AllowAutoRedirect Example: Setting SocketsHttpHandler Properties.
AllowReadStreamBuffering No direct equivalent API Usage of Buffering Properties.
AllowWriteStreamBuffering No direct equivalent API Usage of Buffering Properties.
AuthenticationLevel No direct equivalent API Example: Enabling Mutual Authentication.
AutomaticDecompression AutomaticDecompression Example: Setting SocketsHttpHandler Properties.
CachePolicy No direct equivalent API Example: Apply CachePolicy Headers.
ClientCertificates SslOptions.ClientCertificates Usage of Certificate Related Properties in HttpClient.
Connection Connection Example: Set Request Headers.
ConnectionGroupName No equivalent API No workaround
ContentLength ContentLength Example: Set Content Headers.
ContentType ContentType Example: Set Content Headers.
ContinueDelegate No equivalent API No workaround.
ContinueTimeout Expect100ContinueTimeout Example: Set SocketsHttpHandler Properties.
CookieContainer CookieContainer Example: Set SocketsHttpHandler Properties.
Credentials Credentials Example: Set SocketsHttpHandler Properties.
Date Date Example: Set Request Headers.
DefaultCachePolicy No direct equivalent API Example: Apply CachePolicy Headers.
DefaultMaximumErrorResponseLength No direct equivalent API Example: Set MaximumErrorResponseLength in HttpClient.
DefaultMaximumResponseHeadersLength No equivalent API MaxResponseHeadersLength can be used instead.
DefaultWebProxy No equivalent API Proxy can be used instead.
Expect Expect Example: Set Request Headers.
HaveResponse No equivalent API Implied by having an HttpResponseMessage instance.
Headers Headers Example: Set Request Headers.
Host Host Example: Set Request Headers.
IfModifiedSince IfModifiedSince Example: Set Request Headers.
ImpersonationLevel No direct equivalent API Example: Change ImpersonationLevel.
KeepAlive No direct equivalent API Example: Set Request Headers.
MaximumAutomaticRedirections MaxAutomaticRedirections Example: Setting SocketsHttpHandler Properties.
MaximumResponseHeadersLength MaxResponseHeadersLength Example: Setting SocketsHttpHandler Properties.
MediaType No direct equivalent API Example: Set Content Headers.
Method Method Example: Usage of HttpRequestMessage properties.
Pipelined No equivalent API HttpClient doesn't support pipelining.
PreAuthenticate PreAuthenticate
ProtocolVersion HttpRequestMessage.Version Example: Usage of HttpRequestMessage properties.
Proxy Proxy Example: Setting SocketsHttpHandler Properties.
ReadWriteTimeout No direct equivalent API Usage of SocketsHttpHandler and ConnectCallback.
Referer Referrer Example: Set Request Headers.
RequestUri RequestUri Example: Usage of HttpRequestMessage properties.
SendChunked TransferEncodingChunked Example: Set Request Headers.
ServerCertificateValidationCallback SslOptions.RemoteCertificateValidationCallback Example: Setting SocketsHttpHandler Properties.
ServicePoint No equivalent API ServicePoint is not part of HttpClient.
SupportsCookieContainer No equivalent API This is always true for HttpClient.
Timeout Timeout
TransferEncoding TransferEncoding Example: Set Request Headers.
UnsafeAuthenticatedConnectionSharing No equivalent API No workaround
UseDefaultCredentials No direct equivalent API Example: Setting SocketsHttpHandler Properties.
UserAgent UserAgent Example: Set Request Headers.

Migrate ServicePoint(Manager) usage

You should be aware that ServicePointManager is a static class, meaning that any changes made to its properties will have a global effect on all newly created ServicePoint objects within the application. For example, when you modify a property like ConnectionLimit or Expect100Continue, it impacts every new ServicePoint instance.

Warning

In modern .NET, HttpClient does not take into account any configurations set on ServicePointManager.

ServicePointManager properties mapping

ServicePointManager Old API New API Notes
CheckCertificateRevocationList SslOptions.CertificateRevocationCheckMode Example: Enabling CRL Check with SocketsHttpHandler.
DefaultConnectionLimit MaxConnectionsPerServer Example: Setting SocketsHttpHandler Properties.
DnsRefreshTimeout No equivalent API Example: Enabling Dns Round Robin.
EnableDnsRoundRobin No equivalent API Example: Enabling Dns Round Robin.
EncryptionPolicy SslOptions.EncryptionPolicy Example: Setting SocketsHttpHandler Properties.
Expect100Continue ExpectContinue Example: Set Request Headers.
MaxServicePointIdleTime PooledConnectionIdleTimeout Example: Setting SocketsHttpHandler Properties.
MaxServicePoints No equivalent API ServicePoint is not part of HttpClient.
ReusePort No direct equivalent API Usage of SocketsHttpHandler and ConnectCallback.
SecurityProtocol SslOptions.EnabledSslProtocols Example: Setting SocketsHttpHandler Properties.
ServerCertificateValidationCallback SslOptions.RemoteCertificateValidationCallback Both of them are RemoteCertificateValidationCallback
UseNagleAlgorithm No direct equivalent API Usage of SocketsHttpHandler and ConnectCallback.

Warning

In modern .NET, the default values for the UseNagleAlgorithm and Expect100Continue properties are set to false. These values were true by default in .NET Framework.

ServicePointManager method mapping

ServicePointManager Old API New API Notes
FindServicePoint No equivalent API No workaround
SetTcpKeepAlive No direct equivalent API Usage of SocketsHttpHandler and ConnectCallback.

ServicePoint properties mapping

ServicePoint Old API New API Notes
Address HttpRequestMessage.RequestUri This is request uri, this information can be found under HttpRequestMessage.
BindIPEndPointDelegate No direct equivalent API Usage of SocketsHttpHandler and ConnectCallback.
Certificate No direct equivalent API This information can be fetched from RemoteCertificateValidationCallback. Example: Fetch Certificate.
ClientCertificate No equivalent API Example: Enabling Mutual Authentication.
ConnectionLeaseTimeout SocketsHttpHandler.PooledConnectionLifetime Equivalent setting in HttpClient
ConnectionLimit MaxConnectionsPerServer Example: Setting SocketsHttpHandler Properties.
ConnectionName No equivalent API No workaround
CurrentConnections No equivalent API See Networking telemetry in .NET.
Expect100Continue ExpectContinue Example: Set Request Headers.
IdleSince No equivalent API No workaround
MaxIdleTime PooledConnectionIdleTimeout Example: Setting SocketsHttpHandler Properties.
ProtocolVersion HttpRequestMessage.Version Example: Usage of HttpRequestMessage properties.
ReceiveBufferSize No direct equivalent API Usage of SocketsHttpHandler and ConnectCallback.
SupportsPipelining No equivalent API HttpClient doesn't support pipelining.
UseNagleAlgorithm No direct equivalent API Usage of SocketsHttpHandler and ConnectCallback.

ServicePoint method mapping

ServicePoint Old API New API Notes
CloseConnectionGroup No equivalent No workaround
SetTcpKeepAlive No direct equivalent API Usage of SocketsHttpHandler and ConnectCallback.

Usage of HttpClient and HttpRequestMessage properties

When working with HttpClient in .NET, you have access to a variety of properties that allow you to configure and customize HTTP requests and responses. Understanding these properties can help you make the most of HttpClient and ensure that your application communicates efficiently and securely with web services.

Example: Usage of HttpRequestMessage properties

Here's an example of how to use HttpClient and HttpRequestMessage together:

var client = new HttpClient();

var request = new HttpRequestMessage(HttpMethod.Post, "https://example.com"); // Method and RequestUri usage
var request = new HttpRequestMessage() // Alternative way to set RequestUri and Method
{
    RequestUri = new Uri("https://example.com"),
    Method = HttpMethod.Post
};
request.Headers.Add("Custom-Header", "value");
request.Content = new StringContent("somestring");

using var response = await client.SendAsync(request);
var protocolVersion = response.RequestMessage.Version; // Fetch `ProtocolVersion`.

Example: Fetch redirected URI

Here's an example of how to fetch redirected URI (Same as HttpWebRequest.Address):

var client = new HttpClient();
using var response = await client.GetAsync(uri);
var redirectedUri = response.RequestMessage.RequestUri;

Usage of SocketsHttpHandler and ConnectCallback

The ConnectCallback property in SocketsHttpHandler allows developers to customize the process of establishing a TCP connection. This can be useful for scenarios where you need to control DNS resolution or apply specific socket options on the connection. By using ConnectCallback, you can intercept and modify the connection process before it is used by HttpClient.

Example: Bind IP address to socket

In the old approach using HttpWebRequest, you might have used custom logic to bind a specific IP address to a socket. Here's how you can achieve similar functionality using HttpClient and ConnectCallback:

Old Code Using HttpWebRequest:

HttpWebRequest request = WebRequest.CreateHttp(uri);
request.ServicePoint.BindIPEndPointDelegate = (servicePoint, remoteEndPoint, retryCount) =>
{
    // Bind to a specific IP address
    IPAddress localAddress = IPAddress.Parse("192.168.1.100");
    return new IPEndPoint(localAddress, 0);
};
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();

New Code Using HttpClient and ConnectCallback:

var handler = new SocketsHttpHandler
{
    ConnectCallback = async (context, cancellationToken) =>
    {
        // Bind to a specific IP address
        IPAddress localAddress = IPAddress.Parse("192.168.1.100");
        var socket = new Socket(localAddress.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
        try
        {
            socket.Bind(new IPEndPoint(localAddress, 0));
            await socket.ConnectAsync(context.DnsEndPoint, cancellationToken);
            return new NetworkStream(socket, ownsSocket: true);
        }
        catch
        {
            socket.Dispose();
            throw;
        }
    }
};
var client = new HttpClient(handler);
using var response = await client.GetAsync(uri);

Example: Apply specific socket options

If you need to apply specific socket options, such as enabling TCP keep-alive, you can use ConnectCallback to configure the socket before it is used by HttpClient. In fact, ConnectCallback is more flexible to configure socket options.

Old Code Using HttpWebRequest:

ServicePointManager.ReusePort = true;
HttpWebRequest request = WebRequest.CreateHttp(uri);
request.ServicePoint.SetTcpKeepAlive(true, 60000, 1000);
request.ServicePoint.ReceiveBufferSize = 8192;
request.ServicePoint.UseNagleAlgorithm = false;
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();

New Code Using HttpClient and ConnectCallback:

var handler = new SocketsHttpHandler
{
    ConnectCallback = async (context, cancellationToken) =>
    {
        var socket = new Socket(SocketType.Stream, ProtocolType.Tcp);
        try
        {
            // Setting TCP Keep Alive
            socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.KeepAlive, true);
            socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveTime, 60);
            socket.SetSocketOption(SocketOptionLevel.Tcp, SocketOptionName.TcpKeepAliveInterval, 1);

            // Setting ReceiveBufferSize
            socket.ReceiveBufferSize = 8192;

            // Enabling ReusePort
            socket.SetSocketOption(SocketOptionLevel.Socket, SocketOptionName.ReuseUnicastPort, true);

            // Disabling Nagle Algorithm
            socket.NoDelay = true;

            await socket.ConnectAsync(context.DnsEndPoint, cancellationToken);
            return new NetworkStream(socket, ownsSocket: true);
        }
        catch
        {
            socket.Dispose();
            throw;
        }
    }
};
var client = new HttpClient(handler);
using var response = await client.GetAsync(uri);

Example: Enable DNS round robin

DNS Round Robin is a technique used to distribute network traffic across multiple servers by rotating through a list of IP addresses associated with a single domain name. This helps in load balancing and improving the availability of services. When using HttpClient, you can implement DNS Round Robin by manually handling the DNS resolution and rotating through the IP addresses using the ConnectCallback property of SocketsHttpHandler.

To enable DNS Round Robin with HttpClient, you can use the ConnectCallback property to manually resolve the DNS entries and rotate through the IP addresses. Here's an example for HttpWebRequest and HttpClient:

Old Code Using HttpWebRequest:

ServicePointManager.DnsRefreshTimeout = 60000;
ServicePointManager.EnableDnsRoundRobin = true;
HttpWebRequest request = WebRequest.CreateHttp(uri);
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();

In the older HttpWebRequest API, enabling DNS Round Robin was straightforward due to its built-in support for this feature. However, the newer HttpClient API does not provide the same built-in functionality. Despite this, you can achieve similar behavior by implementing a DnsRoundRobinConnector that manually rotates through the IP addresses returned by DNS resolution.

New Code Using HttpClient:

// This is available as NuGet Package: https://www.nuget.org/packages/DnsRoundRobin/
// The original source code can be found also here: https://github.com/MihaZupan/DnsRoundRobin
public sealed class DnsRoundRobinConnector : IDisposable

You can find implementation of DnsRoundRobinConnector here.

DnsRoundRobinConnector Usage:

private static readonly DnsRoundRobinConnector s_roundRobinConnector = new(
        dnsRefreshInterval: TimeSpan.FromSeconds(10),
        endpointConnectTimeout: TimeSpan.FromSeconds(5));
static async Task DnsRoundRobinConnectAsync()
{
    var handler = new SocketsHttpHandler
    {
        ConnectCallback = async (context, cancellation) =>
        {
            Socket socket = await DnsRoundRobinConnector.Shared.ConnectAsync(context.DnsEndPoint, cancellation);
            // Or you can create and use your custom DnsRoundRobinConnector instance
            // Socket socket = await s_roundRobinConnector.ConnectAsync(context.DnsEndPoint, cancellation);
            return new NetworkStream(socket, ownsSocket: true);
        }
    };
    var client = new HttpClient(handler);
    HttpResponseMessage response = await client.GetAsync(Uri);
}

Example: Set SocketsHttpHandler properties

SocketsHttpHandler is a powerful and flexible handler in .NET that provides advanced configuration options for managing HTTP connections. By setting various properties of SocketsHttpHandler, you can fine-tune the behavior of your HTTP client to meet specific requirements, such as performance optimization, security enhancements, and custom connection handling.

Here's an example of how to configure SocketsHttpHandler with various properties and use it with HttpClient:

var cookieContainer = new CookieContainer();
cookieContainer.Add(new Cookie("cookieName", "cookieValue"));

var handler = new SocketsHttpHandler
{
    AllowAutoRedirect = true,
    AutomaticDecompression = DecompressionMethods.All,
    Expect100ContinueTimeout = TimeSpan.FromSeconds(1),
    CookieContainer = cookieContainer,
    Credentials = new NetworkCredential("user", "pass"),
    MaxAutomaticRedirections = 10,
    MaxResponseHeadersLength = 1,
    Proxy = new WebProxy("http://proxyserver:8080"), // Don't forget to set UseProxy
    UseProxy = true,
};

var client = new HttpClient(handler);
using var response = await client.GetAsync(uri);

Example: Change ImpersonationLevel

This functionality is specific to certain platforms and is somewhat outdated. If you need a workaround, you can refer to this section of the code.

When working with HttpClient, you may need to handle client certificates for various purposes, such as custom validation of server certificates or fetching the server certificate. HttpClient provides several properties and options to manage certificates effectively.

Example: Check certificate revocation list with SocketsHttpHandler

The CheckCertificateRevocationList property in SocketsHttpHandler.SslOptions allows developers to enable or disable the check for certificate revocation lists (CRL) during SSL/TLS handshake. Enabling this property ensures that the client verifies whether the server's certificate has been revoked, enhancing the security of the connection.

Old Code Using HttpWebRequest:

ServicePointManager.CheckCertificateRevocationList = true;
HttpWebRequest request = WebRequest.CreateHttp(uri);
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();

New Code Using HttpClient:

bool checkCertificateRevocationList = true;
var handler = new SocketsHttpHandler
{
    SslOptions =
    {
        CertificateRevocationCheckMode = checkCertificateRevocationList ? X509RevocationMode.Online : X509RevocationMode.NoCheck,
    }
};
var client = new HttpClient(handler);
using var response = await client.GetAsync(uri);

Example: Fetch certificate

To fetch the certificate from the RemoteCertificateValidationCallback in HttpClient, you can use the ServerCertificateCustomValidationCallback property of HttpClientHandler or SocketsHttpHandler.SslOptions. This callback allows you to inspect the server's certificate during the SSL/TLS handshake.

Old Code Using HttpWebRequest:

HttpWebRequest request = WebRequest.CreateHttp(uri);
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();
X509Certificate? serverCertificate = request.ServicePoint.Certificate;

New Code Using HttpClient:

X509Certificate? serverCertificate = null;
var handler = new SocketsHttpHandler
{
    SslOptions = new SslClientAuthenticationOptions
    {
        RemoteCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) =>
        {
            serverCertificate = certificate;

            // Leave the validation as-is.
            return sslPolicyErrors == SslPolicyErrors.None;
        }
    }
};
var client = new HttpClient(handler);
using var response = await client.GetAsync("https://example.com");

Example: Enable mutual authentication

Mutual authentication, also known as two-way SSL or client certificate authentication, is a security process in which both the client and the server authenticate each other. This ensures that both parties are who they claim to be, providing an additional layer of security for sensitive communications. In HttpClient, you can enable mutual authentication by configuring the HttpClientHandler or SocketsHttpHandler to include the client certificate and validate the server's certificate.

To enable mutual authentication, follow these steps:

  • Load the client certificate.
  • Configure the HttpClientHandler or SocketsHttpHandler to include the client certificate.
  • Set up the server certificate validation callback if custom validation is needed.

Here's an example using SocketsHttpHandler:

var handler = new SocketsHttpHandler
{
    SslOptions = new SslClientAuthenticationOptions
    {
        ClientCertificates = new X509CertificateCollection
        {
            // Load the client certificate from a file
            new X509Certificate2("path_to_certificate.pfx", "certificate_password")
        },
        RemoteCertificateValidationCallback = (sender, certificate, chain, sslPolicyErrors) =>
        {
            // Custom validation logic for the server certificate
            return sslPolicyErrors == SslPolicyErrors.None;
        }
    }
};

var client = new HttpClient(handler);
using var response = await client.GetAsync(uri);

Usage of Header Properties

Headers play a crucial role in HTTP communication, providing essential metadata about the request and response. When working with HttpClient in .NET, you can set and manage various header properties to control the behavior of your HTTP requests and responses. Understanding how to use these header properties effectively can help you ensure that your application communicates efficiently and securely with web services.

Set request headers

Request headers are used to provide additional information to the server about the request being made. Common use cases include specifying the content type, setting authentication tokens, and adding custom headers. You can set request headers using the DefaultRequestHeaders property of HttpClient or the Headers property of HttpRequestMessage.

Example: Set custom request headers

Setting Default Custom Request Headers in HttpClient

var client = new HttpClient();
client.DefaultRequestHeaders.Add("Custom-Header", "value");

Setting Custom Request Headers in HttpRequestMessage

var request = new HttpRequestMessage(HttpMethod.Get, uri);
request.Headers.Add("Custom-Header", "value");

Example: Set common request headers

When working with HttpRequestMessage in .NET, setting common request headers is essential for providing additional information to the server about the request being made. These headers can include authentication tokens and more. Properly configuring these headers ensures that your HTTP requests are processed correctly by the server. For a comprehensive list of common properties available in HttpRequestHeaders, see Properties.

To set common request headers in HttpRequestMessage, you can use the Headers property of the HttpRequestMessage object. This property provides access to the HttpRequestHeaders collection, where you can add or modify headers as needed.

Setting Common Default Request Headers in HttpClient

using System.Net.Http.Headers;

var client = new HttpClient();
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", "token");

Setting Common Request Headers in HttpRequestMessage

using System.Net.Http.Headers;

var request = new HttpRequestMessage(HttpMethod.Get, uri);
request.Headers.Authorization = new AuthenticationHeaderValue("Bearer", "token");

Example: Set content headers

Content headers are used to provide additional information about the body of an HTTP request or response. When working with HttpClient in .NET, you can set content headers to specify the media type, encoding, and other metadata related to the content being sent or received. Properly configuring content headers ensures that the server and client can correctly interpret and process the content.

var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Post, uri);

// Create the content and set the content headers
var jsonData = "{\"key\":\"value\"}";
var content = new StringContent(jsonData, Encoding.UTF8, "application/json");

// The following headers are set automatically by `StringContent`. If you wish to override their values, you can do it like so:
// content.Headers.ContentType = new MediaTypeHeaderValue("application/json; charset=utf-8");
// content.Headers.ContentLength = Encoding.UTF8.GetByteCount(jsonData);

// Assign the content to the request
request.Content = content;

using var response = await client.SendAsync(request);

Example: Set MaximumErrorResponseLength in HttpClient

The MaximumErrorResponseLength usage allows developers to specify the maximum length of the error response content that the handler will buffer. This is useful for controlling the amount of data that is read and stored in memory when an error response is received from the server. By using this technique, you can prevent excessive memory usage and improve the performance of your application when handling large error responses.

There are couple of ways to do that, we'll examine TruncatedReadStream technique on this example:

internal sealed class TruncatedReadStream(Stream innerStream, long maxSize) : Stream
{
    private long _maxRemainingLength = maxSize;
    public override bool CanRead => true;
    public override bool CanSeek => false;
    public override bool CanWrite => false;

    public override long Length => throw new NotSupportedException();
    public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); }

    public override void Flush() => throw new NotSupportedException();

    public override int Read(byte[] buffer, int offset, int count)
    {
        return Read(new Span<byte>(buffer, offset, count));
    }

    public override int Read(Span<byte> buffer)
    {
        int readBytes = innerStream.Read(buffer.Slice(0, (int)Math.Min(buffer.Length, _maxRemainingLength)));
        _maxRemainingLength -= readBytes;
        return readBytes;
    }

    public override Task<int> ReadAsync(byte[] buffer, int offset, int count, CancellationToken cancellationToken)
    {
        return ReadAsync(new Memory<byte>(buffer, offset, count), cancellationToken).AsTask();
    }

    public override async ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken = default)
    {
        int readBytes = await innerStream.ReadAsync(buffer.Slice(0, (int)Math.Min(buffer.Length, _maxRemainingLength)), cancellationToken)
            .ConfigureAwait(false);
        _maxRemainingLength -= readBytes;
        return readBytes;
    }

    public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException();
    public override void SetLength(long value) => throw new NotSupportedException();
    public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException();

    public override ValueTask DisposeAsync() => innerStream.DisposeAsync();

    protected override void Dispose(bool disposing)
    {
        if (disposing)
        {
            innerStream.Dispose();
        }
    }
}

And usage example of TruncatedReadStream:

int maxErrorResponseLength = 1 * 1024; // 1 KB

HttpClient client = new HttpClient();
using HttpResponseMessage response = await client.GetAsync(Uri);

if (response.Content is not null)
{
    Stream responseReadStream = await response.Content.ReadAsStreamAsync();
    // If MaxErrorResponseLength is set and the response status code is an error code, then wrap the response stream in a TruncatedReadStream
    if (maxErrorResponseLength >= 0 && !response.IsSuccessStatusCode)
    {
        responseReadStream = new TruncatedReadStream(responseReadStream, maxErrorResponseLength);
    }
    // Read the response stream
    Memory<byte> buffer = new byte[1024];
    int readValue = await responseReadStream.ReadAsync(buffer);
}

Example: Apply CachePolicy headers

Warning

HttpClient does not have built-in logic to cache responses. There is no workaround other than implementing all the caching yourself. Simply setting the headers will not achieve caching.

When migrating from HttpWebRequest to HttpClient, it's important to correctly handle cache-related headers such as pragma and cache-control. These headers control how responses are cached and retrieved, ensuring that your application behaves as expected in terms of performance and data freshness.

In HttpWebRequest, you might have used the CachePolicy property to set these headers. However, in HttpClient, you need to manually set these headers on the request.

Old Code Using HttpWebRequest:

HttpWebRequest request = WebRequest.CreateHttp(uri);
request.CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.NoCacheNoStore);
using HttpWebResponse response = (HttpWebResponse)request.GetResponse();

In the older HttpWebRequest API, applying CachePolicy was straightforward due to its built-in support for this feature. However, the newer HttpClient API does not provide the same built-in functionality. Despite this, you can achieve similar behavior by implementing a AddCacheControlHeaders that manually add cache related headers.

New Code Using HttpClient:

public static class CachePolicy
{
    public static void AddCacheControlHeaders(HttpRequestMessage request, RequestCachePolicy policy)

You can find implementation of AddCacheControlHeaders here.

AddCacheControlHeaders Usage:

static async Task AddCacheControlHeaders()
{
    HttpClient client = new HttpClient();
    HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Get, Uri);
    CachePolicy.AddCacheControlHeaders(requestMessage, new HttpRequestCachePolicy(HttpRequestCacheLevel.NoCacheNoStore));
    HttpResponseMessage response = await client.SendAsync(requestMessage);
}

Usage of buffering properties

When migrating from HttpWebRequest to HttpClient, it's important to understand the differences in how these two APIs handle buffering.

Old Code Using HttpWebRequest:

In HttpWebRequest, you have direct control over buffering properties through the AllowWriteStreamBuffering and AllowReadStreamBuffering properties. These properties enable or disable buffering of data sent to and received from the server.

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.AllowReadStreamBuffering = true; // Default is `false`.
request.AllowWriteStreamBuffering = false; // Default is `true`.

New Code Using HttpClient:

In HttpClient, there are no direct equivalents to the AllowWriteStreamBuffering and AllowReadStreamBuffering properties.

HttpClient does not buffer request bodies on its own, instead delegating the responsibility to the HttpContent used. Contents like StringContent or ByteArrayContent are logically already buffered in memory, while using StreamContent will not incur any buffering by default. To force the content to be buffered, you may call HttpContent.LoadIntoBufferAsync before sending the request. Here's an example:

HttpClient client = new HttpClient();

HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, uri);
request.Content = new StreamContent(yourStream);
await request.Content.LoadIntoBufferAsync();

HttpResponseMessage response = await client.SendAsync(request);

In HttpClient read buffering is enabled by default. To avoid it, you may specify the HttpCompletionOption.ResponseHeadersRead flag, or use the GetStreamAsync helper.

HttpClient client = new HttpClient();

using HttpResponseMessage response = await client.GetAsync(uri, HttpCompletionOption.ResponseHeadersRead);
await using Stream responseStream = await response.Content.ReadAsStreamAsync();

// Or simply
await using Stream responseStream = await client.GetStreamAsync(uri);