Share via

Faster proxy detection for HttpWebRequest

  • Article

Do a web request using System.Net classes, HttpWebRequest or WebClient. For example:

new WebClient().DownloadFile("https://tinyurl.com/d5yy8a", @"C:\Temp\file.xml");

Or:

HttpWebRequest.Create("https://tinyurl.com/d5yy8a").GetResponse();

If the first request takes more than 4 seconds and you think is too much time for your application, then the following information may interest you.

The reason the first request takes more time is because it tries, by default, to detect the proxy setting in a weird way. The framework uses an embedded JavaScript code for doing the detection, but the first time it does it’s job, it spends more time “dynamically compiling” the JavaScipt code than executing the actual proxy detection logic.

One way to skip this detection is by setting the proxy settings yourself, either by code or by using a configuration file like this:

<configuration>
  <system.net>
    <defaultProxy>
      <proxy 
proxyaddress="https://ContosoProxy:80"
autoDetect="False" />
</defaultProxy>
  </system.net>
</configuration>

If that solution doesn’t suits you, another way is do the proxy detection yourself (well, with the help of WinHttp.dll). It happens that there is a library in Windows XP, 2003, Vista, 7, called WinHttp.dll that exposes useful methods for proxy detection and in order to use it we just need some P/Invoke code and P/Invoke is by far faster than compiling the embedded JavaScript.

WinHttpGetProxyForUrl can be used for detecting the proxy for an URL and the function WinHttpGetIEProxyConfigForCurrentUser is useful if you want to read the proxy configuration from Internet Explorer. In this example I won’t do the IE proxy configuration reading just to keep it simple.

WinHttpGetProxyForUrl return a list of proxies to use. That list may look like a list of IP address separated by colons. In the example below I split that list and pre-append the https:// suffix if needed so it can became a valid URL. This example just picks the first proxy address and uses it. You can later on add logic to try multiple proxies if the request fails, just remember that once you use a HttpWebRequest object, you cannot re-use it, you need to create a new one and also keep the tracking of the all the timeout so you don’t end waiting 1 hour before timeout. If you use WebClient, you can override the method GetWebResponse and set the proxy there, just don’t eat the WebException exceptions that have a Status equals WebExceptionStatus.ProtocolError since that is actually a valid response.

Using normal System.Net proxy detection I get the output:

MS:7537
MS:41
MS:39
…

Using WinHttp.dll I get:

MS:410
MS:11
MS:11
…

Here you have:

https://code.msdn.microsoft.com/WinHttp

Comments