Freigeben über

How does maxconnection work for a System.Net.HttpWebRequest using a proxy?

  • Artikel

When using a WebRequest, one important thing to keep in mind is how many connections are allowed to be made to the same server.  The maxConnection setting will affect how many connections you can concurrently have to a given server.  When you set up a proxy, System.Net sees the proxy as the server and so it will limit the number of concurrent connections to that instead of all the various servers.

For example, doing something like this:

 HttpWebRequest wr = (HttpWebRequest)WebRequest.Create(requestUri);
wr.Proxy = new WebProxy("https://myproxy");

will cause the connection to funnel through a proxy.  This means that from the IIS perspective, the “client” is connecting to the proxy as the server.  So the number of connections, as set by maxConnection, will be applied to that proxy, not the Uri that it is eventually going to.  This is based off the RFC: RFC 2616, the RFC for Hypertext Transfer Protocol -- HTTP/1.1:

"Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy. A proxy SHOULD use up to 2*N connections to another server or proxy, where N is the number of simultaneously active users. These guidelines are intended to improve HTTP response times and avoid congestion."

So a few suggesting of what we could do:

- we could just increase maxConnection to 500 or 800 or whatever would be needed, knowing that the proxy can handle that much and pass the requests through.

- We could switch to using an NAT and route the requests through the proxy that way, using Big IP or Cisco Local Director or something like that.  Then we wouldn’t set the proxy on the HttpWebRequest and so it would not see them all as going to one machine.

If you have other suggested methods to solve this, or any thoughts around it, I’d like to hear them.

Comments

  • Anonymous
    September 17, 2008
    It's so helpful to see the real-world solutions that are used.  Thanks for sharing with the Community : )

  • Anonymous
    September 17, 2008
    Hi Tom, I have 2 questions: According to MSDN: <quote> If the Proxy property is specified, then the proxy settings from the Proxy property override the local computer or application config file and the HttpWebRequest instance will use the proxy settings specified. If no proxy is specified in a config file and the Proxy property is unspecified, the HttpWebRequest class uses the proxy settings inherited from Internet Explorer on the local computer. If there are no proxy settings in Internet Explorer, the request is sent directly to the server. </quote>

  1. Wouldn't a best practice be to always set the MaxConnections in the config file just because you can't be sure how things outside of your control are configured?
  2. How does the MaxConections setting interact with HTTP KeepAlives if KeepAlives are enabled on the other end of the wire?

  • Anonymous
    September 17, 2008
    Daveblack, For #1, that can be a good best practice, but you have to remember that this setting is set in machine.config so it will affect all .NET applications that use System.Net.  So you may not want to allow other applications to connect that many times. For #2, great question.  My understanding is that if you have a loop and make multiple requests on the same System.Net object, KeepAlives will be used to go on the same TCP port.  If you have multiple unique requests coming in, they will not reuse the TCP port.  Although I am not 100% sure of that.

  • Anonymous
    September 17, 2008
    I have searched the net and I should say I have not come across an article like this which is so easy to understand and learn the concepts. cheers!!

  • Anonymous
    September 17, 2008
    Hi Tom, According to MSDN, you can use the "connectionManagement" element in both the application config as well as the machine.config.  See http://msdn.microsoft.com/en-us/library/fb6y0fyc.aspx

  • Anonymous
    September 17, 2008
    Daveblack, While that is true, the bulk of the time, it will get changed in machine.config and then you will forget about it and when a new application is deployed, it will use the same settings.