Bi-Directional Affinity in ISA Server

Article
03/12/2008

The reason for having a network load balancing (NLB) Bi-Directional Affinity (BDA) feature in ISA Server is that it guarantees that traffic is handled in both directions by the same array server. This is very important and required if you have a Route Relationship (instead of NAT) between two ISA Server networks (for example between the DMZ and the External network) or if you use a Publishing Rule (Server Publishing or Web Publishing) that is configured with “Requests appear to come from the original client” (this is the default setting for a Server Publishing rule) instead of “Requests appear to come from the ISA Server computer” (this is the default setting for a Web Publishing rule). If for example the BDA feature was not available or for some reason not correctly configured in ISA Server, then you could potentially see that incoming traffic from an external client was handled by one array member, while the return traffic from an internal published server (that is configured to route traffic back out through ISA’s internal Virtual NLB IP as it typically will and should be) back to that external client was handled by another array member. This would cause the connection to fail as there is no connection sharing between ISA array members. Because of this we need to guarantee that traffic is handled in both directions by the same array server, and this is what BDA does for us when we have a Route Relationship between two ISA Server networks or if we use “Requests appear to come from the original client” in a Publishing rule.

Note that when using a Route Relationship between two ISA Server networks or using “Requests appear to come from the original client” in a Publishing rule, then you should make sure that you enable Integrated NLB on both the Source and Destination network interface for the involved networks. Otherwise BDA will likely fail and you may see strange symptoms. If for example you only enable NLB on one network when having a Route Relationship between two networks, then you should see a “NLB Inconsistent Configuration Detected” ISA Server alert being triggered similar to the following:

NLB Inconsistent Configuration Detected

An inconsistency in the Network Load Balancing (NLB) configuration may result in inconsistent handling of traffic between the External network and the DMZ network. When a network rule specifying a route relationship is defined between two networks, NLB must be enabled (or disabled) on both networks. To enable NLB for IPSec remote site networks, enable NLB on the network containing the local tunnel endpoint. To enable NLB for VPN site-to-site and VPN client networks, enable NLB on the selected access networks. Alternatively, for the VPN Client network, you can designate a router for routing traffic according to the static address pool.

Using Integrated NLB in ISA Server means that only the NLB single affinity scheme is supported to distribute the load among the array servers. This is a BDA requirement. Single affinity basically means that only a single IP address (source or destination) is used to make load distribution decisions. ISA Server uses NLB Hook Rules to make decisions either it will use the source IP address or destination IP address as affinity, and you can list these NLB Hook Rules by running “FWEngMon /QueryNLB” from a Command Prompt on one of the ISA Servers in the array. That we hash the affinity based on the source IP address for incoming traffic and based on the destination IP address for the return traffic (or vice versa) is actually how BDA works and what guarantees that traffic is handled in both directions by the same array server.

FWEngMon.exe is available both for ISA Server 2006 and ISA Server 2004 and can be downloaded here:

http://www.microsoft.com/downloads/details.aspx?familyid=01fc5551-5d44-4a99-966a-bd86caeb43d7&displaylang=en

http://www.microsoft.com/downloads/details.aspx?familyid=f3306399-d4f9-4989-865e-c61f8293c330&displaylang=en

When using Integrated NLB, ISA Server works with Windows NLB to automatically configure Bi-Directional Affinity (BDA). Therefore, you as an ISA Server administrator typically do not have to do any additional steps. You just have to make sure you enable Integrated NLB on the ISA Server network interfaces in ISA Management Console and BDA will work by default and by itself. However there’s always exceptions and you may still experience unexpected behavior as you will see below.

Let us take the following scenario to try and explain the above NLB Hook Rules further. Let us say you run two ISA Servers in an array with NLB enabled on all network interfaces. You have a Network Rule using a Route Relationship between the DMZ network and the External network. The DMZ network is defined as Source Network and the External network is defined as Destination Network in the Network Rule. In the DMZ you have a single server that you access from multiple external clients through an Access Rule in ISA Server. You have observed that only one of the ISA Server computers seems to handle all the load instead of the load being equally distributed between the two ISA Server nodes. If you change the configuration to use a Server Publishing Rule instead of an Access Rule, then you notice that the load is being equally distributed between the two nodes. Based on this you run a “FWEngMon /QueryNLB” from a Command Prompt and find the following (We have only listed the relevant NLB Hook Rules. Please note that the NLB Hook Rules will look different depending on your configuration and Networks definitions in ISA Server):

Hash on source .... (forward): 23.1.1.0--23.1.1.255 -> 0.0.0.0--10.1.0.255

Hash on source .... (forward): 23.1.1.0--23.1.1.255 -> 10.1.2.0--10.255.255.254

Hash on source .... (forward): 23.1.1.0--23.1.1.255 -> 11.0.0.0--23.1.0.255

Hash on source .... (forward): 23.1.1.0--23.1.1.255 -> 23.1.2.0--23.255.255.254

Hash on source .... (forward): 23.1.1.0--23.1.1.255 -> 24.0.0.0--126.255.255.255

Hash on source .... (forward): 23.1.1.0--23.1.1.255 -> 128.0.0.0--255.255.255.254

Hash on destination (reverse): 0.0.0.0--10.1.0.255 -> 23.1.1.0--23.1.1.255

Hash on destination (reverse): 10.1.2.0--10.255.255.254 -> 23.1.1.0--23.1.1.255

Hash on destination (reverse): 11.0.0.0--23.1.0.255 -> 23.1.1.0--23.1.1.255

Hash on destination (reverse): 23.1.2.0--23.255.255.254 -> 23.1.1.0--23.1.1.255

Hash on destination (reverse): 24.0.0.0--126.255.255.255 -> 23.1.1.0--23.1.1.255

Hash on destination (reverse): 128.0.0.0--255.255.255.254 -> 23.1.1.0--23.1.1.255

In this example 23.1.1.0--23.1.1.255 is your DMZ network, and you can therefore see from the above that ISA Server will actually hash the IP affinity based on the IP address of the server in the DMZ and not the IP addresses of the external clients. This means that even if you have thousands of external clients accessing the single server in the DMZ, ISA Server will always hash the affinity based on the DMZ range. If you only have one server in the DMZ that is being accessed, this means that a single IP address is used to make load balancing distribution decisions. In turn explaining why all the traffic is always handled by only one of the ISA Servers.

Because of this you try to change the Access Rule to use a Server Publishing Rule (configured with “Requests appear to come from the original client”) instead, and as mentioned above the load is now distributed equally between the two ISA Server nodes. In order to explain this we run “FWEngMon /QueryNLB” again. We can still see the same rules as in the case when using the Access Rule instead of the Server Publishing Rule. However you will also notice additional NLB Hook Rules that are created specifically for the Server Published Server on IP address 23.1.1.20. As you can now see we hash the IP affinity based on the external IP address ranges instead of the IP address of the published DMZ Server (23.1.1.20) and because of this the load is distributed equally between the two ISA Server nodes.