Clients Are Not Prompted to Choose a Certificate When Authenticating to ISA/TMG

 

Recently I have been seeing an increasing number of cases with the same symptom especially in the military and the government sector and even in contractors for the government. In these highly secure environments clients largely rely on the use of a “smart” card known as Common Access Cards (CAC) for authentication to their various types of servers and services.

Symptom

Your Internet Security and Acceleration Server (ISA) or Forefront Threat Management Gateway 2010 (TMG) Server is publishing resources internally/externally and your Web Listener is configure to use SSL Client Certificate Authentication. When clients navigate to the site that is published they would normally be prompted to choose their client certificate. Some or all of the clients are not being prompted to choose the certificate. On the ISA/TMG server, you may see a Warning in your Event Log with an Event ID of 36885.

Event Type: Warning

Event Source: Schannel Event Category: None Event ID: 36885 Date: date Time: time User: Computer: COMPUTERNAME Description: When asking for client authentication, this server sends a list of trusted certificate authorities to the client. The client uses this list to choose a client certificate that is trusted by the server. Currently, this server trusts so many certificate authorities that the list has grown too long. This list has thus been truncated. The administrator of this machine should review the certificate authorities trusted for client authentication and remove those that do not really need to be trusted.

Cause

This issue is caused when there are too many trusted certificate authorities in the Certificate Store on ISA/TMG. This is particularly common for servers that need a long list of Department of Defense (DoD) Certificate Authorities. When the list grows beyond 12,228 bytes (the maximum size the current Schannel security package supports) the list will be truncated. If the client doesn't receive the root CA that it needs because it has been truncated, it will not prompt to choose the certificate.

Resolution

There are a few workarounds for this but the one that is easiest to implement and seems to fit the needs of most organizations is below.

On the server or servers that are running ISA/TMG you will need to set the following registry entry to 0 (false):

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL

Value name: SendTrustedIssuerList

Value type: REG_DWORD Value data: 0

By default the value is 1 (true).

For other possible workarounds please see this KB:

http://support.microsoft.com/kb/933430

Conclusion

Troubleshooting SSL Client Certificate issues can be tricky and time consuming. This issue was certainly difficult to identify the first time I saw it. Hopefully the information I have given you here can save you time, money, and aggravation.

 

Author:

Keith Abluton:

Security Support Escalation Engineer - MSD Security Team

Reviewer:

Richard Barker

Sr. Security Support Escalation Engineer - MSD Security Team