Getting error 12202 intermittently (Authentication Failure) while accessing published resources (SharePoint/Exchange) through TMG 2010

 

I was working on a case where we were seeing an intermittent failure with authentication for the websites (Exchange / Sharepoint ) that were published on TMG Server.

The error generated was:

12202 The Forefront TMG denied the specified Uniform Resource Locator (URL)

Environment:

TMG 2010 , Domain Joined (2003 and 2008 Domain Controllers)
Kerberos Constraint Delegation (KCD) configured as the Delegation method in the Publishing rules.
Exchange / Sharepoint Publishing.

Data Collection :

We collected a Network trace on Internal network of the TMG Server and noticed the following error returned from the DC:

KerberosV5:KRB_ERROR - KDC_ERR_ETYPE_NOSUPP (14)

This error indicates an encryption type negotiation mismatch occurred.

After analyzing the data further, we found that this error only occurred when the TMG server had a secure channel setup with a Windows 2003 Domain Controller.  To verify TMGs’ secure channel, you can run the following NLTEST command:

nltest /sc_verify:domainname

More information:

When the TMG Server (which runs on Windows 2008 server) attempts to establish Kerberos communication, it defaults to AES encryption; however Windows 2003 Domain Controllers supports DES encryption.  Therefore, if the TMG server has a secure channel with a Windows 2003 DC, the negotiation will fail because of the encryption method mis-match.  If the TMG server has a secure channel with a Windows 2008 DC, the negotiation will succeed because they will both default to DES encryption.

Resolution:

To fix the issue , we had two options :

1. Upgrade all DC's to 2008

2. Enable all encryption types on the TMG Server which is a 2008 R2 machine.

Procedure to enable encryption type on Windows Server 2008 R2:

1. Start > Gpedit- which opens local policy, locate the following location:
Computer Configuration\ Windows Settings\ Security Settings\ Local Policies\ Security Options
2. Click to select the Network security: Configure encryption types allowed for Kerberos option.
3. Click to select Define these policy settings and all the six check boxes for the encryption types.
4. Click OK. Close the Gpedit.
5. Reboot the server.

Note:  The policy sets the SupportedEncryptionTypes registry entry to a value of 0x7FFFFFFF. The SupportedEncryptionTypes registry entry is at the following location:
HKLM\Software\Microsoft\Windows\CurrentVersion\Policies\System\Kerberos\parameters\

Ref: http://support.microsoft.com/kb/977321

Author

Junaid Ahmad Jan - Security Support Escalation Engineer, Microsoft CSS Forefront Security Edge Team

Reviewer

Richard Barker - Sr Security Support Escalation Engineer, Microsoft CSS Forefront Security Edge Team

Comments

  • Anonymous
    January 01, 2003
    very informative!!

  • Anonymous
    August 15, 2012
    Really nice..!!! Never knew that before..

  • Anonymous
    August 21, 2012
    In paragraph: If the TMG server has a secure channel with a Windows 2008 DC, the negotiation will succeed because they will both default to DES encryption. Does it mean "defaults to AES encryption"?

  • Anonymous
    May 17, 2013
    exactly what i was thinking seth, i think its a typo; "If the TMG server has a secure channel with a Windows 2008 DC, the negotiation will succeed because they will both default to DES encryption." should be "If the TMG server has a secure channel with a Windows 2008 DC, the negotiation will succeed because they will both default to AES encryption."