Unable to authenticate using FBA with LDAP on ISA Server 2006
Introduction
Consider a scenario where users are not able to authenticate using FBA with LDAP to access the websites published through ISA server. In this scenario ISA server was part of one domain (contoso.com) and users who would access the website are part of another domain (fabrikam.com). FBA with LDAP is used on the web listener of the web publishing rule to authenticate the users from fabrikam.com’s domain controller. But these users are not able to authenticate using this method.
Note: More about Ldap authentication please refer http://technet.microsoft.com/hi-in/library/bb794854(en-us).aspx#ldap
Data Collection
To troubleshoot the issue took network traces while creating LDAP user set on the ISA server (reference http://technet.microsoft.com/hi-in/library/bb794854(en-us).aspx#LDAPUser), which failed with error, “access to LDAP server is denied.”
Data Analysis
In the network captures we found:
1. LDAP Bind Request as below
2. LDAP Bind Response as below
Troubleshooting and Resolution
We tested the user credentials from a machine which is already part of the fabrikam domain and we were able to authenticate using same credentials. Then as per http://blogs.technet.com/b/isablog/archive/2008/04/17/isa-server-2006-form-base-authentication-problem-using-upn-logon-format-on-a-multiple-domain-environment.aspx we checked the HKLM\System\CCS\CONTROL\LSA\LMCompatibilityLevel on the ISA server and it was set to 0x2 (only allow LM and NTLM). Then checked same on the Domain Controller (which it was windows 2008 server) of the domain where users were located and it was set to 0x5(only allow NTLMv2 and block LM /NTLM).Since Domain controller only allows NTLM v2 it was not authenticating the request coming from the ISA server which was sending it with NTLM v1 as we can see in the NTLM challenge response in the network traces.
To resolve the problem we set LMCompatibilityLevel key on the ISA server to 0x3 (although we could also set it to 0x4) and restarted the ISA server. After that users were able to normally logon via FBA.
Author
Suraj Singh
Support Engineer
Microsoft CSS Forefront Security Edge Team
Technical Reviewer
Yuri Diogenes
Sr Support Escalation Engineer
Microsoft CSS Forefront Security Edge Team