Share via

SharePoint 2013: Crawl file shares for Claims ADFS Users

  • Article

Introduction

Recently one customer requested to crawl his file system with SharePoint Server 2013. As the SharePoint infrastructure was Federated with AD FS 3.0, search request was made under a SAML identity anddocuments will never appear in their search results. 

Background

When you crawl a SharePoint site, it will need to be configured to present NTLM authentication for the crawler to make this work. In a ADFS Trusted Provider Deployment, you will need to extend or present double authentication on one zone for the crawl to work.

But what happens when you need to crawl a file system when users are logged on with AD FS?

Remember: Search in SharePoint Server 2013 performs security trimming, which prevents users from seeing information that they do not have access to in the system".

The solution to make it work is really simple when we know. But nobody was previously explaining how to do, so let's demonstrate.

SharePoint Farm - Prerequisites

  • SharePoint Server 2013 SP1.
  • Federation Services 3.0 (should work with 2.0).
  • A Basic Search Center (https://search.domain.com).
  • Search Service Application.
  • File Share to crawl with Read Access.
  • A relying trust for the WebApplication.

The farm is also setup with the two solutions (LDAPCP.WSP andClaimsViewerWebPart) and other related configuration. Including the WebApplicationProxy (Federation Proxy). We will not describe here the configuration for the SharePoint farm to work under AD FS. Please refer here for the general setup.

The File System - Permissions

To demonstrate we have created two folders to index. Permissions are assigned differently between these folders. One is only set with user permissions and the other one is only accessible per Group Permission. In a perfect IT world, all permissions are set with the group. But for testing purpose, we have set up one folder with the only user account.

  • The content source within SharePoint point to : \server.domain.local\SHARE

Notes :

  • Document have inherited permissions and users have Full Control over each file.
  • User 1 is Member of "GS_Access_SHARE_FC" Group.
  • Don't forget to add "Manage auditing and security logs" in local policy if the share is present in a Windows Server.
  • Default content access account "read" NTFS Permission.

The Problem: Missing results with AD FS

The crawl operates on the file system with the Default Content Access Account and uses the NTLM Authentication to access the file. When a user searches across the file share through SharePoint, the results will be trimmed with the user permission onthe file system.

So far so good, this is expected! ... BUT :

  • A logged on NTLM user will search for a file and retrieve all the indexed file where he has permission to read. Either permission is nominative or bulked with Groups.
  • A logged on ADFS user will not retrieve any file.

The administrator will go to crawl log and see the missing file are correctly crawled by the Search Service Application. On the first tests only files with explicit user permission was returned by the search page.

Demonstration with a Basic Search Center

To demonstrate the problem, I've created a Basic Search Centerboth NTLM users or AD FS users can login and search across all documents. You can activate double authentication on the Default one for the diagnostic purpose. This will help you to gather the correct claims for each user logged.

  • Note:  This also happens in any other search context or search scope when results come from the filesystem.

     

For helping to diagnostic what is going on, we will suggest you use a claims viewer tools (webpart or plugins). We are using the helloitsliam.ClaimsViewerWebPart (now offline), a targeted solution for a site collection feature that will enable you to get the Claims issued for each user with a good looking display.

You will see the following for the Claims :

  • For a NTLM User you will see the following Claims (truncated):
 Claims Claims Value Issuer  OriginalIssuer
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier  DOMAIN\user1  SharePoint  SharePoint
http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid   S-1-5-21-4043082085-373456638-2456788878-1123  SharePoint  Windows
http://schemas.microsoft.com/ws/2008/06/identity/claims/primarygroupsid  S-1-5-21-4043082085-373456638-2456788878-513  SharePoint  Windows
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn  user1@domain.local  SharePoint  Windows
http://schemas.microsoft.com/sharepoint/2009/08/claims/userlogonname  DOMAIN\user1  SharePoint  SecurityTokenService
http://schemas.microsoft.com/sharepoint/2009/08/claims/useridq  0#.w|DOMAIN\user1  SharePoint  SecurityTokenService
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name  0#.w|DOMAIN\user1  SharePoint  SecurityTokenService
  • For a ADFS User you will see the following Claims (truncated):
 Claims Claims Value Issuer  OriginalIssuer
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier  user1@domain.com  SharePoint TrustedProvider:
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress  user1@domain.com  SharePoint TrustedProvider:
 http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn  user1@domain.local  SharePoint TrustedProvider:
 http://schemas.microsoft.com/ws/2008/06/identity/claims/role  DOMAIN\Domain Users  SharePoint TrustedProvider:
http://schemas.microsoft.com/sharepoint/2009/08/claims/userid  05.t|domain_sts|user1@domain.com  SharePoint SecurityTokenService
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name   05.t|domain_sts|user1@domain.com  SharePoint SecurityTokenService
 http://schemas.microsoft.com/sharepoint/2009/08/claims/identityprovider  trusted:DOMAIN_STS  SharePoint SecurityTokenService

The difference occurs on the two following claims:

These claims are not present when the user is logged with AD FS TrustedProvider.

AD FS and Claims Rules - Configuration

The results occur with the default setup: emailaddress, upn, role and primarysid as IncomingClaimType and with the default Claims Rules in the SPTrustedIdentityTokenIssuer.

$emailClaimMap= New-SPClaimTypeMapping -IncomingClaimType "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress"-IncomingClaimTypeDisplayName "EmailAddress" -SameAsIncoming
$upnClaimMap= New-SPClaimTypeMapping -IncomingClaimType "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn"-IncomingClaimTypeDisplayName "UPN" -SameAsIncoming
$roleClaimMap= New-SPClaimTypeMapping -IncomingClaimType "http://schemas.microsoft.com/ws/2008/06/identity/claims/role"-IncomingClaimTypeDisplayName "Role" -SameAsIncoming
$sidClaimMap= New-SPClaimTypeMapping -IncomingClaimType "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid"-IncomingClaimTypeDisplayName "SID"

realm ="urn:sharepoint:<WebAppName>"
$signInURL= "https://<YourADFSServerName>/adfs/ls"
$ap= New-SPTrustedIdentityTokenIssuer -Name <ProviderName> -Description <ProviderDescription> -realm$realm -ImportTrustCertificate$cert -ClaimsMappings$emailClaimMap,$upnClaimMap,$roleClaimMap,$sidClaimMap-SignInUrl $signInURL-IdentifierClaim $emailClaimmap.InputClaimType

c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer == "AD AUTHORITY"]
=> issue(store = "Active Directory", types = ("http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress", "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn", "http://schemas.microsoft.com/ws/2008/06/identity/claims/role"), query = ";mail,userPrincipalName,tokenGroups(domainQualifiedName);{0}", param = c.Value);

The Solution - Pass through Claims for Group Membership

As seen above in the different Claims for each logged user, you will need to add these in the Claims Rules:

  • In AD FS: Add a new LDAP Rules to Send GroupSID. Create two additional rules under the first "Send LDAP attributes as Claims"

2 x Pass-Through Or Filter incoming Claims

c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarysid"]

=> issue(claim = c);

c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarygroupsid"]

=> issue(claim = c);

  • In SharePoint: Add a new SPClaimTypeMapping to your SPTrustedIdentityTokenIssuer
$sts= Get-SPTrustedIdentityTokenIssuer "ADFS_NAME"
$sts.ClaimTypes.Add("http://schemas.microsoft.com/ws/2008/06/identity/claims/primarygroupsid")
$sts.Update()
$map= New-SPClaimTypeMapping -IncomingClaimType "http://schemas.microsoft.com/ws/2008/06/identity/claims/primarygroupsid"-IncomingClaimTypeDisplayName "Primary group SID" -SameAsIncoming
Add-SPClaimTypeMapping -Identity$map -TrustedIdentityTokenIssuer$sts

Note: you will need only to do it for Primary Group SID as the PrimarySID is already created with the TechNet procedure.

  • IISReset on SharePoint for the new ClaimType take effect.
  • Re-try the AD FS user and you will see correct results based on user & group permissions.