Creating a Custom Pre-Security Trimmer for SharePoint 2013
What You Will Learn
This blog post will show you how to write your own custom security pre-trimmer for SharePoint 2013. We will take you through the steps of deploying and registering the trimmer before putting the trimmer to work.
Please visit the official MSDN documentation for the overview and definitive source of documentation of this feature: |
Why Use Pre-Security Trimmers
Pre-trimming refers to pre-query evaluation where the backend rewrites the query adding security information before the index lookup in the search index. Post-trimming refers to post-query evaluation where search results are pruned before they are returned to the user.
We recommend the use of pre-trimming for performance and general correctness; pre-trimming prevents information leakage for refiner data and hit count instances.
Requirements
- SharePoint 2013 Server
- Visual Studio 2012
- A Custom Connector sending claims (see previous post on this blog)
The Trimmer Design
Let's create a simple pre-security trimmer. A trimmer that reads group membership data from a text file, performs a user lookup for group membership data and then adds claims to the query tree based upon this. In short, the trimmer code needs to figure out which user that is issuing the query, then perform a group membership lookup on that user and then add claims for that user, depending on the group membership.
The Code
This MSDN article offers useful starting tips on creating the security pre-trimmer project in Visual Studio, by adding references to both the Microsoft.Office.Server.Search.dll and the Microsoft.IdentityModel.dll.
Add the following to the using directives at the beginning of the class file, SearchPreTrimmer.cs:
using System.Security.Principal;
using Microsoft.IdentityModel.Claims;
using Microsoft.Office.Server.Search.Administration;
using Microsoft.Office.Server.Search.Query;
We then define the class as implementing the ISecurityTrimmerPre interface in the class declaration:
public class XmlContentSourcePreTrimmer : ISecurityTrimmerPre
We have to define a few constants at the beginning of the class. These variables may be altered by the static properties given when the trimmer is registered with SharePoint.
private string _claimType = "https://surface.microsoft.com/security/acl";
private string _claimIssuer = "customtrimmer";
private string _claimValueType = ClaimValueTypes.String;
private string _lookupFilePath = "datafile.txt";
The initialization method of the trimmer may modify a few "constant variables", primarily claim type and issuer along with the file path to the input data of this trimmer's group membership data:
/// <summary>
/// Initialize the pre-trimmer.
/// </summary>
/// <param name="staticProperties">Static properties configured for the trimmer.</param>
/// <param name="searchApplication">Search Service Application object</param>
public void Initialize(NameValueCollection staticProperties, SearchServiceApplication searchApplication)
{
if (staticProperties.Get("claimtype") != null)
{
_claimType = staticProperties.Get("claimtype");
}
if (staticProperties.Get("claimissuer") != null)
{
_claimIssuer = staticProperties.Get("claimissuer");
}
if (staticProperties.Get("datafile") != null)
{
_lookupFilePath = staticProperties.Get("datafile");
}
RefreshDataFile();
}
The AddAccess method of the trimmer is responsible for returning claims to be added to the query tree. We will refresh the group membership data if needed and figure out the user id for key lookup into the group membership structure from the text file.
/// <summary>
/// Add custom claims to the query tree
/// </summary>
/// <param name="sessionProperties">Session properties collection</param>
/// <param name="userIdentity">query user identity</param>
/// <returns>An enumerable of tuples with claims</returns>
public IEnumerable<Tuple<Claim, bool>> AddAccess(IDictionary<string, object> sessionProperties, IIdentity userIdentity)
{
if (null == userIdentity)
{
throw new NullReferenceException("Error: AxdAccess method is called with an invalid user identity parameter");
}
RefreshDataFile();
var claims = new LinkedList<Tuple<Claim, bool>>();
var membership = GetMembership(GetUserId(userIdentity));
if (membership != null)
{
foreach (var member in membership)
{
claims.AddLast(new Tuple<Claim, bool>(new Claim(_claimType, member, _claimValueType, _claimIssuer, _claimIssuer), false));
}
}
return claims;
}
We need a class to act as a simple wrapper of a Dictionary<string, string>. It should essentially serve as a key-value lookup from a user-ID to its group membership data. Each user's group membership has the form group1;group2;group3, where ";" is used to separate between each group membership entry. This class is simply called Lookup.
private string GetUserId(IIdentity userIdentity)
{
// Run through all the claims of the claims identity, look for the
// user logon name claim and primary SID to get the current user:
// domain\username
...
...
return strUser;
}
private string[] GetMembership(string key)
{
lock (_lock)
{
var value = _lookup.Get(key);
if (!string.IsNullOrEmpty(value))
{
return value.Split(new[] { ';' }, StringSplitOptions.RemoveEmptyEntries);
}
return null;
}
}
private void RefreshDataFile()
{
if ((DateTime.Now - _lookupFileStamp).Seconds > 30)
{
lock (_lock)
{
_lookupFileStamp = DateTime.Now;
if (File.Exists(_lookupFilePath))
{
_lookup = Lookup.Load(_lookupFilePath);
}
}
}
}
Performance Considerations
Consider the following tips to improve the overall performance with this trimmer as a starting point:
- Use multiple Lookup classes in a hash-table on a given key.
- Use a compressible stream and binary serialize the Lookup data.
- Use IPC (NetPipe) to talk to a local service that holds the more efficient key-value Lookups.
Deploying Trimmer
After you have built the custom security trimmer project, you must deploy it to the global assembly cache on any server in the Query role.
On the Start menu, choose All Programs, choose Microsoft Visual Studio 2010, and then choose Visual Studio Tools and open a Visual Studio command prompt.
To install the SearchPreTrimmer.dll, type the following the command prompt
gacutil /i <ExtractedFolderPath>\PreTrimmer\bin\Debug\SearchPreTrimmer.dll
As the last step of deploying the trimmer, we need to learn about the token of the DLL. Type the following the command prompt
gacutil /l SearchPreTrimmer
Write down the token listed for the newly added DLL.
Registering Trimmer
Open the SharePoint Management Shell.
At the command prompt, type the following command:
New-SPEnterpriseSearchSecurityTrimmer -SearchApplication "Search Service Application"
-typeName "CstSearchPreTrimmer.SearchPreTrimmer, CstSearchPreTrimmer,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=token"where token is the text copied from the gacutil /l command above. Example: New-SPEnterpriseSearchSecurityTrimmer -SearchApplication "Search Service Application" -typeName "CstSearchPreTrimmer.SearchPreTrimmer, SearchPreTrimmer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=4ba2b4aceeb50e6d" -Id 2
Restart the Search Service by typing
net restart sphostcontrollerservice
Testing the Trimmer
Now, you can issue queries and you can see the beauty of the pre-trimmer logic in action for every query evaluated. Try modifying the datafile.txt for different group membership per user (keep in mind that contents of the data file are only loaded every 30 seconds). The 30 second refresh interval is a defined constant in the code. Enjoy!
Acknowledgements
Author: Sveinar Rasmussen (sveinar).
Comments
Anonymous
July 22, 2013
Hi, Thanks for wonderful Article and detailed steps!!!. Am not able to debug the Security Trimmer Dll ,but i added few logs in code initially which am able to see log comments but when i added bit extra logic and extra logging,deployed to GAC i still see still old log comments despite of repetitive deployment (around 20 times) :-( .. Please let me know how to referring to latest to DLL ,sorry if this too native question.. Between can we achieve same using rest API (client) of Fast to consume Search Results and see if security trimming works?Anonymous
July 23, 2013
The comment has been removedAnonymous
October 06, 2013
hi, Im looking for a away to return results regardless of the user access rights, for example, writing the loging name of a user on a metadata field and check that field in Custom Pre-Security Trimmer phase but i dont understand how to add such a claim. can this be done using Custom Pre-Security Trimmer ?Anonymous
October 07, 2013
Lior, you would be looking at adding a claim that everyone can see. It is not possible to disable security trimming in one custom pre-security trimmer. You cannot add a claim to match 'everyone' in a Windows SID terminology (by design, SID claims are not accepted from a pre-security trimmer). That said, it should be possible to add a claim that represents authenticated people in SharePoint. Perhaps you could add a claim of the following type: sharepoint.microsoft.com/.../isauthenticated with value set to true ?Anonymous
November 08, 2013
Hi Sveinar, Thanks for you reply with specific to debugging of security trimmer :-) that helped me a lot we have requirement for having Custom Security Groups(non-AD security groups) which are local to external source and those groups are not a part of AD . Below is what are looking to implement .please let me know how can we achieved ? 1)SP crawler has to index External source which is outside of share point that has it own security groups. 2)While crawling the document i want to set this security group which is local to that external source . 3)while user logs in for searching content,we want to trim the results based on logged in user custom security group (assuming we have those details from http header for that logged in user) . Regards, RK.Anonymous
November 12, 2013
While I haven’t set up a similar case with multiple SharePoint farms where each farm has its own security groups, I do believe that one obstacle here to overcome is the ACLs that will be associated with the external source contents (as seen from the SP web-crawl). The general rule is to use the BCS framework to set the SecurityDescriptor with custom claims, where the custom claims are those other external (and different) security groups. Similarly, when the user logs on to query for content, a Pre-trimmer can be used here to perform an analogous mapping from the local user to its corresponding external security group memberships. Thus, this should be possible but it requires a custom BCS connector to supply the proper claims/ACLs on the content and a pre-trimmer to perform a successful secure lookup. With the SP Crawler, I do believe that it is not possible to customize the ACLs that it puts on the content crawled. For additional information, be sure to check out the blog post on “Creating Custom Connector Sending Claims with SharePoint 2013”.Anonymous
June 20, 2015
Hi Sveinar, what actual value we can pass in pre-security trimming member value of AddAccess method. I have two domain A, and B. so I want to add domain A users claim with the search query in presecurity trimming for the items where domain B have permissions only. Can it be done through this?. what actual value we need to pass in member field(group or loginname of Domain A user) i.e Can we add like domainAuser1 as member value in AddAccess method.Anonymous
June 20, 2015
The comment has been removedAnonymous
June 21, 2015
So if I Pass this like claims.AddLast(new Tuple<Claim, bool>( new Claim("schemas.happy.bdc.microsoft.com/.../acl", loginDomainUser), false)); where "schemas.happy.bdc.microsoft.com/.../acl" as same which you have written and I only change the logindomainuser value as domainAatul, then I would be able to see the document, or do in need to change the claim type url "schemas.happy.bdc.microsoft.com/.../acl" also to map with our environment(Dev, UAT, production)...Anonymous
June 21, 2015
Atul, a claim is just a statement where the "schemas.happy.bdc.microsoft.com" is just a claim type identifier together with a claim value (group or domain name for instance). For Windows security identifiers like SIDs, these are not allowed to pass through the pre-trimmer for security reasons (these would be of the claim type "schemas.microsoft.com/.../primarysid"). If that was allowed, it would enable pre-trimmers to unlock content without changing the crawler source, if the latter was submitting SID security ACEs. Instead, the work must be symmetrical for this to work. E.g. you can define a custom claim type of your own like "https://atuls.happy.claim.type.here.com" with the value of DomainA/atul in the pre-trimmer. Similarly, you need to ensure that the SecurityDescriptor from the BCS framework on the crawler/connector side match up to this same custom claim type. For more details on claims: technet.microsoft.com/.../ee913589.aspxAnonymous
August 24, 2015
I am searching with user domanuser1, but user1 dont have permission on one file. Atul has permission on file so I am passing this like claims.AddLast(new Tuple<Claim, bool>(new Claim(_claimType, "domainatul" _claimValueType, _claimIssuer, _claimIssuer), false)); I have following configuration private string _claimType = "surface.microsoft.com/.../acl"; private string _claimIssuer = "customtrimmer"; private string _claimValueType = ClaimValueTypes.String; private string _lookupFilePath = @"c:membershipdatafile.txt"; During query time page is htting the code in vs2013, means trimmer is registered perfectly. But this is not returning results based on "domaintatul". Do I need to change anything.Anonymous
August 26, 2015
Hello "Claims not working" :-) A few comments to help you on your way... With custom claims, you also need to tag the claim to the documents using the BCS framework. That is, you need to submit documents with the proper claim ACL to the index. Did you follow the steps of blogs.msdn.com/.../creating-custom-connector-sending-claims-to-sharepoint-2013.aspx for that part? On a minor note, I am assuming that the custom connector is in place and that the claim type naturally does not contain any "..." but instead _claimType = "surface.microsoft.com/.../acl" (not "surface.microsoft.com/.../acl") and that the key for 'domainatul' results in a few group membership entries in the datafile.txt.Anonymous
August 27, 2015
Thanks Sveinar for the clarification. But If we write my own BCS connector then we can handle the user permission. I was thinking to send custom claims of a new user in OOTB indexed document while searching. We have some content on which some users dont have permissions. After crawling using OOTB fileshare Sharepoint connector, i want to add the claim of a users during search.Anonymous
August 27, 2015
Indeed, the SharePoint Pre-Trimmer is prevented from allowing SID claims to be processed by design. This was a security concern during development, and we can reassess if this concern is still valid. Can you file a feature request perhaps to lift the restriction to allow pre-trimmers to emit any claims possible? That way, you can then "modify" OOTB indexed document permissions from the SharePoint connector etc. An argument is that pre-trimmers are considered trusted code deployed by trusted parties only with admin access. For more details on this, see my post on 22 Jun 2015 1:21 PM.Anonymous
February 10, 2016
Hello Thanks fo this article. I need to create SharePoint 2013 search service that return all documents in SharePoints (Anythings ). I created the SearchPreTrimmer and in the method: public IEnumerable<Tuple<Claim, bool>> AddAccess I added this code claims.AddLast(new Tuple<Claim, bool>(new Claim(_claimType, "domain\Administrator"), false)); I use local Active Directory. I need this Trimer for the documents in SharePoint. But is not working.Anonymous
February 15, 2016
The comment has been removedAnonymous
February 15, 2016
Hi Sveinar, thanks for the reply. Do you have any suggestion to do what I expected. I overrieded the Search WebPart and runed it with runwithelevatedprivileges with an admin account bu it didn't work. Thanks you for your helpAnonymous
February 15, 2016
Saber, overriding the SearchWebPart with elevated privileges will not have any affect on the behavior here. Sorry. The logic in the claims encoding prevents claims like these SID claims from being encoded in the first place. This translates to no-match for the search engine term lookup - thus, by design, your documents will not be surfaced. I would recommend filing a feature request or use the BCS framework to supply your own custom claims for the documents submitted.