FIM 2010 R2 Search Changes

The following section describes the Search Scope improvements to FIM 2010 R2

Search Scope Improvements

With the release of Forefront Identity Manager R2 2010, the Search Scopes have had some improvements to their overall performance. This was achieved by changing the search scope of DisplayName to use start-with rather than contains. That is, in FIM 2010 for example if we were to do a search in the FIM Portal for all users named John then the query used to look like this:

/Person [(contains(DisplayName, ''john'') or starts-with(AccountName, ''john'') or starts-with(MailNickname, ''john''))]

In FIM 2010 R2 the query would look like this:

/Person [(starts-with(DisplayName, ''john'') or starts-with(AccountName, ''john'') or starts-with(MailNickname, ''john''))]

The reason for this is that in large databases, the Query performs poorly with prefix-term searches. A prefix term refers to a string that is affixed to the front of a word to produce a derivative word or an inflected form. For a single prefix term, any word starting with the specified term will be part of the result set. For example, the term "auto*" matches "automatic", "automobile", and so forth. This will affect anything that uses or builds a Search Scope.

Because of this and that some organizations may want to continue to use the contains operator, two new attributes have been introduced to allow this functionality to continue. They are:

1. AdvancedFilter – Allows you to specify the filter query to use.

2. DefaultSearchScopeName – Allows you to specify the search scope to use on a UocIdentityPicker.

AdvancedFilter

The AdvancedFilter box is available on the Extended Attributes of a Search Scope. You cannot specify the AdvancedFilter when creating a new Search Scope only after it has been created in the FIM Portal and you have access to the Extended Attributes.

To locate the x-path used by a Search Scope, you can use SQL Profiler to see what x-path is being run when you submit a search from the Portal. Appending the x-path to the query is new for FIM 2010 R2. To locate the x-path, use the following procedure.

To locate the x-path

1. On the FIMService database begin a SQL Profiler Trace

2. On the FIM Portal server, in the UI, do a search. In our example we did a search for john using the All Users search scope. This returned no results because we have no users named John.

3. Once the search completes, stop the SQL Profiler trace and look for a line that in the TextData column that says: **exec [fim].ExecuteQuery @spl=N’—**

   Warning

   Be aware that there is a lot more to this line but the portion above is the starting piece.

4. Select the line from above.

5. In SQL Profiler, the box at the bottom will have a very big query. Use the arrows on the side and scroll to the bottom.

6. At the very bottom, you will see the xpath query. In our case /Person [(starts-with(DisplayName, ''john'') or starts-with(AccountName, ''john'') or starts-with(MailNickname, ''john''))]

   

To change the Search Scope, simply add the new scope to the Advanced Filter box and submit the scope. You will need to do an iisreset once the request has been submitted as well. So for example, if you wanted to change the new default starts-with query back to using the contains operator for the All Users Search Scope, you would add the following to the Advanced Filter box of the Extended Attributes of the All Users Search Scope:

/Person [(contains(DisplayName, '%SEARCH_TERM_STRING%') or starts-with(AccountName, '%SEARCH_TERM_STRING%') or starts-with(MailNickname, '%SEARCH_TERM_STRING%'))]

Now after submitting it and doing an iisreset, this is the query that will be run. To test this, you can use a SQL Profiler trace and verify that this is the query that is executing.

Be aware that the Advanced Filter does not validate the xpath query so if there is a typo in there it may be the cause of the Search Scope not functioning properly.

. The FIM service supports the SQL LIKE wildcards for the “starts-with” predicate within the Advanced Filter x-path. These wildcards are _, %, and []. The “contains” predicate does not support any wildcards or any non-alpha-numeric characters. Direct comparisons within the x-path also do not support any wildcards. Be aware that in a search when [ is present in the search term string, it will need to be escaped in order for the search to succeed. Otherwise the search will fail. The [ is escaped by enclosing it in [ ].

Advanced Filter Details

The x-path supplied in the Advanced Filter attribute supports all the tokens that are possible for the standard Filter attribute, such as LoginID and FromToday. In addition, Advanced Filter supports the following tokens:

Token	Example X-Path Clause
%SEARCH_TERM_REFERENCE%	objectId = %SEARCH_TERM_REFERENCE%
%SEARCH_TERM_BOOLEAN%	IsActionActivity = %SEARCH_TERM_BOOLEAN%
%SEARCH_TERM_INTEGER%	FreezeCount = %SEARCH_TERM_INTEGER%

Partial Substring Match

The following is an explanation of using a partial substring match in a search scope. Lets say for example that we have a user with a DisplayName of Ralph Waldo Emerson. We do a search of All Users for Waldo using the FIM 2010 R2 search scope for All Users.

 
/Person [(starts-with(DisplayName, '%SEARCH_TERM_STRING%') or starts-with(AccountName, '%SEARCH_TERM_STRING%') or starts-with(MailNickname, '%SEARCH_TERM_STRING%'))]

This would not return any results. Depending on what results you are looking for this may be sufficient. If you only wanted users whose first name was Waldo then this search scope would work. However if you want all users whose DisplayName contains Waldo regardless of whether or not it is the first, last or middle name then an AdvancedFilter would be needed. If we change the xpath of our All Users AdvancedFilter attribute to revert to how it was in FIM 2010 it will look similar to the one below.

/Person [(contains(DisplayName, '%SEARCH_TERM_STRING%') or starts-with(AccountName, '%SEARCH_TERM_STRING%') or starts-with(MailNickname, '%SEARCH_TERM_STRING%'))]

This search scope would return our user. Another option is to use an xpath with a partial substring match similar to the following:

/Person [(starts-with(DisplayName, '%%SEARCH_TERM_STRING%') or starts-with(AccountName, '%SEARCH_TERM_STRING%') or starts-with(MailNickname, '%SEARCH_TERM_STRING%'))]

The only difference to the new FIM 2010 R2 search scope is the addition of an additional % for our search term string for DisplayName. This would also return our user.  In some cases, this query may be a faster alternative to using contains.

DefaultSearchScopeName

The DefaultSearchScopeName allows for the specification of a default Search Scope within a UocIdentityPicker. So for example, if I wanted to change my search scope to use the contains operator and I wanted the ObjectPicker associated with Manager on User creation to use this search scope I can add this attribute to properties of the Manager control in my Configuration for User Creation RCDC. The DefaultSearchScopeName must specify the name of an existing, valid search scope, otherwise this property is ignored.

To add this attribute, you must modify the RCDC. Below is an example of how to modify the RCDC to use the DefaultSearchScopeName.

<my:Control my:Name="Manager" my:TypeName="UocIdentityPicker" my:Caption="{Binding Source=schema, Path=Manager.DisplayName}" my:Description="">
        <my:Properties>
          <my:Property my:Name="Required" my:Value="{Binding Source=schema, Path=Manager.Required}"/>
          <my:Property my:Name="Mode" my:Value="SingleResult"/>
          <my:Property my:Name="ObjectTypes" my:Value="Person"/>
          <my:Property my:Name="ColumnsToDisplay" my:Value="DisplayName, AccountName, Department"/>
          <my:Property my:Name="AttributesToSearch" my:Value="DisplayName, AccountName"/>
          <my:Property my:Name="UsageKeywords" my:Value="Person"/>
          <my:Property my:Name="ResultObjectType" my:Value="Person"/>
          <my:Property my:Name="Value" my:Value="{Binding Source=object, Path=Manager, Mode=TwoWay}"/>
          <my:Property my:Name="ListViewTitle" my:Value="%SYMBOL_ManagerListViewTitleCaption_END%"/>
          <my:Property my:Name="PreviewTitle" my:Value="%SYMBOL_ManagerPreviewTitleCaption_END%"/>
          <my:Property my:Name="MainSearchScreenText" my:Value="%SYMBOL_ManagerMainSearchScreenTextCaption_END%"/>
          <my:Property my:Name="DefaultSearchScopeName" my:Value="All Users"/>
        </my:Properties>
      </my:Control>

Be aware that ObjectTypes, ColumnsToDisplay, AttributesToSearch, and UsageKeywords are all ignored if DefaultSearchScopeName is specified and references an existing valid search scope.

Additional Useful Search Scopes

The following additional search scopes are not included in FIM 2010 R2 by default but have been very useful with many customers and consultants in the past and are worth noting. They are:

Errors from Today

/Request[Creator='%LoginID%' and CreatedTime>op:subtract-dayTimeDuration-from-dateTime(fn:current-dateTime(), xs:dayTimeDuration('P1D')) and (RequestStatus='PostProcessingError') or (RequestStatus='Failed') or (RequestStatus='Denied')]

All Pending Requests

/Request[(RequestStatus='PostProcessing' or RequestStatus='Authorizing' or RequestStatus='Committed' or RequestStatus='Authenticating' or RequestStatus='Validating' or RequestStatus='Authorized' or RequestStatus='Authenticated' or RequestStatus='Validated' or RequestStatus='Canceling') and CreatedTime>op:subtract-dayTimeDuration-from-dateTime(fn:current-dateTime(), xs:dayTimeDuration('P1D'))]