Query Refinement in SharePoint
Learn how to use SharePoint query refinement features programmatically when you are working with search queries and results.
You can use query refinement features to provide end users with refinement options that are relevant for their queries. These features let the end user drill down into search results by using refinement data computed for the results. Refinement data is calculated by the index component, based on the aggregation of managed property statistics for all of the results of a search query.
Typically, query refinement is used for metadata associated with the indexed items, such as creation date, author, or file types that appear in the item. By using the refinement options, you can refine your query to display only items created during a certain time period, or display only items of a specific file type.
Using refiners in the Query Object Model
There are two queries involved in query refinement:
- You can request for a set of refiners to be returned in the search results by adding a refiner spec to the end user's query. A refiner spec is the input for the Refiners property. This query is run against the search index. The search results will consist of relevant results and refinement data.
- You can use the refinement data to drill down into the search results by, creating a refined query. Add the RefinementFilters property to the query so that the final search results will meet the requirements of both the original query text from the end user and the selected refinement option from the refinement data.
The following sections describe these steps in detail, and provide code examples.
Adding refiners to the end-user's query by using refiner specs
You can specify the requested query refiners by using the Refiners property of the KeywordQuery class. Use the following syntax to specify the requested query refiners:
<refiner>[,<refiner>]*
Each refiner has the following format:
<refiner-name>[(parameter=value[,parameter=value]*)]?
Where:
<refiner-name>is the name of the managed property associated with the refiner. This managed property must be set to Refinable or Sortable in the search schema.- The optional list of
parameter=valuepairs specifies non-default configuration values for the named refiner. If a parameter for a refiner is not listed inside the parentheses, the search schema configuration gives you the default setting. Table 1 lists possible values for theparameter=valuepairs.
Note
When you specify refiners, the minimum requirement is to specify a refiner-name, which is a managed property.
Example
Refiners = "FileType"
Or, you can also use the advanced syntax to adjust the refiner settings:
Refiners = "FileType,Write(discretize=manual/2013-01-01/2013-08-22/2013-09-15/2013-09-21/2013-09-22),companies"
Table 1: List of parameters for refiners
| Parameter | Description |
|---|---|
| deephits |
Overrides the default number of hits that is used as the basis for refinement computation. When refiners are produced, all results for the query will be evaluated. Normally, using this parameter will improve search performance. Syntax deephits=<integer value> Example price(deephits=1000) Note: This limit applies within each index partition. The actual number of hits that are evaluated will be larger than this value due to the aggregation across search partitions. |
| discretize |
Specifies custom intervals (refinement bins) for numeric refiners. Syntax discretize=manual/<threshold>/<threshold>[/<threshold>]* Example write(discretize=manual/2013-01-01/2013-08-22/2013-09-15/2013-09-21/2013-09-22) The <threshold> attribute specifies the threshold for each refinement bin. There is one interval for everything below the first threshold specified, one interval between each consecutive threshold, and one interval for everything above the last threshold. For a refiner of type DateTime, specify the threshold according to one of the following ISO 8601-compatible formats:
|
| sort |
Defines how to sort the bins within a string refiner. Syntax sort=<property>/<direction> The attributes perform the following:
sort=name/ascending Default: frequency/descending |
| filter |
Defines how bins within a refiner of type String are filtered before they are returned to the client. Syntax filter=<bins>/<freq>/<prefix>[<levels>] The attributes perform the following:
|
| cutoff |
Limits the data that must be transferred and processed for deep string refiners. You can configure the refiners to return only the most relevant values (bins). Note: This cutoff filtering is performed within each index partition. This differs from the filter parameter, which performs result-side filtering only. You can combine the two parameters. Syntax cutoff=<frequency>/<minbins>/<maxbins> The attributes perform the following:
|
Example: Adding refiners
The following CSOM example shows how to programmatically request three refiners: FileType, Write, and Companies. Write represents the last modified date for the item, and uses the advanced syntax to return fixed-size date/time bins.
using (var context = new ClientContext("http://<serverName>/<siteCollectionPath>"))
{
var query = new KeywordQuery(context)
{
QueryText = "home",
Refiners = "FileType,Write(discretize=manual/2013-01-01/2013-08-22/2013-09-
15/2013-09-21/2013-09-22),companies"
};
var executor = new SearchExecutor(context);
var results = executor.ExecuteQuery(query);
context.ExecuteQuery();
ResultTable relevantResultsTable = results.Value[0];
ResultTable refinerResultsTable = results.Value[1];
Console.WriteLine(refinerResultsTable.RowCount + " refinement options:");
foreach (var refinementOption in refinerResultsTable.ResultRows)
{
Console.WriteLine("RefinerName: '{0}' RefinementName: '{1}'
RefinementValue: '{2}' RefinementToken: '{3}' RefinementCount: '{4}'",
refinementOption["RefinerName"],
refinementOption["RefinementName"],
refinementOption["RefinementValue"],
refinementOption["RefinementToken"],
refinementOption["RefinementCount"]
);
}
}
Understanding the refinement data in the search result
If you have enabled query refinement for a managed property in your query, the query result contains refinement data split into refinement bins. This data is located in the RefinementResults table ( RefinementResults ) within the ResultTableCollection . One refinement bin represents a specific value or value range for the managed property.The RefinementResults table contains one row per refinement bin, and contains the columns, as specified in Table 2.
Table 2: Data returned for refinement bins
| Parameter | Description |
|---|---|
| RefinerName | The name of the query refiner. |
| RefinementName | The string that represents the refinement bin. This string is typically used when presenting the refinement options to the users on a search result page. |
| RefinementValue | An implementation-specific formatted string that represents refinement. This data is returned for debugging and is typically not required for the client. |
| RefinementToken | A string that represents the refinement bin to use with RefinerName when you perform a refined query. |
| RefinementCount | The result count for this refinement bin. This data represents the number of items (including duplicates) in the search result with a value for the given managed property that falls into this refinement bin. |
Example: Refinement data
Table 3 below contains two rows of refinement data. The first row holds refinement data for indexed items, where the file type is HTML. The second row holds refinement data for indexed items, where the last modified time is from 2013/09/21 until 2013/09/22.
Table 3: Format and contents of refinement data
| RefinerName | RefinementName | RefinementValue | RefinementToken | RefinementCount |
|---|---|---|---|---|
| FileType | Html | Html | "????68746d6c" | 50553 |
| Write | From 2013-09-21T00:00:00Z up to 2013-09-22T00:00:00Z | From 2013-09-21T00:00:00Z up to 2013-09-22T00:00:00Z | range(2013-09-21T00:00:00Z, 2013-09-22T00:00:00Z) | 37 |
Creating a refined query
A search result presents refinement options in the form of string values or value ranges. Each string value or numeric value range is called a refinement bin, and each refinement bin has an associated RefinementToken value. A refinement option is associated with a managed property, which is provided by the RefinerName value.
Both the RefinementToken and RefinerName values are concatenated to create a refinement filter string. This string represents a filter that can be used to limit search result items to include only items where a managed property has a value within a refinement bin. In short:
refinement filter = <RefinerName>:<RefinementToken>
You can provide one or more refinement filters for a refined query by adding refinement filters to the RefinementFilters property of the KeywordQuery class. Multiple refinement filters enable you to provide multilevel drill-down into the search results, and to apply refinement on multivalued properties. For example, you can refine the query to items that have two authors - each represented by a refinement bin - but exclude items that have only one of the authors.
Example 1: Creating a refined query for HTML file types
The following CSOM example shows how to programmatically perform a refinement, to limit the search results to those of HTML file type only. As mentioned in Example: Refinement data, refinement data related to this refinement option has RefinerName set to Filetype and RefinementToken set to "????68746d6c".
using (var context = new ClientContext("http://<serverName>/<siteCollectionPath>"))
{
var query = new KeywordQuery(context)
{
QueryText = "home"
};
query.RefinementFilters.Add("FileType:\\"????68746d6c\\"");
var executor = new SearchExecutor(context);
var results = executor.ExecuteQuery(query);
context.ExecuteQuery();
ResultTable relevantResultsTable = results.Value[0];
var resultCount = 1;
foreach (var relevantResult in relevantResultsTable.ResultRows)
{
Console.WriteLine("Relevant result number {0} has file type {1}.",
resultCount, relevantResult["FileType"]);
resultCount++;
}
}
Example 2: Creating a refined query by using previously obtained refinement data
The following CSOM example shows how to run a query with a refiner spec to create refinement data which is subsequently used to perform refinement. This example simulates the process of an end user selecting the first refinement option.
using (var context = new ClientContext("http://<serverName>/<siteCollectionPath>"))
{
// Step 1: Run the query with refiner spec to provide refinement data in search result
var query = new KeywordQuery(context)
{
QueryText = "home",
Refiners = "FileType,Write(discretize=manual/2013-01-01/2013-08-22/2013-09-15/2013-09-21/2013-09-22),companies"
};
Console.WriteLine("Run query '{0}' with refiner spec '{1}'.", query.QueryText, query.Refiners);
var executor = new SearchExecutor(context);
var results = executor.ExecuteQuery(query);
context.ExecuteQuery();
// The query has been run and we can now look at the refinement data, to view the
// refinement options
ResultTable relevantResultsTable = results.Value[0];
ResultTable refinerResultsTable = results.Value[1];
Console.WriteLine("Got back {0} refinement options in the result:",
refinerResultsTable.RowCount);
foreach (var refinementOption in refinerResultsTable.ResultRows)
{
Console.WriteLine("RefinerName: '{0}' RefinementName: '{1}'
RefinementValue: '{2}' RefinementToken: '{3}' RefinementCount: '{4}'",
refinementOption["RefinerName"],
refinementOption["RefinementName"],
refinementOption["RefinementValue"],
refinementOption["RefinementToken"],
refinementOption["RefinementCount"]
);
}
// Step 2: Run the refined query with refinement filter to drill down into
// the search results. This example uses the first refinement option in the refinement
// data, if available. This simulates an end user selecting this refinement option.
var refinementOptionArray = refinerResultsTable.ResultRows.ToArray();
if (refinementOptionArray.Length > 0)
{
var firstRefinementOption = refinementOptionArray[6];
// Construct the refinement filter by concatenation
var refinementFilter = firstRefinementOption["RefinerName"] + ":" +
firstRefinementOption["RefinementToken"];
var refinedQuery = new KeywordQuery(context)
{
QueryText = query.QueryText
};
refinedQuery.RefinementFilters.Add(refinementFilter);
refinedQuery.SelectProperties.Add("FileType");
refinedQuery.SelectProperties.Add("Write");
refinedQuery.SelectProperties.Add("Companies");
Console.WriteLine("Run query '{0}' with refinement filter '{1}'",
refinedQuery.QueryText, refinementFilter);
var refinedResults = executor.ExecuteQuery(refinedQuery);
context.ExecuteQuery();
ResultTable refinedRelevantResultsTable = refinedResults.Value[0];
var resultCount = 1;
foreach (var relevantResult in refinedRelevantResultsTable.ResultRows)
{
Console.WriteLine("Relevant result number {0} has FileType='{1}',
Write='{2}', Companies='{3}'",
resultCount,
relevantResult["FileType"],
relevantResult["Write"],
relevantResult["Companies"]
);
resultCount++;
}
}
}
See also
- Configure properties of the Refinement web part in SharePoint
- Overview of the search schema in SharePoint