Modify the Content Search Web Part display template and use Windows PowerShell to start Usage analytics in SharePoint Server 2013
This is a blog post in the series “How to display Recommendations and Popular items on a SharePoint Server 2013 site.” In the previous post, I introduced you to the Usage analytics feature. In this blog post:
- Change the mapping of the UsageAnalyticsID managed property
- Modify a Content Search Web Part display template so that usage events are logged correctly
- Why you should simulate the generation of Views usage events
- Run Windows PowerShell scripts to start search analytics and push usage events to the Event store
- View usage events in the Event store
- Prepare usage events files before starting Usage analytics with Windows PowerShell
- Start the Usage analytics job with Windows PowerShell
Note: The examples in this blog series are based on an on-premises installation.
Change the mapping of the UsageAnalyticsID managed property
In our Contoso scenario, we want to recommend one product per product group, that is, we want Usage analytics to ignore the product color. This means that our recommendations must be calculated on Group Number. We can do this, because Group Number is part of the friendly URL (FURL) on our item details page.
In the previous blog post, I told you that the managed property that’s used to specify how recommendations between individual catalog items should be calculated is UsageAnalyticsID. So, in order for Usage analytics to do its calculation on Group Number, we have to change the mapping of the UsageAnalyticsID property. Here’s how you do that:
IMPORTANT: You have to change the property mapping on the Authoring site.
1. On your authoring site, go to Site settings --> Search Schema.
2. On the Managed Properties page, in the Managed property field, type UsageAnalyticsID, and then select the arrow button.
3. From the Property Name field, select Edit/Map Property.
4. On the Edit Managed Property page, select Add a Mapping. Notice that by default, this property is mapped to the crawled property ows_ProductCatalogItemNumber.
5. In the Crawled property selection dialog box, use the Search for crawled property name field to search for the crawled property that you want to map to this managed property.
In our Contoso scenario, we want to map the site column called Group Number. Crawled properties don’t contain spaces, so leave out the space, enter GroupNumber and select Find.
Two crawled properties are found. Select the crawled property with the ows_ prefix, and select OK.
If you are confused because two crawled properties that look about the same are found, then you’re not alone. This is quite tricky. In another blog post, I talk about the naming convention for crawled and managed properties.
If you are interested in a cliff note version of that blog post, here it is: When mapping a crawled property to the UsageAnalyticsID managed property, you should select the crawled property with the ows_ prefix!
6. On the Edit Managed Property page, select the ows_ProductCatalogItemNumber crawled property, and then Remove Mapping.
7. Select OK to save the new mapping.
IMPORTANT: Map only one crawled property to the UsageAnalyticsID managed property. If you map more than one crawled property, the Usage analytics calculation won’t work correctly.
After you have changed the mapping of the UsageAnalyticsID managed property, you have to do a full crawl of your catalog. I describe how you can do this in this blog post.
Modify a Content Search Web Part display template so that usage events are logged correctly
On our Contoso site, we use a Content Search Web Part (CSWP) to display items on the catalog item page. By default, the CSWP does not log usage events. To enable our CSWP to log usage events we have to modify the display template that the CSWP is using. Here’s what you need to do:
1. In your mapped network drive, open the display template that you have applied to your CSWP.
2. In the ManagedPropertyMapping element, add the following two properties:
'Original Path'{Original Path}:'OriginalPath',
'SiteID'{SiteID}:'SiteID',
3. Add the following Java-Script just above the HTML part of your display template:
//Log Views usage event on URL of catalog item
window.LogViewsToEventStore = function(url, site)
{
SP.SOD.executeFunc("sp.js", "SP.ClientContext", function()
{
var spClientContext = SP.ClientContext.get_current();
if(!$isNull(spClientContext))
{
var spWeb = spClientContext.get_web();
var spUser = spWeb.get_currentUser();
var spScope = "{00000000-0000-0000-0000-000000000000}";
SP.Analytics.AnalyticsUsageEntry.logAnalyticsEvent2(spClientContext, 1, url, spScope, site, spUser);spClientContext.executeQueryAsync(null, null);
}
});
};
var originalPath = $getItemValue(ctx, "Original Path");
var originalSite = $getItemValue(ctx, "SiteID");
LogViewsToEventStore(originalPath.value, originalSite.value);
I want to point out one value in this script. In the previous blog post, I told you about the different EventTypeIDs for the usage events. The value 1 in this script represents the EventTypeID of the Views usage event. If you want to log a different usage event, you’ll have to substitute this value with the EventTypeID of the usage event that you want to log.
4. Save the file.
Why you should simulate the generation of Views usage events
Now that our CSWP can correctly log usage events, the next step is to actually generate usage events. In our case, we modified the CSWP to log Views. If the Contoso site was LIVE, visitors would create a Views usage event every time they viewed an item on the website. However, the Contoso site is only a demo site, so it doesn’t have any visitors.
When you set up your site, you’ll most likely want to test the Usage analytics feature before you put it into production. To be able to test the Usage analytics feature, you’ll need to generate usage events. To generate recommendations based on usage events, a minimum of three different users have to click the same items.
There is not one single correct way of simulating the generation of Views usage events. To generate Views usage events for the Contoso site, I invited some of my co-workers to a “click party” (they were bribed with candy). To ensure that recommendations were generated, I gave all users a list of items that they had to click. That way I ensured that at least three different users clicked the same items.
Here’s an example of the instructions that I gave my co-workers:
When Usage analytics is run, SV Keyboard E10 will generate a recommendation for WWI Desktop PC2.30 M2300 (people who viewed WWI Desktop PC2.30 M2300 also viewed SV Keyboard E10) and WWI Desktop PC2.30 M2300 will generate a recommendation for SV Keyboard E10 (people who viewed SV Keyboard E10 also viewed WWI Desktop PC2.30 M2300).
Run Windows PowerShell scripts to start search analytics and push usage events to the Event store
After you have generated Views usage events, you have two options on how to proceed. The Usage analytics timer job runs once every 24 hours. If you are a patient person you can now relax, go home, and let Usage analytics work its magic overnight.
On the other hand, if you are more of an “I want results now” type of person, you can use some Windows PowerShell scripts to speed up the process. Here’s what you need to do:
1. Verify that you meet the following minimum permission requirements.
2. On the server where SharePoint Server 2013 is installed, open the SharePoint 2013 Management Shell as an Administrator.
3. At the Windows PowerShell command prompt, type the following commands to start Search analytics. The output from Search analytics is used by Usage analytics to map usage events against the actual items in the search index.
$a = Get-SPTimerJob -Type Microsoft.Office.Server.Search.Analytics.AnalyticsJobDefinition
$sa = $a.GetAnalysis("Microsoft.Office.Server.Search.Analytics.SearchAnalyticsJob")
$sa.StartAnalysis()
4. Wait for the search analytics job to finish. To check the status of the search analytics job, type the following command:
$sa.GetAnalysisInfo()
As long as the search analytics job is running, State is Running.
The search analytics job is finished when State is Stopped and Status is 100.
5. The usage events are added to the Event store within a 10 minute interval. To push the usage events to the Event store, type the following commands:
$tj = Get-SPTimerJob -Identity ("job-usage-log-file-import")
$tj.RunNow()
View usage events in the Event store
After you have pushed the usage events into the Event store, you should verify that the usage events are recorded correctly. To do that, on the machine where SharePoint Server 2013 is installed, go to the Event store. In most cases you can find the Event store in this file directory:
C:\Program Files\Microsoft Office Servers\15.0\Data\Office Server\Analytics_<GUID>\EventStore
In the Event store, the usage events of each day are stored in a separate folder. In our scenario, we can see that a folder has been added.
In this folder, you’ll see some text files. These files contain our usage events. Notice that all file names start with 1_.
This number is the EventTypeID of the usage event that is logged in the file. Remember, 1 is the EventTypeID of the Views usage event. At this point the only usage event that we are logging is the Views event, so this is a good sign that we are doing things right.
Open one of the files in a text editor, for example Notepad++. This file contains quite a bit of information, but you should really only be looking for two things:
- Verify that the usage events are logged correctly.
- Verify that different users have generated the usage event.
In the previous blog post, I explained that for Usage analytics to work, the usage event must be recorded on the URL of the item. In the Event store file, you’ll see many URLs. Look for URLs that end in dispform.aspx?id= followed by a number. In our file, we can see that it contains many entries with such URLs.
This is a very good sign, because it means that the usage events are being recorded correctly. To verify that the URLs actually belong to one of our catalog items, copy one of the URLs from this file, and paste it into your browser.
To verify that different users have generated the usage event, look in the third column of the file. In our scenario, we can see that we have at least three different user IDs.
Now that we have verified that the usage events are correctly logged, you could be tempted to think that we are ready to run the Usage analytics job. But remember, by using Windows PowerShell to start Usage analytics, we are actually kick-starting timer jobs.
When the Usage analytics timer job kicks off, it’ll take the usage events from yesterday and process them. Since we want to process the files from today, we’ll use a simple trick so that the correct files can be processed by Usage analytics.
Prepare usage events files before starting Usage analytics with Windows PowerShell
1. In your EventStore file directory, create a new folder called myevents.
2. Copy the usage event files that you want Usage analytics to process into your myevents folder.
In our scenario, I copied all files from the 20131206 folder into myevents.
3. Right-click on your myevents folder and select Properties.
4. In the Attributes section, select Read-only (Only applies to files in folder) , and then OK.
5. In the Confirm Attribute Changes, select Apply changes to this folder, subfolder and files and then OK.
Now we’re ready to start the Usage analytics job.
Start the Usage analytics job with Windows PowerShell
1. At the Windows PowerShell command prompt, type the following commands:
$tj = get-sptimerjob -type microsoft.office.server.search.analytics.usageanalyticsjobdefinition
$tj.DisableTimerJobSchedule()
$tj.StartAnalysis(“\\<hostname>\Analytics_<guid>\EventStore\myevents”)
$tj.EnableTimerJobSchedule()
Notice that one command contains two placeholders: hostname and guid. The hostname is the name of the server where SharePoint Server 2013 is installed. You can see the guid in the file path to your EventStore.
2. Check the status of the Usage analytics job by entering the following command:
$tj.GetAnalysisInfo()
The Usage analytics job is finished when State is Stopped and Status is 100.
Now that Usage analytics have processed the usage events, the next step is to display the results of the analysis on our Publishing site. To do that, we’ll add and configure two Web Parts. I’ll show you how to do that in the next blog post.
Next blog post in this series:
Add and configure the Recommended Items and Popular Items Web Part
Comments
Anonymous
January 01, 2003
Hi Bella
Your articles helped me on my development environment, and i can see stats.
But in the qualification environment, it is not working i do not see files on EventStore folder.
The unique difference is that my development environment is SharePoint Entreprise, but the qualification environment is SharePoint Standard.
Is this feature available on Standard edition? in this link they said that it is available http://blog.blksthl.com/2013/01/14/sharepoint-2013-feature-comparison-chart-all-editions/
Thank youAnonymous
January 01, 2003
Hi Wensi,
Please contact MS Support with your question.
BellaAnonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
Thank you so much Bella :)Anonymous
January 01, 2003
Hi Colin,
I’m glad to hear that you like Season 2 :-)
I’m not going to the SharePoint Conference. From what I've seen, there will be several WCM related sessions (http://www.sharepointconference.com/content/sessions), many of which look really interesting.
BellaAnonymous
January 01, 2003
Hi Moudhafer,
The usage analytics features that I describe in this blog series is only available in SharePoint Server 2014 Enterprise. For more information, see this overview:http://technet.microsoft.com/en-us/library/jj819267.aspx#bkmk_FeaturesOnPremise
It's important to see the footnote for "WCM:Analytics" that says that only search analytics is available in SharePoint Server 2013 Standard.
BellaAnonymous
January 24, 2014
Hi Bella, season 1 was fantastic, season 2 is just epic!!! The quality and detail of your posts cannot be praised highly enough.Are you presenting any sessions at the SharePoint Conference in Las Vegas? A session (or 2) of this type would go down a storm.Anonymous
May 14, 2014
...however my question is: do you know if there is any easy way to export search results into csv?Anonymous
November 07, 2014
Hi, I wanted to log events for items (images) when a user clicks a link crafted in a display template for the Search Results WebPart. However, the SiteID property is null, so before I continue, do you have any hopes for this working for images in a Search Results WebPart instead of pages in a Content Search WebPart?Anonymous
December 29, 2014
Hi Bella, I am curious display template is only called when rendering search items, how can click event trigger Log usage event function in display template?