Share via

Usage Reporting in SharePoint (WSS v3 and MOSS)

  • Article

What is Usage Reporting?

SharePoint (WSS or MOSS) have built-in usage reporting feature. Usage reporting is a service that enables site administrators, site collection administrators, and Shared Services Provider (SSP) administrators to monitor statistics about the use of their sites. Usage reporting also includes usage reporting for search queries that can be viewed by SSP administrators for search and site collection administrators.

Usage reporting is very useful for managing complex site hierarchies with many sites, a large number of page hits, and a large number of search queries, and it is recommended that the service be enabled for deployments of complex site hierarchies. For less complex deployments, usage reporting might not be necessary. It is also possible to disable the service temporarily to conserve resources when other those resources are needed for other processes. Here's some information regarding the basic functionality and processing of usage reports.

To configure usage reporting, a farm administrator must first enable Windows SharePoint Services usage logging for the farm that hosts the Web application containing the SSP. The SSP administrator enables and configures the usage reporting service. Then, site collection administrators can activate the 'reporting feature' to enable usage reports on the site collection.

After usage reporting is enabled, site administrators and site collection administrators can view site usage summary pages that have the following information for their sites and site collections (SpUsageSite.aspx):

  • Requests and queries in the last day and the last 30 days.
  • Average number of requests per day over the last 30 days.
  • A chart of requests per day over the last 30 days.
  • A list of the top page requests over the last 30 days.
  • A list of top users over the last 30 days.
  • A chart of top referring hosts over the last 30 days.
  • A chart of top referring pages over the last 30 days.
  • A list of top destination pages over the last 30 days.
  • Top queries for the last 30 days (if search usage reporting is enabled).
  • Search results top destination pages (if search usage reporting is enabled).

SSP administrators for the search service can view a search usage reports page that tracks the following information (SpUsageSiteSearchQueries.aspx):

  • Number of queries per day over the previous 30 days.
  • Number of queries per month over the previous 12 months.
  • Top queries over the previous 30 days.
  • Top site collections originating queries over the previous 30 days.
  • Queries per search scope over the previous 30 days.

Site collection administrators for the SSP site can view a usage summary page that tracks the following information (usage.aspx):

  • Total amount of storage used by the site collection.
  • Percent of storage space used by Web Discussions.
  • Maximum storage space allowed.
  • Number of users for all sites in the hierarchy.
  • Total hits and recent bandwidth usage across all sites.

Windows SharePoint Services (WSS) 3.0 provides the most basic of reports which are primarily text-based. Site collection administrators can also view a site usage report that includes monthly and daily page hit totals filtered by the following criteria

  • Page
  • User
  • Operating system
  • Browser
  • Referrer URL

Enable and Configure Usage Logging

How to Enable Windows SharePoint Services Usage Logging

Enable Windows SharePoint Services usage logging for the farm hosting the Web application containing the SSP. Please refer the following steps to enable usage logging for the farm:

  • On the Central Administration home page, click Operations.
    On the Operations page, in the Logging and Reporting section, click Usage analysis processing. 
  • On the Usage Analysis Processing page, in the Logging Settings section, select Enable logging. 
  • Type a log file location (by default it is set to  '<%Program Files%\Common Files\Microsoft Shared\Web Server Extensions\12\logs' folder ) and number of log files to create. 
  • In the Processing Settings section, select Enable usage analysis processing, and then select a time to run usage processing. 
  • Click OK.

Enable Office SharePoint Usage Reporting

After Windows SharePoint Services usage logging is enabled in the server farm, SSP administrators must enable the Office SharePoint Usage Reporting service. SSP administrators can control the complexity of usage analysis processing, and select whether or not reporting is enabled for search queries. Please refer the following steps to enable portal usage reporting:

  • On Central Administration home page, click the Shared Service Provider listed under Shared Services Administration in the Quick Launch bar.

  • On the SSP home page, in the Office SharePoint Usage Reporting section, click Usage reporting. 

  • On the Configure Advanced Usage Analysis Processing page, in the Processing Settings section, click Enable advanced usage analysis processing. 

  • In the Search Query Logging section, select Enable search query logging. 

  • Click OK.

    Note: If advanced usage analysis processing is not selected, usage reporting statistics will be minimal.

Reset Internet Information Server

  • Go to the Start Button and click Run
  • Type IISReset and click OK.

How this data makes it back to DB

This differs for WSS and MOSS.

  • In WSS, a timer job called Usage Analysis, runs on each WFE and is then responsible for parsing the usage files and updating the appropriate information for each site’s content DB.
  • In MOSS, a timer job called Office SharePoint Usage Analytics Log Import, runs on each WFE and is responsible for parsing the usage files and uploading this data in the SSP DB.
  • the Office SharePoint Usage Analytics Log Processing job is responsible for parsing and populating the usage report data in the SSP DB’s analytics tables (that use the ANL prefix)
  • It runs every 15 min, to check is there’s new data imported (from the Office SharePoint Usage Analytics Log Import job) so that the reports are updated
  • It also expires detailed data (kept only for 30 days) and report data (kept for 365 days)
  • Windows SharePoint Services 3.0 generates usage event logs daily for each Web application when 'Enable logging' is selected on the Usage Analysis Processing page in SharePoint Central Administration 
  • When logging is enabled, Windows SharePoint Services by default creates log files in the 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\Logs' path on the front-end Web server, although you can specify an alternative location
  • The Logs directory contains a folder for each Web application on the Web server, each named with a GUID that identifies the respective Web application
  • Windows SharePoint Services 3.0 inserts an ampersand (&) between the top-level site URL and the sub site URL when it processes the log files 
  • This marks the log file as "processed" and prevents data from being counted twice if the usage processing job is accidentally run again on the same day.

 
If the connection is located in this file and has the ampersand (&) you will see it reflected in the usage data.

Activate Office SharePoint Usage Reporting

After Office SharePoint Usage Reporting is enabled for the SSP, site collection administrators must activate the reporting feature. Until the reporting feature is activated on a site collection, usage reports are not available. 

Please refer the following steps to activate the reporting feature:

- On the Site Actions menu, click Site Settings. 

  • On the Site Settings page, in the Site Collection Administration section, click Site collection features. 
  • On the Site Collection Features page, click the Activate button for the Reporting feature.

More Information

Whenever Usage Analysis is enabled, the Web Application Servers begin creating usage analysis logs in the ' Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\Logs ' path. There will be a separate folder named with a GUID that represents the web application. Within each of these folders will be a subfolder for each days logs which in turn contains usage logs in the format 01.log, 02.log etc. The usage analysis job runs against the data collected from the previous day(s) logs. For this reason running the Usage Analysis job more than once per day will not update usage data.

On a high level, this is how the usage reports are generated

  • IIS keeps all the SharePoint usage records in memory and will only dump it into physical file (the usage log) when memory is full or IISRESET
  • By default, MOSS has daily timer job (Office SharePoint Usage Analytics Log Import ) to process these physical files into database's temp usage tables
  • Be default, MOSS has another hourly or minutely timer job (Office SharePoint Usage Analytics Processing) that process data in those temp/shadow tables into the real usage tables. 
  • Once this is completed, usage report would then be available for viewing.

In MOSS, the two important timer jobs which are responsible for parsing, processing and updating the SSP DB (the ANL tables) are

- Office SharePoint Usage Analytics Log Import

  • SharePoint Usage Analytics Log processing

These jobs run on each WFE in the farm on a daily basis to process the data in the shadow tables and write the final usage data into another set of tables like ANLHit.
 
‘Office SharePoint Usage Analytics Log Import’ job is responsible for parsing and populating the usage report data in the SSP DB’s analytics tables (that use the ANL prefix) and runs daily to pick up yesterday and only yesterday’s usage log files and parse them into the SSP table like 'ANLShadowHit' while 'SharePoint Usage Analytics Log Processing' runs every minute to process the data in the above shadow tables and write the final usage data into another set of tables like ANLHit.

The following categories are the most relevant for usage reports

  • Office SharePoint Usage Analytics Processing
  • Office SharePoint Usage Analytics Log Import
  • Microsoft.SharePoint.Administration.SPUsageAnalysisJobDefinition
  • Microsoft.SharePoint.Portal.Analytics.UsageProcessingJobDefinition
  • Microsoft.SharePoint.Portal.Analytics.LogImportJobDefinition

Report pages (WSS)

- View site collection reports:       http://sitecollection/_layouts/SpUsageSite.aspx

The Site Usage Report page '_layouts/usageDetails.aspx' shows you the 'Total Hits' for the documents which are very useful information.

Report Pages (MOSS)

  • usage.aspx
  • usageDetails.aspx 
  • SpUsageSite.aspx 
  • SpUsageWeb.aspx 
  • SpUsageSiteQueries.aspx 
  • SpUsageSiteResults.aspx

Note:  The information in the 'Advanced' reports (SPUsageSite.aspx)  is not exactly the same as that in the basic WSS reports (usageDetails.aspx).

While the presentation of the Advanced reports is much easier to take in at a glance, it is all cumulative, or "rolled up" in some way. Daily information is always rolled up from all elements (e.g. pages or referrers), while element information is rolled up across all time. If you want to see the daily breakdown of individual elements, you need  to use the WSS Usage Details report.
 

File

Purpose

Unique Characteristics

usage.aspx

Text Mode Site Collection Usage Summary

The only report that shows total registered site collection users, and storage compared to quota.

usageDetails.aspx

Text Mode Web Usage Details

The only built-in reports that can show a cross-tab of items by day. Monthly summary and daily views for each of several metrics.

SpUsageSite.aspx

Site Collection Scope, graphical and text representations of usage.

Includes Quick-launch links for Search Queries and Results reports.

SpUsageWeb.aspx

Individual Site Scope, graphical and text representation of usage.

Easily digestible information on a single Site.

SpUsageSiteQueries.aspx

Site Collection Scope, graphical and text representation of user queries.

Rolling 30 day and 12 month query charts, Top query phrases, and percent of queries by scope.

SpUsageSiteResults.aspx

Site Collection Scope, summary of query results.

Most popular results, and diagnostic tables. Zero result queries, Best Bet info, queries with low click-through to results

There are two report pages that are extremely useful, particularly for slightly smaller sites, that cannot be reached through the GUI interface in MOSS 2007. They are actually from the basic WSS system, and MOSS inexplicably misses out any direct reference though the administration pages.

  • /_layouts/usage.aspx
  • /_layouts/usageDetails.aspx

Description

WSS page:    'Usagedetails.aspx'  - The url ‘/_layouts/usagedetails.aspx’ brings data from the content database, which is the total hit for the page from ALL locations.

MOSS page:    ‘SpUsageSite.aspx'     ("Advanced" Reporting provides the MOSS version of the Site Collection usage summary) The url ‘/_layouts/spUsageSite.aspx’ brings data from SharedServices_DB, which is processed thru multiple SQL table views and stored. It results only the hits to the page FROM A specific site collection.

In short, the data on the ‘usagedetails.aspx’ page is calculated for any hit (success or failure) to the location whereas the data in the ‘spUsageSite.aspx’ page shows the page which was accessed (and the number of times it was access in the Pie chart) FROM the site collection.

Top referring pages (past 30 days)...........................(Site Collection :  SPUsageSite.aspx)
 
Pie Chart 

URL:                            page that any user visited just prior to browsing to a page in the current site collection. The top 5 URLs are shown.
Slice-value (integer):     total number of times this URL was a referring page over the past 30 days.
 

Top destination pages (past 30 days)...........................(Site Collection :  SPUsageSite.aspx)

Pie Chart

URL:                            Site that a user navigated to from the current site/collection
Slice-value (integer):     the total number of times this URL was a destination page over the past 30 days
 

These are the definitions used by WSS and MOSS in summary usage reports (which are stored in the web metainfo):
 

  1. Visit:             a “total hit” that does not come from within the same server; that is, it either has no Referrer header, or it has a Referrer header from another server
  2. Total hit:       any hit that gets logged in the WSS http logs (we don’t log hits that result in error http results, or hits to the _layouts directory)
  3. Hit:               anything in (2) except hits on files with these extensions: "gif", "jpg", "png", "bmp", "css", "mid", "wav", "ico", "xml", "au", "js", "class"
  4. Request:       Requests