Using Office Telemetry Dashboard to see how well your Office solutions perform in Office 2013
This post is written by Makoto Yamagishi, a Program Manager with the Office Solutions Management team
In our last blog post, we introduced the modern Office compatibility process and the Office Telemetry Dashboard. We also showed you how to identify the most important documents and solutions. In this post, let's take a closer look on how you can use the Telemetry Dashboard to see just how well those important solutions are running in Office 2013.
Test & monitor your solutions
This section covers the last two steps of the modern Office Compatibility process:
Step 3 Start a pilot deployment, focusing on the most critical solutions: Proactively monitor how your business-critical documents and add-ins are behaving when they are tested. Quickly figure out resolutions to issues.
Step 4 Deploy Office and continue monitoring your solutions: Deploy the new Office, and look for cases of errors or poor performance to be addressed.
Now that you know what the most popular solutions in your organization are, Office 2013 can report on the stability and performance of individual documents and add-ins. When you start piloting Office 2013, ask your end users to use Office 2013 with the documents and add-ins that they normally run—Office Telemetry will report back on them. If you have critical documents that are only run seasonally, maybe at tax time or year-end, it’s a good idea to try those out in your pilot process as well. As your users experience Office 2013, the columns of data in the Office Telemetry Dashboard’s views will start filling up so you can see how the critical ones are behaving.
The following columns located on the right side of the view are especially important when piloting:
- Success (%) indicates the stability of the documents/add-ins. This is the percentage of sessions where the document loaded properly or ran without a critical error or crash. This number goes down if users experience load failures or if macros are calling methods or properties in the object model that work very differently from earlier versions.
- Trend shows if the success rate is getting better or worse over time.
- Critical shows the number of users who have hit load failures, crashes or other high severity issues.
- Load time shows the average time Office applications takes to load each add-in in seconds.
Here is a screenshot of the Solutions worksheet that shows the columns that I just described:
When there are documents and add-ins with low success rates, you will want to investigate what issues are happening. You can do this by just following the hyperlinks.
By clicking the ‘Critical’ counts, you can drill-through the view to see the details of the each critical issue that users are experiencing. Critical issues are the issues that are making documents or add-ins unusable, such as load failures, crashes, and use of deprecated object model elements. You can drill-through further to get the session log for each issue.
Here's a screenshot that illustrates the critical issues that occurred when users use DocStatusTracker.xlsx.
Even after you are done piloting, you can continue using these views to monitor your documents and add-ins. This can provide peace of mind as your environment changes with new hardware, drivers, applications, or security patches. Or you may want to monitor to keep unauthorized add-ins from invading the corporate network. To see a high-level summary of the Success rate status and trends, see the Overview sheet, which we'll discuss in a future blog post.
How does Office Telemetry work?
So far we’ve talked about the benefits of deploying Office Telemetry and incorporating it into your deployment plans. Let me briefly explain what components are running under the covers to collect data and enable the reporting views.
Office Telemetry consists of five key components:
- Office Telemetry Logging
- Office Telemetry Agent
- Office Telemetry Processor
- Office Telemetry Dashboard
- Office Telemetry Log
The following graphic illustrates the flow of data between some of the components of Office Telemetry.
Although not shown in the previous graphic, Telemetry Logging is integrated into Office 2013 client applications (Word, Excel, PowerPoint, and Outlook). Once enabled, it logs which files and add-ins are being loaded and out-of-the-ordinary events. Examples of these events include load success/failure, crashes, use of deprecated ActiveX controls, and calls to object models with known compatibility issues.
Telemetry Agent sends local telemetry data files to a shared folder on your network. The Telemetry Processor is a Windows service application that takes the data files from the shared folder, extracts the information, and adds the telemetry data to a SQL Server or SQL Server Express database. As mentioned in the previous blog post, Telemetry Agent can be deployed to Office 2003, Office 2007, and Office 2010 as well. Information about compatibility issues, load failures, and other events is only reported from Office 2013 clients.
Telemetry Dashboard is an Excel 2013-based solution that shows the telemetry data that shows various views by connecting to a SQL Server database. A companion tool to Telemetry Dashboard is Office Telemetry Log, which you can use to access Telemetry Logging data. This is another Excel 2013-based solution that displays local Office events, including compatibility issues. It allows you to more quickly troubleshoot the issues on the local PC.
Here is a screenshot that shows Office Telemetry Log and the events that have occurred on the computer where Telemetry log is running.
Here is a screenshot that shows Office Telemetry Log and the events that have occurred on the computer where Telemetry log is running.
Want to learn more?
For details about each component described in this post, take a look at the TechNet article detailing how to deploy the Office Telemetry Dashboard. For Telemetry Log, please also take look at Troubleshooting Office files and custom solutions with the Office Telemetry Log.