Measuring The Impact Of PowerBI: Usage Analytics By User

Having led the design and launch of a major PowerBI project in K-12 in a former life, one of the things that bugged me was the difficulty in truly measuring how much impact and usage of these new reports was actually happening amongst the staff. At the time, I cobbled together a few filters on our Fortigate firewall to show me traffic to PowerBI.com, however I freely acknowledged at the time this was a bit of a blunt force instrument for what I was trying to find out.

Enter PowerBI Usage Metrics back in June. This was a significant step forward in the sense that it started to reveal which reports were being more actively viewed compared to others. Accessing this content was very easy:

To be able to view the Usage Metrics you needed to have Edit permissions to the report itself, thus enabling a simple way to restrict access to this key information.

The Usage Metrics allowed you to slice on a few variables, including whether the report was accessed via the web or the mobile app e.g.

Once you’ve created a copy, you’ll get full access to the underlying dataset, allowing you to fully customize the usage metrics report to your specific needs. You can even use Power BI Desktop to build custom usage metrics reports using the live connection to Power BI service feature.

This announcement, whilst exciting, seemed to lack key feature that I would be looking for if I was trying to drive uptake with this type of tool across a staff group or team – the ability to see exactly who was accessing the reports. It’s imperative to understand which teams or departments are utilizing BI and then try to observe if this is having an impact on performance.

With an announcement today, you can now see exactly who in your organisation is accessing the content meaning you can do a few things:

1. Talk to the most frequent users of reports and find out what is working for them and what they would like to see improved / added to the reports.
2. Identify those users that are still not making use of this type of data and provide some coaching sessions on how they can use this data to improve their performance or add value in their role.

Note on the right showing user access to reports

Tip: the UserGuid (aka Object ID) and UserPrincipalName are both unique identifiers for the user in AAD. That means if you export the usage metrics data, you could join the usage metrics data against more data from your directory, like organizational structure, job title, etc.

Additionally, there are new features for IT Admins to control access to Usage Metrics that are worth reading about here. The following table shows what metrics are available within Usage Metrics:

What metrics are reported?

Metric	Dashboard	Report	Description
Distribution method slicer	yes	yes	How users got access to the content. There are 3 possible methods: users can access the dashboard or report by being members of an app workspace, by having the content shared with them, or by installing a content pack/app. Note that views through an app are counted as “content pack.”
Platforms slicer	yes	yes	Was the dashboard or report accessed via the Power BI service (powerbi.com) or a mobile device? Mobile includes all our iOS, Android, and Windows apps.
Report page slicer	no	yes	If the report has more than 1 page, slice the report by the page(s) that was viewed. If you see a list option for “Blank,” that means a report page was recently added (within 24 hours the actual name of the new page will appear in the slicer list) and/or report pages have been deleted. “Blank” captures these types of situations.
Views per day	yes	yes	Total number of views per day – a view is defined as a user loading a report page or dashboard.
Unique viewers per day	yes	yes	Number of different users who viewed the dashboard or report (based on the AAD user account).
Views per user	yes	yes	Number of views in the past 90 days, broken down by individual users.
Shares per day	yes	no	Number of times the dashboard was shared with another user or group.
Total views	yes	yes	Number of views in the past 90 days.
Total viewers	yes	yes	Number of unique viewers in the past 90 days.
Total shares	yes	no	Number of times the dashboard or report was shared in the past 90 days.
Total in organization	yes	yes	Count of all dashboards or reports in the entire organization which had at least one view in the past 90 days. Used to calculate rank.
Rank: Total views	yes	yes	For total views of all dashboards or reports in the organization over the past 90 days, where does this dashboard or report rank.
Rank: Total shares	yes	no	For total shares of all dashboards in the organization over the past 90 days, where does this dashboard or report rank.