Share via

Review and improve agent effectiveness in Copilot Studio

  • Article

The upgraded analytics page in Copilot Studio provides an aggregated insight into the overall effectiveness of your agent.

Screenshot of the upgraded analytics section in Copilot Studio, showing the new charts and details options. :::

Tip

By default, the Analytics tab opens to the Summary page in the traditional analytics experience.
Toggle the upgraded experience on and off with the New Analytics switch at the top of the page.

There are three core areas to focus on when reviewing and improving agent effectiveness:

  • Outcomes and engagement: Knowing the end result of a conversation helps you begin to identify where your agent is succeeding and where it needs improvement.
  • Knowledge source use: Seeing how often your knowledge sources are used helps you understand how well your agent is able to provide answers to user questions.
  • User feedback: Reviewing user feedback helps you identify new user scenarios and issues and make improvements based directly on what your users are asking for.

Tip

By default, the page shows you key performance indicators for the last seven days. To change the time period, use the date controls at the top of the page.

Outcomes and engagement

The Outcomes and engagement section shows a chart that tracks the type of outcome for each session between your agent and users.

To open a side panel with a piechart breakdown of sessions outcomes, along with the custom topics that led to each outcome, select See details on the chart.

A session falls into one of the following two states:

  • Unengaged: A session starts when a user interacts with your agent or the agent sends a proactive message to the user. The session begins in an unengaged state.
  • Engaged: A session becomes engaged when one of the following occurs:
    • a non-system topic is triggered
    • the session is escalated
    • the fallback topic is triggered
    • the conversational boosting topic is triggered

Once the session becomes engaged, it remains engaged. An engaged session can have one of the following outcomes:

  • Escalated: A session ends and is considered escalated when the Escalate topic is triggered or a Transfer to agent node is run (the current analytics session ends, whether the conversation transfers to a live agent or not).
  • Resolved: A session ends and is considered resolved when either:
    • The Confirmed Success topic is triggered.
    • The End of Conversation topic is triggered and the user confirms that the interaction was a success or lets the session time out.
  • Abandoned: A session ends and is considered abandoned when an engaged session times out and it didn't reach a resolved or escalated state.

See the guidance documentation Measuring copilot engagement for suggestions and best practices on how to measure and improve engagement.

Knowledge source use

The Knowledge source use section shows a chart and metrics that track how often your knowledge sources are used in conversations between your agent and users.

It also shows trend indicators for how often your sources are being used, how many errors are being generated, and how many times the source is being used to provide answers to user questions.

The chart displays the top 5 knowledge sources used over the date period defined at the top of the Analytics page.

To open a side panel with a list of all knowledge sources used in the specified time period, along with trend indicators, select See details on the chart.

User feedback

The User feedback section shows a chart and metrics that track the average customer satisfaction (CSAT) scores for sessions in which customers respond to an end-of-session request to take the survey.