Measuring agent engagement

Tracking conversation engagement rates and outcomes are crucial to measure the agent performance metrics, and spot areas for improvements.

Understanding conversations and analytics sessions

A single conversation between a user and an agent can generate one or multiple analytics sessions, typically happening when the user has new questions after an initial topic conversation path completed.

Note

Analytics sessions and billed sessions are distinct concepts.

Engaged and unengaged analytics sessions

Analytics sessions are either unengaged (default) or engaged.

This notion is used in the Copilot Studio analytics dashboard.

Tip

Analytics session engagement is also tracked in conversation transcript records in Dataverse, if you have a custom analytics strategy.

In a conversation transcript record content, the engagement status of a session (either unengaged or engaged) is available as part of the SessionInfo activity, in the value type.

  • A session starts when a user interacts with the agent or when the agent sends a proactive message to the user.
  • The analytics session begins in an unengaged state.
  • An analytics session becomes engaged when either entering a custom topic or the Escalate topic.
  • The last custom topic triggered—or the first system topic triggered, if no custom topics were triggered—is associated with the analytics session.

Screenshot displaying a sample copilot engagement scenario.

Note

Having an agent that proactively starts a conversation or that's placed on a website can increase the session total and the number of unengaged sessions.