Monitor Agent Conversations with Utterance Analysis
Learning Objectives
After completing this unit, you’ll be able to:
- Understand how Utterance Analysis groups user requests into clusters and categories to identify trends and refine topics and actions.
- Perform a reverse search to identify unsupported requests and improve agent response effectiveness.
- Customize Utterance Analysis reports.
Your agent’s performance is a big piece of the picture, but not the whole picture. If it makes sense for your company, and there aren’t restrictions around data gathering that affect you, such as industry regulations or laws, you can view what users input into your agent.
The requests or questions your users ask your agent are called user utterances. By default, this data isn’t captured as privacy regulations can forbid it. But if that’s not the case for you, you can enable utterance gathering by turning on Feedback in Einstein Feedback and Monitoring Setup.
Utterance Analysis provides you with customizable reports that track user interactions, input, and agent response types. By using these reports, you can gain valuable insights into how your users are engaging with your agents and identify areas for improvement.
Accessing Utterance Analysis
The Utterance Analysis dashboard also lives in the Agentforce (Default) folder. Look for it in the Dashboard tab. The Utterance Analysis dashboard includes reports that track users, their input, and the agent response type. Reports are run on weekends and the dashboard is updated weekly, giving you enough aggregate data for the big picture and fresh enough data to keep on top of trends.
Working with Clusters and Categories
By default, Utterance Analysis brings similar utterances together into groups called Clusters. These clusters are then grouped into Categories. By grouping this information, you can see the overall trends of what users ask for, which helps you identify common requests. You can also see areas where agents might need more support.
You can also do a reverse search and filter for unsupported requests, which are marked as system topics. Your agent can use system topics for things like small talk, but it can also draw on them when it isn’t able to create a meaningful response. Accessing this information helps you determine where gaps are and when to create new topics and actions to meet the needs of your users.
Customizing Utterance Analysis Reports
Utterance Analysis is built on Data Cloud reports, which are easily modified with the flexibility to customize reports to suit your needs. But be sure to clone the report and then modify the clone, not the original, so nothing breaks in the dashboard.
Got the basics? Let’s see how it can be used in practical terms.
The Customer Speaks: Coral Cloud Resorts
Alex Wu wants to know what users are asking to understand how they’re using his agent. Since neither his industry nor his region have any restrictions around gathering this type of data, he enables Utterance Analysis. To do so, he goes to Setup and turns on the Feedback section of the Einstein Feedback and Monitoring Setup page. He knows that Utterance Analysis gathers data throughout the week and populates the reports over the weekend, so he comes back to check on Monday.
Just like the Agentforce Analytics dashboard, the Utterance Analysis dashboard has widgets showing various charts and lists based on reports.
Alex can see how many unique users interacted with his agent and the types of questions and requests they were asking. He notices a lot of requests are in the Greetings and Salutations category, which makes him wonder if he needs to refine an existing topic or create a new one. He opens the report and can see the actual utterances users made. It seems that most of them are being polite and saying hello to his agent. No need for further adjustments.