Use Your Data to Make Better Agents
Learning Objectives
After completing this unit, you’ll be able to:
- Identify issues by using analytics.
- Describe how testing can help narrow down the problem.
- Make modifications to topics or even create new ones using generative AI.
So far we’ve talked about the two main types of analytics for agents: Agentforce Analytics and Utterance Analysis. Having information about your agent, including how it performs, if it’s being used, and if the responses meet user needs is fantastic, but only if you use it to make informed decisions. Let’s take a look at how to do that.
Spotting Trends
The dashboards for Agentforce Analytics and Utterance Analysis can help you see trends, including those that are troubling. Perhaps you notice that the adoption rate is dropping or that utterances that should be handled by topics are instead falling into a general bucket. This is helpful information, but what do you do with it? You test!
Testing Your Agent
Testing Center, found in Setup, lets you use batch testing to check topics and utterances. The simplest way to use it is to let generative AI create the test cases for you, but you can also upload a CSV file with your test cases. Testing Center then runs the test and notifies you when the response doesn’t match the utterance. You can learn how to do this step by step in the Agentforce: Agent Testing module.
Test results = more data! Now what do you do with that? You test in Agentforce Builder. In Setup, open your agent and click on Open in Builder. Here you can test at the conversation level, instead of the batch level, and see what the agent does in real time. By using the failed tests from Testing Center, you also have a head start at pinpointing the problem. Enter the utterance in the conversation panel and watch the results in the planner to see what topics the agent thinks fit the conversation. Not what you expected? You can add or remove topics or edit custom ones until you see the results you want.
Creating a Solution
Once you’ve identified what’s wrong, or at least have a good idea of what needs updating, you can modify or create topics to solve the problem in Agentforce Builder. Generative AI can help you here, too. When you create new topics in Agentforce Builder, you enter what you want the topic to do and click Next.
Generative AI creates everything for you, including all the instructions. You just need to review, make any changes or additions you want, add your actions, and save. Modifying topics is simpler, too, as the tool gives you the option to have generative AI review the description, scope, and instructions and offer improvements. All with just the click of a button.
Check to see if the updates did the trick by restarting the conversation in Agentforce Builder. Repeat these steps until you’re satisfied with the results.
Improving the Coral Cloud Resorts Agent
Sounds easy, right? Alex thinks so. But he knows that he can’t modify an active agent, so he comes up with a plan to reduce the agent’s downtime. First, he identifies clusters of user utterances that don’t look right. He wonders if these clusters are affecting customer satisfaction, so he decides to test them out.
From Setup, Alex enters and selects Testing Center. He creates a new test in the Testing Center and uses generative AI to help create the cases. Next, Alex runs the test. There are some failed utterances where the response doesn’t match the request. He copies a typical request and moves over to Agentforce Builder where he deactivates his agent and inputs the utterance into the conversation panel.
In the planner panel, he sees that the topic the agent picked is not quite right. The agent used a general topic, so Alex thinks a specific topic will likely be his best choice. He clicks New Topic, adds a description, and lets Agentforce Builder's generative AI do the heavy lifting for him by creating the topic name, description, scope, and instructions. He adds the actions he wants the topic to cover and saves it. He restarts the conversation, and this time the agent correctly picks his new topic. Hurray! He reactivates his agent and is back in business.