Test, Launch, and Improve Your Bot

Learning Objectives

After completing this unit, you’ll be able to:

• Test your bot for launch.
• Prepare customer support agents to handle escalated cases.
• Launch your bot into the world.
• Improve your bot over time.

Test Your Bot for Launch

As Harryette browses between tabs in Bot Builder, she finds herself beaming with pride. From planned use cases to custom dialogs, she’s finally brought her chatbot to life. 

Bloomington Bot is almost ready to enter the real world. Almost. To make sure everything is running smoothly before launch, Harryette first has to test her bot’s features. 

Having designed her bot for Salesforce Chat, Harryette can seamlessly test her bot functions through the Preview tab within the Bot Builder. There, she completes the first round of user testing by navigating through each menu option and exploring all the different routes users can take. She looks for any breakdowns in the conversation flow, and she types in unexpected responses to double check the bot’s error-handling. 

To see how the bot will look on the Bloomington Caregivers site, where it will soon live, Harryette collaborates with developers to launch the latest version of Bloomington Bot to an internal site, so that authorized staff can access it. She then troubleshoots and fixes any discrepancies, enlisting the help of developers as needed.

Now that the bot is live on an internal site, Harryette asks members of multiple departments to give the bot a try and collects their feedback on the experience. For this phase of the bot development, she’s primarily testing for a cohesive user flow. When she implements natural language processing in phase two, she'll test for the bot’s understanding of common user requests and validate the accuracy of its intent model. 

Prepare Customer Support Agents

After all the bot’s functions are validated and fine-tuned, Harryette prepares her support agents to pick up where the bot leaves off. She provides a breakdown of the types of requests that agents can receive and a demo to show what each one looks like. Working with the customer support team, she puts together general guidelines for how to handle each request. 

Launch Your Bot into the World

Now for the easy part, the moment Harryette and her team have been patiently waiting for: Bloomington Bot’s official launch! 

To deploy her bot, Harryette migrates it from the sandbox into the production org. All she has to do now is connect the bot to the channel and activate it. She follows the deployment instructions in Salesforce Help to add Chat to her bot. Since she built the first iteration of the bot for Chat, she goes to the Embedded Service tab in Setup to find the code snippet for the chat window. She then follows these directions to add the code snippet to the Bloomington Caregivers website and proceeds to launch the latest version of the site. 

To monitor the bot’s performance after it goes live, Harryette engages her launch day support team, which includes some customer service agents and a developer. They regularly check the Performance tab for any unexpected errors in production and visits the Einstein Bot Reports folder in the Reports tab to track hourly metrics on how the bot picks up traction and performs with customers. 

After all their preparation and building, if everything goes smoothly now, Harryette and her team can finally sit back and watch Bloomington Bot take care of customers by itself. 

Improve Your Bot Over Time

After the launch, Harryette and her team no longer have to monitor the bot on a daily basis. Their post-launch responsibilities fall under maintenance and improvement tasks, and follow an adaptive, iterative process based on how the bot performs. 

After Bloomington Bot’s first week out in the wild, Harryette reviews the data they collected from the first batch of customers and analyzes the reports. She asks herself a few key questions in search of areas for improvement.

• Are there any dialogs that stand out in the reports? These can include dialogs associated with unexpected user behaviors or exceptionally high or low engagement.
• What do these user patterns say about the dialogs?
• Which dialogs are customers ending their sessions on?
• Which dialogs are customers transferring to support reps on?
• Are there any errors that need the developers’ attention?

To gather additional insight on how users behave on each dialog, Harryette uses the Einstein Bot Reports folder in the Reports tab. By looking for unusual behavior on specific dialogs, Harryette can zero in on discrepancies in the bot and improve on them. Beyond the components of the bot, she also evaluates how the bot as a whole has been contributing to Bloomington Caregivers’s business goals. 

At the end of the quarter, Harryette is ready to stack up the bot’s performance data against the KPIs she and her team identified earlier in the planning process. She looks at key data points like the number of cases the bot handled and deflected, customer satisfaction scores, and the bot use cases that the customers used most. 

Using these insights, Harryette assesses whether her bot has met the company’s KPIs, and she meets with stakeholders to discuss whether any big-picture adjustments are necessary. She also looks for growth opportunities that she can apply in future iterations of the Bloomington Bot, makes a plan for improvement, and repeats this process every quarter. 

Now that you’ve followed Harryette through every step of the bot implementation process, you’re ready to create and develop your own bot and send it out into the world!