Test and Improve Prompts

Learning Objectives

After completing this unit, you'll be able to:

• Apply a seven-step iteration cycle to refine prompts.
  
• Test prompts with real data.
  
• Identify and fix common prompt problems.
  
• Deploy and monitor prompts in production.
  
• Continuously improve prompts based on results.
  

The Seven-Step Iteration Cycle

Even the best recipes go through rounds of testing. The first time you bake a soufflé, it might fall flat. You adjust the temperature, tweak the technique, and try again. Prompt iteration works exactly the same way. You rarely nail it on the first attempt, and that's completely expected. The seven-step cycle below gives you a repeatable process for getting to your desired outcome.

Testing and improving prompts is an ongoing process. Follow these seven steps:

1. Write. Create your initial prompt using the six components.
   
2. Test. Run it with sample data and real scenarios.
   
3. Evaluate: Measure the results against your success criteria.
   
4. Identify Issues. Identify what worked and what didn't.
   
5. Refine. Adjust your prompt based on what you learned.
   
6. Re-test. Run the refined prompt with the same sample data.
   
7. Deploy. Once it's working well, put it in production.
   

Work through these steps in order, but don't be afraid to loop back. Iteration is not a sign that something went wrong, it's how good prompts become great ones.

Email Generation Example: From Initial to Final

Here's how the iteration cycle works in practice:

Initial Prompt (Version 1):

Write a follow-up email to a customer
Customer: Sarah
Issue: Billing problem
Status: Resolved

Test Result: The email is generic and doesn't feel personal.

Refined Prompt (Version 2):

Write a follow-up email to Sarah who had a billing issue
Include:
- Apology
- Explanation of what happened
- What we did to fix it
Tone: Professional and empathetic

Test Result: Better, but still missing details about the actual resolution.

Final Prompt (Version 3):

Role: You are a customer service manager
Context: Sarah is a 2-year customer who experienced a billing error. We discovered we overcharged her $150 due to a system error on her third billing cycle. We've issued a full refund and applied a $50 loyalty credit.
Task: Write a follow-up email thanking her for her patience and restoring confidence
"""
Customer: Sarah
Issue: Billing error ($150 overcharge)
Our action: Full refund issued, $50 loyalty credit applied
Customer's response so far: Frustrated but cooperative
"""
Instructions:
- Start with a sincere apology
- Explain the root cause (system error on our part, not her mistake)
- Confirm the refund and loyalty credit
- Thank her for giving us a chance to fix it
- Mention one way we're improving our billing system
- Include a discount code for her next purchase (15% off)
Tone: Warm, professional, genuinely apologetic
Format: Email with subject line
Constraints: Keep body under 200 words

Final Result: The email feels personal, specific, and demonstrates genuine effort to fix the relationship. Sarah feels heard and valued.

Salesforce Platform Constraints

Before you deploy, it's worth understanding how Salesforce handles prompt outputs. The field type you’re targeting directly affects how you design your prompt. A prompt that generates a 500-word summary won't fit in a standard text field. Knowing these limits upfront saves you from redesigning after the fact.

When deploying prompts in Salesforce, understand these field limits:

Text Field

• Character limit: 255 characters
  
• Use for: Short responses, summaries, single-line outputs
  
• Limitation: Can't use for long-form content
  

Long Text Area

• Character limit: 131,072 characters (about 20,000 words)
  
• Use for: Full emails, detailed descriptions, multi-paragraph content
  
• Advantage: Plenty of space for most use cases
  

Rich Text Area

• Character limit: 131,072 characters
  
• Use for: Content with formatting, lists, emphasis
  
• Advantage: Supports bold, italics, bullet points, numbered lists
  
• Perfect for: Formatted emails, detailed guides, structured content
  

Deployment tip: Design your prompt with the target field in mind. If deploying to a Text Field, your constraint should be under 255 characters.

Monitoring and Continuous Improvement

Deployment isn't the finish line. It's the starting line. Prompt Builder includes built-in tools to help you monitor performance after go-live. You can preview prompts with real data inputs, track token usage, monitor response time, and compare results across different models or versions. Use these tools to catch issues early and continuously refine.

Think of it like tasting your dish while you’re in the process of cooking. Don’t wait until it's on the plate to find out something went wrong.

Keep an eye out for these signals that a prompt needs attention:

• Outputs that are inconsistent across similar inputs
  
• Responses that feel generic or miss the point
  
• Outputs that get cut off or truncated
  
• Response times that are noticeably slow
  
• User feedback that the output isn't useful
  

If you notice issues, return to refine your prompt and repeat.

Common Prompt Problems and How to Fix Them

Even well-designed prompts run into issues. Here are the most common problems and how to address them:

Deployment Checklist

Use this checklist before deploying any prompt to production to make sure it's ready for real-world use.

• Tested with at least 20 representative examples.
  
• Success criteria clearly defined and met.
  
• Model selected and tested (not assumed).
  
• Prompt follows the six-component structure.
  
• Field size constraints verified.
  
• Edge cases identified and handled.
  
• Documentation written for the prompt.
  
• Success metrics defined for monitoring.
  
• Plan in place for collecting user feedback.
  
• Monitoring and refinement schedule set.
  

Once deployed, make continuous improvement a regular part of your process. The best prompts evolve over time based on real-world usage.