Discover RAG Best Practices
Learning Objectives
After completing this unit, you’ll be able to:
- Identify common RAG best practices.
- Explain the benefit of developing RAG solutions with best practices.
In this unit, we look back at steps you followed while creating your RAG solution, explain best practices and decisions, and define the expected benefit of these decisions.
Use Clean Content
Garbage in equals garbage out. Your RAG-powered solution is only as good as the data that grounds it. While you used clean sample data in the scenario, unstructured content in the real-world is another matter. Successful RAG results start with a knowledge store of well-curated, high-quality content that’s accurate, complete, current, and organized. The content should also be consistent and does not contain contradictory information.
Here are some common ways to prepare your content for retrieval. Some of these apply only if you’re uploading documents.
- Include only recent, relevant, complete, and consistent data.
- Organize and format content consistently and explicitly. For example, use heading levels to indicate hierarchy. Use descriptive titles for tables and images. Use keywords and terms consistently.
- Add metadata, such as source, created date, updated date, document type, product or version, and region or language.
After optimizing your knowledge for retrieval, you can expect to see more accurate and relevant responses to user queries.
Choose the Best Approach: Agentforce Data Library or Manual Setup
In this badge, you set up RAG manually with Data 360. Agentforce also has an out-of-the-box RAG solution called Agentforce Data Libraries (ADLs) that support Knowledge articles and file uploads. The scenario in this badge relies on the Case DMO, not Knowledge articles or file uploads, so ADLs can’t support this badge’s scenario. That’s why you manually set up RAG in Data 360.
Define Your Retrieval Strategy
When creating your search index and retriever, you made specific decisions to define your retrieval strategy.
Use Hybrid Search
You chose hybrid search over vector search. Let’s compare them to understand that decision.
Vector Search | Hybrid Search |
|---|---|
|
|
Your agent answers queries about specific issues with toaster ovens, such as “Why does my toast burn?” So hybrid search is best. Learn more in Search Index Types in Data 360: Quick Look.
Tune and Enrich Your Content Chunks
Your chunking strategy influences how effectively your solution retrieves relevant knowledge. Make sure that the embedding model and chunk size are optimized for your content. Use enhancement features such as section-aware chunking and prepended chunks. Learn more in Chunking Strategies.
Set a Sufficient Retrieval Window
The retrieval window is the number of results your retriever surfaces. You set the number of retrieved results to 5 in the prompt template, which is often sufficient. A narrow retrieval window leaves out lower-ranked documents, so you can miss out on important context. For example, 2 is often too narrow. If you find that your solution isn’t returning enough information, widen the retrieval window.
Index Long-Form Text Fields
Only index long-form text fields that contain at least one semantically coherent, full sentence. Don’t index short fields such as Status, which typically only include one word. In this badge, when creating the search index, you indexed the Subject and Description fields in the Case object. Let’s look at an example of what those fields contain.
- Subject: Power cord caught fire.
- Description: The power cord caught fire the first time I tried to use my toaster oven to cook a pizza.
Both fields are long-form and contain a full sentence.
After choosing hybrid search, setting a wide retrieval window, and indexing long-form fields, you can expect to see more directly relevant material in retrieved results.
Give Explicit Instructions in the Prompt Template
Let’s look at parts of the prompt template to understand how it was strategically written to prevent hallucinations.
The prompt explicitly instructs the model to use only retrieved context for responses.
Find the article-chunk(s) that is most relevant to answer the user query and VERBATIM extract the ID of the article to set <source_id> field in the response JSON. If you are unable to find the relevant article, set <source_id> to NONE.
b. Use the relevant article-chunk to generate the response that exactly answers the user's question and set the <generated response> field.The prompt explicitly tells the model it’s okay to say “I don’t know” if it can’t find relevant information. This sets a negative guardrail and prevents the model from answering off-topic questions.
c. If the user request cannot be answered by the case information provided, set the <source_id> to NONE and <generated_response> to "Sorry, I can't find an answer based on the available articles."
Now, you can expect to see less hallucination and more clear, explainable answers.
Wrap Up
Congratulations! In this badge, you created a grounded service agent that answers customer questions based on Case information. You know how to create advanced RAG-powered solutions in Data 360 with a search index, retriever, and prompt template. You’re saving your company’s service team hours answering customer inquiries, while maintaining a high-quality, personalized customer experience. You also know best practices for effective RAG solutions and how to apply them to your own business.