The other half of RAG
For six sessions we made retrieval good: embeddings, chunking, vector stores, metadata, hybrid search, reranking. But a perfect top-k is worthless if the model then ignores it and answers from memory. Grounding is the contract that the answer must come from the retrieved context - and only from it. Add a citation to every claim so a reader can check it, and a refusal path for when the context does not contain the answer, and you have the difference between a demo and a support bot you can put in front of customers.
Grounded generation: use only what you were handed 8 min live
The generation step of RAG takes the retrieved passages, the user's question, and a system instruction, and asks the model to write an answer. Grounding is what you put in that instruction: answer using only the provided context; if it is not there, say you do not know; cite the source of every claim. Get those three sentences right and most hallucinations disappear.
LiveThe three moves of a grounding prompt3 min▶
Anthropic's "Reduce hallucinations" guidance reads almost like a checklist for a RAG system prompt. Three moves do most of the work:
- Restrict to the provided context. Tell the model explicitly that the answer must come from the passages you supplied, not its training data. This is the single highest-leverage sentence in the whole system.
- Allow "I do not know." Give the model permission to say it cannot find the answer. Models hallucinate partly because they feel obliged to produce something; remove that obligation and the guessing drops.
- Quote first, then answer. Ask the model to pull the supporting sentence out of the context before it writes the answer. If it cannot find a supporting quote, it should not make the claim - and it should retract anything it cannot back with a quote.
LiveWriting the grounding prompt for Recall3 min▶
Here is the pattern applied to Recall over Corpus C. Notice it does three things: sets the rule, formats each passage with a visible source id, and asks for a citation on every claim.
The model now has everything it needs to answer and nothing it needs to make up. The source ids in the context are what make citations possible - the model can point back to exactly the passage it used.
Self-studyThree ways to hand context to the model3 min read▶
The same retrieved passages can be presented to the model in ways that range from careless to rigorous. Know all three, and reach for the strongest your latency budget allows.
| Technique | What it does | Grounding strength |
|---|---|---|
| Stuffing context | Drop all top-k passages into the prompt, no instruction to stick to them | Weak - model may still answer from memory |
| Restrict-to-context | Add "answer using only these passages; say you do not know otherwise" | Strong - cuts most hallucinations |
| Quote-first-then-answer | Model extracts the supporting sentence, then answers, and retracts any claim with no quote | Strongest - evidence before assertion |
The confident wrong answer. A support team (anonymized) shipped a bot that stuffed context but never told the model to stay inside it. Asked about a policy the KB did not cover, it invented a plausible-sounding 90-day return window from its training data. One sentence - "if it is not in the context, say you do not know" - turned that into an honest "I do not have that in our documents." Same retrieval, one line of prompt.
Citations and refusal: the two trust primitives 8 min live
Grounding tells the model to use the context. Citations let a reader verify it used the context. Refusal handles the case the context does not cover. Together they are what make a RAG answer auditable - a claim you can click through to its source, and an honest blank where there is no source.
LiveWhy refusal is a feature, not a failure2 min▶
New builders treat "I do not know" as a bug to be minimized. In a support bot it is the opposite - it is the safety rail. The cost of a confident wrong answer ("yes, we offer a 90-day refund") is a broken promise, a support ticket, and lost trust. The cost of an honest refusal is one more click for the user. Refusal is cheaper, every time.
- The threshold is your dial. If the best retrieval score is below some bar, the answer probably is not in your KB - so refuse. Tune the bar on real questions (that is session b9).
- Refusal protects the citation contract. A model that must cite a source cannot cite one that does not exist. So "no good source" naturally becomes "no answer".
- It is testable. You can write out-of-KB questions and assert the bot refuses. A bot that never refuses is a bot that hallucinates on the edges.
LiveAnthropic Citations on a document block3 min▶
You can ground and cite with pure prompting, but Anthropic's Citations feature does it structurally: pass each retrieved chunk as a document content block with citations enabled, and the API returns the exact cited_text and its location for every claim.
cited_text does not count toward your output tokens - the quoting is effectively free, so there is no cost reason to skip citations.
Self-studyCitation location types3 min read▶
The location Anthropic returns with each citation depends on how you supplied the document. Matching the document type to how your users read the source makes the citations clickable in your UI.
| Document type | How it is chunked | Location returned |
|---|---|---|
| Plain text | Split into sentences | Character-index range (start, end) |
| By page | Page location | |
| Custom content | Your own blocks | Block-index location |
For a RAG pipeline, plain-text document blocks - one per chunk - give the finest, sentence-level citations, which is usually what you want in a support answer. Custom content blocks are the move when you have already chunked deliberately and want the citation to point back to your exact chunk boundaries.
Self-studyPrompt-only citations vs the Citations feature2 min read▶
You can get citations two ways, and it is worth knowing when each earns its keep. Asking the model to write [C-refund] in its prose costs nothing to set up but is only as reliable as the prompt. Enabling the Citations feature returns the exact supporting sentence and its location as structured data you can render as a clickable link.
| Approach | What you get | When to reach for it |
|---|---|---|
| Prompt-only | Source ids written into the answer text | Quick prototypes, logs, internal tools |
| Citations feature | Structured cited_text + location per claim | Customer-facing UIs where users click through to the source |
The two are not exclusive - a production support bot often uses the Citations feature for the clickable evidence and still keeps the grounding instruction in the system prompt, because the instruction is what stops the model answering off-context in the first place. Citations prove the answer; grounding causes it.
★ Feel the refuse case: Recall over Corpus C ★ 8 min · live playground
This is Recall reading Corpus C, a company help-center knowledge base - the same shape of documents a support bot answers from. Ask a real support question and watch the right chunk rise. Then ask something the KB does not cover and watch the tool hit its refuse case - the exact decision your grounding prompt has to make.
Ask "how do I reset my password?" or "how long for a refund?" and confirm the right KB chunk ranks first - that is the answer your grounding prompt would cite. Then ask "what is your stock price?" or "do you offer student discounts?" - things Corpus C simply does not contain - and watch the refuse panel appear. That panel is session b7 in one screen: when the best match is too weak, refuse instead of guessing.
Write Recall's grounding-and-refusal prompt ★ 12 min
A grounding prompt is only trustworthy once you have tried to break it. Write Recall's system prompt, then test it against questions it should answer and questions it must refuse.
Draft the prompt. Write a system prompt that (1) restricts Recall to the provided context, (2) permits and specifies the exact refusal string, and (3) requires a source id citation after every claim. Reuse the Part 1 template as your skeleton.
Three in-KB questions. Test against questions Corpus C answers: refund timing, password reset, shipping options. Confirm each answer is correct, stays inside the context, and carries the right citation like [C-refund].
Two out-of-KB questions. Test "what is your stock price?" and "do you sell gift cards?". Confirm Recall returns your exact refusal string and does not invent a plausible policy.
Tune the tone. A bare refusal feels curt. Add one helpful line ("You might try contacting support Monday to Friday") - but only from context. Notice how the citation rule stops you from padding the refusal with invented specifics.
Before session b8 ◐ 40 min total
- Take your grounding prompt and run it (on paper or against a real key) over the five questions from the exercise. Log which answers cited correctly and whether both out-of-KB questions refused.
- Convert one Corpus C chunk into an Anthropic document content block with
"citations": {"enabled": true}and predict whatcited_textand location you would get back for the question "how long for a refund?". - Write three more out-of-KB questions that are close to Corpus C topics (e.g. "can I get a refund after 45 days?" when the policy says 30). These near-misses are the hardest refusals and become evaluation cases in b9.
- Read: Anthropic "Reduce hallucinations" (the allow-"I do not know" and quote-first sections) and the Anthropic Citations guide.
Official sources covered
Taught from Anthropic's official guidance on hallucination control and citations. This page covers the RAG-relevant core of both; the rest (full API reference, tool-use nuances) stays with the source.
Three questions before you go 🎯 ◐ 90 seconds
1 · What is the single highest-leverage sentence in a RAG grounding prompt?
Restricting the model to the provided context and permitting "I do not know" is what removes most hallucinations - it is Anthropic's core reduce-hallucinations move.
2 · When the best retrieved passage scores far below your threshold, the right behavior is to...
A weak best-match signals the answer is not in the KB. Refusing beats a confident wrong answer; refusal is a feature, not a failure.
3 · With Anthropic Citations enabled on a plain-text document block, what do you get back and at what token cost?
Plain text is chunked into sentences and cited by character-index location; the returned cited_text is free of output-token cost, so there is no cost reason to skip citations.