Where every session lands
RAG is three verbs: retrieve, augment, generate. Retrieve is b1 through b6 - embeddings, chunking, the store, metadata, hybrid search, reranking. Augment is the step we name today: taking the retrieved passages and folding them into a prompt with a grounding instruction. Generate is the Claude API call from b7's grounding rules. Today we write the loop that runs all three in sequence, over Corpus C, and streams one cited answer out the other end. That loop is Recall v1.
The whole loop, wired 8 min live
Here is the runtime path of a single question, start to finish. Nothing in it is new - you built every box in an earlier session. What is new is seeing them run as one function, where the output of retrieval becomes the input to augmentation, which becomes the prompt for generation.
LiveThe augmentation step, up close3 min▶
Augmentation is the step people skim past, but it is where grounding lives. It takes the retrieved chunks and formats them into the prompt so the model can both use them and cite them. Two habits matter:
- Tag each passage with its source id. Prefix every chunk with a marker like
[C-refund]so the model can point back to it. No source markers in the prompt means no citations in the answer. - Keep the grounding instruction in the system prompt. The augmented prompt is the sum of three things: the system grounding rule (from b7), the retrieved passages tagged with ids, and the user's question. Same three ingredients, every request.
LiveWiring the loop in Python3 min▶
Here is the whole loop as one function. Embed the query, ask the store for top-k, format the context with source ids, and call the Claude Messages API with the grounding system prompt. This is concept-level - your embed and collection come from earlier sessions - but the shape is exactly production.
model="claude-sonnet-5" is the balanced default for a support bot. Reach for claude-opus-4-8 when answers need the most reasoning, or claude-haiku-4-5-20251001 when latency and cost matter most. The rest of the call is identical across all three - the pipeline does not change, only the model string.
Self-studyWhy each chunk gets its own document block3 min read▶
The Part 1 code stuffs the tagged context into one text block, which is the simplest wiring. To get structured, sentence-level citations back from the API (b7), pass each retrieved chunk as its own document content block with citations enabled instead of concatenating them into one string.
One chunk per plain-text document block gives you sentence-level citations with character-index locations, and the returned cited_text is free of output-token cost. This is the difference between "the model mentioned a source" and "the API told me exactly which sentence backs this claim".
Streaming, and the assembled Recall 8 min live
A support answer that appears all at once after a three-second pause feels broken; the same answer streamed token by token feels instant. Streaming does not change the pipeline - it changes how the last step delivers. And once you stream, you can attach the citations to the text as it lands, so the answer and its evidence arrive together.
LiveStreaming the answer with the Messages API3 min▶
Swap messages.create for messages.stream and iterate the text as it arrives. The retrieve and augment steps are unchanged - only the delivery of the generate step differs.
Self-studyWhere reranking and refusal slot in3 min read▶
The loop from Part 1 is the spine. The last two sessions plug into named joints on it - you do not rebuild anything, you insert a step.
| Step from b6/b7 | Where it slots into the loop | What it changes |
|---|---|---|
| Reranking (b6) | Right after retrieve, before augment | Reorders the top-k so the most relevant chunk leads the context |
| Refusal (b7) | Between retrieve and augment | If the best score is below threshold, short-circuit and refuse - never reach generate |
| Grounding (b7) | Inside augment (the system prompt) | Constrains generate to the context and requires citations |
Refuse before you spend a token. A team (anonymized) put the refusal check after the generate call, so every out-of-scope question still paid for a full Claude response before being thrown away. Moving the threshold check to right after retrieve - refuse before augment - cut their wasted API spend on junk questions to zero. The order of the joints matters, not just their presence.
Self-studyHand-wired loop vs a framework2 min read▶
Frameworks like LangChain give you the retrieve-augment-generate loop as a few lines of chain configuration. We hand-wired it here on purpose - once you have written the loop yourself, the framework becomes a convenience rather than a black box.
| Hand-wired | Framework (e.g. LangChain) | |
|---|---|---|
| Control | Total - you see every step | Abstracted behind chain objects |
| Speed to first demo | Slower | Faster |
| Debugging | Straightforward - it is your code | Depends on the framework internals |
The advice for a first production system: hand-wire it, so you understand exactly where reranking, refusal, and citations live. Reach for a framework later, when the loop is boring and you want the boilerplate handled. Either way the three verbs are identical underneath - the framework is just sugar over retrieve, augment, generate.
★ See the retrieve step that feeds generate ★ 6 min · live playground
Before you wire the whole loop, watch its first real step in isolation. This is Recall's retrieve over Corpus C - the exact chunks that would become the augmented prompt's context. Whatever ranks in the top-k here is what your build_prompt would tag with source ids and hand to Claude.
Ask "how do I cancel my subscription?" and note the ranked chunks. Those top-k results - [C-subscription] and its neighbors - are precisely the context your loop would format with source ids and pass to messages.create. Seeing retrieve in isolation makes the augment step concrete: the prompt is just these chunks, tagged, plus the grounding rule, plus the question.
Assemble the full Recall script ★ 15 min
Time to put the whole thing on one page. Assemble the retrieve-augment-generate loop over Corpus C, then run five real support questions through it and read the answers critically.
Assemble the loop. Combine the Part 1 answer() function with your b7 grounding system prompt. Use Corpus C's eight chunks as the collection. Keep it one file - retrieve, augment, generate, in order.
Run five questions. Refund timing, password reset, shipping options, subscription cancel, and account deletion. Confirm each answer is grounded in the right chunk and carries a citation like [C-reset].
Add the streaming variant. Swap in messages.stream and watch the same answer arrive token by token. Note the pipeline code above the call did not change at all.
Slot in refusal. Add a threshold check right after retrieve: if the best score is too low, return the refusal string and never call Claude. Test it with "what is your stock price?" and confirm it short-circuits before generate.
Before session b9 ◐ 45 min total
- Write out the full
answer()loop from memory - embed, query, augment with source ids,messages.create- without looking. Gaps you hit are the parts to reread. - Convert the loop to the document-block form (one plain-text block per chunk, citations enabled) and predict the
cited_textyou would get back for "how do I delete my account?". - Add the refusal short-circuit after retrieve and list three questions it should catch before ever reaching Claude.
- Read: the LangChain RAG pipeline tutorial (for the retrieve-augment-generate framing) and the Anthropic Citations + Messages API pages for the exact call shapes.
Official sources covered
Taught from Anthropic's Messages API and Citations docs, with LangChain for the pipeline framing. Honest note: the code on this page is concept-level - correct in shape and API surface, but not a cloned, runnable repo. Wire it against your own store and key.
Three questions before you go 🎯 ◐ 90 seconds
1 · What are the three verbs of the RAG runtime loop, in order?
Retrieve (top-k from the store), augment (build the prompt with source-tagged context and the grounding rule), generate (call the Claude API). That loop is the whole pipeline.
2 · What does the augmentation step actually do?
Augment = system grounding rule + retrieved passages each tagged with a source id + the user question. The source tags are what make citations possible.
3 · Switching from messages.create to messages.stream changes...
Streaming is purely a delivery choice at the generate step. The pipeline above the call is unchanged; tokens just arrive live, dropping perceived latency.