learn-rag-with-phoebe / Builder session 8 of 10
Learn RAG with Phoebe · Builder track · Session 8 of 10

The full pipeline: retrieve, augment, generate, ship

Seven sessions of parts. This is the session they assemble into. You will wire the whole runtime loop - embed the question, retrieve the top-k chunks from the store, build an augmented prompt that carries the context and the grounding instruction, call the Claude API, and stream back one cited, grounded answer. Reranking from b6 and refusal from b7 slot into this loop at named places. By the end, Recall v1 runs end to end over Corpus C. This is concept-level code, not a runnable repo - but every line is the real shape.

🔴 Builder track Practitioners · some Python Live playground included Session 8 of 10
0-3 · Welcome 3-18 · Concepts 18-40 · Build-along 40-45 · Q&A
Part 0

Where every session lands

RAG is three verbs: retrieve, augment, generate. Retrieve is b1 through b6 - embeddings, chunking, the store, metadata, hybrid search, reranking. Augment is the step we name today: taking the retrieved passages and folding them into a prompt with a grounding instruction. Generate is the Claude API call from b7's grounding rules. Today we write the loop that runs all three in sequence, over Corpus C, and streams one cited answer out the other end. That loop is Recall v1.

Live - presented in session Self-study - full depth after class ★ Try it now Sources covered at the end
★ What you build today The end-to-end Recall runtime: a function that takes a question, retrieves top-k from the store, augments a prompt with source-tagged context, calls the Claude Messages API, and streams a grounded, cited answer. You will see exactly where reranking and refusal plug in, and run it over five real support questions.
Part 1 · covers the retrieve-augment-generate loop

The whole loop, wired 8 min live

Here is the runtime path of a single question, start to finish. Nothing in it is new - you built every box in an earlier session. What is new is seeing them run as one function, where the output of retrieval becomes the input to augmentation, which becomes the prompt for generation.

Question "refund time?" Embed query → vector Retrieve top-k store.query() Augment prompt context + question + grounding rule Claude generates messages.create Cited answer [C-refund] One function, five steps. Rerank (b6) sits after retrieve; refuse (b7) sits between retrieve and augment when the top score is too weak.
🔍 Click to zoom - the end-to-end RAG runtime: embed, retrieve, augment, generate, cite
LiveThe augmentation step, up close3 min

Augmentation is the step people skim past, but it is where grounding lives. It takes the retrieved chunks and formats them into the prompt so the model can both use them and cite them. Two habits matter:

  • Tag each passage with its source id. Prefix every chunk with a marker like [C-refund] so the model can point back to it. No source markers in the prompt means no citations in the answer.
  • Keep the grounding instruction in the system prompt. The augmented prompt is the sum of three things: the system grounding rule (from b7), the retrieved passages tagged with ids, and the user's question. Same three ingredients, every request.
LiveWiring the loop in Python3 min

Here is the whole loop as one function. Embed the query, ask the store for top-k, format the context with source ids, and call the Claude Messages API with the grounding system prompt. This is concept-level - your embed and collection come from earlier sessions - but the shape is exactly production.

Python · retrieve → augment → generateimport anthropic client = anthropic.Anthropic() SYSTEM = ("You are Recall. Answer using ONLY the context passages. " "If the answer is not present, say you do not know. " "Cite the source id in [brackets] after each claim.") def answer(question, collection, embed): # 1. retrieve: embed the query, ask the store for top-k q_vec = embed([question])[0] hits = collection.query(query_embeddings=[q_vec], n_results=3) docs = hits["documents"][0] ids = hits["ids"][0] # 2. augment: format each passage with its source id context = "\n\n".join(f"[{i}] {d}" for i, d in zip(ids, docs)) user = f"Context:\n{context}\n\nQuestion: {question}" # 3. generate: call the Claude Messages API resp = client.messages.create( model="claude-sonnet-5", max_tokens=1024, system=SYSTEM, messages=[{"role": "user", "content": user}], ) return resp.content[0].text print(answer("how long does a refund take?", collection, embed))
Model ids you can pass today model="claude-sonnet-5" is the balanced default for a support bot. Reach for claude-opus-4-8 when answers need the most reasoning, or claude-haiku-4-5-20251001 when latency and cost matter most. The rest of the call is identical across all three - the pipeline does not change, only the model string.
Self-studyWhy each chunk gets its own document block3 min read

The Part 1 code stuffs the tagged context into one text block, which is the simplest wiring. To get structured, sentence-level citations back from the API (b7), pass each retrieved chunk as its own document content block with citations enabled instead of concatenating them into one string.

Python · one document block per retrieved chunkdef to_documents(hits): docs = hits["documents"][0] ids = hits["ids"][0] return [ { "type": "document", "source": {"type": "text", "media_type": "text/plain", "data": d}, "title": i, "citations": {"enabled": True}, } for i, d in zip(ids, docs) ] content = to_documents(hits) + [{"type": "text", "text": question}] resp = client.messages.create( model="claude-sonnet-5", max_tokens=1024, system=SYSTEM, messages=[{"role": "user", "content": content}], )

One chunk per plain-text document block gives you sentence-level citations with character-index locations, and the returned cited_text is free of output-token cost. This is the difference between "the model mentioned a source" and "the API told me exactly which sentence backs this claim".

Part 2 · covers streaming + the assembled system

Streaming, and the assembled Recall 8 min live

A support answer that appears all at once after a three-second pause feels broken; the same answer streamed token by token feels instant. Streaming does not change the pipeline - it changes how the last step delivers. And once you stream, you can attach the citations to the text as it lands, so the answer and its evidence arrive together.

Claude messages.stream Refunds take 5-7 days tokens stream in, left to right "Refunds take 5-7 business days." [C-refund] ← citation attached Same answer, better UX. Citations attach to the claims as the text lands, so evidence arrives with the words.
🔍 Click to zoom - streaming delivers the answer token by token, with citations attached to their claims
LiveStreaming the answer with the Messages API3 min

Swap messages.create for messages.stream and iterate the text as it arrives. The retrieve and augment steps are unchanged - only the delivery of the generate step differs.

Python · stream a grounded answer with citations enabledcontent = to_documents(hits) + [{"type": "text", "text": question}] with client.messages.stream( model="claude-sonnet-5", max_tokens=1024, system=SYSTEM, messages=[{"role": "user", "content": content}], ) as stream: for text in stream.text_stream: print(text, end="", flush=True) # tokens arrive live final = stream.get_final_message() # cited_text + location travel with the final message, # free of output-token cost
Streaming UX, without the pipeline changing The whole retrieve-augment-generate loop is identical whether you stream or not. Streaming is purely a delivery choice at the generate step: it drops perceived latency to near zero because the first token appears immediately, while the rest write themselves out as the user reads. Attach each citation to its claim as the text lands and the answer never appears "unsourced" even mid-stream.
Self-studyWhere reranking and refusal slot in3 min read

The loop from Part 1 is the spine. The last two sessions plug into named joints on it - you do not rebuild anything, you insert a step.

Step from b6/b7Where it slots into the loopWhat it changes
Reranking (b6)Right after retrieve, before augmentReorders the top-k so the most relevant chunk leads the context
Refusal (b7)Between retrieve and augmentIf the best score is below threshold, short-circuit and refuse - never reach generate
Grounding (b7)Inside augment (the system prompt)Constrains generate to the context and requires citations
Real world

Refuse before you spend a token. A team (anonymized) put the refusal check after the generate call, so every out-of-scope question still paid for a full Claude response before being thrown away. Moving the threshold check to right after retrieve - refuse before augment - cut their wasted API spend on junk questions to zero. The order of the joints matters, not just their presence.

Self-studyHand-wired loop vs a framework2 min read

Frameworks like LangChain give you the retrieve-augment-generate loop as a few lines of chain configuration. We hand-wired it here on purpose - once you have written the loop yourself, the framework becomes a convenience rather than a black box.

Hand-wiredFramework (e.g. LangChain)
ControlTotal - you see every stepAbstracted behind chain objects
Speed to first demoSlowerFaster
DebuggingStraightforward - it is your codeDepends on the framework internals

The advice for a first production system: hand-wire it, so you understand exactly where reranking, refusal, and citations live. Reach for a framework later, when the loop is boring and you want the boilerplate handled. Either way the three verbs are identical underneath - the framework is just sugar over retrieve, augment, generate.

Build-along · covers the retrieve step feeding the loop

★ See the retrieve step that feeds generate ★ 6 min · live playground

Before you wire the whole loop, watch its first real step in isolation. This is Recall's retrieve over Corpus C - the exact chunks that would become the augmented prompt's context. Whatever ranks in the top-k here is what your build_prompt would tag with source ids and hand to Claude.

Honesty note - what this playground really does This tool runs a simplified lexical embedding offline - term frequency over a small vocabulary with a synonym map, not a neural model. But the ranking mechanic is the real one: cosine similarity over vectors, exactly as production retrieval does it. The end-to-end pipeline on this page is concept-level code, not a runnable repo - this playground shows you only the retrieve step that feeds it, so you can see what the augmentation step actually receives.

Ask "how do I cancel my subscription?" and note the ranked chunks. Those top-k results - [C-subscription] and its neighbors - are precisely the context your loop would format with source ids and pass to messages.create. Seeing retrieve in isolation makes the augment step concrete: the prompt is just these chunks, tagged, plus the grounding rule, plus the question.

Build-along · take it further

Assemble the full Recall script ★ 15 min

Time to put the whole thing on one page. Assemble the retrieve-augment-generate loop over Corpus C, then run five real support questions through it and read the answers critically.

Assemble the loop. Combine the Part 1 answer() function with your b7 grounding system prompt. Use Corpus C's eight chunks as the collection. Keep it one file - retrieve, augment, generate, in order.

Run five questions. Refund timing, password reset, shipping options, subscription cancel, and account deletion. Confirm each answer is grounded in the right chunk and carries a citation like [C-reset].

Add the streaming variant. Swap in messages.stream and watch the same answer arrive token by token. Note the pipeline code above the call did not change at all.

Slot in refusal. Add a threshold check right after retrieve: if the best score is too low, return the refusal string and never call Claude. Test it with "what is your stock price?" and confirm it short-circuits before generate.

★ Recall's status after b8 Recall v1 runs end to end: a question goes in, top-k chunks come out of the store, an augmented prompt carries them to Claude with source ids and a grounding rule, and a cited, streamed answer comes back - with refusal guarding the edges. It is a working support bot. In b9 we stop trusting it by feel and start measuring it - retrieval and answer quality, on a real evaluation set.
Homework

Before session b9 ◐ 45 min total

Source material

Official sources covered

Taught from Anthropic's Messages API and Citations docs, with LangChain for the pipeline framing. Honest note: the code on this page is concept-level - correct in shape and API surface, but not a cloned, runnable repo. Wire it against your own store and key.

Anthropic · Messages APIPart 1-2 · client.messages.create / .stream · model, max_tokens, system, content blocks · claude-sonnet-5 / claude-opus-4-8 / claude-haiku-4-5
Anthropic · CitationsPart 1-2 · document blocks with "citations": {"enabled": true} · cited_text + location free of output tokens
LangChain · RAG pipeline (retrieve → augment → generate)Part 1 · the three-verb framing and loop structure; we hand-wire it rather than use the abstractions
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · What are the three verbs of the RAG runtime loop, in order?

Retrieve (top-k from the store), augment (build the prompt with source-tagged context and the grounding rule), generate (call the Claude API). That loop is the whole pipeline.

2 · What does the augmentation step actually do?

Augment = system grounding rule + retrieved passages each tagged with a source id + the user question. The source tags are what make citations possible.

3 · Switching from messages.create to messages.stream changes...

Streaming is purely a delivery choice at the generate step. The pipeline above the call is unchanged; tokens just arrive live, dropping perceived latency.

Builder session 8 cheat sheet · pin this

The three verbsRetrieve → augment → generate. The whole RAG runtime is these three steps in order.
RetrieveEmbed the query, collection.query for top-k. This is b1-b6 running as one call.
Augmentsystem grounding rule + passages tagged with source ids + the question. The augmented prompt.
Generateclient.messages.create(model, max_tokens, system, messages). Grounded, cited answer out.
Model idsclaude-sonnet-5 (balanced) · claude-opus-4-8 (deepest) · claude-haiku-4-5-20251001 (fastest).
CitationsOne plain-text document block per chunk, "citations": {"enabled": true}. cited_text is free.
Streamingclient.messages.stream(...); iterate text_stream. Same pipeline, tokens arrive live.
The jointsRerank (b6) after retrieve; refuse (b7) between retrieve and augment - short-circuit before generate.