Learn RAG with Phoebe

Two tracks from "what is retrieval" to a RAG system you can trust

Retrieval-augmented generation is how you make an LLM answer from YOUR documents instead of guessing. A leader track teaches what RAG is, when it pays off, and how to judge one - no code. A builder track grows a real retrieval assistant across three escalating corpora: embeddings, chunking, a vector database, hybrid retrieval, reranking, grounding with citations, and evaluation. Taught from the official docs and short courses. Feeds straight into learn-langchain-with-phoebe.

2tracks
16sessions
3escalating corpora
45min per session

The leader track 🤝 · for C-level, managers & curious minds · 6 x 45 min · no code, ever

Thinking mode: what RAG actually is, when it beats fine-tuning, what it costs, how to judge whether a RAG system is trustworthy, and how to govern it - each session ends with questions to ask your data team.

🎯 Leader 1 · easy

What RAG is, and why it beats a bigger prompt

Retrieval in plain English, RAG vs fine-tuning vs long-context, and the one question that tells you when RAG pays off.

▶ Leaders start here 🧩 Leader 2 · easy

Anatomy of a RAG system

The nine moving parts you fund - ingest, chunk, embed, store, retrieve, rerank, generate, cite, evaluate - each as the question it lets you ask.

The whole pipeline 💰 Leader 3 · medium

Cost vs quality: the real tradeoffs

Where money goes, where quality is won, build vs buy, and the levers that move accuracy without moving the bill much.

Investment view 📊 Leader 4 · medium

How to judge a RAG system

Groundedness, citations, hallucination, retrieval quality - the RAG Triad scorecard you demand before you trust an answer.

Scorecard session 🛡️ Leader 5 · deciding

Risk & governance

PII in embeddings, who-can-retrieve-what, stale answers, vendor lock-in, and refusal as a control you can insist on.

Governance exercise 🗺️ Leader 6 · deciding

From POC to production

The roadmap, the team, RAG vs agents, and where retrieval is heading in 2026 - contextual retrieval and agentic RAG.

Roadmap exercise

The builder track 🛠️ · for practitioners · 10 x 45 min · Python + a browser playground

One growing artifact: Recall, a retrieval assistant. Every session upgrades the same system - and every idea is felt in the in-browser playground first, so the vector database never gets to be magic.

🔢 Builder 1 · easy

Embeddings 101

What a vector is, why cosine similarity ranks meaning, and why questions rarely share words with their answers. Live playground.

▶ Builders start here ✂️ Builder 2 · easy

Chunking strategies

Fixed, recursive, semantic, sentence-window, parent-document - size and overlap, and why a bad chunk sinks good retrieval.

Recall indexes Corpus A 🗄️ Builder 3 · medium

Vector databases

Chroma end to end - client, collection, add, query - then the pgvector and FAISS deltas, and HNSW vs IVF vs flat.

Recall gets a store 🔎 Builder 4 · medium

Retrieval basics

top-k similarity search, the retriever interface, and how to read a ranked result set with a critical eye.

Recall answers questions 🏷️ Builder 5 · hands-on

Metadata + hybrid retrieval

where-filters, temporal queries over dated incidents, and dense + sparse/BM25 hybrid with the alpha lever.

Recall reads Corpus B 🎚️ Builder 6 · hands-on

Reranking

Bi-encoder vs cross-encoder, retrieve-wide-then-rerank-narrow, MMR diversity, and query expansion / HyDE.

Recall gets precise 📎 Builder 7 · hands-on

Grounding & citations

Answer only from context, cite the source, and the refuse-when-unknown guardrail that kills the confident wrong answer.

Recall reads Corpus C 🔗 Builder 8 · hardest

The full pipeline

Wire retrieve → augment → generate with the Claude API, and stream one cited, grounded answer end to end.

Recall ships v1 🧪 Builder 9 · hardest

RAG evaluation

A golden set, faithfulness, answer relevancy, context precision and recall, hit-rate and MRR, and the LLM-as-judge.

Recall gets a scorecard 🚀 Builder 10 · hardest

Production RAG

Re-indexing and freshness, caching, the latency budget, monitoring, and cost - then the handoff to LangChain.

Recall graduates
easy - everyone medium hands-on / deciding hardest

The running project: three escalating corpora 📚

The builder track grows one assistant, Recall, over the same three document sets - each harder to retrieve well than the last.

Corpus A · Personal

An MD's chief-of-staff notes

Emails, board notes, travel prefs. Pure semantic search: "what did I commit to in the Q2 board call?" Sessions b1-b4.

Corpus B · Product / Ops

Dated Jira incident tickets

Each ticket has a date and a severity. Now retrieval needs metadata and time: "what SEV1s happened in March?" Sessions b5-b6.

Corpus C · Company KB

A help-center for a support bot

Policy articles a customer bot answers from - grounded, cited, and willing to say "I don't know". Sessions b7-b10.

Choose your path 🗺️

Two doors, one course. Leaders never see code; builders never sit through theory they have not first felt in the playground.

🤝 Leader (no code, ever) a1→ a2→ a3→ a4→ a5→ a6
🛠️ Builder (grow Recall) b1→ b2→ b3→ b4→ b5→ b6→ b7→ b8→ b9→ b10
🌱 Curious follower (a taste) a1→ a4→ b1
🚀 Full journey (lead AND build) a1-a6→ b1-b10→ LangChain ↗

The knowledge map 🧠

Both tracks at a glance - hover a session to spotlight its concepts, click any node to jump in.

The whole course on one sheet 📐

Seven modules: both tracks, the refusal that makes RAG trustworthy, the pipeline stage by stage, the builder ladder from embeddings to production, where retrieval quietly fails, and what good actually means when you measure it.

Download for print (2800px) Every word here is real selectable text, so it stays sharp however far you zoom.

learn RAG
with phoebe

Answer from the sources, or not at all
16 sessions - two tracks - one growing assistant, Recall - three escalating corpora

SEC-01 TWO TRACKS

LEADER TRACK
judge one, no code

  1. a1What RAG is, and why it beats a bigger prompt
  2. a2Anatomy of a RAG system
  3. a3Cost vs quality: the real tradeoffs
  4. a4How to judge a RAG system
  5. a5Risk and governance
  6. a6From POC to production

BUILDER TRACK
grow Recall yourself

  1. b1Embeddings 101
  2. b2Chunking strategies
  3. b3Vector databases
  4. b4Retrieval basics
  5. b5Metadata and hybrid retrieval
  6. b6Reranking
  7. b7Grounding and citations
  8. b8The full pipeline
  9. b9RAG evaluation
  10. b10Production RAG
16sessions
6leader
10builder
3corpora

SEC-02 THE REFUSAL

Session b7 names the standard

Answer from the sources, or not at all. A system that will always produce something is not grounded, it is fluent. The ability to decline is the feature, not a gap in coverage.

What grounding actually requires
every claim -> a retrieved chunk every chunk -> a citation no chunk -> no answer
This is also what makes a RAG system auditable. A citation is a claim someone else can check.

SEC-03 WHY NOT A BIGGER PROMPT

a1
retrieval scalesThe corpus grows without the prompt growing with it.
a3
cost follows tokensSending everything every time is the expensive way to be wrong.
a5
the documents stay yoursRetrieval reads what you point it at, and governance is about what that is.
b7
answers become checkableA cited chunk can be opened. A confident paragraph cannot.
Track A ends at POC to production, because judging a RAG system is a leadership skill before it is an engineering one.

SEC-04 THE BUILDER LADDER

1
embeddings turning meaning into numbers
b1
2
chunking cutting documents so retrieval works
b2
3
vector database where Recall's chunks live
b3
4
retrieval basics ask, rank, take the top few
b4
5
metadata, hybrid when meaning alone is not enough
b5
6
reranking retrieve wide, then get precise
b6
7
grounding, citations answer from the sources, or not at all
b7
8
the full pipeline retrieve, augment, generate, ship
b8
9
evaluation measuring what good means
b9
10
production keeping Recall alive, and the handoff
b10
How to climb it

Rungs 1 to 4 get an answer out at all. Rungs 5 and 6 are where quality actually comes from, and they are the two most people skip. Rung 7 is what makes the answer trustworthy, and rung 9 is what lets you prove any of it.

SEC-05 THE PIPELINE

1
RETRIEVEask, rank, take the top few
2
RERANKretrieve wide, then get precise
3
AUGMENTput the chunks in front of the model
4
GENERATEanswer only from what was retrieved
5
CITE OR REFUSEno supporting chunk, no claim
Session b8 builds the whole of it end to end, on the corpus grown across b1 to b7.

SEC-06 WHERE RETRIEVAL QUIETLY FAILS

The chunk boundaryThe answer was split across two chunks, so neither one contains it. Ask: what happens to a table, or a clause, at the cut?
Meaning is not enoughSemantically close, factually the wrong document. Wrong year, wrong entity, wrong version. Ask: is there metadata we should be filtering on?
Top-k too narrowThe right chunk was ranked seventh and k was five. Ask: do we retrieve wide and then rerank?
Confidently ungroundedThe model answered anyway, from its own memory, in the same tone. Ask: can this system decline?
None of these announce themselves. They all return a fluent answer, which is exactly why session b9 exists.

SEC-07 WHAT GOOD MEANS

Session b9 makes it measurable

Judging a RAG system means judging two things separately: whether the right context was retrieved at all, and whether the answer stayed inside it. A single end-to-end score hides which half broke.

step
retrieval qualityWas the answer-bearing chunk in what came back?
step
groundednessIs every claim supported by a retrieved chunk?
step
the handoffWho keeps it alive after launch, and how do they know it drifted?
learn RAG with phoebe retrieve - augment - generate - or refuse by Phoebe Fu