The demo is the start line, not the finish
Everything you have learned across five sessions converges here. You know what RAG is, how it is built, what it costs, how to judge it, and how to govern it. Now the question is operational: how do you move from "it worked in the demo" to "we trust it in production, and we can prove it"? The answer is a series of gates, each adding one thing the last one lacked. A team that ships the demo skips the gates and pays for it later. A team that names its current gate - and what it must pass next - is doing the actual work of productionizing AI. That naming is what you leave this course able to do.
The POC-to-prod roadmap 9 min live
Four stages, each one adding what the previous lacked. You cannot skip a stage; you can only pretend to, and the pretending shows up in production. Everything from sessions a1 through a5 slots onto this strip.
LiveThe trap of shipping the POC3 min▶
A proof-of-concept answers exactly one question: can retrieval work at all for this use case? That is genuinely valuable - it kills bad ideas cheaply. But it is a single hand-picked demo, not evidence of reliability. The trap is treating "it worked when I tried it" as "it works".
- A POC has no scorecard. You watched a few answers look good. You do not know the faithfulness rate, the refusal behavior, or how it does on the hard 10% of questions.
- A POC has no governance. No access filtering, no audit log, no refusal policy. Fine for a demo, a liability in production.
- A POC is unmonitored. When quality drifts - and it will, as documents change - nobody is watching.
LiveWhat each stage adds3 min▶
Read the roadmap as a stack of guarantees, each stage earning the right to the next.
| Stage | Adds | Gate to pass |
|---|---|---|
| POC | Proof retrieval can work | Does a good answer come out at all? |
| Evaluated | A scorecard on a golden set | Do faithfulness and recall clear our thresholds? |
| Governed | Access, refusal, citations, audit | Can it refuse, cite, and respect permissions? |
| Production | Monitoring, freshness, cost caps | Will we notice when it drifts, and can we afford it? |
Notice how the whole course maps on: a1-a2 got you to a POC, a3 made it affordable, a4 made it evaluated, a5 made it governed. This session is about not stopping until production.
Self-studyThe team that carries a RAG system2 min read▶
Production RAG is not one engineer's side project. Four roles keep it honest, and as a leader your job is to make sure each one has an owner:
- Data owner. Decides what goes in the index, keeps it fresh, and retires stale documents. Owns the answer to "is our knowledge current?"
- Retrieval engineer. Owns the search step - chunking, embeddings, re-ranking. The person you call when context recall is low.
- Evaluator. Owns the scorecard and the golden set. Reports faithfulness and recall over time, and proves whether a change actually helped.
- Domain expert. The person who knows whether an answer is actually right in the real world - the source of truth for the golden set, and the final judge of quality.
These need not be four people, but they are four hats. A system missing the evaluator hat ships on vibes; one missing the domain-expert hat optimizes numbers that do not match reality.
RAG vs agents, and where this is heading 8 min live
The frontier is moving fast, so here is the honest, durable version. The big shift: RAG is becoming something an agent decides to use, rather than a fixed pipeline that always runs. Retrieval turns into a tool the model can call when it judges it needs to.
LiveWhen plain RAG is enough vs when you need an agent3 min▶
Do not reach for an agent because it sounds advanced. The distinction is about the shape of the task.
- Plain RAG is enough when the job is "answer this question from our documents". One retrieve, one generate, done. This covers the large majority of real use cases - support bots, policy assistants, research helpers - and it is simpler to build, cheaper to run, and easier to govern.
- You need an agent when the task requires several steps, or a choice between actions: search our docs, then look something up externally, then decide whether to retrieve again. An agent can call retrieval as one of several tools and decide when to use it.
- The plain-language test: if the task is "look it up and tell me", that is RAG. If it is "figure out what to do, using lookup as one of your moves", that is an agent that uses RAG.
LiveContextual retrieval - today's quality frontier3 min▶
If you want one concrete upgrade to ask about, it is contextual retrieval. The problem it solves: when you chop a document into chunks, each chunk loses the context around it - a paragraph that says "the threshold was raised to 30" no longer knows which threshold or when. Contextual retrieval adds a short blurb of surrounding context to each chunk before indexing, so the retrieved passage carries its own meaning.
- Why it matters to you: it meaningfully reduces retrieval failures - the wrong-passage problem from session a4 - which is the root of a lot of bad answers. Sessions a3 and a4 carry the specific numbers.
- How to treat it: as the current best-practice frontier for quality, worth asking your team whether they have adopted. Not a silver bullet - it still needs the same evaluation and governance - but a real step up.
The chunk that forgot what it meant. A team (anonymized) kept seeing their assistant confuse two similarly worded policies. The chunks had been split so finely that neither carried enough context to tell them apart. Adding a one-line context header to each chunk before indexing - contextual retrieval - separated them cleanly, without touching the model at all.
Self-studyThe honest "re-verify the tooling" note, and where to go next2 min read▶
One honest caveat to close on: this space moves fast. The frameworks, the tool names, the exact best-practice recipe - all of it shifts on a timescale of months. Agentic RAG in particular is an active frontier, and the specific way a framework wires it up this quarter may look different next quarter.
- What is durable is everything in this course: the retrieve-augment-generate idea, the three doors, the RAG Triad, the governance controls, the roadmap. These are concepts, and concepts age well.
- What to re-verify is the tooling - which library, which feature, which vendor. Treat any specific tool claim as "true as of when we checked", and check again before you commit.
And that is the leader track. Six sessions ago, "AI that uses our data" was a phrase in a vendor pitch. Now you can define RAG in a breath, sort the three doors, judge an answer on the Triad, govern the risks, and map a project from demo to production. That is a genuinely rare literacy. If you want to feel how it is actually built - the embeddings, the retrieval, the live playground - the builder track starts at b1, and the natural next step for hands-on orchestration is LangChain ↗. Thank you for spending the hours. Go ask your team the good questions.
Place your project on the roadmap ★ 12 min · pen and paper
The whole course pays off in this one honest placement. No code - just your real project and the four gates.
Name a real RAG project (or proposed one) in your organization. Write the four stages - POC · Evaluated · Governed · Production - as a line.
Place your project honestly. Most "live" systems are actually at POC or Evaluated wearing a production badge. Resist the flattering placement.
Name the next gate it must pass and the one thing that gate requires - a golden set, an access filter, a monitoring plan. Be specific: "we need a 50-example golden set owned by [name]" beats "improve evaluation".
Write the sentence you will say in your next project meeting: "We're at ___, the next gate is ___, and to pass it we need ___ by ___." That sentence is your entire course, applied.
The RAG-vs-agent decision for your top use case ★ 8 min · decision drill
One last decision to make with a clear head, before someone sells you an agent you do not need.
Take your top use case and describe the task in one sentence, starting with a verb: "answer questions about...", "look up and summarize...", "decide and act on...".
Apply the test: is it "look it up and tell me" (plain RAG) or "figure out what to do, using lookup as one move" (an agent that uses RAG)? Most use cases are the former.
If you leaned agent, list the multiple steps or choices that justify it. If you cannot name at least two genuine steps or a real branch, it is plain RAG dressed up - go simpler.
Write your verdict and the reason. Then note what quality frontier you would ask about first - contextual retrieval is a strong default - and who on your team owns finding out.
After the leader track ◐ 30 min total
- Finish placing your top project on the roadmap and take the "we're at ___, next gate is ___" sentence into a real meeting this week.
- Make the RAG-vs-agent call on your top use case and write down the reason, so future-you can check whether it held.
- Ask your team whether they have adopted contextual retrieval, and treat any tooling answer as "true as of today - let's re-verify before we commit".
- Optional and encouraged: open the builder track's b1 session to see retrieval built for real, and skim LangChain ↗ for where hands-on orchestration goes next.
- Honestly, which roadmap stage is each of our RAG projects at - and are we treating any POC as if it were production?
- What is the next gate our flagship project must pass, and what single thing does it need to get there?
- Do our four roles - data owner, retrieval engineer, evaluator, domain expert - each have a named owner?
- For our top use case, is this really plain RAG, or does it genuinely need an agent that calls retrieval as one tool?
- Have we adopted contextual retrieval, and how do we keep verifying our tooling as this space keeps moving?
Official sources covered
The leader track teaches from published engineering guidance - no vendor marketing. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · The four stages of the POC-to-production roadmap, in order, are...
Each stage adds what the last lacked: proof it works, then a scorecard, then governance, then production monitoring and freshness. Skipping a gate shows up in production.
2 · When is a full agent warranted over plain RAG?
"Look it up and tell me" is plain RAG. "Figure out what to do, using lookup as one move" is an agent that calls retrieval as a tool. Start simple.
3 · Contextual retrieval improves quality by...
Chunking strips a passage of its surrounding context; contextual retrieval restores it before indexing, cutting the wrong-passage failures that cause bad answers.