Why "good" needs three words, not one
Most business AI today is a lookup system: given a question, it first fetches relevant material - policy documents, past tickets, product specs - and then writes an answer from what it found. This is called retrieval-augmented generation, RAG for short, and you do not need the acronym, just the shape: find, then answer. The reason it matters for leaders is simple. When the answer is wrong, there are two very different places it could have gone wrong, and if you cannot name which one, you cannot fix it or even ask the right question. This hour gives you the naming.
The RAG Triad 9 min live
For a find-then-answer system, quality is not one thing - it is a triangle of three checks, and each corner catches a different failure. When all three pass together, you have strong evidence the answer is grounded in real material rather than invented. This is the single most useful mental model in the whole course.
LiveContext Relevance - did we even find the right material?3 min▶
Context Relevance asks a question about the lookup step, before a single word of the answer is written: of the material the system pulled up to answer with, how much of it actually relates to what the user asked?
- Plain English: "Did we fetch the right pages?" If someone asks about refund timing and the system retrieves the shipping policy, context relevance is low - the answer was doomed before it started.
- The failure it catches: the system confidently answering from the wrong source material. A perfect writer cannot save a wrong document.
LiveGroundedness - is every claim actually backed by the material?3 min▶
Groundedness is the anti-hallucination check. It asks: is each statement in the answer actually supported by the material that was retrieved, or did the model add things from thin air?
- How it is scored, in plain terms: break the answer into its individual claims, check each claim against the retrieved context, and score the fraction that are supported. Nine of ten claims backed by the documents is a groundedness of 0.9. This "break into claims, check each" method is worth remembering - it is exactly how the machine does it too.
- The failure it catches: the fluent invention - a plausible detail, number, or policy that appears nowhere in the source. This is the failure that quietly misinforms customers.
LiveAnswer Relevance - did we actually answer the question asked?3 min▶
Answer Relevance closes the loop back to the user. An answer can be built from the right documents and be perfectly grounded in them, and still miss - because it answered a slightly different question, or buried the point, or trailed off into a related topic.
- Plain English: "Was this a reply to what they actually asked?" Correct and grounded but off-target is still a bad answer.
- The failure it catches: the technically-true non-answer - accurate, sourced, and unhelpful.
Retrieval quality vs answer quality 8 min live
Step back from the three corners and there are really two halves to a find-then-answer system: the half that finds the information, and the half that writes with it. Measuring both is not optional - because either half can pass while the other fails, and the two failures need completely different fixes.
LiveWhy you must measure both halves3 min▶
It is tempting to measure just the final answer - that is what the user sees, after all. But a single answer score cannot tell you which half to fix, and worse, it can lull you.
- Great retrieval + bad answer = still bad. The system found the perfect document and then wrote a muddled, off-target reply. The retrieval score looks great; the customer is unhappy.
- Bad retrieval dooms even a great generator. The best writer in the world, handed the wrong documents, produces a confident wrong answer. No amount of writing skill recovers from being fed the wrong facts.
- Two halves, two teams, two fixes. Fixing retrieval means better search and indexing. Fixing generation means better prompting and instructions. If you only know "the answer was bad", you do not know which team to send it to.
LiveThe one-line diagnostic every leader should keep3 min▶
Here is the takeaway to write on a sticky note. When an answer is bad, the Triad tells you where to look:
- Low groundedness - the answer contains claims the documents do not support - means the generation half is the problem. The right info may have been there; the model invented or drifted. Fix the writing side.
- Missing information - the right material was never retrieved, so low context relevance - means the retrieval half is the problem. Fix the search side; no prompt tweak will conjure a document that was never found.
The two-week wild goose chase. A team (anonymized) spent two weeks rewriting prompts to fix a bot that kept giving incomplete answers. Quality barely moved. When they finally looked at the retrieval half, they found the search was only ever pulling the first page of each policy - the answers were incomplete because the material was incomplete. The prompt was never the problem. The Triad would have pointed them at retrieval on day one.
Self-studyWhere the Triad comes from, and its cousins2 min read▶
The RAG Triad framing was popularized by DeepLearning.AI's short course on building and evaluating advanced RAG, taught with the open-source TruLens tool, which scores Context Relevance, Groundedness, and Answer Relevance on real traffic. A parallel open-source framework, RAGAS, offers a similar family of metrics under slightly different names - faithfulness for groundedness, and so on. You do not need to pick a tool today. You need the concept: three checks, two halves, and a diagnostic that tells you which half broke. The builder track puts real numbers on all three.
Score a real answer on the Triad ★ 12 min · pen and paper
You do not need code to grade the Triad - just judgment. Here is one question, the material the system retrieved, and the answer it gave. Rate each corner pass or fail, and say which half broke.
Question: "How many vacation days do first-year employees get?"
Retrieved context: "The employee parking policy allows two reserved spaces per department. Badge access is granted on the first day."
Answer given: "First-year employees receive 15 vacation days per year."
Context Relevance - pass or fail? Does the retrieved material relate to the vacation question? Decide, and note what that tells you about the lookup step.
Groundedness - pass or fail? Is the "15 days" claim supported by the retrieved context? Check the answer's claim against the material in front of you.
Answer Relevance - pass or fail? Taken alone, does the answer address the question that was asked?
Which half broke? You should find context relevance failed (wrong material), groundedness failed (the number is invented - it appears nowhere), and answer relevance passed on its face. Wrong material plus a confident invented number - the failure started in retrieval and the model papered over it. This is the classic hallucination pattern.
Sort the complaints ★ 8 min · your next AI review
Real quality complaints arrive as vague grumbles. The skill is sorting each into "retrieval problem" or "answer problem" - because that routes it to the right fix. Sort these six.
"It made up a discount that does not exist." → Retrieval or answer? (The claim is unsupported - that is a groundedness failure, an answer problem.)
"It said it could not find anything about our return policy, but we have a whole page on it." → (The right document was never fetched - a retrieval problem.)
"Everything it said was true, but it never actually answered my question." → (Low answer relevance - an answer problem.)
"It answered about the wrong product entirely." → (Wrong material pulled up - a retrieval problem.)
"The facts were right but it added a warranty term that is not in our policy." → (An invented claim on top of good material - a groundedness failure, an answer problem.)
"It only ever quotes the summary and misses the detailed clauses." → (Incomplete material retrieved - a retrieval problem.)
Before session a3 ◐ 30 min total
- Take three real bad answers from an AI system you know, and score each on the Triad - context relevance, groundedness, answer relevance - pass or fail.
- For each, write one sentence: "This is a retrieval problem" or "This is an answer problem." Notice the split.
- Bring the most surprising one to your next AI review and ask the team to confirm your diagnosis.
- Optional: skim the builder track's b2 to see the same three numbers scored automatically - you do not need the code, just the shape.
- For our main AI system, do we measure retrieval quality and answer quality separately, or only the final answer?
- Do we track something like groundedness - the fraction of the answer's claims actually supported by the retrieved material?
- When an answer is wrong, can we tell whether the right information was found but misused, or never found at all?
- Do we check that the retrieved documents were even relevant to the question, before judging the answer?
- Would passing all three - context relevance, groundedness, answer relevance - give us confidence the answer is not hallucinated?
Official sources covered
The leader track teaches from published engineering guidance and course curricula - no vendor marketing. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · The three corners of the RAG Triad are...
Context Relevance (right material found?), Groundedness (answer supported by that material?), and Answer Relevance (does it address the question?). All three passing is strong evidence the answer is not hallucinated.
2 · Groundedness is scored, in plain terms, by...
Break the answer into claims, check each against the context, and the score is the fraction supported. Nine of ten claims backed = groundedness of 0.9. It is the anti-hallucination check.
3 · An answer is wrong and its groundedness is low. The diagnostic says...
Low groundedness means claims are not supported by the retrieved context - the answer half misused or invented. Missing context, by contrast, points at the retrieval half.