learn-ai-evals-with-phoebe / Leader session 6 of 6
Learn Evals with Phoebe · Leader track · Session 6 of 6

Building an eval culture a habit, not a heroic one-off

You now know why evaluation matters, what good means, the difference between offline and online, the scorecard to demand, and the ways it can lie. This final session answers the only question that decides whether any of it sticks: how do you make evaluation a habit your team does by default, instead of a heroic scramble someone runs the week before a launch and then forgets? That is a leadership job, and this hour hands you the playbook.

🟠 Deciding Leaders: C-level · managers · curious minds No code, ever 45 min
0-3 · Welcome 3-20 · Concepts 20-40 · Exercises 40-45 · Q&A
Part 0

Evaluation is a habit or it is nothing

Almost every team can run one evaluation. Very few make it a standing part of how they build. The gap between the two is not technical - it is cultural, and culture is set by what leaders reward, schedule, and fund. An eval that happens once, heroically, before a launch, is a photograph. An eval woven into how the team builds is a heartbeat. This session is about installing the heartbeat: writing the eval first, growing the golden set, automating the gate, and naming who owns it.

Live - presented in session Self-study - read after class ★ Try it now prompt Sources covered at the end
★ What you walk out with today The eval-driven development loop, the golden set as a growing asset, why eval must be automated or it lapses, the roles and cadence that keep it alive, a maturity ladder to place your org on, and five questions to put to your data team this week.
Part 1 · covers eval-driven development

Eval-driven development 9 min live

The single most important shift is when the eval gets written. In a mature team, the eval comes first - you define what success means before you build the thing, exactly as Anthropic advises. Then build, measure, ship only past the gate, monitor, and feed every real failure back into the golden set. It is a loop, and it never fully stops.

1 · Write eval first define success 2 · Build to hit the criteria 3 · Measure against the eval 4 · Ship past gate only if it clears the bar 5 · Monitor on live traffic 6 · Feed failures back into golden set The eval comes first and the loop never closes - every failure grows the next round's test.
🔍 Click to zoom - the eval-driven development loop
LiveEval before build - define success first3 min

The instinct most teams have is to build the thing, get it working, and then - if there is time - test it. Reverse that. Anthropic's guidance to builders is to define your success criteria first, then create empirical evaluations against them, and only then build toward those numbers. Writing the eval first forces the hard conversation early: what does good actually mean here, and how would we know? A team that cannot write its eval does not yet understand what it is building.

  • The eval is the spec. When success is defined as measurable criteria up front, "done" stops being a feeling and becomes a threshold. Everyone builds toward the same bar.
  • It surfaces disagreement early. Writing the eval first is where you discover that two stakeholders meant different things by "good". Far cheaper to find that on day one than after launch.
  • The leader's move: ask to see the eval before you approve the build, not after. "What is the test this has to pass, and did we write it before we started?"
LiveThe golden set as a growing asset3 min

The golden set - your collection of test questions with known-good answers - is the most valuable thing an eval culture builds, and it should never stop growing. Anthropic advises prioritizing the volume of automated evaluation cases over a handful of hand-graded ones. The best fuel for that volume is free and arrives every day: real production failures.

  • Every real failure is a gift. When the system gets something wrong in production, that case, with its correct answer, gets added to the golden set. The system can never quietly regress on that failure again - the eval now guards it forever.
  • The set compounds. A golden set that grows with real failures gets more representative of reality every month, while a set frozen at launch drifts further from it. This is the antidote to the stale-set risk from session a5.
  • Volume beats artistry. A large set of automated cases catches more than a small set of lovingly hand-graded ones. Breadth is the goal.
Real world

The failure log that became the moat. A team (anonymized) made one rule: every customer-reported wrong answer became a new eval case by end of week, with the verified correct answer attached. Within a year their golden set had grown tenfold, drawn entirely from real misses. Regressions on old bugs dropped to near zero, because the eval now remembered every mistake the team had ever made.

Self-studyAutomate it, and give it an owner2 min read

Two things quietly kill an eval culture: manual effort and diffuse ownership. If running the eval is a person's occasional side task, it will lapse the first busy week - and busy weeks are exactly when a risky change ships. The fix is to automate it so the eval runs on every change without anyone remembering to trigger it, and to give the whole discipline a named owner with a schedule. Not "the team" - a person, with a cadence: the golden set is refreshed monthly, drift is reviewed weekly, the gate runs on every release. Ownership plus automation is what turns good intentions into a habit that survives turnover, deadlines, and enthusiasm running out. This is the least glamorous slide in the course and the one that decides whether the other five sessions matter.

Part 2 · covers the roadmap and what is next

The roadmap and where this goes 8 min live

Eval culture is not all-or-nothing - it is a ladder teams climb. Most start at vibes, then add an offline gate, then guard every change with a regression eval, then watch production, and finally reach a culture where eval leads. Knowing your rung tells you the next one. And the ground is shifting: agents are raising the stakes on everything you have learned.

1 · Vibes it seemed fine in the demo 2 · Offline gate a measured bar before launch 3 · Regression in CI every change re-tested automatically 4 · Online monitoring watching live traffic for drift 5 · Eval-driven culture eval leads; the loop never stops You do not need rung 5 tomorrow. You need to know your rung and name the next one.
🔍 Click to zoom - the eval maturity ladder
LiveThe maturity ladder and the roles it needs3 min

Each rung is a real, fundable step - and each needs people, not just tools.

  • Vibes to offline gate. The first climb is from "it seemed fine" to a measured threshold before launch. This is where most of the value arrives, and it needs someone to own the golden set.
  • Offline gate to regression in CI. The eval now runs automatically on every change, so nobody has to remember. This needs an engineer to wire it into the release process once.
  • Regression to online monitoring. You start watching live traffic for the drift you met in a5. This needs someone accountable for reviewing the monitor on a cadence.
  • Online to eval-driven culture. The top rung is where evals are written first and failures feed the set by default. This needs leadership - it is a habit, and habits are set by what leaders reward.
Name the rung, name the next step The most useful thing a leader can do is say honestly where the org sits today and fund exactly one rung up. Trying to leap from vibes to a full culture in one quarter usually collapses back to vibes.
LiveAgents raise the stakes3 min

Everything in this course was about systems that answer a question. The frontier now is agents - systems that take actions, call tools, and pursue multi-step goals with real autonomy. More autonomy means more to evaluate, and the surface grows fast.

  • New things to measure. Beyond answer quality, agents introduce tool-call accuracy (did it use the right tool correctly), agent goal accuracy (did it actually achieve what the user wanted), and topic adherence (did it stay in bounds across many steps). These become new columns on the scorecard.
  • Failures compound across steps. A small error early in a ten-step task can cascade. Evaluating only the final answer misses where things went wrong - you increasingly have to evaluate the path, not just the destination.
  • The leader takeaway: the discipline you learned here does not shrink as systems get more capable - it grows. This is a preview, not a lesson; the depth belongs to a later course.
Self-studyA warm close - where you go next2 min read

You started this course with a simple, uncomfortable truth: a model always answers, fluently, whether it is right or wrong, and only evaluation tells the two apart. Six sessions later you can demand a scorecard, read it without a data-science degree, tell offline from online, spot the four ways an eval lies, and install the culture that keeps it honest. That is a leader's fluency in evaluation - not building the tests, but knowing what to require and how to read the answer. If you want to see where the answers come from, the systems being evaluated here are often retrieval-based, and the RAG course is the natural companion to this one. And when you are ready to see how these systems get built and chained into the agents we just previewed, the LangChain course picks up the thread. Thank you for spending these six hours making measurement a habit. It is the least glamorous discipline in AI and the one that decides whether any of it can be trusted.

Real world

From heroics to habit. A team (anonymized) that spent a year at "vibes" adopted just two rules from this course: write the eval before the build, and turn every production failure into a permanent test case. They did not buy new tools. Eighteen months on, shipping an AI change felt as routine as shipping any other software - because the gate was always there, and it was always current.

Exercise 1 of 2

Place your org on the ladder ★ 12 min · pen and paper

Be honest about which rung your organization actually stands on today for one real AI system - then name the single next rung and what it would take to reach it.

Read the five rungs - vibes, offline gate, regression in CI, online monitoring, eval-driven culture - and mark where this system genuinely sits. Judge by what happens by default, not by what happens heroically once.

Write the evidence for your placement. "We have a gate but it only runs when someone remembers" is rung 2 reaching for 3, not rung 3.

Name the next rung and the one concrete thing that would get you there - wiring the eval into the release process, or assigning someone to review the drift monitor weekly.

Estimate the cost of that one step in people and time. It is almost always smaller than leaders fear - and the return is every future launch getting safer.

★ Your maturity-ladder partner (paste into any chat AI)Help me place an AI team on an eval maturity ladder with five rungs: (1) vibes, (2) offline gate before launch, (3) regression eval in CI on every change, (4) online monitoring of live traffic, (5) eval-driven culture where evals are written first and failures feed the test set. I will describe how my team works today. Ask me clarifying questions, tell me which rung I am genuinely on, then name the single next rung and the one concrete change that would get me there. My team works like this: [describe it]
Exercise 2 of 2

Assign an owner and a cadence ★ 8 min · make it real

An eval culture dies without a named owner and a schedule. For one real system, assign both - specifically enough that it would actually happen.

Name a single person as the eval owner for this system. Not a team, not a rotation with no lead - one accountable name.

Set the cadence for each recurring task: how often the golden set is refreshed with new failures, how often the drift monitor is reviewed, and when the gate runs (ideally on every release, automatically).

Write the one rule that keeps the golden set growing - for example, "every production failure becomes an eval case with its correct answer by end of week".

Decide how the owner reports up: a one-line monthly note to you on the five scorecard numbers and any drift. Make the habit visible to leadership so it stays funded.

What good looks like "Priya owns eval for the support assistant. The gate runs on every release; she refreshes the golden set monthly from the week's production failures and reviews the drift monitor every Monday. She sends me a one-line scorecard note each month." That sentence is the difference between a culture and a wish.
Homework

Carry it forward ◐ 30 min total

★ Questions to ask your data team this week
  1. Do we write the eval before we build, or after - and could we define success as measurable criteria up front for our next project?
  2. Does our golden set grow from real production failures, and when did we last add a case from a customer-reported miss?
  3. Does our eval run automatically on every change, or only when someone remembers to trigger it?
  4. Who is the single named owner of evaluation for each system, and what is their cadence for refreshing the set and reviewing drift?
  5. Which rung of the maturity ladder are we honestly on, and what is the one next rung we should fund this quarter?
Source material

Official sources covered

The leader track teaches from published engineering guidance and course curricula - no vendor marketing. This page covers:

Anthropic · Define success criteria & develop test casesPart 1 · define success first, then build empirical evals; prioritize volume of automated cases
LangSmith · Online evaluationPart 2 · monitoring live traffic as the fourth rung of the ladder
Agent evaluation outlook (tool-call, goal, topic adherence)Part 2 · previewed at a high level - the frontier beyond this course
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · In eval-driven development, when should the eval be written?

Anthropic advises defining success criteria first, then creating empirical evaluations against them, then building. Writing the eval first turns "done" into a threshold and surfaces stakeholder disagreement early.

2 · The best fuel for a golden set that keeps growing is...

Every real failure is a free, perfectly representative test case. Feeding failures back in makes the set more like reality every month and guards against ever repeating that mistake - the antidote to a stale set.

3 · As systems become agents that take actions and pursue multi-step goals, evaluation...

More autonomy means more to evaluate. Agents add tool-call accuracy, agent goal accuracy, and topic adherence, and errors can cascade across steps - so you increasingly evaluate the whole path, not just the destination.

Leader session 6 cheat sheet · pin this

Eval firstDefine success as measurable criteria before you build. The eval is the spec; "done" becomes a threshold.
The loop never closesWrite eval, build, measure, ship past the gate, monitor, feed failures back. Repeat forever.
Golden set = assetIt grows from real production failures. A set frozen at launch drifts from reality; a growing one compounds.
Volume beats artistryMany automated cases catch more than a few hand-graded ones. Breadth is the goal.
Automate or it lapsesManual eval dies the first busy week. Wire it into every release so nobody has to remember.
Name an ownerOne accountable person plus a cadence - set refreshed monthly, drift reviewed weekly, gate on every release.
The ladderVibes → offline gate → regression in CI → online monitoring → eval-driven culture. Know your rung; fund the next.
Agents raise the stakesMore autonomy, more to measure: tool-call accuracy, goal accuracy, topic adherence. Evaluate the path.