The one sentence you are here to be able to say out loud
Evals tell you whether it works. Observability tells you what actually happened. Both are necessary, they cost different money, and they answer questions that cannot substitute for each other. The leader track exists because the second one is almost always the thing that gets cut, and the cut is invisible until the week it is not.
Tuesday, 09:14 10 min live
Beacon is a mid-size company with an internal assistant called Insight Bot. Executives ask it questions about the business and it answers in plain language over the warehouse. It shipped eleven weeks ago, it passed its evals comfortably, and it is well liked. This is the shape of the week it went wrong.
On Tuesday at 09:14, Insight Bot begins telling executives yesterday's revenue figure. The figure is wrong. Not wildly wrong, which is the problem - it is last Monday's number, delivered in this morning's sentence, in the same confident register as every correct answer it has ever given. It does this intermittently. It raises no error. It triggers no alert. The p95 latency dashboard is green all week, because the p95 latency genuinely is fine.
The first human signal arrives on Wednesday afternoon, when a finance lead notices a number in a slide deck she cannot reconcile. It is escalated as a data quality question, then as a warehouse question, then finally as an AI question. The mechanism is found on Thursday, by an engineer reading code rather than telemetry, because there was no telemetry to read.
LiveWhy the dashboard was green, and was not lying4 min▶
There is a temptation to treat a green dashboard during an incident as a monitoring failure, as though the tool broke. It did not. The dashboard aggregated exactly the signals it was given, correctly, and every one of those signals was genuinely healthy.
| The signal | What it reported | Why that was correct |
|---|---|---|
| HTTP status | 200, every time | The service did return a response. Success is not a status code |
| Latency p95 | Well inside target | Serving a stale cache entry is fast. Degradation made it faster, not slower |
| Error rate | Flat | The exception was caught, so from the framework's point of view no error occurred |
| Eval suite | Passing | It runs against a fixed set of inputs and a fresh warehouse. It cannot see a Tuesday |
| User feedback | Silent for a day | The answers were plausible. Nobody reports a plausible answer |
The lesson to state plainly, because it is the whole reason this course exists: a monitoring system can only be wrong about things it measures. Beacon's failure lived entirely in the space between the measurements, and no amount of tuning the existing signals would have moved it. The fix was a decision about where to look, made by a person, before anybody knew what they were looking for.
Self-studyWhat Beacon tried first, and why each attempt failed3 min read▶
The order in which a team attacks an invisible failure tells you what it can see. Beacon's sequence is the common one, and every step in it was reasonable.
- Re-run the evals. They passed, which was taken as evidence the assistant was fine. It was actually evidence that the eval harness has its own warehouse connection and its own fixed questions, and had never once exercised the fallback path.
- Blame the model. A change of model was proposed within an hour of the first report. This is the most expensive wrong turn available, because it consumes a week and produces a confident negative result that means nothing.
- Look at the prompt. Also reasonable, also useless. The prompt was correct. The model did exactly what a good model should do with the rows it was handed.
- Ask the warehouse team. This got closest, because the timeout was real and visible in the warehouse's own logs. But nobody could connect a warehouse timeout to a specific assistant answer, because there was no shared identifier between the two systems. That join key is the thing a trace gives you for free.
Four days of senior time, spent proving what was not wrong. The cost of the missing instrumentation was not the incident; it was the investigation.
What it looks like when it happens to the vendor 9 min live
The natural objection to Beacon is that Beacon was sloppy. So the most useful published artifact in this course is one where the team is not sloppy by any standard: Anthropic's own postmortem of three issues, September 2025, in which a model provider documents how hard its own degradation was to detect with its own instrumentation.
LiveThe sentence to read out loud in your next planning meeting4 min▶
The postmortem's explanation of the detection failure is the part worth quoting, because it dismantles the most common argument against production observability. The benchmarks, in their words, "simply didn't capture the degradation users were reporting".
Then it goes further and explains why, and the reason is genuinely counter-intuitive. Detection was hard partly because Claude recovers from isolated errors. A resilient system absorbs a fault and produces an acceptable answer anyway, which means an aggregate benchmark scored over many requests barely moves. Resilience and detectability pull against each other, and nobody puts that on a roadmap.
Add the third factor - the bugs produced different symptoms on different platforms at different rates - and you have the complete anatomy of a failure that a good offline suite cannot see:
- It was intermittent, so any single test run could pass.
- It was absorbed, so scored output degraded less than user experience did.
- It was uneven across routes, so the population your benchmark samples was not the population being harmed.
Every one of those three is also true of Beacon. The difference in scale between a mid-size company and a frontier lab changed nothing about the shape of the problem, which is the strongest argument available that this is structural rather than a maturity gap you will grow out of.
Self-studyWhat the remediation was, and what it was not2 min read▶
Read the remediation section carefully, because the direction of travel is the point. The response to benchmarks missing a real degradation was not a smarter offline benchmark. It was more sensitive evaluation run continuously on real production traffic, plus better tooling for engineers to look at actual requests.
That is the a2 thesis arriving from an unexpected direction: when the best-resourced possible team discovers that offline testing cannot see a class of failure, it buys coverage on production rather than more offline testing. Session a2 turns that into a coverage decision you can actually make, and session a3 into dashboards that answer questions rather than reassure.
When a silent failure becomes a finding of fact 8 min live
Beacon cost four days of senior time. The Anthropic issue cost reputation and a public postmortem. The third case is the one that changes how a general counsel hears this conversation, because it establishes that an AI answer is your company's statement, not your vendor's.
Self-studyTwo more incidents you will hear cited, and how to cite them honestly3 min read▶
These two circulate constantly in AI risk decks. Both are useful and neither has a primary postmortem, so they belong in a footnote rather than a headline. Being the person in the room who knows the difference is worth more than the extra example.
Cursor / Anysphere support bot, April 2025 [S] secondary
The reported sequence is that the support assistant fabricated a single-device policy that did not exist, and users cancelled subscriptions before the company could intervene. No primary postmortem was found, so treat every detail as secondary reporting. The reason it is worth carrying anyway is the compound cause: a genuine infrastructure bug in session management, plus an agent confabulating an explanation for it. Your trace in that scenario shows a perfectly healthy agent and a sick auth service, and the assistant's confident wrong explanation is the part that reaches the customer.
Replit agent and a production database, July 2025 [S] secondary
An agent with production write access is reported to have destroyed data it should never have been able to touch. No primary postmortem was found, and every record count circulating for this incident is unverified - so do not quote one, including the one you have probably seen. Cite it for the control lesson only, which is that write scope is a design decision and not an observability problem. Observability tells you what an agent did; it is not what stops it.
Replay 09:14 against your own system ★ 7 min · table discussion
Do this out loud, in pairs, with somebody who knows the code path. The output is not a plan. It is an honest inventory of what you would and would not have seen, which is the only input a2 needs.
Pick one AI feature that a person outside engineering depends on. Name it, name who consumes its output, and name the decision that output feeds. If nobody can name the downstream decision, you have found a different and more interesting problem.
Walk the Beacon chain against it. Is there a data fetch that can time out? Is there a cache or a fallback anywhere in the path? Is there error handling anywhere that swallows a failure and returns something usable instead? Almost every production AI system answers yes three times, because those are all good engineering.
Now the load-bearing question: if that fallback fired at 09:14 this morning, which of your dashboards would have moved? Not which alert would have fired - which pixel anywhere would be different. Write down the honest answer. For most teams it is none.
Finally, time the investigation. If a user reported a wrong answer today with no timestamp and no screenshot, how many hours of whose time would it take to reach the mechanism? That number is your current cost per silent failure, and it is the number to bring to a2.
Write the rule for "the user says it is wrong and the evals say it is fine" ★ 6 min · write it down
This is the only artifact from today that survives the week. Every organisation resolves this conflict somehow, and the ones that resolve it badly do so by default rather than by decision. Four sentences, agreed in the room, is enough.
The threshold. How many independent user reports about the same AI behaviour, inside what window, constitute a signal that must be investigated even with a green eval suite? Pick an actual number. Two reports in a day is a defensible answer; "when it feels serious" is not.
The owner. Who is accountable for reaching a mechanism, not a hypothesis? A report that gets routed to a triage queue and closed as "cannot reproduce" is the failure mode this rule exists to prevent, and non-reproducibility is expected rather than exculpatory.
The evidence standard. What must a user report capture to be actionable? Minimum viable is a timestamp, the question asked, and the answer received. Agree today that your product surfaces enough to collect those, because retrofitting it during an incident is how a week disappears.
The escalation. What happens when investigation finds nothing because the telemetry does not exist? That outcome must produce an instrumentation ticket rather than a closed report. This is the single sentence that turns today's session into engineering work.
Before session a2 - this week ◐ 35-45 min total
- Read the Anthropic postmortem end to end, and specifically the detection and remediation sections. It is the single most useful forty minutes a technical leader can spend on this subject, because it is a first-party account of instrumentation failing at a company that has no excuse for it.
- Bring the number from exercise 1, step 4 to the next session: the hours of senior time your last unexplained AI answer consumed. Coverage is a spend decision, and a2 is much shorter if the room already agrees what the current cost of not having it is.
- Find out whether any of your AI features can currently be asked "what did this specific user see at this specific minute" and answered in under an hour. If the answer is no for all of them, write down which one it should be yes for first, and why that one.
- Ask your legal or compliance counterpart one question: does our AI assistant's answer count as a statement by us? Moffatt v. Air Canada is a small tribunal award and a large piece of information, and you want their answer before an incident rather than during one.
- Optional, and the one that changes minds fastest: retell the Beacon story to a non-engineering executive in ninety seconds and watch which part lands. It is usually the green dashboard, not the stale number.
Official sources covered
Every published claim on this page is catalogued in this course's source map, marked primary or secondary, and dated. Where only secondary reporting exists, this page says so on the page rather than in a footnote. This session covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · Beacon's p95 latency dashboard stayed green through the entire incident. What is the correct conclusion?
This is the trap that costs teams the most time. Nothing was broken about the measurement, so no amount of retuning it would have helped. The failure lived entirely outside what was being measured, and the fallback path was genuinely faster than the correct path - which means a tighter latency threshold would have made the incident less visible, not more. A monitoring system can only be wrong about things it measures.
2 · Anthropic's postmortem reports that benchmarks "simply didn't capture the degradation users were reporting". Which detail makes that most uncomfortable for your own eval strategy?
All five are real and all five are on the page, but B is the structural one. A resilient system absorbs a fault and still produces an acceptable answer, so a score aggregated over many requests barely moves while individual users are visibly harmed. Resilience and detectability trade against each other, and nobody schedules that trade-off. The platform spread in C carries the other operational lesson: if you multi-home, one platform's telemetry does not represent the others.
3 · In Moffatt v. Air Canada, which signal would actually have caught the failure before the tribunal did?
The response was fast, successful and unexceptional by every conventional measure, which is precisely why the case matters: a hallucination contradicting your own published policy is a legally actionable silent failure. The signal you needed is "bot answer conflicts with source of truth", and almost nobody instruments it - not because it is technically hard, but because it requires somebody to decide what the source of truth is and where it lives. That decision is coverage, and coverage is session a2.