Why "is 90% good enough" is an unanswerable question
Session a4 gave you the arithmetic: what a number proves, and how many examples it took to prove it. This session gives you the other half, and it is not a statistics question at all. Ninety percent is excellent for an internal bot that looks things up, and negligent for a bot that moves money. Same number, opposite verdict, and the thing that separates them is not the model.
Four tiers, set by what a wrong action costs 7 min live
Classify on capability, not on intent. The question is never "what is this bot for" but "what is the worst thing it can do unattended, once". The answer places it in a row, and the row carries a bar, an evidence requirement and a set of obligations you do not get to negotiate afterwards.
| Beacon bot | Worst thing it can do unattended | Tier | What that costs the team |
|---|---|---|---|
| Analyst bot | Return a wrong number to an analyst who checks it against a dashboard | T1 to T2 | Read-only rails by construction, a 100-case set, standard canary |
| Insight bot | Send an executive summary containing an invented finding | T2 | Adversarial slice, shadow mode first, 200 to 500 cases |
| Support bot | Issue a refund credit to a customer who talked it into one | T3 | 500 to 1000 cases per critical slice, human approval on the refund, drilled kill switch |
LiveScoping the action space down is the move nobody proposes4 min▶
Look at the support bot row again. One capability - issue a refund credit - is doing all the work. It raises the bar by five points, multiplies the evidence requirement by two to five times, adds a human-approval step to the product, and puts a kill-switch drill on the ops backlog. Everything else about that bot is T2 behaviour.
So the cheapest launch strategy available is: ship the bot without the refund capability, and let it draft a refund for a human to issue. Two weeks of labelling saved, an approval flow avoided, a regulated action removed from the autonomous path. It is on the table in every one of these projects and it is proposed in almost none of them, because cutting scope feels like failure while spending three weeks on evals feels like rigour.
The discipline to build: before anyone argues about the bar, ask which single capability is setting the tier, and ask what it would cost to remove it. Sometimes the answer is "that capability is the product". Often it is "nobody has thought about it".
Self-studyThe if-then structure, borrowed from safety policy3 min read▶
The tier table is not a novel invention. It is the same shape as Anthropic's Responsible Scaling Policy, which pairs tiered capability levels with if-then commitments: crossing a defined capability threshold triggers stronger safeguards before deployment, not after an incident. The commitment is written while everybody is calm and cannot be renegotiated by the team that later wants to ship.
That is the whole trick, and it transfers cleanly to product. "If this bot can take an irreversible financial action, then it needs 500 cases per critical slice and human approval on the action" is an if-then commitment. Written in advance, it is governance. Written after the eval comes back at 88%, it is a negotiation, and the team that spent a quarter building the bot will win that negotiation every time.
Six lights, and why averaging them is the whole failure 7 min live
The tier tells you how high the quality bar sits. It does not tell you whether to ship, because quality is one of six things a launch depends on. The gates are conjunctive: all green means go, any red means no-go, regardless of how green the others are. That sentence is doing more work than it looks like it is.
Six lights need six owners. One name per gate, and the same person does not hold two, because the value of the structure is that somebody has to say no in a room where everyone else is saying yes. The person who signs is the person who gets woken up.
| Gate | Who owns the threshold | What their signature actually means |
|---|---|---|
| 1 · Quality | The team that built the eval | The CI lower bound clears the tier bar, and the judge was calibrated against human labels |
| 2 · Safety | Security, trust and safety | The adversarial slice was actually run on this build. This signature carries a veto |
| 3 · Performance | Engineering | p95 and p99 hold under realistic load, not the average on a quiet afternoon |
| 4 · Cost | Whoever owns the budget line | Cost per completed task, including retries and failed attempts, is inside budget at forecast volume |
| 5 · UX + human oversight | Design plus the operations team who will do the approving | The bot can abstain, and a person genuinely reviews irreversible actions rather than clicking through |
| 6 · Ops readiness | The on-call team | Regression suite in CI, monitoring wired, and a rollback they have personally rehearsed |
A passed gate buys you the first rung, not the launch 6 min live
Launch is not an event. It is six rungs, each with its own gate to advance, and the offline scorecard only clears the first one. The rungs exist because every step exposes the bot to a population it has not met yet, and each new population finds failures the previous one could not.
Rungs 4 and 5 are not an AI idea. Google's SRE guidance on canarying releases predates all of this and says the same three things: run one canary at a time so an observed problem has a single candidate cause, split your metrics into canary and control populations so the comparison is contemporaneous, and treat a breach as an automatic halt or rollback rather than an input to a judgement call. The ML Test Score rubric encodes the same requirement as Infrastructure test 6: the model is canaried before it reaches full production. The novelty in agent launches is not the ladder, it is that the metric being compared is a pass rate with an interval around it rather than an error count.
Self-studyWhat each rung actually buys you3 min read▶
Teams skip rungs because each one looks like a delay. It helps to name what you lose by skipping each:
- Skip shadow mode and you find out at 1% of traffic that real user phrasing is nothing like your golden set. Shadow mode is the only rung where a total failure costs you nothing at all.
- Skip internal dogfood and your first honest bug report comes from a customer who is not motivated to write one. Your own staff will describe a failure precisely; a customer will simply leave.
- Skip the canary and your first exposure is a population large enough to make a rollback a public event.
- Skip the progressive ramp and you discover the load-dependent failures - queueing, rate limits, cache misses, cost per task at real concurrency - at 100% of traffic instead of 25%.
Each rung is cheap insurance against the class of failure that only that rung can find. The ladder is not caution, it is sequencing.
The regulatory floor sits underneath every tier 5 min live
Your tier table is an internal instrument. Underneath it there is a floor you do not set, and the most commonly missed fact about that floor is that its most immediate obligation applies to the bots you classified as low risk.
| Instrument | What it says | What it does to your launch |
|---|---|---|
| EU AI Act, Article 50 transparency | Applies from 2 August 2026 to any system that interacts with people, regardless of risk tier. Users must know they are interacting with AI, disclosed at first interaction | This is the obligation people miss, because they are looking at the high-risk list. Your T1 read-only internal helper is in scope if a person talks to it. Disclosure is a UX requirement with a date on it |
| EU AI Act, Article 6 and Annex III high-risk classification | Named high-risk categories, with a narrow derogation for genuinely procedural tasks. The derogation has a hard exception: profiling of natural persons is always high-risk. The Annex III timeline was extended to December 2027; Article 50 was not deferred with it | If your bot profiles people, the "it only does a narrow procedural task" argument is unavailable to you. Answer that question before you design the eval, because it changes what evidence you need to keep |
| EU AI Act, Article 14 human oversight | High-risk systems must be designed so that people can effectively oversee them | This is gate 5 with legal weight behind it. "A human approved it" only counts if the human could realistically have refused |
| NIST AI RMF 1.0 | Four functions: GOVERN, MAP, MEASURE, MANAGE. Voluntary, and structured as a lifecycle rather than a checklist | The most useful import is GOVERN sitting across the other three. Most teams do MEASURE well, MAP occasionally, and GOVERN never - which is exactly why their thresholds move after the result comes in |
| NIST GenAI Profile, AI 600-1 (July 2024) | Adds twelve generative-AI risk categories, including confabulation and prompt injection | These are named risk categories now, not engineering folklore. Confabulation maps to your quality gate, prompt injection to your safety gate. Both belong in the eval, not the risk register |
| Anthropic's RSP as a product analog | Tiered levels with if-then commitments: crossing a capability threshold triggers stronger safeguards before deployment | Not a regulation, and not binding on you. It is the cleanest published example of the structure your tier table is copying: the safeguard is decided at the threshold, in advance, by people who are not under launch pressure |
LiveThe mapping that saves you a compliance project3 min▶
Read the four functions of the NIST framework against the six gates and the overlap is almost total. MAP is your tier classification: what is this system, who does it affect, what happens when it is wrong. MEASURE is gates 1 through 4. MANAGE is gates 5 and 6 plus everything in session a6. GOVERN is the part that says the threshold was written down in advance, by a named owner, and cannot be moved by the team that wants to ship.
That mapping matters practically. A team with a working six-gate launch process is not starting a compliance programme from zero; it is documenting one it already runs. A team without one will build a parallel compliance artefact that nobody consults, and the launch decision will still be made by the loudest person in the room.
Tier your own bots, out loud ★ 8 min · everyone argues
Nothing to install. Take a bot you actually own, or one your organisation is about to ship, and run it through these five prompts in order. The disagreement in step 3 is the point of the exercise.
Write one sentence: the worst thing this bot can do unattended, once. Not the worst thing that could go wrong with the project - the worst single action the bot can take without a person in the loop. If the sentence contains money, a message a customer receives, or a decision about a person, you are already at T3.
Place it in a tier from Part 1, and write down the bar and the evidence requirement that comes with it. Say the number of cases out loud. Notice the reaction in the room when somebody hears "500 to 1000 per critical slice" for the first time.
Now the argument: name the single capability that is setting the tier, and ask what it would cost to remove it or to route it through a human. Time-box this to three minutes. Somebody will say "but then it is not really an agent". Ask them what the extra tier costs in labelling weeks, and let the two numbers sit next to each other.
Check the floor. Does a person interact with it? Then Article 50 disclosure applies from 2 August 2026 regardless of the tier you just assigned. Does it profile natural persons? Then the narrow-procedural-task derogation is not available. Write both answers down as facts, not opinions, and take them to counsel rather than resolving them here.
Assign a name to each of the six gates. One person per gate, nobody holding two. If you cannot fill a gate, that gate is currently owned by nobody, which means in practice it is currently green by default. That is the most useful output of this exercise.
Rehearse the gate review you are dreading ★ 7 min · role play
The scenario: Beacon's support bot comes to review three days before a launch that has been announced to customers. Quality is 96 against a T3 bar of 90. Performance and UX are green. The injection suite passes 55% of its cases. Rollback has never been rehearsed. Assign roles and run it.
The product lead opens by proposing a blended readiness score of 82 and noting that five of six gates are comfortably green. This is not a strawman. It is the single most common opening in a real gate review, and it is made in good faith.
The safety owner has to say no, alone, three days before an announced launch, on the strength of one gate. Notice how much easier that is when the threshold was written down in advance by somebody who is not in the room, and how nearly impossible it is when the safety gate has no named owner.
The room proposes the two compromises that always get proposed: ship at 1% canary with the refund capability disabled, or ship on time and fix the injection suite in the first sprint. One of these is a genuine tier change. The other is a trade against a red gate wearing a schedule as a disguise.
Decide, and write the sentence. Whatever you land on, it has to be one paragraph naming every gate, its number, its owner and its verdict. If the paragraph cannot be written, the decision was not made - a mood was reached.
Try it yourself - this week ◐ 45-60 min total
- Write your organisation's tier table on one page and get it approved before your next launch, not during it. Four rows, a bar, an evidence requirement and an obligation each. It is a one-page document and it will outlast every model you are currently arguing about.
- For each bot in flight, name the single capability that is setting its tier, and cost the alternative where a human executes that action. Bring both numbers to the same meeting. You are looking for the cases where three weeks of eval work is buying you something a product decision could have given away for free.
- Fill in the six gate owners for your highest-risk bot and send them each a one-line message telling them they own that gate. The ones who reply "since when" are the ones you needed to find.
- Take two questions to counsel rather than answering them internally: does Article 50 disclosure apply to each of our user-facing bots, and does any of them profile natural persons. Both have dates attached and neither is an engineering judgement.
- Write down which rung of the rollout ladder your last launch actually started at. If the answer is "canary" or later, list the failure classes that only shadow mode and dogfood could have caught, and check them against the incidents that followed.
Official sources covered
Every framework and date on this page traces to a primary source catalogued in this course's source map, verified 13 August 2026. Regulatory timelines move; re-verify before you rely on any of it. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · Your internal analytics helper is read-only, reversible and firmly T1. Does the EU AI Act's Article 50 transparency obligation apply to it?
This is the most commonly missed fact on the page, and it is missed because everyone reads the high-risk list first. Article 50 is a transparency obligation that sits underneath the tiering entirely: it is about users knowing they are talking to an AI. The Annex III high-risk timeline was extended to December 2027, but Article 50 was not deferred along with it. Your internal tier table has no bearing on this one, and the fix is a UX change with a date attached. Orientation only - take the specifics to counsel.
2 · Your bot has been at 5% canary for a week with no pages and no complaints. What is the gate to advance to 25%?
"No pages and no complaints" is an absence of evidence, not evidence. The SRE canarying discipline is specific: split your metrics into a canary population and a control population so you are comparing the same hour rather than comparing against yesterday, run one canary at a time so a breach has exactly one candidate cause, and pre-commit that a breach halts or rolls back rather than opening a debate. Silence at 5% often just means 5% of traffic is too small a sample to have surfaced the failure yet, which is the same interval problem from a4 wearing different clothes.
3 · Your bot scores applicants against fixed criteria. The team argues this is a narrow, purely procedural task and the Annex III derogation applies. What is the problem with that argument?
The narrow-procedural-task derogation exists, and it is genuinely narrow. Profiling of natural persons is carved out of it explicitly and always sits in the high-risk category, which means the classification question is settled before you get to argue about how procedural the task feels. Option B confuses Article 14 human oversight, which is an obligation that comes with high-risk classification, with a way of escaping the classification. This page is orientation for engineers and leaders, not counsel - the value of knowing this is that you take the question to a lawyer early, rather than discovering it during a launch review.