Where the five rules came from
Each session contributed exactly one lever to tonight's simulator. Session 1: the triage question. Session 2: free tier first. Session 3: the classic ML check. Session 4: the GenAI fit test. Session 5: count the full cost. Tonight is the first time all five run at once, and the first time you see what each is worth in dollars and trust.
★ What you walk out with tonight. A routing playbook your organization can actually hold: the one-page tree, the numbers that justify it rung by rung, and the review cadence that keeps a routing decision honest after the workload underneath it changes.
LiveWhat the five rules are, in one breath each3 min▶
- S1 · The triage question. Can the logic be written down, and is the definition aligned? Two yeses mean deterministic work. This rule alone converts silent errors into caught errors, because you now know which answers are checkable.
- S2 · Free tier first. Verified plus aligned goes to the dashboard. Not because it is humble, but because verified-once-then-cents is an economic property no model can match.
- S3 · The classic ML check. Structured prediction from history goes to a trained, backtested model. The middle rung is the cheapest trustworthy home for pattern work.
- S4 · The GenAI fit test. Unstructured language and judgment goes to the LLM, and the money spent there is money well spent. This course is a router, not a refusal.
- S5 · Count the full cost. Every probabilistic route carries its eval set and monitoring on its back. The honest number is bigger, and it is the only number worth budgeting.
The one-page decision tree
8 min liveThis is the whole course on one page. It routes a workload in under a minute, and every branch carries the session that justifies it, so when someone challenges a routing decision, the answer is a page number rather than an opinion.
LiveThe cascade: what "start small" means on the GenAI branch5 min▶
Even after a workload correctly lands on the GenAI branch, the routing is not finished, because the branch has rungs of its own: rules where they hold, a small model where it suffices, the frontier model only where it earns its price. Two real, published escalation stories:
- Checkr routed background-check classification off GPT-4 onto a fine-tuned Llama-3-8B: roughly $12,000 a month down to under $800, about 30x faster, and accuracy on the hardest record class up from the 55 to 72 percent range to 85 to 90 percent plus. Reported by Computerworld, with the caveat that the fine-tuning vendor was involved in the telling.
- Google Cloud published the same shape for analytical SQL: replace the majority of per-row LLM calls with lightweight proxy models and escalate only the uncertain rows - more than 100x cheaper and faster on the workloads they show.
The cascade rule. The frontier model is the specialist you consult, not the front desk. Route every item to the cheapest tier that can handle it, and escalate the residue. Both published cases put the savings between 5x and 100x with accuracy flat or better.
Self-studyPortfolio triage: routing at the roadmap level4 min▶
The same tree that routes a workload routes a quarter's initiative list. Two questions per initiative: which tier does the core work actually live on, and what is the full-cost number at that tier? Then three honest buckets:
| Bucket | What it looks like | The leader's move |
|---|---|---|
| Fund | Judgment work with a real fit test passed, cascade designed, evals budgeted | Spend confidently - this is the 5 percent that reaches the P&L |
| Reroute | Deterministic or pattern work wearing an AI costume | Deliver it a rung down, bank the difference, keep the credibility |
| Stop | Contested definitions, no data, or beyond the state of the art | Name the real blocker - RAND's causes 1, 2 and 5 - and fix that instead |
The reroute bucket is the politically hard one, because it means telling someone their AI project is a dashboard. The playbook makes that conversation survivable: the routing is a documented decision with numbers, not a taste judgment.
Route the backlog, live
★ 22 min · everyone buildsThe full simulator. Ten workloads, five routing rules, one anti-lever. The dashboard answers below are genuinely computed from the embedded dataset when you route them there, and every price is a real published rate. Have your session 1 paper triage next to you - tonight it gets scored.
Read the baseline first: $2,315 true cost, zero answers you can trust, seven silently wrong. This is the 2026 default nobody chose on purpose: metrics flowing to the model because the integration was already open, language work stuck on keyword tooling because it was already there. The API bill is $5 - the damage was never in the token bill.
Switch on S1, the triage question, and watch the true cost fall to $225 before a single workload moves. Nothing got rerouted. You simply know which seven answers are checkable, so you check them: silently wrong becomes caught wrong, and $2,310 of cleanup becomes $220 of verification labor. Knowing what is verifiable is worth 10x before any tool changes.
Switch on S2 and S3. Free tier first routes the four deterministic workloads to the dashboard - their answers now compute live from the real data, and trust reaches 4/10. The classic ML check moves the three pattern workloads to trained models: $62 true cost, 7/10 trusted, and that $62 is the cheapest trustworthy configuration in the whole simulator. Sit on this rung a moment - this is BI and classic ML carrying seven-tenths of the company.
Switch on S4 and watch the cost RISE to $176 - and understand why that is correct. The three judgment workloads move off keyword tooling onto the LLM, and trust reaches 10/10. This is the course's other half: where language and judgment are the actual work, the tokens are the bargain. Routing is not refusal.
Switch on S5 and let the number get honest: $416. Nothing moved. The evals, monitoring and drift-watching that every probabilistic route needs just entered the price. The number went up and the budget got MORE defensible, because this is the figure that survives an audit.
Now the anti-lever: AI-everything. On top of the full playbook it pays $739 to trust 3 answers out of ten. From the raw baseline it pays $2,429. In every configuration it pays more to trust less - measure it yourself, then quote your own measurement in the next budget meeting.
The sentence the whole course compresses into. The route from $2,315 and 0/10 to $416 and 10/10 never once required refusing AI. It required three workloads' worth of well-placed spending and seven workloads' worth of well-placed restraint.
Routing decisions expire
5 min liveA routing decision is correct at a point in time. Volumes grow, definitions drift, model prices fall by half in a year. The playbook needs a review trigger on every route, or it fossilizes into the next generation's "we have always done it this way".
LiveThe 5 percent, revisited from the finish line3 min▶
Session 1 opened with the MIT finding: 95 percent of GenAI pilots never reached the P&L, and the winners picked one pain point and executed. Read it again from here and it is a description of the playbook: the winners routed. They put judgment work, and only judgment work, on the expensive tier, and executed the boring tiers with the respect boring tiers deserve.
The standout leader move is not enthusiasm and it is not skepticism. It is a defensible route for every workload, and the willingness to be boring in public about the ones that only needed a WHERE clause.
Self-studyRunning the playbook as a team ritual3 min▶
- New workload intake: the requester answers the two triage questions in the ticket template. Thirty seconds of friction that prevents the default-to-LLM drift permanently.
- Quarterly re-route: top ten workloads by spend get the tree re-run. Expect one to move per quarter; celebrate the moves down as loudly as the moves up.
- The routing changelog: one line per decision - date, workload, tier, reason, review trigger. This document is what makes the discipline outlive its author, and it is the artifact this course was always building.
After the course
45 min- Score your session 1 paper triage against the simulator. Toggle the full playbook on, compare your ten routes to the live ones, and write down where you differed and why.
- Draft your one-page tree. Copy the Part 1 diagram's structure, but with your organization's own workload examples on the branches.
- Route your team's top five workloads by spend, for real. Tier, reason, full-cost estimate, review trigger. That document is the playbook, version one.
- Find your reroute-down candidate. The politically hard one. Bring the simulator's numbers to the conversation instead of an opinion.
- Set the quarterly re-route reminder now, before the discipline evaporates. The playbook that is not on a calendar does not exist.
Three questions
Clear all three for the final passport stamp.
1 · Switching on the triage question alone dropped true cost from $2,315 to $225 before any workload changed tier. What produced the saving?
The API bill barely moved - it was $5 all along. The $2,310 was the cost of confidently wrong numbers reaching decisions unverified, at six analyst-hours per incident to unwind. Triage does not fix the answers; it tells you which ones can be checked, and checkable-and-checked is a different risk class from fluent-and-unverified.
2 · Switching on the GenAI fit test RAISED the monthly cost from $62 to $176, and the course calls that correct. Why?
The $62 configuration was the cheapest trustworthy one, but it left ticket theming, email drafting and transcript extraction on tools that structurally cannot do them. Routing is not refusal: the discipline that saves $1,900 on metrics is the same discipline that spends $114 on judgment work without apology.
3 · A quarter after adopting the playbook, the LLM vendor halves its prices. What does the playbook say happens next?
Price is one of the four standing review triggers, and it cuts both ways. But note what a price change can never do: it cannot make an unverifiable answer verifiable. The deterministic tier keeps its workloads at any token price, because its advantage was never only the money - it was correct-by-construction.
What this session teaches, and where it came from
Every fact on this page, and its verification tier, is recorded in the course's source map.
Session 6 cheat sheet · pin this
Session 1 asked the question. Sessions 2, 3 and 4 gave each tier its due: the dashboard's verified-once economics, classic ML's cheap trained judgment, GenAI's genuine wins. Session 5 made the accounting honest. Tonight put it all on one page. The mosquito is dead, the nuclear stockpile is intact, and the budget line for the ox-cleaver was approved without a fight - because the routing was written down.