Seven routed, three still bleeding
Session 2 sent four deterministic workloads to the free tier. Session 3 sent three pattern workloads to classic ML, and the simulator's trusted counter reached 7 out of 10 at a $62 bill. The three workloads still unrouted - theming 500 support tickets, drafting personalized onboarding emails, extracting action items from transcripts - are not waiting politely. They are running on keyword tooling right now, and failing at it every day.
★ What you walk out with tonight. The fit test that identifies genuine GenAI work in one line. Two published cases where the cascade pattern cut GenAI cost 5x to over 100x without giving up the win. And a paper cascade design for all three of Cadence's judgment workloads, ready for session 5's pricing.
LiveThe router is not anti-AI, and tonight proves it3 min▶
Everything so far has been the discipline of saying no: no to sending metrics through a language model, no to paying token prices for arithmetic. Tonight is the other half of the same discipline. A router that can only say no is just a wall, and a wall does not theme a single support ticket.
Here is the tell that the remaining three workloads belong on this branch: every cheap tool has already been tried on them, and every cheap tool has failed. The keyword counter misses paraphrases. The one-size template moves no metric. The regex extracts nothing a human would call an action item. When the bottom rungs genuinely refuse the work, climbing is not extravagance. It is the two-question test from session 1 giving its honest answer: no writable rule exists, so spend.
Self-studyWhy conviction matters as much as caution2 min▶
A half-hearted yes is the most expensive answer of all. Teams that route judgment work to a language model but never trust it end up paying twice: once for the tokens, and once for the human who redoes the work anyway. If the fit test says yes, commit - budget the evals, define the escalation, set the human gate where one is needed, and then let the tool do the job it is genuinely the best on earth at.
The alternative is what Cadence has been living: language work jammed into keyword tooling because keywords feel safe. The output of that safety is a support queue themed by words nobody uses, onboarding emails nobody opens, and action items nobody captured. Safe tooling doing the wrong work is not safe. It is just quiet.
Where language is the actual work
7 min liveUnstructured language, synthesis, generation. No writable rule exists and no table contains the answer. That is the home ground, and all three of Cadence's remaining workloads live squarely on it.
LiveThe fit test: no writable rule, no table with the answer4 min▶
Run each workload back through session 1's first question. Theming 500 tickets: can you write the rule that maps "app eats my notes" to the same theme as "data loss"? You can enumerate keywords for a week and a customer will paraphrase past your list on day eight. Drafting an onboarding email that reflects what this specific customer signed up to do: there is no template slot for that. Extracting "Sam said he'd send the pricing doc by Friday" from a transcript: no regex on earth, because the commitment is in the meaning, not the characters.
A genuine no on the triage question is a real license to spend. Not a shrug, a license. The same test that protected the deterministic tier from token prices now protects the judgment tier from keyword tooling. It is one test, honest in both directions.
The one-line fit test, written down. "The input is unstructured language, the output requires synthesis or generation, and no writable rule connects them." If all three clauses hold, the workload is on GenAI's home ground. If any clause fails, a cheaper rung wants the work back.
LiveWhy paying more here is the correct decision4 min▶
Watch what the simulator does when the GenAI fit lever comes on: the bill rises from $62 to $176, and the trusted counter goes from 7 out of 10 to 10 out of 10. Cost UP $114, and it is the best $114 in the whole course. Session 2's lesson was never "cheap is good". It was "pay for what the work needs and nothing else". These three workloads need judgment over language, and $114 a month is what judgment over language costs at Cadence's volume.
Compare the alternative honestly. The keyword tooling was nearly free and produced ticket themes nobody used, emails nobody opened, action items nobody caught. Free output that no one trusts has a price too - it is just paid in missed renewals and dropped commitments instead of invoices. Where language is the actual work, the tokens are the bargain.
Cadence's support lead spent two quarters extending the keyword list for ticket theming. Every month it caught last month's phrasings. The dashboard tier is magnificent at counting things - session 2 proved it - but a dashboard cannot know that "my notes vanished mid-call" and "sync ate my meeting" are the same complaint. That knowing IS the work, and it is exactly the work language models were built for.
The cascade: cheapest capable tier first, escalate the residue
6 min liveSaying yes to GenAI does not mean saying yes to the biggest model for every call. Even on this branch the routing instinct holds: send each request to the cheapest tier that can handle it, and escalate only what that tier cannot. Two published cases show the size of the prize.
LiveCheckr: frontier to fine-tuned small model, roughly 5x cheaper3 min▶
Checkr runs background checks, and classifying messy records is genuine judgment-over-language work. As reported by Computerworld, they moved that classification off GPT-4 onto a fine-tuned Llama-3-8B: the bill went from roughly $12,000 a month to under $800 - about 5x cheaper - while latency dropped from around 15 seconds to 0.15-0.5 seconds, roughly 30x faster. Accuracy on their hardest, messiest record class went from the 55-72 percent range to 85-90 percent or better.
Two honesty notes travel with this case. First, it is a small-model-replaces-frontier story, not a regex-replaces-LLM story - the work stayed on the GenAI branch, it just moved to a cheaper capable tier. Second, the fine-tuning vendor was involved in telling it, so the numbers here are the Computerworld-reported version, and a healthy reader holds vendor-adjacent case studies a notch looser than audited figures.
LiveGoogle Cloud: proxy models, over 100x on per-row LLM calls3 min▶
Google Cloud published the same pattern at a different scale: analytical SQL pipelines that called an LLM per row were rebuilt so that lightweight proxy models handle the majority of rows, and only the rows the proxy is uncertain about escalate to the full LLM. Reported result: more than 100x reduction in cost and latency for the pipeline as a whole.
Notice what both cases have in common. Neither one abandoned GenAI - the frontier model is still in the loop, doing the residue that genuinely needs it. The saving came from refusing to pay frontier prices for the easy majority. That is the routing instinct from sessions 1 through 3 applied INSIDE the GenAI branch, and it is why the cascade belongs in tonight's session and not in a cost-cutting appendix.
The cascade in one sentence. Route every request to the cheapest tier that can handle it, define what "cannot handle it" looks like before you ship, and let only that residue climb. The published range for the prize is 5x to over 100x.
Where LLMs quietly fail: the wrong work
5 min liveFull conviction on the home ground requires equal clarity about the border. Language models fail in a specific, quiet way when handed deterministic work, and one famous benchmark trap makes them look better on tables than they are.
LiveFluent when wrong: the $11k that read perfectly3 min▶
The simulator's baseline configuration shows the failure in its natural habitat: a monthly revenue total answered by a language model comes back in a well-formed sentence, confidently wrong by $11,000, and nothing about the output signals it. It surfaced only at month-end reconciliation - which is to say, it was caught by the deterministic tier doing its job weeks late. Seven of the baseline's ten answers are silently wrong like this, which is what makes its true cost $2,315 while its API bill is $5.
This is not a capability gap that the next model generation fixes, because the problem is structural: a checkable answer delivered through an unverifiable channel is unverifiable by construction. The words carry no audit trail. On judgment work that is fine - a draft email has a human reader as its verification step. On a revenue number, the fluency is camouflage.
Self-studyThe contamination trap: acing a famous CSV proves memorization4 min▶
One more border marker, from the tabular evidence stack session 3 introduced. Bordt, Nori, Caruana and colleagues showed at COLM 2024 that large language models have memorized popular tabular datasets verbatim - the famous CSVs that circulate in every tutorial were in the training data, row by row.
The consequence for a router: an LLM acing a well-known benchmark table is evidence of memorization, not of tabular capability. If a demo shows a language model predicting beautifully on a dataset you recognize from a textbook, the demo has shown you a recall test. Session 3's evidence stack stands: on typical structured prediction, trained tabular models remain the tool of record, and the honest way to evaluate any model on YOUR table is data the model has never seen.
A vendor pitched Cadence's team with an LLM demo scoring near-perfectly on a churn dataset the data lead recognized within a minute - it was a public teaching dataset with its own Wikipedia-adjacent fame. On Cadence's own churn table, the same setup performed no better than the classic model from session 3, at many times the cost per prediction. The demo was not dishonest. It was contaminated, which from a budget's point of view is the same thing.
Design the cascade, on paper
★ 18 min · everyone buildsThree judgment workloads, one cascade design each. No code tonight - the design decisions are the hard part, and they fit on a page. Session 5 prices exactly what you write down now.
Write the fit test line for each workload. One sentence per workload: is the input unstructured language or judgment? Does no writable rule exist? For ticket theming, email drafting, and action-item extraction the honest answer is yes and yes - but write the sentence anyway, because the sentence is what you defend in the budget meeting.
Pick the cheapest starting tier for each. Template, small model, or frontier? Action-item extraction is well-scoped per transcript - start small. Ticket theming needs consistent taxonomy over messy paraphrase - small model with a clear theme list, escalate the weird ones. Personalized onboarding emails carry Cadence's voice to a brand-new customer - that draft may earn a frontier tier, and writing WHY is the exercise.
Define the escalation trigger before you ship anything. What does "this tier cannot handle it" look like, concretely? Low confidence on theme assignment, a transcript over a length threshold, an email for an unusual plan type. If you cannot describe the trigger, you have not designed a cascade - you have just picked a model.
Set the human gate where the blast radius demands one. Onboarding emails leave the building with Cadence's name on them: a human approves every send. Ticket themes feed an internal dashboard: spot-check weekly, no gate per item. Action items go to account owners who will notice a bad one: light review. The gate is sized to the cost of a miss, not to a general anxiety.
Sanity-check the whole design against the lever. Your three cascades together are what the simulator prices at $176 a month against $62 without them - $114 for three workloads moving from failing to trusted. Write the one-line justification under your design. If it does not convince you, it will not convince finance.
Bring the designs to session 5. Tonight you chose tiers, triggers, and gates. Session 5 puts real per-token prices on every choice you just made - and adds the layers of cost that never appear on the API invoice, which is where most GenAI budgets quietly go wrong.
Before session 5
30 min- Finish the three cascade designs - fit test line, starting tier, escalation trigger, human gate for each of Cadence's judgment workloads, one page total.
- Find one workload in your own shop that passes the fit test - unstructured language in, synthesis or generation out, no writable rule - and is currently limping on keyword or template tooling. Write down what the limping costs, even roughly.
- Find one LLM call in your shop that a cascade would demote. A frontier model doing something a small model or a template could handle for the easy majority. Estimate the share of requests that actually need the top tier.
- Read the Computerworld Checkr piece. Ten minutes. Notice both the numbers and who is telling the story - reading a vendor-adjacent case with the caveat attached is a skill worth practicing on a friendly example.
- Optional: take one famous public dataset an LLM demo has impressed you on, and check its fame. If you recognize it, so did the training data.
Three questions
Clear all three and session 4 gets stamped in your passport.
1 · Turning on the GenAI fit lever raises the simulator's bill from $62 to $176. Why is this correct routing rather than a routing failure?
The router optimizes trust per dollar, not dollars alone. The three judgment workloads were failing on keyword tooling - nearly free and nearly useless. $114 a month moves them from failing to trusted, taking the config to 10/10. Paying more where language is the actual work is the pro-GenAI half of the same discipline that keeps metrics off the token meter.
2 · Checkr moved background-check classification from GPT-4 to a fine-tuned Llama-3-8B. What happened, per the Computerworld report?
All three improved at once: about 5x cheaper, about 30x faster, and more accurate on the messiest class. The caveats matter as much as the numbers: this is a small-model-replaces-frontier case, not regex-replaces-LLM - the work stayed on the GenAI branch - and the fine-tuning vendor was involved in telling the story, so quote the Computerworld-reported version and hold it accordingly.
3 · An LLM scores near-perfectly on a famous public tabular dataset. Per Bordt et al. (COLM 2024), what does this demonstrate?
Bordt, Nori, Caruana and colleagues showed popular tabular datasets sit verbatim in LLM training data, so pre-cutoff benchmark wins are contamination. The honest evaluation for your own table is data the model has never seen - and session 3's evidence stack on typical tabular work is unmoved by contaminated demos.
What this session teaches, and where it came from
Every fact on this page, and its verification tier, is recorded in the course's source map.
Session 4 cheat sheet · pin this
Tonight the router said yes and designed the cascade. Session 5 prices everything: real per-token math on real September 2026 rates, and the layers of true cost that never appear on the API invoice - evals, monitoring, verification labor, and the $330 cleanup behind every silently wrong number. The honest number is bigger than the bill, and it is the number that survives an audit.