When classic ML wins

Run session 1's test on "which workspaces will cancel next month". Can the logic be written down? Try it: tenure under 90 days AND fewer than 3 notes a week AND... every rule you write leaks. Churn has a hundred quiet precursors and no human can enumerate them. Q1 is a genuine no - this is the license to spend that the deterministic pile never issues.

But look at what the work is made of: rows, columns, timestamps, counts. Structured history. No paragraph needs understanding, nothing needs generating. The rule you cannot write is sitting inside data you already have, waiting to be learned. That combination - no writable rule, but history contains it - is the signature of the middle rung, and it fits all three homeless workloads exactly.

• Churn prediction - which paying workspaces cancel soon. The precursors live in usage history nobody can fully articulate.
• Ticket-volume forecast - how many support tickets arrive next week, so staffing is a decision instead of a scramble. Seasonality plus release spikes, learnable from two years of counts.
• Anomalous API usage - which integration partners' traffic looks wrong. "Wrong" resists definition; "unlike its own history" does not.

Notice what they share: the answer is a number or a flag about the future, graded eventually by reality itself. That grading is what tonight's whole argument stands on.

Train the churn model only on data up to six months ago, then ask it to predict the churn that followed - churn that has already happened, so every prediction is gradeable against reality. Before the model touches a single live decision, you hold a number: how well it did on a future it could not have memorized.

That is a real exam with a real answer key, and it is the middle rung's answer to "verified once". Not correct by construction - correct by demonstration, on your own data, before deployment. The LLM route cannot cheaply imitate this: to earn the same certificate it needs a labeled eval set built and graded by hand, per task, forever. The trained model's exam comes free with the method, because history already contains the answers.

In the simulator, switching the three pattern workloads onto classic ML lands the whole configuration at $62 a month true cost, with 7 of 10 answers trusted and zero silently wrong - the cheapest trustworthy configuration in the course. One modest inference box serves all three models; inference on tabular models costs cents per thousand calls, so the workloads share the hardware the way queries share a warehouse.

Compare the neighbors: the free-tier-only config sits at $224 with 4/10 trusted, because the pattern pile is still being mangled elsewhere. The AI-everything anti-lever pays $2,429 from baseline to trust 3 of 10. The middle rung is where the cost curve and the trust curve cross - which is exactly why it is the most skipped rung in 2026: it is nobody's demo and everybody's savings.

Grinsztajn and colleagues found tree-based models still state of the art - on typical, medium-sized tabular data, around 10K samples, across 45 datasets. The scope is in the paper's own title, and it is the finding, not fine print. This is not "trees always win"; it is "on the shape of data most businesses actually have, the boring tool is still the benchmark to beat".

McElfresh and colleagues widened the lens to 176 datasets and found the wins between GBDTs and neural networks vary by dataset profile - but a lightly-tuned GBDT is the strongest default, and tuning matters more than algorithm choice. The routing lesson hides in that last clause: the team arguing about which tool would be better spent tuning the one they have.

Together the stack licenses one modest claim: for Cadence's three pattern workloads - all typical, all medium-sized, all tabular - a gradient-boosted model is the evidence-backed default. It does not license sneering at neural nets on large or exotic data. Scoped claims are the ones that survive meetings.

Someone will show you a frontier LLM scoring brilliantly on a famous public dataset and conclude the middle rung is obsolete. Bordt and colleagues tested exactly this and found that LLMs had memorized popular tabular datasets verbatim - they could reproduce rows character for character. An impressive score on a dataset the model has seen during training is contamination, not capability: the exam was in the training set.

Your churn table was not. That is the quiet, decisive advantage of the backtest from Part 1: your own held-out history is an exam the model provably has not seen. The demo dazzles on Titanic; the routing decision is about your data, where the only honest certificate is the one you grade yourself.

The world the model learned keeps moving: a pricing change reshapes churn, a product launch reshapes ticket volume, a new integration reshapes "normal" API traffic. That is drift, and it is not a malfunction - it is the weather on rung 2. The defense is the loop in the figure: a maintained eval set, monitoring that compares live performance against the backtest number, and retraining when the gap opens.

The simulator prices that discipline at $40 per probabilistic workload per month - the upkeep line its S5 lever adds, which is why the honest full-cost configuration reads $416 rather than $176. Budget the loop on day one. A model priced without its eval set is a quote for half a bridge.

Google's Rules of ML puts a gate in front of this whole session: "If you think that machine learning will give you a 100% boost, then a heuristic will get you 50% of the way there." Before Cadence trains anything, the honest first move is the dumb baseline: for churn, flag workspaces with no notes created in 14 days; for the forecast, next week equals the same week last year plus trend.

The heuristic is not the destination - it is the yardstick. It costs an afternoon, it is deterministic, and it sets the number the model must beat by enough to pay its $40-a-month loop. Sometimes the heuristic wins outright and a workload quietly moves DOWN a rung, which is not a failure of ambition. It is the routing skill working.