A model that aces the past can still fail the future
Last session we said machine learning learns its rules from examples. Here is the catch nobody puts on the pitch slide: those examples are all from the past, and the model's actual job is on new cases it has never seen. A model that looks perfect on the data it studied can be useless the moment a fresh visitor arrives. The entire risk in machine learning - the thing your judgement has to protect - is whether the model generalises beyond what it memorised. Lumen Skincare's team learned this the hard way, and this session is how you avoid their mistake without touching the maths.
Learning from examples, and grading honestly 8 min live
A model learns by studying a pile of past examples - Lumen's case, thousands of website visits, each labelled with whether the visitor bought. But if we then grade the model on those very same examples, we learn nothing: it has already seen the answers. So before training even starts, we split the pile in two. The model studies one part and is graded on the other - a part it never saw. That held-out slice is the only honest measure of how good the model really is.
LiveWhat training actually does3 min▶
Training is the once-off process of finding the pattern that fits the examples. The model tries a rule, checks how wrong it is on the training pile, nudges the rule to be a little less wrong, and repeats - thousands of times - until it fits those examples well. That is all "the model is learning" means: it is tuning itself to match the answers it can see.
- Input - the training pile: past visits, each with its known outcome (bought, or did not).
- The dial-turning - the model adjusts its internal rule to reduce error on those examples.
- Output - a fixed model, the learned rule, ready to score new visitors.
Notice the trap hiding here: the model is rewarded for fitting the training pile. Nothing in training forces it to work on anything else. That is exactly why we hold a test set back.
LiveWhy hold out a test set3 min▶
The only honest measure of a model is its performance on data it has never seen - because that is the only situation the model will ever face in the real world. So before training, we lock away a slice of the examples (Lumen uses about 20%) and never let the model study it. After training, we grade the model on that locked slice. If it does well there, it has genuinely found a pattern. If it does well on the training pile but poorly on the held-out set, it just memorised.
Early on, Lumen's team graded their conversion model on the exact same visits it had trained on, saw 98%, and told the CMO it was ready. It went live and flopped - it had memorised the past visitors, not learned who buys. Once they re-graded on a held-out slice the real story appeared: a modest, honest signal (a ROC-AUC around 0.71), useful but nowhere near 98%. Same model, two scores - only the held-out one was true.
The two ways a model fails 7 min live
Once you grade on held-out data, every failure sorts into one of two piles. Underfitting: the model is too simple, so it misses the pattern and does badly on both the training set and the test set. Overfitting: the model is so complex it memorises the noise in the training set - it does brilliantly on training but badly on the test set. The gap between the two scores is the tell. Picture a seesaw: too simple on one end, too complex on the other, and the sweet spot balanced in the middle.
The clearest way to feel this is to watch it happen. Below is a live model fit on Lumen's data: slide the decision tree's depth and watch the training error fall while the test error makes a U - dropping to a sweet spot, then climbing back up as the model starts memorising.
LiveOverfitting, defined properly2 min▶
Overfitting is not "high error" - it is a very specific signature: low training error and high test error. The model has fit the training pile so tightly that it has learned the noise, the random quirks of those particular examples, as if they were the pattern. On new data those quirks do not repeat, so it fails.
| Tree depth | Train R² | Test R² | Diagnosis |
|---|---|---|---|
| 1 | 0.31 | 0.22 | Underfit - too simple |
| 3 | 0.46 | 0.36 | Sweet spot (peak) |
| 5 | 0.51 | 0.29 | Starting to overfit |
| 8 | 0.62 | 0.15 | Overfit |
| 15 | 0.84 | negative | Badly overfit |
Read the last row slowly. At depth 15 the model scores a superb 0.84 on training and a negative test R² - meaning on new data it is literally worse than just guessing the average order value. It memorised the training rows perfectly and learned nothing useful. That is overfitting in one line, and no amount of training-set glory hides it once you look at the held-out score.
LiveBias-variance in one sentence2 min▶
You will hear engineers say "bias-variance trade-off." Here is the whole thing in plain English: a simple model has high bias (it underfits), a complex model has high variance (it overfits), and you cannot drive both to zero - you trade one for the other. The goal is not zero of either; it is the depth where the two together are smallest - the bottom of that U in the test curve.
- High bias - the model is too rigid to bend to the real pattern. More data will not save it; it needs to be less simple.
- High variance - the model bends to every wiggle, including noise. It needs to be less complex, or fed more data.
This week ◐ 25 min total
- Ask the question. Find one model your org relies on and ask whoever owns it: "how was this tested - on data the model actually held out and never trained on?" Note the answer, and how confidently it comes.
- Tell an overfit story. Describe, in a few lines, a time something (a model, a strategy, a person) performed brilliantly on the practice cases and fell apart on real ones. That is overfitting; naming it in the wild makes it stick.
- Predict the failure. Pick a model you know of. Guess whether it is more likely too simple (underfit) or too complex (overfit), and why. You will not always be right - the point is to build the instinct.
Three questions before you go 🎯 ◐ 90 seconds
1 · What is the signature of an overfit model?
Overfitting means the model fit the training pile so tightly it learned the noise: low training error, high test error. The gap between the two is the tell. High error on both is underfitting.
2 · Why do we hold back a test set the model never sees?
The model's real job is on cases it never saw. Grading on the training data is like giving students the exam answers to memorise; only a held-out set tells you the truth.
3 · A model is too simple and does badly on both training and test data. In bias-variance terms, what is that?
A simple model that misses the pattern on both sets has high bias and is underfitting. High variance is the opposite failure - a complex model that overfits, doing well on training but poorly on test.
What this session covers
This session teaches the framing behind the "overfitting and generalisation" week of the major intro courses - the intuition a decision-maker needs. The hands-on Python (the decision-tree depth loop that produces the table above) lives in Builder Session 4; the sources below go deeper.