Attribution, twice, before the bench
The last two sessions before the bench both answer the same kind of question: given an error, which component produced it. Tonight the components are a scoring function and a search. Next session they are the two stages of a pipeline. Both tests replace a debate with a measurement, and both are cheap enough to run on twenty examples in an afternoon.
★ What you walk out with tonight. A three-line function that turns any wrong output of a search-based system into one of two verdicts. A tally over twenty examples that says which verdict dominates. And the sentence you take to the sprint planning: the next quarter belongs to the search team or to the modelling team, and here is the count that decided it.
LiveWhy guessing here is expensive3 min▶
The two failures cost different things to fix. A search failure is fixed with beam width, better candidate generation, more inference-time compute: engineering work, often measurable in days, and it does not touch the model. A scoring failure is fixed with more data, a better objective, a bigger or differently shaped model: research work, measurable in months, and it does not touch the search.
Teams routinely fund one while suffering from the other. The symptom that gets reported is the same either way, "the system said elk and it was a deer", and the loudest voice in the room decides. The test below removes the room from the decision.
Lanternfish spent six weeks widening the candidate list for its species pipeline because the field kept reporting confident wrong answers. The correct species had been in the list every time, sitting below the wrong one. Six weeks of search work on a scoring problem, and the extra candidates made the ranking noisier.
Self-studyThe vocabulary for tonight3 min▶
- Scoring function - the thing that takes an input and a candidate output and returns a number. Higher is meant to mean better.
- Search - the procedure that walks the space of possible outputs looking for the highest-scoring one. Beam search, greedy decoding, sampling with reranking, a planner, a shortlist of candidates from a cheaper model.
- Approximate maximization - what the search actually does. It rarely finds the true maximum, and the gap between what it found and what exists is the whole of the first failure mode.
- The reference output - the answer a human would give. You need one to run the test, and it has to be scoreable by the same function.
- The verdict - one of two words, "search" or "scoring", per example. The tally over many examples is the deliverable.
"The Optimization Verification test"
9 min liveChapter 44 of Machine Learning Yearning is titled "The Optimization Verification test", and it exists because one class of system hides its own failures. Once a system searches rather than classifies, the output you see is the joint product of two components, and no amount of staring at outputs separates them. One comparison does.
LiveThe shape of system this applies to3 min▶
The test applies wherever the system produces an output by searching for the best-scoring candidate rather than by naming a class directly. If your model ends in a softmax over twelve species and you take the argmax, there is no search and no test to run: the argmax is exact. The moment the output space is too large to enumerate, a search appears, and with it a second place to be wrong.
- Speech recognition. A scoring function rates transcripts; a decoder searches the space of word sequences. The space is unbounded, so the decoder is approximate.
- Machine translation. Same shape. Beam search over sentences that could have been produced, scored by the model.
- Any planner or router. A cost function over routes, a search that cannot enumerate all routes.
- Retrieve-then-rerank. A cheap retriever proposes candidates, an expensive scorer ranks them. The retriever is the search.
- Lanternfish's pipeline. The species stage emits a ranked candidate list per frame with a score each, produced by a cheap shortlist step and rated by the classifier head.
LiveThe two failure modes, and why they look identical3 min▶
Write the system as two components and the failures fall out cleanly.
- Failure one: the scoring function is wrong. It gave the correct output a lower score than a wrong one. The search did its job perfectly and returned exactly the highest-scoring candidate, which happened to be wrong. Widening the search here makes things worse, because a wider search finds more high-scoring wrong answers.
- Failure two: the search is wrong. The scoring function would have preferred the correct output, and the search never brought it forward to be scored, or brought it forward and pruned it before the end. The model is fine. Every hour spent on training data here is wasted.
From the outside both produce the same artefact: a confident wrong answer. That is why the test compares scores rather than outputs. The score of the correct answer is the piece of evidence that the output alone never carries.
LiveThe test, in its general form3 min▶
Chapter 45 gives the general form, and it is three steps.
- Take an example the system got wrong.
- Take the correct output, the one a human would give, and score it with the system's own scoring function.
- Compare that against the score the system assigned to its own output.
If the correct output scores higher, the scoring function preferred the right answer and the search failed to surface it. The search is at fault. If the correct output scores lower, the search returned the true best-scoring candidate and the scoring function is what ranked the right answer down. The scoring function is at fault.
Self-studyWhat the test does not require3 min▶
This is the property that makes it worth running on a Monday morning rather than scheduling.
- No understanding of the model. The scoring function is used as a black box that takes an input and a candidate and returns a number.
- No understanding of the search. You never inspect the beam, the pruning rule or the ordering. The search is judged entirely by what it failed to bring you.
- No retraining. Both scores come from the model you already have deployed.
- No new labelled data, beyond the correct outputs, which for a classification-shaped output space you already have in the dev set labels.
- No statistical machinery. Twenty examples give a tally, and a tally of fourteen to four is not a result that needs a confidence interval to act on.
Why this saves a quarter
8 min liveThe verdict is not a diagnosis for its own sake. It is a budget allocation. The two branches fund different work, cost different amounts, and in most organisations are staffed by different people who are each convinced the problem is on the other side.
| Verdict | What it means | What the sprint funds | Typical owner | Typical horizon |
|---|---|---|---|---|
| Search at fault | The correct output scores higher and was never surfaced | Beam width, candidate generation, reranking depth, inference-time compute | Systems and inference engineers | Days to weeks |
| Scoring at fault | The correct output scores lower than the wrong one | Training data, the objective, model capacity, the loss | Modelling and data people | Weeks to a quarter |
| Near-tie | The two scores sit within noise of each other | An objective that separates the two cases at all | Modelling, but a different job | Its own finding |
LiveThe same test outside supervised learning3 min▶
Chapter 46 runs the identical argument in a reinforcement learning setting, where the two components are a reward function and the learning algorithm that maximises it. An agent behaves badly; the question is whether the reward function scored the good behaviour below the bad behaviour, or whether the reward function was correct and the algorithm simply failed to find the good behaviour.
The procedure does not change. Take a trajectory the agent produced, take a trajectory a human would call correct, evaluate the reward function on both. Correct trajectory scores higher means the optimisation is at fault and you tune the algorithm. Correct trajectory scores lower means the reward function is at fault and no amount of better optimisation will help, because it will optimise harder toward the wrong thing.
The pattern to carry. Any system built as "a thing that scores" plus "a thing that maximises the score" can be split by this test, whatever the two things are called in your stack: reward and policy, cost and planner, relevance model and retriever, likelihood and decoder.
LiveLanternfish's version of the test3 min▶
The species stage does not name one animal. For each frame it emits a ranked list of candidate species with a score each, and the pipeline reports the top of that list. That makes the shortlist step the search and the classifier head the scoring function, and the test becomes two concrete questions per frame.
- Was the correct species in the candidate list at all? If not, score it directly with the classifier head anyway. Absence from the list is a property of the shortlist, not of the score.
- How did it score against the species the pipeline chose? Higher means the shortlist and the ranking failed to bring a species the model would have preferred. Lower means the model preferred the wrong animal, and no shortlist change rescues that.
On the twenty frames run tonight, deer scored above the chosen elk on fourteen of them. The species the field wanted was already preferred by the model and was being ranked out by a shortlist tuned for latency. That is a two-week change to candidate generation, not the eight-week specialist model the team had costed.
Self-studyThe honest limits of the test4 min▶
- You need a correct output that can be scored. For a species label that is free, it is in the dev set. For an open-ended output, a translation or a summary, somebody has to write the correct answer first, and the test is only as good as that reference. Budget the human time before promising the result.
- A tie is a finding, not a failure of the test. If the correct and chosen outputs score within noise of each other, the scoring function is not ranking the right answer down; it is failing to distinguish them at all. That funds a different piece of modelling work, an objective that separates the two cases, and it will not be found by widening a beam.
- The reference must be reachable in principle. If the correct output is something the system's output format cannot express, the test reports "scoring" and it is really a representation problem. Check the format before reading the verdict.
- The verdict is per example, not per system. A mixed tally is normal and useful. Fourteen to four says where the next sprint goes; it does not say the scoring function is perfect.
- Twenty examples is a tally, not an estimate. It is enough to choose a sprint. It is not enough to publish a percentage, and it should not be written down as one.
Run the test on twenty frames
★ 20 min · everyone buildsBring a system of yours that searches, and the last hundred outputs it got wrong. If you have nothing to hand, use the Lanternfish candidate tables: one row per frame per candidate species, with a rank and a score from the classifier head.
Pull twenty misclassified frames that came back with a ranked candidate list. Filter to the dev split so the examples are ones you are allowed to look at, and take them in a hash order rather than by date, so you do not accidentally sample one camera or one week.
Obtain the correct output and locate it in the list. The dev set label is the correct species. Record its rank if it appears and record its absence if it does not. Absence is common and is not yet a verdict.
Score both candidates with the same scoring function. The chosen species already has a score. The correct species has one too if it was in the list; if it was not, call the scoring function on it directly. Never compare a score from one function against a score from another, and never compare against a probability that has been renormalised over a different candidate set.
Tabulate the verdict per example. Correct scores higher means search. Correct scores lower means scoring. Within a small epsilon means near-tie, and write the epsilon down: on Lanternfish's head, 0.01 on a 0 to 1 score.
Count the split. Fourteen search, four scoring, two near-ties. Look at the two near-ties by hand before you file them; they are frequently the most interesting frames in the set.
Write which team the next sprint belongs to, in one sentence, with the count in it. "Fourteen of twenty dev errors are search failures, so the next two weeks are candidate generation." That sentence survives a planning meeting; "I think it is the shortlist" does not.
Run it before the planning meeting, not after. The test costs an afternoon and twenty labelled examples you already own. The sprint it redirects costs six weeks. Lanternfish ran it in the seventh week of a six-week search project.
Before session 9
30 min- Decide whether any system you own actually searches. Write down its scoring function and its search procedure by name. If you cannot name both, there is no test to run and that is a useful answer.
- Run the test on ten wrong outputs. Ten is enough to see whether the split is lopsided, and lopsided is the only reading you need.
- Check what your team funded last quarter against the verdict you just computed. Note whether the money went to the branch the tally points at.
- For any near-ties, write one sentence on what the objective would have to reward for the two outputs to separate. That sentence is the start of the next objective.
- Sketch your production system as a sequence of stages, one box each, and mark which stage produces the output the user complains about. Session 9 starts from that sketch.
Three questions
A frame is labelled deer and the pipeline said elk. The classifier head scores deer at 0.83 and elk at 0.71. Which component is at fault?
Correct output scores higher means the scoring function already preferred the right answer, so the failure is that the search did not bring it forward. Fund beam width and candidate generation, not the model.
A team plans to widen the beam because the system keeps returning confident wrong answers. The test says the correct output scores lower than the chosen one on eighteen of twenty examples. What happens if they widen the beam anyway?
When the scoring function is at fault, a better search optimises harder toward the wrong answer. The work is the model, the data and the objective.
On two frames the correct species and the chosen species score within 0.004 of each other. What should be recorded?
A tie says the function cannot tell the two outputs apart. That is a modelling job, but a different one from ranking the right answer down, and widening the search will not touch it.