Three projects, and the one that is not a model

The top box is her test, and it is written in her words, not yours. At Harbourline it reads: fewer complaint calls about empty shelves, which she counts herself, and she said plainly that she does not care what the accuracy number is. The second half is the guard rail: an over-order costs expiry write-off, and head office measures her on write-off. Say that half back to her as your reading, because she named the cost and not a target; a project that cut the calls by stuffing every shelf would pass the first half and fail her review.

Test the top box by turning the tree over. If what sits at the top is "use AI" or "predict stock-outs", the tree is upside down: a solution is pretending to be an outcome, and every branch under it will be a feature of the model, a horizon, a threshold, a dashboard, rather than a pain of hers. The board wants the word AI on a slide; she wants fewer calls. Those are different projects, and only one of them is hers to sponsor.

Keeping her words is not politeness. "Fewer complaint calls" is a number she already collects, by hand, without you, so a project that points at it is one she can check without trusting your metric. The moment you rewrite it as "improve stock-out recall", it is your project again, judged by a number she cannot see.

The second row is a transcript, not a brainstorm. Each box is a pain she voiced behind doors 3, 4 and 5: nobody records a stock-out anywhere, the only record is the complaint call; product codes differ between the two countries, and two outlets renamed products themselves last year; three versions of the stock spreadsheet are in circulation; the spreadsheet is overwritten every Friday, so last week's stock level does not exist; the warning today is the pharmacist looking at the shelf and sending a WhatsApp; the area managers read things on their phones, in the van. Six boxes, six things she said.

The discipline is to write nothing in this row she did not say. "No labelled training data" is your diagnosis; "nobody records a stock-out" is her pain. The first pulls a modelling project in behind it and a feature store behind that. Torres's rule, as reported, is that opportunities are framed from the customer's side, and the reason is practical: a pain in her words is one she will recognise on the page and fight for, and a diagnosis in yours is one she will nod at and forget.

Which opportunity is strongest is the first real judgement on the page, and for a scientist it is a judgement about what the outcome can be checked against. The missing record earns the mustard edge, because every other box, and every model you might build, needs to know when a shelf was empty, and today that fact lives only as a phone call to her.

Solutions are the only layer you invent, and the rule is at least two per opportunity you take seriously. Under "nobody records a stock-out": a daily log, one sheet per outlet, filled in by the person who sees the empty shelf; keeping each Friday count instead of overwriting it, so there is a stock history from next week on; and a model on the three years of sales history. Three shapes, from cheapest to dearest. Writing the model down here is the honest move, not the dangerous one: on the page, next to the log, anyone can see that it answers a different question.

Torres's fourth layer, assumption tests, is the one scientists skip and should not: under each solution, the thing that must be true for it to work, and the cheapest way to find out. Under the daily log the assumption is that someone will write a line on the day it happens; the test is the pharmacist in Johor, who has offered to do exactly that if someone gives her a sheet. Under the model on sales, the assumption is that sales predict empty shelves, and the vendor already tested it for you. The Product Thinking course teaches the tree as a discipline, with the interviewing cadence around it; here it is borrowed for one job, ordering what she already told you.

Pruning has one rule: one candidate under each of the three strongest opportunities, never three solutions to one. Three ways to predict a stock-out is a menu, and a menu makes her choose a build. Three projects under three different pains is a portfolio, and a portfolio lets her choose a problem, which is the choice she is qualified to make.

Each candidate is written the same way, in four lines: what it is, in words from her tree; the first step, something that can start this week; who does it, a named person on her side and on yours; and what it needs. "A daily stock-out log in four outlets, the Johor pharmacist first" is a candidate: she offered to log every day if someone gives her a sheet, and a sheet costs nothing. "Improve data readiness for ML" is a wish. The test is whether Priya in finance, who can give two hours a week, could read the four lines and know what she is doing on Monday.

Notice what the prune did to the request. "Predict which outlets will run out" is not one of the three. It is the fourth line, and it is honest about why: a model that predicts stock-outs needs to have seen stock-outs, and the only record of one today is a phone call to Adeline. Candidate 1 is the first project because it creates the label. Candidate 3 is on the list because it is the thing any model will later have to beat.

Sean Ellis and Morgan Brown's ICE score, reported here from Hacking Growth, rates each candidate on impact, confidence and ease, each from 1 to 10. How the three are combined is contested: some teams multiply, some average, and the summaries that quote the book describe its practice differently. So the honest move is not to pick the right rule. It is to write down which one you used, above the grid, before you fill in a digit.

The two rules can disagree, and on this grid they do, for second place. The product master scores 6, 8 and 3; the alert scores 7, 4 and 6. Averaged, both come to 5.7 and the grid cannot separate them. Multiplied, the product master falls to 144 and the alert rises to 168, because multiplying punishes the weakest leg and the product master's ease of 3, nobody owns it and two outlets renamed products themselves, is weaker than the alert's confidence of 4. The winner holds either way: the log multiplies to 648 and averages 8.7. Neither rule is wrong. An unwritten rule is.

The digits are yours, and that is the point rather than a weakness. A grid with the rule above it is a page she can argue with: "why is confidence only 4 on the alert?" is a conversation about the shop, and the answer, that the Friday count it reads is typed by hand and exists in three versions, usually ends with a better digit and a better project.

Google's machine learning problem-framing course, read at source, makes two points that belong on this grid. Before building a model, check that the current non-ML solution is as good as it can be; and define how success will be measured before you build anything. At Harbourline the current solution is a pharmacist looking at a shelf and sending a WhatsApp, which catches most of them. Any model has to beat a person looking at a shelf, and before that it has to beat a rule anyone can read: if the days of stock left for a chronic product are fewer than the supplier lead time, five days in Singapore and nine in Malaysia, flag it for Thursday's van. The success measure is already written: her own count of complaint calls.

That is why the alert is candidate 3 and not a throwaway. It is the non-ML solution made explicit, so that eight weeks from now a model is scored against something rather than against nothing. A model that cannot beat "stock left under lead time" on the stock-outs the log recorded is not worth the POS export. And the export has its own assumption to test: Kenneth, the only IT person, holds the POS admin login, which is a question to ask by name before the alert depends on it. The Decision Intelligence course owns the wider discipline of choosing under uncertainty; the grid here is a single page of it.

Parking the model is not the same as dropping it. Write it on the page with the condition that would let you score it: eight weeks of logged stock-outs in the four outlets, and the alert's record over the same weeks. Then do the arithmetic out loud, because it is the base rate, and the base rate is the first number any prediction is judged against. About 30 stock-outs a month across 38 outlets is under one per outlet per month. Four average outlets over eight weeks would log roughly six. That figure is arithmetic on the constructed facts, not a finding, and it teaches the right thing: six events is enough to check an alert against and far too few to train a model on.

So the eight-week review asks a question rather than delivering a model: is the log telling us something the alert misses? If the four outlets are the ones that keep her up at night, the count will be higher than average, and that is a reason to choose them, not a trick. If the alert already catches what the log records, the best model on the page is the rule, and saying so is the most useful thing a scientist can put on a board slide.

A pilot is the winning candidate with four things pinned to it: a scope, partners, a test and a date. All four came out of doors 5 and 6, so none of them is yours to invent. Scope: four outlets, because she said she would run a pilot in four first and never all 38. Partners: the Johor pharmacist, who logs, and Priya, who gives two hours a week. Test: her own count of complaint calls about empty shelves. Date: one slide for the board in six weeks.

Her test beats any score you could offer, and the reason is the whole of door 1. An accuracy number measures the model; her count measures the outcome. A model that says "no stock-out" every week is right almost every week, because most shelves are fine most weeks, and it leaves every complaint call exactly where it was. Her count cannot be gamed that way, and she already keeps it. Google's second point, define success before you build, is met the moment you write "her count of calls" on the pilot line.

The constraint shapes the first step more than the ambition does. No budget this quarter means the log is a sheet the pharmacist fills in on the day, and the alert, when it arrives in week 4, is a formula on the Friday count and three years of daily sales. When the SGD 15,000 opens in Q1, the pilot has produced the one thing a budget conversation needs: a record, from four of her own outlets, of when the shelf was empty and whether a plain rule saw it coming.

The pharmacist in Johor is not a data source on the plan. She is the reason the label exists. The person who logs a stock-out becomes the person who owns the record of it, and she volunteered, which is Rob Fitzpatrick's test from door 6 already met: the other side gave up something before you asked. Put her outlet first because she offered, not because it is convenient, and design the sheet with her in the first week: one line per empty shelf, the product, the day, and whether a patient left without it. Anything longer than that will not be filled in on a busy day.

Priya is the reason the work survives you. Her two hours a week turn the four sheets into the count set beside Adeline's calls, and she is the person who will notice that the two countries use different product codes the first time she tries to add them up. That is candidate 2 arriving on its own schedule, with a named owner already sitting next to the problem. Torres's weekly contact, reported here, is the cadence: the same two hours every week, in the calendar.

Check the pilot against what you actually asked. On session 3's bench, the scientist-tuned order spends its 45 minutes on eleven open questions and stops at the one about where a prediction would be read. The supplier lead-time question and the "when do you next have to show something upward?" question are further down its list and never get asked. So a scientist who ran that order has her test, her partners, her four outlets and her Q1 budget in the notes, but not the five and nine days the alert depends on, and not the six-week board date the pilot is aimed at.

That is not a flaw in the order; 45 minutes is 45 minutes. It is the reason every line of the pilot is marked with where it came from. A line that came from her is written as a fact; a line you are assuming is written as a question, and session 6 sends those questions in the covering note with the brief.