learn-asking-right-questions-as-da-with-phoebe / Session 5 of 6
Learn Asking the Right Questions as a Data Analyst with Phoebe · Session 5 of 6

From pains to three joint projects

You walked out of the meeting with a page of her answers: a write-off target, a Monday phone call that changes nothing, three versions of a stock spreadsheet, product codes that differ by country, a colleague with two hours a week. That page is not a project. Left alone it becomes the thing she first asked for, a dashboard, because that is the only noun anyone wrote down. Tonight turns the answers into a tree with her outcome at the top, prunes the tree to three candidate projects with a first step each, scores them with a rule written where she can argue with it, and shapes the winner into a pilot small enough to start on Monday with no budget. Doors 1 and 6, outcome and success, do most of the work; session 1 has the corridor.

🟡 Core Analysts, and anyone who gets asked for numbers No code. A pen. 45 min
0-3 · Welcome 3-18 · The tree, the three, the pilot 18-40 · Build-along: your tree, your three, your first step 40-45 · Q&A
Part 0

Where we start

Sessions 1 to 4 were about getting facts out of a person who did not know she had them. Adeline Tan, operations director of Harbourline, an invented chain of 38 pharmacy and clinic outlets, gave you 24 of them in 45 minutes if you walked the doors in order. The danger now is the opposite of the first meeting's. Then, you had too little; now you have a page of pains and no way to choose. Most analysts choose by picking the one they know how to build, which is how the dashboard comes back. The move tonight is to let her outcome choose, and to write the choosing down so she can see it and disagree.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = write it yourself

★ What you walk out with tonight. One page for a request you actually hold: her outcome in her words at the top of a tree, the pains she named as its branches, three candidate joint projects with a first step, a named person and what each needs, a scoring grid with the combination rule written above it, and a five-line pilot with a scope, a partner, a test and a date. Session 6 folds that page into the brief you send her within 24 hours.

Part 1 · covers her answers as a tree

Her answers as a tree

5 min live

Teresa Torres's opportunity solution tree, reported here from her 2021 book, has four layers: a desired outcome at the top, the opportunities underneath it, solutions under the opportunities, and assumption tests under the solutions. The layers matter more than the drawing. An opportunity is a need or a pain the customer voiced; a solution is a thing you could build; the outcome is what would be different if the pain went away. Adeline's meeting gave you all three layers, in her words, in the wrong order. The tree puts them back in order.

HER ANSWERS, PUT BACK IN ORDER outcome on top, in her words · pains she named underneath · things to build come last Adeline OUTCOME, in her words write-off from 2.1 to 1.5 percent of cost of goods, and no chronic-medication stock-outs in the six Malaysian outlets OPPORTUNITIES: THE PAINS SHE NAMED Priya's six hours, every Monday three stock spreadsheets in circulation product codes differ by country orders over WhatsApp, no order table the Monday number changes nothing today managers on phones, in the outlets SOLUTIONS, MORE THAN ONE, UNDER THE STRONGEST one spreadsheet, typed once, with a Friday check stock count through a form on the phone stock module from the POS vendor, from Q1 the upside-down tree: "build a dashboard" on top Every box is something she said. The analyst added only the order, and the order is the argument.
🔍 Click to zoom - the terracotta box came out of door 1, the hatched row is her pains as she said them, and the white boxes are the only layer you are allowed to invent
LiveOutcome first, in her words2 min▶

The top box is the answer to door 1, and it is written in her words, not yours. At Harbourline it reads: expired-stock write-off from 2.1 percent of cost of goods to 1.5, and no chronic-medication stock-outs in the six Malaysian outlets. Two things she is measured on, two things she can see. A dashboard is not on it, and neither is a data warehouse, because she never said either was the point.

Test the top box by turning the tree over. If what sits at the top is "build a dashboard", the tree is upside down: a solution is pretending to be an outcome, and every branch under it will be a feature of the dashboard rather than a pain of hers. That is the tree that produces the vendor pilot nobody opened. An outcome box you can check a solution against reads like a fact about the shop in six months, and it has a number in it if she gave you one.

Keeping her words is not politeness. The sentence "write-off to 1.5 percent" is the one the regional GM and the board already recognise, so a project that points at it is a project she can defend upward without translating. The moment you rewrite it as "improve inventory visibility", it is your project again.

LiveOpportunities are pains she named2 min▶

The second row is a transcript, not a brainstorm. Each box is a pain she voiced behind doors 2, 3 and 4: Priya's six hours every Monday; three versions of the stock spreadsheet in circulation; product codes that differ between the Singapore and Malaysia outlets; supplier orders over WhatsApp with no order table anywhere; the Monday number that changes nothing, because when it looks bad they phone the outlet; area managers on their phones in the outlets, not at a laptop. Six boxes, six things she said.

The discipline is to write nothing in this row she did not say. "No single source of truth" is your diagnosis; "three spreadsheets in circulation" is her pain. The first is a solution wearing a problem's clothes, and it will pull a warehouse in behind it. Torres's rule, as reported, is that opportunities are framed from the customer's side, and the reason is practical: a pain in her words is one she will recognise on the page and fight for, and a diagnosis in yours is one she will nod at and forget.

Which opportunity is strongest is the first real judgement on the page, and it is a judgement about the outcome, not about the data. The stock spreadsheet earns the terracotta edge because both halves of her outcome, write-off and stock-outs, are decided by whether the Friday stock count is right, and today there are three of them.

Self-studySolutions come last, more than one per opportunity2 min▶

Solutions are the only layer you invent, and the rule is at least two per opportunity you take seriously. Under the stock spreadsheet: one spreadsheet, typed once, with a Friday check; a stock count entered through a form on the phone; a stock module from the POS vendor once the Q1 budget opens. Three shapes, from cheapest to dearest. The point of writing three is not to choose the best on the spot. It is to stop the first one you thought of from becoming the plan because it was alone on the page.

Torres's fourth layer, assumption tests, is the one analysts skip and should not: under each solution, the thing that must be true for it to work, and the cheapest way to find out. Under "one spreadsheet, typed once" the assumption is that outlet managers will type into a shared file on Friday if it is the only one; the test is to ask the four pilot outlets to do it for two weeks. The Product Thinking course teaches the tree as a discipline, with the interviewing cadence around it; here it is borrowed for one job, ordering what she already told you.

Real world

The tree with "dashboard" at the top. A constructed case, but the shape is the common one. The year before your meeting, a BI vendor came to Harbourline with a tree whose top box read "a management dashboard". The branches under it were the dashboard's features: sales by outlet, trend lines, a mobile view. The dashboards were beautiful. Nobody opened them after week two, and Adeline has been wary of the word since, which is why the request reached you as "a dashboard" with a shrug. Nothing in that tree pointed at a write-off number or a stock-out, so nothing in the build could move either, and the one fact that predicted the failure, that the area managers phone the outlet and do not change the order, was never on the page.

Part 2 · covers pruning and scoring

Three candidates, written down

5 min live

A tree with six branches and nine leaves is a map, not a plan. The prune is to three candidate joint projects, one under each of the three strongest opportunities, never three solutions to the same one. At Harbourline: one product master for both countries, with an owner named; one stock spreadsheet, typed once, with a Friday check; and the Monday number as a phone-readable outlet-level list with a suggested reorder, for four pilot outlets. Then a score, written where she can see it, with the combination rule above the grid.

THREE CANDIDATES, ONE GRID, THE RULE WRITTEN ABOVE IT RULE, written first: impact x confidence x ease, 1 to 10 each. We multiply. CANDIDATE IMPACT CONFIDENCE EASE I x C x E 1 One product master, both countries, owner named 7 8 6 336 2 One stock spreadsheet, typed once, with a Friday check 8 9 8 576 winner 3 Monday number as a phone list with a suggested reorder, four pilot outlets 9 6 5 270 The digits are the analyst's judgement, written down so she can argue with them. Averaged, the same row wins here; it does not always. Row 3's confidence rises once row 2 gives a count to trust.
🔍 Click to zoom - the rule sits above the grid because a score whose rule is unwritten cannot be argued with, only accepted
LiveFrom tree to three2 min▶

Pruning has one rule: one candidate under each of the three strongest opportunities, never three solutions to one. Three ways to fix the stock spreadsheet is a menu, and a menu makes her choose a build. Three projects under three different pains is a portfolio, and a portfolio lets her choose a problem, which is the choice she is qualified to make.

Each candidate is written the same way, in four lines: what it is, in words from her tree; the first step, something that can start this week; who does it, a named person on her side and on yours; and what it needs, which at Harbourline is nothing until Q1 for two of the three. "One product master for both countries, owner named" is a candidate. "Improve data quality" is a wish. The test is whether Priya could read the four lines and know what she is doing on Monday.

Notice what the prune did to the request. "A dashboard" is not one of the three. The phone list is the nearest thing to it, and it is candidate 3 not candidate 1, because its confidence score is honest about the fact that a suggested reorder built on three disagreeing stock counts is a suggested guess.

LiveScore it, and write the rule down2 min▶

Sean Ellis and Morgan Brown's ICE score, reported here from Hacking Growth, rates each candidate on impact, confidence and ease, each from 1 to 10. How the three are combined is contested: some teams multiply, some average, and the book's own practice is described differently across the summaries that quote it. So the honest move is not to pick the right rule. It is to write down which one you used, above the grid, before you fill in a digit. The two rules can disagree: a candidate scored 9, 9 and 2 averages 6.7 and multiplies to 162; one scored 6, 6 and 6 averages 6.0 and multiplies to 216. Multiplying punishes a weak leg; averaging forgives it. Neither is wrong. An unwritten rule is.

The digits are yours, and that is the point rather than a weakness. A grid with the rule above it is a page she can argue with: "why is confidence 6 on the phone list?" is a conversation about the shop, and it usually ends with a better digit. A ranking with no rule and no digits is a verdict, and a verdict she did not make is one she will not defend to the board.

Self-studyThe simple fix first1 min▶

Google's machine learning problem-framing course, read at source, gives one line that belongs on every prioritisation grid: verify that the current non-ML solution is optimized before you build anything cleverer. The course is about models, and the line is about analytics too. At Harbourline the suggested reorder is the clever thing, and the spreadsheet that feeds it is the current solution, and it is not optimized; it exists in three versions. Fix the spreadsheet first and the reorder gets a confidence score above 6. Build the reorder first and it will be right about a stock count that is wrong.

This is also why the winner on the grid needs no budget. The cheapest candidate that moves her outcome is nearly always a fix to the current way, because the current way was never designed, it accreted. The Decision Intelligence course owns the wider discipline of choosing between options under uncertainty; the grid here is a single page of it.

Part 3 · covers the pilot

Four outlets, not 38

4 min live

Door 6 is where a candidate becomes a pilot, and everything the pilot needs was in her answers. Her test: the area managers stop phoning her on Monday afternoon. Her partner: Priya, two hours a week, and Priya asked for a course. Her scope: four outlets first, never all 38. Her constraint: no budget until Q1, so the first step uses what exists. Her date: one slide for the board in six weeks, not a demo. Five facts from door 6 and door 5, and the pilot writes itself.

SIX WEEKS, FOUR OUTLETS, ONE SLIDE the constructed timeline for the winning candidate · no budget spent before Q1 0 1 2 4 6 WEEK the brief one page, sent within 24 hours code list agreed with Priya, both countries, one owner one spreadsheet typed once, Friday check, four outlets first Monday list on phones, outlet level, one reorder one board slide four outlets, one number she can see the test a quiet phone on Monday afternoon Nothing on the line costs money: the code list and the spreadsheet use what exists, the phone list is a shared sheet. Weeks 3 and 5 are empty on purpose: Priya has two hours a week, and a plan that fills every week cannot be kept.
🔍 Click to zoom - the slide is terracotta because it is the only thing on the line she asked for by name, and the phone above the line is how she will know
LiveThe pilot is the first step, not the plan2 min▶

A pilot is the winning candidate with four things pinned to it: a scope, a partner, a test and a date. All four came out of doors 5 and 6, so none of them is yours to invent. Scope: four outlets, because she said she would sponsor four and never 38 at once. Partner: Priya, two hours a week. Test: the area managers stop phoning her on Monday afternoon. Date: one slide for the board in six weeks.

Four outlets beat 38 for a reason she can feel and you can measure. In four outlets the single spreadsheet is a conversation with four managers; across 38 it is a policy, and policies are the thing the previous pilot was. A quiet phone beats a login count because the login count measures the tool and the quiet phone measures the outcome; a dashboard opened twice and a dashboard opened two hundred times can both leave the stock-outs exactly where they were. Session 1's door 1 line applies: a tool is checked against an outcome, never the other way round.

The constraint shapes the first step more than the ambition does. No budget until Q1 means the product-code list is a shared sheet Priya and you agree in week 1, and the stock spreadsheet is the existing file with two versions retired, not a new system. When the SGD 15,000 opens in Q1, the pilot has produced the one thing a budget conversation needs: a fact from four outlets about what happened to the Friday count.

Self-studyWhat the partner changes2 min▶

Priya is not a resource on the plan. She is the reason the work survives you. She builds the Monday report today, six hours every Monday, so she knows every column and every fudge in it; she asked for a course, so she wants the two hours to be a lesson and not a hand-over; and she is on leave for two weeks in November, which is the first test of whether the single spreadsheet runs without her. A pilot with no named partner on her side has a shelf life of one analyst.

Torres's weekly contact with the people you are building for, reported here, is the cadence for the analyst too: the two hours are the same two hours every week, in the calendar, with the four outlet managers on a call for the first fifteen minutes of it. Rob Fitzpatrick's test from door 6, whether the other side gave up something, is met the moment Priya's Tuesday afternoon is blocked. Everything after that is the two of you finding out what the Friday count is actually like, which is the fact the board slide will be built on.

Real world

The pilot the other 34 asked for. A constructed case, with the shape that recurs. The four Harbourline outlets ran one stock spreadsheet for four weeks, and their Monday list arrived on the area managers' phones with an outlet-level number and one suggested reorder. In week five an area manager from an outlet outside the pilot phoned Adeline on a Monday afternoon, which was the call the test was supposed to make stop, and asked why his outlets were not on the list. The rollout to 38 was never proposed by the analyst. It was requested by the people who had watched four neighbours stop phoning. That is the difference between a pilot and a demo: a demo is shown, a pilot is missed.

Build-along

Your tree, your three, your first step

★ 22 min · everyone writes

One page, three columns. Take the answers you collected for your real request, or Adeline's if you have none yet, and build the page in six steps: the outcome in their words, the pains as they said them, two solutions under each of the three strongest, three candidates with a first step, the grid with its rule, and the pilot. Every line is written tonight; nothing on the page is a decision the business person has not already given you the facts for.

THE PAGE, THREE COLUMNS, LEFT TO RIGHT 1 TREE outcome, their words pain pain pain two solutions under each of the three strongest 2 THREE CANDIDATES what · first step who · needs what · first step who · needs what · first step who · needs I C E rule 3 PILOT scope: four outlets partner: Priya, 2 h a week test: a quiet phone date: one slide, week 6 needs: nothing before Q1 Left to right, as the meeting ran: her outcome picks the pains, the pains pick the three, the rule picks the one.
🔍 Click to zoom - everything terracotta on the page traces back to a door 1 or door 6 answer, which is how you can tell the page is hers

Write the outcome in their words at the top. Not a goal you would set for them. The sentence they said when you asked what would be different in six months, with the number in it if they gave one. If the sentence you have is "a dashboard", you do not have door 1 yet, and the tree cannot start; write the question you will ask to get it instead.

★ Step 1 · the outcome, her wordsOUTCOME (Adeline, door 1): "Write-off down from 2.1 percent of cost of goods to 1.5 this year, and none of the six Malaysian outlets running out of chronic meds." Measured on: write-off, by head office. Seen by: her, every month.

List the opportunities as they said them. One line each, their words, no diagnosis. Five to seven is normal for a 45-minute meeting. Mark the three that bear most directly on the outcome box; that mark is your first judgement, and you should be able to say why in one sentence per mark.

★ Step 2 · the pains, as said1. "Priya spends six hours on it every Monday." 2. "There are three versions of the stock spreadsheet going round." <- strongest 3. "Singapore and Malaysia use different product codes." <- strong 4. "Orders go on WhatsApp. There's no list of them anywhere." 5. "When it looks bad they phone the outlet. They don't change the order." <- strong 6. "The area managers are on their phones in the outlets on Monday."

Two solutions under each of the three strongest. Cheapest first, dearest second. Under each, one line for the assumption that has to be true, because that line is what the first step will test.

★ Step 3 · solutions and the assumption under eachUnder 2 (three spreadsheets): a. one shared file, typed once, Friday check assumes: managers will type into one file if it is the only one b. stock module from the POS vendor, Q1 assumes: SGD 15,000 covers it, and Kenneth's login survives January Under 3 (codes differ): a. one product master, an owner named assumes: someone will own it; today nobody does b. a mapping table kept by Priya assumes: Priya has time, which she does not on Mondays Under 5 (number changes nothing): a. outlet-level list on phones with a suggested reorder assumes: a trusted stock count exists b. a reorder rule the pharmacist applies assumes: product-level detail reaches the outlet

Write the three candidates, four lines each. What, first step, who, needs. One candidate per strong opportunity. The first step must be something a named person can do this week without asking for money.

★ Step 4 · three candidatesC1 One product master for both countries, owner named first step: Priya and I list the codes that differ, one afternoon who: Priya, me needs: nothing C2 One stock spreadsheet, typed once, with a Friday check first step: four outlets retire two versions, keep one, for two weeks who: Priya, four managers needs: nothing C3 Monday number as a phone list, outlet level, one suggested reorder first step: a shared sheet opened on a phone, four outlets who: me, three area managers needs: C2 first

Score the three, with the rule written above the grid. Impact, confidence, ease, 1 to 10. Write the combination rule first, then the digits. If your winner changes when you switch multiply for average, say so under the grid; that is the honest state of the choice.

★ Step 5 · the grid, rule firstRULE: I x C x E, each 1 to 10. (Some teams average. We multiply, because a weak leg should hurt.) I C E I x C x E C1 product master 7 8 6 336 C2 one spreadsheet 8 9 8 576 <- winner C3 phone list 9 6 5 270 Averaged: 7.0 / 8.3 / 6.7, same winner. Confidence on C3 rises the week C2 produces a count we trust.

Write the pilot in five lines. Scope, partner, test, date, and what it needs. Every line comes from door 5 or door 6. If a line is yours rather than theirs, mark it as a guess to confirm; session 6 sends the page within 24 hours and the guesses are the questions in the covering note.

★ Step 6 · the pilotPILOT: C2, one stock spreadsheet, typed once, with a Friday check scope: four outlets (she offered four, never 38) partner: Priya, two hours a week, Tuesday afternoons test: the area managers stop phoning her on Monday afternoon date: one board slide in six weeks, with the four outlets' Friday count on it needs: nothing before Q1; the SGD 15,000 conversation happens with a fact in hand

The first project needs no budget and produces a fact she can see. That is the test of a first step, and it is stricter than it sounds. A fact she can see is a number from her own outlets on a page she opened, not a demo you showed. If the first step needs money, it is the second step; if it produces a tool rather than a fact, it is the previous pilot again.

Homework

Before session 6

45 min
  1. Build the tree for your real request. Their outcome at the top in their words, their pains as the row, two solutions and one assumption under each of the three strongest. If you cannot fill the top box, the homework is the question that fills it, and you ask it this week.
  2. Prune to three and score them with the combination rule written above the grid. Run the grid a second time with the other rule and write one line on whether the winner moved.
  3. Write the pilot in five lines: scope, partner, test, date, needs. Mark every line that is your guess rather than their answer.
  4. Bring all of it to session 6, printed, with the Six Doors sheet from session 1 and the data-state map from session 2. The mock meeting runs on your own sheet, and the brief is written from these pages.
Check yourself

Three questions

Clear all three and session 5 gets stamped in your passport.

1 · Why does the top box of the tree hold the outcome in her words, and what does a tree with "build a dashboard" at the top produce?

At Harbourline the top box reads "write-off from 2.1 to 1.5 percent, no chronic-med stock-outs in six outlets". Nothing in "a management dashboard" pointed at either, so nothing built could move either, and the fact that predicted the failure, phoning the outlet instead of changing the order, was never on the page.

2 · Why must the ICE combination rule be written above the grid, and what about ICE is contested?

A candidate scored 9, 9, 2 averages 6.7 and multiplies to 162; one scored 6, 6, 6 averages 6.0 and multiplies to 216. Neither rule is wrong. Writing which one you used, before the digits, is what makes the grid a conversation about the shop instead of a ranking she has to accept.

3 · Why do four outlets and a quiet phone beat a 38-outlet rollout with a login count?

Door 6 gave you the test in her words: "if my phone is quiet, it worked." A pilot is the candidate with her scope, her partner, her test and her date pinned to it, and a rollout is requested by the 34 who watched four neighbours stop phoning, not proposed by the analyst.

Sources covered

What this session teaches, and where it came from

◐Opportunity solution tree: outcome, opportunities, solutions, assumption tests; weekly contactTeresa Torres, Continuous Discovery Habits, Product Talk, 2021; reported from summaries, and taught in depth in the Product Thinking course
◐ICE: impact, confidence, ease, each 1 to 10Sean Ellis and Morgan Brown, Hacking Growth, Currency, 2017; reported, and whether the three are averaged or multiplied is contested across summaries, so the page says write down which
✓Verify the current non-ML solution is optimized before building anything clevererGoogle for Developers, "Introduction to Machine Learning Problem Framing", developers.google.com/machine-learning/problem-framing, accessed 2026-09-29; primary, read at source
◐Harbourline, Adeline Tan, Priya and every fact in the treeConstructed. The 24 facts are printed in the bench's bank file and invented to be typical, not sourced; the ICE digits are the analyst's judgement, as the page says
◐The six-week timeline and the 34 outlets that askedConstructed to show the shape of a pilot that produces a fact; no claim is made about how long a real spreadsheet fix takes

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 5 cheat sheet · pin this

Outcome on top, her wordsA tree with a solution on top is upside down. Turn it over before you branch.
Opportunities are a transcriptPains as she said them. A diagnosis in your words is a solution in disguise.
Three candidates, three painsOne per strong opportunity, four lines each: what, first step, who, needs.
Write the rule above the gridMultiply or average is contested. Write which. Then the digits, then argue.
The simple fix firstCheck the current way is fixed before anything cleverer. The spreadsheet before the reorder.
Pilot: scope, partner, test, dateFour outlets, Priya, a quiet phone, one slide in six weeks. No budget, one fact she can see.