learn-asking-right-questions-as-da-with-phoebe / Session 3 of 6
Learn Asking the Right Questions as a Data Analyst with Phoebe · Session 3 of 6

The analyst's questions, and the bench

Sessions 1 and 2 argued that the order of questions decides what you learn. Tonight the argument is replaced by a count. Adeline Tan sits in this page with a fact sheet you cannot see: 24 things that are true about Harbourline, worth 67 points, three of which she will only say once she trusts you. You have 45 minutes and a bank of 37 questions, sorted into five kinds. The page runs the same meeting five ways in front of you: the technical opener most of us were trained to give, a shuffle, the Six Doors in printed order, the doors tuned for an analyst, and the fast yes-or-no interview that looks like the best meeting in the room. Then it hands you the chair.

🔴 Bench night Bring your Six Doors sheet from session 1 No code tonight, all clicking 45 min
0-3 · Welcome 3-14 · Five kinds of question 14-40 · The bench, then your own meeting 40-45 · Q&A
Part 0

Where we start

A bench is a fixed person and a fixed clock, so that when two ways of asking get two different results the only thing that differed was the asking. The person is constructed: Harbourline is invented and so is Adeline, and every fact on her sheet, every minute cost and every trust gate is printed in a file you can open and change. What the bench claims is not a number about pharmacies. It is a mechanism, shown as an ordering you can reproduce: jargon costs twice, leading questions record beliefs that are wrong, and the facts that matter most only come out once trust exists.

🎫 Clear all three quiz questions to stamp your passport counted = computed from the printed bank, in your browser constructed = the company, the person, the sheet

★ What you walk out with tonight. Your own question bank sorted into the five kinds, the counted answer to "what is the framework worth against a shuffle" (48 against 28, on this sheet), the reason the fastest interview scores lowest, and one run of the meeting in your own hands with a score beside the presets.

Part 1 · covers the five kinds, and the analyst's door

Five kinds of question, one of which earns trust

6 min live

Every question in the bank is one of five kinds, and Adeline answers each kind differently. The kinds are not about wording. They are about what comes back: a fact, nothing, a generic, a request to explain yourself, or an agreement with something you assumed. The analyst's door, decision, has the highest-value fact on the sheet behind it, and it is opened by one kind only.

FIVE KINDS, AND WHAT COMES BACK the kind is not the wording; it is what she can do with it OPEN "Walk me through Monday morning." reorder, Tuesday noon GM asks every Monday trust, one notch up CLOSED "Is the report weekly?" "yes" one fact at most, nothing you did not ask VAGUE "What are your main pain points?" "the usual" a generic, and three minutes gone JARGON "What's your refresh SLA on it?" ? "sorry, what do you mean?" and a notch lost five minutes, not three LEADING "You'd want this daily, right?" "I suppose so." "daily" is now in your notes. It is wrong. Only the first kind moves the trust meter up, and three of the 24 facts on the sheet are behind that meter.
🔍 Click to zoom - the two right-hand cards cost more than they look: one in minutes and trust, one in a build that goes wrong later
LiveWhat each kind gets back, and what it costs3 min▶

Open. About the past or the present, answerable by anyone who runs the shop. "Walk me through Monday morning." It costs three to five minutes, it uncovers the facts behind its door, and it earns a point of trust, which is the currency the sensitive facts are priced in. Every open question in the bank carries a teach-back line.

Closed. Yes or no. "Is the report weekly?" One minute, one fact at most, and only the fact you already had the shape of. Cheap, and useful for confirming, but a meeting made of them learns nothing you did not walk in expecting.

Vague. "What are your pain points?" It sounds open. It is not, because it gives her nothing to be specific about, and what comes back is a generic: "the usual, stock, staff, the reports being late." Three minutes for a sentence you could have written yourself.

Jargon. "What is your refresh SLA?" She has to ask what you mean, so it costs the question plus an explanation, five minutes rather than three, and she is a little less sure you understand her shop. On the bench that is a point of trust lost, and it is the only kind that loses one.

Leading. "You would want this daily, right?" She says "I suppose so", because the question told her the expected answer and she is being polite. Nothing true came back. Something false went into your notes, and it costs later, when you build a daily refresh nobody reads. On the bench a leading question records a belief with a cost attached, and the fast interview records five of them.

LiveThe analyst's door is decision, and here is its bank3 min▶

An analyst's output is a number somebody acts on. The facts that decide whether the number is right are the action, the person and the deadline, and they live behind door 2. On Adeline's sheet the single most valuable fact, worth 5 of 67, is that the Monday decision is next week's reorder per outlet, made by three area managers, due Tuesday noon. Three more decision facts are worth 8 between them, and one of those, that today they phone the outlet rather than change the order, only comes out once she trusts you.

The four open questions that open that door, each with its teach-back line, are walk me through Monday morning; when a number looks bad who makes the call and by when; if the numbers arrived a day earlier what would you do differently; and tell me about the last time the report changed a decision. The analyst-tuned order on the bench asks the first three of them first. By minute 13 they have put 15 points in your notes and taken trust to 3, which is exactly the level the sheet's most sensitive fact is priced at. The fourth is held back as a spare, because the first three already reach all four decision facts on her sheet.

Self-studyThe whole bank, by kind3 min▶

Thirty-seven questions: twenty open (two to four per door), four vague, five jargon, five leading, three closed. The open ones are the sheet you built in session 1 with Adeline's answers already attached. The others are there because they are what gets asked in real meetings, and the bench needs them to show what they cost. Every one is printed with its kind, its minute cost and the facts it reaches in the bank file the page loads; the widget below draws its buttons from that file, so what you click is what it counts.

KindIn the bankMinutesWhat comes backTrust
Open203 to 5The facts behind its door, when trust is high enough for them+1
Closed31 to 2One fact at most0
Vague43 to 4A generic; at most one low-value fact0
Jargon55"Sorry, what do you mean?"-1
Leading51 to 2"I suppose so", and a false belief with a cost of 2 to 40
Build-along · the bench

Five ways of asking, counted in front of you

★ 26 min · the main event

Work down the five presets in order. For each, read the verdict line, then the seven counters, then the transcript underneath: what you asked, what she said, what it was worth, and where trust stood. Say out loud what changed before moving to the next. The last preset is the anti-lever, the interview that looks most efficient, and the bench shows what it costs. Then the lower half of the widget gives you the chair: same person, same 45 minutes, your choice of questions, scored beside the presets.

Read it in this order. Net value, then false beliefs, then doors opened, then the "after 20 minutes" counter. Net value is what you can build on. False beliefs are what you will build wrong. Doors opened is whether you can write a brief at all. The 20-minute counter is what is in your notes if she gets called away, which happens.

Part 2 · covers what the bench finds

Same person, five answers

5 min live

The bench above computes these live; the numbers here are the ones it produces on the printed bank, written down so they can be argued with after class. Change a value in the bank file and they move. The ordering does not.

NET VALUE OUT OF 67, FIVE WAYS OF ASKING THE SAME PERSON wide bar = net value after 45 minutes · thin bar = what was in your notes after 20 · scale: 8 px per point, from x = 230 The efficient interview 15 5 false beliefs · after 20 min: -2 The technical opener 17 6 facts, 3 doors · after 20 min: 0 Any order at all 28 mean of 200 range 8 to 47 after 20 min: 14 Six Doors, in order 48 after 20 min: 19 Tuned for an analyst 52 after 20 min: 21 the ceiling, 61 an optimizer that can read her sheet Same person, same clock. The way of asking, not the person, decided whether the analyst left with 15 points or 52.
🔍 Click to zoom - the top bar is the meeting most of us have run, and it is the shortest
Way of askingNet valueAfter 20 minFactsDoorsFalse beliefsQuestions
The efficient interview (leading and closed first)15-2114516
The technical opener (five jargon questions first)17063010
Any order at all (mean of 200 shuffles)28.214.011.95.32.413.8
Six Doors, door by door4819176011
Six Doors, tuned for an analyst5221186010
Ceiling: an optimizer that can read the sheet6133

All counted from the printed bank. Random is a mean over 200 seeded shuffles; its best single draw is 47 and its worst is 8.

LiveFinding 1: the framework is not optimal, it is reproducible2 min▶

Six Doors in printed order gets 48 of 67 with no false beliefs and every door opened. Tuning the order for an analyst, decision door first, adds four points and, more usefully, puts 21 in your notes by minute 20 instead of 19. A search that is allowed to read the fact sheet finds an order worth 61, and it opens with "what happens when Priya is away" before the interviewer has heard the name Priya. Nobody in a real meeting can read the sheet. Optimal against a known answer is not the same thing as good; the framework is the thing you can do on Tuesday, blind, and get 48.

LiveFinding 2: a shuffle is not neutral, it is a spread2 min▶

Two hundred shuffles of the same bank average 28 and range from 8 to 47. The average is the meeting most people run when they have a good list and no order: some of the jargon lands early and costs trust, a leading question or two gets recorded, and one door stays shut. The spread is the argument for an order at all. A method that scores 48 every time is worth more than one that scores 47 on a good day and 8 on a bad one, and the bad days are the ones with the board in six weeks.

Self-studyFinding 3: jargon costs twice, and the second cost is the one that matters2 min▶

Five technical questions in a row spend 25 of the 45 minutes and take trust to -3. The obvious cost is the minutes. The one that matters is what happens to the open questions that follow: the sensitive facts stay unsaid because she is no longer sure you understand her shop, and the whole meeting ends with six facts and three doors. The technical opener is not wrong because the questions are bad; every one of them is a question you will need answered by Kenneth in week two. It is wrong because they were asked of the wrong person, first.

Part 3 · covers the anti-lever

The efficient interview is the worst meeting on the bench

4 min live

Sixteen questions in 44 minutes, eight answers in the first nine. It feels like the best-run meeting in the building, and on the bench it scores 15, the lowest of the five, because five of the things in your notes are things she agreed to rather than things that are true.

ONE LEADING QUESTION, SIX WEEKS LATER LEADING "The stock numbers come out of the POS system, correct?" "I think so." Monday report notes stock = POS export week six: a stock chart wired to the till data -4 and a week of rework WHAT WAS TRUE, ON HER SHEET stock is a shared spreadsheet per outlet, typed by outlet managers on Fridays, with three versions in circulation. The POS has never held stock. THE OPEN VERSION, 4 MINUTES "Where do the stock numbers in it come from, step by step?" gets the spreadsheet, the Fridays and the three versions She was not lying. The question told her the answer you wanted, and she is polite. The cost lands on you, six weeks later.
🔍 Click to zoom - the red line in the notebook is the only thing the fast interview produces quickly
LiveWhy "I suppose so" is worse than silence2 min▶

A closed question that gets a "no" has taught you something. A leading question almost never gets a "no", because it carries its expected answer and the person across the table is being courteous, busy, or both. Fitzpatrick's rule for customer conversations applies without change: people will say yes to be polite. What comes back is not information about her shop; it is information about how agreeable she is, filed as if it were the first thing.

The bench prices each false belief at what building on it wastes. "Daily refresh" costs 3: a pipeline nobody reads. "Product-level for the area managers" costs 2: a screen they scroll past. "Stock comes from the POS" costs 4: a chart wired to a source that has never held stock, discovered in week six. "A dashboard is the deliverable" costs 3, because the last vendor's dashboards went unopened after week two and the word itself is the thing she is wary of. Five leading questions in five minutes, and the meeting starts at minus 14 before the first true fact.

Self-studyThe honest nuance: when a closed question is right2 min▶

Closed questions are not the villain; leading ones are. "Is Priya the only one who builds it?" is closed and gets a real fact for two minutes, and the bench scores it so. The rule is about sequence: confirm with a closed question after an open one has shown you the shape of the answer, never before. And the anti-lever carries its own limit: a fast interview that asked only closed questions, no leading ones, would score higher than 15 and lower than the framework, because it would still learn nothing it did not walk in expecting. The bench does not pretend that speed is the sin. Assuming is.

Real world

The interview that impressed everyone. A constructed case with a familiar shape. An analyst runs a tight 40-minute session with a department head, sixteen questions, every one answered, a requirements list by lunchtime. Six weeks later the daily refresh nobody asked for is the reason the build is late, and the stock figure is wired to a system that never held it. The department head, asked why she agreed to all of it, says what people say: "You seemed to know what you wanted. I assumed you knew." The fix costs nothing but the sequence: open first, confirm second, never assume out loud.

Homework

Before session 4

30 min
  1. Run your own meeting twice in the lower half of the bench: once with your session 1 sheet in your hand, once without. Write down the two scores and which doors you left shut the second time.
  2. Sort your last real interview. Take the notes from a discovery meeting you actually ran and mark each question you asked with its kind. Count the leading ones. Each is a belief in your notes that nobody checked.
  3. Change one number in the bank file and re-run. Double the cost of every false belief, or halve the minute cost of jargon. Write one sentence on what moved and what did not; the ordering should survive, and if it does not, you have found the assumption the bench rests on.
  4. Bring the twelve teach-back lines from your sheet. Session 4 rewrites them for a person who has never seen a table, and picks the one concept per meeting worth explaining properly.
Check yourself

Three questions

Clear all three and session 3 gets stamped in your passport.

1 · The efficient interview asks 16 questions and uncovers 11 facts, more questions than any other preset. Why does it score 15, the lowest on the bench?

The raw value of the facts it finds is 29, which would be a middling meeting. The 14 points of false beliefs are the cost of building on "daily", "product-level", "stock from the POS", "a dashboard" and "laptops", none of which is true. Speed is not the defect. Assuming out loud is.

2 · An optimizer that can read Adeline's fact sheet finds an order worth 61, against 48 for the framework. Why does the course still teach the framework?

The ceiling is printed so the framework's number can be read honestly: 48 is not the best possible meeting, it is the best reproducible one. The 200-shuffle spread, 8 to 47, is the other half of the argument: an order is worth having because the bad days are the ones that matter.

3 · The technical opener asks five perfectly good data questions first and ends with six facts and three doors. What went wrong?

Door 4 is opened fourth, and with the plain version: which systems does a sale touch, who can log in. The warehouse and the schema are real questions for the person who holds the login, and the bench's point is that asking them of the operations director costs twice, in minutes and in the trust the rest of the meeting is priced in.

Sources covered

What this session teaches, and where it came from

✓Every count on the benchComputed in the browser from the printed bank file, and verified headlessly in node before any page quoted a number; the two agree to the point
✓People say yes to be polite; ask about the past, not the futureRob Fitzpatrick, The Mom Test, 2013; the leading-question mechanism on the bench is his observation made countable
◐Harbourline, Adeline Tan, and every value on her sheetConstructed to be typical. The bench claims the ordering and the mechanism, never the numbers as facts about any real business; change the bank and re-run
◐How often real meetings run like the efficient interviewNot claimed. No count of real discovery meetings by question kind was found, and none is asserted

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 3 cheat sheet · pin this

Five kindsOpen, closed, vague, jargon, leading. Only open earns trust; only jargon loses it; only leading records a lie.
The bench, on this sheetFast 15 · jargon 17 · shuffle 28 (8 to 47) · Six Doors 48 · analyst-tuned 52 · ceiling 61.
Read net, then false, then doorsWhat you can build on, what you will build wrong, whether a brief is possible.
The 20-minute counterWhat is in your notes if she is called away. Decision door first puts 21 there.
Open first, confirm secondA closed question after an open one is cheap and true. Before one, it is a guess with a yes attached.
Optimal is not good61 needs her sheet. 48 needs a printed page and works blind.