learn-asking-right-questions-as-da-with-phoebe / Session 1 of 6
Learn Asking the Right Questions as a Data Analyst with Phoebe · Session 1 of 6

Why the gap exists, and the Six Doors

An operations director asks for a sales dashboard. An analyst hears a data request and starts asking about exports and refresh schedules. Six weeks later there is a dashboard, it is correct, and nobody opens it, because the thing she was worried about was six outlets running out of chronic medication and the word she had for that was "dashboard". Neither of them was careless. They were using the same words for different things, and nobody in the room had a way of noticing. Tonight is the way of noticing: why the gap opens every time, the six doors every discovery meeting has, and the one sentence that lets the business person learn how data work thinks while she is answering you.

🟡 Core Analysts, and anyone who gets asked for numbers No code. A pen. 45 min
0-3 · Welcome 3-18 · The gap, the doors, the teach-back 18-40 · Build-along: your Six Doors sheet 40-45 · Q&A
Part 0

Where we start

Most of what goes wrong in data work goes wrong in the first meeting, and it goes wrong quietly. The business person says what they want in the vocabulary they have, the data person translates it into the vocabulary they have, and both walk out believing they agreed. The running case for all six sessions is Harbourline, an invented chain of 38 pharmacy and clinic outlets across Singapore and Malaysia, and its operations director Adeline Tan, who has never heard the word pipeline and does not need to. She has a request. You have 45 minutes with her. Everything in the six sessions is about what you ask in those minutes.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = write it yourself

★ What you walk out with tonight. A one-page Six Doors sheet for a request you actually have on your desk, two open questions behind each door written in the past tense, a plain teach-back line attached to each one, and a mark on the door your role should open first. Session 3 puts that sheet in front of Adeline and counts what it gets out of her.

Part 1 · covers why the gap opens

Two people, one word, two meanings

5 min live

In 1990 a Stanford psychology student, Elizabeth Newton, split people into tappers and listeners. Tappers picked a well-known song and tapped its rhythm on a table; listeners had to name it. Before tapping, tappers guessed the listeners would get about half the songs. Listeners got 3 of 120. The tapper cannot stop hearing the tune, so the taps sound obvious; the listener hears knocking. Every discovery meeting has a tapper on each side of the table.

TAPPERS AND LISTENERS, 1990 the tapper hears the whole song; the listener hears a table being knocked the song, whole tapper knock, knock, knock-knock listener tappers predicted about 50% of songs would be named listeners named 3 of 120 2.5 percent six outlets short of chronic meds Adeline "we need a dashboard" the analyst WHAT GETS BUILT a correct sales dashboard, six weeks, opened twice, and the stock-outs continue
🔍 Click to zoom - the bottom row is the top row with the tune replaced by an operations problem and the knocking replaced by a word
LiveThe tune in your head, and why neither side can hear the other's3 min▶

Newton's tappers were not bad at tapping. They were unable to un-know the song while they tapped, so every knock arrived with the melody attached, and they judged the listener's chances from inside that experience. Economists had named this a year earlier: Camerer, Loewenstein and Weber called it the curse of knowledge, the finding that better-informed people cannot fully set their information aside when predicting what less-informed people will understand.

In a discovery meeting there are two tunes. The business person's tune is the operational worry: which outlets, which supplier, which Monday phone call. The data person's tune is the machinery: exports, joins, refresh cadences, the shape of a table. Each hears their own tune behind every word the other says. "Dashboard" is knocking. To Adeline it means "make the Monday problem go away". To the analyst it means a page with charts on it. Both leave the meeting sure they agreed.

The gap is not closed by either side learning the other's vocabulary. It is closed by questions that make the tune audible: about the decision, the current way of working and the real state of the data, asked in words that need no translation, with the reason for asking said out loud. That is the rest of tonight.

Self-studyThe translator role, and why you should not wait for one2 min▶

In 2018 three McKinsey partners argued in Harvard Business Review that analytics teams needed a named role, the analytics translator, whose job is to make sure the analytics work actually solves the business problem. The article is worth reading for the shape of the role. The number of translators the firm predicted the economy would need circulates widely and was not read at source, so it is not quoted here.

The practical point: most teams do not have one. The translation happens, or fails to happen, in the first meeting, and the person running that meeting is you. Treating the discovery conversation as part of the analyst's craft, rather than somebody else's job, is the position these six sessions take.

Real world

The stand-in request. A constructed case, but the shape is the most common one there is. Harbourline's Monday sales report is late and untrusted, so the request that reaches the analyst is "a dashboard". The worry underneath is stock-outs of chronic medication in six Malaysian outlets, and the decision the report is supposed to feed is next week's reorder quantity per outlet, due Tuesday noon. Nothing in the word "dashboard" carries any of that. It comes out in about four minutes if you ask what happens on Monday morning, and never comes out at all if you ask about the data warehouse.

Part 2 · covers the framework

Six Doors

6 min live

A discovery meeting is a corridor with six doors. Behind each one is a kind of fact you need before you can build anything, and each opens to a different kind of question. Most analysts walk straight to the fourth door, data state, because that is where the work lives, and leave the first two shut. The framework is the corridor drawn out, so you can see which doors you opened and which you walked past.

THE CORRIDOR: SIX DOORS, ONE MEETING the question on each door is the one that opens it · walk them in order 1 2 3 4 5 6 Outcome If this worked, what is different in six months? the worry behind the request Decision Who does what with the number, and by when? the action, who, and the deadline Current way How does it happen today? Show me the file. the spec nobody wrote down Data state Where do the numbers live, who can log in? where analysts start, and stay Constraints What is off the table? Budget, legal, device. the shape of a yes Success How will we both know, and who works with me? the test, and the partner stood here for 25 minutes, asking about the warehouse start here Doors 1 and 2 are the ones a data person walks past. They are also the ones that decide whether anything gets used.
🔍 Click to zoom - the hatched door is the one most of us were trained to open first, and it is the fourth
DoorThe question that opens itWhat you walk out withA teach-back line to attach
1 · OutcomeIf this worked, what would be different in six months? Which outlets keep you up at night?The worry the request stands in for. Harbourline: stock-outs of chronic meds in six outlets, and a write-off target of 1.5 percent"A dashboard is a tool. I need the outcome, because I can check a tool against an outcome and not the other way round."
2 · DecisionWalk me through Monday morning. Who opens it, what do they do next, by when? When the number is bad, what changes that week?The action, the person and the deadline. Harbourline: next week's reorder per outlet, three area managers, Tuesday noon"The deadline tells me how fresh the data has to be. Fresher costs more, so I only build fresh where a decision needs it."
3 · Current wayHow is it made today? Can you show me the file and tell me who builds it? Where does each number come from, step by step?The unwritten spec. Harbourline: Priya, six hours every Monday, an adjustments column nobody can explain"The current file is the spec. Every column in it is a decision somebody once made, and I would rather inherit those than guess."
4 · Data stateWhich systems does one sale touch? Who can log in and pull an export? Do both countries use the same product codes?Sources, owners, access, freshness. Harbourline: one POS vendor, per-outlet CSV, codes that differ by country, one IT person"Every hand-off between systems is a place a number can change. I am counting the hand-offs."
5 · ConstraintsWhat is off the table? Budget, time, legal, people. Where do they read it, laptop or phone? When must you show something upward?The shape of a yes. Harbourline: no budget until Q1, phones on Monday, a board slide in six weeks"I ask for the limits first so I do not propose something you could never approve."
6 · Success and partnershipHow will you know it worked, and what would you stop doing? Have you tried this before? Who could spend two hours a week with me?The test, the history and the partner. Harbourline: a quiet phone on Monday afternoon, a vendor pilot nobody opened, Priya"If we agree on the test now, neither of us has to argue about it later."
LiveDoors 1 and 2: start from the decision, because a number nobody acts on measures nothing3 min▶

Douglas Hubbard's definition of measurement is the whole of door 2 in one line: a measurement is a quantitatively expressed reduction of uncertainty based on one or more observations. Uncertainty about what? About a decision. His method begins by defining the decision before anything is counted, and if you cannot name a decision the number will change, you have not found a measurement yet, you have found a chart.

So the first door is the outcome and the second is the decision, and they come before any question about data. "What happens on Monday morning" gets you the decision (reorder quantities), the person (three area managers), the deadline (Tuesday noon) and, if you listen, the fact that today the number changes nothing because when it looks bad they phone the outlet. That last fact is worth more than any export schedule. It says the report needs to be different, not faster.

Teresa Torres's discovery work puts the same thing in product language: a desired outcome sits above the opportunities, which sit above the solutions. A dashboard is a solution. Asking for the outcome first is how you find out what it is a solution to.

LiveDoors 3 and 4: specifics in the past, and "show me the file"3 min▶

Rob Fitzpatrick wrote The Mom Test for founders talking to customers, and its three rules transfer whole: talk about their life instead of your idea; ask about specifics in the past instead of generics or opinions about the future; talk less and listen more. His verified line on the second rule is the one to remember: "The world's most deadly fluff is: 'I would definitely buy that.' But folks are wildly optimistic about what they would do in the future." For an analyst, "would you use a daily number?" is that fluff. "Tell me about the last time the report changed a decision" is the past, and the past is where the facts are.

Door 3, the current way, is opened by one request: show me the file. The Monday Excel is the spec nobody wrote. Every column is a decision somebody made once, the adjustments column is a fudge that encodes a real problem, and the six hours Priya spends are the cost you are being asked to remove. Door 4, data state, is the door analysts open first and should open fourth: which systems a sale touches, who owns each, who can log in, whether Singapore and Malaysia use the same product codes. DalleMule and Davenport's 2017 frame for data strategy asks whether a business needs a single source of truth or several fit-for-purpose versions; Harbourline, with three versions of the stock spreadsheet in circulation, has the second by accident.

Taiichi Ohno's five whys belong at door 3 as well. His example walks from a stopped machine through a blown fuse, a dry bearing and a failing pump to a missing strainer. "Why is the report late" walks from Monday to Priya to the POS export to product codes that differ by country. Stop at "Priya is slow" and you build the wrong thing.

Self-studyDoors 5 and 6: the shape of a yes, and the partner3 min▶

Constraints are asked for early because they define what a proposal can look like, not because they are interesting. No budget until Q1 means the first deliverable is built with what exists. Phones on Monday means a table nobody will scroll on a phone is not read. A board in six weeks means one honest number by then beats a suite after it. Malaysian loyalty data under PDPA with a lawyer who takes four weeks means anything touching it is a second phase.

Door 6 is two questions. The first is the test: how will you know it worked, and what would you stop doing? Adeline's answer is the best kind, observable and hers: the area managers stop phoning her on Monday afternoon. The second is the partner: who on her side spends two hours a week with you. Torres argues for weekly contact with the people you are building for; the analyst's version is a named person, not a stakeholder group. The person who works with you is the person who can run it after you leave. Fitzpatrick's test for whether a conversation went well is whether the other side gave up something, time, a name, an introduction; Priya, two hours a week, is that something.

Walk the doors in order the first time. Session 3 shows what the order is worth: on the bench, the same questions in a shuffled order average 28 points against 48 in door order, and the worst shuffle scores 8. Once you can walk them blind, session 3 also shows which door an analyst should open widest, and it is door 2.

Part 3 · covers the teach-back line

Why I am asking, in one sentence

4 min live

Every open question in the bank comes with a second sentence: a plain statement of why you are asking it. "The deadline tells me how fresh the data has to be, and fresher costs more." Said out loud, it does three things at once. It makes the question feel like a reason rather than an interrogation, it earns trust, and it teaches the business person one true thing about how data work is decided. By the end of a 45-minute meeting she has learned ten of them, without a slide.

ONE QUESTION, TWO SENTENCES, THREE THINGS BACK THE QUESTION "When a number looks bad, who decides, and by when?" THE TEACH-BACK "The deadline tells me how fresh the data has to be. Fresher costs more, so I only build fresh where a decision needs it." A fact comes back "Tuesday noon. And honestly they phone the outlet, they don't change the order." A lesson stays with her freshness is a cost, not a default; weekly is a choice trust, one notch up The second sentence is why she answers the next question more fully, and the only lesson she takes home.
🔍 Click to zoom - the terracotta box is the part of the meeting that outlives it
LivePlain, true, one sentence, said out loud2 min▶

A teach-back line has three properties. It is plain: no word she would have to ask about, so "freshness" not "latency", "the file" not "the extract". It is true: a real principle of the craft, not a pleasantry. And it is one sentence, because two is a lecture and she stops answering.

Three that earn their place in almost any meeting. Before a data-state question: "Every hand-off between systems is a place a number can change, so I am counting the hand-offs." Before a current-way question: "The current file is the spec. Every column is a decision somebody made once, and I would rather inherit those than guess." Before a success question: "If we agree on the test now, neither of us has to argue about it later." Session 4 builds twelve of these and shows how to choose the one concept per meeting that is worth explaining properly.

Self-studyFive kinds of question, and only one of them earns trust2 min▶

Session 3 sorts every question in the bank into five kinds, and the simulator scores them differently. Open questions about the past or present uncover facts and earn a point of trust. Closed yes-or-no questions are cheap and get you one fact at most. Vague questions ("what are your pain points?") cost minutes and get a generic back. Jargon costs twice: she has to ask what you mean, and she is a little less sure you understand her shop. Leading questions are the dangerous kind: "you'd want this daily, right?" gets a "sure", and a belief that is wrong goes into your notes and later into your build.

The trust point is not decoration. On the bench, three of the 24 facts on Adeline's sheet only come out once trust exists, and one of them, that the only IT person is leaving in January, is the one that decides whether anything you build survives the quarter.

Build-along

Your Six Doors sheet, for a request you have right now

★ 22 min · everyone writes

One page, six rows. Take a real request from your own inbox, or use Adeline's if you do not have one, and build the sheet you will carry into the meeting. Every line is written before the meeting; the meeting is for listening.

THE SHEET, ONE PAGE, WRITTEN BEFORE THE MEETING in their words: "we need a sales dashboard, the Monday report is always late" DOOR TWO OPEN QUESTIONS, PAST TENSE TEACH-BACK, ONE LINE EACH 1 Outcome last time it worked, what changed? tools are checked against outcomes 2 Decision walk me through Monday, by when? the deadline sets the freshness 3 Current way show me the file. where from? the file is the spec 4 Data state which systems? who can log in? I am counting the hand-offs 5 Constraints what is off the table? phone? limits first, so I propose a yes 6 Success how did you know? who has 2 hours? agree the test now, not later the mark: the door your role opens first and widest
🔍 Click to zoom - twelve questions and twelve second sentences fit on one page, and the meeting is for listening

Write the request as it was received, in their words. Not your translation. If it arrived as "we need a dashboard", write "we need a dashboard". You will want this sentence back at the end of the meeting to see how far the real request was from it.

★ Step 1 · the requestRequest, verbatim: "We need a sales dashboard. The Monday report is always late and nobody trusts the numbers in it." From: Adeline Tan, operations director Received: Thursday, by email, forwarded from the regional GM

Two open questions per door, in the past or present tense. "Tell me about the last time", "walk me through", "show me", "where did that come from". Nothing with "would you". Nothing that can be answered with yes. Write them so a person who has never seen a database can answer every one.

★ Step 2 · door 2, decision, two questionsWalk me through Monday morning. Who opens the report, and what do they do next? When a number looks bad, what changes that week? Who makes the call, and by when?
The same door, as it is usually openedWhat's your refresh SLA on the Monday report? Would a daily view be useful?

Attach a teach-back line to each question. Plain, true, one sentence. Read each pair aloud once; if the second sentence has a word she would have to ask about, change the word.

★ Step 3 · the pairQ: When a number looks bad, what changes that week? Who makes the call, and by when? Why I am asking: The deadline tells me how fresh the data has to be. Fresher costs more, so I only build fresh where a decision needs it.

Mark the door your role opens first. For an analyst it is door 2, decision: your output is a number somebody acts on, so the action, the person and the deadline are the facts you cannot build without. Session 3 measures what that choice is worth. If you are a scientist, an engineer or building with AI, the sibling course for your role marks a different door, and links back here for the corridor.

★ Step 4 · the order for tonight2 Decision (widest) -> 3 Current way -> 1 Outcome -> 4 Data state -> 5 Constraints -> 6 Success Budget: 45 minutes. About ten open questions fit. The sheet has twelve; the last two are spares.

Rehearse the opener against Adeline. Her first line is on the page: "Thanks for coming. We need a sales dashboard." Say your first question out loud, with its teach-back. If it took more than twenty seconds, it is two questions; split it.

★ Step 5 · the first thirty secondsAdeline: "Thanks for coming. We need a sales dashboard. The Monday report is always late and honestly nobody trusts the numbers in it." You: "Before the dashboard, can I ask about the Monday? Walk me through Monday morning: who opens the report, and what do they do next? I start there because a report is only worth what someone does after reading it."

Keep the sheet after the meeting. Session 6 turns a filled-in sheet into the one-page brief you send her within 24 hours, and session 5 turns her answers into three candidate projects. The same page carries all of it.

Homework

Before session 2

30 min
  1. Finish the sheet for your real request: twelve questions, twelve teach-back lines, one mark. Bring it printed; session 3 puts it on the bench.
  2. Find the tune. For the request you chose, write the one sentence you think the business person would say if you asked "what would be different in six months?" Then write the sentence you fear they would say. Both are guesses; the meeting replaces them.
  3. Re-read a past request you built something for, and mark which of the six doors you opened. Most people find two. Write down what you would have learned at each door you skipped.
  4. Read the current-way file for your request if one exists, the Excel or the deck. List every column you cannot explain. Session 2 starts there.
Check yourself

Three questions

Clear all three and session 1 gets stamped in your passport.

1 · Newton's tappers predicted listeners would name about half the songs; listeners named 3 of 120. What does the study say about a discovery meeting?

The curse of knowledge runs both ways. Adeline hears "stock-outs" behind "dashboard"; the analyst hears "charts". Neither is careless, and neither notices, which is why the doors are walked in order rather than trusted to instinct.

2 · Why do the outcome and decision doors come before the data-state door, when the data is where the analyst's work lives?

At Harbourline, "who makes the call and by when" yields Tuesday noon, three area managers, and the fact that today the number changes nothing. That last fact says the report needs to be different, not faster, and no export schedule would have surfaced it.

3 · What is the teach-back line, and why is it not just politeness?

On the bench, three of Adeline's 24 facts only surface once trust exists, and one of them, the only IT person leaving in January, decides whether anything built survives the quarter. The second sentence is how trust is built while you are still asking.

Sources covered

What this session teaches, and where it came from

✓Tappers and listeners, about 50 percent predicted, 3 of 120 namedElizabeth Newton, "The Rocky Road from Actions to Intentions", PhD dissertation, Stanford, 1990; figures as reported in Chip Heath and Dan Heath, Made to Stick, Random House, 2007, ch. 1. The dissertation itself was not read at source
✓The curse of knowledge, the termCamerer, Loewenstein and Weber, "The Curse of Knowledge in Economic Settings", Journal of Political Economy 97(5), 1989
✓Measurement as a reduction of uncertainty about a decisionDouglas W. Hubbard, How to Measure Anything, 3rd ed., Wiley, 2014; definition quoted as it appears across summaries, book not opened here
✓Specifics in the past, not opinions about the future; the three rulesRob Fitzpatrick, The Mom Test, 2013; the quoted line is the verified one, and the "over-optimistic lie" paraphrase that circulates is not his wording
◐Outcome before opportunity before solution; weekly contactTeresa Torres, Continuous Discovery Habits, 2021; reported from summaries, and taught in depth in the Product Thinking course
◐The analytics translator roleHenke, Levine and McInerney, Harvard Business Review, 5 Feb 2018; authors and date confirmed, body paywalled, and the headcount figure that circulates is not quoted
◐Harbourline and Adeline TanConstructed. Every fact on her sheet is printed in the bench's bank file and is invented to be typical, not sourced

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 1 cheat sheet · pin this

Two tunesShe hears the operational worry behind every word; you hear the machinery. Neither can switch it off.
Six Doors, in orderOutcome · Decision · Current way · Data state · Constraints · Success and partnership.
Decision before dataA number nobody acts on measures nothing. Get the action, the person, the deadline first.
Specifics in the past"Tell me about the last time", "show me the file". Never "would you".
The teach-back linePlain, true, one sentence, before the question. Trust up, and she learns one real thing.
The analyst's doorDoor 2, decision, opened first and widest. Session 3 counts what that is worth.