learn-asking-right-questions-as-de-with-phoebe / Session 3 of 6
Learn Asking the Right Questions as a Data Engineer with Phoebe · Session 3 of 6

The engineer's questions, and the bench

Sessions 1 and 2 argued that the order of questions decides what you learn, and that every system a number passes through is a place it can change. Tonight the argument is replaced by a count. Adeline Tan sits in this page with a fact sheet you cannot see: 24 things that are true about Harbourline, worth 68 points, seven of which she will only say once she trusts you. She opens with "Our numbers never match. Finance says one thing, the POS says another, and the auditor asked which one was right." You have 45 minutes and a bank of 37 questions, sorted into five kinds. The page runs the same meeting five ways in front of you: the technical opener most engineers were trained to give, a shuffle, the Six Doors in printed order, the doors tuned for an engineer, and the fast yes-or-no interview that looks like the best meeting in the room. Then it hands you the chair.

🔴 Bench night Bring your system list from session 2 No code tonight, all clicking 45 min
0-3 · Welcome 3-15 · Five kinds of question 15-40 · The bench, then your own meeting 40-45 · Q&A
Part 0

Where we start

A bench is a fixed person and a fixed clock, so that when two ways of asking get two different results the only thing that differed was the asking. The person is constructed: Harbourline is invented and so is Adeline, and every fact on her sheet, every minute cost and every trust gate is printed in a file you can open and change. What the bench claims is not a number about pharmacies. It is a mechanism, shown as an ordering you can reproduce: jargon costs twice, leading questions record beliefs that are wrong, and the fact an engineer most needs, who holds the only login, only comes out once trust exists.

🎫 Clear all three quiz questions to stamp your passport counted = computed from the printed bank, in your browser constructed = the company, the person, the sheet

★ What you walk out with tonight. Your own question bank sorted into the five kinds, the counted answer to "what is the framework worth against a shuffle" (47 against 27.6, on this sheet), the reason the fastest interview writes five wrong things into your notes before its first true fact, and one run of the meeting in your own hands with a score beside the presets.

Part 1 · covers the five kinds, and the engineer's door

Five kinds of question, one of which earns trust

6 min live

Every question in the bank is one of five kinds, and Adeline answers each kind differently. The kinds are not about wording. They are about what comes back: a fact, nothing, a generic, a request to explain yourself, or an agreement with something you assumed. The engineer's door, data state, has the most facts on the sheet behind it, and the most sensitive one, and it is opened by one kind only.

FIVE KINDS, AND WHAT COMES BACK the kind is not the wording; it is what she can do with it OPEN "Which systems does one sale touch?" POS: CSV per outlet ledger: monthly export trust, one notch up CLOSED "Is the close on the fifth working day?" "yes" one fact at most, nothing you did not ask VAGUE "What does good data look like to you?" "accurate?" a generic, and three minutes gone JARGON "Change data capture on the POS?" ? "sorry, what do you mean?" and a notch lost five minutes, not three LEADING "The POS is the source of truth, right?" "I suppose so." now in your notes, and it is wrong Only the first kind moves the trust meter up, and seven of the 24 facts on the sheet sit behind that meter.
🔍 Click to zoom - the two right-hand cards cost more than they look: one in minutes and trust, one in a pipeline that goes wrong later
LiveWhat each kind gets back, and what it costs3 min▶

Open. About the past or the present, answerable by anyone who runs the business. "Which systems does one sale touch, from the till to the bank to the ledger?" It costs three to five minutes, it uncovers the facts behind its door, and it earns a point of trust, which is the currency the sensitive facts are priced in. Every open question in the bank carries a teach-back line.

Closed. Yes or no. "Is the close on the fifth working day?" One or two minutes, one fact at most, and only the fact you already had the shape of. Cheap, and useful for confirming, but a meeting made of them learns nothing you did not walk in expecting.

Vague. "What does good data look like to you?" It sounds open. It is not, because it gives her nothing to be specific about, and what comes back is a generic: accurate, on time, the usual. Three minutes for a sentence you could have written yourself, and on this bank that one reaches no fact at all.

Jargon. "Do you have change data capture on the POS, or should we batch?" She has to ask what you mean, so it costs the question plus an explanation, five minutes, and she is a little less sure you understand her business. On the bench that is a point of trust lost, and it is the only kind that loses one. Engineers pay this cost more than anyone, because the vocabulary is the job.

Leading. "The POS is the source of truth, right?" She says "I suppose so", because the question told her the expected answer and she is being polite. Nothing true came back. Something false went into your notes, and it costs later, when you build a pipeline that forces the ledger to agree with the till. On the bench a leading question records a belief with a cost attached, and the fast interview records five of them.

LiveThe engineer's door is data state, and here is its bank3 min▶

An engineer's output is a pipeline other people trust. The facts that decide whether it is right are every system a number passes through, who can log in to each, which copy is declared real and how often each is read, and they live behind door 4 of the Six Doors corridor. On Adeline's sheet that door holds six facts worth 19 of 68. One sale touches the till, a bank settlement, a ledger run by an outsourced bookkeeper who exports once a month by contract, and a stock spreadsheet that exists three times, with the copy Priya treats as real sitting on her laptop. And one fact, worth 4, is priced at trust 3: Kenneth, the only IT person and the only POS admin login, is leaving in January, and she has not told the team.

The five open questions that open that door, each with the teach-back line you say out loud:

  • g08 "Which systems does one sale touch, from the till to the bank to the ledger?" Teach-back: "Every hand-off between systems is a place a number can change. I am counting the hand-offs."
  • g09 "Who can log in to each of those, and pull an export?" Teach-back: "Access is the first blocker on most data projects, so I ask early and I ask by name."
  • g10 "Do the Singapore and Malaysia outlets use the same product codes? Who decides a new one?" Teach-back: "Two names for one thing is the most common reason a total is wrong. It is a naming problem before it is a data problem."
  • g11 "Where do the stock numbers live, and which copy is the real one?" Teach-back: "A file that exists three times has no real copy until someone declares one. I need that declared before I move it anywhere."
  • g19 "How often does each system get exported today, and by whom?" Teach-back: "How often a system is read today is a ceiling I inherit until a contract or a login changes it."

The tuned order does not open with them. It opens with three quick questions from other doors: g01, "Tell me about the last time two numbers did not match", then who decides at month-end, then show me the file. They earn three points of trust and put the auditor's letter, the day-five deadline and Priya's six hours on the table before a single data-state question is asked. Then it goes through decision and the current file, and opens the data-state door widest: five of its first eight questions are door 4, and by the time g09 asks who can log in, six open questions have gone before it and the trust Kenneth's fact needs is already there.

Self-studyThe whole bank, by kind3 min▶

Thirty-seven questions: twenty open (two to six per door), four vague, five jargon, five leading, three closed. The open ones are the list you built in session 2 with Adeline's answers already attached. The others are there because they are what gets asked in real meetings, and the bench needs them to show what they cost. The five jargon questions are the ones an engineer asks another engineer: change data capture, the SLA on the ledger sync, medallion layers, how refunds are modelled, the idempotency of the export. Every one is printed with its kind, its minute cost and the facts it reaches in the bank file the page loads; the widget below draws its buttons from that file, so what you click is what it counts.

KindIn the bankMinutesWhat comes backTrust
Open203 to 5The facts behind its door, when trust is high enough for them+1
Closed31 to 2One fact at most0
Vague43 to 4A generic; at most one low-value fact0
Jargon55"Sorry, what do you mean?"-1
Leading51 to 2"I suppose so", and a false belief with a cost of 3 to 40
Real world

The login nobody asked about. A constructed case with a familiar shape. An engineer scopes a sync between a till system and a ledger, agrees a design in the first meeting, and books the build. Week three, the only person with admin rights on the till system hands in his notice, and the export the whole design rests on needs a login nobody else holds. Nobody lied. Nobody was asked, by name, who could log in; and the person who knew was not going to say it to a stranger in the first ten minutes.

Build-along · the bench

Five ways of asking, counted in front of you

★ 26 min · the main event

Work down the five presets in order. For each, read the verdict line, then the seven counters, then the transcript underneath: what you asked, what she said, what it was worth, and where trust stood. Say out loud what changed before moving to the next. The last preset is the anti-lever, the interview that looks most efficient, and the bench shows what it costs. Then the lower half of the widget gives you the chair: same person, same 45 minutes, your choice of questions, scored beside the presets.

Read it in this order. Net value, then false beliefs, then doors opened, then the "after 20 minutes" counter. Net value is what you can build on. False beliefs are what you will build wrong, and for an engineer a wrong belief about which copy is real becomes a pipeline that runs every night. Doors opened is whether you can write a brief at all. The 20-minute counter is what is in your notes if she gets called away, which happens.

Part 2 · covers what the bench finds

Same person, five answers

5 min live

The bench above computes these live; the numbers here are the ones it produces on the printed bank, written down so they can be argued with after class. Change a value in the bank file and they move. The ordering does not.

NET VALUE OUT OF 68, FIVE WAYS OF ASKING THE SAME PERSON wide bar = net value after 45 minutes · thin bar = what was in your notes after 20 · scale: 7 px per point, from x = 230 The technical opener 15 5 facts, 2 doors · after 20 min: 0 The efficient interview 17 5 false beliefs, -16 · after 20 min: 0 Any order at all 27.6 mean of 200, range 5 to 45 after 20 min: 13.8 Six Doors, in order 47 after 20 min: 20 Tuned for an engineer 51 after 20 min: 24 the ceiling, 62 an optimizer that can read her sheet; after 20 min: 34 Same person, same clock. The way of asking, not the person, decided whether the engineer left with 15 points or 51.
🔍 Click to zoom - the top two bars are the meetings most engineers have run, and they are the shortest
Way of askingNet valueAfter 20 minFactsDoorsFalse beliefs
The technical opener (five jargon questions first)150520
The efficient interview (leading and closed first)1701155
Any order at all (mean of 200 shuffles)27.613.812.35.52.5
Six Doors, door by door47201660
Six Doors, tuned for an engineer (data state widest)51241760
Ceiling: an optimizer that can read the sheet6234

All counted from the printed bank. Random is a mean over 200 seeded shuffles; its best single draw is 45 and its worst is 5.

LiveFinding 1: the framework is not optimal, it is reproducible2 min▶

Six Doors in printed order gets 47 of 68 with no false beliefs and every door opened. Tuning the order for an engineer, three trust-earning questions and then the data-state door widest, adds four points and, more usefully, puts 24 in your notes by minute 20 instead of 20. A search that is allowed to read the fact sheet finds an order worth 62, and it opens with "what happens when Priya is away" before the interviewer has heard the name Priya. Nobody in a real meeting can read the sheet. Optimal against a known answer is not the same thing as good; the framework is the thing you can do on Tuesday, blind, and get 47.

LiveFinding 2: a shuffle is not neutral, it is a spread2 min▶

Two hundred shuffles of the same bank average 27.6 and range from 5 to 45. The average is the meeting most people run when they have a good list and no order: some of the jargon lands early and costs trust, a leading question or two gets recorded, and a door stays shut, often the one with the login behind it. The spread is the argument for an order at all. A method that scores 47 every time is worth more than one that scores 45 on a good day and 5 on a bad one, and the bad days are the ones with the auditor's letter due in eight weeks.

Self-studyFinding 3: jargon costs twice, and the second cost is the one that matters2 min▶

Five technical questions in a row spend 25 of the 45 minutes and take trust to the bottom of the meter. The obvious cost is the minutes. The one that matters is what happens to the open questions that follow: the sensitive facts stay unsaid because she is no longer sure you understand her business, and the whole meeting ends with five facts and two doors, the lowest score on the bench. The technical opener is not wrong because the questions are bad; change data capture, the ledger sync and the idempotency of the export are all things you will need answered. It is wrong because they were asked of the wrong person, first. They belong to Kenneth, in week two, and before January.

Part 3 · covers the anti-lever

The efficient interview writes five wrong things down first

4 min live

Yes-or-no and leading questions, a minute or two each, eight answers in the first ten minutes. It feels like the best-run meeting in the building, and on the bench it scores 17, more than ten points below a shuffle, with nothing in your notes at minute 20. It finds 11 true facts worth 33. It also records five things she agreed to rather than things that are true, and those cost 16.

ONE LEADING QUESTION, SIX WEEKS LATER LEADING "The POS is the source of truth, right?" "I think so." month-end notes truth = POS export WEEK SIX a pipeline that forces the ledger to match the POS -4 and the auditor's question is still open WHAT WAS TRUE, ON HER SHEET the two percent is mostly settlement timing: cards land two days late, cash weekly. Refunds exist only in accounting; the POS never sees them. THE OPEN VERSION, 4 MINUTES "Where does finance's revenue number come from, step by step?" gets the settlement lag and the refunds, and earns a point of trust Neither number was wrong. They were built along two paths, and the pipeline made one of them win faster.
🔍 Click to zoom - a pipeline that forces agreement moves the disagreement faster; it does not settle which copy is real
LiveWhy "I suppose so" is worse than silence2 min▶

A closed question that gets a "no" has taught you something. A leading question almost never gets a "no", because it carries its expected answer and the person across the table is being courteous, busy, or both. Fitzpatrick's rule for customer conversations applies without change: people will say yes to be polite. What comes back is not information about her systems; it is information about how agreeable she is, filed as if it were the first thing.

The bench prices each false belief at what building on it wastes. "The POS is the source of truth" costs 4: a pipeline that overwrites the ledger, when the gap was settlement timing and refunds all along. "Pull everything daily" costs 3: month-end needs the number by day three and Monday needs it weekly, so a daily job buys nobody anything. "The vendor API is already in use" costs 3: nobody has ever touched it. "Finance's number is simply wrong" costs 3, because refunds only exist in accounting. "Stock lives in the POS" costs 3: it lives in three spreadsheets. Five leading questions in five or six minutes, and the meeting starts at minus 16 before the first true fact.

Self-studyThe honest nuance: when a closed question is right2 min▶

Closed questions are not the villain; leading ones are. "Is the close on the fifth working day?" is closed and gets a real fact for one minute, and the bench scores it so. The rule is about sequence: confirm with a closed question after an open one has shown you the shape of the answer, never before. And the anti-lever carries its own limit: a fast interview that asked only closed questions, no leading ones, would score higher than 17 and lower than the framework, because it would still learn nothing it did not walk in expecting. On this sheet the technical opener scores lower still, at 15. The bench does not pretend that speed is the sin. Assuming is.

Real world

The reconciliation that reconciled nothing. A constructed case with a familiar shape. An engineer hears "the POS is right, finance is off", nods, and builds a nightly job that adjusts the ledger to the till totals. The dashboards agree by the end of the month. Then the auditor asks the same question again, now with a new one attached: who changed the ledger, and on what authority? The disagreement had not been settled, only moved faster and hidden. The question that would have saved six weeks was the open one: where does finance's number come from, step by step.

Homework

Before session 4

30 min
  1. Run your own meeting twice in the lower half of the bench: once with your session 2 system list in your hand, once without. Write down the two scores and which doors you left shut the second time; check whether Kenneth's login came out at all.
  2. Sort your last real interview. Take the notes from a discovery or requirements meeting you actually ran and mark each question you asked with its kind. Count the leading ones and the jargon ones. Each leading question is a belief in your notes that nobody checked; each jargon one was probably asked of the wrong person.
  3. Change one number in the bank file and re-run. Double the cost of every false belief, or halve the minute cost of jargon. Write one sentence on what moved and what did not; the ordering should survive, and if it does not, you have found the assumption the bench rests on.
  4. Bring your teach-back lines for the five data-state questions. Session 4 rewrites them for a person who has never seen a schema, and picks the one concept per meeting worth explaining properly.
Check yourself

Three questions

Clear all three and session 3 gets stamped in your passport.

1 · The efficient interview uncovers 11 true facts worth 33, far more than the technical opener's five. Why does it end at 17, more than ten points below a shuffle?

The 16 points are the cost of building on "the POS is the source of truth", "pull everything daily", "the vendor API is in use", "finance is simply wrong" and "stock lives in the POS", none of which is true. Speed is not the defect. Assuming out loud is.

2 · An optimizer that can read Adeline's fact sheet finds an order worth 62, against 47 for the framework in printed order. Why teach the framework anyway?

The ceiling is printed so the framework's number can be read honestly: 47 is not the best possible meeting, it is the best reproducible one, and tuning it for an engineer takes it to 51. The 200-shuffle spread, 5 to 45, is the other half of the argument: an order is worth having because the bad days are the ones that matter.

3 · The technical opener asks five perfectly reasonable engineering questions first and ends with five facts, two doors and the lowest score on the bench. What went wrong?

Data state is the engineer's widest door, and it is opened with the plain version: which systems does one sale touch, who can log in, which copy is real, how often each is read. Change data capture and idempotency are real questions for the person who holds the login, and the bench's point is that asking them of the operations director costs twice, in minutes and in the trust the rest of the meeting is priced in.

Sources covered

What this session teaches, and where it came from

✓Every count on the benchComputed in the browser from the printed bank file, and verified headlessly in node before any page quoted a number
✓People say yes to be polite; ask about the past, not the futureRob Fitzpatrick, The Mom Test, 2013; the leading-question mechanism on the bench is his observation made countable
◐Harbourline, Adeline Tan, Kenneth, Priya, and every value on her sheetConstructed to be typical, including the settlement timing behind the two percent gap. The bench claims the ordering and the mechanism, never the numbers as facts about any real business; change the bank and re-run
◐How often real meetings run like the efficient interview or the technical openerNot claimed. No count of real discovery meetings by question kind was found, and none is asserted

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 3 cheat sheet · pin this

Five kindsOpen, closed, vague, jargon, leading. Only open earns trust; only jargon loses it; only leading records a lie.
The bench, on this sheetFast 17 · jargon 15 · shuffle 27.6 (5 to 45) · Six Doors 47 · engineer-tuned 51 · ceiling 62.
Read net, then false, then doorsWhat you can build on, what you will build wrong, whether a brief is possible.
The 20-minute counterWhat is in your notes if she is called away. Three trust-earning questions, then data state widest, puts 24 there.
Ask the login by nameWho can log in, which copy is real, how often each is read. The login is the fact that needs trust first.
Optimal is not good62 needs her sheet. 47 needs a printed page and works blind.