learn-asking-right-questions-as-ai-with-phoebe / Session 6 of 6
Learn Asking the Right Questions as an AI Developer with Phoebe · Session 6 of 6

The mock meeting and the one-page brief

Five sessions built the parts: the gap between the request and the tune behind it, one real thread walked end to end, the kinds of question and what each costs, the second sentence that teaches, and the tree that turned her pains into a decision log, a confirmation format and an assistant that only drafts. Tonight they run end to end. You take your own sheet into a full 45-minute meeting with Adeline on the bench, read your score against the ladder, and then do the thing that decides whether the meeting was worth having: write the one page she reads the next morning. The thread as she walked it, in her words. Which replies are lookups and which are decisions. What you will build first, and the line the assistant will never cross. How you will both know. Who does what next week, with every line the meeting never reached marked as a question for her. It goes out within 24 hours, or the meeting fades into "the AI person came by once". Door 3, the current way, feeds most of it; the corridor is stated in full here.

🟠 Capstone Bring the sheet, the thread map, the tree No code. A pen and the bench. 45 min
0-3 · Welcome 3-18 · The run, the brief, the gate 18-40 · Build-along: the brief she reads 40-45 · Q&A
Part 0

Where we start

A discovery meeting produces two things: what is in your notes, and what is in her head. The bench in session 3 counted the first. The brief is how the second is fixed in place before it fades, and it is also the first piece of work she sees from you. For an AI developer that matters twice over, because the last thing that arrived at Harbourline labelled AI was a vendor chatbot she switched off within a week. A brief that reads like that chatbot, confident and unchecked, is the first evidence she has that this will go the same way. A brief that hands her own thread back to her, with the one decision it will never take, is evidence of the opposite. Harbourline and Adeline Tan are constructed, as on every page; the brief you write tonight is for a real request if you have one, and for hers if you do not.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = write it yourself

★ What you walk out with tonight. One scored run of the full meeting with your own sheet, a one-page brief ready to send with its "to confirm with you" lines already written as questions, and the eight-row ship gate that says whether it goes out tomorrow morning or needs another hour first.

Part 1 · covers the full run

Your meeting, scored against the ladder

5 min live

Open the bench on session 3 in a second tab, reset the lower half, and run the whole 45 minutes with your printed sheet beside you. Do not look at the preset transcripts first. When the clock reaches zero, write down four numbers: net value, false beliefs, doors opened, and the value in your notes at minute 20. Then write a fifth thing, which is not a number: the facts on your side of the sheet that you never heard. Read the numbers against the ladder below; the list of unheard facts becomes the "to confirm with you" lines in the brief.

WHERE YOUR RUN LANDS net value out of 69 on the AI developer bank · 11 px per point, from x = 60 45 min 0 69 jargon 11 fast interview 21 shuffle 30.1 Six Doors 54 tuned 55 63, reads her sheet your run below 30 the order is costing you 30 to 54 the doors are there, the order is not 54 and above you walked the corridor Read false beliefs before net value. A run at 44 with none is a better meeting than a run at 48 with two.
🔍 Click to zoom - the card is yours to place; tuned and printed order sit one point apart on this bank, and the page says so rather than widening the gap
LiveReading your four numbers3 min▶

False beliefs first. Any number above zero means a leading question slipped through, and each one is a build decision you will get wrong in week six. On this bank the costliest is "the assistant replies to suppliers on its own": it is the vendor chatbot again, and it is wrong because only the area manager may say yes to a substitute. Find the leading question in the transcript, write the open version of it on your sheet ("In a thread, which replies need a person to decide, and which are just looking something up?"), and carry that version into the real meeting.

Doors opened. Six is the only answer that lets you write a brief without guessing. A shut door is a block of the brief you cannot fill. For an AI developer the door that matters most is door 3: if you never walked one thread end to end, block 1 has no workflow in it and block 3 cannot say where each half of the thread lives.

Net value against the ladder. The Six Doors in printed order score 54 blind; tuned for an AI developer, 55. That one point is honest and small. The printed order already asks the lookups-versus-decisions question, a decision-door question that does the current-way work, by minute 12, and it hears the one-in-twenty error rate at minute 16. A run in that region means the sheet works. A run near the shuffle's mean of 30.1 means the questions are good and the order is not: usually the thread was never walked, or jargon crept in early and cost trust the rest of the meeting needed.

The 20-minute counter. The tuned order puts 23 points in your notes by minute 20, printed order 22, a shuffle about 14.3 on average. If she is called away halfway, that is the meeting you actually had. The tuned order's 23 already holds the thread, the error rate, the lookup-or-decision line and the worst case, which is most of a block 1 and a block 2.

Self-studyWhat the run did not hear, and what the bench cannot see2 min▶

The tuned run asks eleven questions in 45 minutes, opens all six doors, and still leaves six facts on her side unsaid: that the board wants the word AI on a slide; who holds the POS back-office login, the most sensitive fact on the bank, which needs the most trust before she says it; that customer names sometimes appear in the threads; that anything the managers use has to live inside WhatsApp or on the phone, in the van; that the board meets in six weeks and the supplier contracts renew in three months; and that Farid will trial anything and Priya in finance can give two hours a week. None of that is a failure of the sheet. It is the shape of 45 minutes. What matters is what you do with it: every one of those is a line the brief marks "to confirm with you" or a question in the covering note, and never a sentence written as if she had said it.

The bench scores what you asked and what came back. It does not hear your tone, see whether you looked at her or at your laptop, or notice the moment she leaned forward because a question was about her managers' actual Tuesday. Those matter, and no widget measures them honestly. The run is a rehearsal for the order and the kinds, not a verdict on you as an interviewer. If you can, run the real meeting with a colleague taking notes, and ask them one thing afterwards: when did she start giving longer answers?

Real world

The rehearsal that found the leading question. A constructed case. An AI developer ran the bench twice before a real discovery meeting about a messaging assistant. The first run had one false belief: "So the assistant should reply to the suppliers automatically?" had slipped onto the sheet as a confirmation meant for the end. The developer rewrote it as "Tell me about the last substitution you approved. How did you decide?" and moved it up behind the walk through one thread. In the real meeting that open question produced the rule that only the manager may accept a substitute, and the brief sent the next morning carried, in its fourth block, the line the first draft of the sheet would have got exactly backwards.

Part 2 · covers the one-page brief

One page, in her words, sent within 24 hours

6 min live

The brief is not minutes and it is not a proposal. It is the meeting handed back to her in a form she can check, correct and forward to the board without rewriting. It has six blocks, each fed by a door, and every sentence in it is either something she said, something you will do, or a question marked as one. For an AI developer the block that carries the weight is block 4. It is where "Can AI answer the supplier WhatsApps?" comes back to her as a decision log first, and as an assistant that drafts and never approves a substitute. If she reads that and says "yes, that is what I meant", the gap from session 1 is closed. If she corrects it, the brief was worth sending for that alone.

THE BRIEF: SIX BLOCKS, EACH FED BY A DOOR 1 2 3 4 5 6 outcome, current way decision current way, data state decision, constraints success partnership What you told me, in your words "My three area managers spend half their day on them." Which replies are lookups, which are decisions only the area manager says yes to a substitute · 1 in 20 wrong Where each half of the thread lives supplier chat, outlet group, no order table the map WHAT I WILL BUILD FIRST, AND WHAT I NEVER WILL a decision log first · never approves a substitute How we will both know two outlet visits a day, in the visit log you read Who does what next week the log starts Monday · who trials it: to confirm with you within 24 hours read on her phone
🔍 Click to zoom - block 4 is plum because it is the one this role adds: where the request comes back as a log first and a line the assistant never crosses
LiveThe six blocks, and the rule for each4 min▶

1 · What you told me, in your words. Two or three sentences she said, quoted, and at least one of them is the thread as she walked you through it. Not your summary. "My three area managers spend half their day on them" is her sentence; "high manual message-handling load" is yours, and she cannot correct a tune she never hears. This is the block that closes the gap from session 1: if what you heard is not what she meant, she says so here, and it costs one email instead of six weeks.

2 · Which replies are lookups, and which are decisions. Session 4's one concept, written as a fact about her shop. Inside a thread one kind of reply needs a person, accepting a substitute or a price change, and only the area manager may say yes to it; everything else is looking something up. Put the error rate today beside it: about one confirmation in twenty carries a wrong price or a substitute, and the managers catch most of them. That number is the bar any assistant has to beat, and how they catch it is the part the assistant has to keep.

3 · Where each half of the thread lives. The thread map from session 2, as a thumbnail or six lines of text. A thread has two halves and they live in different places: the supplier's confirmation in a chat on a manager's personal phone, the outlet's original order in another WhatsApp group, in three languages with nicknames. End it with one question: "which of these is wrong?"

4 · What I will build first, and what I never will. The pilot from session 5 in three short lines, in its order: the decision log first, the confirmation format at the next contract renewal, and only then an assistant that drafts the plain-OK replies for one manager and four suppliers, shows the match it made, and never approves a substitute or a price change. Then one sentence naming what it will not do. "The assistant will never approve a substitute" is the line that most needs her agreement, because it is her door 2 answer written as a specification, and because she has already switched off one system that answered on its own.

5 · How we will both know. Her test, in her words: the area managers visit two outlets a day, and she reads the visit log herself. Not an accuracy figure. She told you she does not care what the assistant's accuracy number is, and a brief that leads with one tells her you were not listening.

6 · Who does what next week. Names, a day, a time, one task each. Rob Fitzpatrick's test in The Mom Test (2013, reported from summaries) for whether a conversation went anywhere is whether the other side committed something: time, a name, an introduction. This block is where her commitment is written down so it cannot quietly lapse, and where anything the meeting never reached is marked "to confirm with you" instead of being filled in.

LiveWhy 24 hours, and why one page2 min▶

Within a day she still remembers what she meant, so a correction is cheap and specific. After a week she remembers that you came by, and a brief arriving then reads as a proposal to be evaluated rather than a record to be checked. No study is quoted for the 24 hours; it is a working norm, and the reason is the one above.

One page because the people who act on it read on their phones: her managers are in the van, and she reads between outlets. Two pages is a document; a document goes into a folder to be read later; later is after the board. If a block needs more than three lines, the detail goes in an attachment she can ignore.

Self-studyWhat the brief must never contain2 min▶

A word from the technical opener: fine-tune, retrieval, hallucination rate, eval set, vector store, guardrails. An accuracy figure for an assistant that does not exist yet. A customer's name, or a quoted message that carries one: a patient name in a message makes it personal data under Malaysia's PDPA, and the head office lawyer takes four weeks to answer anything, so the brief says "no message with a customer's name in it goes into phase one" and shows none. A sentence she never said, written as if she had; the six facts the run did not reach are questions, not statements.

And a fact she told you only once trust was high. If your run reached the question about who can log in to the POS back office, and she told you what she has not yet told her team, that fact shapes block 4, because the step that needs the product list must not depend on one login, but it goes into your plan and not into a document she may forward. The brief is hers to forward. Write it as if it will be.

Real world

The brief that led with a percentage. A constructed case, with the shape that recurs. An AI developer's first draft opened block 4 with "the assistant will handle 95 percent of supplier confirmations", a constructed figure nobody had measured. The operations director replied with one line: what happens to the other five? That was the right question, and the brief should have answered it before she asked. The second draft opened block 4 with the decision log, put the drafting assistant third, and said in plain words that any confirmation with a substitute or a price change goes to the manager, with the difference shown. She forwarded that version to the board unchanged.

Part 3 · the final scorecard

The ship gate for a discovery brief

4 min live

Eight rows, each scored 0, 1 or 2 against your own brief. This gate is strict on purpose: a brief that half-separates lookups from decisions is read by the business as a brief that has separated them, and the gap it leaves opens again the first time a draft says yes to something it should have handed over. Five rows are the AI developer's own: the thread in her words, lookups and decisions apart, today's error rate stated, no patient data in phase one, nothing that replies on its own. Three are the rows every discovery brief carries: no belief from a leading question, the read-aloud test, names and days. Anything under 16 of 16 gets another hour before it goes out.

THE GATE: EIGHT ROWS, 0 TO 2 EACH, 16 TO SEND the thread, quoted block 1 · sessions 1, 2 lookups, decisions block 2 · session 4 error rate today block 2 · session 3 no patient data block 4 · session 5 never sends alone block 4 · session 5 no leading guess every block · session 3 read-aloud test every line · session 4 names and days block 6 · session 5 16 of 16? under 16 16 another hour back to the session it names send it within 24 hours A row at 1 is read as a 2 by the person who acts on it. One partial row holds the whole brief back.
🔍 Click to zoom - the top row holds four of the five rows an AI developer adds; every slot names the session to go back to
Block 1 quotes her own words, and at least one quote is the thread as she walked me through it, message by message. None of it is my summary. Sessions 1 and 2. The workflow as it happens today is the spec; a summary is you tapping the tune you heard, and she cannot correct what she never sees.
Block 2 separates the lookups from the decisions, and names who may make each decision: only the area manager may accept a substitute or a price change. Session 4. Assistants do lookups, people keep decisions, and the line between them is hers to draw; if the brief blurs it, the build will too.
Block 2 states today's error rate, about one confirmation in twenty, and how the managers catch it, as the bar any assistant has to beat. Session 3. The one-in-twenty rate sits behind door 3 and a little trust; without it there is no bar, only a demo that looks good.
Block 4 says no message with a customer's name in it goes into phase one, and the brief itself carries no customer name and no quoted message that holds one. Session 5. A patient name makes a message personal data under Malaysia's PDPA, and a lawyer's four weeks would eat the pilot before it starts.
Nothing in block 4 replies on its own: the assistant drafts, shows the match it made, never approves a substitute or a price change, and the brief flags that line as the one I need her to agree with. Session 5. She switched off the vendor's chatbot within a week because it answered on its own; a brief that promises another one ends the project at block 4.
Nothing in my notes, and nothing in the brief, came from a leading question I did not later confirm with an open one. Session 3. On the bench the efficient interview recorded five beliefs that were wrong and scored 21, below a shuffle's 30.1 and far below the doors' 54.
Every sentence passes the read-aloud test: no word she would have to ask about. Session 4. The technical opener scored 11 on the bench, because jargon costs minutes and trust; in a brief it costs the correction she never makes because she did not understand the line.
Block 6 has names, a day, a time and one task each, and every line the meeting never reached is marked "to confirm with you" instead of being filled in. Session 5. The tuned run never heard who would trial it, who could give two hours a week, or when the contracts renew; a partner who was never asked is not a partner.
Real world

The row that held the brief back an hour. A constructed case. An AI developer scored a brief at 15 of 16. The row at 1 was block 6: it said "Priya in finance reviews the log every Friday", copied from a pilot sheet where it had been a guess. Nobody had asked Priya, or asked Adeline whether Priya could be asked. The developer rewrote the line as "to confirm with you: who on your side could give two hours a week to check the log?" and sent the brief an hour later. The reply named Farid for the trial and Priya for the review, and added that she had a quarter-end close for the next two weeks. The log started with Farid alone, and the review started in week three. The guessed line would have booked a person into a fortnight she did not have.

Build-along

Write the brief she reads tomorrow morning

★ 22 min · everyone writes

Six blocks, one page, built from the three artifacts you already have: the Six Doors sheet from session 1, the thread map from session 2, the tree, the three candidates and the pilot from session 5. Nothing new is invented tonight. Every line is something she said, something you will do, or a question marked "to confirm with you". The worked example below is Adeline's, written from her fact sheet and limited to what the tuned run actually heard; the facts it never reached are written as questions, exactly as you would have to write them.

THREE THINGS YOU ALREADY HAVE, ONE PAGE SHE READS Six Doors sheet session 1 · blocks 1, 2, 5 The thread map session 2 · block 3 Tree, three, pilot session 5 · blocks 4, 6 THE BRIEF 1 in your words 2 lookups, decisions · 3 halves 4 the log first, never a yes 5 how we know · 6 next week one page, her phone Nothing new is invented tonight. What the meeting never reached goes in as a question, marked as one.
🔍 Click to zoom - if a block has no arrow feeding it, a door was left shut and the brief should say so

Block 1, her words, and the thread. Open your sheet and copy two or three of her sentences exactly: the request as she said it, the thread as she walked it, and the outcome behind it. If your sheet has the request and no walk through a thread, door 3 never opened, and the brief should ask for one thread rather than describe one.

★ Step 1 · what you told meWhat you told me, in your words: "Can AI answer the supplier WhatsApps? My three area managers spend half their day on them." "The supplier sends a confirmation with the items and prices. The manager checks it against what the outlet asked for, which is in a different WhatsApp group, and replies OK or queries it." "Head office expects two outlet visits a day. They manage one, because of the phone."

Blocks 2 and 3, the line and the two halves. Block 2 draws the line between lookups and decisions and states today's error rate. Block 3 says where each half of the thread lives, one line each, ending with the question.

★ Step 2 · lookups, decisions, and where the halves liveWhich replies are lookups, and which are decisions: - Decision: accepting a substitute or a price change. Only the area manager may say yes. A wrong yes is a compliance problem: pharmacy products need a substitution record by law. - Lookups: everything else. Checking items and prices against the outlet's order; passing on a delivery time. - Today: about forty threads a day per manager, about four in five end in a plain OK. About one confirmation in twenty has a wrong price or a substitute in it; your managers catch most of them. Where each half of the thread lives: - The supplier's half: a WhatsApp chat on the manager's personal phone, no business account - The outlet's half: another WhatsApp group, typed by the outlet manager, in English, Malay and Chinese, with nicknames - The product list: the POS; the codes differ between Singapore and Malaysia, and the nicknames are not in it - What was decided: only in the thread; when a manager left in March, her phone went with her - An order table: there is none; a chat can be exported one at a time, by hand - The suppliers: 14 in Singapore, 22 in Malaysia, each writing its confirmation differently Which of these is wrong?

Block 4, first, and never. The three candidates from session 5 in their order, one line each, then the never line on its own, flagged as the one you need her to agree with. The first step needs no budget this quarter and no model. Anything the run never heard, such as when the supplier contracts renew or whether it must live inside WhatsApp, goes in as a question.

★ Step 3 · what I will build first, and what it will never doWhat I will build first: 1. A decision log: one shared sheet, one line for every substitute or price change a manager accepts (supplier, item, what changed, who said yes). It needs no budget and no model. 2. At the next supplier contract renewal: a clause asking every supplier for one confirmation format, items, quantities and prices as a numbered list. (To confirm with you: when do the contracts renew?) 3. From Q1, when the SGD 15,000 tooling budget opens: an assistant that drafts the plain-OK replies for one manager and four suppliers, shows the match it made, and hands every mismatch to the manager with the difference shown. (To confirm with you: does it have to live inside WhatsApp?) What it will never do: The assistant will never approve a substitute or a price change, and it will never send a reply on its own. This is the line I most need you to agree with. Not in phase one: No message with a customer's name in it.

Blocks 5 and 6, how we know and who does what. Her test in her words. Then names, a day, a time and one task each, and a "to confirm with you" line for every person or date the meeting never reached. Do not name a person she never named.

★ Step 4 · the test and next weekHow we will both know: Your area managers visit two outlets a day, and you read it in the visit log yourself. Not the assistant's accuracy number; you told me that is not what you watch. Who does what next week: - Me, Monday 10:00: the decision-log sheet set up, and ten minutes walking one manager through it - To confirm with you: which manager trials it first, and with which four suppliers (you said one manager and four suppliers first, never all 36) - To confirm with you: who on your side could give two hours a week to look over the log - To confirm with you: when you next need to show the board something

Read it on your phone, then score it. Send the draft to yourself, read it on the phone standing up, and run it through the eight-row gate above. Anything under 16 gets fixed before it is sent. Then send it, within 24 hours of the meeting, with a covering note that points at the two blocks you most want corrected and counts the questions you still owe her.

★ Step 5 · the covering noteSubject: What I heard on Thursday, and the first step Hi Adeline, here is what I understood from our meeting, on one page. Two parts I most want you to check: the first block (is that how a thread goes?) and the never line in block 4 (do you agree with it?). Five things I did not get to ask are marked "to confirm with you". One more for when we next speak: who can log in to the POS back office and pull the product list?

The reply is the second meeting. Whatever she corrects is the most valuable fact you will get this month, because it is the one you would otherwise have built wrong. Her answers to the "to confirm with you" lines are the rest of door 6 arriving in writing. Put each one straight into the sheet, the thread map or the pilot, and send a one-line thank-you that says what changed.

Homework

After the last session

1 week
  1. Run the real meeting. Take your sheet to the real request, walk one thread end to end before anything else, and send the brief within 24 hours. Score it on the gate before it goes.
  2. Log the reply. Write down every correction she made, and which block it was in. Block 1 corrections mean you heard the request and missed the thread; block 2 corrections mean you drew the line between lookups and decisions in the wrong place, which is the most expensive correction to get after a build.
  3. Re-run the bench with the question you would now ask first, and compare the score and the 20-minute counter with your session 3 run. Then compare the list of unheard facts: the shorter it is, the fewer "to confirm with you" lines you have to send.
  4. Teach one teach-back line to a colleague who builds with models, and ask them to use it in their next meeting. "I split every workflow into lookups and decisions" is a good first one. A framework is only a framework once two people use it the same way.
Check yourself

Three questions

Clear all three and session 6 gets stamped in your passport.

1 · Your run on the bench scores 48 with two false beliefs. A colleague's scores 44 with none. Whose meeting was better, and what do you do next?

Read false beliefs before net value. The efficient interview records 37 points of facts on this bank, and its five false beliefs take it to 21, below a shuffle's mean of 30.1. Two false beliefs in your run are two blocks of the brief that will be confidently wrong, and one of them may be "the assistant replies on its own".

2 · Why does block 4 carry "the assistant will never approve a substitute" on its own line, flagged as the one you most need her to agree with?

The line is not a caveat about the model. It is the decision a person keeps, and it would stay a decision however accurate a model became. Flagging it invites the one correction that would change the whole build.

3 · The tuned run on the bench never reached who would trial the pilot or who could give two hours a week. What does block 6 say about them?

A guessed partner reads as an agreed one. A question she can answer in one line turns the gap in your 45 minutes into her commitment, which is the thing Fitzpatrick's test says a good conversation produces.

Sources covered

What this session teaches, and where it came from

✓The preset scores the run is read againstCounted by the bench on session 3 from the printed bank file and verified headlessly: technical opener 11, efficient interview 21, shuffle 30.1 (after 20 minutes 14.3), Six Doors 54, tuned 55, ceiling 63, of 69
✓What the tuned run did and did not hearRead from the tuned preset's transcript on the same bank: eleven questions, six doors, six facts unsaid, which become the "to confirm with you" lines
✓A conversation went somewhere if the other side committed somethingRob Fitzpatrick, The Mom Test, self-published, 2013, reported (secondary, consistent across summaries); block 6 is that test written down
◐The 24-hour norm and the one-page ruleWorking norms, not research findings. No study is quoted for either, and the reasons given are the ones on the page
◐Harbourline, Adeline Tan, and the worked briefConstructed. The worked brief is written from her printed fact sheet; the 95 percent figure in the Part 2 story is constructed too. The bench claims the ordering and the mechanism, never its numbers as facts about a real business

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 6 cheat sheet · pin this

Read your run in orderFalse beliefs, doors, net value, the 20-minute counter, then the facts you never heard.
Six blocksHer words · lookups and decisions · the two halves · first and never · how we know · who does what.
Block 1 holds the threadHer words, message by message. Not your summary; it is the block she corrects if you misheard.
The never line"The assistant will never approve a substitute" is the line that most needs her agreement.
Unheard is a questionEvery fact the meeting never reached goes in as "to confirm with you", never as a guess.
16 of 16, or another hourA half-drawn line between lookups and decisions is read as a drawn one by the person who acts on it.