learn-asking-right-questions-as-ai-with-phoebe / Session 4 of 6
Learn Asking the Right Questions as an AI Developer with Phoebe · Session 4 of 6

Teaching while you ask

Every open question on your sheet has a second sentence: why you are asking, said out loud. Session 1 showed what it buys, a point of trust and one true thing learned. Tonight is about the sentence itself, because the version most AI developers write is true and useless. "Retrieval grounds the model, so hallucination drops" is a correct principle that teaches Adeline nothing, since three of its words are words she would have to ask about, and she will not ask. She will nod. The work of this session is making twelve second sentences plain enough that a person who runs pharmacies learns one real thing from each, without anyone giving a lecture.

🟡 Core AI developers, and anyone asked to "add AI" No code. A pen. 45 min
0-3 · Welcome 3-18 · The curse, one concept, lesson not lecture 18-40 · Build-along: twelve lines, made plain 40-45 · Q&A
Part 0

Where we start

You have a sheet of open questions with a teach-back line beside each, and session 3 put it on the bench. The questions are fine. The second sentences are the risk, because you wrote them in the vocabulary you build in, and you cannot hear that from the inside. Adeline Tan, operations director of Harbourline, a constructed chain of 38 pharmacy and clinic outlets, is the reader tonight. She asked "Can AI answer the supplier WhatsApps?", and the door that matters most for that request is the current way, door 3 of the corridor stated in full here. The aim tonight is narrow: by the end of a 45-minute meeting she can repeat one idea about AI work to her board in her own words, and the other lines have quietly landed as single sentences she did not have to stop and decode. The Communication course owns presenting and influence; this is only the sentence you say before a question.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = write it yourself

★ What you walk out with tonight. Twelve teach-back lines with every word she would have to ask about replaced, each still one sentence; one concept chosen for the meeting, "the assistant drafts the lookups and a person keeps the decisions", with its ninety-second version written in three plain sentences; and the sentence you hope she says to her board. Session 5 takes the same sheet and turns her answers into three projects, one of which is an assistant that only drafts.

Part 1 · covers why you cannot hear your own jargon

Your curse of knowledge, not hers

5 min live

In 1989 Camerer, Loewenstein and Weber showed that better-informed people cannot fully set their information aside when predicting what less-informed people will understand, and named it the curse of knowledge. Tech Writing session 1 teaches it in depth, with what it does to documentation. The AI developer's version is smaller and sharper: a teach-back line written in the words of the build. "Retrieval grounds the model, so hallucination drops" is tapping a tune. The developer hears the whole song, the vector store, the context window, the eval that showed it, and judges the line clear from inside that experience. Adeline hears three knocks and one word she has read in the newspaper.

ONE LINE, DRAWN TWICE the claim does not change; the three words she would have to ask about do BEFORE, IN THE DEVELOPER'S WORDS three words she would ask about; she nods instead Retrieval grounds the model, so the hallucination rate drops. "sorry, what do you mean?" AFTER, IN HERS if it looks in your product list first, before it writes a reply, it makes up fewer answers. same claim, no word to ask about, still one sentence; she learns the product list is part of the build The circled words were invisible to the person who wrote the line. Reading it aloud to someone else is the only way to see them.
🔍 Click to zoom - the top line is correct and the bottom line is the same line; only the bottom one teaches her anything
LiveThe words she would have to ask about3 min▶

Take any teach-back line on your sheet and read it aloud to someone who does not build software. Any word that makes them say "sorry, what do you mean?" is a jargon cost, and session 3's bench prices it exactly: a jargon question takes five minutes, because you have to explain, and it costs a point of trust, because she is a little less sure you understand her shop. The line was meant to earn trust. With one wrong word it spends it.

The usual offenders for an AI developer, with the plain words that replace each. Model: the part that reads a message and writes a reply. Prompt: the written instructions it is given before every message. Fine-tune: teaching it from your own past threads so it writes the way your managers do. RAG (retrieval-augmented generation): letting it look things up in your product list before it answers. Embedding: a way of finding two messages that mean the same thing even when the words differ, like two nicknames for one product. Hallucination: when it writes something confident that is not true. Eval: a set of real past threads with the right replies, kept to test it against. Guardrail: a rule it cannot break, such as never saying yes to a substitute. None of the plain versions is longer than a breath, and every one of them is a thing she can picture in a WhatsApp thread.

The test is not "would a smart person understand this". Adeline is a smart person, and she has read about AI in the papers, which makes it worse: "hallucination" sounds familiar enough to nod at and means nothing precise to her. The test is whether the word is one she uses at work, and the honest way to find out is her face, which is why the reading is done aloud and never on paper.

Self-studyWhy plain is not dumbed down2 min▶

Douglas Hubbard's definition of measurement, a quantitatively expressed reduction of uncertainty based on one or more observations, is itself a plain sentence. It contains no term of art, and an operations director can check it against her own experience without a glossary. Precision and plainness are not in tension; the words that sound precise to a developer are usually just familiar.

There is a professional reason plain is the norm and not a courtesy, and for an AI developer it is sharper than for most. The person who signs off an assistant has to be able to check what it will do. A teach-back line she cannot decode is a claim she cannot test, and a claim she cannot test about something that talks to 36 suppliers is one she either accepts on faith or quietly vetoes. Adeline has already switched off one thing that replied on its own, the POS vendor's chatbot, which answered a customer's prescription question wrongly last year; she switched it off within a week. Plain words are how you make your reasoning inspectable by the person who remembers that week.

Real world

The nod that cost six weeks. A constructed case, and a common shape. A developer told Adeline "we will put a human in the loop for low-confidence intents", watched her nod, and moved on. Six weeks later, reviewing the first demo, she asked why the assistant had replied OK to a supplier's substitute on its own. To her, "low-confidence" had meant the messages the assistant found hard to read, and a clearly written substitute is easy to read. To the developer it had meant something else again. Nobody lied and nobody was careless. A phrase the developer could not hear as jargon went past a person too polite to stop the meeting, and the belief it left behind, that the assistant could say yes when the message was clear, was not visible until a demo had been built on top of it.

Part 2 · covers the one idea that gets explained properly

One concept per meeting

5 min live

Twelve teach-back lines, and she will remember one. That is not a failure of the other eleven; single sentences land as background, and background is what most of them should be. But one idea per meeting deserves ninety seconds of proper explanation: what it is, why it matters, what it means for a real person's morning. The choice is not the idea you find most interesting. It is the idea a false belief would attach to if you left it unexplained, because for an AI developer that belief does not stay in the notes. It ships as behaviour, in a thread with a supplier.

TWELVE LINES, ONE PULLED OUT eleven stay single sentences; one gets ninety seconds, chosen by the false belief it prevents THE ONE: THE ASSISTANT DRAFTS, A PERSON KEEPS THE DECISIONS What it is: it drafts the lookups; a person reads every draft and presses send. Why: a yes to a substitute is the manager's alone, and the law wants a record. Farid's morning: the plain OKs arrive drafted; the one in twenty arrives flagged. ninety seconds, three sentences, and "it replies on its own" never forms Farid 1 2 3 4 5 6 7 8 9 10 11 12 pulled 1-2 outcome · 3-4 decision · 5-6 current way · 7-8 data state · 9-10 constraints · 11-12 success
🔍 Click to zoom - the pulled card is the only one that gets three sentences, and it is chosen by the belief it prevents
LiveChoosing the one2 min▶

Look at the five false beliefs the bench records on the fast run, because each one is a concept that was never explained. "The assistant replies to suppliers on its own" attaches to drafts versus decisions. "The orders are in English" attaches to what the assistant has to read: English, Malay and Chinese, with product nicknames. "There is an order table to match against" attaches to where the other half of the match lives, which is another WhatsApp group. "Mistakes are rare and cheap" attaches to the error rate today as the bar. "The managers will use a separate app" attaches to the tool living where the people already are. Pick the belief most likely to form in this meeting with this person, and the concept behind it is your one.

For Harbourline there are two honest candidates. The assistant drafts the lookups and a person keeps the decisions, because "so the assistant should reply to the suppliers automatically?" is the belief waiting to happen the moment anyone says the word AI, and on the bench it is the first question of the fast run, recorded in minute one at a cost of 4, the highest in the bank alongside the order table. Or the other half of the match lives somewhere else, because she will assume the assistant can check a confirmation against an order, and there is no order table anywhere, only another group typed in three languages. Either is right. Choose one, not both; two concepts in ninety seconds each is a lecture with an interval. For an AI developer facing this request, the first wins, because it is the one that decides whether anything ever replies on its own.

LiveThe ninety-second version2 min▶

Three sentences, in a fixed order: what it is, why it matters, what it means for a real morning. Written in full for this concept, as it would be said after "in a thread, which replies need a person to decide, and which are just looking something up?" and her answer that only the area manager may say yes to a substitute or a price change:

What it is. "An assistant can do the looking-up half of a thread, checking the supplier's confirmation against what the outlet asked for and writing the reply, but it only ever drafts; a person reads the draft and presses send, every time." Why a substitution is a decision. "Saying yes to a substitute product or a price change is not a lookup, it is a choice only the area manager may make, because a pharmacy product needs a substitution record by law and a wrong drug on a shelf is not a customer-service problem." Farid's morning. "Of his forty or so threads a day, about four in five end in a plain OK, so those arrive as drafts he checks and sends, and the one confirmation in twenty with a wrong price or a substitute in it arrives in front of him marked, with the match shown, so his attention goes where the decisions are."

Ninety seconds, then back to the next question. The tell that it worked is that she uses the word "draft" later in the meeting on her own, as in "and the drafts would still come to Farid?", rather than asking whether it can be switched to automatic.

Self-studyThe test that she could repeat it to her board2 min▶

The board meets in six weeks and wants the word AI on a slide. When a director asks Adeline what the AI developer said, if the one concept landed she can say something like "the assistant writes the easy replies, and a manager still says yes or no to every substitute, so nothing reaches a supplier without a person." That sentence is the whole test: it is hers, it is short, it carries the reason, and it would survive the sceptical follow-up she has already asked herself, the one about the chatbot she switched off.

Write the sentence you hope she says before the meeting. It forces the ninety seconds to be built backwards from a line a person who runs pharmacies could carry into a boardroom, and it is the cheapest rehearsal there is. It also tells you something about the choice itself: if you cannot write a one-line version a board member would repeat, you have picked an idea that matters to you and not to her.

Real world

The wrong one. Constructed. A developer chose "the product list is the assistant's dictionary" as the concept for Adeline's meeting, explained it well, and she repeated it perfectly. Two weeks later the first demo sent a plain OK to a supplier on its own, and the confirmation it had approved carried a substitute on its seventh line. Nobody had said out loud that every reply is a draft, and "so it would just reply to them?" had gone unchallenged in the first minute. The concept she learned was true. It was the wrong one for that meeting, because the belief waiting to form was about who presses send, not about what the assistant can look up.

Part 3 · covers the difference between teaching and talking

Lesson, not lecture

4 min live

A teach-back line passes three tests: plain, true, one sentence. The third is the one developers fail, because the moment a line touches how the thing would be built it wants a second sentence about the architecture, and then a third about the trade-off. The tell of a lecture is easy to hear from across the room: she stops answering and starts nodding. Rob Fitzpatrick's third rule for talking to customers, talk less, is the whole discipline, and it applies with extra force to the person who is supposedly asking the questions.

TWO MEETINGS, 45 MINUTES EACH hatched is the developer talking; white is her talking; plum is a teach-back line · drawn, not measured LECTURE she stops answering and starts nodding; the gaps are her saying "right" 0 min 15 30 45 LESSON she talks; each tick is one plain sentence before a question, and the white stretches are the facts arriving 0 min 15 30 45 Both rails are constructed illustrations of a shape, not recordings of a meeting. Teaching happens in the ticks.
🔍 Click to zoom - constructed, not measured; the lesson rail is mostly white because most of a good meeting is her talking
LivePlain, true, one sentence2 min▶

Failing, before "who can log in to the POS back office and pull the product list?": "Credential provisioning is usually the critical path on AI projects, because if the POS admin account is a single point of failure we would want a service account and scoped API access before any ingestion work starts." True, and it fails all three tests at once: half a dozen words she would ask about, two claims joined into one, and a second sentence hiding inside the first.

Passing, same question, the line from the bank: "Access is the first blocker on most projects, so I ask early and I ask by name." Plain, because every word is one she uses; true, because it is a real principle of how these projects stall; one sentence, so she is answering again within three seconds. The failing version would have earned a nod. The passing version, asked after a run of open questions had built trust, earns the fact behind the highest trust gate on her sheet: Kenneth is the only IT person, holds the POS admin login, and is leaving in January, and she has not told the team.

Self-studyWhen she asks a follow-up, you have a lesson2 min▶

There is only one proof that a line taught something, and it is a follow-up question from her. "So who presses send?" means drafts versus decisions landed. "Wait, so it would have to read the outlets' group as well?" means the other half of the match landed. A nod proves nothing, and a polite "makes sense" proves slightly less than nothing, because it closes the topic. Silence after a teach-back line is a reason to ask your next question, not to add a sentence.

On the bench this is the mechanism, not a metaphor. Every open question earns a point of trust, and eight of the facts on her sheet sit behind trust gates she will not cross unprompted: seven need one point, and the Kenneth fact needs three. The teach-back line is where the point is earned: she answers the next question more fully because the last one came with a reason. A lecture spends minutes and earns nothing, which is the same arithmetic as the technical opener in session 3 with a friendlier face.

Real world

The follow-up that changed the build. Constructed. After "if the record is a phone, the first thing an assistant should produce is a log, before it produces a single answer", Adeline asked: "so when the area manager left in March and took her phone, every substitute she ever approved went with her?" That question was the lesson landing and the project changing shape in the same breath. It moved the first piece of work from an assistant that writes replies to a shared record of every accepted substitute and price change, which is where session 5 will find it.

Build-along

Make your twelve lines plain for Adeline

★ 22 min · everyone writes

The sheet from session 1, teach-back column only. Twelve lines go in as you wrote them and come out with every word she would have to ask about replaced, still one sentence each. Then one is chosen and given its ninety seconds. If you did not bring a sheet, the twelve lines from the bank are printed in step 1, two per door in the printed order, and they need the same work; several were written by a developer too. For scale: on the bench, the tuned order gets through eleven open questions before the 45 minutes run out, so twelve lines is a full meeting's worth.

THE SHEET AGAIN, TEACH-BACK COLUMN WIDE request, in her words: "Can AI answer the supplier WhatsApps?" LINE, DOOR THE QUESTION TEACH-BACK, BEFORE AND AFTER 3 Decision which replies need a person to decide? the model handles lookup intents; the rest go to HITL the assistant drafts the lookups, and a person keeps every yes to a substitute 5 Current way show me one thread from yesterday the as-is workflow is the spec for the automation the way you do it today is the plan, because I cannot copy a step I have not watched a person do 8 Data state personal phones, or a business account? no WhatsApp Business API means no ingestion path where the messages live decides whether the assistant can even read them the one concept, line 3 Struck lines are true. They are struck because she would have had to ask about them, and she would not have asked.
🔍 Click to zoom - the column that was narrowest on the session 1 sheet is the one that outlives the meeting

Paste your twelve teach-back lines as they are. No editing yet. If you have no sheet, these twelve are the bank's, two per door in the printed order, quoted exactly, and they are the raw material for the rest of the steps.

★ Step 1 · the twelve, as written1 A chatbot is a tool. I need the outcome, because I can check a tool against an outcome and not the other way round. 2 Whatever they are measured on is the number the assistant should move. Everything else is nice to have. 3 I split every workflow into lookups and decisions. Assistants do lookups, people keep decisions, and the line between them is yours to draw. 4 The worst case sets how much checking the assistant needs. A wrong delivery time and a wrong drug are not the same mistake. 5 The workflow as it happens today is the spec. I cannot automate a step I have not watched a person do. 6 The error rate today is the bar any assistant has to beat, and how you catch errors is the part the assistant has to keep. 7 An assistant can only match what it can look up. The product list is its dictionary. 8 Where the messages live decides whether an assistant can even read them. That is a setup question, not a model question. 9 I ask for the limits first so I do not propose something you could never approve. 10 A tool outside the app people already use is a tool nobody opens. The device and the app decide the design. 11 If we agree on the test now, neither of us argues about it later. A visit log beats a satisfaction score. 12 The person who trials it is the person who will defend it. I pick them before I pick a model.

Circle every word she would have to ask about. Read each line aloud to the person next to you. Mark the word at the moment their face changes, not the word you suspect. In the bank's twelve, look hard at "workflow", "spec", "setup", "model" and "the device": each is borderline, and borderline means circle it.

★ Step 2 · the circles3 I split every [workflow] into lookups and decisions ... 5 The [workflow] as it happens today is the [spec]. I cannot automate a step ... 8 ... That is a [setup] question, not a [model] question. 10 ... [The device] and the app decide the design. 12 ... I pick them before I pick a [model]. Your own sheet will have more. "Prompt", "fine-tune", "RAG", "eval" and "guardrail" are the usual five.

Replace each circled word, keeping one sentence. The replacement is the plain thing the word stands for, and it is allowed to be longer than the word it replaces. It is not allowed to become a second sentence.

★ Step 3 · the replacements5 The way you do it today is the plan, because I cannot copy a step I have not watched a person do. 8 Where the messages live decides whether the assistant can even read them, and that is a question about your phones, not about the AI. 10 A tool outside WhatsApp is a tool nobody opens, so the phone and the app decide what it looks like. 12 The person who trials it is the person who will defend it, so I pick them before I pick any software.

Read all twelve aloud and cut any that became two. The test is a breath: if you had to take one in the middle, it is two sentences wearing a full stop or a semicolon. Cut the weaker half. If the second half was the important half, keep that and cut the first. Line 3 is the clearest case in the bank.

★ Step 4 · one that became two, and the cutBefore: I split every workflow into lookups and decisions. Assistants do lookups, people keep decisions, and the line between them is yours to draw. After: The assistant does the looking-up, people keep the decisions, and the line between them is yours to draw.

Choose the one concept and write its ninety-second version. Pick by the false belief most likely to form in this meeting. Three sentences in the fixed order: what it is, why it matters, what it means for a real morning. Mark that row on the sheet.

★ Step 5 · the one, chosen and writtenConcept: the assistant drafts the lookups, a person keeps the decisions (line 3, decision door) Belief it prevents: "So the assistant should reply to the suppliers automatically?" What it is: It checks the confirmation against the outlet's order and writes the reply, but it only drafts; a person presses send. Why it matters: A yes to a substitute or a price change is the area manager's alone, and a pharmacy substitute needs a record by law. Farid's morning: The plain OKs, about four in five, arrive drafted; the one in twenty with a wrong price or a substitute arrives marked.

Write the sentence you hope she says to her board. One line, in her voice, that a board member would accept in six weeks' time. If you cannot write it, go back to step 5 and pick a concept that matters to her rather than to you.

★ Step 6 · her sentence"The assistant writes the easy replies, and a manager still says yes or no to every substitute, so nothing reaches a supplier without a person."

The teach-back lines are the part of the meeting that outlives it. The facts go into your notes and then into session 5's three projects. The lines go into her, and she carries them into the board meeting you are not in. Keep the plain column; session 6's one-page brief reuses the same words, so that what she reads in 24 hours matches what she heard, and "draft" means the same thing on the page as it did in the room.

Homework

Before session 5

30 min
  1. Make the lines plain for a real request. Take the sheet you built in session 1 for something on your own desk, ideally a request that starts "can AI do...", and run steps 2 to 4 on its teach-back column. Keep the before and after side by side; the difference is the lesson.
  2. Record yourself asking three questions, each with its teach-back line, on your phone. Play it back and count the words the business person would have to ask about. Model words hide best, because the newspapers have made them sound familiar; most people find one per line on the first pass and none on the third.
  3. Pick the one concept for your next meeting. Name the false belief it prevents, write the three sentences, and write the line you hope the business person repeats upward. If the request involves anything that would reply, send or approve, test whether drafts versus decisions is the one before you consider any other.
  4. Bring the plain sheet to session 5. Her answers become three projects there, and the words you settled on tonight, draft, decision, record, become the words those projects are named in.
Check yourself

Three questions

Clear all three and session 4 gets stamped in your passport.

1 · What does the curse of knowledge predict about an AI developer's own teach-back lines, and why is reading them aloud the test?

Camerer, Loewenstein and Weber's finding is that the better-informed person cannot fully set their knowledge aside when predicting what the less-informed person understands. "Retrieval" and "grounds" are invisible to the person who uses them daily, and "hallucination" is worse, because it sounds familiar enough to nod at. The listener's "sorry, what do you mean?" is the only reliable detector, and she will usually nod instead, which is why you read to a colleague first.

2 · Why is "the assistant drafts the lookups and a person keeps the decisions" the one concept for Adeline's meeting?

The runner-up is "the other half of the match lives somewhere else", because there is no order table, only another group in three languages. Both are chosen by the belief they prevent, not by how interesting they are. A developer who picks "the product list is its dictionary" and explains it beautifully can still demo an assistant that says OK on its own.

3 · What separates a lesson from a lecture, and what is the only proof that a line taught something?

Fitzpatrick's third rule, talk less, is the whole discipline. On the bench the mechanism is trust: every open question earns a point, eight facts sit behind trust gates, and the teach-back line is where the point is earned. A lecture spends minutes and earns nothing, the same arithmetic as the technical opener with a friendlier face.

Sources covered

What this session teaches, and where it came from

✓The curse of knowledge, the term and the findingCamerer, Loewenstein and Weber, "The Curse of Knowledge in Economic Settings", Journal of Political Economy 97(5), 1989. One paragraph here; Tech Writing session 1 teaches it in depth
✓Talk less; the third ruleRob Fitzpatrick, The Mom Test, 2013; reported from summaries, and only his verified wording is quoted anywhere on these pages
✓Measurement as a plain sentenceDouglas W. Hubbard, How to Measure Anything, 3rd ed., Wiley, 2014; definition reported as it appears across sources, book not opened here
✓The twelve teach-back lines, the five false beliefs and the trust gatesQuoted from the bench's printed bank file; the fast-run order and the trust mechanism are counted by the bench in session 3
◐The plain replacements for model wordsWorking definitions for a meeting with a person who runs pharmacies, not technical definitions; each is shorter and looser than the term it stands in for, on purpose
◐The lecture and lesson timelinesConstructed illustrations of a shape. No study of talk time in discovery meetings is cited, because none was read, and no number is claimed for either rail
◐Harbourline, Adeline Tan, Farid and the three storiesConstructed. The in-the-loop nod, the wrong concept and the lost-phone follow-up are invented to be typical, and every fact behind them is printed in the bench's bank file

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 4 cheat sheet · pin this

Your curse, not hersYou cannot hear your own jargon. Read the line to someone else and watch their face.
The usual offendersModel, prompt, fine-tune, RAG, embedding, hallucination, eval, guardrail. Each has a plain phrase.
Plain is not dumbed downThe person who signs off an assistant has to be able to check what it will do.
One concept per meetingNinety seconds, three sentences, chosen by the belief it prevents. Here: it drafts, a person decides.
Plain, true, one sentenceA second sentence about the architecture is a lecture. The tell: she stops answering and nods.
The only proofA follow-up question from her, like "so who presses send?". A nod proves nothing.