learn-asking-right-questions-as-ai-with-phoebe / Session 2 of 6
Learn Asking the Right Questions as an AI Developer with Phoebe · Session 2 of 6

The thread end to end, and its data

Last session you asked to see one thread. Tonight you watch it, message by message, and then ask the question an AI developer usually skips: what would an assistant actually have to read to do the manager's job, and where does each piece of it live? Door 3, the current way, leads straight into door 4, the data state. The supplier's confirmation is on a manager's personal phone. The outlet's order is in a different group, in three languages, with nicknames no system knows. The product list that would act as the assistant's dictionary sits behind one login held by one person. And some threads carry a patient's name. Those facts decide the build before any model does. Tonight is how to find each of them in plain words, and the one-page map you send her before you build anything.

🟡 Core AI developers, and anyone asked to "add AI" No code. A pen. 45 min
0-3 · Welcome 3-18 · The thread, its origins, the dictionary and the key 18-40 · Build-along: the thread map 40-45 · Q&A
Part 0

Where we start

An AI developer handed "can AI answer the WhatsApps" tends to ask for the data first: the chat history, a sample of messages, something to prompt against. Adeline Tan, operations director of Harbourline, a constructed chain of 38 pharmacy and clinic outlets across Singapore and Malaysia, cannot give you any of that in a form you would recognise, and asking for it first gets you a polite "I will ask IT". What she can do is open her phone and show you yesterday. The thread comes before the data because the thread tells you which data matters: what the manager looked at, what she compared it with, what she decided, and where that decision went afterwards. Part 1 watches the thread. Part 2 asks where each piece was born. Part 3 finds the dictionary, the names and the one key. The build-along turns all of it into a map she can correct. The six doors are stated in full at the corridor, and session 1 has the thread's first look.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = write it yourself

★ What you walk out with tonight. A one-page thread map for a real "can AI do X" request: every kind of message in the workflow, where each one lives, whether it was typed, recorded or never written down, which step is a lookup and which a decision, and what an assistant would need to read to do the lookups, with the dictionary, the login holder by name and the one box it must never see marked on the page. Session 3 puts the same facts on the bench and prices them.

Part 1 · covers the current way, watched before any data is read

Watch one thread before you read any data

5 min live

The thread looks like one conversation on the manager's phone. It is two. The supplier sends a confirmation with items and prices into one chat. The outlet's original order, the thing the confirmation has to match, was typed by the outlet manager into another WhatsApp group, in a mix of English, Malay and Chinese, using product nicknames. The area manager holds both in their head, matches them by eye, and replies OK or queries. About forty times a day. An assistant that reads only the supplier chat is reading half the job.

ONE THREAD NEEDS TWO CHATS, AND THE MATCH HAPPENS IN THE MANAGER'S HEAD watched on one area manager's phone, yesterday's messages SUPPLIER THREAD Confirmation, its format items and prices, in its words 14 SG + 22 MY suppliers Sometimes a substitute about 1 in 20 is wrong Reply: OK or query 4 in 5 end in OK OUTLET ORDER GROUP Outlet's original order typed by the outlet manager English, Malay, Chinese Product nicknames not in the POS product list No order table behind it this chat is the order happens in the manager's head THE MATCH, BY EYE items against items, prices against memory, nickname against name nothing links the two chats Supplier chat, a personal phone Outlet chat, another group The only link between the two chats is the manager, about forty times a day. An assistant inherits that step.
🔍 Click to zoom - the plum box is the work: not replying to one message, but matching two that live in different places
LiveOne thread, message by message3 min▶

Open with the bank's question g01, teach-back first: "The workflow as it happens today is the spec. I cannot automate a step I have not watched a person do." Then: "Show me one thread from yesterday, start to finish. What did you do at each message?" Adeline calls Farid over and he scrolls. The supplier's confirmation arrives with items and prices in the supplier's own layout. He opens a second group, the one the outlet posts its orders in, and finds what the outlet asked for. He reads down both, item by item. He replies OK.

Stop him at the second group. That is the fact that reshapes the project, and it would never have come out of a request for "the chat history". The follow-up is g03, with its teach-back: "An assistant checks one message against another. If the other one lives somewhere else in another language, that is the real project." Then "Where is the outlet's original order, and what does it look like?" The answer: typed by the outlet manager, in a mix of English, Malay and Chinese, using product nicknames. There is no order table anywhere.

Write down what he did at each message, not what the message said. "Read the confirmation, opened the outlet group, matched by eye, replied" is a spec. "Supplier sent a confirmation" is a log line. The difference is the part a person does between messages, and that part is exactly what an assistant would be asked to take over.

Self-studyForty a day, thirty-six ways2 min▶

Two numbers turn one watched thread into a workload. The first is volume: about forty threads a day per manager, and about four in five end in a plain OK. The bank's question g19 asks for exactly that, with its teach-back: "The share of plain OKs is the share an assistant can safely draft. The rest is where the person stays." It gives you the size of the routine part and, just as usefully, the size of the part that is not routine.

The second is variety: fourteen suppliers in Singapore and twenty-two in Malaysia, each writing its confirmation differently. Thirty-six formats is not a detail for later. It means there is no single message shape to parse, which is the first thing a developer assumes and the first thing the thread disproves. Ask to see a second thread from a different supplier before you leave; the difference between the two is the variety, in one glance.

Neither number needs trust to come out; they are facts she knows and states freely. The error rate, about one confirmation in twenty with a wrong price or a substitute, needs a little, and the question that gets it is in session 1. Watching the thread first is part of how that trust is earned: you have shown you care about how the work is done before you ask how often it goes wrong.

Real world

The nickname only one outlet uses. A constructed case, but every multi-site business has its version. An outlet manager types an order using the name her team calls a product, not the name on the POS and not the name the supplier uses. The area manager reads it correctly, because they have visited that outlet and know what the team means. A developer who builds from the supplier chat alone sees a clean confirmation that matches nothing. A developer who watched the thread sees three names for one item, and understands that the knowledge doing the matching is in the manager's head, gathered on outlet visits. That is also why the managers are the scarce resource: the knowledge that makes them good at the thread is the knowledge they gather in the outlets they no longer have time to visit.

Part 2 · covers where each piece was born, and how mature the habit is

Recorded, typed, or not recorded, for an assistant

5 min live

Once you have watched the thread, list what an assistant would have to read to do the manager's lookups, and ask where each thing was born. There are only three answers. Recorded by a system as a side effect of work. Typed by a person. Or not recorded anywhere. At Harbourline three of the four things an assistant needs were typed by a person on a phone, and the fourth, what was decided, was never written down at all. There is no order table anywhere, a chat can be exported only one at a time by hand, and the phones are the managers' own.

WHERE EACH THING THE ASSISTANT MUST READ WAS BORN one question per item: "where does this live, step by step?" The supplier's confirmation items, prices, substitutes The outlet's original order what the outlet asked for The product list to know which item is which What was decided who said yes to a substitute TYPED, by the supplier in a chat on a manager's personal phone; 36 formats TYPED, by the outlet manager in another group; English, Malay and Chinese; product nicknames RECORDED, by the POS codes differ between SG and MY; the nicknames are not in it NOT RECORDED anywhere the thread is the record; in March one left with her phone How an assistant could reach any of it today no order table; chats export one at a time, by hand; personal phones, no business account, no API Three of the four were typed on a phone; the one a pharmacy most needs kept was never written down.
🔍 Click to zoom - the plum row is the gap an assistant cannot read around: a decision nobody recorded
LiveWhere the messages live, and where the decision goes3 min▶

Two questions from the bank find the whole left side of that figure. The first is g11, teach-back first: "Where the messages live decides whether an assistant can even read them. That is a setup question, not a model question." Then: "Are the WhatsApp numbers personal phones or a business account? Can a chat be exported?" Adeline's answer is short and decisive. The numbers are the managers' personal phones. There is no business account and no API. A chat can be exported, one chat at a time, by hand.

Say what that means out loud, because it is the moment the project changes shape. An assistant cannot sit behind a personal WhatsApp number and read along. Anything it reads, somebody has to hand it, one chat at a time. That is not a problem a better model solves. It is a setup fact, and it belongs on the data-state map before any design is drawn.

The second is g04, with its teach-back: "If the record is a phone, the first thing an assistant should produce is a log, before it produces a single answer." Then "What happens to a thread once it is done? Where does the decision live?" This one needs a little trust, and it is why you ask it after the thread, not before. Nobody keeps a record of what was decided. The WhatsApp thread is the record. An area manager left in March and her phone went with her. For a pharmacy, where substitutions need a record by law, that is the most important box on the page, and it is empty.

LiveFive rungs, asked about the thread2 min▶

Where a thing was born tells you what it is. How mature the habit around it is tells you how far you can lean on it. DAMA-DMBOK, 2nd edition, chapter 15, names five levels of data management maturity: Initial/Ad hoc, Repeatable, Defined, Managed, Optimized. Five, with no level 0. The names are borrowed, as reported from summaries; the plain questions are ours, and they are what Adeline hears. The ladder is taught in depth in session 2 of the data analyst sibling; for an AI developer it is five questions about the thread.

"When a confirmation is wrong, who catches it, and how do they know?" The manager, by eye, because she knows the outlet. "Does each manager do it the same way?" You do not know yet, and neither does the fact sheet; ask, and write down the answer she gives rather than the one you expect. "Is any of it written down?" No: the method is in three heads and the record is on three phones. "Does anyone check afterwards whether the OK was right?" Nothing she has said so far suggests anyone does. "Has it got better on its own?" Ask. On what she has told you, the matching habit reads as the bottom rung, with the second and fourth questions still open. That reading goes in your notes, never into the meeting as a grade.

The rung matters more for an assistant than for a report. A report built on an ad hoc habit is a report nobody trusts. An assistant built on one inherits the habit's variety as its training data and its test cases, with no written method to check its drafts against. The real maturity assessment is a governance discipline, taught in Governance 101; in a discovery meeting it is five questions and a note.

Self-studyWhy preparation eats the project, quoted as it stands2 min▶

Two figures are usually run together into one, and they should not be. In 2014 Steve Lohr reported in The New York Times that data scientists spend 50 percent to 80 percent of their time collecting and preparing data. In 2016 CrowdFlower's data science survey reported 60 percent of time spent cleaning and organising data and a further 19 percent collecting it. Different methods, different shapes of answer: a range for two activities together, and two separate figures. Quote whichever you use as it stands.

For an assistant at Harbourline, preparation has a concrete shape, and every piece of it is visible in the figure above. Somebody exports chats by hand from three personal phones. Somebody builds a dictionary from outlet nicknames to POS codes, twice, because the codes differ between the two countries. Somebody decides what to do about the decisions that were never recorded. None of that is model work, all of it comes before model work, and every piece was found by asking where a thing was born rather than asking for "the data".

Real world

The history that was not a dataset. Constructed, and a request every AI developer has made. The developer asks for six months of thread history to test a prototype against. It sounds like one export. At Harbourline it is each area manager opening each supplier chat on their own phone and exporting it by hand, one chat at a time, then doing the same for the outlet groups, because the confirmations are useless without the orders they answer. The threads of the manager who left in March are not there at all: they went with her phone. And somewhere in those exports is a patient's name, which Part 3 is about. "Can I have the history?" was the wrong first question. "Where does it live, and who would have to export it?" finds all of that in two minutes.

Part 3 · covers the dictionary, the names in the threads, and the one key

The dictionary, the names, and the one key

5 min live

An assistant that drafts the lookups has to know which item is which, so it needs a dictionary: the POS product list, whose codes differ between the two countries and which holds none of the outlet nicknames. The dictionary sits behind one login, held by Kenneth, the only IT person, who is leaving in January. And some threads carry a patient's name, which turns a supplier tool into a personal-data tool. Three facts, three different people who have to say yes, and none of them about a model.

THE DATA-STATE MAP FOR AN ASSISTANT, AS SHE CAN READ IT what it could read, what it needs a dictionary for, what it must never see CAN READ, ONCE EXPORTED Supplier confirmations one chat at a time, by hand, from each manager's own phone 36 formats, no shared layout NEEDS A DICTIONARY Outlet orders nicknames, three languages dictionary: the POS product list codes differ SG vs MY the nicknames are not in it MUST NEVER SEE Patient names sometimes in a thread, when an outlet orders for a named person Malaysia's PDPA: personal data What was decided not recorded; the thread is it first output: a log, not an answer POS product list admin login: Kenneth, only him leaving January; team not told Who has to approve a name makes it personal data head office lawyer: four weeks Each box says who can say yes. The plum box changes who has to approve the whole tool.
🔍 Click to zoom - the plum box is the one an assistant must never be shown, and it quietly moves the approval to a lawyer
LiveThe product list is its dictionary2 min▶

Every lookup the assistant drafts is a match: this line on the confirmation is that line on the outlet's order. To match, it has to know that two different names are one product. The bank's question g09 asks for the thing that would let it: "Which system holds the product list, and do the suppliers use your codes?" Teach-back first: "An assistant can only match what it can look up. The product list is its dictionary."

Adeline's answer has two halves, and both land on the map. The POS holds the product list, so there is a dictionary. But the codes differ between the two countries, and the nicknames the outlets use are not in it. The dictionary is real and incomplete in exactly the place the matching happens. Filling that gap, nickname to product, outlet by outlet, is work a person has to do with the managers, and it is worth more to any assistant than a better prompt.

LiveThe login, asked by name2 min▶

Question g10 has to be asked with a name in the answer: "Who can log in to the POS back office and pull the product list?" Teach-back: "Access is the first blocker on most projects, so I ask early and I ask by name." "IT can" is not an answer. "Kenneth can" is.

The rest of the answer does not come out on the first asking. Kenneth is the only IT person, he holds the POS admin login, he is leaving in January, and she has not told the team. On the bench that fact sits behind the highest trust gate on her sheet: she volunteers it only after several open questions with their reasons said aloud have shown her you are trying to help. It is worth 3 points, and it is the difference between an assistant with a dictionary and an assistant whose dictionary nobody can update after January. That is also why this question comes after the thread and not before it: asked cold, it gets "our IT guy".

LiveA name in the thread changes who approves2 min▶

Question g12 is short and easy to skip, and skipping it is how a supplier tool becomes a legal problem: "Do customer names ever appear in these threads?" Teach-back: "A name in a message turns a supplier tool into a personal-data tool, and that changes who has to approve it."

They do, sometimes, when an outlet orders a special item for a named customer. She mentions it once she trusts you a little. A patient name in a message makes it personal data under Malaysia's Personal Data Protection Act, and the head office lawyer takes four weeks to answer anything. So the box goes on the map in plum, labelled "must never see", and the approver changes from Adeline alone to Adeline and the lawyer. What the law requires in detail is not something to improvise in a discovery meeting; what you need is the fact that names appear, and the name of the person who will have to say yes.

Self-studyWhat the map is for2 min▶

The map above is the first thing you send Adeline after the meeting, before any prototype, before any proposal. Six boxes, a name or "nobody" wherever someone has to say yes, one key, one plum box. No word on it she would have to ask about. She can look at it on her phone and say "two of the outlets post their orders in a different group", or "Kenneth has a deputy who can log in too", and either correction is worth more than a week of building.

It is also your own honesty check before you design anything. A box that says "not recorded" means the project has a logging task before it has an assistant. A key on one person who is leaving means a continuity task before a dictionary. A plum box means an approval task before a pilot. An assistant proposed on top of a map with all three unaddressed is a demo, and session 5 is where those three tasks become candidate projects in their own right.

Real world

The export that carried a name. Constructed, and one way a small AI pilot turns into a large one. A developer who never asked g12 exports a few supplier threads to try a drafting prompt against. One of them is a special order, and the outlet's message has the customer's name in it, typed so the right bottle goes to the right person. The prompt works. The pilot is now a tool that has read personal data, on a laptop nobody approved, and the next conversation is with a lawyer who takes four weeks. Asked in the meeting, the same question costs three minutes and puts one plum box on the map.

Build-along

Map the thread, then ask which box is wrong

★ 22 min · everyone writes

One page, written by hand from what you watched and what she told you. Use Adeline's supplier thread, or a workflow from your own "can AI do X" request, and build the thread map you will send within a day of the meeting. One row per kind of message, not per message. Every cell is something she can correct, and none of it is a schema.

THE THREAD MAP: ONE ROW PER KIND OF MESSAGE, SIX QUESTIONS IN THE MARGIN MESSAGE WHERE IT LIVES ORIGIN STEP ASSISTANT NEEDS Confirmation supplier chat typed lookup the outlet order Outlet order outlet group typed lookup the dictionary Substitute supplier chat typed DECISION a person, by name your next row IN THE MARGIN 1 Every kind of message? 2 Where does it live? 3 Typed, recorded, absent? 4 Lookup or decision? 5 What must it read? 6 "Which box is wrong?" the line you send it with One row per kind of message, not per message. Forty threads a day share a handful of kinds.
🔍 Click to zoom - three rows are filled to show the depth; the plum row is the one that keeps a person, and the map says who

List every kind of message in one thread, in the order it arrives. From the thread you watched, not from what a thread ought to contain. Include the message that lives somewhere else; it is the one most maps leave out.

★ Step 1 · the kinds of message, in order1. Supplier confirmation: items and prices, in the supplier's own format 2. The outlet's original order: in another WhatsApp group, typed earlier 3. Sometimes: a substitute offered, or a price change 4. The manager's reply: OK, or a query 5. Delivery time: just needs to reach the outlet

Where each one lives, and how it was born. Typed by a person, recorded by a system, or not recorded anywhere. Beside each, write the question that found it, so you can repeat it next time. Say the teach-back first every time.

★ Step 2 · where it lives, with the question that found itConfirmation TYPED supplier chat, manager's personal phone found by: "show me one thread from yesterday" Outlet order TYPED outlet group; English, Malay, Chinese; nicknames found by: "where is the outlet's original order?" Product list RECORDED the POS; codes differ SG vs MY; no nicknames found by: "which system holds the product list?" Decision made ABSENT the thread is the record; one phone left in March found by: "where does the decision live?"

Tag each step lookup or decision, and name the owner of each decision. A lookup has a right answer that already exists somewhere. A decision is a choice a person owns. Write the name of the person who may say yes, never a role in general.

★ Step 3 · lookups, decisions, ownersMatch confirmation to order LOOKUP done by eye today Check the price LOOKUP against what? there is no order table Accept a substitute or price DECISION area manager only; needs a record Pass on the delivery time LOOKUP nobody decides anything about it Reply OK LOOKUP about four in five threads end here

What the assistant would need to read, and who opens each door to it. For every lookup, write what it would have to look in, and the name of the person who can give it that. Then add the one box it must never see.

★ Step 4 · what it reads, and who says yesSupplier chats exported one at a time, by hand, by each manager no business account, no API Outlet orders the same, from the outlet groups needs the dictionary to match Dictionary POS product list login: Kenneth only; leaving January, team not told Nickname list does not exist yet made together with the managers, outlet by outlet NEVER SEE patient names in special orders personal data under Malaysia's PDPA; lawyer, four weeks

Draw the map. Boxes for what it could read, what it needs a dictionary for, and what it must never see; a box for the decision record, dashed if it does not exist; the key on the one login. A monospace sketch in a message is fine; a photo of a hand drawing is better, because it looks like something she is allowed to correct.

★ Step 5 · the map, as it goes in the message+----------------------+ +----------------------+ +----------------------+ | CAN READ, exported | | NEEDS A DICTIONARY | | MUST NEVER SEE | | supplier chats | | outlet orders | | patient names | | by hand, per phone | | nicknames, 3 langs | | PDPA, lawyer 4 weeks | +----------------------+ +----------+-----------+ +----------------------+ ^ | :......................: +----------+-----------+ : DECISIONS: not kept : | POS product list | : first output: a log : | [key] Kenneth only | :......................: | leaving January | +----------------------+

Write the one line you send with it. Not a summary, not a proposal, not a prototype. One sentence that says what the map is and asks her to correct it. Send it within a day, while she still remembers what she told you.

★ Step 6 · the covering lineHi Adeline, here is what I understood about one supplier thread and where each piece of it lives. Which box is wrong?
The same message, as it is usually sentHi Adeline, attached is a proposed architecture for an LLM-based WhatsApp agent with retrieval over thread history and a guardrail layer. Could IT share an export of the chats so we can start testing?

Send the map, not a prototype, first. A prototype says "I have started building"; a map says "I have started listening". She can correct the map on her phone in a minute, and every correction is a build week you do not lose. Session 5 turns the dashed box and the missing nickname list into candidate projects of their own, beside the assistant that only drafts; session 6 folds the corrected map into the one-page brief.

Homework

Before session 3

30 min
  1. Watch one real thread. For a workflow someone wants an assistant for, sit with the person who does it and watch one instance end to end. Write what they did at each step, not what each message said. Mark the moment they opened something else.
  2. Map it. One row per kind of message: where it lives, typed or recorded or absent, lookup or decision, and what an assistant would have to read. Put a name or "nobody" wherever someone has to say yes.
  3. Find the dictionary and its key. Name the list the assistant would look things up in, what is missing from it, and the one person who can log in to it. Write down what happens the week that person is away.
  4. Ask the name question. Find out whether anything personal ever appears in the messages. If it does, write down who would have to approve, and how long they take. Bring the map to session 3.
Check yourself

Three questions

Clear all three and session 2 gets stamped in your passport.

1 · Why does an AI developer ask to watch one thread end to end before asking for any chat history or sample data?

"The workflow as it happens today is the spec." The second group, the nicknames and the match by eye only appear when you watch; no export of the supplier chat contains them.

2 · The WhatsApp numbers are the managers' personal phones, with no business account and no API. Why does that belong on the data-state map rather than in a later technical note?

"That is a setup question, not a model question." It is also why the manager who left in March took her threads with her: the record was a personal phone.

3 · Patient names sometimes appear in the threads, when an outlet orders a special item for a named customer. What does that change?

"A name in a message turns a supplier tool into a personal-data tool, and that changes who has to approve it." Asked in the meeting it costs three minutes; discovered in a test export, it costs a pilot.

Sources covered

What this session teaches, and where it came from

✓Five maturity level names: Initial/Ad hoc, Repeatable, Defined, Managed, OptimizedDAMA International, DAMA-DMBOK, 2nd ed., Technics Publications, 2017, ch. 15; reported from summaries. DMBOK has no level 0. The plain questions beside each rung are ours, not DAMA's; the ladder in depth is in session 2 of the data analyst sibling
✓50 percent to 80 percent of time collecting and preparing dataSteve Lohr, "For Big-Data Scientists, 'Janitor Work' Is Key Hurdle to Insights", The New York Times, 17 Aug 2014; quoted as a range, as printed
✓60 percent cleaning and organising, plus 19 percent collectingCrowdFlower, 2016 Data Science Report; two separate figures, never combined with Lohr's range into one number
✓Door 3, current way, leading into door 4, data stateThe Six Doors are stated in full, with the source of each move, in session 1 of the data analyst sibling, the canonical corridor
◐Malaysia's Personal Data Protection Act 2010 (PDPA)Named because the constructed case sits under it, as a typical touchpoint rather than a sourced finding. What the Act requires was not read at source and is not claimed; the page teaches only that a name in a message changes who approves
◐Harbourline, Adeline Tan, Kenneth, Farid and the supplier threadsConstructed. The two chats, the nicknames, the personal phones, the missing order table, the login and the patient names are typical, not sourced; every value, including the 3 points on the login fact, is printed in the bench's bank file

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 2 cheat sheet · pin this

Watch before you read"Show me one thread from yesterday, start to finish." What they did between messages is the spec.
Two chats, one matchThe confirmation in one chat, the outlet's order in another, matched by eye. An assistant reading one chat reads half the job.
Typed, recorded, or absentAsk where each thing was born. Here three were typed on a phone and the decision was never written down.
Setup, not modelPersonal phones, no business account, no API, one export at a time. Where messages live decides whether it can read them.
The dictionary and its keyThe product list is the assistant's dictionary. Ask who can open it, by name, and what happens when they leave.
The box it must never seeA name in a message makes it a personal-data tool. Ask early; it changes who approves. Send the map with "which box is wrong?"