learn-data-thinking-with-phoebe / Session 2 of 6
Learn Data Thinking with Phoebe · Session 2 of 6

What you would need to know

You have three questions with all four slots filled, and no idea yet whether a single one of them can be answered, because nobody has said out loud what would answer them. The move almost nobody makes is to write the ideal table first - the columns you would need if you could have anything at all - and only then open the file to see what is actually there. In that order the gaps are visible, nameable and sendable. In the other order the question quietly shrinks to whatever the spreadsheet happens to support, and nobody notices the substitution, including you.

🟢 Everyone Bring the three rewritten questions No maths, no code 45 min
0-3 · Welcome 3-18 · The ideal table, and the gap list 18-40 · Build-along: three ideal tables 40-45 · Q&A
Part 0

Where we start

Session 1 ended with three requests rewritten so that each one carries an outcome, a population, a timeframe and a decision. That makes them answerable in principle. Whether any of them is answerable in practice is still completely unknown, because nobody has said what would answer them. Daybreak, the coffee subscription company, ran its autumn promotion, and the three askers are still waiting: marketing wants conversion, finance wants revenue per order, and the founder wants to know whether the customers the offer brought in are the sort who stay.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class

★ What you walk out with tonight. An ideal table sketched for each of your three questions, and a gap list naming exactly which columns are missing, what each one would let you answer, and who can change it. The gap list is not a note to yourself. It is a message you can send.

Where this sits on the shelf. Writing the SQL that pulls the rows once you know which rows you want is learn-sql-with-phoebe. Designing a new collection so that the missing column exists next time, properly, with a control group, is learn-experimentation-with-phoebe. Here the subject is working out what you would need, on paper, before you open anything at all.

Part 1 · covers the ideal table

Sketch the table you wish you had

6 min live

Take one question with its four slots filled, and draw the table that would answer it if the data gods were generous. Not the table you have. The one you would order from a catalogue. It takes about four minutes per question and it is done with a pen, because the moment a real file is open the sketch stops being ideal and starts being a description of that file.

SKETCHED BEFORE ANY FILE IS OPENED Did conversion rise among new subscribers after the offer? WHICH SLOT EACH COLUMN SERVES order_id order_date customer_id is_new channel converted value 10241 09 Oct C-8842 yes email yes 24.00 10242 09 Oct C-1077 no social no 0.00 10243 10 Oct C-9310 yes social yes 31.50 ONE ROW = ONE ORDER the founder's question needs one row per customer instead POPULATION -> is_new, channel a column you can filter on OUTCOME -> converted the thing being measured TIMEFRAME -> order_date so two windows can be cut DECISION -> channel so the answer can be acted on Nothing on this sketch has been checked against a real file yet. That is the point of drawing it first. A column you cannot name a purpose for does not belong on the sketch. Cross it off before you ask for it.
🔍 Click to zoom - seven columns, four slots, and a grain stated in the first line, all of it written before a file is opened
LiveOne row per what?4 min▶

The single most useful question you can ask about any analysis, yours or somebody else's, is what does one row represent. An order. A customer. A customer-week. A session. A support ticket. It takes four words to ask and it is the commonest structural error in the whole field, because it is invisible once the numbers come out.

Here is why it matters at Daybreak, concretely. Marketing's question is about orders, so the ideal table has one row per order and the conversion rate is a count of converted orders over all orders. The founder's question is about people, so the ideal table has one row per customer. Those are two different tables, and answering the founder's question on the order table does something specific and silent: a customer who ordered forty times contributes forty rows, a customer who ordered once contributes one, and the answer you get describes the habits of heavy buyers wearing the label "our customers". Nobody sees this happen. The number looks completely ordinary.

So write the grain on the first line of the sketch, in words, before any column name. "One row per order, for orders placed between 15 September and 8 December." If you cannot write that sentence, you are not ready to ask anyone for data yet, and five more minutes with the pen is cheaper than the two days that follow.

Self-studyColumns come from slots4 min▶

The ideal table is not invented from nothing. Each of the four slots from session 1 demands a specific kind of column, and if you work through them in order you will have most of the sketch without any creative effort at all.

SlotWhat it demandsDaybreak example
PopulationA column you can filter on, with one value per row, so that the group in the question becomes a row filter rather than a wordis_new, which turns "new subscribers" into orders where the customer had no earlier order
OutcomeThe thing being measured, recorded on every row, in the units the question usedconverted for marketing, order_value for the finance version of the same request
TimeframeA date on every row, so the two windows in the question can actually be cut apartorder_date, covering the twelve weeks either side of the offer, not just the after period
DecisionA grouping column, because a decision is rarely taken for everybody at once and the answer usually has to differ by groupchannel, since the offer can be repeated on email and dropped on social, which is a real spring decision

The decision slot is the one people leave off the sketch, and leaving it off is how you end up with a single global number that nobody can act on. If the spring choice is per channel, an answer that does not carry a channel column is not an answer to the question that was asked, however correct it is.

Two more columns earn their place on most sketches even though no slot demands them: an identifier, so you can check whether a row is duplicated, and a date of record, so you can tell when a value was written rather than only what it says.

Part 2 · covers the gap list

The gap list, and what to do with each kind of gap

5 min live

Now open the file. Go down the sketch, tick every column that exists, and mark every one that does not. What is left is the gap list, and it is worth much more than it looks, because a gap is not a complaint. Each kind of gap has a different cause, a different fix and a different person, and telling them apart takes about a minute per column.

IDEAL COLUMNS, CHECKED AGAINST THE REAL TABLE WANTED THREE KINDS OF GAP, THREE DIFFERENT REPLIES ✓ order_id have it ✓ order_date have it ✓ customer_id have it ✓ is_new have it ✓ channel have it ✓ converted have it ✓ order_value have it renewal_status missing email_opened missing signup_source missing NEVER COLLECTED nobody ever wrote it down, so no file anywhere contains it Ask the owner of the system to start recording it, and say from which date. COLLECTED BUT NOT KEPT it existed, then was overwritten, truncated or aged out of a window Ask who still has a copy and how far back it goes, before the window shuts. EXISTS BUT NOT JOINABLE another team holds it under an identifier yours does not share Ask what the shared key is, and whether anyone has mapped the two before. Not one of these three replies says the data is bad. Each of them names a person, a column and a date. A gap you cannot attach a name to is not actionable yet. Find the owner before you write the message.
🔍 Click to zoom - three missing columns, three causes, three different people, and not a single sentence about bad data
LiveThree kinds of gap, three different replies4 min▶

Never collected. Nothing anywhere holds this value, because no system was ever asked to write it. The reply goes to whoever owns the system that would write it, it asks for collection to begin, and it names the date from which you need it. The cost of this gap is time: the answer arrives one collection period from now, not today, and saying that plainly on day one is what lets somebody plan around it.

Collected but not kept. The value existed and was overwritten, truncated or aged out of a retention window. This is the gap most worth chasing hard and fastest, because a copy is often sitting in an export on somebody's laptop, in a vendor dashboard that keeps ninety days, or in a monthly report nobody deleted. The reply asks who has a copy and how far back it goes, and it is urgent in a way the other two are not, because the window is still closing while you write.

Exists but not joinable. Another team has the column, under an identifier yours does not share. This is a plumbing question, not a data question, and the reply is one line: what is the shared key, and has anybody mapped these two before? The answer is frequently yes, and frequently from someone who tried it last year and can save you a week.

What none of these replies says is "the data is bad", which is the sentence that ends conversations. Every gap message has the same three parts: what is missing, what it would let you answer, and who can change it. Three parts, one line each, and the third part is the one that turns a complaint into a request.

Self-studyWhy ideal-before-actual is the whole point4 min▶

The order is not a formality. Open the spreadsheet first and something specific happens within about ninety seconds: you see which columns are there, your sense of what is askable rearranges itself around them, and the question you end up answering becomes the question that file supports. The original question is not rejected. It is quietly replaced, and the replacement feels like the original, because nothing was ever written down to compare it against.

Writing the ideal table first is the only way that substitution becomes visible. The sketch is a record of what you wanted made before you knew what you could have, and holding the real table against it is the moment the difference shows up as a list rather than as a feeling. That list is the gap list, and it exists only because the sketch came first.

Be straight about the standing of this, because it is craft rather than a research finding. No study shows that teams who sketch the ideal table produce better analyses than teams who do not, and nobody appears to have run one. What can be said is that every process model in the field independently orders the work this way: CRISP-DM, published by a cross-industry consortium in 2000, puts business understanding ahead of data understanding among its six phases, which encodes exactly this rule. Agreement between process models is not evidence of effectiveness, and it should not be presented as if it were.

Real world

The founder's question needs a column that records what a customer did in the renewal cycle after the promotion. The rows stop at week 24. There is no such column and there is no file that has one, because nothing was ever asked to record it. That is a never-collected gap, and the reply is one sentence: nothing records renewal behaviour after week 24, so this question gets answered in the spring rather than now, and here is who has to switch the recording on today for that to happen.

Part 3 · covers proxies

Proxies, and the honest way to use one

4 min live

Some gaps will not close in time, and the pressure then is to measure something nearby instead. That is a proxy, it is completely normal, and every organisation runs on them. What separates an honest proxy from a quiet fabrication is not the choice of substitute. It is whether the substitution is said out loud, in the same breath as the number.

ONE REAL OUTCOME, THREE THINGS PEOPLE MEASURE INSTEAD THE REAL OUTCOME Did the customer still have an active subscription at the next renewal? Ordered again within 12 weeks USABLE, SAY SO Captures: they came back and paid, which is most of what staying means. Misses: a lapsed customer who buys one bag once still counts as stayed. Opened the renewal email TOO FAR - DO NOT Captures: attention, at the moment the email landed in an inbox. Misses: payment entirely. A silent, happy, paying customer scores zero. Completed onboarding in week one EARLY ONLY Captures: the first week, which does predict the first month fairly well. Misses: the renewal cycle, which is the thing the founder asked about. A proxy is fine. Naming it in the same sentence as the answer is the thing that makes it fine.
🔍 Click to zoom - three candidates for one missing column, and the one that measures attention instead of money is not a substitute at all
LiveA proxy is a different question wearing the name of the first one4 min▶

When you cannot measure whether customers stayed, you measure whether they ordered again within twelve weeks. That is a reasonable thing to do. It is also, strictly, an answer to a different question, and the whole of the honesty here lives in one habit: you name the substitution in the same sentence as the number, every time, and never in a footnote.

Compare the two sentences. "Sixty-one percent of promotion customers stayed" is a claim about retention that nobody measured. "Sixty-one percent of promotion customers ordered again within twelve weeks, which is what we are using in place of whether they stayed" is the same number with the substitution attached, and the second version costs fourteen extra words. Those fourteen words are the entire difference between an honest proxy and a misleading one.

The reason it has to be the same sentence is mechanical rather than moral. Numbers travel and their footnotes do not. Your caveat survives one slide, maybe two, and then somebody pastes the figure into an email and the qualification is gone forever. If the substitution is inside the sentence, it travels with the number, and by the time it reaches the board the claim is still the claim you made.

Self-studyWhen a proxy is not acceptable3 min▶

When the decision is expensive and hard to reverse. A proxy is a loan against the truth, and the interest is paid by whoever acts on it. For a reversible weekly choice, borrow freely. For hiring a team, signing a year of spend, or telling a board that a channel works, the gap between the proxy and the real outcome is being converted into money that cannot come back.

When the proxy and the real outcome are known to diverge for the group you care about. This is the failure that looks most like success. Email opens are a defensible proxy for engagement across a whole customer base, and a terrible one for the subset who pay by standing order and never open anything. If the divergence lands precisely on the population in the question, the proxy is not weaker than the real thing, it points the other way.

When the substitution would not survive being said out loud to the person who asked. This is the most practical test of the three, and you can run it in your head in five seconds. Say the sentence with the substitution in it, to the founder, in your imagination. If it comes out as "we measured whether they opened an email, and we are calling that staying", you already have your answer, and no amount of careful wording fixes it.

Build-along

Three ideal tables, three gap lists

★ 22 min · your own three questions

Your three rewritten questions from session 1, one sketch each. Pen first, file later, and the order is the part that does the work.

Decide what one row represents for each of the three questions, and write it as a sentence before any column name. An order, a customer, a customer-week, a ticket. Where two of your questions need different grains, note that, because it means two tables and not one.

Write the ideal columns from the four slots. Population needs something to filter on, outcome needs the thing being measured, timeframe needs a date on every row, decision needs a grouping column. Add an identifier so duplicates are detectable. Do not open anything yet.

Now open the real data and tick what exists. Go down the sketch in order and mark each column present or absent. Resist rewriting the sketch as you go; the columns you wanted and did not get are the entire product of this exercise.

Sort every missing column into never collected, collected but not kept, or exists but not joinable. If you cannot tell which, that is itself a question for somebody, and it is usually answered in one reply.

Write one message per gap, three parts each: what is missing, what it would let you answer, who can change it. Put the name in before you send. A gap addressed to nobody stays open indefinitely and then gets described as a data problem.

Sketch on paper, not in a spreadsheet. Opening a spreadsheet to draw the ideal table means opening a spreadsheet, and within a minute you are looking at column names that exist instead of column names you want.

Write the grain as a full sentence, with the date range in it. "One row per customer who placed a first order between 15 September and 8 December" is a grain. "Customers" is a word.

Include the columns you expect to be missing. The sketch is a wish list, and a wish list that pre-censors itself produces a gap list with nothing on it, which is the one outcome guaranteed to be wrong.

Chase the not-kept gaps first. Those are the ones where a copy still exists somewhere and the window is closing. The never-collected ones cannot get worse; the not-kept ones can.

Name a person, not a team. "Engineering" does not read messages. One human being does, and finding out which one takes a single question to whoever sits nearest the system.

Keep all three sketches. Session 3 opens the data behind the closest one and spends twenty minutes on it before calculating anything at all.

If you take one thing from tonight. Do not open the data until stage three. Everything before it is worth doing precisely because it was done in ignorance of what the file contains, and the ignorance is not a defect in the method, it is the method. Once you have seen the columns you cannot unsee them, and the sketch you draw afterwards will be a description of the file with a wish list's name on it.

Homework

Before session 3

25 min
  1. Send the gap messages and note which ones get an answer and how fast. The pattern is informative: not-kept gaps usually come back within a day because somebody knows exactly where the copy is, and never-collected gaps take longer because the reply requires a decision rather than a lookup.
  2. Find one analysis in your organisation that used a proxy without saying so. A retention figure built on repeat orders, an engagement number built on logins, a satisfaction score built on survey responders only. You will not have to look hard, and the point is not to catch anyone out, it is to see how completely invisible the substitution is once the number is in a slide.
  3. Pick the question whose ideal table is closest to what exists. That is the one session 3 opens and looks at properly, and closest means fewest missing columns, not most interesting subject.
  4. Bring the actual file or table for that question, whatever state it is in. Not a summary of it, not a screenshot of the tidy tab, the real thing with the real number of rows.
Check yourself

Three questions

Clear all three and session 2 gets stamped in your passport.

1 · Why is "what does one row represent" worth asking before anything else?

A customer with forty orders contributes forty rows and one with a single order contributes one, so the answer describes heavy buyers while carrying the label "our customers". The number looks entirely ordinary when it comes out, which is exactly why the grain has to be settled in words at the top of the sketch. It is the commonest structural error in the field and the cheapest one to avoid.

2 · You have three missing columns. What does sorting them into kinds actually buy you?

Three kinds, three replies, three different people, and none of them is the sentence "the data is bad", which ends conversations rather than starting them. The not-kept gaps are the urgent ones because a copy often still exists and the retention window is closing while you write. Every message has the same three parts: what is missing, what it would let you answer, and who can change it.

3 · You cannot measure whether customers stayed, so you use "ordered again within 12 weeks" instead. What is the rule?

Numbers travel and their footnotes do not; a caveat survives one slide and then somebody pastes the figure into an email without it. Putting the substitution inside the sentence costs about fourteen words and means the qualification arrives wherever the number arrives. Proxies are legitimate and unavoidable, so the honesty has to live in how the answer is phrased rather than in whether a proxy is used.

Sources covered

What this session teaches, and where it came from

✓The four slots mapped to columnsPractitioner craft, built directly on session 1's four slots; each slot demands a specific kind of column and the mapping is stated as method, not as finding
◐The ideal-before-actual disciplinePractitioner craft. No study shows teams that sketch the ideal table first produce better analyses; the source map records that nobody appears to have run one, and the page says so
✓CRISP-DM puts business understanding before data understandingRead from the primary consortium document, published 2000; six phases, in that order
✓Proxy discipline, and the same-sentence ruleCraft with no citation behind it, offered as a working norm rather than as evidence; the reason given is that footnotes do not travel with numbers
✓The founder's gap at DaybreakA property of the dataset rather than a claim about the world, and verifiable directly in session 4's bench

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 2 cheat sheet · pin this

One row per whatWrite the grain as a sentence before any column name.
Columns come from slotsFilter, measure, date, group. Four slots, four columns.
Ideal first, file secondOpen it first and the question becomes what the file supports.
Three kinds of gapNever collected, not kept, not joinable. Different person each time.
Every gap has a nameMissing what, answers what, who can change it.
Proxies go in the sentenceNot the footnote. Footnotes do not travel with the number.