Where we start
Session 1 ended with three requests rewritten so that each one carries an outcome, a population, a timeframe and a decision. That makes them answerable in principle. Whether any of them is answerable in practice is still completely unknown, because nobody has said what would answer them. Daybreak, the coffee subscription company, ran its autumn promotion, and the three askers are still waiting: marketing wants conversion, finance wants revenue per order, and the founder wants to know whether the customers the offer brought in are the sort who stay.
★ What you walk out with tonight. An ideal table sketched for each of your three questions, and a gap list naming exactly which columns are missing, what each one would let you answer, and who can change it. The gap list is not a note to yourself. It is a message you can send.
Where this sits on the shelf. Writing the SQL that pulls the rows once you know which rows you want is learn-sql-with-phoebe. Designing a new collection so that the missing column exists next time, properly, with a control group, is learn-experimentation-with-phoebe. Here the subject is working out what you would need, on paper, before you open anything at all.
Sketch the table you wish you had
6 min liveTake one question with its four slots filled, and draw the table that would answer it if the data gods were generous. Not the table you have. The one you would order from a catalogue. It takes about four minutes per question and it is done with a pen, because the moment a real file is open the sketch stops being ideal and starts being a description of that file.
LiveOne row per what?4 min▶
The single most useful question you can ask about any analysis, yours or somebody else's, is what does one row represent. An order. A customer. A customer-week. A session. A support ticket. It takes four words to ask and it is the commonest structural error in the whole field, because it is invisible once the numbers come out.
Here is why it matters at Daybreak, concretely. Marketing's question is about orders, so the ideal table has one row per order and the conversion rate is a count of converted orders over all orders. The founder's question is about people, so the ideal table has one row per customer. Those are two different tables, and answering the founder's question on the order table does something specific and silent: a customer who ordered forty times contributes forty rows, a customer who ordered once contributes one, and the answer you get describes the habits of heavy buyers wearing the label "our customers". Nobody sees this happen. The number looks completely ordinary.
So write the grain on the first line of the sketch, in words, before any column name. "One row per order, for orders placed between 15 September and 8 December." If you cannot write that sentence, you are not ready to ask anyone for data yet, and five more minutes with the pen is cheaper than the two days that follow.
Self-studyColumns come from slots4 min▶
The ideal table is not invented from nothing. Each of the four slots from session 1 demands a specific kind of column, and if you work through them in order you will have most of the sketch without any creative effort at all.
| Slot | What it demands | Daybreak example |
|---|---|---|
| Population | A column you can filter on, with one value per row, so that the group in the question becomes a row filter rather than a word | is_new, which turns "new subscribers" into orders where the customer had no earlier order |
| Outcome | The thing being measured, recorded on every row, in the units the question used | converted for marketing, order_value for the finance version of the same request |
| Timeframe | A date on every row, so the two windows in the question can actually be cut apart | order_date, covering the twelve weeks either side of the offer, not just the after period |
| Decision | A grouping column, because a decision is rarely taken for everybody at once and the answer usually has to differ by group | channel, since the offer can be repeated on email and dropped on social, which is a real spring decision |
The decision slot is the one people leave off the sketch, and leaving it off is how you end up with a single global number that nobody can act on. If the spring choice is per channel, an answer that does not carry a channel column is not an answer to the question that was asked, however correct it is.
Two more columns earn their place on most sketches even though no slot demands them: an identifier, so you can check whether a row is duplicated, and a date of record, so you can tell when a value was written rather than only what it says.
The gap list, and what to do with each kind of gap
5 min liveNow open the file. Go down the sketch, tick every column that exists, and mark every one that does not. What is left is the gap list, and it is worth much more than it looks, because a gap is not a complaint. Each kind of gap has a different cause, a different fix and a different person, and telling them apart takes about a minute per column.
LiveThree kinds of gap, three different replies4 min▶
Never collected. Nothing anywhere holds this value, because no system was ever asked to write it. The reply goes to whoever owns the system that would write it, it asks for collection to begin, and it names the date from which you need it. The cost of this gap is time: the answer arrives one collection period from now, not today, and saying that plainly on day one is what lets somebody plan around it.
Collected but not kept. The value existed and was overwritten, truncated or aged out of a retention window. This is the gap most worth chasing hard and fastest, because a copy is often sitting in an export on somebody's laptop, in a vendor dashboard that keeps ninety days, or in a monthly report nobody deleted. The reply asks who has a copy and how far back it goes, and it is urgent in a way the other two are not, because the window is still closing while you write.
Exists but not joinable. Another team has the column, under an identifier yours does not share. This is a plumbing question, not a data question, and the reply is one line: what is the shared key, and has anybody mapped these two before? The answer is frequently yes, and frequently from someone who tried it last year and can save you a week.
What none of these replies says is "the data is bad", which is the sentence that ends conversations. Every gap message has the same three parts: what is missing, what it would let you answer, and who can change it. Three parts, one line each, and the third part is the one that turns a complaint into a request.
Self-studyWhy ideal-before-actual is the whole point4 min▶
The order is not a formality. Open the spreadsheet first and something specific happens within about ninety seconds: you see which columns are there, your sense of what is askable rearranges itself around them, and the question you end up answering becomes the question that file supports. The original question is not rejected. It is quietly replaced, and the replacement feels like the original, because nothing was ever written down to compare it against.
Writing the ideal table first is the only way that substitution becomes visible. The sketch is a record of what you wanted made before you knew what you could have, and holding the real table against it is the moment the difference shows up as a list rather than as a feeling. That list is the gap list, and it exists only because the sketch came first.
Be straight about the standing of this, because it is craft rather than a research finding. No study shows that teams who sketch the ideal table produce better analyses than teams who do not, and nobody appears to have run one. What can be said is that every process model in the field independently orders the work this way: CRISP-DM, published by a cross-industry consortium in 2000, puts business understanding ahead of data understanding among its six phases, which encodes exactly this rule. Agreement between process models is not evidence of effectiveness, and it should not be presented as if it were.
The founder's question needs a column that records what a customer did in the renewal cycle after the promotion. The rows stop at week 24. There is no such column and there is no file that has one, because nothing was ever asked to record it. That is a never-collected gap, and the reply is one sentence: nothing records renewal behaviour after week 24, so this question gets answered in the spring rather than now, and here is who has to switch the recording on today for that to happen.
Proxies, and the honest way to use one
4 min liveSome gaps will not close in time, and the pressure then is to measure something nearby instead. That is a proxy, it is completely normal, and every organisation runs on them. What separates an honest proxy from a quiet fabrication is not the choice of substitute. It is whether the substitution is said out loud, in the same breath as the number.
LiveA proxy is a different question wearing the name of the first one4 min▶
When you cannot measure whether customers stayed, you measure whether they ordered again within twelve weeks. That is a reasonable thing to do. It is also, strictly, an answer to a different question, and the whole of the honesty here lives in one habit: you name the substitution in the same sentence as the number, every time, and never in a footnote.
Compare the two sentences. "Sixty-one percent of promotion customers stayed" is a claim about retention that nobody measured. "Sixty-one percent of promotion customers ordered again within twelve weeks, which is what we are using in place of whether they stayed" is the same number with the substitution attached, and the second version costs fourteen extra words. Those fourteen words are the entire difference between an honest proxy and a misleading one.
The reason it has to be the same sentence is mechanical rather than moral. Numbers travel and their footnotes do not. Your caveat survives one slide, maybe two, and then somebody pastes the figure into an email and the qualification is gone forever. If the substitution is inside the sentence, it travels with the number, and by the time it reaches the board the claim is still the claim you made.
Self-studyWhen a proxy is not acceptable3 min▶
When the decision is expensive and hard to reverse. A proxy is a loan against the truth, and the interest is paid by whoever acts on it. For a reversible weekly choice, borrow freely. For hiring a team, signing a year of spend, or telling a board that a channel works, the gap between the proxy and the real outcome is being converted into money that cannot come back.
When the proxy and the real outcome are known to diverge for the group you care about. This is the failure that looks most like success. Email opens are a defensible proxy for engagement across a whole customer base, and a terrible one for the subset who pay by standing order and never open anything. If the divergence lands precisely on the population in the question, the proxy is not weaker than the real thing, it points the other way.
When the substitution would not survive being said out loud to the person who asked. This is the most practical test of the three, and you can run it in your head in five seconds. Say the sentence with the substitution in it, to the founder, in your imagination. If it comes out as "we measured whether they opened an email, and we are calling that staying", you already have your answer, and no amount of careful wording fixes it.
Three ideal tables, three gap lists
★ 22 min · your own three questionsYour three rewritten questions from session 1, one sketch each. Pen first, file later, and the order is the part that does the work.
Decide what one row represents for each of the three questions, and write it as a sentence before any column name. An order, a customer, a customer-week, a ticket. Where two of your questions need different grains, note that, because it means two tables and not one.
Write the ideal columns from the four slots. Population needs something to filter on, outcome needs the thing being measured, timeframe needs a date on every row, decision needs a grouping column. Add an identifier so duplicates are detectable. Do not open anything yet.
Now open the real data and tick what exists. Go down the sketch in order and mark each column present or absent. Resist rewriting the sketch as you go; the columns you wanted and did not get are the entire product of this exercise.
Sort every missing column into never collected, collected but not kept, or exists but not joinable. If you cannot tell which, that is itself a question for somebody, and it is usually answered in one reply.
Write one message per gap, three parts each: what is missing, what it would let you answer, who can change it. Put the name in before you send. A gap addressed to nobody stays open indefinitely and then gets described as a data problem.
Sketch on paper, not in a spreadsheet. Opening a spreadsheet to draw the ideal table means opening a spreadsheet, and within a minute you are looking at column names that exist instead of column names you want.
Write the grain as a full sentence, with the date range in it. "One row per customer who placed a first order between 15 September and 8 December" is a grain. "Customers" is a word.
Include the columns you expect to be missing. The sketch is a wish list, and a wish list that pre-censors itself produces a gap list with nothing on it, which is the one outcome guaranteed to be wrong.
Chase the not-kept gaps first. Those are the ones where a copy still exists somewhere and the window is closing. The never-collected ones cannot get worse; the not-kept ones can.
Name a person, not a team. "Engineering" does not read messages. One human being does, and finding out which one takes a single question to whoever sits nearest the system.
Keep all three sketches. Session 3 opens the data behind the closest one and spends twenty minutes on it before calculating anything at all.
If you take one thing from tonight. Do not open the data until stage three. Everything before it is worth doing precisely because it was done in ignorance of what the file contains, and the ignorance is not a defect in the method, it is the method. Once you have seen the columns you cannot unsee them, and the sketch you draw afterwards will be a description of the file with a wish list's name on it.
Before session 3
25 min- Send the gap messages and note which ones get an answer and how fast. The pattern is informative: not-kept gaps usually come back within a day because somebody knows exactly where the copy is, and never-collected gaps take longer because the reply requires a decision rather than a lookup.
- Find one analysis in your organisation that used a proxy without saying so. A retention figure built on repeat orders, an engagement number built on logins, a satisfaction score built on survey responders only. You will not have to look hard, and the point is not to catch anyone out, it is to see how completely invisible the substitution is once the number is in a slide.
- Pick the question whose ideal table is closest to what exists. That is the one session 3 opens and looks at properly, and closest means fewest missing columns, not most interesting subject.
- Bring the actual file or table for that question, whatever state it is in. Not a summary of it, not a screenshot of the tidy tab, the real thing with the real number of rows.
Three questions
Clear all three and session 2 gets stamped in your passport.
1 · Why is "what does one row represent" worth asking before anything else?
A customer with forty orders contributes forty rows and one with a single order contributes one, so the answer describes heavy buyers while carrying the label "our customers". The number looks entirely ordinary when it comes out, which is exactly why the grain has to be settled in words at the top of the sketch. It is the commonest structural error in the field and the cheapest one to avoid.
2 · You have three missing columns. What does sorting them into kinds actually buy you?
Three kinds, three replies, three different people, and none of them is the sentence "the data is bad", which ends conversations rather than starting them. The not-kept gaps are the urgent ones because a copy often still exists and the retention window is closing while you write. Every message has the same three parts: what is missing, what it would let you answer, and who can change it.
3 · You cannot measure whether customers stayed, so you use "ordered again within 12 weeks" instead. What is the rule?
Numbers travel and their footnotes do not; a caveat survives one slide and then somebody pastes the figure into an email without it. Putting the substitution inside the sentence costs about fourteen words and means the qualification arrives wherever the number arrives. Proxies are legitimate and unavoidable, so the honesty has to live in how the answer is phrased rather than in whether a proxy is used.
What this session teaches, and where it came from
Every fact on this page, and its verification tier, is recorded in the course's source map.
Session 2 cheat sheet · pin this
You now know what you would need and what you actually have, which is more than most analyses ever establish. Session 3 opens the file behind your closest question and spends twenty minutes looking at it before calculating anything: how many rows, what is missing, what is duplicated, what is impossible. The problems that break every later step are visible in those first twenty minutes and essentially nowhere else, because once a number has been computed it looks like a number rather than like a symptom.