Scoping a project on this

Four of the six have a property worth noticing: a vague answer and a bad answer look the same from outside, and both mean the same thing in practice. If nobody can say what stays off the permanent record, the boundary has not been drawn. If nobody can name a reorg depth, the lag was chosen by default. If nobody can name the party who benefits from the metric, nobody has looked.

That makes these questions cheap to ask and hard to deflect, which is the point. You are not asking anybody to predict the future or defend a technology choice. You are asking whether a specific decision has been made yet, and the absence of an answer is itself the finding.

• The boundary. What goes on the permanent record and what deliberately stays off it. Session 4, and the first question.
• Counterparty. Any party that has to accept the record as authoritative. Session 5, and the second question.
• Unit of identity. What one row of your reporting counts. Addresses, entities or people, and they are three different numbers. Session 2.
• Error direction. Which way a method is wrong. One-sided error gives a bound, two-sided error gives a number you must trust.
• Reorg depth. How far back the recent history can be replaced. The number the fifth question is about, and the one nobody writes down.
• Finality lag. How long you wait before loading. A defence that works only if it is at least as deep as the reorg.
• Retraction. Removing already loaded rows when their source is replaced. A separate defence with a separate cost, and the two are easily confused.
• Paying metric. Any published number that causes money or resource to move. Session 3, and the sixth question.

Question one is the boundary from session 4, and the reason it goes first is that it is the only decision in the whole arrangement that is genuinely reversible before the build and genuinely irreversible afterwards. Everything else on this list can be improved later at some cost. What went on the permanent record cannot.

Question two is the two-question test from session 5 compressed into one, and it is the one people resent most, because it asks about relationships in a meeting that was supposed to be about technology. Ask it anyway, and ask it in the form that does the work: not "do you trust each other" but "who would keep the record if we did not do it this way, and would everyone accept them". A name that everyone accepts means the machinery is buying tolerance of a distrust that is not present, and session 5 was clear about what that costs.

If the first two come back well, the project may be worth doing. If either comes back vague, nothing further on the list matters yet, because you are scoping something that has not been decided.

Questions three, four and five are the standing operational load, and they are the three that land on a data team specifically. Three is session 2: what your reporting counts and which way it is wrong. The acceptance criterion is the shape of the answer rather than its accuracy, because a ceiling with a stated method survives being repeated by somebody who was not in the room, and a point estimate with a caveat does not.

Four is session 4, and it is the question that produces the most informative silence on the list. You are not asking for a policy. You are asking somebody to narrate an incident: a record is wrong, what happens next, who does what, and who ends up reading the wrong value anyway. Anyone who has drawn the boundary can do this in twenty seconds.

Five is the one with a number, which is why it gets part 2 rather than a sentence. The honest version of this question is not "have you thought about reorgs", which everybody answers yes to. It is "what depth are you designing for and where did the number come from", and then a lag at least that deep, with retraction discussed separately.

Session 3's finding is arguably the most severe on the list, so putting it sixth looks like an ordering mistake. It is deliberate. The first five are about whether the thing can be built and what it will cost to operate. The sixth is about whether the output will mean anything, and that question only becomes answerable once you know what is being published to whom.

There is a second reason. The first five can be answered by the people proposing the project. The sixth usually cannot, because the party who benefits from the metric is often somewhere else in the organisation, or outside it entirely, and asking the question in the design meeting is how you find out that nobody owns the answer. That discovery is worth more than the answer would have been.

The failure mode to watch for is a confident "nobody benefits, it is just a reporting metric". Session 3 measured what that reasoning costs: the conditions for manufactured identity are permanently present, the cost of one more of them is one more row, and the resulting rows do not trip anything. A metric that nobody is paid to move is a claim about the incentive landscape, and it needs the same evidence as any other claim.

Waiting and retracting are different mechanisms with different costs, and the box above puts them on separate controls for a reason. Waiting keeps rows out of the report until they are settled, so it costs freshness and it costs nothing else. Retracting lets rows in and removes them afterwards, so it costs nothing in freshness and it means your published figures move after publication.

They are easy to blur together because both end up with a clean warehouse, and blurring them produces a specific error: a shallow lag that appears to solve a deep reorg because the retraction was quietly doing the work. An earlier version of this bench made exactly that mistake and made a one block lag look like a fix for a three block reorg. The practitioner track spends a whole session on how that was caught.

The practical form of the question is therefore two questions in a trenchcoat, and you should separate them when you ask. What depth are we waiting for, and do we retract as well. A project that answers only the second has chosen to publish figures that move. A project that answers only the first has chosen staleness. Both are legitimate and neither is the default anybody meant to choose.

Five sessions have produced a consistent pattern that is worth stating plainly at the end rather than implying throughout. The decision to build on this is made once, quickly, by people who are evaluating a product idea. The consequences are continuous, and they arrive at a different desk.

That is not an accusation of bad faith, and treating it as one gets you nowhere. It is a structural feature of how the costs are shaped: the boundary decision looks like an implementation detail in the room where it is made, and looks like the whole job eighteen months later. The read path, the bound that has to be re-explained every quarter, the loader lag, the metric that no amount of engineering will fix.

The remedy is not louder objection. It is the six questions, asked before the decision rather than after, by the people who will be carrying it. That is the entire argument for spending forty five minutes on a list of questions.