You cannot take it back

It would be easy to spend forty five minutes being appalled, and it would be a waste of the time. An append-only record is doing exactly what it was chosen to do, and session 5 makes the case for it properly. The useful posture tonight is the one a finance function takes with a fixed cost: name it, size it, put it in the right budget line, and stop being surprised by it every quarter.

What makes it awkward is that the cost is invisible at the moment of the decision and visible every month afterwards. Nobody experiences the loss of a correction path on the day the design is signed off. They experience it eighteen months later, in a read path that has accumulated a dozen rules nobody can safely remove, maintained by people who were not in the room.

• Append only. New records can be added, existing ones cannot be altered. Session 1 named it; tonight it has consequences.
• Annotation. A later record that says something about an earlier one. The only correction shape available, and it is not a correction.
• Supersede. The relationship an annotation asserts. It has meaning only if the reader applies it, and nothing makes them.
• Read path. Everything between the stored rows and the answer. This is where every correction ends up living, permanently.
• Restatement. Reissuing a published period on a corrected basis. Standard practice elsewhere, and it has no mechanism here.
• Erasure. Removing personal data on request. What any regime obliges is not settled here; the obligations live in learn-gdpr-with-phoebe.
• Opaque reference. An identifier that means nothing without a separate table, kept somewhere you can still destroy.
• The boundary. The line between what goes on the record and what stays off it. Part 2, and the only decision here that is really yours.

The mechanical part is simple enough that it hides the problem. A wrong record stays exactly where it is and a new record is appended saying what should have been recorded. The pair of them is now the truth, and neither one alone is.

Everything difficult follows from that last clause. A query written against the raw rows returns the wrong answer, and it returns it confidently, with no error, no null and no flag. The correctness of every figure now depends on a piece of logic sitting in the read path, and that logic has to be present in every read path: the warehouse view, the notebook somebody wrote in a hurry, the export a partner pulls, the query an auditor runs themselves because the data is public and they can.

That last one is the part people miss. When the rows are readable by outsiders, you do not control who reads them or how, so you cannot guarantee that your correction logic is applied. You can publish it, and publishing it is the right move, and it is still a request rather than a control.

A correction in a normal warehouse has a lifespan. The row is fixed, the incident is closed, and in three years nothing in the codebase remembers it happened. That is the property being given up, and it is worth naming precisely because nobody misses it until it is gone.

Here every correction is a permanent addition to the read path, and the reason it is permanent is not laziness. The wrong row is still there, still readable, and will still be read by somebody in three years, so the rule that supersedes it cannot be retired while the row exists, which is always. A read path with twelve supersede rules in it is not a badly maintained read path. It is a correctly maintained one, after twelve mistakes.

What a leader should take from this is a budget line rather than a lament. The maintenance cost of the reporting layer grows with the count of historical errors and never shrinks, so the thing worth managing down is the error rate at the point of writing, and the thing worth resourcing is the read path itself.

Restatement is the move a reporting function reaches for when a published period turns out to have been computed on a wrong basis. The period is recomputed, the new version is issued, the old version is withdrawn, and there is an established etiquette for saying so. It works because the published figures were an assertion by one party who can withdraw them.

Here the inputs were not asserted by you and cannot be withdrawn by you. You can recompute and reissue your own derived figure, and you should, but the rows the old figure was computed from are still sitting there, still public, and still produce the old figure for anyone who runs the obvious query. A restatement in this setting is a claim competing with a computation, and the computation is the one a sceptical reader will run.

There is a version of this that works and it is unglamorous: publish the derivation, version it, and make the superseding relationship part of what you publish rather than part of what you announce. Then the sceptical reader running their own query is running yours. It does not remove the old answer from the world, because nothing can, and it does give the new one somewhere to stand.

Switch the loader to retract and the wrong rows go to zero. That is worth pausing on, because it is the clearest demonstration of what the boundary actually buys. The warehouse is yours, so inside it the verbs from part 1 all still work: you can correct, you can restate, you can remove. The record upstream has none of them.

So the boundary is not a compliance diagram or a governance artefact. It is the line between the part of your estate where a mistake is recoverable and the part where it is not, and every item you move across it to the permanent side is a capability you are giving up for that item, forever, in exchange for something. Part of the job in a design review is making sure somebody says out loud what the something is.

Two honest notes about this box. The upstream event causing rows to be replaced is a chain reorganisation, which is taught properly on the practitioner track and returns in session 6 as a scoping question. And holding rows back until they settle is a separate defence from retracting them, with a different cost; conflating the two is a known trap and session 6 keeps them apart.

Most hard things in a data estate are hard in a way that yields to effort. A field is scattered across nine systems, so you find the nine, you write the deletes, you prove it. Expensive, tedious, done. The reason this one is different in kind is that no amount of effort produces a mechanism, because there is nothing to call and nobody to ask. The record does not have an owner who could remove it if they wanted to.

What obligations attach to any of this belongs in learn-gdpr-with-phoebe and is not settled here. What belongs here is the design consequence, which holds regardless of which regime you are under: if a capability might be required of you later and is structurally unavailable on the permanent side, the only place to provide it is on the other side of the boundary, and the only time to arrange that is before the first record is written.

In practice that means the lever people reach for is destroying the link rather than the record: keep the map from an opaque reference to a person in a store you control, and destroy the map. Whether that is a remedy or the appearance of one is the argument below, and it is a real argument.

One camp holds that nothing capable of being linked to a person belongs on a permanent public record, in any form, including an opaque reference. Their argument is that an opaque reference is only opaque against the linking ability available today. The record lasts indefinitely, linking ability improves, and session 2 and session 3 already showed what structure alone gives you without any external table at all. On this view, promising that a reference cannot be resolved is a promise about the future that nobody is able to make, and the design that avoids making it is the only one that stays honest.

The other camp holds that this collapses a real distinction and, taken seriously, forbids most of what the arrangement is for. A reference with no table behind it is not information about a person in any usable sense, and holding the table somewhere destructible gives you a genuine remedy with a genuine effect: destroy it and the linkage is gone from your side permanently. They also point out that the first position is not free. Move everything identity-shaped off and what stays on the record is too thin for a third party to check anything, which throws away the one advantage session 1 credited on the plus side and session 5 argues for at full strength.

Both camps have shipped systems and both have been right in different projects. The genuinely unresolved part is empirical and about the future: how resolvable a reference becomes over the life of a permanent record, which nobody knows. Notice that the two positions fail differently rather than one failing worse. The first fails quietly, by building something too thin to be worth the machinery, and that failure is survivable. The second fails loudly and late, when a reference that was opaque at design time stops being opaque and the record is still there. Which failure you would rather explain is a legitimate way to choose, and it is not the same answer for every product.