The incident package

A postmortem written from memory has three problems that no amount of good intent fixes. The times are wrong, because nobody at 02:00 was writing them down. The shape is different every time, because each author decides what matters. And it can be argued with, because a recollection is an opinion about what happened. A timeline produced by a query has none of those problems: the timestamps were recorded by machines at the moment, the six fields are the same for every incident, and the only argument available is about the query, which is a better argument to have.

The partition log from session 1 and the alert log from session 3 already hold everything the timeline needs. The package is a view over two tables plus three lines a human writes: conditions, actions, and who reads the stale data.

• First bad partition - the earliest partition hour in the incident that fails any of freshness, correctness or coverage. Partition log.
• Page time - when the alert fired. Alert log. Under burn-rate alerting this is the first bad hour of the incident; under threshold paging it is the same moment, followed by thirteen more on night 9.
• Acknowledgement - when a human took the page. Alert log. Page to acknowledgement is the number the rota design in leader session 3 is judged on.
• Safe state reached - when the runbook's safe-state step completed: downstream refresh paused, partitions marked stale, readers told. Alert log, written by the runbook step itself.
• Last bad partition - the latest partition hour in the incident that failed. Partition log.
• Backfill complete - when the last bad partition was replaced by a good one. Partition log, the backfilled_at column session 5's idempotent job writes.

The partition log gives the first bad partition, the last bad partition and the backfill completion. The alert log gives the page, the acknowledgement and the safe state. Join them on the incident window and the timeline is a single row. Because it is a query, night 4's timeline has the same six columns as night 9's, and the leader reading five packages can compare page-to-acknowledgement across all of them without translating five different prose styles.

The elapsed times fall out by subtraction: an hour and four minutes from the first bad partition to the page, because the partition's deadline is the hour plus sixty minutes of grace and the detector ran four minutes after; seven minutes to acknowledgement; twenty-nine minutes to a safe state. Those three intervals are what the rota and the runbook are for, and the timeline is the only place they are measured.

"Severe" and "major" are not numbers, and a budget conversation needs a number. The impact section states how many partitions went bad and what share of the error budget that was at the current target. Night 9 was 14 partitions. At a 97 percent target the month's budget is 21.6 partitions, so night 9 alone spent two thirds of it, and with night 4's three already gone the budget was exhausted six nights later on night 15. Stated that way, the outage is not a story; it is the reason the settlement team was promised something the pipeline could not keep.

The second half of impact is who read the stale data. The lineage table lists which downstream jobs read which partitions, so the package can say "the settlement report read all 14 stale partitions; the seller payouts batch read 6 before the backfill landed" instead of "downstream reports were affected".

An action is a change to the system, the runbook or the contract that would have made the incident shorter, smaller or impossible. It carries three fields and is rejected without any one of them:

• Owner - one named person, not a team. A team owning an action is the same as nobody owning it, because everybody on the team assumes somebody else is on it.
• Date - a day, not a sprint or a quarter. The set review in a month checks the date against the calendar.
• Verifiable when done - a check that returns true or false: a row exists in the contracts table, a runbook step exists and was dry-run, a query returns a count. If nobody can tell whether it is done, it is not an action.

Two or three actions per package is normal. Eight is a sign the package is listing everything that could be better rather than what would have changed this night.

Night 9's upstream feed went down for twelve hours. That is a fact, and it is where a root-cause postmortem stops: the cause was upstream, the fix is theirs, closed. But the same feed had been flaky for a year without a 14-partition incident, so the outage alone does not explain the night. Two more conditions were true. There was no dependency SLO with the source team, so nobody at Harborlight had a number for how late the feed was allowed to be, and nobody at the source knew a settlement report depended on it. And the runbook stopped at diagnosis: it told the on-call how to confirm the feed was down and nothing about what to do next, so the safe state took twenty-nine minutes of improvisation.

Each condition maps to an action Harborlight can own. The outage maps to a fix another team owns and may never make. Contributing conditions are not a softer way of saying root cause; they are the list of things you can actually change.

Run every proposed action through three questions: who, by when, and how will anyone know. "Add a coverage check on shard count, owner Mara, by the 20th, verified by a row in the indicators table" passes all three. "Be more careful with upstream changes" fails all three, and it is the most common action in postmortems everywhere, because it feels like accountability and costs nothing.

A wish in an actions list is worse than an empty list, because it lets the package close. The set review a month later finds it unverifiable and the same conditions still true.

The package is the input to the postmortem review on the leader track, and the reader there was asleep on night 9. Everything they need has to be on the page, which changes how it is written:

• No pronouns without names. "We paused the refresh" becomes "the on-call (Tomas) paused the settlement refresh at 02:40".
• No system nicknames. The feed has a table name and an owning team; the package uses both.
• Impact before narrative. The leader reads the budget share first and decides whether to read the rest. Fourteen partitions and two thirds of the budget earns the rest.
• Blameless by construction. The timeline is a query and the conditions are properties of the system, so there is no sentence in the package that is about a person's judgement. That is what makes people willing to write the honest version.

Because the packages land in a table with the same columns, the five incidents of the month can be queried together. Night 21's flapping connector and night 27's partial load look unrelated in their own packages: one is a network problem, the other a source that shipped two shards of three. Read together, both list the same contributing condition: no coverage check on shard count, so partitions landed on time, reconciled, and missing a third of their rows, and nothing noticed for hours. One action, owned once, closes a condition that two separate packages would each have asked for separately or not at all.

The set review is also where actions are checked against their dates. An action past due with no verification is a wish that was well dressed.