learn-web3-with-phoebe / Leader session 5 of 6
Learn Web3 with Phoebe · Leader session 5 of 6

What actually gets better

Four sessions of costs earns the credit side a fair hearing, and it deserves one on the merits rather than as a courtesy. There are four things a shared append-only ledger does for a data team that a warehouse cannot do at any price, and each of them is worth having. There is also a boundary around all four, and it is sharper than the enthusiasts admit and narrower than the sceptics think. Tonight is both, at full strength, in that order.

🟠 Leader track Anyone who has decided this is all cost Anyone about to approve a consortium project 45 min
0-3 · Welcome 3-18 · The four 18-30 · The boundary 30-45 · Build-along and Q&A
Part 0

The sentence you wrote down, and the four it was probably one of

Session 4 asked you to write, in a sentence, the one thing this arrangement does better. Most answers land on some version of trust, which is the right instinct and too vague to do any work. Tonight splits it into four separate properties that are usually blurred together, because they have different costs, they fail in different circumstances, and a project can easily need one of them and none of the other three.

Live = presented in the 45 minutes Self-study = the full depth, after class 🔍 Every diagram zooms

★ What you walk out with tonight. Four genuine advantages stated in terms a data team can act on, with the mechanism drawn for each rather than asserted. The boundary that applies to all four at once, which is that every one of them is about shared state between parties who do not trust each other. A four cell picture of when the machinery is worth its cost and when a database with an audit log wins on every dimension including honesty. And the argument that actually decides most real projects, which is about the parties rather than the technology, left open because it is open.

LiveWhy "trust" is too coarse to plan with4 min▶

Trust gets used to mean at least four different things in the same meeting, and the ambiguity is expensive because the four have different remedies. Knowing where a figure came from is not the same as knowing that a record has not been altered. Being able to check something yourself is not the same as several parties sharing one version of a history. A project can have a hard requirement for exactly one of these and no interest in the rest.

Splitting them up also makes it possible to be honest about the cheaper option. For three of the four there is a conventional answer that works well when one condition holds, and naming the condition is more useful than arguing about the technology. That condition turns out to be the same one every time, which is part 2.

Real world

Larkfield's partner asked for a ledger and meant one specific thing: they wanted to recompute the reward figures themselves rather than receive them in a monthly file. That is one of the four. The other three were in the proposal because they came in the same box, and two of them were never going to be used.

Self-studyThe words that keep coming back4 min▶
  • Provenance. Where a figure came from and what produced it. The strong version is a derivation a stranger can run, not a document describing one.
  • Recompute. To arrive at the figure yourself from the same inputs. The difference between this and reading a report is the whole of the first advantage.
  • Shared history. One sequence of events that several parties read as authoritative, without any of them maintaining it.
  • The keeper. Whoever holds the copy that counts. In most arrangements there is one, and agreeing who it is can be the hardest part of the deal.
  • Integrity. Evidence that a record has not been altered since it was written. Checkable by the reader here, asserted by the publisher elsewhere.
  • Audit trail. A record of what happened and when. The property that matters is whether the party being audited can revise it.
  • Distrusting parties. Parties whose interests differ enough that none of them will accept another's copy. The condition all four advantages depend on.
  • Trust minimising. Machinery that reduces how much any party has to take on faith. It is not free and part 2 says what it costs.
Part 1 · covers the four advantages

Four things a warehouse cannot do, at any price

15 min live

Each of these is a genuine capability and none of them is a matter of degree. In every case the conventional arrangement can get close and then stops at the same place, which is the point where somebody outside has to take somebody inside at their word. That gap is not a quality problem and it does not close with more effort or better tooling.

Four capabilities, each drawn on both sides of the same line. 1 · PROVENANCE YOU CAN RECOMPUTE USUALLY publisher the figure the reader has to take it ON A SHARED LEDGER the rows the reader runs the derivation themselves A derivation a stranger can run beats a document describing one, and no warehouse offers it. 2 · A HISTORY WITH NO KEEPER USUALLY party A party B the keeper somebody has to own the copy that counts ON A SHARED LEDGER one history nobody owns it, so nobody has to be agreed on Agreeing who keeps the record is often the hardest clause in the deal. Here there is no such clause. 3 · INTEGRITY THE READER CHECKS USUALLY we checked it an assertion, with nothing behind it to test ON A SHARED LEDGER the rows the reader verifies it without asking anyone Checkable by whoever doubts it, rather than promised by whoever would be embarrassed by the answer. 4 · A TRAIL NOBODY CAN REVISE USUALLY whoever keeps the log can revise the log ON A SHARED LEDGER no party can revise it, including you The property that matters is whether the party being audited can quietly edit the evidence. Here, no. All four are about shared state between parties who do not trust each other. That is the boundary, and it is a hard one. Inside one organisation with a keeper everybody accepts, a database with an audit log delivers every one of these more cheaply, and it can also correct a row. What the machinery buys is tolerance of distrust, and sessions 1 to 4 are what it costs.
🔍 Click to zoom - on the left of every divider somebody outside has to take somebody inside at their word. That gap does not close with better tooling, which is why these four are capabilities rather than quality levels.
LiveProvenance a third party recomputes4 min▶

Every serious data function already does provenance. Lineage is captured, transformations are documented, a figure can be traced back through the pipeline to its inputs. That work is real and it is valuable, and it produces something with a hard limit: an account of the derivation, offered by the party whose figure it is.

The strong version is different in kind rather than in quality. When the inputs are readable by anyone and the derivation is published, a doubter does not read your account of how the figure was produced. They produce it. If they get your number, the question is settled without your involvement; if they do not, you have a discrepancy with two candidate causes, both of which are findable.

Notice what that removes. It removes the escalation, it removes the audit engagement, it removes the meeting in which two teams compare two numbers neither can inspect, and it removes the situation where the party with the better relationship wins the argument. Those are not small operational costs, and a warehouse cannot remove them at any budget, because the bottleneck is not data quality. It is that the other party has to be granted access and then has to believe the grant was complete.

LiveA shared history, with nobody holding the pen4 min▶

This is the advantage that sounds most abstract and is most often the actual reason a project exists. When several parties need one version of a sequence of events, somebody normally maintains it, and that somebody has to be agreed. Negotiating who keeps the record can take longer than building anything, because the keeper gets to resolve every ambiguity, sees everybody's data, and cannot be removed without unwinding the arrangement.

The usual alternative is worse rather than better: every party keeps its own copy, the copies disagree at the edges, and a reconciliation function exists permanently to argue about the differences. Most people reading this have staffed one. The cost is continuous and it is mostly paid in senior time.

A shared append-only history removes the role rather than assigning it well. There is no keeper to appoint, no keeper to audit, and no keeper to renegotiate when one party is acquired by another. That is a structural answer to a political problem, which is rare enough to be worth a lot when the problem is genuinely political.

LiveIntegrity checked rather than asserted, and a trail nobody can revise4 min▶

The third and fourth run together and are usually confused with each other. Integrity is about whether a record you are reading now is the record that was written. An audit trail is about whether the sequence of what happened can be reconstructed later. A conventional system can do both to a high standard, and in both cases the assurance ultimately rests on the party operating the system.

The honest way to put the difference is this: a well run audit log is excellent evidence against accident and mediocre evidence against the log's own operator. That is not a criticism of anyone's controls. It is a structural fact about who holds the write access, and every auditor knows it, which is why audit is built on sampling and corroboration rather than on reading the log.

Here the operator cannot revise it either, and neither can you, which is exactly the property sessions 1 to 4 spent so long complaining about. That is the trade in its clearest form. The thing that makes correction impossible is the same thing that makes quiet revision impossible, and you do not get to buy one without the other.

Self-studyWhat a warehouse gets close to, and where it stops5 min▶

It is worth being precise about the near misses, because a good engineer will offer each of them and each is a reasonable answer under the right condition. For provenance, a conventional system can publish its transformations and grant read access to its inputs, which gets a long way. It stops at completeness: the outside party can recompute from what they were given and cannot establish that what they were given was everything.

For a shared history, a conventional system can appoint a neutral keeper, and a well chosen one works for years. It stops when the neutrality is challenged, which tends to happen at exactly the moment the record matters most, and there is no way to settle the challenge from inside the arrangement.

For integrity and the audit trail, a conventional system can sign records, write to storage that resists modification, and ship the log to a third party. Each of these raises the cost of quiet revision substantially. All of them are configured, operated and can be reconfigured by the same party whose behaviour is in question. That is the stopping point in all four cases and it is always the same one: somewhere in the chain, an outsider is relying on an insider's good faith. Part 2 is about whether that reliance is actually a problem in your situation, because very often it is not.

Part 2 · covers the boundary

The boundary: shared state, distrusting parties, and nothing else

12 min live

The four advantages have one condition in common and it is not a technical one. Every single one of them is about state that several parties share, where no party will accept another's copy. Take away either half of that and the case collapses quickly, in a direction that is uncomfortable for anyone who has already chosen.

Two questions decide it, and neither of them is about technology. PARTIES MOSTLY TRUST EACH OTHER PARTIES DO NOT TRUST EACH OTHER THE CONTESTED CELL A database with an agreed keeper and an audit log does all four more cheaply, and it can still correct a row. Whether the machinery is worth paying for here is the argument below. WHERE IT EARNS ITS COST No party can be the keeper, because agreeing on one is the thing that cannot be done. A history nobody owns is the only shape left, and the four properties come with it. NOT A CANDIDATE A database. None of the costs in sessions 1 to 4 buys anything here, because there is nobody outside who has to be convinced of anything. NOT A CANDIDATE EITHER Distrust inside one organisation is an access control and separation of duties problem, and that has its own well understood answers. STATE IS SHARED BETWEEN PARTIES STATE IS INSIDE ONE ORGANISATION One cell of the four is a case for the machinery, and it is the one with no keeper anybody would accept. That is not a small cell. Settlement between competitors, shared registries, consortium reporting and multi-party supply records all sit in it. It is also not most proposals, and most proposals sit in the top left, which is why the argument below decides things.
🔍 Click to zoom - the amber cell is not the wrong answer. It is the cell where two reasonable people look at the same facts and choose differently, and where most real decisions are actually made.
Live The first advantage, in this page. Larkfield publishes a figure; you recompute it from the same generator rather than taking it.

...

...
LiveYou just did the thing that cannot be bought4 min▶

Switch the published figure to the inflated version and read the last row. You did not escalate, you did not request access, you did not ask anyone to confirm anything, and you did not have to be trusted by Larkfield in order to do it. You ran the derivation and compared. That is the whole of the first advantage and it fits in one interaction.

Notice also what it did not do. Catching the discrepancy does not tell you who was at fault, whether it was a definitional difference or a mistake, or what the right figure is. It tells you that two numbers do not agree and that both of them can be produced by anybody. What follows is still a conversation, and it is a conversation between two parties who can both see the evidence, which is a substantially different conversation from the one that happens when only one of them can.

The honest caveat is the one from part 1 and it applies here too. This works because the inputs and the method are both available. If Larkfield published the figure and kept the method to itself, as session 3's second camp would have it do, you would be back to taking the number. The advantage is not a property of the ledger alone; it is a property of the ledger plus a published derivation, and the second half is a choice somebody makes every time.

LiveInside one building, a database wins on every count4 min▶

This is the part that gets left out of the enthusiastic version and it needs saying plainly. Where the state lives inside one organisation, and there is a keeper everybody accepts, a relational database with a well designed audit log gives you provenance, a single history, integrity checking and a trail. It gives them faster, cheaper, with tooling everybody already knows and staff you can already hire.

It also gives you something the ledger cannot: the ability to fix a row. Everything in session 4 is a cost you would simply not be paying. No annotation-only correction path, no read path accumulating rules forever, no restatement with nothing to restate, no structurally unavailable erasure.

So the comparison is not close in that setting, and a data leader who chooses the ledger there has usually done so for a reason that is not about the four advantages. Being able to say that out loud, with the four advantages in hand rather than as a general scepticism, is what makes the objection land with people who have already fallen in love with the idea.

Self-studyWhere serious people disagree5 min▶

The live argument is about the contested cell: shared state between parties who mostly do trust each other, which is where a large share of real proposals actually sit. One camp says the machinery is not worth paying for there, and their case is concrete. Trust that exists is cheap to use and expensive to replace with mathematics. You would be buying the four advantages and paying for them in the three costs of sessions 1 to 4, permanently, to solve a problem the parties are currently solving by picking up the phone. They point at the long record of consortium projects that were technically finished and never used, where the quieter option kept winning because it was quicker for everyone involved.

The other camp says trust is a snapshot and the arrangement outlives it. Parties who trust each other today are one acquisition, one change of management or one bad quarter away from not doing so, and the machinery is much harder to retrofit than to start with, because the history you would want to have been keeping is the history you did not keep. They also make a sharper point about the phrase "accepted keeper": in most arrangements the keeper is the largest party, and the smaller ones accept it because the alternative is not being in the arrangement. Calling that trust flatters it. On this view, adopting the machinery while relations are good is buying an option on their deterioration, at a price you can only pay in advance.

Both camps are staffed by people who have run these programmes and both can name projects that vindicate them. The disagreement is not resolvable from the outside, because it turns on a forecast about a specific set of relationships over a specific horizon, which is exactly the thing nobody has data on. What can be said is which way each side fails. The first fails by being caught without a shared record on the day the relationship changes, which is recoverable and slow and humiliating. The second fails by carrying sessions 1 to 4 as a permanent cost for a risk that never materialises, which is survivable and quiet and never shows up as a failure at all. Leaders systematically under-count the second because nothing ever goes wrong, and that asymmetry is worth naming before you choose.

Part 3 · build-along

Put a real project in one of the four cells

15 min hands on

No code. Six steps on a project you are actually being asked about, said out loud in the room. The aim is to leave with one project in one cell, and with the specific advantage it needs named rather than all four assumed.

Name the four without notes

Recomputable provenance, a history with no keeper, integrity the reader checks, a trail nobody can revise. If you can only get three, the one you lost is usually the second, which is the one that most often turns out to be the real requirement.

Pick a live proposal and find its cell

Two questions only: is the state shared between parties, and would any of them accept another's copy. Put it in a cell out loud. Nobody gets to say it is between cells at this stage.

Name which of the four it actually needs

Often exactly one. Write down which, and then write what happens if the other three are absent. If the honest answer is nothing, you have just narrowed the project considerably.

Catch the discrepancy in the live box

Switch the published figure to the inflated version and watch the comparison row. Then say in one sentence what you would have had to do to reach the same conclusion if the inputs had been private.

Argue the cheaper option properly

For your project, make the case for a database with an agreed keeper and an audit log. Say who the keeper would be and why the others would accept them. If you cannot name a keeper the others would accept, you have found your answer.

Say which way you would rather fail

Caught without a shared record when relations change, or carrying four sessions of cost for a risk that never arrives. Pick one and say why. Keep the answer, because session 6 turns all of this into questions you ask before agreeing to build.

The habit to keep. When somebody proposes this machinery, ask which of the four advantages the project needs and then ask who the keeper would be if it did not use it. The second question does more work than the first. If a keeper can be named and the others would accept them, you are in the contested cell and the decision is about a forecast rather than about technology. If nobody can be named, you are in the cell where the cost is worth paying and the conversation can move on.

Check yourself

Three questions

A conventional warehouse publishes its transformations and grants an outside party read access to its inputs. Which part of the first advantage does that still not deliver?

That arrangement gets a long way and stops at the same place all four do. Somewhere in the chain an outsider is relying on an insider's good faith, here about the completeness of the grant, and no amount of tooling closes that.

A team wants the machinery for a dataset that lives entirely inside one organisation, with an audit requirement. What is the strongest honest response?

All four advantages are about shared state between distrusting parties. With one organisation and an accepted keeper, both halves of the condition are absent, so the comparison is not close and the ability to fix a row is a decisive extra.

Two experienced leaders disagree about a shared project between parties who currently get on well. What is the honest description of that disagreement?

One side pays permanently for a risk that may never arrive, and that failure is quiet and never gets recorded as a failure. The other is caught without a shared record on the day the relationship changes, and that failure is loud. Choosing which you would rather explain is a legitimate method, and the asymmetry in how visible they are is worth naming out loud.

Before session 6

Homework

Twenty minutes, on the project you placed in step 2 and on everything five sessions have handed you.

Write the two-question test down

Is the state shared, and would any party accept another's copy. Then apply it to two more things your organisation is considering and write which cell each lands in. Bring any that land in the top left.

Draft the questions you would ask

From sessions 1 to 5, write the questions you would want answered before agreeing to build on this. Aim for six and do not polish them. Session 6 builds its own list and the interesting part is what you have that it does not.

Covered tonight

What this session covers

✓Provenance a stranger recomputesA derivation somebody else can run, rather than an account of one offered by the party whose figure it is.
✓A shared history with no keeperThe role is removed rather than assigned well, which is a structural answer to a political problem.
✓Integrity the reader checksCheckable by whoever doubts it, instead of asserted by whoever would be embarrassed by the answer.
✓A trail nobody can reviseIncluding you, which is the same property session 4 spent the whole evening complaining about.
✓The boundary around all fourShared state, distrusting parties. Remove either half and a database with an audit log wins on every dimension.
✓The four cell testIs the state shared, and would any party accept another's copy. Two questions, neither of them technical.
✓The argument about the contested cellShared state and existing trust, argued both ways and left open, with each side's failure mode named.
◐Turning all of this into scoping questionsStarted tonight in the homework. Session 6 builds the list and pairs each question with what would change your mind.
○How the integrity guarantee is constructedHashes, blocks and consensus are assumed from learn-blockchain-with-phoebe rather than repeated.
○What any law or regulation requiresNot stated here in any form. The obligations live in learn-gdpr-with-phoebe.
○Anything about price, markets or holdingsOut of scope by design, and nothing here is advice of any kind.
Take this with you

Cheat sheet

The one sentence. Four genuine advantages, and all four are about shared state between parties who do not trust each other.
Advantage 1. Provenance a third party recomputes instead of trusting. Removes the escalation, not just the doubt.
Advantage 2. A shared history with no keeper to appoint, audit or renegotiate. A structural answer to a political problem.
Advantage 3. Integrity the reader checks, rather than the publisher asserts.
Advantage 4. A trail nobody can quietly revise, including you. The same property that makes correction impossible.
The boundary. One organisation, accepted keeper: a database with an audit log does all four more cheaply and fixes rows too.
The two questions. Is the state shared, and would any party accept another's copy. Neither is technical.
The open argument. Shared state plus existing trust. One side pays forever for a risk that may not come, the other is caught without a record when it does.