learn-blockchain-with-phoebe / Practitioner session 3 of 10
Learn Blockchain with Phoebe · Practitioner session 3 of 10

Breaking the chain

Verification does not return a verdict, it returns a report. Two fields of that report do the work: which block failed, and what sort of failure it was. Tonight the Harbourgate register is edited in front of you, the report is read properly at each step, and the attacker is allowed to keep repairing until the register verifies clean again. It takes four blocks and, at difficulty 0, a fraction of a millisecond.

🟡 Foundations Engineers and data people Anyone who will audit a register 45 min
0-4 · Welcome 4-14 · What verify walks 14-30 · Two stages, live 30-45 · Reforge and build-along
Part 0

The attacker already has write access

Assume nothing exotic. No hash function is broken, no secret is guessed and nothing is stolen. Somebody who is already allowed to write to the Harbourgate register decides that block 2 should read differently, and they are patient enough to keep going after the first thing they try fails. Every result below comes from running that against the same six block register assembled in session 2, at difficulty 0.

Live = presented in the 45 minutes Self-study = the full depth, after class 🔍 Every diagram zooms

★ What you walk out with tonight. The three fields verification returns and what each is for. The two failure kinds, and what each one tells you about what the attacker did and where to look. And the honest ending: a register with no work attached to its blocks can be rewritten completely and will then pass every check you can run on it.

Self-studyThe words that will keep coming back4 min▶
  • verify. Recompute every block's hash and compare every block's prev against the block above. Returns ok, problems and firstBad.
  • problem. One entry in that list: an index and a kind.
  • kind hash. The block's stored hash does not match a fresh hash of its own fields.
  • kind link. The block's prev does not match the hash of the block above it.
  • firstBad. The index of the first problem found, or nothing when there are none.
  • tamper. Change one block's data and leave every stored hash alone, which is what editing the database does.
  • hashBlock. Rehash one block from its own fields. The attacker's repair tool, and the auditor's.
  • reforge. Redo a block and every block after it, so the register is internally consistent again.
Part 1 · covers verification

Two checks per block, and three fields back

10 min live

Verification is a walk down the register doing two comparisons at each stop. Recompute this block's hash from its own five fields and compare it with the hash the block stores. Then compare this block's previous hash field with the hash of the block above. Block 0 has no block above it, so only the first comparison runs there.

What verification does at every stop CHECK 1 · RECOMPUTE THE HASH block 2, five fields index, timestamp, data, prev, nonce SHA-256 recomputed hash f325e5d96bee6a1d compared with the stored hash a difference is kind hash CHECK 2 · COMPARE THE LINK block 1 hash 86202a0bab83df26 block 2 prev field 86202a0bab83df26 the same 64 characters, or the link is broken a difference is kind link What comes back ok, a list of problems, and firstBad. Every problem in the list carries an index and a kind, and those two fields between them say which block failed and what sort of failure it was. Both checks run on every block, so the list is not limited to one entry. Reading only ok throws away everything that makes the report worth having.
🔍 Click to zoom - the first check asks whether a block is internally consistent. The second asks whether it agrees with its neighbour.
LiveThe boolean is the least useful thing in the report4 min▶

A verifier that returns true or false tells you a register is broken and leaves you to find out how. This one returns the index of every failure alongside a kind, and the kind is what turns a failed check into an account of what happened. Two registers can both fail with exactly one problem and be the result of completely different actions.

So the habit worth building is to read the problem list before the boolean. If you only ever look at ok, you have built a smoke alarm with no address on it.

LiveWhy block 0 gets one check and not two3 min▶

The link comparison needs a block above, and block 0 has none, so the walk skips it there. Its previous hash field holds 64 zero characters that nothing computed and nothing can check, which means block 0 can be rehashed and it will be internally consistent whatever it says.

The consequence is worth stating before the attack begins. An attacker who rewrites from block 0 forwards is not detected by anything in this report, because there is no earlier value for the report to disagree with. What catches that attacker is somebody else holding a copy, which is a different mechanism entirely.

Self-studyWhere the disagreement really is4 min▶

One camp says a verifier should stop at the first failure. After the first mismatch nothing downstream is trustworthy, a long list of consequent problems buries the one that matters, and an operator who is shown forty red rows will read none of them. On that view firstBad is the answer and the rest of the list is noise dressed as thoroughness.

The other camp says the pattern of problems is the evidence. One hash mismatch, one link mismatch, a hash and a link together, or a run of them down the register are all different situations, and truncating at the first one throws the forensics away to save a reader some scrolling. On that view a verifier that stops early has decided for you what the incident was.

The engine on this page declines to choose: it returns the whole list and the first index as separate fields, so the same call serves the operator and the investigator. That is a reasonable answer and it is not a free one, because a register with a million blocks and a systematic fault produces a list nobody can hold, and then somebody has to decide what to truncate after all.

Part 2 · covers the two stages

The edit, and the repair that moves the failure

16 min live

Stage one edits the entry and touches nothing else, which is what changing a row in a database does. Stage two is what an attacker does next once the first attempt is caught: recompute the edited block's hash so the block agrees with itself. Both stages produce exactly one problem, and the two problems are not the same problem.

One problem each time, and they say different things STAGE ONE · the entry is edited, hashes untouched 1 hash matches, link matches 2 data changed, stored hash left alone 3 link still matches, nothing moved verify() returns 1 problem · index 2 · kind hash STAGE TWO · the attacker rehashes block 2 1 hash matches, link matches 2 now agrees with itself, edit and all 3 prev holds the hash block 2 used to have verify() returns 1 problem · index 3 · kind link One problem before the repair. One problem after it. The count did not improve and the index moved forward by one. The kind changed from hash to link, and that change is the part saying what the attacker did rather than only that something somewhere is wrong.
🔍 Click to zoom - the two panels report the same number of problems. Anyone counting problems alone learns nothing from the repair.
Live The real engine, in this page. Edit block 2, then let the attacker keep repairing and watch where the failure goes.

...

...
LiveFollow the repair to the end of the register4 min▶

Press the repair button repeatedly and the report alternates. A hash mismatch at some block, then a link mismatch at the next one, then a hash mismatch there, and so on. Each repair satisfies the check it was aimed at and breaks the next one, because every fix changes a value that a later block already copied.

The alternation stops in one place only, which is the last block. Nothing copies the last block's hash, so there is no later check left to break. That is the mechanical reason a rewrite has to run all the way to the end of the register, and the reason the position of the edit, rather than its size, decides the workload.

LiveWhat tamper deliberately does not touch3 min▶

The tamper call changes one block's data field and copies every other field across untouched, stored hash included. That is not a weakened attack, it is the realistic one: somebody with database access edits a row, and the row does not know it is part of a chain. The stored hash sits there as a witness to what the row said when it was written.

This is also why a register whose hashes are recomputed on read is worth nothing. If the verifier rehashes a block and stores the result, every edit repairs itself and the first check can never fail. The stored hash has to be the old one for the comparison to mean anything.

Self-studyWhy the entry text was capitalised rather than reworded3 min▶

The edit used here turns one word of block 2 into capitals and changes nothing else about the length or the shape of the line. That is deliberate. A reader scanning the register sees almost the same sentence, and every check still fails as hard as it would for a rewritten paragraph, because the digest has no notion of how large a change is.

It follows that there is no such thing as a small edit to hash protected data. A typo correction and a fabricated inspection result produce failures of identical severity, which is convenient for detection and inconvenient for anybody who has to run a register where legitimate corrections happen.

Part 3 · covers the forensics

The kind tells you what was done, the index tells you where

7 min live

Two kinds, two accounts. Neither of them needs access to the data, the logs or the attacker. Both are read off the shape of the failure, which is what makes them useful in the hour after somebody says the register looks wrong.

Reading one problem without reading the data KIND WHAT THE ATTACKER DID WHERE THE EDIT IS WHAT TO ASK NEXT hash at N Edited block N in place and did nothing else at all. In block N itself. Its stored hash is the old one. Which field of block N moved, and who could. link at N Rebuilt block N minus 1 and stopped before its successors. One block above, at N minus 1. Block N kept the old hash. Whether the repair stopped there or went on. The index of a link problem points one block upwards at the edit. A register that verifies clean says only that nobody stopped halfway. Somebody who rebuilds every block from the edit to the end of the register leaves no problem at all, which is why a second copy held by somebody else is the other half of the check rather than an optional extra.
🔍 Click to zoom - both rows are read from the report alone. Neither of them required looking at what block 2 now says.
LiveReading a report you did not generate4 min▶

Somebody hands you a report saying one problem, index 4, kind link. Without seeing the register you can already say that block 3 was rebuilt, that whoever did it did not continue past it, and that blocks 0 to 2 were left alone. You can also say that block 3's own hash is consistent with block 3's current contents, because otherwise there would be a hash problem at 3 as well.

What you cannot say is what block 3 now contains, whether the change was malicious, or whether it was a botched migration. The report is about structure. Attributing intent to it is the commonest way these findings get misreported upwards.

LiveTwo problems at once, and what that looks like3 min▶

Both checks run at every index, so a single block can contribute a hash problem and a link problem in the same pass, and several blocks can contribute their own. A list with many entries usually means something structural rather than something surgical: a failed load, a truncated copy, two writers with different field orders.

A list with exactly one entry is the interesting case, because somebody was careful and stopped one step too early. That is the shape both stages above produce, and it is the shape worth recognising on sight.

Self-studyWhat the report cannot see3 min▶

Verification compares the register with itself. It has no second opinion, so anything consistently applied across the whole register is invisible to it: an entry that was never written, a register rebuilt from block 0, a set of timestamps that were all wrong at the moment of writing.

This is why the useful question after a clean report is not whether the hashes match but who else holds a copy and when they last compared. A clean report from a single copy is a statement about one file, and treating it as a statement about history is the mistake the whole subject invites.

Part 4 · covers the rewrite

Reforge, and a register that passes every check

6 min live

The alternating repair is the slow way to do it. Reforging from the edited block does the whole thing at once: redo that block and every block after it, carrying each new hash forward into the next block's previous hash field. At Harbourgate that is 4 blocks, 2 through 5, and afterwards verification has nothing to report.

reforge(chain, 2, 0) Blocks before the edit are never touched. Everything from the edit to the end is rebuilt. BLOCK 0 untouched original hash BLOCK 1 untouched original hash BLOCK 2 edited, redone new hash BLOCK 3 redone new prev, new hash BLOCK 4 redone new prev, new hash BLOCK 5 redone new prev, new hash 4 blocks, 2 through 5. Each hash is computed once, because difficulty 0 asks for no search. verify() on the result: ok, and an empty problem list. The register is internally consistent again, and block 2 now says something it never said. Detection was never the weak part of this arrangement. What is missing is any reason for the attacker to stop. Four hash computations is not a deterrent, and no amount of care in reading the report changes that, because after a full rewrite there is no report to read.
🔍 Click to zoom - the two untouched blocks on the left are the only reason this rewrite was not the whole register.
Live Reforge the edited register from any block, timed in your own browser at difficulty 0.

...

...
LivePosition is the whole cost, and it is the wrong way round4 min▶

Reforging from block 5 redoes one block. From block 2 it redoes four. The workload is the distance from the edit to the end of the register and has nothing to do with how much text was changed, so the oldest entries are the most expensive to alter and the newest are nearly free.

That ordering is useful and it is also the opposite of what most disputes need. Arguments are usually about something recorded recently, and recent entries sit at the cheap end. A register that has only just written the entry you are relying on is protecting it with the least work it will ever have.

LiveWhat the timing in the box is and is not3 min▶

The elapsed figure is measured in your browser on this run, on whatever machine you are holding, and it will differ between two presses on the same laptop. It is quoted here as an order of magnitude and nothing finer, because the point survives any reasonable variation: a handful of hash computations completes faster than a person can notice.

Do not carry that number anywhere. Session 4 measures attempts instead, which is the quantity that does not depend on the machine, and it does it over many runs rather than one.

Self-studyWhat actually stops the rewrite at difficulty 03 min▶

Nothing inside the register does. What stops it is everything outside: somebody else holding a copy of the old version, a published root that was recorded somewhere the attacker does not control, an append only store, backups nobody with write access can reach.

All of those are ordinary operational controls, and they are doing the work that people often attribute to the chain. The structure contributes one thing, which is that comparing two copies is a one line check rather than a reconciliation project. That is worth having and it is not the same as making the register hard to change.

Part 5 · build-along

Run both stages yourself

15 min hands on

Console open on this page. The same register, the same engine, and a report to read at every step rather than a boolean to glance at.

Consolevar ENTRIES = [ "Harbourgate opens the register", "Consignment 41 signed over to Pell and Sons", "Consignment 41 inspected, seal intact", "Consignment 42 signed over to Ardwick Cold Store", "Consignment 41 released to the buyer", "Consignment 42 inspected, seal broken" ]; var chain = CHAIN.makeChain(ENTRIES, { difficulty: 0 }); CHAIN.verify(chain) var edited = CHAIN.tamper(chain, 2, "Consignment 41 inspected, seal BROKEN"); CHAIN.verify(edited).problems edited[2].hash = CHAIN.hashBlock(edited[2]); CHAIN.verify(edited).problems var forged = CHAIN.reforge(edited, 2, 0); CHAIN.verify(forged.chain)

Read all three fields before anything is broken

Run CHAIN.verify(chain) on the untampered register and look at ok, problems and firstBad together. Say out loud what each one holds when there is nothing wrong, so you recognise the difference when there is.

Predict the kind and the index

Before running the tamper, write down which of the two kinds you expect and at which index. Then run it and read problems. Getting this right is the difference between knowing the mechanism and having read about it.

Make the repair an attacker would make

Set edited[2].hash = CHAIN.hashBlock(edited[2]) and run verification again. Count the problems before you read them. The count is unchanged, which is the part that catches people out.

Point at the edit from the report alone

From the second report, name the block that was actually altered without looking at any block's data. Then check yourself against edited[2].data. Write the rule you used in one sentence.

Walk the repair one block further

Set edited[3].prev = edited[2].hash and verify. Then set edited[3].hash = CHAIN.hashBlock(edited[3]) and verify again. Note the kind and index each time and say where the sequence can possibly stop.

Reforge, and read what it cost

Run CHAIN.reforge(edited, 2, 0) and inspect both halves of what comes back: the rebuilt chain, and attempts. Verify the rebuilt chain. Keep the attempts figure, because session 4 changes it by four orders of magnitude without changing a line of this code.

The habit to keep. When a verification fails, write down the kind and the index before you open the data. Those two values name a suspect and a location, and the account you build from them is testable. An investigation that starts by reading the changed row tends to end up explaining the row rather than the edit.

Check yourself

Three questions

Block 2's entry is edited and every stored hash is left alone. What does verification return?

The link check at block 3 compares block 3's prev with block 2's stored hash, and that stored hash was not touched, so the link still matches. Only block 2's own consistency has broken. The link failure is what stage two produces, after the attacker repairs the hash.

A register you did not build reports exactly one problem: index 4, kind link. What can you say?

A link problem at N is about the join between N minus 1 and N, and it points upwards. Block 3 is internally consistent, otherwise a hash problem would be reported at 3 as well, so block 3 was rebuilt rather than merely edited. What the report cannot tell you is what block 3 now says or why.

After reforging from block 2 the register verifies clean. What has that established?

A clean result is a statement about internal consistency and nothing else. The rebuilt register is consistent with a history that did not happen, and this copy has no memory of the old hashes to disagree with. Comparing against a copy held elsewhere is the check that survives a full rewrite.

Before session 4

Homework

Twenty five minutes, on the register you assembled for session 3 or on a real audit trail you are responsible for.

Give your verifier a kind

If your own check returns true or false, change it to return an index and a kind for every failure, then break your register in two different ways and confirm the two reports differ. A verifier that cannot tell an in place edit from a stopped repair is a verifier that cannot help an investigation.

Time a full rewrite of your own register

Rebuild it from its second entry onwards and record how many blocks that touched and how long it took. Bring both numbers to session 4, which is about making the first number expensive rather than making the second one larger.

Covered tonight

What this session covers

✓Reading verification output properlyok, problems and firstBad, and the index plus kind carried by every problem.
✓Stage one, the edit alonetamper on block 2 leaves stored hashes alone and produces one problem: a hash mismatch at block 2.
✓Stage two, the attacker's repairRecomputing that block's hash leaves one problem: a link mismatch at block 3.
✓The kind as forensicsA hash mismatch means an in place edit; a link mismatch means a rebuild whose successors were not rebuilt.
✓Reforging to a clean register4 blocks, 2 through 5, after which verification has nothing to report at all.
◐Making the rewrite expensiveNamed tonight as the missing piece. Nonces, difficulty and measured attempts are session 4.
◐Who else holds a copyThe other half of the check, raised here and worked through in session 8.
○Who is allowed to write an entryNothing in verification asks. Signatures and identity are session 7.
○Any cryptocurrency, price, market or walletNot here. Harbourgate keeps a custody register, and nothing on this page is investment advice.
Take this with you

Cheat sheet

The one sentence. Verification names the block and the kind of failure, and the kind is the diagnostic.
Two checks per block. Rehash it from its own fields; compare its prev with the block above.
Stage one. Edit block 2 only: 1 problem, index 2, kind hash.
Stage two. Rehash block 2: still 1 problem, index 3, kind link.
hash at N. Block N was edited in place and nothing else was done.
link at N. Block N minus 1 was rebuilt and its successors were not. Look one block up.
Reforge from 2. 4 blocks, 2 through 5, and verification returns ok.
The gap. At difficulty 0 the rewrite is four hash computations, which is what session 4 makes expensive.