learn-blockchain-with-phoebe / Practitioner session 1 of 10
Learn Blockchain with Phoebe · Practitioner session 1 of 10

Hashing, hands on

Every guarantee Harbourgate's custody register makes rests on one function. Not on a network, not on a consensus rule, not on anybody's good behaviour. One function that takes bytes and returns 64 hexadecimal characters. Tonight you run it, you break a prediction with it, and you check it against the vectors that say it is the function it claims to be. Everything after this session is arrangement. This is the primitive.

🟡 Foundations Engineers and data people Anyone who will build or audit one 45 min
0-3 · Welcome 3-15 · Four properties 15-27 · Vectors and avalanche 27-45 · Build-along and Q&A
Part 0

One function, and ten sessions built on it

The practitioner track runs ten sessions against a real engine: a full SHA-256 written from the specification, a six block Harbourgate register, and the routines that tamper with it, mine it, reforge it and prove membership in it. Nothing on these pages is a stored screenshot of a result. Session 1 is the function on its own, with no chain around it, because a chain built on a hash you have not checked is a chain of confident nonsense.

Live = presented in the 45 minutes Self-study = the full depth, after class 🔍 Every diagram zooms

★ What you walk out with tonight. Four properties you can name without hedging, and the ability to say which one is doing the work. A working habit: never trust a hash implementation that does not carry its test vectors. And one measured number you can quote from memory, 58 of 64, which is what a single letter does to a digest.

LiveWhat a hash is, in one paragraph4 min▶

A hash function takes an input of any size and returns an output of one fixed size. SHA-256 returns 256 bits, written as 64 hexadecimal characters. It holds no state, it stores nothing, and it has no key. Give it the same bytes and it gives you the same 64 characters on any machine, in any language, in any year.

That is the whole interface. What makes it useful for a register is not the interface but four properties of the mapping underneath, and one of those four is the load bearing one.

Real world

The engine behind these pages exposes it as CHAIN.sha256(str). One argument in, 64 characters out, no options and no configuration. If a hashing API offers you settings, you are looking at something else wrapped around a hash.

Self-studyThe words that will keep coming back4 min▶
  • Digest. The output. Also called a hash or a fingerprint. Always 64 hexadecimal characters here.
  • Test vector. A published input and the digest every correct implementation must return for it.
  • Self test. Running the vectors at load time instead of hoping. CHAIN.selfTest() does it in this page.
  • Avalanche. A tiny change to the input producing an unrecognisable digest.
  • Preimage. An input that produces a given digest. Finding one has no method better than guessing.
  • Collision. Two different inputs with the same digest. They must exist by counting, and none is known for SHA-256.
  • Block hash. The digest of a whole assembled block rather than of its entry text. Session 2.
  • Key management. Who holds which secret and how it is rotated. A discipline of its own, in the security courses.
Part 1 · covers the function

Four properties, and which one is load bearing

12 min live

Three of the four are conveniences. They make the thing practical, and a register could survive weaker versions of any of them. The fourth is the one an attacker runs into, and without it the other three are a fast way to produce useless numbers.

Three conveniences and one defence 1 · Deterministic Same bytes in, same 64 characters out. Any machine, any year. Two parties who distrust each other can check the same thing and get the same answer without asking a third. 2 · Fast One pass over the bytes. Nothing to look up, nothing to wait for. Producing a digest and checking one cost the same, which is why a sceptic can recompute a whole register for free. 3 · Fixed length Three characters in or three million in. 64 characters out. The digest is a stable sized handle for something of any size, which is what lets one field stand for a whole block. 4 · No usable gradient Nothing in a digest points back at the input that made it. Nothing points towards a target digest either, so there is no warmer and no colder. Only guess, check, guess again. The first three make it practical. The fourth makes it worth building on. A function with properties 1 to 3 and not 4 is a very fast way to produce numbers nobody should rely on, and it would pass every casual inspection, because its output looks exactly the same.
🔍 Click to zoom - properties 1 to 3 are about the interface. Property 4 is about the mapping, and it is the only one an attacker has to defeat.
LiveWhy a gradient would end the argument4 min▶

Suppose digests behaved sensibly, so that similar inputs produced similar outputs. Forging an entry would become a hill climb. Write the line you want, compare its digest with the one you need to match, nudge a character, compare again, keep whatever got you closer. Any search with feedback finishes eventually.

The absence of a gradient turns that into a blind lottery. Each attempt is independent, nothing carries over, and there is no signal to follow. This is also the reason a hash is called one way. Forwards costs one pass. Backwards has no known method better than trying inputs until one lands.

LiveFixed length has a consequence people skip3 min▶

Inputs are unlimited and outputs are not, so different inputs must share digests somewhere. That is arithmetic, not a defect, and it is true of every hash function that has ever existed. The engineering claim is narrower: no two colliding inputs for SHA-256 have been found, and nobody has a method for constructing a pair.

Keep the distinction handy, because the two sentences get merged in conversation. Collisions exist. Nobody can produce one. Only the second sentence is a property of SHA-256, and only the second sentence can expire.

Self-studyFast is a property with two edges3 min▶

Speed is what lets a buyer recompute a six block register in the time it takes to read this sentence. It is also what lets an attacker try a very large number of inputs, because their guesses cost the same as your checks. For a register that is fine, since the attacker still has no gradient and no shortcut.

It stops being fine the moment the input is small and predictable, which is why storing short human chosen secrets under a fast hash is a different problem with different tools. That problem belongs to the security courses, and the register does not have it.

Part 2 · covers the test vectors

Why an implementation ships its test vectors

7 min live

A hash implementation with one constant mistyped does not crash and does not warn. It returns 64 hexadecimal characters that look exactly as convincing as correct ones, and every guarantee built on top of it quietly stops being true. There is no way to notice by looking. There is a way to notice by checking.

A wrong hash fails silently, and it fails beautifully A subtly wrong hash One constant mistyped 64 plausible hex characters No crash, no warning, no clue Every guarantee gone And the screen looks identical PUBLISHED INPUT THE DIGEST EVERY CORRECT IMPLEMENTATION RETURNS abc ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad the empty string e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 abcdbcdecdefdefgefghfghi jhijkijkljklmklmnlmnomnopnopq 248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1 All three pass in the engine running on this page. The implementation was additionally cross-checked against node's own crypto module on four further inputs, including a 1,000 character string, because three vectors are a floor and not a ceiling.
🔍 Click to zoom - the vectors are the only thing in the picture that can tell the two implementations apart.
Live selfTest() running in this page. The second button swaps in a function that returns plausible hex and nothing else.

...

...
LiveReading the impostor's output3 min▶

Press the second button and look at the digests before you look at the verdicts. They are 64 characters, they are hexadecimal, they are different for different inputs, and they are stable across reloads. Every property you can see by eye is satisfied. The function behind them is not a hash function in any useful sense and nothing on screen says so.

That is the argument for shipping vectors inside the implementation and running them at load time rather than in a test suite somebody remembers to run. A digest that is wrong is indistinguishable from a digest that is right, right up until somebody else recomputes it and disagrees with you in public.

LiveWhere the vectors come from3 min▶

They are published alongside the specification, which is the point of them. The digest of abc is not an opinion held by this engine, it is the value any conforming implementation must return, and it has been the same value since SHA-256 was standardised. Three inputs were chosen to exercise different paths: a short input, an empty one, and one long enough to force a second compression block.

The empty string is the one people leave out, and it is the one that catches padding bugs, because it is entirely padding.

Self-studyWhere the disagreement really is4 min▶

One camp holds a firm rule: never write your own implementation of a cryptographic primitive. Use the platform's, which is reviewed by specialists, hardened against timing attacks and maintained by people whose job it is. A hand written SHA-256 in a page like this one is, on that view, exactly the practice that produces the silent failures the vectors are there to catch.

The other camp holds that an implementation you can read end to end and check against published vectors is worth more, in a teaching or auditing setting, than a platform call whose behaviour you take on faith. The vectors are what make the claim checkable by the reader rather than by reputation, and reputation is not evidence.

The engine on these pages sits squarely in the second camp and is honest about which camp that is. Both positions are held by people who have shipped real systems, and the sides do not even fully disagree: the first camp's rule is about production and the second camp's rule is about understanding. Where they genuinely collide is the system that quietly grows from a demonstration into production, which is most of them.

Part 3 · covers avalanche

Avalanche, with the numbers attached

8 min live

"A small change makes a big difference" is a slogan until somebody counts. Two pairs were measured against this engine. A three letter input against its one letter neighbour, and a Harbourgate entry against the same entry with one letter capitalised. Both come out around three quarters of the digest.

Measured, not asserted Pair one · two three letter inputs abc ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad abd a52d159f262b2c6ddb724a61840befc36eb30c88877a4030b65cbe86298449c9 One letter of input changed. Nothing of the digest survives in place. 58 of the 64 characters differ Pair two · one capital letter inside a Harbourgate entry Consignment 41 inspected, seal intact f0c5a3b1469adfcafafc37934b52a3c180d65419382858fb242ea484a10f4a4a Consignment 41 inspected, seal Intact 9ab0c83cecdb404d8089e91684d00c3704835e035897cd6b155ba763fb3989f8 60 of 64 differ from one capital letter Read the small print on these four digests. They are hashes of the entry text alone. They are not the block hashes in the Harbourgate register, because a block hash is taken over the whole assembled block. Session 2 is exactly that difference.
🔍 Click to zoom - two pairs, two counts, both around three quarters. That ratio is what "no gradient" looks like when you count it.
Live Real SHA-256 in this page. Each of the 64 positions is marked where the two digests disagree.

...

...

...

...
LiveWhy it is not 64 out of 643 min▶

Two unrelated digests agree at any given position about one time in sixteen, because there are sixteen hexadecimal characters. Across 64 positions that predicts roughly four accidental matches, which is why measured counts sit near 60 rather than at 64. A pair that differed in all 64 positions every time would be the suspicious result, not the reassuring one.

So the honest reading of 58 and 60 is not "almost everything changed". It is "the second digest looks like a digest drawn at random with no relationship to the first", which is the property the register needs.

LiveWhat this rules out for an attacker3 min▶

Someone rewriting a custody entry needs a different sentence with the same digest. The measurement says the nearest possible sentences, the ones differing by a single character, land nowhere near the target. There is no sequence of small edits that walks a digest towards a value, because each edit throws the digest somewhere unrelated.

What is left is guessing, and the space is 64 hexadecimal characters wide. That is the whole of the defence, and it is enough, which is why sessions 3 and 4 are about a completely different attack: not breaking the hash, but recomputing everything downstream of the edit.

Self-studyCounting characters is a proxy, and a rough one3 min▶

Counting differing hexadecimal characters is easy to see and easy to explain, and it is not the measure a cryptographer would use. The proper statement of avalanche is at the bit level: flipping one input bit should flip each of the 256 output bits with probability one half. A character counts as differing if any of its four bits moved, which inflates the visible ratio.

The character count is used here because it is checkable on screen in a page anyone can open. Take 58 and 60 as demonstrations of the shape of the property, not as its definition.

Part 4 · build-along

Drive the function yourself

18 min hands on

Open the browser console on this page. The engine is already loaded and exposed as CHAIN, so every line below runs against the same implementation the diagrams were measured with.

ConsoleCHAIN.selfTest().ok CHAIN.selfTest().results CHAIN.sha256("abc") CHAIN.sha256("abd") CHAIN.sha256("") CHAIN.sha256("placeholder")

Check before you use

Run CHAIN.selfTest().ok first. If it is not true, stop, because nothing else on the page means anything. Then run .results and read the three rows: the input, the expected digest, the digest this browser produced.

Match the first vector by eye

Run CHAIN.sha256("abc") and compare the first eight characters with ba7816bf in the diagram above. Then compare the last eight. Two spot checks at both ends is how a digest is compared by a human, and it is why digests get truncated in tables.

Hash the empty string

CHAIN.sha256("") returns a digest, not an error. Say out loud why a function that hashes nothing still has to return 64 characters, and what would break in the register if it returned an empty result instead.

Predict, then count

In the avalanche box, before pressing anything, write down how many of the 64 positions you expect to change if you add a full stop to the end of input A. Then type it and read the count. Most people guess too low.

Try to steer it

Set input B to something whose digest begins with the same character as input A's. You are allowed unlimited tries. Notice that nothing you learn from one attempt helps with the next, and that this is what having no gradient feels like from the inside.

Write the sentence you will need in session 2

One line, no metaphor: what a digest proves about the bytes it was taken over. Keep it. The next session hands that sentence a block with five fields in it and asks whether it still holds.

The habit to keep. When a library hands you a hash, find its test vectors before you find its documentation. An implementation that does not carry them is asking you to take its arithmetic on trust, and its output will look correct either way.

Check yourself

Three questions

An implementation has one constant mistyped. What does its output look like?

This is the whole reason vectors ship inside the implementation. A wrong hash is still a function: same input, same output, plausible hex, no warning. The only way to tell is to compare against a published value, which is what selfTest does at load time.

The digests of abc and abd differ in 58 of 64 characters rather than all 64. Why is that the expected result?

Sixteen possible characters per position means roughly four coincidental matches across 64 positions. A count of 58 or 60 says the second digest behaves like an unrelated draw. A count of exactly 64 every time would be a pattern, and patterns are what an attacker wants.

The digest f0c5a3b1 and so on was measured for the text "Consignment 41 inspected, seal intact". Where does that digest appear in the Harbourgate register?

Block 2's hash begins f325e5d96bee6a1d, which is a different value entirely, because the block carries its index, timestamp, previous hash and nonce alongside the entry text and all of them go into the digest. Conflating the two is the most common error when reading a register, and session 2 is built on the difference.

Before session 2

Homework

Twenty minutes, in whatever language you actually work in.

Find the vectors in your own stack

Take the hashing library you would reach for at work and hash the three published inputs from Part 2. Confirm all three match. Note how long it took you to find out whether that library runs its own vectors at load time, because that answer is the finding.

Measure your own avalanche

Hash a real record from your work, change one character, hash it again, and count the differing positions yourself. Bring the number to session 2. If it is nowhere near the high fifties, something in your comparison is wrong and that is worth finding.

Covered tonight

What this session covers

✓SHA-256 as a function, and its four propertiesDeterministic, fast, fixed length, and no usable gradient, which is the one that does the work.
✓The three published test vectorsRun live by selfTest, and why an implementation ships them rather than assuming.
✓The avalanche effect, measured58 of 64 characters for abc against abd; 60 of 64 for one capital letter in a Harbourgate entry.
✓Driving sha256 and selfTest liveThe same engine the diagrams were measured with, in the console and in two boxes on this page.
◐What a block hash is taken overNamed tonight, because it is not the same as hashing the entry text. Session 2.
◐Collisions, and why none is knownStated as a distinction, not worked through. The security courses go further.
○Key management, rotation and certificate hygieneA discipline of its own, covered in the security and access control courses.
○Named commercial chains and their roadmapsThey move faster than a course can, and none of the mechanism on this page depends on which one.
Take this with you

Cheat sheet

The function. CHAIN.sha256(str). Any input, 64 hexadecimal characters, no options.
Deterministic. Same bytes, same digest, any machine, any year.
Fast. One pass. Checking costs what producing costs, which is why a sceptic can verify for free.
Fixed length. Three characters or three million, 64 out. Collisions must exist; none is known.
No gradient. Nothing points back at the input and nothing points at a target. Guess, check, guess again.
Vectors. abc gives ba7816bf and so on. A wrong hash still prints plausible hex, so check rather than assume.
Avalanche. 58 of 64 for abc against abd. 60 of 64 for one capital letter in an entry.
The trap. Those digests are of the entry text alone. A block hash covers the whole assembled block.