learn-web3-with-phoebe / Leader session 3 of 6
Learn Web3 with Phoebe · Leader session 3 of 6

Numbers people are paid to move

A measure that becomes a target stops being a good measure. Everybody knows that sentence and almost nobody has met the version Larkfield is living in, where the target is published in the mechanism that pays it and the party responding to it can make more of itself rather than merely behave differently. Session 2 measured half of this already: 4 operators running 60 addresses each, 240 rows that no address count can tell apart from 240 people. Tonight is what those two extra features do to every metric Larkfield reports.

🟠 Leader track Anyone who signs off a metric that triggers a payment Anyone planning detection work 45 min
0-3 · Welcome 3-17 · Two extra features 17-30 · The measurement 30-45 · Build-along and Q&A
Part 0

The adversary you were going to assess, who is already in the data

Session 1 named the fourth coupling and left it as a sentence: the metric and the incentive to move it sit at the same address. Session 2 then measured a gap of about 400 between a reported ceiling and the truth and found that most of it was ordinary people, not operators. Tonight takes the operators seriously anyway, because the reason they matter is not the size of their contribution to that particular gap. It is that they are evidence of a standing property of the dataset, and that property changes what a metric is.

Live = presented in the 45 minutes Self-study = the full depth, after class 🔍 Every diagram zooms

★ What you walk out with tonight. Goodhart's problem with the two features it does not usually carry, named separately so you can check for each one. The measured version of the second feature: 240 addresses made by 4 parties, sitting inside a count of about a thousand, with the clustering removing only part of them. A working rule for reading any published metric as a target from the day it is published rather than from the day somebody abuses it. And the genuine disagreement about whether publishing the definition of a metric makes the gaming worse or only makes it visible.

LiveWhy an adversary is a default and not a risk item4 min▶

A risk register asks how likely something is and what it would cost. That framing works when the thing is an event. It does not work here, because on a public ledger the conditions for the behaviour are permanently present: the rule is readable, participation is open, and one party can hold as many addresses as it cares to create. There is no state of the world in which those conditions are absent, so there is no probability to estimate.

The shift that follows is small to say and large to run. You stop asking whether the user base is adversarial and start asking which share of it is, and in which direction that biases each number you publish. The first question has a yes or no answer that is always yes. The second has an answer you can measure, report and act on, which is what the live box in part 2 does.

Real world

Larkfield's risk review scored this as low likelihood, on the reasoning that the rewards were modest and the product was young. Both of those were true. Neither of them was relevant, because the cost of presenting as sixty parties instead of one is sixty rows, and the review had implicitly priced the behaviour as if it required sixty people.

Self-studyThe words that keep coming back4 min▶
  • Target. A measure that somebody is rewarded against. The moment a measure becomes one, its relationship to the thing it was measuring starts to come apart.
  • Published target. A target whose exact definition is readable by the parties responding to it, without being told and without asking. Part 1.
  • Manufactured identity. An address created so that a count of addresses reads higher. Not a disguise and not a breach, just a row.
  • Operator. In Larkfield's world, a party deliberately running many addresses. There are 4 of them and they run 60 each.
  • Adversarial data. A dataset where some rows exist because somebody wanted your metric to read differently. Session 1 named it, tonight measures it.
  • Ceiling. The clustered figure from session 2. An upper bound on the entity count, with no floor.
  • Detection. Work aimed at finding deliberate multi-address behaviour. Session 2 measured how much of the gap it can possibly recover, and the answer was small.
  • Record linkage. Joining records about the same party by fuzzy attributes and probability. That craft lives in learn-entity-resolution-with-phoebe; the evidence here is transaction structure, so the machinery differs.
Part 1 · covers the two extra features

Goodhart, with the two parts nobody warns you about

14 min live

The familiar version has a target set inside a building, an actor who has to guess what is counted, one identity per actor, and feedback that arrives slowly and incompletely. Every mitigation ever written for it assumes at least one of those. On a public ledger the first three are gone and the fourth runs the other way, and the honest way to see that is to draw the same three parts twice.

Three parts of a targeted metric, drawn twice. Nothing was redesigned; the wall came off. NORMALLY, ON A TARGET YOU SET 1 · THE TARGET THE RULE what counts as active ? the rule is not readable from outside the wall A target somebody has to guess at before gaming it. 2 · THE ACTOR one party changed behaviour one identity, and it is the only one available To move the number the party has to act differently. 3 · THE FEEDBACK metric published once a month, in aggregate the actor learns late, partially and by inference Slow feedback, so gaming is costly and uncertain. WHEN THE TARGET IS ON THE LEDGER 1 · THE TARGET THE RULE what counts as active anyone can read the exact rule and what it rewards The target is published as part of the mechanism. 2 · THE ACTOR one party, as many identities as it decides to create Moving the number needs no change of behaviour. 3 · THE FEEDBACK metric the actor reads its own effect as soon as it lands Immediate and exact, so gaming is cheap to verify. Two features the usual version does not have: the target is published, and the actor can make more of itself. Either one on its own is manageable with the usual playbook. Together they mean the metric is a specification, the specification is public, and the population it counts can be manufactured by anyone who reads it. Part 2 puts measured numbers on the second.
🔍 Click to zoom - the bottom middle panel is the one with no equivalent in the familiar version. The actor did not behave differently. It became more numerous.
LiveFeature one: the target is published4 min▶

Every mitigation written for a targeted metric leans somewhere on the actor not knowing the exact rule. Hold the definition back. Change it without announcing. Keep the thresholds in a document that does not circulate. Measure something adjacent so the obvious lever is not the one being counted. All of these work by putting a gap between what the actor thinks is counted and what actually is.

On a public ledger that gap cannot be created, because the rule that issues the reward is readable by anyone in exactly the form that runs. There is no version of it held back. An actor does not have to infer what triggers the reward from its own results over three months, which is the slow and error-prone route; the actor reads it, once, and is then correct about it.

The consequence is that secrecy stops being available as a control, and any plan that quietly depended on it has to be replaced rather than tightened. That is not the same as saying nothing can be done, and part 2's build-along works on what is left.

LiveFeature two: the actor can make more of itself4 min▶

The second feature is the larger one and it is the one with no precedent in the usual framing. In the familiar version an actor facing a target changes what it does. It works the measured thing harder and the unmeasured thing less, and the damage is that your measure and your intent come apart. The actor remains one actor throughout, which is why the harm is describable as distortion.

Here the actor has a second move. Rather than changing behaviour it can change how many of it there are, because an identity in this dataset is an address and an address is a row. A party wanting a per-party count to read higher does not need to do anything differently. It needs to be more numerous, and being more numerous is free of everything except the rows.

That breaks the mitigation of last resort as well. When you cannot hide the rule, the standard fallback is to normalise the metric per user, so that working the lever harder does not help. Per user normalisation assumes the user count is a fact about the world. It is a fact about the data, and the data is what the actor is adding to.

Self-studyWhy the two features have to be checked separately4 min▶

It is tempting to treat these as one phenomenon, and the reason not to is that they come apart in other settings and the remedies differ. A published target with a fixed identity set is a familiar situation: published benchmark criteria, published inspection standards, published eligibility rules. It is uncomfortable and it is workable, and the working answer is usually to make the target hard to satisfy without also doing the underlying thing.

A manufacturable identity set with a private target is also familiar, in a smaller way: a referral programme where one person can hold several accounts, but nobody outside knows exactly what qualifies. The working answer there is verification at the point of identity, and the cost of it is borne once per identity.

Larkfield has both at once, so neither answer survives on its own. Making the target hard to satisfy without doing the underlying thing does not help when the underlying thing can be done sixty times by one party. Verifying identity is the right instinct and it is a decision about the product rather than the pipeline, since the only place to attach a verification is before the address becomes eligible. Session 6 puts that in the scoping questions, where it belongs, because by the time a data team meets this the choice has already been made.

Part 2 · covers the measurement

240 addresses, 4 parties, and a count that cannot tell

13 min live

Session 2 ran the co-spend method over Larkfield's month and reported three numbers. The same run also contains the answer to a different question, which is what share of the count exists because the metric pays for it. In Larkfield's world 4 operators run 60 addresses each. That is 240 addresses, made on purpose, sitting inside a count of about a thousand.

One month of Larkfield, counted three ways, on one scale. ADDRESSES SEEN 300 people holding one to four addresses each 240 made by 4 about 1,000 AFTER THE CO-SPEND METHOD it joins some of them and then runs out of evidence 676 to 707 clusters THE TRUTH 300 people and 4 operators 304 entities 0 250 500 750 1,000 By address count, 4 parties and 240 people are the same thing. Nothing in the rows separates them and nothing outside the rows can be joined in to settle it. The co-spend method removes about half of the total error and leaves a ceiling of 676 to 707 against a truth of 304. Both of those are session 2 results. What is new tonight: those 240 addresses exist because the metric pays for them.
🔍 Click to zoom - the amber block is not a data quality defect. It is the product working exactly as specified, counted by a metric that was not designed to notice.
Live The generator and the clustering, both running in this page. Move the operators and watch which of the four numbers responds.

...

...
LiveDrag it to zero, and then read the truth row4 min▶

Set the operators to run no addresses at all and the manufactured rows disappear from the count, which is what you would expect. The row worth staring at is the one below: the truth drops by 4 and the reported ceiling drops by a great deal more than 4. The four operators were four entities. They were being counted as something close to two hundred.

Now walk the slider back up and watch how the reported ceiling responds. It rises, and it rises by less than the number of addresses added, because the co-spend method does catch some of them. It never catches all of them, and the shortfall goes straight into the number Larkfield publishes as a bound.

Two things are worth saying out loud at this point. The first is that this is not a detection failure, since detection is exactly what the method is doing and it is doing it correctly. The second is that a party who wanted to defeat the method entirely would not have to work at it; session 2 measured that operator discipline moves the clustered count by 31 against a gap of 392, so the method is not being beaten by tradecraft. It is being outnumbered.

LiveRead every published metric as a target from day one4 min▶

The practical rule that comes out of this is narrow enough to apply on a Monday. When a metric is published and a payment is attached to it, you do not wait for evidence of gaming before treating the figure as gamed. You write down, at the moment of publication, which direction the metric can be pushed and what one additional identity buys the pusher. Then you decide whether you are prepared to report it at all.

The reason to do this at publication rather than at first incident is that the first incident is not observable. There is no alert for an address that behaved normally. The 240 rows in Larkfield's month did not trip anything, would not trip anything, and are only visible here because the generator knows the answer. On real rows you would have the count and nothing else.

The version of this that a leader can hold people to is a single line in the definition of every metric that triggers a payment: who benefits if this number goes up, and what is the cheapest way for them to make it go up. If that line is empty the metric has not been reviewed, whatever else was checked.

Self-studyWhere serious people disagree5 min▶

One camp holds that you should publish the reporting definition in full, alongside the reward rule that is already public. Their argument is that the definition is effectively recoverable anyway: a determined party can infer it by watching what gets counted, and will, so withholding it disadvantages only the honest participants, your own analysts and anyone auditing you. Publishing also forces the definition to be good, because a rule that embarrasses you when read is a rule that should not be paying anyone.

The other camp holds that recoverable in principle and handed over in writing are different things, and the difference is measured in how many parties bother. Every published clause is a specification for satisfying it cheaply, and the people who read specifications carefully are exactly the ones you do not want optimising. On this view the reward rule has to be public because it executes, and the reporting definition does not, so keep them apart: pay on one and report on another, and never publish the second.

Both are held by people who have watched a metric collapse. The disagreement is empirical rather than philosophical, and it turns on how many parties will do the inference work if you do not do it for them. Notice that the second position has a cost the first does not: a reporting definition nobody outside can see is a figure nobody outside can check, which gives up the one advantage session 1 credited to the public arrangement in the first place.

Part 3 · build-along

Find the metric somebody is already paid to move

15 min hands on

No code. Six steps on the live box above and on a metric your own organisation publishes, said out loud in the room. The aim is to leave with one metric you would now describe differently, and the sentence you would use.

Say the two features back

Published target, manufacturable identity. One sentence each, without notes. Then say which of the two your own organisation already has somewhere, because most have the first without ever calling it that.

Run the slider to both ends

Set the operators to 0 and then to 60. Write down what the truth row does and what the reported ceiling does. The two movements are different sizes and the difference is the whole point of the box.

Pick a metric that triggers a payment

Anything where a number moving causes money or resource to move: a bonus, a rebate, a tier, a budget allocation, a supplier credit. Write down who receives it and how the number reaches them.

Write the cheapest way to move it

Not the most likely way, the cheapest. One line. Then write what it costs the person doing it, in their currency rather than yours. If the cost is one row, say so plainly and sit with it.

Check whether you were relying on secrecy

Look at your answer to step 4 and ask what stops it today. If any part of the answer is that they do not know the rule, mark it, because that control is unavailable on a public mechanism and you have just found the shape of the replacement you need.

Decide what you would report instead

Given the direction the metric can be pushed, write the sentence you would put in a board pack. Session 2's shape is the model: a bound with a stated direction beats a point estimate. Keep it. Session 4 is about what happens when you have to correct one of these after the fact.

The habit to keep. Before asking how accurate a metric is, ask who is paid by it and what one more identity buys them. Accuracy is a property of the pipeline and it can be improved by engineering. The other thing is a property of the arrangement, it cannot, and a clean pipeline reporting a gamed metric is a faster route to the same wrong decision.

Check yourself

Three questions

Which two features does the Larkfield version of a targeted metric have that the familiar version does not?

Both are structural rather than incidental. A published target removes secrecy as a control, and a manufacturable identity set removes per user normalisation as a fallback, because the user count is itself made of the thing the actor is adding.

A team proposes normalising Larkfield's headline figure per active user, so that raw activity cannot inflate it. Why does that not work here?

Per user normalisation assumes the user count is an independent fact about the world. Here it is a count of rows in the same dataset the actor is writing to, so it is not independent of the behaviour it is supposed to control for.

Larkfield's risk review scored deliberate multi-address behaviour as low likelihood because the rewards were modest. What is wrong with that reasoning?

A likelihood estimate fits an event. This is a standing property of the arrangement: the rule is readable, participation is open and identity is free to create. There is no state of the world in which those are absent, so the useful question is what share, in which direction, not whether.

Before session 4

Homework

Twenty minutes, on the metric you chose in step 3 and on one figure your team has already published.

Add the missing line to one metric definition

Who benefits if this number goes up, and what is the cheapest way for them to make it go up. Two clauses, in the definition itself rather than in a separate risk document. Bring the before and the after.

Find a published figure you would now withdraw

One number your team has already reported that you would state differently after tonight. Write the corrected sentence. Then try to work out how you would actually issue the correction, and bring what you find, because session 4 is about a dataset that has no mechanism for taking anything back.

Covered tonight

What this session covers

✓Goodhart with two extra featuresA published target and a manufacturable identity set, drawn as one mechanism twice and checked separately.
✓Secrecy is not available as a controlThe rule that pays is readable in the form that runs, so every mitigation that depends on a definition gap has to be replaced.
✓240 addresses made by 4 parties4 operators running 60 addresses each, inside a count of about 1,000, with the clustering removing only part of them.
✓Why per user normalisation fails hereThe denominator is built from the same rows the actor is adding to, so it is not independent of what it is meant to control for.
✓Adversarial as a default, not a risk itemThe conditions are permanently present, so the question is what share and in which direction rather than how likely.
✓The line every paying metric needsWho benefits if this goes up, and what is the cheapest way for them to make it go up.
◐Correcting a figure you already publishedReached tonight in the homework. Session 4 is the correction path, or the absence of one.
◐Verification before an address becomes eligibleNamed tonight as a product decision rather than a pipeline one. Session 6 puts it in the scoping questions.
○The clustering method itselfMeasured in session 2 and taught to working depth on the practitioner track.
○Probabilistic record linkageFuzzy matching, blocking and scoring live in learn-entity-resolution-with-phoebe.
○Chain mechanics, and anything about price or marketsAssumed from learn-blockchain-with-phoebe, and out of scope by design. Nothing here is advice of any kind.
Take this with you

Cheat sheet

The one sentence. The metric and the incentive live in the same place, and the population being counted can be written to by the people being counted.
Feature one. The target is published in the form that runs. Secrecy is gone as a control, not weakened.
Feature two. The actor can be more numerous instead of behaving differently. One more identity costs one more row.
The measured version. 4 operators, 60 addresses each, 240 rows inside a count of about 1,000.
Why normalising fails. The denominator is made of the same rows the actor is adding to.
Not a risk item. The conditions never go away, so ask what share and which direction, never how likely.
The missing line. Who benefits if this goes up, and what is the cheapest way for them to make it go up.
The open question. You published a figure and it was wrong. Session 4 is what taking it back involves here.