The memo's information verdict
Every analysis request is an implicit purchase order: someone's time, a delay to the decision, sometimes a literal invoice. "Let's get more data first" sounds prudent precisely because nobody prices it. Tonight the memo gains the section that does - the information verdict: what more information is worth to this decision, what it costs, and therefore whether to buy it. Arithmetic, not politics.
★ What you walk out with tonight. EVPI computed by hand on the voucher numbers - and the do-not-buy surprise. The tent-shaped law of where information is valuable, felt on the live calculator. And Hubbard's measurement inversion: why organizations routinely measure the variables worth the least.
LiveThe question nobody asks of an analysis3 min▶
Teams ask whether an analysis is feasible, rigorous, fast. Almost nobody asks the economic question: what is knowing this worth? Ronald Howard made it computable in his 1966 "Information Value Theory": the expected value of perfect information is the expected value of deciding AFTER the uncertainty resolves, minus the expected value of the best decision you can make NOW. That difference is the ceiling on what any study, test, or dataset can possibly be worth to this decision - no real study, being imperfect, can beat it.
Howard drew the line explicitly against Shannon's information theory: Shannon measures information in bits, blind to what it is FOR. A megabyte that cannot change your decision is worth $0; a single bit that flips a $12k call can be worth thousands. Value of information is decision-first by construction - which is why it belongs in this course and in your memo.
Session 1's six-word question already priced one analysis: the stakeholder who said "nothing would change my mind" was requesting a $0-value study. Tonight that intuition becomes a formula - EVPI for an unflippable decision computes to exactly $0, and for the voucher case it computes to a number with a verdict attached.
Self-studyTonight's vocabulary2 min▶
- EV(act now) - the probability-weighted payoff of the best action under current uncertainty. The memo's numbers already contain it.
- EV(perfect info) - the payoff if a perfectly reliable oracle told you the truth BEFORE you chose: act when it says true, skip when it says false.
- EVPI - the difference between the two. The ceiling on any study's worth to this decision.
- Ceiling, not price - real studies deliver imperfect information, so they are worth strictly less than EVPI. A study that fails the ceiling test needs no further analysis.
- Measurement inversion - Hubbard's practice finding: the variables organizations measure most are usually the ones worth the least to measure.
EVPI on the voucher numbers
6 min liveThe memo already holds everything the formula needs. The campaign costs $12k; if the voucher hypothesis is true it nets +$18k, if false it nets -$12k; the calibrated, base-rated estimate from sessions 2-3 puts P(true) at 0.7; and the offered segmentation study costs $5k. Walk the three lines.
LiveThe three lines, slowly3 min▶
| Line | Arithmetic | Result |
|---|---|---|
| Best decision now | act = 0.7 x 18 - 0.3 x 12 = 12.6 - 3.6; skip = 0; take the better | $9.0k |
| With perfect information | act only when told true: 0.7 x 18 + 0.3 x 0 | $12.6k |
| EVPI | 12.6 - 9.0 | $3.6k |
Notice where perfect information makes its money: only in the false branch, where the oracle saves you from a -$12k mistake you would otherwise take 30% of the time (0.3 x 12 = $3.6k - the same number from the other side). And the punchline the team will not enjoy: even a PERFECT study is worth $3.6k here, and the imperfect one on offer costs $5k. The prudent-sounding move loses money before it starts.
Why the verdict feels wrong and is not. At p = 0.7 the decision is not close - acting is already clearly better, so an oracle rarely changes the choice. Studies feel valuable in proportion to anxiety; they ARE valuable in proportion to how likely they are to flip the decision. Those are different quantities, and only one of them belongs in the memo.
The tent-shaped law, and the inversion
5 min liveRerun the same arithmetic at p = 0.5: act = 0.5 x 18 - 0.5 x 12 = $3.0k; perfect info = 0.5 x 18 = $9.0k; EVPI = 9.0 - 3.0 = $6.0k, which beats the $5k study - buy it. Same decision, same payoffs, opposite verdict. The only thing that moved was how CLOSE the call is - and that is the law.
LiveThe law that falls out of the tent3 min▶
Three readings of the same shape:
- Information is worth most where the decision is genuinely close. The peak sits at the decision boundary (p = 0.4 here, where acting breaks even) - the region where an oracle flips the choice most often.
- Near-certainty makes studies worthless. At either end of the tent, the oracle almost never changes what you would do anyway, so EVPI slides to $0.
- A decision the data cannot flip has EVPI = $0. Session 1's "nothing would change my mind" stakeholder, now with arithmetic: if no answer alters the action, the study's ceiling is zero - decoration, priced.
The tent also explains why the p = 0.7 verdict stung: the team's own confidence is what made the study worthless. You cannot simultaneously claim to be 70% sure and claim the study is essential - the memo makes that tension visible, in dollars.
LiveHubbard's measurement inversion3 min▶
Douglas Hubbard ran this arithmetic across his consulting practice - computed information values on 60+ business cases and 4,000+ variables - and reports a pattern he calls the measurement inversion:
Organizations measure what is easy, familiar, and already instrumented - and the variables that would actually move their decisions sit unmeasured, because nobody priced them. His working rule of thumb for that price, verbatim from his deck: "the value of information is simply the cost of being wrong times the chance of being wrong." (Check it against Part 1: cost of the wrong act = $12k, chance = 0.3, product = $3.6k - the EVPI.)
Sourcing character, honestly: these are Hubbard's practice findings from proprietary consulting data, not peer-reviewed research. The course teaches the inversion as a practitioner's well-documented field observation - useful as a prior to check on your own variables, not as settled science.
Fermi decomposition: which uncertainty carries the value
4 min liveEVPI so far priced the whole hypothesis. Real decisions decompose - and so does the value of information. Break the campaign's value into factors, ask which factor's uncertainty most moves the decision, and you have found where the VoI actually lives. Per the inversion, it is usually not the factor anyone proposed to measure.
LiveDecompose, then price the right factor3 min▶
The move takes five minutes in any meeting: write the value as a product of factors (uptake rate x retained value per save x segment size - cost), put your current 90% interval on each factor, and ask which interval's WIDTH most moves the final decision. That factor carries the VoI. The others - however easy or traditional to measure - carry little or none.
For the voucher case the decomposition exposes the request: the offered study would sharpen segment size, the factor everyone already knows, while uptake - the factor that decides whether the campaign nets +$18k or -$12k - stays a guess. If any study is worth running here, it is a small uptake pilot, not a $5k segmentation.
Ceiling, honestly. Real studies deliver IMPERFECT information, so EVPI is a ceiling, not a price: a study that would resolve about half your uncertainty is worth at most about half the ceiling. The calculator prices the ceiling anyway, because the ceiling test is the cheap first gate - a study that cannot clear it even if perfect, like tonight's $5k against a $3.6k ceiling, needs no further analysis at all.
The EVPI calculator, live
★ 20 min · everyone buildsPure live arithmetic - every number below recomputes as you drag. The preset is the voucher case exactly as the memo carries it: $12k campaign, +$18k if true, -$12k if false, $5k study on offer, p = 0.7.
Confirm the canon at the defaults. Read the board: EV of acting now $9.0k, EV with perfect information $12.6k, EVPI $3.6k - and the verdict line says do NOT buy the $5k study. That is Part 1's hand arithmetic, now live. If your board disagrees with your pencil, one of them has a typo and it is not the board.
Drag p down to 0.5 and watch the tent by hand. EVPI climbs as you drag - $3.6k at 0.7, $6.0k at 0.5 - and the verdict flips to buy along the way. You are walking down the right slope of the tent toward the peak: every step closer to the decision boundary makes the same study worth more.
Find the break-even points where EVPI = $5k. Drag slowly until the verdict flips and note BOTH flip points: coming down from high p the verdict flips to buy just below p = 0.58 (exactly 7/12), and coming up from low p it flips at about p = 0.28 (exactly 5/18). Between them is the buy zone; outside it, the study loses money even if perfect. Write both numbers - they are the study's shelf life as your belief moves.
Stress the payoffs: double the loss to $24k. Set the loss to 24 and watch the buy zone widen - at the p = 0.7 preset the verdict now says buy (EVPI $7.2k), and the zone stretches to roughly p = 0.28 through 0.79. Asymmetric stakes buy more information: the bigger the mistake an oracle can save you from, the more the oracle is worth. Reset to 12 when done.
Write two information verdicts. First the voucher case, into the memo: "EVPI ceiling $3.6k at our p = 0.7; study costs $5k; verdict: do not buy - decide now." Then one analysis request from your own week: its ceiling, its cost, its verdict - one sentence each. If you cannot state the decision it serves and the p it would move, you have found another $0 study.
What to bring to session 6. Your memo, now complete except the last section: default and criteria (s1), calibrated estimates (s2), base rates and updates (s3), graded uncertainty (s4), and tonight's information verdict. Session 6 assembles it and teaches the part that decides whether anyone trusts it: communicating the uncertainty without losing the room.
Before session 6
30 min- Price one live analysis request at work with the ceiling test: the decision it serves, the p, the payoffs, EVPI, the cost. One paragraph, verdict included. Expect at least one $0.
- Run the Fermi decomposition on one decision you own: factors, 90% intervals per factor, which width moves the call. Check for the inversion - is the measured factor the valuable one?
- Find your tent position - for your priced request, is the decision near a boundary (peak, information valuable) or near certainty (tail, decide now)? One sentence on which and why.
- Apply Hubbard's rule of thumb - cost of being wrong times chance of being wrong - to the same request and compare with your EVPI line. They should land close; reconcile them if not.
- Optional: Howard, "Information Value Theory" (IEEE 1966) - the original six pages; and Hubbard's measurement-inversion essay at hubbardresearch.com for the practice-data version.
Three questions
Clear all three and session 5 gets stamped in your passport.
1 · New decision: p = 0.6 that acting pays +$10k, else it loses $10k; skipping pays $0. What is the EVPI?
EVPI is always the DIFFERENCE: EV(decide after the truth is known) minus EV(best decision now). Answer A stops at the perfect-info value without subtracting; B stops at the act-now value. The $4k gap is the ceiling on any study for this decision - equal, per Hubbard's rule of thumb, to the $10k cost of being wrong times the 0.4 chance of being wrong.
2 · Why does EVPI peak near the decision boundary and vanish at both certainty ends?
Value of information is decision-first: it counts only the scenarios where knowing the truth changes what you do. Close call = frequent flips = high EVPI; near-certainty = the oracle confirms your plan = $0. The same logic prices "nothing would change my mind" at exactly $0 - session 1's detector, now in dollars.
3 · What does Hubbard's measurement inversion claim, and what kind of evidence backs it?
The claim is a "usually", not an "always" - and its basis is Hubbard's proprietary consulting data, honestly labeled as practice findings. That does not make it useless: it makes it a strong prior to CHECK with tonight's tools - decompose your decision, compare uncertainty widths, and see whether the measured factor is the valuable one.
What this session teaches, and where it came from
Every fact on this page, and its verification tier, is recorded in the course's source map.
Session 5 cheat sheet · pin this
The memo is built - default, criteria, calibrated estimates, base rates, graded uncertainty, information verdict. Session 6 delivers it: how to state uncertainty so trust survives it (the evidence says precise numeric ranges cost almost nothing, while vague verbal hedges are what erode you), which metrics survive being decided on, and the finished one-page memo your whole team can run. Bring everything; leave with the artifact.