Why this session is the heart of the course
Sessions 1-3 taught you to read data honestly. This session teaches you to run a business with it. Metrics are the steering wheel of management - and most companies drive with a broken one: too many numbers, the wrong numbers, or numbers that reward exactly the behavior nobody wants. Today we pick the right ones, wire them into a tree, and practice diagnosing a revenue drop the way an analyst does: mechanically, branch by branch.
What is not measured cannot be improved 4 min live
Peter Drucker's principle built modern management: no measurement, no improvement. But it has a dark twin that costs companies billions - what IS measured gets gamed. Data literacy means holding both truths at once.
LiveDrucker's principle - and Goodhart's revenge2 min▶
Drucker: you cannot manage what you do not measure. Goodhart's law is the counterweight: when a measure becomes a target, it ceases to be a good measure - because people optimize the number instead of the goal behind it. Every metric you put a bonus on will be gamed; the only question is how expensively.
Wells Fargo, 2016: the target was "eight accounts per customer" ("eight is great"). Under relentless pressure, employees opened roughly 3.5 million accounts customers never asked for. The metric looked magnificent right up until the $3 billion in fines and a CEO resignation. The measure became the target, and the target ate the bank's reputation. Every target needs a guardrail - we build those in Part 2.
LiveWhat makes a metric good - and the vanity trap2 min▶
Croll & Yoskovitz (Lean Analytics) give the working checklist. A good metric is:
| Criterion | Meaning | Quick test |
|---|---|---|
| Comparable | vs last month, vs a cohort, vs a competitor | "is 12% good?" should have an answer |
| Understandable | explainable in one breath, no footnotes | can the whole exec team define it the same way? |
| A ratio or rate | ratios beat raw counts - they carry context | revenue per buyer beats revenue; churn rate beats churn count |
| Behavior-changing | if it moves, someone does something different | "what would we do if this halved?" has an answer |
Vanity vs actionable: registered users, total downloads, cumulative anything - numbers that only go up and change no decisions. Weekly active buyers, repeat purchase rate, revenue per visitor - numbers a team can move and be judged by. The vanity test is brutal and simple: what would we do differently if this number were half?
Color Labs, 2011: raised $41M pre-launch, hit the top of the app-store charts, and the download chart looked like a rocket. But activation cratered - most users opened the photo app once, found an empty network, and never returned. The team celebrated downloads while the metric that mattered (users actually sharing photos in week one) was near zero. Shut down within two years. Downloads are a vanity metric wearing a party hat.
North Star and the metric system 4 min live
A metric system has three parts: one North Star that captures value actually delivered to customers, a handful of input metrics teams can pull on this quarter, and guardrail metrics that stop the gaming Goodhart promised. Star, levers, brakes.
LivePicking the star - value delivered, not value promised2 min▶
- Airbnb: nights booked. Not registered users, not listings - nights people actually slept. Both sides of the marketplace show up in one number.
- Spotify: time listening. Subscriptions are the lagging receipt; listening is the value that renews them.
- Amazon: no single star - a flywheel. Lower prices → more customers → more sellers → more selection → lower cost structure → lower prices. Their answer is hundreds of input metrics reviewed weekly (Part 3) rather than one star. Big enough companies graduate from a star to a system.
- The pattern: a North Star counts value delivered. If your candidate metric can go up while customers get nothing (downloads, sign-ups, pipeline), it is an input at best and a vanity metric at worst.
Self-studyOKRs vs KPIs in one line1 min read▶
KPIs are the dashboard you always watch (health); OKRs are the quarterly bet you are trying to move (change). A KPI tree tells you what is true; an OKR picks which branch you will fight over next quarter. Same numbers, different job - do not let a planning workshop blur them.
KPI trees: the P&L as arithmetic 7 min live
Any money metric decomposes into arithmetic: income = revenue - cost; revenue = buyers × basket size × item value; buyers = new + returning + resurrected - churned. Each leaf is ownable by one team and testable with data. Decomposition makes diagnosis mechanical - no meetings, just subtraction.
LiveDiagnosing "revenue down 8%" - the walk4 min▶
- Layer 1: income = revenue - cost. Revenue fell, cost flat → go left.
- Layer 2: revenue = buyers × basket size × item value. Basket and item value flat, buyers down → go to the buyers branch.
- Layer 3: buyers = new + returning + resurrected - churned. New buyers flat, returning down 12% → found it.
- The payoff: the same "revenue down 8%" headline sends the Monday call to a different team depending on which leaf moved. Returning buyers down → retention and CRM. Basket size down → merchandising. New buyers down → marketing. Item value down → pricing. Without the tree, all four teams sit in one long defensive meeting; with it, three of them get their morning back.
Amazon's Weekly Business Review: hundreds of input metrics - not the outputs - reviewed weekly against expectations, out loud. Anomalies get an owner before they get an excuse. The discipline is the point: outputs like revenue are lagging and ungovernable; inputs like price, selection and delivery speed are where hands touch levers. The WBR is a KPI tree read aloud every week.
Self-studyGrowth accounting - the honest growth mirror2 min read▶
The buyers layer has a name: growth accounting. This month's buyers = last month's + new + resurrected - churned. Run it monthly and you learn whether growth is real acquisition or a leaky bucket refilled faster - two stories with identical headline numbers and opposite futures. A company adding 500 new buyers while churning 450 is not growing 500; it is renting growth at full price. Ask your analyst for this one chart: monthly buyers split into new / returning / resurrected / churned bars. It takes an hour to build and it ends most growth debates.
Drive the tree yourself ★ 10 min · everyone drives
The tree from Part 3, live. Every leaf is a slider - move one and watch the arithmetic ripple to income. Your mission: develop a feel for which levers punch hardest.
Read the baseline out loud first: 400 new + 700 returning + 100 resurrected - 200 churned = 1,000 buyers; × 3 items × $25 = $75,000 revenue; - $52,000 cost = $23,000 income. Every number on screen is one leaf of the Part 3 tree.
Challenge 1 - the churn shock: churn doubles from 200 to 400. Predict income before you move the slider, then move it. Most people underestimate the damage.
Challenge 2 - the merchandising play: reset, then raise basket size from 3 to 3.5 instead. Compare the income change with challenge 1. Which lever moved more per "unit of effort"?
Challenge 3 - the recovery: apply the churn shock again, then find the cheapest combination of other levers that gets income back to $23,000. Argue with your neighbor about whose path is more realistic to execute.
Build YOUR tree ★ 6 min · everyone draws
Take your business's top metric (revenue, GMV, bookings, ARR - whatever the board watches) and write it at the top of a page.
Decompose it one layer using the formula trick: any metric = its arithmetic parts. Who does it, times how often, times how much - or units × price, or accounts × seats × price per seat. There is always a multiplication hiding inside.
Name one owner per branch. If a branch has no owner, circle it - you just found out why it never improves.
Compare with a neighbor: the most common discovery is a "returning customer" branch that nobody formally owns while three teams own the "new customer" branch.
Vanity audit of your board pack ★ 5 min · everyone judges
List the 5 headline metrics from your last board pack or monthly review, from memory. (Struggling to remember them is itself a finding.)
Tag each one: actionable (a ratio or rate, comparable, would change a decision if it halved) or vanity (only goes up, changes nothing).
Take your worst vanity metric and propose the replacement: cumulative registered users → weekly active buyers; total downloads → week-one activation rate; pipeline added → win rate.
Write the sentence you will use to propose the swap at the next review. Bonus points for saying "what would we do differently if this were half?" out loud.
This week ◐ 30 min total
- Finish your KPI tree from Apply-along 2: two layers deep, one owner per leaf. Share it with the named owners and watch the conversation it starts.
- Run the vanity audit for real: open the actual board pack, tag all metrics, and bring your worst offender + proposed replacement to next session.
- Ask your analyst for the growth-accounting chart (new / returning / resurrected / churned buyers by month). It is the self-study card made real.
- Optional deep end: Croll & Yoskovitz, Lean Analytics, chapters 2-3 - the vanity vs actionable distinction and the One Metric That Matters, in full.
Three questions before you go 🎯 ◐ 90 seconds
1 · Goodhart's law says...
Targets are fine - naked targets are not. Wells Fargo's "eight accounts per customer" produced millions of fake accounts. The fix is pairing every target with a guardrail metric, not abandoning measurement.
2 · Which is the best North Star for a meal-kit subscription business?
A North Star counts value delivered, not value promised. Accounts and downloads can climb while nobody cooks anything (the Color Labs trap). Meals cooked predicts retention, referrals and revenue - and every team can trace its work to it.
3 · Revenue is down 8% but buyer count is flat. Which branch of the tree do you check first?
Revenue = buyers × basket size × item value. If buyers are flat, arithmetic says the basket branch moved - a merchandising or pricing call, not a marketing one. The tree tells you who gets the Monday call before anyone opens a slide deck.
Concepts covered & their origins
This session teaches the working 80% of each idea and credits the canon - the books stay the deep end for anyone who wants more.