learn-metric-decomposition-with-phoebe / Builder session 3 of 10
Learn Metric Decomposition with Phoebe · Builder track · Session 3 of 10

Leading indicators & the playbook

A tree tells you what a metric is made of. Two things turn that into a working diagnostic habit. First, leading indicators - the drivers near the leaves that move first and warn early, so you catch a fall before the board does. Second, a diagnosis playbook - the fixed four-step runbook you follow every time a top line drops, so you never freelance under pressure. Today you learn both and run them live on real store data.

🟢 Builder track Analysts · PMs · founders · ops Live tree simulator + SQL 45 min
0-3 · Recap 3-16 · Leading indicators 16-42 · The playbook, live 42-45 · Q&A
Part 0

Recap, and today's job

You can now build a tree (b1) and name its shape - product, sum, or bridge (b2). But a tree at rest is just a diagram. Its value shows up the morning a number is down and everyone is looking at you. This session gives you the two things that turn a static tree into a diagnostic reflex: leading indicators that warn you early from near the leaves, and a four-step playbook you run the same way every single time. By the end you will have diagnosed a live GMV drop end to end - tree, then SQL, then a named cause.

Live - presented in session Self-study - read after class ▶ Live tree - editable & runnable Framework sources covered
★ What you walk out with today A test for whether a metric qualifies as a leading indicator (predictive, early, controllable), and the four-question playbook - is it real? which branch? which segment? cause confirmed? - run live from a coral tree trail into a GROUP BY.
Part 1 · covers Amplitude's North Star, leading inputs

Finding leading indicators 6 min live

A top-line metric is lagging - it reports after the fact. A leading indicator is a driver that moves first, giving you lead time to act before the lagging number lands. But not every early number is useful. A good leading indicator passes three tests: it is predictive (it correlates with the future outcome), early (it moves with real lead time, not the same day), and controllable (a team can actually push it). Miss any one and you have a distraction, not a signal.

Week 1 Week 3 Week 5 Week 7 Lagging: GMV Leading: add-to-cart rate leading dips at week 3... ...lagging follows at week 4 A real leading indicator moves first and predicts the fall - that gap is your reaction time.
🔍 Click to zoom - the leading driver dips before the lagging outcome; the gap is lead time to act
LiveThe three tests: predictive, early, controllable3 min

Deeper in a tree, drivers get more leading. GMV lags; traffic is more leading than GMV; add-to-cart rate leads traffic's revenue effect; ad impressions lead all of it. But depth alone is not enough - a leading indicator earns its place only if you can act on it. Run every candidate through three tests.

  • Predictive: does last period's move in this metric actually forecast this period's outcome? If there is no correlation, it is noise.
  • Early: does it move with real lead time - days or weeks before the outcome, not the same afternoon? Same-day is not a warning.
  • Controllable: can a team change it on purpose? A weather index may predict sales but nobody owns the weather.
Real world

Facebook's "7 friends in 10 days". The classic leading indicator: new users who added 7 friends within 10 days almost always stuck. It was predictive (it forecast retention), early (visible in week two, not month six), and controllable (onboarding could nudge it). Three tests, all passed - so the whole company steered by it.

Self-studyValidating a leading indicator against your own data2 min read

The predictive test is not a vibe - it is a query. Line up your candidate leading metric from last period next to the outcome this period, across many periods, and check whether they move together. If last month's add-to-cart rate reliably predicts this month's GMV, you have a real leading indicator. If the scatter is a cloud, you have a coincidence someone fell in love with.

  • Lag it: shift the candidate back one period and correlate with the outcome. A leading indicator correlates when lagged; a lagging one only correlates at zero lag.
  • Hold it up over time: one lucky month proves nothing. Look for the relationship holding across a stretch of periods.
  • Beware reverse causation: sometimes the outcome drives the "leading" metric, not the other way. Direction matters.
Part 2 · covers Amazon WBR, the diagnosis runbook

The diagnosis playbook 7 min live

When a top line drops, panic makes people freelance - they jump to a favourite theory and go hunting for confirming numbers. The playbook replaces that with a fixed order of four questions, run the same way every time: is it real, which branch, which segment, cause confirmed? The order matters. Skip "is it real?" and you fire-drill over noise. Skip "which branch?" and you segment the wrong driver. Discipline in the sequence is the whole edge.

1 · Is it real? noise vs signal - check a baseline first 2 · Which branch? walk the tree, find the driver that moved 3 · Which segment? GROUP BY the guilty driver 4 · Cause confirmed? Each step narrows the question. Run them in order - skipping one is how diagnoses go wrong.
🔍 Click to zoom - four questions, narrowing from "is this even real?" to a confirmed cause
LiveStep 2 in the simulator: walk the tree to the moved driver3 min

Step 1 is judgement against a baseline (Part 3). Step 2 is mechanical: walk the tree and find the coral. Press Simulate a drop below - one driver falls, the path to GMV lights up, and step 2 is answered in a glance. That is the entire point of having built the tree in b1: the branch question becomes a five-second read instead of an argument.

{
  "unit": "$",
  "root": {
    "label": "GMV", "op": "x",
    "children": [
      { "label": "Traffic",    "value": 120000, "unit": "visits" },
      { "label": "Conversion", "value": 0.028,  "pct": true },
      { "label": "AOV",        "value": 62,     "unit": "$" }
    ]
  }
}
LiveStep 3 in SQL: segment the guilty driver3 min

The tree named the driver. Step 3 splits it. If the tree pointed at conversion or revenue, the first cut is almost always by channel - it is cheap, it is owned, and it usually isolates the problem to one place. Run this to see revenue split by channel.

SELECT o.channel,
       COUNT(DISTINCT o.order_id)                 AS orders,
       ROUND(SUM(oi.quantity * oi.unit_price), 0) AS revenue
FROM orders o
JOIN order_items oi ON o.order_id = oi.order_id
WHERE o.status = 'completed'
GROUP BY o.channel
ORDER BY revenue DESC;
Part 3 · covers Amazon WBR baselining

Baselines: is the drop even real? 5 min live

Step 1 of the playbook - "is it real?" - lives and dies on the baseline you compare against. Week-over-week catches fast breaks but trips on weekly seasonality. Year-over-year controls for seasonality but hides recent trend. Versus-plan tells you if you are off target but not why. The classic trap is comparing a Monday to a Saturday and declaring a crisis. Pick the baseline that matches the metric's natural rhythm before you call anything a drop.

LiveCompare this period to a baseline period4 min

Before you diagnose a fall, prove it against a baseline. This query pulls completed revenue by month, then uses a window function to show each month's change against the prior month - the simplest real baseline. A dip that is just the usual month-to-month wobble is not a signal.

WITH monthly AS (
  SELECT substr(o.order_date, 1, 7)          AS month,
         SUM(oi.quantity * oi.unit_price)     AS revenue
  FROM orders o
  JOIN order_items oi ON o.order_id = oi.order_id
  WHERE o.status = 'completed'
  GROUP BY substr(o.order_date, 1, 7)
)
SELECT month,
       ROUND(revenue, 0)                                      AS revenue,
       ROUND(revenue - LAG(revenue) OVER (ORDER BY month), 0) AS vs_prior_month
FROM monthly
ORDER BY month;
Match the baseline to the rhythm Daily metric with weekly seasonality? Compare same-day-of-week or use year-over-year. Slow monthly metric? Month-over-month is fine. Never compare across a seasonal boundary and call the gap a problem.
Demo 1 of 2

Run the playbook end to end ★ 12 min · everyone builds

Now all four questions in one pass. Simulate a drop on the tree (steps 1 and 2), then segment the guilty driver in SQL (step 3), then read the outlier and name a cause (step 4). This is exactly the sequence you will run the next time a real dashboard is red - practise it until the order is muscle memory.

{
  "unit": "$",
  "root": {
    "label": "GMV", "op": "x",
    "children": [
      { "label": "Traffic",    "value": 120000, "unit": "visits" },
      { "label": "Conversion", "value": 0.028,  "pct": true },
      { "label": "AOV",        "value": 62,     "unit": "$" }
    ]
  }
}
SELECT o.channel,
       COUNT(DISTINCT o.order_id)                 AS orders,
       ROUND(SUM(oi.quantity * oi.unit_price), 0) AS revenue,
       ROUND(SUM(oi.quantity * oi.unit_price) * 1.0
             / COUNT(DISTINCT o.order_id), 2)     AS avg_order_value
FROM orders o
JOIN order_items oi ON o.order_id = oi.order_id
WHERE o.status = 'completed'
GROUP BY o.channel
ORDER BY revenue DESC;

Q1 is it real? Simulate a drop, then ask whether a fall this size would clear your baseline. A 2% wobble is noise; a 20% coral trail is a signal.

Q2 which branch? Read the coral path from the moved leaf up to GMV. Say which driver moved and by what percent.

Q3 which segment? Run the SQL. Split revenue and AOV by channel and find the channel that is dragging the total down.

Q4 cause confirmed? Form a hypothesis ("app checkout broke") and name the one follow-up query that would confirm it. Reset and run the whole loop again.

Real world

This is the Amazon weekly business review in miniature. The WBR ritual is not magic - it is this exact loop run on a cadence: is the number off its baseline, which input moved, which segment owns it, what is the corrective action. Teams that run it well are not smarter; they refuse to skip a step.

Demo 2 of 2

Your turn: segment and baseline ★ 10 min · everyone builds

Two live queries and one prose answer. First check whether a drop is concentrated in one channel, then check this period against a baseline, then define a leading indicator for a metric you actually own.

LiveQ1 · Refund rate by channel - is the drop concentrated?3 min

A GMV drop can hide in refunds, not just missing orders. Run this to see the refund-and-cancel rate by channel. If one channel refunds far more than the other, the fall is concentrated - and you have your lead.

SELECT channel,
       COUNT(*)                                              AS all_orders,
       COUNT(*) FILTER (WHERE status IN ('refunded','cancelled')) AS lost_orders,
       ROUND(100.0 * COUNT(*) FILTER (WHERE status IN ('refunded','cancelled'))
             / COUNT(*), 1)                                  AS lost_rate_pct
FROM orders
GROUP BY channel
ORDER BY lost_rate_pct DESC;
LiveQ2 · Revenue this month vs a baseline month3 min

Prove a drop before you chase it. This pulls completed revenue and distinct orders by month, so you can eyeball whether the latest month is genuinely below its recent baseline or just wobbling.

SELECT substr(o.order_date, 1, 7)               AS month,
       COUNT(DISTINCT o.order_id)              AS orders,
       ROUND(SUM(oi.quantity * oi.unit_price), 0) AS revenue
FROM orders o
JOIN order_items oi ON o.order_id = oi.order_id
WHERE o.status = 'completed'
GROUP BY substr(o.order_date, 1, 7)
ORDER BY month;
Self-studyQ3 · Name a leading indicator for your own metric2 min write

Pick a lagging metric you report. In two or three sentences, name one leading indicator for it and argue why it qualifies on all three tests: is it predictive (does last period's move forecast the outcome?), early (does it move with real lead time?), and controllable (can a team push it on purpose?). If it fails any one test, say which - and what you would use instead.

The honesty check Most "leading indicators" people quote fail the controllable test - they predict but nobody owns them. A leading indicator you cannot move is a weather forecast, not a lever.
Homework

Try it yourself - this week ◐ 20-30 min total

Framework sources

Frameworks this session draws on

Leading indicators and the diagnosis runbook are distilled from a handful of durable frameworks, applied here live instead of described. This page draws on:

Amplitude North Star Playbook - inputs that lead the outputPart 1 · predictive, early, controllable inputs vs the lagging star
Amazon Weekly Business Review - diagnosis on a cadenceParts 2-3 & Demo 1 · the four-step loop and baselining; full ritual in a5
Lean Analytics (Croll & Yoskovitz) - leading vs lagging, actionable metricsPart 1 · a metric you cannot act on is a distraction
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · What are the three tests a good leading indicator must pass?

A leading indicator must forecast the outcome (predictive), move with real lead time (early), and be something a team can push on purpose (controllable). Fail one and it is a distraction, not a signal.

2 · What is the correct order of the diagnosis playbook?

Prove the drop against a baseline first, then walk the tree to the moved driver, then GROUP BY that driver to find the slice, then confirm a cause. Skipping "is it real?" starts fire drills over noise.

3 · Why does the choice of baseline matter so much?

A baseline that ignores the metric's rhythm manufactures false drops. Match the comparison to the seasonality - same-day-of-week or year-over-year for seasonal metrics - before you call anything a fall.

Builder session 3 cheat sheet · pin this

Leading indicatorA driver that moves before the outcome and gives you time to act. Near the leaves of the tree, not the root.
Three testsPredictive (forecasts the outcome), early (real lead time), controllable (a team can push it). Fail one, drop it.
Validate by laggingShift the candidate back one period and correlate with the outcome. Holds across many periods = real signal.
The playbookIs it real? -> which branch? -> which segment? -> cause confirmed? Same order, every time.
Step 1 is a baselineProve the drop clears normal noise before diagnosing. No baseline, no drop - just a wobble.
Tree then SQLStep 2 reads the coral branch; step 3 is a GROUP BY on that driver. Never SQL-first.
Match the baselineWoW for fast breaks, YoY for seasonality, vs plan for targets. Never compare across a seasonal boundary.
Running skillWarn early with leading indicators, diagnose in four steps. Next: b4, the ecommerce deep dive.