learn-user-journey-analytics-with-phoebe / Analyst session 7 of 8
Learn User Journey Analytics with Phoebe · Analyst track · Session 7 of 8

Diagnose a journey leak

Monday morning. GMV is down 9% week over week, and the metrics course's driver tree has already done its job: the conversion lever fell, not sessions, not AOV. That is where the prequel stopped - and where this course earns its keep. Today you run the tree's verdict down through the journey layers: which gate, which door, which path, which screen, which step. By the end the leak has an address and an owner.

🟡 Analyst track Analysts · PMs · UX · growth ▶ Live flow simulator 45 min
0-3 · Welcome 3-14 · The extended workflow 14-40 · Live diagnosis + the finding 40-45 · Q&A
Part 0

Where you are

This is the second simulator session - b4's machine, run in reverse. In b4 you turned dials up and watched GMV grow; today a dial has been turned DOWN somewhere and you must find which one, using everything since b1: the map, the doors (b2), the paths (b3), the PDP instruments (b5), and the checkout micro-funnel (b6). The prequel's diagnosis session (its b5) covered the tree; this session covers everything underneath it.

Live - presented in session Self-study - read after class ▶ Live flow - editable & runnable Sources covered
★ What you walk out with today The full six-step drill-down from a GMV alarm to a single screen, the simulator proof of why the top line alone cannot locate a leak, and a fill-in-the-blanks finding template that ends with a screen and an owner.
Part 1 · the drill-down from money to screen

The extended diagnosis workflow 11 min live

The metrics course taught steps 1-2: confirm the drop is real, then let the driver tree name the lever. This course adds four more layers underneath the conversion lever. Six steps, each one narrowing the search space, each one using an instrument you have already built in this track.

Confirm it is real. Tracking outage? Seasonality? Check event volumes and the same week last year before anyone panics. A fake drop diagnosed as real wastes a week.

Tree: which lever. GMV = sessions x CR x AOV. The prequel's driver tree already ran: sessions held, AOV held, CR fell. Done - inherited from the metrics course.

Gates: which of the four fell. CR is a product of PDP penetration x add-to-cart x checkout start x completion. Pull the four gate rates WoW and find the one that moved.

Doors + paths: mix or quality. A gate is an average across five doors and two paths. Use b2's decomposition: did the door MIX shift, or did one door's QUALITY fall?

Screen / step: where exactly. Drill into the screen-level instruments - b5's engagement events on the PDP, b6's step completions in checkout - until you can name one screen or one step.

Write the screen-level finding. One paragraph: the drop, the lever, the gate, the door, the screen, the suspected cause, the owner, the action, the re-check date. Demo 2 gives you the template.

Monday alarm: GMV down 9% WoW the report line everyone sees - and where most meetings stop steps 1-2 · confirm real, then the driver tree (metrics course b5) Lever: conversion rate fell sessions held · AOV held - the tree already ruled them out step 3 · the four gates, WoW Gate: PDP penetration fell 40% to 36.4% add-to-cart, checkout start, completion all held step 4 · doors and paths - mix or quality? (b2, b3) Door: Home quality fell, mix held Home reach-PDP 20% to 16% - every other door held step 5 · screen and step instruments (b5, b6) Screen: the homepage banner takeover buried the catalog modules step 6 · write it down The finding: screen + owner + action + re-check one paragraph, no meeting required Each level shrinks the search space: from all of GMV to one screen with one owner, in six steps.
🔍 Click to zoom - the six-step drill-down: from a 9% GMV alarm to one screen with one owner
LiveWhy the extra layers exist at all4 min▶

The prequel could say "CR fell" - and then it had to stop, because CR is a blend of four gates, five doors, two paths, and a dozen screens. Everything you built in b1-b6 exists for exactly this Monday:

  • The gates (b4's machine) split CR into four owned rates - the first cut.
  • The doors (b2's decomposition) split each gate into mix versus quality - the second cut.
  • The screen instruments (b5's engagement events, b6's checkout steps) point at pixels - the final cut.

Without the layers, "CR fell 9%" becomes a room full of teams each explaining why it is not them. With the layers, it becomes one team's Tuesday ticket.

Demo 1 of 2

Run the diagnosis live ★ 15 min · everyone works

Here is b4's machine again, loaded with three competing hypotheses instead of four fixes. Each lever recreates this week's reported drop through a different gate. Your job: prove the top line cannot tell them apart, then break the tie with gate-level data.

{
  "sessions": 360000, "aov": 68,
  "stages": [
    { "label": "Sessions" },
    { "label": "PDP sessions",      "rate": 0.40, "rateLabel": "PDP penetration" },
    { "label": "Cart sessions",     "rate": 0.25, "rateLabel": "add-to-cart" },
    { "label": "Checkout sessions", "rate": 0.45, "rateLabel": "checkout start" },
    { "label": "Orders",            "rate": 0.60, "rateLabel": "completion" }
  ],
  "levers": [
    { "stage": 1, "to": 0.364, "label": "This week: PDP penetration fell", "desc": "40% down to 36.4% - the reported drop, recreated" },
    { "stage": 2, "to": 0.2275, "label": "Alt hypothesis: add-to-cart fell", "desc": "25% down to 22.75% - produces the same -9%" },
    { "stage": 4, "to": 0.546, "label": "Alt hypothesis: completion fell", "desc": "60% down to 54.6% - also the same -9%" }
  ]
}

Toggle each hypothesis alone. All three land on roughly the same orders line, about 9% below baseline. No coincidence: 0.364/0.40 = 0.2275/0.25 = 0.546/0.60 = 0.91, and in a multiplicative flow a 0.91x on ANY single gate passes straight through as a 0.91x on orders. The top line literally cannot tell them apart - that is WHY the journey layers exist.

Break the tie with gate-level counts. Now pull the stage counts, not the ratios of ratios. This week PDP sessions fell to about 131,000 (360,000 x 36.4%), while cart-rate-of-PDP and both checkout gates held at 25% / 45% / 60%. Hypothesis 1 wins: the leak is above the PDP, in getting people TO products.

Drill into the doors with b2's table. Door mix held - same five shares as always. But Home's reach-PDP quality fell from 20% to 16% after a homepage banner takeover buried the catalog modules; every other door held its rate. The leak has an address: the homepage.

Three hypotheses, one top line: why -9% cannot locate a leak GATE 1 · PDP PENETRATION 40% → 36.4% 0.364 / 0.40 = 0.91 orders about -9% indistinguishable on the top line GATE 2 · ADD-TO-CART 25% → 22.75% 0.2275 / 0.25 = 0.91 orders about -9% indistinguishable on the top line GATE 4 · COMPLETION 60% → 54.6% 0.546 / 0.60 = 0.91 orders about -9% indistinguishable on the top line Break the tie with stage COUNTS, not ratios of ratios PDP sessions fell to about 131,000 (360,000 x 36.4%) while 25% / 45% / 60% all held - hypothesis 1 wins In a multiplicative flow a 0.91x on ANY single gate passes straight through as a 0.91x on orders. That is why b8's scorecard tracks the four gate counts weekly, not just the blended conversion rate.
🔍 Click to zoom - identical top lines, three different owners; only the counts can tell them apart
Real world

This ambiguity is the most common diagnosis failure in the wild. A war room stares at "CR down 9%" and three teams each run their own analysis proving it is someone else's gate. Gate-level session counts settle it in one query - which is why b8's scorecard tracks the four gate COUNTS weekly, not just the blended CR.

Demo 2 of 2

Write the screen-level finding ★ 10 min · everyone writes

A diagnosis that lives in a dashboard dies in a dashboard. The deliverable is one paragraph, in a fixed shape, that any executive can read in twenty seconds and any team can act on. Memorize the template; fill it every time.

LiveThe template, then the filled version6 min▶

The template: "GMV fell [X]% WoW. Tree: CR lever. Gate: [gate] fell [from] to [to]. Doors: [mix or quality, which door]. Screen: [the address]. Suspected cause: [change/incident]. Owner: [team]. Action: [fix]. Re-check: [date]."

FieldThis week, filled in
The dropGMV fell 9% WoW
TreeCR lever - sessions and AOV held (metrics course b5)
GatePDP penetration fell 40% to 36.4% (PDP sessions ~131,000); other three gates held
DoorsQuality, not mix: Home reach-PDP 20% to 16%, every other door held
ScreenHomepage
Suspected causeMonday's banner takeover buried the catalog modules below the fold
OwnerHomepage / merchandising team
ActionRestore the catalog modules above the fold
Re-checkFriday - Home reach-PDP back at ~20%, PDP sessions back at ~144,000
A journey finding ends with a SCREEN and an OWNER, not a metric "CR fell 9%" is a fact; "the homepage banner buried the catalog, merchandising restores the modules, re-check Friday" is a finding. If your last sentence names a number instead of a screen and a team, you stopped one layer too early - go back to step 5.
Self-studyThe false-positive checklist - before you accuse a screen4 min read▶

Three impostors produce exactly this signature without any real behavior change. Run this checklist between step 1 and step 6, every time:

  • Instrumentation outage. A screen stopped firing its events - b1's trap. A silent view_item makes PDP penetration "fall" while shoppers browse happily. Check raw event volumes per screen BEFORE any business panic; a screen at near-zero events is a bug report, not a leak.
  • App release the same day. A new version can change what fires and when. Split every gate by app version: if the drop lives entirely in the new build, suspect the build - either its tracking or a genuinely broken screen it shipped.
  • Campaign mix. A paid flood through a weak door (say, a burst of Other-door retargeting) drags the blended gate down while every door's own quality holds. That is a mix effect, not a quality leak - b2's decomposition separates them in one table.
Before you accuse a screen: three impostors with the same signature A gate rate fell run this before step 6 Screen events at near-zero? b1's trap - a silent view_item App release the same day? split every gate by app version Paid flood through a weak door? b2's decomposition separates it yes yes yes BUG REPORT check raw event volumes BUILD ISSUE the drop lives in one build MIX EFFECT every door's quality held No to all three: it is a real quality leak - go to step 5 and name the screen. Run it between step 1 and step 6, every time - all three produce the signature with no behaviour change.
🔍 Click to zoom - three of the cheapest checks in analytics stand between a dashboard and a false accusation
Homework

Try it yourself - this week ◐ 25-35 min total

Sources

Sources this session draws on

Diagnosis is where the whole two-course stack pays off at once. This page draws on:

✓Metrics-course diagnosis workflow - its b5, diagnose the drop (steps 1-2)Part 1 · the tree verdict this session inherits and extends
✓Mango Lane journey canon - gates, doors, paths, all python-verified (b1-b3)Demo 1 · every count in the diagnosis cross-checks
✓Instrumentation traps - silent screens and unfired events (this course's b1)Self-study · false positive number one
◐The finding template and false-positive checklist - practitioner craft, not a fetched primary sourceDemo 2 · field-tested shape, honestly flagged as craft
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · Three different gate drops all produced the same -9% orders in the simulator. Why?

0.364/0.40 = 0.2275/0.25 = 0.546/0.60 = 0.91. The top line carries the product of the gates, so it cannot say which factor moved - that ambiguity is the whole reason the journey layers exist.

2 · What data breaks the tie between the three hypotheses?

The blended CR is ambiguous; the per-stage counts are not. One query showing PDP sessions down and cart/checkout/completion rates flat localizes the leak above the PDP immediately.

3 · Which of these is a COMPLETE journey finding?

A finding ends with a screen and an owner, not a metric. Option A stops at step 3 of 6; the template forces you all the way down to the homepage, merchandising, and Friday's re-check.

Analyst session 7 cheat sheet · pin this

The six stepsConfirm real → tree (lever) → gates → doors + paths → screen/step → write the finding.
Top-line ambiguityA 0.91x on any gate = 0.91x on orders. The blended number cannot localize a leak.
The tie-breakerGate-level COUNTS: this week PDP sessions ~131,000, later gate rates held. Hypothesis 1 wins.
Mix vs qualityb2's decomposition: door shares held, Home reach-PDP fell 20% → 16%. Quality leak, one door.
The finding templateDrop, tree, gate, doors, screen, cause, owner, action, re-check. One paragraph, fixed shape.
Screen + owner ruleA finding ends with a screen and an owner, not a metric. Otherwise go back one layer.
False positivesInstrumentation outage, same-day app release, campaign mix flood. Check all three first.
No-action findings count"Seasonal gravity, no action" is a valid ending. Next: b8 makes the whole drill a standing scorecard.