learn-customer-retention-with-phoebe / Stakeholder session 3 of 4
Learn Customer Retention with Phoebe · Stakeholder track · Session 3 of 4

Levers that move retention

Sessions 1 and 2 taught you why retention is the money metric and how to read the charts. Today is the fun part: what actually moves the number. We walk the best-published case in the industry end to end - Duolingo's four-year climb - then visit the studio that kills its own games for missing a retention bar, and finish with the loyalty and winback economics of ecommerce. Every story on this page carries an honesty label, because retention lore is full of numbers nobody can source.

🟡 Stakeholder track Leaders · PMs · no code Case studies, folklore-checked 45 min
0-3 · Welcome 3-20 · Duolingo end to end 20-40 · More levers 40-45 · Q&A
Part 0

Where this session sits

Four stakeholder sessions, no code, one goal: make you the person in the room who asks the right retention questions. s1 made the money case, s2 taught you to read the cohort chart. Today is the lever menu - what teams actually pull, with the published evidence for each pull. s4 closes the loop with the questions to ask your data analyst. One standard holds throughout - the same one worth enforcing in your own meetings: a case only counts if you can say who published it and how it was measured. The labels DOCUMENTED and SEMI-FOLKLORE appear next to every story, and learning to demand those labels is half of today's value.

Live - presented in session Self-study - read after class ✓ Documented vs ~ folklore - every case is labeled Sources covered
★ What you walk out with today Duolingo's growth state machine and why one metric (CURR) beat every other candidate, the Supercell kill gate and what it says about retention versus monetization order, a lever menu sorted by which part of the retention curve each lever moves, and a folklore scorecard so you can tell a documented case from a conference legend in ten seconds.
Part 1 · covers the Duolingo CURR case (Jorge Mazal, Lenny's Newsletter - DOCUMENTED)

Duolingo: the best-published retention case 17 min live

Most retention stories are one sentence and a suspiciously round number. Duolingo's is different: Jorge Mazal, the former Chief Product Officer, wrote the whole playbook up in Lenny's Newsletter - the model, the metric contest, the levers, the results. It starts with a decision that sounds boring and turns out to be everything: stop treating users as one blended pool, and model them as a state machine. Seven buckets, and every user is in exactly one.

The 7-bucket growth state machine: four buckets fill DAU, three states leak Counted in today's DAU New Current Reactivated Resurrected CURR - the north star Current users who stay Current Leaking out of DAU At-risk WAU At-risk MAU Dormant Dark red arrows are the leak; dashed teal arrows are the comebacks (Reactivated, Resurrected). Sensitivity analysis ranked every arrow: moving CURR had ~5x the DAU impact of the next-best metric.
🔍 Click to zoom - seven buckets, one highlighted loop: keep Current users Current and everything downstream compounds
LiveWalk the machine: seven buckets, one loop that matters7 min▶

Every Duolingo user sits in exactly one bucket on any given day. Four buckets are active today and sum to DAU: New (first day ever), Current (active today and recently active), Reactivated (back after a short absence), and Resurrected (back after a long absence, 30+ days away). Three states are the leak: At-risk WAU (slipping this week), At-risk MAU (gone for weeks), and Dormant (gone a month or more). Users flow along the arrows every day.

  • Why this beats "our retention is 34%": a single blended number cannot tell you where the leak is. The machine can - it shows exactly which arrow is too thick and which comeback arrow is too thin.
  • The metric contest: the growth team ran a sensitivity analysis - nudge each transition rate, watch what happens to DAU. One rate crushed the field: CURR, the current-user retention rate (the self-loop on Current). Moving it had roughly 5x the DAU impact of the next-best metric. It became the north star not by opinion but by arithmetic.
  • Why CURR wins mechanically: Current is the biggest bucket, and a user kept Current today is exposed to being kept again tomorrow. Small gains to the self-loop compound daily. Reactivation flows, by contrast, act on smaller populations and only once per lapse.
  • The result: about four years of focused work later, CURR was up 21% and DAU roughly 4.5x - with acquisition spend flat. Nobody bought that growth. They kept it.

The levers behind those four years, per Mazal: leaderboards and leagues (a documented +17% lift in learning time), streaks with the streak freeze (the habit loop, plus forgiveness so one missed day does not destroy the habit), and notification discipline (fewer, smarter pushes rather than more).

Folklore flag on this very case 🚩 Two things circulate about Duolingo that are NOT first-party. The "gamification took retention from 12% to 55%" stat comes from third-party blog posts, not from Duolingo - Mazal's writeup never says it, so leave it out of your decks. And "The Growth Handbook" often cited as Duolingo's playbook is actually Intercom's book. The documented case is impressive enough; it does not need the legends.

The machine above names the states; the chart below runs one real month of this course's dataset through it, every flow sized to its count.

One month's volumes through the machine: May 2026 April: where they were May: where they land 326 new in 817 stay current 450 out to at-risk 181 resurrected New in May Current base (Apr) 1,267 Dormant pool Active in May 1,324 At-risk / dormant 450 Counts from the course dataset's May 2026 month: 326 new + 817 retained + 181 resurrected = 1,324 actives. The 450 outflow is the balance: of April's 1,267 actives, only 817 stayed current.
🔍 Click to zoom - the same machine with May's real volumes: three inflows, one leak, 1,324 actives
Self-studyWhy a state machine beats one blended number4 min read▶

Imagine your product reports "monthly retention: 34%" and it dips to 32%. What do you do? You genuinely cannot answer - the blended number mixes brand-new users, loyal regulars, and returners, and each responds to different levers. The state machine splits the one number into named flows, and that buys you three things:

  • Diagnosis: a DAU dip decomposes into "the New inflow slowed" (a marketing problem) versus "the Current self-loop weakened" (a product problem) versus "comebacks dried up" (a winback problem). Three different owners, three different fixes.
  • Prioritization: once flows are separate you can rank them by impact, which is exactly the sensitivity analysis that crowned CURR. A blended metric cannot be ranked against itself.
  • Honest targets: each team owns one arrow. The winback team is not blamed for an onboarding leak, and nobody claims credit for a mix shift (s4 shows you how that trick works).

You do not need Duolingo's scale to use this. Amplitude's lifecycle framing - every active user is new, current, or resurrected; every loss is churn - is the same idea, available in most analytics tools your team already runs.

Part 2 · covers the soft-launch kill gate (Supercell, PocketGamer coverage - DOCUMENTED)

The go/no-go gate: games 9 min live

Duolingo shows retention as a lever to pull. The game industry shows it as a gate to pass. Supercell - the studio behind Clash of Clans - is famous for the second: it launches every new game quietly in a few test markets, watches the retention cohorts, and kills the game if they miss the bar. Not "iterates on". Kills.

The Supercell gate: soft launch, measure, then scale or kill 1 · Soft launch Test markets only, small UA budget 2 · Watch the cohorts D1 / D7 retention vs the bars set by live hits 3 · The bar beat it or not? Scale global + UA Kill fast, no rescue Killed at this gate: Spooky Pop (Mar 2015), Smash Land (Sept 2015 - despite players saying they liked it), and 30+ titles in total across the studio's history. Why so brutal: only 2-5% of players ever pay, and whales emerge from stayers.
🔍 Click to zoom - the gate is the strategy: retention decides whether acquisition money ever gets spent
LiveWhy a studio kills a game its players say they like6 min▶

Spooky Pop died in March 2015. Smash Land died in September 2015 - and the team openly noted that player feedback was positive. Across its history Supercell has killed 30+ titles this way, against a handful it has ever shipped globally. The pattern, as covered by PocketGamer and the studio's own communications, is always the same: soft launch in small test markets, judge the retention cohorts against the bars set by the live hits, and kill fast when the bar is missed.

  • Feedback is not behavior. Players said nice things about Smash Land and then did not come back. The cohort table saw what the reviews could not. When words and retention curves disagree, believe the curves.
  • Retention gates UA spend. Free-to-play economics are ruthless: only 2-5% of players ever pay, and the big spenders emerge over months from the pool of people who stayed. Pour acquisition money into a leaky game and you are renting traffic that leaves before it can pay - the leaky bucket from s1, at industrial scale.
  • The order is fixed: retention first, monetization second. A game that retains can always be monetized later. A game that monetizes early but leaks cannot be saved by more spend - the arithmetic only gets worse as you scale it.
  • The stakeholder lesson travels. You may never kill a product, but every launch decision has the same shape: is the cohort evidence above the bar we set BEFORE we looked? If your organization has no pre-agreed bar, the launch decision will be made by whoever tells the best story.
Real world

DOCUMENTED, and worth copying at any scale. An ecommerce equivalent: pilot the new subscription box in two cities, pre-commit the repeat-purchase bar it must clear by day 90, and let the cohort table - not the pilot team's enthusiasm - make the rollout call. The discipline is the transferable part, not the genre.

Part 3 · covers loyalty economics + winback (CIRP, Second Measure, McKinsey - labels in text)

Loyalty and winback: ecommerce 11 min live

Games gate on retention; ecommerce buys it with economics. The published numbers here are strong - but this is also where honesty labels matter most, because several of the famous figures are third-party estimates with a selection effect baked in. We will use them AND name the caveat, because naming the caveat is exactly the skill s4 trains.

The lever menu, mapped onto the curve each lever moves this gap, x customers, x spend = the money Onboarding slow the early cliff Habit loops keep the middle alive Loyalty economics raise the flat floor Winback bend the tail up months since first purchase Raise the asymptote (onboarding, habit, loyalty) or bend the tail (winback) - every lever is one of two moves.
🔍 Click to zoom - the menu is short: three levers raise the floor, one bends the tail
LiveThe lever menu, by mechanism8 min▶

Every retention lever ever proposed does one of two things to the curve: raise the asymptote (get more of each cohort into the durable core) or bend the tail up (bring the churned back). Sort the famous cases by mechanism and the menu writes itself:

  • Raise the asymptote - onboarding: get new customers to the core value fast, before the early cliff takes them. Duolingo's first-session design and streak seeding are this lever; so is a first-order-to-second-order nudge in ecommerce.
  • Raise the asymptote - habit loops: streaks, leagues, replenishment reminders - anything that gives the product a natural revisit rhythm. Duolingo's +17% learning-time lift from leaderboards lives here. DOCUMENTED.
  • Raise the asymptote - loyalty economics: make staying financially rational. Amazon Prime renews at roughly 93% after year one and about 98% after year two, and members spend about 2x what non-members do - per CIRP, which is a THIRD-PARTY survey, and partly a selection effect: the heaviest shoppers are exactly who joins Prime. Quote it, and quote the caveat. Sephora Beauty Insider: about 46M members driving roughly 80% of North America sales. Dollar Shave Club: about 50% six-month retention in Second Measure's transaction data - category-leading, and the durability story behind its $1B acquisition.
  • Bend the tail - winback: the only lever that acts on people already gone. McKinsey pegs subscription-box reality at about 40% of subscribers cancelling, with a median lifetime around 125 days - so a winback motion is not optional in that business. Realistic winback email reactivation runs 10-30%. Not a miracle, but applied to a large churned pool it is real money.
How to cost a lever in one line Money impact = delta retention x cohort size x average revenue per customer. A 3-point retention gain on a 10,000-customer monthly cohort is 300 extra retained customers - multiply by what a retained customer spends and you have the number that belongs in the business case. s4 makes this your standard demand for every readout.
Self-studyReading loyalty numbers without getting fooled4 min read▶

The Prime caveat deserves its own card, because it is the single most useful skeptical habit in loyalty analytics. "Members spend 2x" is true AND mostly not caused by the membership - people who already shop constantly are the ones who buy the membership. Three questions to ask of any loyalty stat:

  • Who measured it? First-party disclosure, third-party panel (CIRP, Second Measure), or a conference slide? Each is a different grade of evidence. CIRP is survey-based estimates; Second Measure is transaction data - stronger.
  • Selection or causation? Did the program make members loyal, or did loyal customers join the program? Usually both; the honest answer is "we cannot fully split them without an experiment".
  • Would we act differently either way? Often yes: a program that merely identifies your best customers is still worth having - but you should not project the 2x spend gap onto marginal joiners in the business case.
The Prime 2x gap: two stories fit the same bars What CIRP's survey shows non-members members ~2x Third-party survey, not Amazon's disclosure. Two mechanisms hide inside the gap Selection heavy shoppers join Causation perks drive more spend Observed 2x gap Only an experiment can split them - never project the full 2x onto marginal joiners in a business case. Quote the number AND the caveat: a program that merely identifies your best customers is still worth having.
🔍 Click to zoom - the 2x gap is selection and causation mixed; only an experiment can split them
Real world

The folklore scorecard - grade the classics before you quote them. Facebook's "7 friends in 10 days": DOCUMENTED (Chamath Palihapitiya, 2013) - but Mode's caveat applies: it was a correlational threshold chosen for communicability, not a proven causal switch. Slack's "customers who send 2,000 team messages: 93% stayed": DOCUMENTED (Stewart Butterfield, First Round Review). Dropbox's "get one file into one folder": SEMI-FOLKLORE - no first-party writeup exists, so it is a plausible story, not evidence. The habit to build: every case you repeat in a meeting should carry its label out loud.

Homework

Try it yourself - this week ◐ 30 min total

Sources covered

What this session draws on

Every case above carries its label in the text. The underlying sources, and how far this session takes each one:

✓Jorge Mazal - "How Duolingo reignited user growth" (Lenny's Newsletter)Part 1 · the state machine, CURR, and the four-year result, end to end
✓PocketGamer coverage of Supercell's soft-launch killsPart 2 · Spooky Pop, Smash Land, the 30+ pattern
◐CIRP - Amazon Prime renewal and member-spend surveysPart 3 · quoted with the third-party + selection-effect caveat
◐Second Measure - Dollar Shave Club transaction dataPart 3 · the ~50% six-month figure; full chart treatment in a2/a3
◐McKinsey - subscription ecommerce churn studyPart 3 · the ~40% cancel / ~125-day median behind the winback case
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · Duolingo's growth team had a dozen candidate metrics. Why did CURR win the north-star contest?

The choice was arithmetic, not taste: nudge each transition rate in the state machine, watch DAU. CURR crushed the field because Current is the biggest bucket and a user kept today is exposed to being kept again tomorrow - the gain compounds. Four years later: CURR +21%, DAU ~4.5x, flat acquisition spend.

2 · Supercell killed Smash Land even though player feedback was positive. What does the kill pattern teach about the order of retention and monetization?

Retention gates everything downstream. The paying few surface over months from the pool of people who stayed, so scaling UA into a game below the retention bar is renting traffic that leaves before it pays. That is why the gate kills fast - and why feedback lost to the cohort table.

3 · Which of these famous retention stories is semi-folklore rather than documented?

The Dropbox story has no first-party writeup - plausible, but a legend. Slack's number is documented (Stewart Butterfield, First Round Review). Facebook's is documented too (Chamath, 2013), though with Mode's caveat: a correlational threshold chosen for communicability, not a proven causal switch.

Stakeholder session 3 cheat sheet · pin this

The 7-bucket machineNew + Current + Reactivated + Resurrected = DAU. At-risk WAU, At-risk MAU, Dormant are the leak. Model flows, not one blended number.
CURR, the north starCurrent-user retention rate. Sensitivity analysis: ~5x the DAU impact of the next metric. Result: CURR +21%, DAU ~4.5x in ~4 years, flat spend.
Duolingo's leversLeaderboards/leagues (learning time +17%), streaks + streak freeze, notification discipline. All documented via Mazal in Lenny's Newsletter.
Folklore filter"12% to 55% from gamification" is third-party, not Duolingo's. "The Growth Handbook" is Intercom's book. Dropbox "one file" has no first-party source.
The Supercell gateSoft launch in test markets, judge D1/D7 vs live-hit bars, kill fast. Spooky Pop (Mar 2015), Smash Land (Sept 2015, liked but leaky), 30+ total.
Retention before monetizationOnly 2-5% of players ever pay; whales emerge from stayers. Retention decides whether acquisition money gets spent at all.
Loyalty economicsPrime ~93% then ~98% renewal, ~2x spend (CIRP - third-party, selection effect). Sephora ~46M members, ~80% of NA sales. DSC ~50% at 6 months.
Winback + the money math~40% of subscription-box subs cancel, median ~125 days (McKinsey); winback emails reactivate 10-30%. Cost a lever: delta x cohort size x ARPU.