Where this session sits
Four stakeholder sessions, no code, one goal: make you the person in the room who asks the right retention questions. s1 made the money case, s2 taught you to read the cohort chart. Today is the lever menu - what teams actually pull, with the published evidence for each pull. s4 closes the loop with the questions to ask your data analyst. One standard holds throughout - the same one worth enforcing in your own meetings: a case only counts if you can say who published it and how it was measured. The labels DOCUMENTED and SEMI-FOLKLORE appear next to every story, and learning to demand those labels is half of today's value.
Duolingo: the best-published retention case 17 min live
Most retention stories are one sentence and a suspiciously round number. Duolingo's is different: Jorge Mazal, the former Chief Product Officer, wrote the whole playbook up in Lenny's Newsletter - the model, the metric contest, the levers, the results. It starts with a decision that sounds boring and turns out to be everything: stop treating users as one blended pool, and model them as a state machine. Seven buckets, and every user is in exactly one.
LiveWalk the machine: seven buckets, one loop that matters7 min▶
Every Duolingo user sits in exactly one bucket on any given day. Four buckets are active today and sum to DAU: New (first day ever), Current (active today and recently active), Reactivated (back after a short absence), and Resurrected (back after a long absence, 30+ days away). Three states are the leak: At-risk WAU (slipping this week), At-risk MAU (gone for weeks), and Dormant (gone a month or more). Users flow along the arrows every day.
- Why this beats "our retention is 34%": a single blended number cannot tell you where the leak is. The machine can - it shows exactly which arrow is too thick and which comeback arrow is too thin.
- The metric contest: the growth team ran a sensitivity analysis - nudge each transition rate, watch what happens to DAU. One rate crushed the field: CURR, the current-user retention rate (the self-loop on Current). Moving it had roughly 5x the DAU impact of the next-best metric. It became the north star not by opinion but by arithmetic.
- Why CURR wins mechanically: Current is the biggest bucket, and a user kept Current today is exposed to being kept again tomorrow. Small gains to the self-loop compound daily. Reactivation flows, by contrast, act on smaller populations and only once per lapse.
- The result: about four years of focused work later, CURR was up 21% and DAU roughly 4.5x - with acquisition spend flat. Nobody bought that growth. They kept it.
The levers behind those four years, per Mazal: leaderboards and leagues (a documented +17% lift in learning time), streaks with the streak freeze (the habit loop, plus forgiveness so one missed day does not destroy the habit), and notification discipline (fewer, smarter pushes rather than more).
The machine above names the states; the chart below runs one real month of this course's dataset through it, every flow sized to its count.
Self-studyWhy a state machine beats one blended number4 min read▶
Imagine your product reports "monthly retention: 34%" and it dips to 32%. What do you do? You genuinely cannot answer - the blended number mixes brand-new users, loyal regulars, and returners, and each responds to different levers. The state machine splits the one number into named flows, and that buys you three things:
- Diagnosis: a DAU dip decomposes into "the New inflow slowed" (a marketing problem) versus "the Current self-loop weakened" (a product problem) versus "comebacks dried up" (a winback problem). Three different owners, three different fixes.
- Prioritization: once flows are separate you can rank them by impact, which is exactly the sensitivity analysis that crowned CURR. A blended metric cannot be ranked against itself.
- Honest targets: each team owns one arrow. The winback team is not blamed for an onboarding leak, and nobody claims credit for a mix shift (s4 shows you how that trick works).
You do not need Duolingo's scale to use this. Amplitude's lifecycle framing - every active user is new, current, or resurrected; every loss is churn - is the same idea, available in most analytics tools your team already runs.
The go/no-go gate: games 9 min live
Duolingo shows retention as a lever to pull. The game industry shows it as a gate to pass. Supercell - the studio behind Clash of Clans - is famous for the second: it launches every new game quietly in a few test markets, watches the retention cohorts, and kills the game if they miss the bar. Not "iterates on". Kills.
LiveWhy a studio kills a game its players say they like6 min▶
Spooky Pop died in March 2015. Smash Land died in September 2015 - and the team openly noted that player feedback was positive. Across its history Supercell has killed 30+ titles this way, against a handful it has ever shipped globally. The pattern, as covered by PocketGamer and the studio's own communications, is always the same: soft launch in small test markets, judge the retention cohorts against the bars set by the live hits, and kill fast when the bar is missed.
- Feedback is not behavior. Players said nice things about Smash Land and then did not come back. The cohort table saw what the reviews could not. When words and retention curves disagree, believe the curves.
- Retention gates UA spend. Free-to-play economics are ruthless: only 2-5% of players ever pay, and the big spenders emerge over months from the pool of people who stayed. Pour acquisition money into a leaky game and you are renting traffic that leaves before it can pay - the leaky bucket from s1, at industrial scale.
- The order is fixed: retention first, monetization second. A game that retains can always be monetized later. A game that monetizes early but leaks cannot be saved by more spend - the arithmetic only gets worse as you scale it.
- The stakeholder lesson travels. You may never kill a product, but every launch decision has the same shape: is the cohort evidence above the bar we set BEFORE we looked? If your organization has no pre-agreed bar, the launch decision will be made by whoever tells the best story.
DOCUMENTED, and worth copying at any scale. An ecommerce equivalent: pilot the new subscription box in two cities, pre-commit the repeat-purchase bar it must clear by day 90, and let the cohort table - not the pilot team's enthusiasm - make the rollout call. The discipline is the transferable part, not the genre.
Loyalty and winback: ecommerce 11 min live
Games gate on retention; ecommerce buys it with economics. The published numbers here are strong - but this is also where honesty labels matter most, because several of the famous figures are third-party estimates with a selection effect baked in. We will use them AND name the caveat, because naming the caveat is exactly the skill s4 trains.
LiveThe lever menu, by mechanism8 min▶
Every retention lever ever proposed does one of two things to the curve: raise the asymptote (get more of each cohort into the durable core) or bend the tail up (bring the churned back). Sort the famous cases by mechanism and the menu writes itself:
- Raise the asymptote - onboarding: get new customers to the core value fast, before the early cliff takes them. Duolingo's first-session design and streak seeding are this lever; so is a first-order-to-second-order nudge in ecommerce.
- Raise the asymptote - habit loops: streaks, leagues, replenishment reminders - anything that gives the product a natural revisit rhythm. Duolingo's +17% learning-time lift from leaderboards lives here. DOCUMENTED.
- Raise the asymptote - loyalty economics: make staying financially rational. Amazon Prime renews at roughly 93% after year one and about 98% after year two, and members spend about 2x what non-members do - per CIRP, which is a THIRD-PARTY survey, and partly a selection effect: the heaviest shoppers are exactly who joins Prime. Quote it, and quote the caveat. Sephora Beauty Insider: about 46M members driving roughly 80% of North America sales. Dollar Shave Club: about 50% six-month retention in Second Measure's transaction data - category-leading, and the durability story behind its $1B acquisition.
- Bend the tail - winback: the only lever that acts on people already gone. McKinsey pegs subscription-box reality at about 40% of subscribers cancelling, with a median lifetime around 125 days - so a winback motion is not optional in that business. Realistic winback email reactivation runs 10-30%. Not a miracle, but applied to a large churned pool it is real money.
Self-studyReading loyalty numbers without getting fooled4 min read▶
The Prime caveat deserves its own card, because it is the single most useful skeptical habit in loyalty analytics. "Members spend 2x" is true AND mostly not caused by the membership - people who already shop constantly are the ones who buy the membership. Three questions to ask of any loyalty stat:
- Who measured it? First-party disclosure, third-party panel (CIRP, Second Measure), or a conference slide? Each is a different grade of evidence. CIRP is survey-based estimates; Second Measure is transaction data - stronger.
- Selection or causation? Did the program make members loyal, or did loyal customers join the program? Usually both; the honest answer is "we cannot fully split them without an experiment".
- Would we act differently either way? Often yes: a program that merely identifies your best customers is still worth having - but you should not project the 2x spend gap onto marginal joiners in the business case.
The folklore scorecard - grade the classics before you quote them. Facebook's "7 friends in 10 days": DOCUMENTED (Chamath Palihapitiya, 2013) - but Mode's caveat applies: it was a correlational threshold chosen for communicability, not a proven causal switch. Slack's "customers who send 2,000 team messages: 93% stayed": DOCUMENTED (Stewart Butterfield, First Round Review). Dropbox's "get one file into one folder": SEMI-FOLKLORE - no first-party writeup exists, so it is a plausible story, not evidence. The habit to build: every case you repeat in a meeting should carry its label out loud.
Try it yourself - this week ◐ 30 min total
- Pick ONE lever from the menu (onboarding, habit loop, loyalty economics, or winback) that fits your product's weakest part of the curve - use s2's chart-reading to find where the leak is.
- Estimate its money impact with the one-liner: delta retention x cohort size x average revenue per customer. Use a modest delta - a 2-3 point gain is already a strong result for most levers.
- Write down, before anyone builds anything, what evidence would convince you the lever worked: which cohorts, compared to which baseline, at what age. (If you cannot name it now, you will accept a victory slide later.)
- Grade one retention case you have heard quoted at your company: DOCUMENTED or SEMI-FOLKLORE? Who published it, and how was it measured?
What this session draws on
Every case above carries its label in the text. The underlying sources, and how far this session takes each one:
Three questions before you go 🎯 ◐ 90 seconds
1 · Duolingo's growth team had a dozen candidate metrics. Why did CURR win the north-star contest?
The choice was arithmetic, not taste: nudge each transition rate in the state machine, watch DAU. CURR crushed the field because Current is the biggest bucket and a user kept today is exposed to being kept again tomorrow - the gain compounds. Four years later: CURR +21%, DAU ~4.5x, flat acquisition spend.
2 · Supercell killed Smash Land even though player feedback was positive. What does the kill pattern teach about the order of retention and monetization?
Retention gates everything downstream. The paying few surface over months from the pool of people who stayed, so scaling UA into a game below the retention bar is renting traffic that leaves before it pays. That is why the gate kills fast - and why feedback lost to the cohort table.
3 · Which of these famous retention stories is semi-folklore rather than documented?
The Dropbox story has no first-party writeup - plausible, but a legend. Slack's number is documented (Stewart Butterfield, First Round Review). Facebook's is documented too (Chamath, 2013), though with Mode's caveat: a correlational threshold chosen for communicability, not a proven causal switch.