Where we start
Session 1 left you with a one-page tracking plan: five moments your product writes down. One of those moments, or a small count of one of them inside a short window, is going to become your activation metric, the early behaviour you watch because people who do it tend to come back and people who do not tend to vanish. The trap is that the two most famous activation metrics in the industry are quoted as laws when one is a second-hand transcript and the other is a founder's rule of thumb with a built-in survivorship problem. The second trap is subtler: even a real, measured lift tells you who is likely to stay, not what to do about it. We take both traps apart, then you write your own candidates in a form that can be checked.
★ What you walk out with tonight. An activation hypothesis sheet: three candidate definitions from your own tracking plan, each written as "at least k X within N days", each with your predicted share activated, your predicted lift, and the experiment that would show whether it is a lever or only a signal. Plus the two sentences that end most magic-number arguments: "what tier is that claim?" and "predicts, or causes?"
Activation is a definition you choose
5 min liveTwo words get used as if they were one. The aha moment is something a person feels: the instant the product made sense and became worth coming back to. Activation is something a team counts: an early, measurable behaviour, chosen because the people who do it come back at a much higher rate than the people who do not. The metric is a proxy for the feeling. Nobody can log a feeling, so we log the nearest behaviour and check, weeks later, whether it earned its place.
LiveActivation is a definition you choose3 min▶
An activation metric has three parts and every one of them is a decision. A behaviour from the tracking plan: a push, a second day of activity, a second person joining. A count: at least one, at least three. A window from the birth of the thing: the first day, the first seven days. Write it as "at least k X within N days" and it stops being a slogan and becomes something an analyst can run.
The figure is that sentence run for real. The cohort is 8,348 public repositories created in one hour on 4 March 2024, the first one of each non-bot account, because GH Archive records a repository's birth and never records an account's. The behaviour is "seen on at least two separate days in the first seven". 1,186 repositories pass, 14.2 percent. Weeks 5 and 6 later, 13.0 percent of them had somebody touch them again, against 2.4 percent of the other 7,162. That gap, 5.44 times, is what earns a definition its place. A definition with no gap is just a thing people do.
Notice the base rate first. Only 325 of 8,348 came back at all, 3.9 percent. Most new repositories are created and abandoned, the way most trial workspaces are. Any activation story starts from that floor, and a definition is judged by how far above the floor it lifts the people it selects.
Self-studyAha is felt, activation is counted2 min▶
Keep the two words apart in meetings. When a designer says "the aha moment is when they see their first report", that is a hypothesis about a feeling, and it is useful: it tells you where to look. When an analyst says "activation is Report Viewed at least once within 3 days", that is a measurable proxy, and the only test of it is the later return gap. The feeling can be real and the proxy can still be wrong: perhaps the people who view a report early were already committed, and the report did nothing. The proxy can also be right for the wrong reason, which is Part 3.
A good working rule: the product team owns the aha hypothesis, the analytics team owns the activation definition, and the two meet once a quarter to check that the definition still separates stayers from leavers. Definitions drift as the product changes. Slack's founder said theirs was based on experience of which companies stuck and which did not; that is a definition earned from data and it would need re-earning as the product grew.
Activated, by whose definition? A common pattern, no numbers attached: a growth team reports activation at 60 percent and a product team reports it at 20 percent for the same month. Neither is lying. One counts "completed onboarding" (a behaviour the product forces) and the other counts "returned on a second day" (a behaviour the user chooses). Forced behaviours are nearly universal and predict almost nothing; chosen behaviours are rarer and predict more. The fix is the sentence form: write both as "at least k X within N days", put the later return rate next to each, and let the gap decide which one is the activation metric.
Two magic numbers, traced to their source
6 min liveBoth stories are told the same way: the team found a threshold in the data, made it the whole company's goal, and growth followed. Trace each to the earliest thing you can actually read and they come apart differently. One survives only as a transcript on somebody else's website. The other is a founder speaking on the record, with a number that quietly proves less than it seems to.
LiveRally cry versus evidence3 min▶
The Facebook line is reported. Chamath Palihapitiya gave a talk in October 2012, "How we put Facebook on the path to 1 billion users", and a transcript excerpt quoted by Startup Archive has him saying "The single biggest thing we realized was to get any individual to 7 friends in 10 days". The video was not watched for this build, so the words are second-hand, the tier is reported, and the honest way to teach it is as a claim about what the team believed and organised around, not as a threshold found in data. Benn Stancil at Mode wrote in January 2015 that it reads as a memorable rally cry rather than a threshold, and Mixpanel's blog reports Andrew Chen saying it could as easily have been 10 friends in 12 days or 5 in 1. Nobody outside the company has seen the analysis, so nobody outside the company can say the number was special.
The Slack line is read at source. Stewart Butterfield told First Round Review in February 2015: "we decided that any team that has exchanged 2,000 messages in its history has tried Slack - really tried it", and "after 2,000 messages, 93% of those customers are still using Slack today". Notice his own framing: "we decided". It is a definition chosen from experience of which companies stuck, which is exactly what Part 1 said an activation metric is. That is a stronger claim than Facebook's, because you can read it. It still proves less than the 93 percent suggests.
LiveThe survivorship problem2 min▶
A team cannot send 2,000 messages in an afternoon. To reach that count a team has already used Slack for a while, which means the group "teams with 2,000 messages" is selected from teams that had already stayed. Of course most of them are still there: the definition filtered out the early leavers before it counted anyone. The 93 percent tells you that teams who stayed long enough to reach 2,000 tended to keep staying. It does not tell you that the 2,000th message did anything, and it does not tell you what share of all new teams ever got there.
The same shape is in the figure from Part 1. "Seen on 2+ separate days in the first 7" is, very nearly, "came back once already". A repository that came back on day 3 is more likely to come back in week 5 for the plain reason that it has shown it comes back. The lift is real and useful for prediction; it is partly the outcome measuring itself early. Every activation metric built from a repeat behaviour carries this, and the page should say so rather than hide it.
Self-studyReading a claim's tier before you repeat it2 min▶
Three questions to ask of any famous number before it goes on a slide. Can I read it? A quote on the speaker's own page or in a published interview is read; a transcript on a third-party site is reported; a number in a conference deck photographed from the audience is barely that. Was the analysis published? For both Facebook and Slack the answer is no; only the conclusion travelled. Would the speaker have said it if the number had been different? Butterfield chose 2,000 as a round threshold from experience; Chen's point, as reported, is that 7 and 10 were one defensible pair among several. A number that could have been several other numbers is a rally cry, and rally cries are fine as long as nobody mistakes them for measurement.
Copying 7 friends into a product that has no friends. A common pattern: a team building a B2B analytics tool decides its activation metric is "7 dashboards viewed in 10 days" because the shape of the Facebook line felt right. Nobody checks whether users who view 7 dashboards return at a higher rate than users who view 3, and nobody checks what share of new accounts ever reach 7. Six months later the metric sits at a single-digit percentage, the team is discouraged, and the honest analyst finally runs the comparison and finds that "invited a second teammate within a week" separates stayers far more cleanly. The borrowed number cost two quarters. The sentence form, "at least k X within N days", with the return gap next to it, would have cost an afternoon.
A lift predicts; it does not pull
4 min livePut eight candidate definitions on the same 8,348 repositories and the lifts range from 0.88x to 5.45x. Every one of them is a correlation: repositories that did X early came back at some multiple of the rate of repositories that did not. None of them says that making a repository do X would change anything. The strongest predictors are usually early versions of the outcome itself, or things the owner cannot control at all, which is exactly why they predict so well and lever so badly.
LivePredictor versus lever2 min▶
Read the bench the way it was measured. "1+ push on the first day" calls 33.3 percent of repositories activated and lifts return 1.74x. "3+ events of any kind in the first 7 days" calls 46.0 percent activated at 2.93x and catches 71.4 percent of everything that eventually came back, because it is broad. "Seen on 2+ separate days in the first 7" calls 14.2 percent activated at 5.44x. Same repositories, same return window; the repeat-day rule separates far better per head, the volume rule catches more of the returners by naming almost half the cohort. Which one you want depends on whether you are trying to predict or to target, and that is a decision, not a discovery.
Then the trap. "10+ events in the birth hour" sounds like the most engaged repositories of all. Their lift is 0.88x: 4 of 117 came back, against 3.9 percent for the rest. The group is small enough that the honest claim is "no signal", never "harm"; a seeded shuffle of the same 117 rows, keeping the group size and breaking the link, reads 1.32x, which is how far a group that size wanders by chance. Setup activity is not engagement. Ten events in the first hour is somebody unpacking boxes.
Every one of these is a correlation. Repositories that did X came back more often. Nothing here says that causing a repository to do X would change its odds. The gap between "predicts" and "causes" is closed by one thing only: an experiment, where a random half of new users is nudged toward the behaviour and the other half is not. The design of that experiment is its own subject: learn-experimentation-with-phoebe.
Self-studyThe strongest predictors are early versions of the outcome2 min▶
Look at what tops the table. "Seen on 2+ days" at 5.44x is coming back, measured early. "2+ distinct people in the first 7 days" at 5.45x, 288 repositories, is a project that already has a team, and a team is the thing that keeps a project alive. "1+ star in the first 7 days" at 3.35x, 194 repositories, is something the owner does not do at all: a stranger did it. These predict well because they are downstream of the thing you care about, or outside your control entirely. Neither kind is a button you can press.
The behaviours you can actually cause, a push on day one, some events in week one, sit lower, at 1.74x and 2.93x. This is the usual shape and it is worth expecting: the more a behaviour is something the product can force, the less it tends to separate stayers from leavers, because everyone gets forced through it. The activation metric you can defend is often not the one you can most easily move, and knowing that before the roadmap is written saves a quarter.
Self-studySmall groups wander1 min▶
Three of the eight definitions select fewer than 300 repositories: 2+ people (288), 1+ issue or PR opened (214), 1+ star (194), and the birth-hour burst (117). Their lifts should be read with a wide margin. The shuffle results say how wide: with the link deliberately broken, those small groups still read 1.07x, 1.33x, 1.33x and 1.32x, while the big groups sit near 1.0. A lift of 1.3x on 200 rows is indistinguishable from nothing. Ask for the group size next to every lift, every time.
The roadmap that chased its own tail. A common pattern, no numbers attached: an analyst finds that customers who use the export feature in week one renew at several times the rate of those who do not. The next quarter's roadmap becomes "get everyone to export in week one": a banner, an onboarding step, an email. Export-in-week-one rises. Renewal does not move, because the people who exported early were the ones who already had a real job for the product; the export was a symptom of commitment, not a source of it. One holdout experiment before the roadmap, half the new accounts seeing the banner and half not, would have shown the flat line in a month instead of a year.
Your activation hypothesis sheet
★ 22 min · everyone buildsNo tools and no data access. Your session 1 tracking plan, a blank sheet, and three rows. Each row is one candidate activation definition written so that an analyst can run it and an experiment can test it. The example row below is constructed for a fictional workspace product; your numbers will differ, and the two "predicted" columns are guesses you write down now so that the real numbers can embarrass you later. That embarrassment is the point.
Pick three candidates from your session 1 tracking plan. One should be a chosen behaviour (the user did it without being forced), one should involve a second person or a second day, and one may be the obvious volume rule your team already uses, so that it can be compared honestly. Do not pick "completed onboarding"; forced steps predict almost nothing because everyone passes through them.
Write each one as "at least k X within N days". X is an event name from the plan, k is a count, N is a window from the birth of the account, workspace or project. If a candidate cannot be written this way it is an aha hypothesis, not an activation definition, and it goes in the "why" column of a different row.
Write why it might predict return, in one line, and be suspicious of your own answer. If the line says "because people who do this have already decided to stay", you have written a survivorship problem, like 2,000 messages. That does not disqualify the row; it means the row will predict well and lever badly, and you should say so now.
Guess the share activated and the lift, and write the guesses down before anyone runs anything. A guess on paper is the only thing that lets the real number teach you. The shapes to expect from the bench: broad rules call 30 to 50 percent activated at lifts under 3x; narrow repeat or second-person rules call under 15 percent at lifts around 5x; a busy first hour can read under 1x. Your product will differ, which is why the column exists.
Write the experiment that would test it as a lever. Always the same shape: a random half is nudged toward the behaviour, the other half is not, and the later return rates of the two halves are compared. If nobody can name a nudge, the candidate is a pure predictor, which is still useful for forecasting and for targeting help, and you should label it as one. Hand the sheet to your analyst; session 4 of the analyst track runs exactly this sweep on real rows.
Keep the guesses. When the real numbers arrive, the gap between your guess and the measurement is the most useful thing on the sheet. A team that guessed 3x and measured 1.2x has learned that its favourite behaviour is nearly universal and nearly meaningless. A team that guessed 20 percent activated and measured 2 percent has learned that its aha moment is real but rare, which is a different roadmap entirely.
Before session 3
30 min- Find your product's current activation definition, if it has one, and rewrite it as "at least k X within N days". If it cannot be written that way, or nobody can say who chose it, you have found the first thing to fix.
- Ask for the return gap. For that definition, ask the analyst for two numbers: the share of new accounts that met it, and the later return rate of those who did against those who did not. If the answer is "we have never checked", say so in the next product review.
- Trace one borrowed number. Pick a magic number your team quotes, from any company, and find the earliest source you can read. Write down its tier: read at source, reported, or nowhere. Bring the tier, not the number.
- Bring the hypothesis sheet. Session 3 turns from the first week to the long run: how often people come back, why one ratio hides most of it, and which of your "active" users are actually automation.
Three questions
Clear all three and leader session 2 gets stamped in your passport.
1 · Slack's founder said that after 2,000 messages, 93 percent of those customers were still using Slack. Why does that number prove less than it seems to?
Survivorship. The definition filters out early leavers before anyone is counted, so of course the survivors mostly survive. The 93 percent says teams who stayed long enough to reach 2,000 tended to keep staying; it says nothing about what the 2,000th message did.
2 · What evidence tier does Facebook's "7 friends in 10 days" have, and how should it be taught?
The words reach us through a transcript excerpt quoted by Startup Archive; the video was not watched and the analysis was never published. Stancil calls it a memorable rally cry, and Chen is reported as saying 10 in 12 or 5 in 1 would have done as well. Repeat the tier with the number.
3 · "Seen on 2+ separate days in the first 7" has a lift of 5.44x on 8,348 real new repositories. What does that prove, and what does it not?
A lift is a correlation, and this one is partly the outcome measuring itself early: coming back on day 3 is a small version of coming back in week 5. Useful for forecasting and targeting; silent about cause. Only a random-half experiment turns a predictor into a lever.
What this session teaches, and where it came from
Every fact on this page, and its verification tier, is recorded in the course's source map.
Leader session 2 cheat sheet · pin this
Activation is the first week. Session 3 is everything after it: how many of your "active" users are active on one day a month, why DAU over MAU hides that, and how the power-user curve from Li Jin and Andrew Chen shows the shape in one picture. On a 5 percent sample of GitHub accounts across 28 days, three definitions of "active" give three different monthly totals, 79,555 against 48,946 against 8,993, and the right tail turns out to hold a surprising amount of automation. Bring your hypothesis sheet.