Where we start
Sessions 4 and 5 asked which accounts stay and how often they show up. Adoption asks a narrower question about one feature at a time: of everyone who could have used it, who did, how much, how soon, and did using it go with staying. The unit is the same new public repository as the bench, born in the hour 2024-03-04 15:00 UTC, one per non-bot account, 8,348 in all, 325 of them seen again in days 28 to 41. A "feature" is a GitHub capability read straight from the event type, and every count is "seen in the sampled hours", one hour in two.
★ What you walk out with tonight. A per-feature adoption table with four columns (breadth, depth, time to first use, came-back rate of users against the rest), a breadth gradient from 0 to 4+ features, and the habit of asking, before any adoption number goes in a deck, how much of it the product did by itself.
Who tried each feature in week one
5 min liveBreadth is the first column of any adoption table: the share of the cohort that used a feature at least once inside a fixed window. Here the window is the first 7 days (sampled hours 0 to 166) and the denominator is all 8,348 repositories, whether or not anyone ever looked at the feature. Six features, six very different numbers, and the ranking already hides a trap.
LiveBreadth is a share of the whole cohort3 min▶
The script tests each repository's week-one rows against one rule per feature: push is a PushEvent, branch is a CreateEvent whose detail is "branch", issue and PR are IssuesEvent and PullRequestEvent with action "opened", release is a ReleaseEvent, wiki is a GollumEvent. Any match in hours 0 to 166 makes the repository a user. Divide by 8,348 and you have breadth.
Read the ranking top to bottom: branch 68.8 percent, push 39.1, PR 2.2, issue 0.5, release 0.4, wiki 0.1. Four of the six features reach fewer than 300 repositories, so every rate built on them later is a small-group rate and moves a lot when one repository changes. That is not a reason to drop them; it is a reason to print the group size next to every rate.
Self-studyThe denominator is a decision2 min▶
A launch review usually wants "adoption among active users", because the number is bigger. Pick the denominator before you look: the whole cohort answers "how far did the feature reach", the active base answers "how many of the people who were around used it", and the two are not interchangeable across time, because the active base moves too. Here the denominator is the whole cohort, and 2,171 repositories (26.0 percent) used none of the six features in week one at all.
The window is a decision too. Seven days favours features used at setup; a feature people reach for in month two will look unadopted in any week-one table. Part 2 measures how much of each feature's adoption already falls outside day one.
The feature every account "adopted" on day one. A common pattern, not a measured case: a team collaboration product creates a default channel, project or folder for every new workspace, and the launch dashboard counts "workspaces using channels". Adoption reads near total from the first week, the feature team is congratulated, and nobody has asked how many of those workspaces ever created a second one. Whenever the product performs an action on the user's behalf, count the user's own actions separately.
How much, and how soon
5 min liveBreadth says a repository used a feature once. Two more columns turn it into a feature funnel per repository: born, used once, used again (depth), and how many hours passed between birth and the first use (time to adopt). These are feature funnels inside one product, not purchase funnels; checkout paths and page-to-page journeys belong to user journey analytics.
LiveDepth: the used-again step of the funnel3 min▶
Depth is counted only among users: how many sampled uses each user repository had in week one, reported as a median and a mean. The funnel reads born (8,348), used once (the breadth count), used again (two or more uses). The table gives only medians and means, so read the used-again step from what a median can tell you and compute nothing it cannot:
- Push: median 2, mean 3.6 among 3,268 users. A median of 2 means at least half of the pushing repositories pushed again. It is the only feature with that property.
- Every other feature: median 1. At least half of each feature's users used it exactly once in the week, branch (mean 1.4) most of all.
- Mean above median (issue 2.5, PR 3.3, wiki 3.9) says a few users went deep while most stopped at one. With 11 wiki repositories, that mean is a handful of repositories and nothing more.
Depth counts events in sampled hours, not separate days. Ten pushes in one afternoon and one push on each of ten days both read as depth 10 here; the bench's "seen on 2+ days" rule (5.44x) against "3+ events" (2.93x) is the reason separate days are the better measure of returning.
LiveTime to first use, and who arrives late2 min▶
Time to adopt is hours from the birth hour to the first sampled use. Push and branch have a median of 0: they happen in the hour the repository is born, because that is how a repository gets its first code. Wiki (2 h), release (3 h) and PR (6 h) follow, and issues come last at 15 h.
The right-hand column matters as much as the median. 17 of the 44 issue-opening repositories first opened one after day one, and 64 of 183 for pull requests. A day-one adoption metric would miss those repositories entirely. The grid samples one hour in two, so every time here is precise to about two hours; that is fine for comparing features and too coarse for anything finer.
Self-studyWhat a funnel per repository adds over a funnel per feature2 min▶
A funnel per feature is one row of the table: breadth, then depth, then time. A funnel per repository asks, for each repository, which features it reached and in what order. That second view is the one that produces the breadth gradient in Part 3, because it lets you count features per repository. Keep both: the first tells a feature team how their feature did, the second tells the product team what kinds of repository exist.
Slack's 2,000 messages as a depth threshold. Stewart Butterfield told First Round Review in 2015 that a team which had exchanged 2,000 messages had "really tried" Slack, based on which companies stuck with them. That is a depth cut-off on the core feature, and it carries a survivorship problem: a team only reaches 2,000 messages by staying long enough to send them, so depth and staying are partly the same fact counted twice. Measure depth inside a fixed early window, as the table here does, and the problem shrinks.
Adopted is not valued
5 min liveThe fourth column is the one that earns the table its place in a review: of the repositories that used a feature in week one, how many were seen again in days 28 to 41, against the repositories that did not use it. Then count features per repository and the same question produces a gradient. Both are correlations. Neither says a feature makes a repository come back.
LiveUsers against the rest, feature by feature3 min▶
Every repository in the cohort is either a user of a feature in week one or not, and each side has a came-back rate. The whole cohort's rate is 3.9 percent (325 of 8,348).
| Feature | Users | Came back, users | Came back, the rest |
|---|---|---|---|
| branch created | 5,744 · 68.8% | 4.3% | 3.0% |
| push | 3,268 · 39.1% | 6.4% | 2.3% |
| PR opened | 183 · 2.2% · small | 12.0% | 3.7% |
| issue opened | 44 · 0.5% · small | 29.5% | 3.8% |
| release | 36 · 0.4% · small | 11.1% | 3.9% |
| wiki edit | 11 · 0.1% · small | 9.1% | 3.9% |
The ordering inverts. The widest feature, branch, separates least: 4.3 against 3.0. The rarest deliberate features separate most, with issues at 29.5 against 3.8 on only 44 repositories. Wide adoption and a strong signal are different properties, and a table that only shows breadth rewards the wrong feature.
LiveWhy branch is the widest and the weakest2 min▶
Most of the "branch created" events in week one are the default branch, created as a side effect of the first push into an empty repository. Nobody decided to use branching; the product did it for them. That is why its time to adopt is 0 hours, its depth median is 1, and its users come back barely more often than the rest. "Adopted" was counted from a setup artefact.
The fix is in the definition, not the dashboard: count the user's own action. For branches that means a branch other than the first one, or a branch created after day one. The same rule applies to any feature your product switches on by default.
Self-studyA predictor, not a lever2 min▶
Repositories that open issues or ship releases are, for the most part, projects someone is actively working on, and projects being worked on come back. The gradient is what that looks like counted by features. It does not say that prompting a new repository to open an issue would lift its return rate. Proving a lever needs a randomised test; that design belongs to experimentation, and full return curves by cohort to customer retention.
Three honest limits on the numbers: public repositories only; bots without the [bot] suffix stay in; every count is from one hour in two. The four features under 300 users and the 3-feature and 4+ groups are small groups, and their rates carry that label wherever they are quoted.
GitHub's own default branch, measured. On the real cohort, the feature a naive launch dashboard would celebrate, reaching 68.8 percent of new repositories in a week, is mostly GitHub creating the default branch on first push. The measured version of "adopted is not valued" needed no survey: the came-back rate of its users is 4.3 percent against 3.0 for the rest. GitHub's documentation for its Copilot metrics gives the reading rule in one line: "Look for patterns across these signals rather than focusing on any single number".
Six features, one cohort, four numbers each
★ 22 min · everyone buildsStandard-library Python on the same cache the bench uses. The numbers printed as comments below are what this code prints on that cache; the cohort is deterministic, so yours should match exactly. Building the cache is the slow part: it streams 252 GH Archive hourly files starting at https://data.gharchive.org/2024-03-04-15.json.gz, about 37 GB, and keeps only the cohort's 44,573 events. If you built it for sessions 4 or 8, skip step 1.
Build the cohort cache. Two scripts from materials/, run once. The first streams the hours and keeps only the cohort's events; adoption-numbers.py then runs steps 2 to 5 in one go, and the steps below take it apart.
Write the feature tests. One rule per feature, on the event type and the detail field. This dictionary is the adoption equivalent of the tracking plan in session 1.
Load the events and pick the cohort. The first new repository of each non-bot account, bot rows dropped, and "came back" as any row in offsets 672 to 1006 (days 28 to 41).
Four numbers per feature. Uses in offsets 0 to 166, then breadth, depth among users, hours to first use, how many first uses came after day one, and the came-back rate of users against the rest. Each printed line is split over two comment lines below so it fits.
The breadth gradient. Count how many of the six features each repository used in week one, group 0, 1, 2, 3 and 4 or more, and print each group's size and came-back rate.
Print the group size next to every rate. The script does, and it is the habit that stops a 16-repository 37.5 percent from being read as a law. Any row under 300 gets a "small" flag in the report you hand on, and the flag travels with the number into every deck.
Before session 7
30 min- Separate the setup artefact. Change the branch test so the first branch created in the birth hour does not count, re-run step 4, and write down whether branch's gap between users and the rest widens or narrows.
- Move the window. Replace
e[0] < 168withe[0] < 24(day one only) and compare breadth for issues and pull requests against the week-one table. Say in one sentence which features a day-one metric undersells. - Add a seventh feature, for example a fork or a star received, from the event types already in the cache, and place it in the table. Flag it as small if it has fewer than 300 users.
- Write the four-column adoption table for one feature in your own product, with its denominator and window stated above it, and mark which column you could not fill today.
Three questions
Clear all three and analyst session 6 gets stamped in your passport.
1 · "Branch created" reaches 68.8 percent of new repositories in week one, yet its users come back at 4.3 percent against 3.0 for the rest. What explains the weak signal?
5,744 repositories is the largest group in the table, so size is not the problem. Its time to adopt of 0 hours and depth median of 1 point the same way: count the user's own branches, not the one the product made.
2 · Push has a depth median of 2; every other feature has a median of 1. What can you say about the used-again step of each funnel without computing anything new?
A median splits the users in half, so it bounds the used-again share from one side. A mean above the median, like wiki's 3.9 on 11 repositories, says a few users went deep, and on a small group that is a handful of rows.
3 · Repositories using 4 or more features come back at 37.5 percent. How should that number appear in a review?
The three large groups carry the gradient; the 16-repository group shows its direction. Repositories being worked on use more features and come back, and only a randomised test can say whether pushing a feature changes return.
What this session teaches, and where it came from
adoption-numbers.py, the code on this pageEvery fact on this page, and its verification tier, is recorded in the course's source map.
Analyst session 6 cheat sheet · pin this
Session 7 takes the four columns to an AI feature, where "used" splits again: a suggestion is shown, accepted, then kept, edited or deleted. On a constructed code-assistant stream of 119,905 suggestions, the long-suggestion version wins on acceptance (0.309 against 0.262) and the short one wins on what is still there ten minutes later, so the metric has to be chosen for the decision.