learn-product-analytics-with-phoebe / Analyst session 5 of 8
Learn Product Analytics with Phoebe · Analyst track · Session 5 of 8

The power-user curve from real rows

Session 3 gave the same accounts three MAUs and one ratio each. A ratio is one number, and one number cannot say whether your monthly actives are 79,000 people who each dropped by once or 5,000 people who live in the product. The power-user curve answers that by counting, per account, how many of the 28 days it was seen, and drawing the histogram. Tonight we build that histogram from real GH Archive rows in Python and matplotlib, read where the account-days concentrate, and then walk into the right tail, where the accounts seen on 22 or more days turn out not all to be people.

🟠 Hands-on Product and data analysts Python + matplotlib 45 min
0-3 · Welcome 3-18 · Histogram, concentration, the tail 18-40 · Build-along: the L28 from 56 hours 40-45 · Q&A
Part 0

Where we start

The rows are the ones session 3 built: every GitHub account whose id is divisible by 20, a deterministic 5 percent sample, watched in two sampled hours a day (03:00 and 15:00 UTC) for the 28 days from 2024-03-04 to 2024-03-31. That is 56 hourly files from GH Archive, 79,555 accounts seen at least once, 98 accounts with a [bot] suffix dropped. An account is "active on a day" if it did anything in either sampled hour of that UTC day. Two hours out of 24 is a thin slice, and the slice matters for everything on this page: it pushes every account to the left of the curve, and it is why the curve is taught as what the sample can see, never as GitHub's real stickiness.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = run the code yourself

★ What you walk out with tonight. A Python script that turns 56 GH Archive hours into a days-per-account count, an L28 histogram saved as a 300 DPI PNG and an SVG, the six band shares it prints (66.7, 24.0, 7.5, 1.6, 0.2 and 0.1 percent), two concentration numbers (a tenth of accounts hold 33.5 percent of account-days; a hundredth hold 30.1 percent of events), and a name check that finds 17 of the 105 accounts on 22 or more days wearing bot-like names without the suffix.

Part 1 · covers the L28 as a histogram

Count the days, then draw the count

6 min live

Li Jin and Andrew Chen (a16z, 2018) define the power-user curve as "a histogram of users' engagement by the total number of days they were active in a month", and note it is "also commonly called the activity histogram or the 'L30' (coined by the Facebook growth team)". Ours is an L28 because the sample covers 28 days. The x axis is days active, 1 to 28; the y axis is how many accounts landed on each day count. Every account contributes exactly one mark. That is the whole construction, and it is why the curve is more honest than DAU/MAU, which the same post says "is a single number and so blurs this variance".

L28, ANY EVENT 79,555 sampled accounts · 28 days · two sampled hours a day · [bot] logins dropped 66.7% 24.0% 7.5% 1.6% 0.2% 0.1% 53,046 accounts 19,081 5,946 1,257 120 105 1 day 2-3 days 4-7 8-14 15-21 22-28 the "smile" Jin and Chen describe would rise again on the right this one keeps falling one account, seen on 3 of 28 days Every account is one mark on this axis. The two right-hand bands are drawn at a 3 px floor; to scale they would be invisible.
🔍 Click to zoom - two thirds of the sampled accounts were seen on exactly one day, and the curve never turns back up
LiveFrom day-marks to a histogram3 min▶

The construction has two steps and no statistics. First, per account, collect the set of UTC days on which it was seen; the size of that set is its score, 1 to 28. Second, count how many accounts landed on each score. That gives a list of 28 counts, and the histogram is that list drawn as bars. For the sample under "any event" the first count is 53,046 and the last is 62: 53,046 accounts seen on exactly one day, 62 seen on all 28.

Grouping into bands is a presentation choice, and the one on the figure (1, 2-3, 4-7, 8-14, 15-21, 22-28) is the one the script prints. Bands make the shape readable across the room; the 28-bar version is what you inspect at your desk. Keep both. The band shares under "any event" are 66.7, 24.0, 7.5, 1.6, 0.2 and 0.1 percent, and they sum to the MAU of 79,555 because every active account is in exactly one band.

Self-studyThree definitions, three curves2 min▶

Session 3's three definitions of active each produce their own L28, and the shape moves with the definition. Under "pushed code" (MAU 48,946) the bands read 64.2, 26.0, 8.0, 1.5, 0.1 and 0.2 percent, with 57 accounts on all 28 days. Under "opened an issue or pull request" (MAU 8,993) they read 77.5, 18.3, 3.7 and 0.4 percent for the first four bands, then almost nothing: 1 account on all 28 days, 4 on 22 or more. The rarer the behaviour, the further left the curve leans. Publish the definition with the curve, every time, because the curve is unreadable without it.

Jin and Chen's post says Facebook "would have a very right-leaning smile, with 60%+ of its MAUs coming back daily". That is their description, not a number anyone outside Facebook has checked, so hold it as a claim about what a social product can look like, not as a benchmark to hit.

Real world

The ratio that hid two products. A common pattern, no numbers attached: a B2B tool reports a healthy DAU/MAU and a flat MAU, so nobody looks. The L28 shows a spike at one day and a second bump at 20-plus days, with almost nothing between. Two populations are sharing one ratio: people who evaluated it once and left, and a core team that lives in it. The average of those two groups describes nobody, and the histogram is the cheapest way to find out you have two products wearing one name.

Part 2 · covers concentration

Who owns the account-days

5 min live

A left-leaning curve says most accounts are occasional. Concentration asks the complementary question: of all the activity, how much sits with the few? The unit for the L28 is the account-day, one account seen on one day. Rank the sampled accounts by days and the top 10 percent hold 33.5 percent of all account-days. Rank them by events and the top 1 percent hold 30.1 percent of all sampled events. The median account was seen on one day. Half of your monthly actives, in this sample, are one-day visitors.

WHERE THE ACTIVITY SITS same 79,555 accounts, any event, ranked two ways ACCOUNT-DAYS (one account seen on one day) EVENTS (everything a sampled account did) THE MEDIAN ACCOUNT 33.5% 66.5% top 10 percent of accounts, ranked by days the other 90 percent of accounts 30.1% 69.9% top 1 percent of accounts, ranked by events the other 99 percent of accounts 28 day-cards 1 day-card median: 1 of 28 Median is 1 day of 28. Half the accounts are one-day visitors, and a tenth of them hold a third of the account-days.
🔍 Click to zoom - the gold slices are the accounts a "monthly active" headline is silently describing
LiveTwo rankings, two shares3 min▶

Rank by days and you measure breadth of presence: how many of the 28 days an account showed up. Rank by events and you measure volume: how much it did. They point the same way here (33.5 percent of account-days with the top tenth; 30.1 percent of events with the top hundredth) but they are different questions, and a product team should choose the one that matches what it sells. A daily habit product cares about days. A tool billed on usage cares about events.

Concentration is also a warning about denominators. When a tenth of accounts hold a third of the account-days, any per-account average is being pulled toward that tenth, and the median (1 day) is the number that describes the typical account. Session 2 met the same shape inside one hour, where the top 1 percent of actors produced 38.7 percent of the events. It is the normal shape of a public event stream, and the reason the median is printed next to every mean.

Self-studyA share is not a segment2 min▶

"The top 10 percent" is a slice of a ranking, not a list of named people you can act on. The cut point moves every month as the population changes, and an account that was in the top tenth in March may drop out in April without changing its own behaviour at all, simply because others moved. Segments that a team acts on (a mailing list, a feature flag) need a fixed rule in absolute terms, such as "seen on 8 or more days", so that membership is stable and comparable across months. Ranked shares are for describing the shape; absolute thresholds are for building lists.

Real world

The average customer who did not exist. A common pattern, no numbers attached: a usage-billed API reports the average calls per customer to a pricing review, and the new tier is designed around that average. The histogram, drawn later, shows the median customer far below it and a handful of accounts far above. The tier priced for the average is too expensive for almost everyone and too cheap for the few. A concentration line, "the top 1 percent of accounts make X percent of calls", printed next to the average would have stopped the review in its first minute.

Part 3 · covers where automation hides and what sampling does

The right tail is not all people

4 min live

The 22-28 band holds 105 accounts, and on a power-user curve those are the accounts a team would call its core. Run a crude name check over their logins (bot, -ci, ci-, sync, deploy, commenter, assistant) and 17 of the 105 match, without any [bot] suffix to give them away. The suffix filter from session 2 removed 98 accounts; it did not remove these. A name check is a weak instrument, it misses bots with plain names and it can flag a person whose login happens to contain "sync", so the honest reading is that some unknown share of the tail is automation, and the 17 is a floor for how many we can see.

THE TAIL, ZOOMED: DAYS 22 TO 28 TWO SAMPLED HOURS OF 24 7 6 8 3 5 14 62 22 23 24 25 26 27 28 days 105 17 bot-like names no [bot] suffix 88 not matched by the name check on 22+ days 03:00 15:00 22 hours unseen an account active only at other hours is never counted as active that day Two hours a day shift every account left. This is what the sample can see, never GitHub's true stickiness.
🔍 Click to zoom - the tallest bar in the tail is the one a team would celebrate, and a sixth of the band has a machine's name
LiveTwo ways the tail lies2 min▶

Automation looks like devotion. A scheduled job runs every day, so it lands on day 28 of the L28 and sits at the very top of the curve. The 62 accounts seen on all 28 days are exactly where a scheduled job lands, and the crude check cannot say how many of them are scripts with ordinary logins. Under "pushed code" the same tail reads 92 accounts on 22 or more days, 11 of them bot-like; under "opened an issue or pull request" it is 4 accounts, 1 bot-like. The narrower the definition, the fewer machines survive it, which is one more argument for choosing "active" as a behaviour that requires a person.

Sampling looks like disengagement. The other direction is just as important. A real daily user who works from 06:00 to 14:00 UTC touches neither sampled hour and scores zero days in the month; a user who overlaps one sampled hour on some days scores a handful. Two hours of 24 is 1/12 of the day, and the curve is left-leaning partly because of that. Part of the shape is real (most public GitHub accounts are occasional) and part is the instrument, and the rows cannot tell you the split. Say both when you present it.

Self-studyWhat to do about names2 min▶

Do not silently drop the 17. Print them, with their day counts and event counts, and read the list. A name check is a hypothesis generator; the confirmation comes from behaviour: an account that acts at the same minute every day, only ever pushes, and never opens, stars or comments is behaving like a schedule. Keep the rule you settle on in code next to the [bot] suffix filter, so that the next person sees both filters and the reasoning, and report the curve with and without the flagged accounts. When the two curves disagree materially, the automation share is a finding, not a footnote.

In a product you run, the same problem appears as internal accounts, QA users, monitoring probes and integrations, and it appears first in the right tail, because that is where anything that never sleeps ends up.

Real world

The core users who were a monitor. A common pattern, no numbers attached: a team celebrates its most engaged accounts in a quarterly review and, when someone finally pulls the list, the top of it is an uptime probe hitting the login page every few minutes, a QA account exercising every feature nightly, and an integration that syncs a calendar on a schedule. None of them will ever churn, and none of them will ever pay. The fix was one line in the definition of active, and a printed list of the top accounts by days that somebody reads before every review.

Build-along

Build the L28 from 56 GH Archive hours

★ 22 min · everyone builds

Standard-library Python for the counting, matplotlib for the drawing. The sample is deterministic (account id divisible by 20), so the numbers printed as comments below are what this code prints on the 56 files, and yours should match them exactly. Step 1 is the slow part: 56 files, each about the size of the session 1 hour (150,807,371 bytes), streamed once. Start it before class, or download the first few files and confirm the shape of actor-days.json before letting it run.

FIVE STEPS, FIFTY-SIX FILES 1 · 56 files 03:00 + 15:00 UTC 2 · one JSON id % 20 == 0 only 3 · days seen a set per account 4 · histogram hist[29], 6 bands 5 · plot + save PNG 300 DPI + SVG 28 days of March 2024 79,555 accounts 53,046 on 1 day, 62 on 28 two files on disk Steps 3 to 5 work unchanged on your product's export. Only step 1 and the definition of a "day active" are yours.
🔍 Click to zoom - the histogram is a 29-slot list; everything after step 3 is arithmetic and drawing

Fetch the 56 sampled hours and keep the 5 percent. GH Archive names hourly files by UTC date and hour with no zero padding, so 03:00 is -3 and 15:00 is -15. For each sampled account the file keeps its login, its event count, its push count and how many issues or pull requests it opened, per hour. This is the only step that touches the network.

★ Step 1 · actor-days.json from 56 GH Archive files (slow: run it ahead)import gzip, json, urllib.request DAYS = ["2024-03-%02d" % d for d in range(4, 32)] # 28 days hours = {} for day in DAYS: for h in (3, 15): name = "%s-%d.json.gz" % (day, h) urllib.request.urlretrieve("https://data.gharchive.org/" + name, name) actors = {} for line in gzip.open(name, "rt"): e = json.loads(line) aid = e["actor"]["id"] if aid % 20: continue # the 5 percent sample v = actors.setdefault(str(aid), [e["actor"]["login"], 0, 0, 0]) v[1] += 1 v[2] += int(e["type"] == "PushEvent") p = e.get("payload") or {} v[3] += int(e["type"] in ("IssuesEvent", "PullRequestEvent") and p.get("action") == "opened") hours["%s-%d" % (day, h)] = {"actors": actors} json.dump(hours, open("actor-days.json", "w"))

Days seen per account. One set of UTC days per account, under "any event". The [bot] suffix filter from session 2 rides along, and the counts it prints are the ones to check first.

★ Step 2 · a set of days per accountimport json, re from collections import defaultdict hours = json.load(open("actor-days.json")) days = sorted({h.rsplit("-", 1)[0] for h in hours}) assert len(days) == 28 seen, events, logins, bots = defaultdict(set), defaultdict(int), {}, set() for h, blob in hours.items(): day = h.rsplit("-", 1)[0] for aid, v in blob["actors"].items(): if v[0].endswith("[bot]"): bots.add(aid); continue logins[aid] = v[0] events[aid] += v[1] if v[1] > 0: # any event: swap for v[2] (push) or v[3] (open) seen[aid].add(day) print(len(seen), len(bots)) # 79555 98

The histogram and its bands. A 29-slot list so that index equals days; slot 0 stays empty because every account in seen has at least one day.

★ Step 3 · hist[29] and six band sharesldays = [len(s) for s in seen.values()] mau = len(ldays) hist = [0] * 29 for x in ldays: hist[x] += 1 BANDS = ((1, 1), (2, 3), (4, 7), (8, 14), (15, 21), (22, 28)) share = [100 * sum(hist[a:b + 1]) / mau for a, b in BANDS] print(mau, hist[1], hist[28]) # 79555 53046 62 print(" ".join("%.1f" % s for s in share)) # 66.7 24.0 7.5 1.6 0.2 0.1 print(sorted(ldays)[mau // 2]) # 1 (median days)

Concentration and the tail. Two rankings, then the crude name check over the accounts on 22 or more days. Print the flagged logins and read them; the number alone is not the finding.

★ Step 4 · two shares and the name checktop = sorted(ldays, reverse=True)[: mau // 10] print("%.1f" % (100 * sum(top) / sum(ldays))) # 33.5 (account-days, top 10 percent) ev = sorted(events.values(), reverse=True) print("%.1f" % (100 * sum(ev[: len(ev) // 100]) / sum(ev))) # 30.1 (events, top 1 percent) tail = [a for a, s in seen.items() if len(s) >= 22] botish = [a for a in tail if re.search(r"bot|-ci\b|ci-|sync|deploy|commenter|assistant", logins[a], re.I)] print(len(tail), len(botish)) # 105 17 for a in botish: print(logins[a], len(seen[a]), events[a]) # read this list

Draw it and save both formats. Bars for all 28 days, band shares annotated above the first bar of each band, a title that names the definition and the sample. The PNG at 300 DPI is for a slide; the SVG is for anything that will be edited later.

★ Step 5 · matplotlib, PNG 300 DPI and SVGimport matplotlib matplotlib.use("Agg") import matplotlib.pyplot as plt fig, ax = plt.subplots(figsize=(9, 4.5)) ax.bar(range(1, 29), hist[1:], color="#1D4AFF", width=0.8) for (a, b), s in zip(BANDS, share): ax.annotate("%.1f%%" % s, ((a + b) / 2, hist[a]), ha="center", va="bottom", fontsize=8) ax.set_xticks(range(1, 29)) ax.set_xlabel("days active out of 28 (two sampled hours a day)") ax.set_ylabel("accounts") ax.set_title("L28, any event, 5 percent GitHub account sample, March 2024") fig.tight_layout() fig.savefig("l28-any.png", dpi=300) fig.savefig("l28-any.svg") # two files written; 53,046 in the first bar, 62 in the last

Then draw it again on a log axis. Add ax.set_yscale("log") and save a second pair of files. On the linear axis the 62 accounts on day 28 are a sliver next to 53,046, which is the truth about their share; on the log axis you can see the tail's shape, which is where the automation question lives. Show the linear one to the room and keep the log one for yourself.

Homework

Before session 6

30 min
  1. Draw the other two curves. Swap v[1] for v[2] (pushed code) and then v[3] (opened an issue or pull request) in step 2, re-run steps 3 and 5, and check your band shares against the ones in Part 1's self-study card. Write one sentence on which way the curve moved and why.
  2. Read the 17. Print the flagged logins with their day and event counts, then print the 28-day accounts that the name check did not flag. Decide a behavioural rule (for example: pushes only, never opens or stars) and report how the 22-28 band changes with it. Your rule will differ from your neighbour's; the point is to write it down.
  3. Save the log-axis pair. Produce l28-any-log.png and .svg, and note which single day count is the first one you can read on the log axis but not on the linear one.
  4. Self-study, no numbers to check: split the sampled accounts by the first sampled day they were seen and draw one L28 per first-seen week. This is a cohort view, and it has the trap session 8's cohort decision names: first seen is not sign-up. Cohort curves proper belong to customer retention.
Check yourself

Three questions

Clear all three and analyst session 5 gets stamped in your passport.

1 · Under "any event", 66.7 percent of the sampled accounts sit in the one-day band. What may you conclude about GitHub's real stickiness?

Part of the left lean is real and part is sampling, and the rows cannot split the two. The honest sentence is "this is what two sampled hours a day can see", and the ratio in B is also the sample's, not GitHub's.

2 · The top 10 percent of accounts by days hold 33.5 percent of account-days, and the median account was seen on 1 day. Which number should sit next to "average days active" in a report?

Concentration is a warning about denominators. Session 2 met the same shape inside one hour (the top 1 percent of actors making 38.7 percent of events), and the rule is the same: print the median beside every mean.

3 · 17 of the 105 accounts on 22 or more days match a crude bot-name pattern and carry no [bot] suffix. What is the right next step?

A name check misses scripts with plain logins and can flag a person; behaviour confirms it. Two curves, side by side, turn the automation share into a finding the reader can weigh instead of a footnote they cannot check.

Sources covered

What this session teaches, and where it came from

✓Power-user curve definition, the L30 name, and the DAU/MAU critiqueJin, L. and Chen, A. (2018). "The Power User Curve: The best way to understand your most engaged users." a16z, 6 August 2018. Read at source; quoted exactly
✓Every L28 count, band share, concentration figure and tail countMeasured on GH Archive, 56 sampled hours (03:00 and 15:00 UTC, 2024-03-04 to 2024-03-31), 5 percent account sample, by the code on this page
✓GH Archive file naming and contentsGH Archive. https://www.gharchive.org/ (hourly archives, data.gharchive.org); GitHub Docs. "GitHub event types."
◐Facebook as a "right-leaning smile" with 60 percent or more of MAUs dailyThe same a16z post; their description, not verified against any published Facebook number, and taught as a claim
◐Which tail accounts are automationA crude name pattern only; 17 of 105 is a floor and the true share is unknown

Every fact on this page, and its verification tier, is recorded in the course's source map.

Analyst session 5 cheat sheet · pin this

L28 = days per account, then countOne mark per account; hist[29], index equals days.
Six bands, any event66.7 · 24.0 · 7.5 · 1.6 · 0.2 · 0.1 percent of 79,555.
Median 1 day, mean beside itTop tenth holds 33.5 percent of account-days; top hundredth 30.1 percent of events.
Publish the definition with the curvePushed and opened lean further left; the curve is unreadable without it.
The tail is not all people105 on 22+ days, 17 bot-like names, no suffix. Print, confirm, report both curves.
Sampling shifts everyone leftTwo hours of 24. What the sample sees, never GitHub's real stickiness.