learn-rfm-modeling-with-phoebe / Session 4 of 6
Learn RFM Modeling with Phoebe · Session 4 of 6

Beyond purchase

RFM assumes the purchase table describes the customer. On a marketplace like Olist that is nearly true: everyone in the file bought at least once. On a platform where most people browse, watch or share and never buy, the purchase table describes almost nobody, and an RFM built on it is a segmentation of the minority who happened to check out. Tonight three real platforms each break one of the three letters, and each one needs a different fix: replace an axis, redefine an axis, or add one. Knowing which move a platform needs is the skill; the pandas is the easy part.

🟡 Core Analysts, growth and platform teams Python, pandas 45 min
0-3 · Welcome 3-18 · Three platforms where the order table is the wrong table 18-40 · Build-along: an engagement RFM on an events table 40-45 · Q&A
Part 0

Where we start

Sessions 1 to 3 moved the knife on one order table and watched who was called best. Every one of those cuts shared a silent assumption: that the people worth segmenting are the people in the order table. A shopping platform where nearly everyone in a given week only looked, a live room where the loyal regulars watch every night and buy twice a year, a team-purchase app where the most valuable customer is the one who brings two friends. In each of those, a purchase RFM does not score the interesting people low. It gives them no row. This session is about noticing which letter is empty, and choosing between three repairs that are not interchangeable.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = run the code yourself

★ What you walk out with tonight. A working engagement RFM built on an events table instead of an orders table, with two recency clocks and a weighted engagement score whose weights are written down as a decision. A three-way rule for any platform: replace a letter when nobody has it, redefine a letter when it counts the wrong thing, add a letter when the behaviour is genuinely new. And the discipline to say "no primary source states that" when a number everyone repeats has no floor under it.

Part 1 · covers engagement in place of spend, RFE

When the missing letter is the one nobody has

6 min live

Taobao's UserBehavior dataset is nine days of one platform, 25 November to 3 December 2017: about 100 million rows from 987,994 users on 4,162,024 items, and every row is one of four things a person did, a page view (pv), a favourite (fav), an add to cart (cart) or a purchase (buy). Published analyses of it report that page views dominate and purchases are a low single-digit share of rows. Build a purchase RFM on that and most of the 987,994 users have no row at all. Not a low score. No row.

FOUR BEHAVIOURS, ONE THIN SLIVER Taobao UserBehavior: pv, fav, cart, buy over nine days in 2017 RFE: E REPLACES M pv · page views most of the rows fav · favourites cart · add to cart buy · a low single-digit share published analyses, not recomputed PURCHASE RFM R F M buyers only never bought: no row R F E engagement replaces spend E = 1·pv + 2·fav + 3·cart + 5·buy one common choice, not a fact Replace, do not add. When the letter nobody has is M, a fourth axis would be as empty as the third.
🔍 Click to zoom - the purchase table reads only from the sliver, and the dashed box is where most of the platform's users live
LiveReplace, do not add, when the missing letter is the one nobody has4 min▶

On a platform like this the letter that is empty is M. Most users have never spent anything, so a monetary column is zero for nearly everyone and the knife, wherever you put it, sorts almost the whole base onto one side. The tempting repair is to keep M and bolt an engagement axis on beside it. That gives you four letters, of which one is still empty, and sixteen corners of which eight hold almost nobody.

The cleaner repair is RFE: keep R and F, and put E where M was. R becomes days since the last action of any kind. F becomes the count of actions, or of sessions if the events carry a session id. E is an engagement score: a weighted count of what the person did, where a purchase is worth more than a page view because it costs the person more. One common choice of weights is pv 1, fav 2, cart 3, buy 5. Those numbers are a decision, not a measurement. Nobody has shown that a cart is worth exactly three page views; someone decided that ordering felt right and wrote it down. That is fine, as long as it is written down and can be argued with.

The test for "replace" is simple: if the letter you are about to add would be empty for most of the base, you are not adding, you are replacing, and you should say so. Part 3 shows the opposite case, where the old letters are all still full and a genuinely new behaviour earns a fourth column.

Self-studyTwo recencies: last action and last purchase, and why a platform keeps both2 min▶

Replacing M with E does not mean forgetting purchase. It means purchase recency and action recency become two different columns, and they answer two different questions. R_action, days since anything happened, tells you whether the person is still around. R_buy, days since the last purchase and missing for anyone who never bought, tells you whether they have ever crossed the line and how long ago. A user with R_action 1 and R_buy missing is alive and has never converted: an onboarding problem. A user with R_action 40 and R_buy 41 bought once and left the same day: a different problem entirely.

Keep both. Segment on R_action, because it is defined for everyone, and carry R_buy as a column you read when you choose the action. The build-along computes both, and the missing values in the second are the point, not a data-quality problem to fill.

Real world

A dataset that is mostly looking. The Tianchi page for UserBehavior gives the shape: nine days, about 100 million rows, 987,994 users, 4,162,024 items, four behaviours. The split between those behaviours is not restated here as a recomputed figure, because the 3.4 GB file sits behind a login and was not downloaded for this build; the "page views dominate, purchases are a low single-digit share" statement is what published analyses of it report, and it is cited as that. The honest limit matters for the lesson: even without the exact split, a platform where a buy is a rare row is a platform where a purchase RFM segments the exception and ignores the rule.

Part 2 · covers live commerce and room loyalty

The room regular the order table cannot see

5 min live

Douyin sells through live rooms. The 索象集团 (Suoxiang Group) and 中国直播产业研究院 (China Live Streaming Industry Research Institute) white paper on Douyin live commerce (2025) puts 2024 Douyin e-commerce GMV above 3 trillion yuan with live scenes over 58 percent of sales, and splits the live side into 店播 (brand rooms, a brand broadcasting from its own room) at 41 percent, against 达播 (influencer rooms) at 38 percent for 2024. CBNData (2025) reports 店播 (brand rooms) revenue at around 70 percent of live revenue and a shelf / 店播 (brand rooms) / 达播 (influencer rooms) mix of roughly 4:3:3. The two reports do not agree, and their denominators are not the same; both are quoted here by name and neither is reconciled. What matters for RFM is what a brand room does to the letter F.

ONE CUSTOMER, TWO CLOCKS F = sessions attended + orders; M stays GMV; R has two clocks LIVE buy a brand's own room R_watch · last session in the room resets every night she shows up R_buy · last order from the room missing for the regular who never buys THE CELL A PURCHASE RFM CANNOT SEE zero spend some spend high low WATCH FREQUENCY SPEND watches often never buys no row in RFM watches often and buys the room's core drops in never buys a passer-by rarely watches buys anyway a shelf buyer Redefine F as sessions plus orders and the hatched cell appears. Purchase RFM gives it no row, not a low score.
🔍 Click to zoom - the hatched cell is the audience a brand room operator most wants to name, and the order table has never heard of them
LiveTwo clocks, one customer3 min▶

A brand room is not a shop with a video on it. It runs at a fixed hour, the same faces come back, and the thing a regular does most is watch. So the letter that breaks is not M this time, it is F. Counting orders makes the nightly regular who buys twice a year look like a two-time buyer, indistinguishable from someone who found the room through a shelf search and never came back. The repair is not to replace F but to redefine it: frequency becomes sessions attended in the room plus orders placed, so attendance counts as the loyalty it is.

Recency splits into two clocks. R_watch is days since the last session attended, defined for everyone who has ever been in the room. R_buy is days since the last order, and missing for anyone who never bought. Monetary stays what it always was, the person's GMV in the room, because on a live platform it is the one letter that still means what it says.

The segment this exposes is the one a 店播 (brand rooms) operator cares about most: high watch frequency, zero spend. In a purchase RFM that person does not have a low score, they have no row. Under the redefined F they are a named group, and session 5 gives them an action and a holdout.

Self-study店播 (brand rooms) against 达播 (influencer rooms): whose customer is it2 min▶

The white paper's split between 店播 (brand rooms) and 达播 (influencer rooms) is not just a market-share table, it is a question about who owns the RFM row. In a 店播 (brand rooms) room the brand runs the broadcast, the regular's loyalty is to the brand, and every session she attends is a data point the brand may keep and act on. In a 达播 (influencer rooms) room the loyalty is to the host. The brand buys a slot, the sale is real, but the room regular is the host's customer, and a "frequency" built on attendance in a host's room is measuring a relationship the brand does not have. The RFM row belongs to the room owner, and only in 店播 (brand rooms) is that the brand.

This is why a brand that sells through both channels should build the redefined RFM on its own room only, and treat 达播 (influencer rooms) orders as acquisition events feeding the purchase side, not as frequency. Mixing them inflates F with sessions the brand never hosted.

Real world

What the reports say, and what they do not. The 索象集团 (Suoxiang Group) and 中国直播产业研究院 (China Live Streaming Industry Research Institute) white paper (2025): 2024 Douyin e-commerce GMV over 3 trillion yuan, live scenes over 58 percent of sales, 店播 (brand rooms) 41 percent against 达播 (influencer rooms) 38 percent in 2024. CBNData (2025): 店播 (brand rooms) revenue around 70 percent of live revenue, shelf / 店播 (brand rooms) / 达播 (influencer rooms) roughly 4:3:3. Read those two 店播 (brand rooms) figures side by side and they do not match; one is a share of all sales and the other a share of live revenue, and the methods behind each are the reports' own. Neither report, and no other primary source found for this build, publishes a repeat-purchase rate for live rooms. Any "复购率 (repeat-purchase rate)" figure you meet for Douyin rooms is a vendor or press claim, and none is stated here as fact.

Part 3 · covers the referral axis

The customer who is a channel

5 min live

Pinduoduo's team purchase is described in PDD Holdings' Form 20-F: buyers "share product information on social networks, and invite their friends, family and social contacts to form shopping teams to enjoy the more attractive prices available under the team purchase option". The lower price is the reward for the share. That makes the invitation a behaviour with a direct commercial value, and a purchase RFM has no column for it. Here the old letters are all still full, so the move is not to replace one. It is to add one.

EIGHT CORNERS GROW A FOURTH AXIS R, F, M from session 1; I counts invitations or teams initiated R · MORE RECENT F · MORE OFTEN M · BIGGER SPEND 000 111 from session 1 I · INVITATIONS the axis out of the page one buyer I = 2 friend friend team price reward for sharing RFM-I: sixteen corners, or keep eight and flag I separately low M, high I is an acquisition channel, a customer the order table cannot see ADD, do not replace: M is full on Pinduoduo Add an axis when the behaviour is new. Replace one when the old letter is empty. Never both by reflex.
🔍 Click to zoom - the tangerine arrow is the only axis that does not come from the order table, and the buyer with I = 2 is worth more than her basket says
LiveAdd an axis when the behaviour is new, replace one when the old letter is empty3 min▶

On Pinduoduo every buyer has an order, so R, F and M are all defined and all mean what they say. What the order table misses is a fourth thing the platform is built on: the invitation. A customer who buys a small basket but forms three teams a month has brought the platform more new buyers than a customer with a large basket and no shares, and in a three-letter RFM the two are indistinguishable or, worse, the sharer scores lower.

So the move is RFM-I: add I, a count of invitations sent or teams initiated in the window. Now the customer with low M and high I has a name, acquisition channel, and an action that is nothing like the action for a low-M non-sharer. The distinction between this and Part 1 is the whole session: there, the letter you would add is empty for almost everyone, so you replace. Here, the letters you already have are full and the new one is too, so you add. Confusing the two either buries a real behaviour inside a score or multiplies your corners by two for nothing.

On disclosure: PDD Holdings' 20-F describes the mechanism in the words quoted above but no longer discloses an active-buyer count. The 882 million figure that circulates for 2023 is a secondary number and is not used here.

Self-studySixteen corners is too many to act on: flag, do not multiply2 min▶

A fourth binary axis doubles the corners to sixteen, and sixteen groups is more than any team writes distinct actions for. The alternative that survives in practice is to keep the eight-corner code from session 1 exactly as it was and carry I as a flag beside it: a "sharer" column set when I clears its own knife. Every one of the eight groups then has a sharer and a non-sharer variant, and the operator chooses which groups the flag changes the action for (usually the low-M ones, where the referral is the value) and ignores it elsewhere.

The same logic applies to any new axis. A second dimension that changes the action in two groups and nothing in six is a flag, not a corner. Multiply corners only when the new letter changes the action nearly everywhere, and be honest that it rarely does.

Real world

The team price is the reward for the share. That is the mechanism in PDD Holdings' own filing: a buyer shares, contacts join, and the team earns "the more attractive prices available under the team purchase option". The discount is not a loyalty gesture, it is payment for acquisition, and the buyer who shares is doing a marketing job. Any behaviour a platform pays for is a behaviour worth a column, because the platform has already decided it has value; the analyst's job is only to count it. On a platform with no such mechanic, an I column would be zero for everyone and the Part 1 rule applies instead.

Part 4 · covers weighting

Does R weigh the same as M?

3 min live

Every RFM tutorial eventually asks whether the three letters should count equally, and most then hand you a set of weights as if it had been measured. Hughes (1994) scored the three in sequence and equally: sort by R into fifths, each fifth by F, each of those by M, no letter heavier than another. Stone (1995) is widely cited for proposing unequal weights, but the weights could not be read at source for this build, so no canonical numbers are given here. Vendor content for content apps proposes yet another ordering, engagement and frequency over recency for heavy users. Three answers, none of them measured on your customers.

HOW MUCH DOES EACH LETTER WEIGH? three weights, one business judgement, no canonical numbers R M F ? w_R = ? w_F = ? w_M = ? Hughes 1994: scored equally, in sequence Stone 1995: weights often quoted, not read at source THREE STARTING POINTS Equal weights one order table, all three letters filled Recency-heavy perishable or seasonal buying: quiet means gone Engagement-heavy a content app: heavy users over recent ones which fits yours? a holdout answers, a citation does not A weight is a hypothesis about your customers. Write it down, then let the session 5 holdout say if it held.
🔍 Click to zoom - the only thing that settles the question mark is a test on your own base, never a number inherited from a slide
WeightingWhat it saysThe situation it fits
EqualR, F and M matter the same; Hughes's sequential scoring in effectAn order table where every letter is filled and nobody has a strong prior. The honest default
Recency-heavyA quiet customer is a lost customer, whatever they spentPerishable, seasonal or subscription-like buying, where the gap since the last order predicts the next one
Engagement-heavyHow much and how often beats how lately, especially on EContent and platform apps with heavy users; proposed in vendor content, not measured in a source read here
LiveWeights are a business judgement: state them, then test them3 min▶

There is no measured answer to how much recency should weigh against spend, and the literature does not pretend otherwise once you read it rather than quote it. Hughes weighted nothing. Stone is cited for weights that could not be verified at source here. Vendors propose weights that suit the product they sell. What all three have in common is that the weights encode a belief about which behaviour predicts the next purchase, and a belief about your customers is something you can test.

So the working rule: write the weights down as a dictionary with a one-line reason each, apply them, and hold out a slice of every segment when you act on it. Session 5 builds that holdout. If the recency-heavy weighting produces a segment that responds and the equal weighting does not, you have learned something about your base that no citation could have told you. If both respond the same, the weights did not matter and the simpler one wins.

Build-along

An engagement RFM on an events table, in pandas

★ 22 min · everyone builds

Seven steps on a constructed events table shaped like Taobao's UserBehavior: four columns, four behaviours, nine days. It is generated with a seed, so it runs anywhere, and it is not the real dataset; the share of each behaviour is a stated choice in the code, not the platform's. Your numbers will differ from your neighbour's and neither of you should copy one from a slide. What transfers is the shape of the pipeline: one table swapped, one column replaced, everything after the knife unchanged.

THE PIPELINE, SEVEN STEPS, ONE TABLE SWAPPED 1 · events user_id · item_id behavior · timestamp pv fav cart buy constructed, seeded orders orders table, set aside tonight 2 · SNAPSHOT a literal, day after 3 · one row per user R_action, R_buy F_actions, F_buys 4 · E, weighted sum over actions replaces M 5 · print zero-buy share 6 · knife · median 6 · 2x2x2 · eight codes on R_action, F_actions, E 7 · vs purchase-only RFM Same pipeline as session 1: events in place of orders, E in place of M. Everything after the knife is unchanged.
🔍 Click to zoom - the dashed orders table is the one every RFM tutorial starts from, and tonight it is the wrong table

Generate a constructed events table. Four columns, four behaviours, nine days, one seed. The behaviour shares in P are a choice made to look roughly like a browse-heavy platform; they are not Taobao's split and should not be quoted as one. If you have a real events export, load it here instead and skip the generator.

★ Step 1 · events, constructedimport numpy as np import pandas as pd from datetime import datetime rng = np.random.default_rng(20171203) N_USERS, N_EVENTS = 20_000, 400_000 BEHAVIOURS = ["pv", "fav", "cart", "buy"] P = [0.90, 0.03, 0.05, 0.02] # constructed shares, a choice, not the dataset's start = pd.Timestamp("2017-11-25") events = pd.DataFrame({ "user_id": rng.integers(1, N_USERS + 1, N_EVENTS), "item_id": rng.integers(1, 200_001, N_EVENTS), "behavior": rng.choice(BEHAVIOURS, N_EVENTS, p=P), "timestamp": start + pd.to_timedelta(rng.integers(0, 9 * 86400, N_EVENTS), unit="s"), }) print(events.shape) # (400000, 4) print(events["behavior"].value_counts(normalize=True).round(3))

Freeze the clock. The window ends on 3 December 2017, so the snapshot is the first day after it, as a literal with the reason beside it. Same rule as session 1: the end of the last complete period, never max().

★ Step 2 · the snapshot# the nine-day window runs 2017-11-25 to 2017-12-03; snapshot is the day after SNAPSHOT = datetime(2017, 12, 4) events = events[events["timestamp"] < SNAPSHOT]

One row per user, with two recencies and two frequencies. R_action and F_actions are defined for everyone. R_buy is missing for anyone who never bought, and that missingness is information, so it is left as NaN rather than filled.

★ Step 3 · R_action, R_buy, F_actions, F_buysbuys = events[events["behavior"] == "buy"] per_user = events.groupby("user_id").agg( last_action=("timestamp", "max"), F_actions=("timestamp", "size")) per_buy = buys.groupby("user_id").agg( last_buy=("timestamp", "max"), F_buys=("timestamp", "size")) rfe = per_user.join(per_buy, how="left") rfe["F_buys"] = rfe["F_buys"].fillna(0).astype(int) rfe["R_action"] = (SNAPSHOT - rfe["last_action"]).dt.days rfe["R_buy"] = (SNAPSHOT - rfe["last_buy"]).dt.days # NaN if never bought

E, the weighted sum. The weights are a dictionary at the top of the file, one common choice, with the reason for each written next to it. Change them and E changes; that is the point of having them in one place.

★ Step 4 · E replaces MWEIGHTS = {"pv": 1, # looked "fav": 2, # wanted to find it again "cart": 3, # nearly bought "buy": 5} # bought; a decision, not a measurement events["w"] = events["behavior"].map(WEIGHTS) rfe["E"] = events.groupby("user_id")["w"].sum()

How many users a purchase table would never see. Print the share with zero buys. Do not write the number into a slide before you have run it; on the constructed data it follows from P, and on your real data it is the single most important fact about your base.

★ Step 5 · the zero-buy sharezero_buy = (rfe["F_buys"] == 0).mean() print(f"users with zero buys: {zero_buy:.1%}") # your number will differ; do not copy one

The knife, then the eight codes. Median tonight, because E and F_actions are as skewed as any spend column; swap in .mean() and watch the shares move, as they did in session 2. Recency is reversed, the other two read the normal way, and the eight codes are the same eight from session 1.

★ Step 6 · 2x2x2 on R_action, F_actions, Eknife = rfe[["R_action", "F_actions", "E"]].median() # or .mean(); say which rfe["r"] = (rfe["R_action"] < knife["R_action"]).astype(int) # reversed rfe["f"] = (rfe["F_actions"] > knife["F_actions"]).astype(int) rfe["e"] = (rfe["E"] > knife["E"]).astype(int) rfe["code"] = rfe["r"].astype(str) + rfe["f"].astype(str) + rfe["e"].astype(str) print(rfe["code"].value_counts(normalize=True).round(3)) # eight rows, yours

Compare with a purchase-only RFM. Keep only users with a purchase and count what was dropped. Every dropped row is a person who looked, saved or carted and never bought: the audience Part 1 said the order table has no row for.

★ Step 7 · what the purchase table dropsrfm_purchase = rfe.dropna(subset=["R_buy"]) dropped = len(rfe) - len(rfm_purchase) print(f"purchase-only RFM keeps {len(rfm_purchase)} of {len(rfe)} users, drops {dropped}") print(f"that is {dropped / len(rfe):.1%} of the base with no row at all") # on the constructed table this follows from P; on yours it is the finding

Save the weights with the table. rfe_2017-12-04_w1-2-3-5.csv, not rfe.csv. A segment table built on E is only comparable with another one if both the snapshot and the weights are on the label, and session 6 will compare two of them.

Homework

Before session 5

30 min
  1. Find the event table behind your own product. Not the orders table: the one that records views, saves, sessions, shares, whatever your users actually do. Write down its columns and how many rows it holds against the orders table.
  2. List the behaviours it records. For each one, note roughly what share of users have ever done it. The letter that is empty for most of your base is the one you are about to decide on.
  3. Decide, per letter, replace or add. If the letter you want to add would be empty for most users, you are replacing an old one; say which. If all three old letters are full and the new behaviour is too, add it as a flag beside the eight codes.
  4. Write the weights down with a reason each, as a dictionary with a comment per line. Bring them: session 5 turns one segment into an action and a holdout, and the holdout is how a weight stops being an opinion.
Check yourself

Three questions

Clear all three and session 4 gets stamped in your passport.

1 · Taobao needs E in place of M; Pinduoduo needs I beside R, F and M. What decides whether a letter is replaced or added?

The test is about the base, not the analyst. If the letter you would add would be empty for most people, you are not adding, you are replacing, and you should say so. If the old letters are full and the new one is too, add it, and in practice carry it as a flag beside the eight codes rather than doubling the corners.

2 · A brand room's most loyal viewer watches every night and has never bought. Why does a purchase-only RFM fail to flag her?

The difference between a low score and no row is the whole point. A purchase RFM is a segmentation of buyers; the people it drops are not at the bottom of the table, they are absent from it, and no knife placement brings them back. Redefining F and adding a watch clock does.

3 · Two industry reports on Douyin live commerce are quoted by name, but no repeat-purchase rate for live rooms is stated. Why?

The professional norm is to attribute every figure to a named report and to say plainly when a widely repeated number has no primary source. The two reports are quoted with their own denominators and are not reconciled; the missing repeat rate is left missing.

Sources covered

What this session teaches, and where it came from

✓Taobao UserBehavior, the dataset's shapeTianchi dataset 649: 2017-11-25 to 2017-12-03, about 100 million rows, 987,994 users, 4,162,024 items, behaviours pv / fav / cart / buy; counts taken from the dataset page
◐The behaviour split, page views dominantCited to published analyses of the dataset, not recomputed; the 3.4 GB file is login-walled and was not downloaded for this build
✓Douyin live commerce figures索象集团 (Suoxiang Group) and 中国直播产业研究院 (China Live Streaming Industry Research Institute) white paper (2025) for GMV, live share and the 店播 (brand rooms) / 达播 (influencer rooms) split; CBNData (2025) for the 70 percent and 4:3:3 figures; each attributed, the two not reconciled; no repeat-purchase rate exists in a primary source and none is claimed
✓Pinduoduo team purchase mechanicsQuoted from PDD Holdings' Form 20-F; active-buyer counts are no longer disclosed there and the secondary 882 million figure is not used
◐Stone 1995 on weighting R, F and MWidely cited for unequal weights; not read at source in this build, so no canonical weights are given and weighting is taught as a live disagreement
◐Hughes 1994, equal sequential scoringDescribed from secondary summaries of Strategic Database Marketing; the book itself was not opened, as stated in session 1

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 4 cheat sheet · pin this

No row, not a low scoreA purchase RFM segments buyers. Everyone who never bought is absent, and no knife brings them back.
Replace when the letter is emptyBrowse-heavy platform: M is zero for most, so E takes its place. RFE, not RFME.
Redefine when it counts the wrong thingLive room: F = sessions attended + orders. Two clocks, R_watch for everyone, R_buy where it exists.
Add when the behaviour is newTeam purchase: I for invitations. Low M, high I = acquisition channel. Flag it beside the eight codes.
Weights are a decisionpv 1, fav 2, cart 3, buy 5 is one common choice. Write yours down with a reason each; test them in session 5.
Attribute, and say what is missingTwo Douyin reports, quoted by name, not reconciled. No primary repeat-purchase rate exists, so none is stated.