learn-rfm-modeling-with-phoebe / Session 1 of 6
Learn RFM Modeling with Phoebe · Session 1 of 6

Three numbers and a snapshot date

Every customer a shop has ever had can be described with three numbers: how long since they last bought, how many times they have bought, and how much they have spent. That is RFM, the oldest segmentation still in daily use, and it survives because it is cheap, honest and needs nothing but an order table. The catch is that none of the three numbers exists until you have chosen a date to measure them from, and moving that date by six weeks moves every customer. Tonight we build the three columns on 93,358 real customers, freeze the clock, and turn the columns into the eight named groups that most operations teams actually run.

🟡 Core Analysts, growth and CRM teams Python, pandas 45 min
0-3 · Welcome 3-18 · R, F, M and the frozen clock 18-40 · Build-along: raw orders to eight groups 40-45 · Q&A
Part 0

Where we start

RFM looks like a scoring exercise and is really a chain of small decisions: what counts as an order, which id is the customer, what "spend" means, what date the clock stops at, and where the knife goes between high and low. Each decision is quick, each one changes who lands in which group, and most tutorials make all of them silently. These six sessions make them out loud, on a real marketplace where 97 percent of customers bought exactly once, because that is the shape of most retail data and it is the shape that breaks the defaults.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = run the code yourself

★ What you walk out with tonight. A working RFM table on 93,358 real customers with the snapshot date written into the code, the eight-group taxonomy you can read as a mechanism instead of a poster, and the one sentence that stops the first argument every RFM project has: "as of which date?"

Part 1 · covers recency, frequency, monetary

One customer becomes three numbers

6 min live

Take one real customer from the Olist marketplace. Three delivered orders between October 2017 and August 2018, R$ 515 in total. Measured from 1 September 2018 she is R 11, F 3, M 515. Every one of those three carries a choice you have to make before the number exists.

ONE CUSTOMER, AS THE ORDER TABLE SEES HER customer f8bc1d81, three delivered orders 2017-10-06 one more order 2018-08-20 snapshot 2018-09-01 she is one of 93,358 R · recency 11 days F · frequency 3 orders M · monetary 515 R$ total THE CHOICE INSIDE IT from which date? and last order placed, or delivered? THE CHOICE INSIDE IT all time, or a window? and do cancelled orders count? THE CHOICE INSIDE IT total or average basket? payments, or items x price? Three numbers, six decisions. Every tutorial makes them for you and most of them never say so.
🔍 Click to zoom - the right-hand column is where two analysts on the same data get two different tables
LiveWhat each letter measures, and the choice hiding inside each4 min▶

R, recency: days between the customer's last order and the snapshot date. Smaller is better, which is why R is the one dimension scored in reverse later on. Two choices: which date is "last order" (placed, approved, or delivered) and which date is "now". On Olist the placed timestamp is the honest one, because delivered dates depend on the carrier rather than on the customer.

F, frequency: the number of orders. Choices: all time or a window (the last 12 months is common, and it changes who counts as frequent), and whether cancelled, unavailable and undelivered orders count. Here only delivered orders count, which drops 2,963 of 99,441.

M, monetary: what the customer is worth in money. Choices: total spend or average basket (total rewards frequency twice, since F is already counted; average basket does not), and which column is "spend". Olist has both an items table (price plus freight per line) and a payments table (what was actually charged, including vouchers and instalments). The payments table is used here because it is the money that moved; 2,961 orders were paid with more than one payment and they are summed.

None of these is the right answer everywhere. What matters is that each is written down in the code, once, so that when a second analyst gets a different table the difference is a diff and not an argument.

Self-studyWhere RFM came from, and the date that has no source3 min▶

RFM is direct-mail technology. The version most people mean is Arthur Hughes's from Strategic Database Marketing (1994): sort the customer file by recency into fifths, sort each fifth by frequency into fifths, each of those by monetary, and you have 125 cells; then mail a random tenth of every cell, read the response rate per cell, and mail the rest of only the cells that paid. The scoring was never the point. The test mailing was, and session 5 is about the modern version of that step.

Many tutorials say RFM has been "in use since 1961". That date travels from page to page without a citation, and a search for its origin during this build found none. The direct-mail roots are real and usually dated to the 1960s catalogue business, but there is no founding paper and no single year to defend. Hughes 1994 is the citable anchor. When a fact has been copied more often than it has been checked, teach the uncertainty rather than the number.

Real world

The same shop, two tables. An analyst counts frequency over all time; the marketing platform the shop uses counts the trailing twelve months. A customer who bought monthly in 2023 and never since is "frequent" in one table and "once" in the other, and the win-back campaign built on the first table mails people the second table would call lost. Neither is wrong. Not writing the window down is what was wrong.

Part 2 · covers the snapshot date

The frozen clock

5 min live

Recency is a distance, and a distance needs two ends. If "now" is whatever day the notebook happens to run, every customer's R changes every day, the groups drift, and a segment table from March cannot be compared with one from May because they were measured from different places. The fix is one line: a snapshot date, chosen deliberately and written into the code.

A DRIFTING "TODAY" the notebook runs three times, the customer never moves, her number does last order 2018-08-20 R = 11 run on 1 Sep R = 42 run on 2 Oct R = 73 run on 2 Nov, now "quiet" A FROZEN SNAPSHOT 2018-09-01, in the code R = 11, every run and March can be compared with May THE RAGGED EDGE orders per month at the end of the file Jun Jul Aug Sep Oct 6,000 to 6,500 a month, then 20 orders in two months and none of the 20 is delivered Freeze the clock at the end of the last complete month. "The last date in the file" is 47 days of nobody buying.
🔍 Click to zoom - the top rail is why a segment table without a date on it cannot be trusted or compared
LiveChoosing the date: the end of the last complete month, not the last row3 min▶

The Olist export's last timestamp is 17 October 2018, so the obvious snapshot is df["order_purchase_timestamp"].max(). It is wrong by 47 days. Only 20 orders are dated September or October and not one of them was delivered: the export stopped at the end of August and a handful of stragglers leaked in. Measured from 17 October, every customer looks a month and a half staler than the data can support, and the whole recency column shifts.

The rule that survives: plot orders per month, find the last complete one, snapshot on the first day after it. Here that is 1 September 2018. Write it as a literal in the code, not as a computed max, so that re-running next quarter is a deliberate edit rather than a silent drift.

This is the same frozen-history discipline the retention course applies to cohort tables, and it matters here for the same reason: a segment view recomputed against a moving "today" quietly restates the past every time it runs.

Self-studyWhich id is the customer, and why the answer costs 3 percent of the rows2 min▶

Olist's orders table carries customer_id, and there are 99,441 of them for 99,441 orders: it is an order-side key, one per purchase. The customers table maps it to customer_unique_id, of which there are 96,096. Group by the first and every customer bought exactly once, by construction, and frequency is meaningless. Group by the second and 2,801 customers turn out to have bought more than once.

Every order system has this pair somewhere: a session or checkout id and a person id. Frequency is only defined against the person, and checking which one you have is the first line of any RFM build, before any date is chosen.

Part 3 · covers the eight-group taxonomy

Eight corners of one box

4 min live

The simplest RFM model splits each of the three columns into high and low, which gives every customer a three-digit code of ones and zeros and puts them in one of eight corners of a box. Each corner has a name and a first action. It is a mechanism, and it is worth being able to read a code without the poster.

EIGHT CORNERS, ONE BOX first digit R, second F, third M · 1 = above the knife on that axis R · MORE RECENT F · MORE OFTEN M · BIGGER SPEND 000 Low value quiet, once, small 100 General develop recent, once, small 010 General retain quiet, often, small 110 General potential recent, often, small 001 Key potential quiet, once, big 101 Key develop recent, once, big 011 Key win-back quiet, often, big 111 High value recent, often, big READING A CODE 011: not recently, but often and big. A proven customer going quiet. The one group a win-back message is for. R is read in reverse: fewer days is the "1". F and M read the normal way. The knife decides everything. Session 2 puts it at the mean, the median and three other places on the same customers.
🔍 Click to zoom - the far top-right corner is the only one where all three digits are 1, and on Olist it holds 1.28 percent of customers
CodeGroupWho they areFirst action
111High valueRecent, frequent, big spend. The coreProtect. Early access, a human touch at key moments, never a discount they did not need
101Key developRecent and big, but only onceA second purchase. Category cross-sell, a reason to return before the memory fades
011Key win-backFrequent and big, gone quietWin back. A relevant new arrival, not a generic coupon; find out why they left
001Key potentialOne big order, long agoReactivate cheaply. If it fails, let go
110General potentialRecent and frequent, small basketsRaise the basket. Bundles, thresholds for free shipping
100General developRecent, once, smallOnboarding. Get the second order at low cost
010General retainFrequent once, small, gone quietLight touch. Content, not spend
000Low valueQuiet, once, smallDeprioritise. Spend the budget elsewhere
A CODE IS THREE SWITCHES, EACH ONE TEST AGAINST ONE KNIFE two real Olist customers, knife at the mean · knob right = 1 · recency is read in reverse, so recent is the 1 011 Key win-back 0 R · recent? mean 240 days 372 days: no 1 F · often? mean 1.03 3 orders: yes 1 M · big? mean R$ 165 R$ 919: yes 101 Key develop 1 R · recent? mean 240 days 36 days: yes 0 F · often? mean 1.03 1 order: no 1 M · big? mean R$ 165 R$ 362: yes Read left to right and you have the code. Move a knife and a switch can flip without the customer doing anything.
🔍 Click to zoom - the second customer is in the corner that holds a third of the revenue on Olist, and she has bought exactly once
WHERE 93,358 REAL CUSTOMERS LAND IN THE BOX knife at the mean · circle area = share of customers · second line = share of revenue 000 Low value 31.9% · 16.2% of revenue 100 General develop 37.9% of people · 19.3% of revenue 010 General retain 0.42% · 0.29% of revenue 110 General potential 0.52% · 0.36% of revenue 001 Key potential 12.1% · 26.6% of revenue 101 Key develop 15.1% of people · 32.3% of revenue 011 Key win-back 0.79% · 1.89% of revenue 111 High value 1.28% of people · 3.06% of revenue HOW A CORNER FILLS Each digit is one test against one knife. 111 needs all three: recent (54.8% pass), two or more orders (3.0% pass), spend above R$ 165 (29.3% pass). The F test is the bottleneck. The two front-bottom corners hold seven in ten customers. The money sits in the two "big spend, bought once" corners at the back.
🔍 Click to zoom - the top corner everybody aims at is a dot, and the corner nobody names holds a third of the revenue
Self-studyWhat "high" turns out to mean on Olist, before session 2 makes it worse3 min▶

Put the knife at the mean of each column, the most common tutorial choice, and read what the eight groups contain on 93,358 real customers. The mean order count is 1.033, so "frequent" means two or more orders and only 3.0 percent of customers clear it. The mean spend is R$ 165 against a median of R$ 108, so the spend column is skewed right and only 29.3 percent are "big". The mean recency is 239.5 days and 54.8 percent are more recent than that.

The result: High value (111) is 1.28 percent of customers and holds 3.1 percent of revenue. The biggest group is General develop (100), recent one-time buyers with a small basket, at 37.9 percent. And the group holding the most revenue is not the corner with three ones at all: Key develop (101), recent single big baskets, is 15.1 percent of customers and 32.3 percent of revenue.

None of this is a flaw in the taxonomy. It is what a marketplace where 97 percent buy once looks like through it, and it is the reason the eight names should be read as coordinates rather than as verdicts. Session 2 moves the knife and watches every one of these shares change.

Build-along

Raw orders to eight groups, on 93,358 real customers

★ 22 min · everyone builds

Seven steps in pandas on the public Olist dataset. Every number printed below is what the code produces; if yours differs, the difference is one of the six decisions and you should be able to name which.

THE PIPELINE, SEVEN STEPS orders 99,441 customers 96,096 payments 103,886 1 · delivered only 96,478 orders 2 · join both ids customer_unique_id 3 · sum payments 4 · one row each R · F · M 93,358 customers 6 · the knife mean, tonight 7 · code and name "011" = Key win-back 8 groups, 8 counts Steps 1 to 5 are the same in every RFM build. Step 6 is where the six sessions live.
🔍 Click to zoom - the funnel drops 2,963 undelivered orders and the join is what makes frequency mean anything

Load the three tables and keep delivered orders. Two of the six decisions are made here: undelivered orders do not count, and the placed timestamp is the order date.

★ Step 1 · load and filterimport pandas as pd from datetime import datetime orders = pd.read_csv("data/olist_orders_dataset.csv", parse_dates=["order_purchase_timestamp"]) customers = pd.read_csv("data/olist_customers_dataset.csv") payments = pd.read_csv("data/olist_order_payments_dataset.csv") delivered = orders[orders["order_status"] == "delivered"] print(len(orders), len(delivered)) # 99441 96478

Attach the person id and the money. Decision three: the customer is customer_unique_id. Decision four: spend is the payments table, summed per order.

★ Step 2 · two joinsd = delivered.merge(customers[["customer_id", "customer_unique_id"]], on="customer_id") paid = payments.groupby("order_id")["payment_value"].sum().rename("amount") d = d.merge(paid, left_on="order_id", right_index=True, how="left")

Freeze the clock. Decision five, as a literal, with the reason in a comment. Everything dated after it is dropped so the snapshot and the data agree.

★ Step 3 · the snapshot# last complete month is August 2018; Sep-Oct hold 20 undelivered stragglers SNAPSHOT = datetime(2018, 9, 1) d = d[d["order_purchase_timestamp"] < SNAPSHOT]

One row per customer. Recency from the last order to the snapshot, frequency as a count of distinct orders, monetary as total spend. Decision six: total, not average.

★ Step 4 · R, F, Mrfm = d.groupby("customer_unique_id").agg( last=("order_purchase_timestamp", "max"), F=("order_id", "nunique"), M=("amount", "sum")) rfm["R"] = (SNAPSHOT - rfm["last"]).dt.days print(len(rfm)) # 93358 print((rfm["F"] == 1).mean().round(3)) # 0.970 print(rfm[["R", "F", "M"]].describe().round(1))

Look at the three columns before cutting them. One bar chart per column, as shares, and a histogram for spend. Thirty seconds of looking tells you where a knife can and cannot go: a column that is one tall bar has nowhere to cut.

★ Step 5 · the three shapesimport matplotlib.pyplot as plt print((rfm["F"].value_counts(normalize=True) * 100).round(2).head(4)) # 1 97.00 · 2 2.76 · 3 0.19 · 4 0.03 fig, ax = plt.subplots(1, 3, figsize=(12, 3.2)) (rfm["R"] // 100 * 100).value_counts(normalize=True).sort_index().plot.bar( ax=ax[0], title="R · share by 100-day band") rfm["F"].clip(upper=3).value_counts(normalize=True).sort_index().plot.bar( ax=ax[1], title="F · share by order count (3 = 3+)") rfm["M"].clip(upper=1000).plot.hist(bins=100, ax=ax[2], title="M · total spend, clipped at 1,000") ax[2].axvline(rfm["M"].mean(), linestyle="--") # the mean, R$ 165 ax[2].axvline(rfm["M"].median(), linestyle=":") # the median, R$ 108 plt.tight_layout() plt.savefig("rfm_shapes_2018-09-01.png", dpi=150) # R: 21 / 25 / 23 / 14 / 11 / 6 percent by band, a hump then a tail # F: one bar at 1 holding 97 percent # M: 46 percent under R$ 100, 32 percent at 100-200, a long tail past 1,000

The knife, at the mean tonight. Step 5 showed why: the mean is a knife the reader can see on the picture. Above the mean is 1, below is 0, and recency is reversed because fewer days is better.

★ Step 6 · high or low on each axisrfm["r"] = (rfm["R"] < rfm["R"].mean()).astype(int) # reversed rfm["f"] = (rfm["F"] > rfm["F"].mean()).astype(int) rfm["m"] = (rfm["M"] > rfm["M"].mean()).astype(int) rfm["code"] = rfm["r"].astype(str) + rfm["f"].astype(str) + rfm["m"].astype(str) print(rfm["F"].mean().round(3), rfm["f"].mean().round(3)) # 1.033 0.030 print(rfm["M"].mean().round(2), rfm["m"].mean().round(3)) # 165.20 0.293

Name the corners and count them. The mapping is a dictionary, not a chain of ifs, so it can be swapped for another taxonomy in session 5 without touching the scoring.

★ Step 7 · eight groupsGROUPS = {"111": "High value", "101": "Key develop", "011": "Key win-back", "001": "Key potential", "110": "General potential", "100": "General develop", "010": "General retain", "000": "Low value"} rfm["group"] = rfm["code"].map(GROUPS) out = rfm.groupby("group").agg(customers=("F", "size"), revenue=("M", "sum")) out["customers_pct"] = (out["customers"] / len(rfm) * 100).round(2) out["revenue_pct"] = (out["revenue"] / rfm["M"].sum() * 100).round(2) print(out.sort_values("customers", ascending=False)) # General develop 37.88 / 19.28 · Low value 31.91 / 16.16 · Key develop 15.10 / 32.33 # Key potential 12.11 / 26.63 · High value 1.28 / 3.06 · Key win-back 0.79 / 1.89 ...

What step 5 draws, on the real 93,358. Three columns, three completely different shapes, and the shapes decide where a knife can go.

THREE COLUMNS, THREE SHAPES share of 93,358 customers in each bar · dashed = mean · dotted = median R · days since last order a hump at 100 to 300 days, then a tail 21 25 23 14 11 6 0 300 600 days mean 240 and median 220 sit together any knife near the middle splits it in half F · delivered orders one bar, and it is the whole column 97 2.8 0.2 1 order 2 3 or more mean 1.03: "high" means 2 or more, 3 percent fifths cannot be drawn here at all M · total spend, R$ nearly half under 100, a long tail past 1,000 46 32 10 4.5 0 300 1,000+ median 108, mean dragged to 165 by the tail the two knives give 50 and 29 percent "high" Same three columns every RFM build starts from. Look first: the shape tells you which knives are honest and which cannot be drawn.
🔍 Click to zoom - the middle panel is the whole reason session 2 exists: 97 percent of a column is one value

Save the table with the date in the filename. rfm_2018-09-01.csv, not rfm.csv. Session 6 compares two of these against each other, and the comparison is only meaningful because each one says when it was taken.

Homework

Before session 2

30 min
  1. Run the six steps on your own order table, or on Olist if you do not have one. Write the six decisions as comments at the top of the script before you write any code below them.
  2. Move the snapshot forward one month and re-run. Count how many customers change group. That number is the cost of an undated segment table.
  3. Swap total spend for average basket in step 4 and re-run step 6. Watch Key develop and Key potential trade places with General groups. Neither is wrong; write down which question each answers.
  4. Bring the table. Session 2 moves the knife five times on it, and session 3 puts a sample of it on a bench where every share is counted live.
Check yourself

Three questions

Clear all three and session 1 gets stamped in your passport.

1 · The Olist export's last order is dated 17 October 2018. Why is 1 September 2018 the better snapshot?

The snapshot goes at the end of the last complete month, found by plotting orders per month, and it is written as a literal so that re-running later is a deliberate edit rather than a silent drift. A is a convention that happens to coincide; the reason is the ragged edge.

2 · Grouping Olist by customer_id gives 99,441 customers who each bought exactly once. What has gone wrong?

Every order system carries a checkout id and a person id. Frequency is only defined against the person, and checking which id you have is the first line of the build, before any date or knife is chosen.

3 · Which corner of the eight holds the most revenue on Olist with the knife at the mean, and why is that not a paradox?

The names are coordinates, not verdicts. High value means all three digits above the knife; on a one-time-buyer marketplace that is a small corner. Where the revenue actually sits depends on the base, and the taxonomy is a way of seeing it, not a promise about it.

Sources covered

What this session teaches, and where it came from

✓Recency, frequency, monetary and their definitionsStandard practice, with the six decisions named explicitly; every Olist figure recomputed from the public dataset in this build
✓The eight-group 2x2x2 taxonomy and its action tableTaught as the mechanism it is; the group names are the common English renderings of the taxonomy used across Chinese growth practice and vendor tooling
◐Hughes 1994, the quintile cells and the test mailingDescribed from secondary summaries of Strategic Database Marketing; the book itself was not opened in this build, and the page says so
◐"RFM since 1961"Not claimed. The date circulates without a citation and none was found; Hughes 1994 is the citable anchor
✓The Olist dataset and its ragged edgeBrazilian E-Commerce Public Dataset by Olist, Kaggle, CC BY-NC-SA 4.0; monthly counts, id cardinalities and the 20 undelivered stragglers all measured here

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 1 cheat sheet · pin this

R · F · MDays since last order, count of orders, total spend. Each hides two decisions.
Six decisions, written downPerson id, which orders count, which column is spend, total or average, window, snapshot date.
Snapshot = end of last complete monthNever the last row in the file. Plot orders per month first.
Recency is reversedFewer days is the 1. F and M read the normal way.
Eight corners, three digits111 High value · 011 Key win-back · 101 Key develop · 000 Low value.
Names are coordinatesOn Olist, 101 holds a third of the revenue and 111 is 1.28 percent of customers.