Where we start
Five sessions built one table: three columns on 93,358 real customers, a knife whose position decides who is called best, eight named corners with an action each, and a holdout behind every action. Every one of those numbers was measured from 1 September 2018. That date is the table's expiry stamp. Most RFM projects die not because the model was wrong but because it was built once, presented once, and then quietly re-run against a moving "today" by someone who had never read the original decisions. The last session is about the part after the build: the second photograph, the drift between the two, the page that makes the table reproducible, and the question of when RFM should stop being the model.
★ What you walk out with tonight. Two dated RFM tables and the code that compares their knives, their group sizes and their movers; a ten-field spec written as a JSON file a colleague could rebuild the table from; a check that fails loudly if the snapshot ever stops being a literal; and an eight-row gate that says whether the table is allowed to leave the notebook.
Two photographs, three months apart
5 min liveRecompute on a declared cadence: monthly for a marketplace, weekly for a live-commerce room where a customer's whole relationship can run in a fortnight, quarterly for a small shop whose customers buy a few times a year. Each run is saved with its snapshot date in the filename, rfm_2018-09-01.csv, never rfm.csv. Between any two dated snapshots three things drift, and the one everyone forgets is the knife itself.
LiveThree things drift, and the knife is the one people forget3 min▶
First, the knife. With the cut at the mean, "high spend" means above the mean of M in that snapshot. New customers arrive, a few large baskets land, and the mean moves. A customer who spent R$ 160 has not changed, but in one snapshot she is below the line and in the next she is above it. The groups look different and nobody bought anything. The same happens to the frequency mean and to the recency mean, which drifts on its own as the whole base ages against a later snapshot date.
Second, the group sizes. Even with the knife frozen, the shares move because the base moves: a marketplace that acquires 6,000 customers a month adds them almost entirely to General develop and Low value, so the Key groups shrink as a share without losing a single member.
Third, individual customers moving corner to corner. This is the drift that pays for the cadence. The migration crosstab between two snapshots, one row per June group and one column per September group, is built step by step in the retention course's analyst session 4 and is described here in one paragraph rather than rebuilt: join the two tables on the person id, cross-tabulate the old group against the new, and read the off-diagonal cells. The cell that matters most is Key win-back to Low value, because it is the customer who was frequent and big, went quiet, and was not reached before "quiet" became "gone".
Self-studyFix the knife or let it move: recomputing the mean each snapshot versus freezing the threshold from the first3 min▶
There are two defensible policies and they answer different questions. A frozen threshold, the mean of M from the first snapshot written into the spec as a number, keeps the groups comparable across time: a customer only changes group when she changes. That is what you want when the question is "are we moving people up", because then the movement is the signal. A moving knife, the mean recomputed each run, keeps the groups honest to the current base: "high" means high among the customers you have now. That is what you want when the question is "who are the top people to call this month", because a threshold frozen in a year when baskets were smaller will eventually call half the base "high".
The trade-off is real and neither side wins it. What is not defensible is not knowing which one you are running. A team that recomputes the mean each month and then reports "Key win-back grew 20 percent" is reporting knife movement as customer movement. Write the policy into field 6 of the spec, and if it is the moving knife, print the knife's value in every run so a reader can see how much of the change is the line moving.
Why the Olist knife is so easy to move. On the 93,358 customers the mean spend is R$ 165.20 and the median is R$ 107.78. When the mean sits that far above the median, the spend column is dragged right by a few large baskets, and only 29.3 percent of customers clear the mean line. One quarter with a handful of unusually large orders lifts the mean for everyone, and a customer who has not bought since spring slips from Key potential to Low value without doing anything. A median knife would move far less between snapshots, at the cost of calling half the base "high" by construction. Whichever you choose, the reason it moved has to be readable in the run log, not reconstructed in a meeting.
Ten fields on one page
5 min liveSession 1 opened with six decisions that every RFM build makes and most tutorials make silently. Sessions 2 to 5 added four more. Written down, the ten fit on one page, and that page is the difference between a table someone can rebuild and a table that lives in one analyst's notebook until she leaves. If any field is missing, two analysts will disagree about the table and both will be right.
LiveTen fields, one page, no adjectives3 min▶
Each field is a literal value, not a description. 1, person id: the column name, customer_unique_id, and one line on why it is not customer_id. 2, which orders count: order_status == "delivered". 3, spend: the column, payment_value summed per order, and the word total or average. 4, window: all time, or the trailing number of months. 5, snapshot date and the rule that chose it: 2018-09-01, end of the last complete month, found by plotting orders per month. 6, the cut and why: mean split, because the tutorial the team learned from used it, or median split because the spend column is skewed, or Klaviyo's thresholds because the tool decides. The why is what a successor needs when she wants to change it. 7, the taxonomy and the action per group: the eight names from session 1 and the action table from session 5, as a table, not a paragraph. 8, holdout share: the fraction of every audience that is not messaged, so the action can be proved. 9, cadence: monthly, weekly or quarterly, and the day of the run. 10, owner: a name.
Adjectives are the failure mode. "Recent customers with meaningful spend" is a sentence two analysts read two ways. "R below 239.5 days, M above R$ 165.20" is a sentence that produces one table.
Self-studyThe spec as a data contract: a change to any field arrives as a diff to review, not as a surprise in the segment counts2 min▶
Once the spec is a file, it can be put under version control next to the code, and then a change to the model is a change to the file. Someone moves the knife from the mean to the median: that is a one-line diff in field 6, and the reviewer asks "why" before it runs, instead of asking "why did Key win-back double" after the segment counts land in a dashboard. Someone shortens the window to twelve months: a diff in field 4, and the reviewer knows to expect the frequency column to fall. The eight-group taxonomy from session 1 lives in field 7 as a dictionary, so swapping it for Klaviyo's six or Putler's eleven is also a diff, and the reviewer can pull up the finding from sessions 2 and 3 that under those two maps nine in ten "Champions" on a one-time-buyer base bought exactly once.
This is the same discipline any shared dataset gets from a schema: the consumers of the table are protected from silent change because the definition is a reviewable artefact. RFM tables rarely get it because they start life in a notebook, and the spec is how they leave it.
When the tool is the spec. A shop that runs its segments inside Klaviyo has field 6 filled in by the vendor: recency scored 3 within 180 days and 2 within 365, frequency and monetary by thirds, six groups with published code lists, as the help centre states them. That is a legitimate spec, and it is a good one to copy into the page precisely because it is a vendor default: the day the shop exports its customers to a warehouse and an analyst rebuilds the segments in pandas, she needs to know that "Champions" meant those thresholds and not the mean split. Without the page, the two systems disagree, the marketing team trusts the one with the prettier dashboard, and both were right.
Three questions RFM cannot answer, and one it usually can
5 min liveRFM answers "who, right now, by three coordinates". It is a description of the present. Three questions sound similar and are not: what is this customer worth over the next year, which customers are about to leave, and are there natural groups I have not drawn. Each has a model built for it, each takes the RFM table as an input, and none of them should be reached for until the question is actually being asked.
LiveThree questions RFM cannot answer, and one it usually can3 min▶
"What is this customer worth over the next year?" is a forecast, and RFM has no time axis in its output. The buy-till-you-die family, BG/NBD for how many purchases and Gamma-Gamma for how much each one is worth, answers it, and it takes recency, frequency and tenure per customer at the snapshot as its inputs. It is built in the retention course's analyst session 6.
"Which customers are about to leave?" is a probability per person, and it needs a label: a definition of "left" that can be checked on past customers. RFM's "quiet" corner is a heuristic for this, not a model of it. The churn model in the retention course's analyst session 7 is the tool, and R, F and M are usually three of its strongest features.
"Are there natural groups I have not drawn?" is the one question the eight corners cannot answer by construction, because the corners were drawn first. Clustering lets the data propose the groups, and the Unsupervised Learning course uses the three-number RFM portrait as exactly those features.
And the one it usually can: "who do I message this month, and with what". For a shop with one channel and a monthly send, three digits and eight actions are the whole model, and reaching for anything heavier is a way of avoiding the holdout question from session 5. Say so plainly in field 6 of the spec: "RFM is enough, revisit when the question changes".
Self-studyWhat each handoff needs from the RFM table2 min▶
Lifetime value wants three columns per customer measured at the snapshot: recency (in the buy-till-you-die convention, the age of the customer at her last purchase), frequency as repeat purchases, and tenure, the time from first purchase to the snapshot. Tenure is not in the standard RFM table, so the handoff means adding one aggregation, first=("order_purchase_timestamp", "min"), to the session 1 pipeline. Everything else transfers as is, including the frozen snapshot, which the model needs for exactly the same reason RFM does.
Clustering wants the three raw columns, not the scores. The 0 and 1 digits are the answer to a question already asked; feeding them to k-means asks the algorithm to rediscover the corners you drew. Feed it R, F and M scaled, usually log-transformed first because M and F are heavily right-skewed on any retail base, and let it propose groups the taxonomy has no name for. Then compare its clusters against the eight corners with the same crosstab from Part 1.
A churn model wants a label before it wants features. The label is the part RFM cannot provide: "did not buy within N days after the snapshot" is one honest definition, and it requires a later snapshot to exist. The dated files from Part 1 are therefore also the training data for the handoff, which is another reason to save every one of them.
A lifetime-value model on a base that mostly bought once. On Olist 97.0 percent of the 93,358 customers have exactly one delivered order, and the 2,801 repeat buyers hold 5.6 percent of revenue. A buy-till-you-die model learns the repeat rhythm of a base from the customers who have one, and here that is 2,801 people out of 93,358. The model will fit; its forecast for the other 90,557 will lean almost entirely on the population prior, because each of them has contributed one event. That is not a reason never to build it. It is the reason the handoff decision should be made after looking at the frequency column, and why "RFM is enough" is the honest exit for most marketplaces with this shape.
Eight checks before the table leaves the notebook
4 min liveSessions 1 to 3 asked halfway questions about definitions. This is the ship gate: eight things that are either true of the table in front of you or not, and a table that fails any one of them is a table someone will act on for the wrong reason. Score the table you have, not the one you intend to build.
The empty rooms, counted. Row seven exists because of a specific replication. Putler's eleven-segment map, run on the 93,358 Olist customers with rank quintiles, produced 7 non-empty segments: New, Promising, Can't lose and Lost held nobody, because averaging the frequency and monetary quintiles never reaches the corner values when 97 percent of frequency is a single tied value. An action table copied from the vendor's page would have assigned a message, a budget line and a KPI to four rooms with no one in them. The check takes one line, value_counts() on the group column, and it belongs in the gate because the taxonomy's poster never shows the counts.
Two snapshots and the one-page spec
★ 22 min · everyone buildsSeven steps in pandas. The September table is the one from session 1. The June table is session 1's homework: the same six steps with SNAPSHOT = datetime(2018, 6, 1), saved as rfm_2018-06-01.csv. Only the September figures below are quoted as measured; wherever a line depends on your June table, the comment says so, and your numbers will differ from the person next to you if any of the ten fields differs.
Load the two dated tables. The September file is session 1's output. The June file is your homework from session 1, the same pipeline with the snapshot moved back three months. If you do not have it yet, build it now; it takes the six steps and one changed literal.
Compare the knives. With a mean split, the line between "high spend" and "low spend" is the mean of M in each table. Print both. If they differ, some of the group movement you are about to see is the line moving, not the customers.
Group sizes side by side. Shares, not counts, because the September base is larger. Read the Key groups first; on a marketplace that acquires fast, they shrink as a share even when nobody leaves them.
Join on the person and cross-tabulate. One line. Customers who did not exist in June appear as "new". How to read the off-diagonal cells, and the persona version of the same table, is taught in the retention course's analyst session 4; here the one cell to look at is the June Key win-back row.
The ten-field spec as a dictionary. Literal values only. Field 6 carries the policy from Part 1 as well as the cut. Field 10 is a name, not a team.
A check that fails loudly. Two assertions the gate cares about most: the snapshot in the script is a literal, not a computed maximum, and the person id named in the spec is actually a column in the table. Run it before every save.
The dated save. The table and the crosstab both carry the snapshot in their names. Nothing is overwritten; the stack at the bottom of the figure grows by one every cadence.
Put the check in the same file as the build. A check that lives in a separate notebook is a check nobody runs. check_spec goes at the bottom of rfm_build.py, before the save, so the file refuses to write a table whose snapshot has quietly become .max().
After the last session
45 min- Write the ten-field spec for your own table, as a JSON file next to the code, with literal values in every field. If a field makes you hesitate, that hesitation is the decision the table has been making silently.
- Schedule the next snapshot. A date in the calendar, the cadence from field 9, and the reminder addressed to the owner in field 10. Then run the check from step 6 against the current script and fix whatever it refuses.
- Decide the handoff question honestly. Look at your frequency column first. If most of your base bought once, write "RFM is enough, revisit when the question changes" in the spec and mean it. If you have real repeat behaviour and the question is money next year, the retention course's analyst session 6 is the next build.
- Send the spec to one colleague and ask them to rebuild the table from it alone. No notebook, no call, just the page. If their group counts match yours, the spec is finished. If they do not, the difference is a missing field, and finding it is the whole point.
Three questions
Clear all three and session 6 gets stamped in your passport.
1 · Between the June and September snapshots, the share of customers in Key win-back changed. Why can that number not be read as "customers moved" without more information?
Three things drift between snapshots: the knife, the group sizes and the individuals. The knife is the one people forget, and on Olist it is especially mobile because the mean spend of R$ 165.20 sits well above the median of R$ 107.78, so a few large baskets move the line for everyone. Neither policy is wrong; not knowing which one you run is.
2 · A shop with one channel and a monthly send asks "which customers are about to leave?" and, separately, "who do I message this month?" Which model answers which?
RFM answers "who, right now, by three coordinates". Money next year is lifetime value, leaving is a churn model, undrawn groups is clustering, and each takes the RFM table as an input. For most shops with a monthly send the honest exit from the decision tree is the tangerine box, and on a 97 percent one-time-buyer base a lifetime-value model has 2,801 repeat buyers' worth of rhythm to learn from.
3 · Two analysts rebuild the RFM table from the same spec and get different group counts. The spec has no window field. Which of them made the mistake?
A missing field is a decision the table makes silently, and two people will make it two ways. The fix is never a meeting; it is a tenth line on the page. Once the spec is a file under version control, the window becomes a diff someone reviews before the run, not a surprise someone explains after it.
What this session teaches, and where it came from
Every fact on this page, and its verification tier, is recorded in the course's source map.
Session 6 cheat sheet · pin this
You have a table that knows its date, a spec that lets someone else rebuild it, and a gate it has to clear every cadence. When the question changes from "who, right now" to "how much, next year", the retention course's analyst session 6 builds lifetime value on the same three columns plus tenure, and its analyst session 7 turns "quiet" into a probability. When the question is whether the eight corners are the right shapes at all, Unsupervised Learning lets the data draw its own groups from the raw R, F and M. And the whole shelf, with every course that shares a dataset with this one, is at the Learn with Phoebe hub.