learn-rfm-modeling-with-phoebe / Session 5 of 6
Learn RFM Modeling with Phoebe · Session 5 of 6

From segment to action, and proving it

A segment is only worth the action it drives, and an action is only worth what a holdout says it added. "Message everyone in win-back" is the most expensive sentence in RFM, because which people are in win-back was decided by the knife in session 2, and on the bench in session 3 that audience ran from 33 people to 2,555 on the same customers. Tonight we give each of the eight corners one action, one channel and a frequency cap, read a vendor's cut before trusting its segment names, hold out a tenth of the audience so the lift can be measured, and write the campaign spec before a single message goes out.

🟡 Core CRM, growth and lifecycle teams Python, pandas 45 min
0-3 · Welcome 3-18 · One action per corner, and the holdout 18-40 · Build-along: the campaign spec with a holdout 40-45 · Q&A
Part 0

Where we start

Four sessions in, you have a table with a three-digit code on every customer and you know that the code depends on where the knife went. The temptation now is to export the win-back group to the email tool and press send. Every step of that is a decision with a price: which action fits the corner, through which channel, how often, to how many, and how you will know afterwards whether it did anything. The last question is the one most RFM projects never ask, and it is the one that decides whether the segmentation was worth building.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class ★ Build-along = run the code yourself

★ What you walk out with tonight. An action table that reads each corner's definition backwards into a first move, a channel and a cap; the habit of opening a vendor's docs to find its cut before funding anything aimed at its "Champions"; a seeded holdout on your own win-back audience; and a one-page campaign spec with the measurement written into it before the send, so that the result is a lift and not a story.

Part 1 · covers the action table

One action per corner, and a cap on each

5 min live

Each of the eight codes is a sentence about a customer: recent or not, once or often, big or small. Read the sentence backwards and it tells you the first move. A customer who is frequent and big but quiet does not need a discount, she needs a reason and a question. A customer who is recent, once and small needs the second order and nothing expensive. Give every corner exactly one action, one channel, and a frequency cap, and write the three down together.

ONE ACTION PER CORNER the action, then the channel and a frequency cap, read off the code 111 High value Protect early access, a human touch, no unneeded discount 101 Key develop Get the second order email cross-sell · cap 2 a month 011 Key win-back Win back, and ask why they left email, a relevant new arrival · 1 send, then stop 001 Key potential Reactivate cheaply, then let go one email · cap 1 110 General potential Raise the basket at checkout: bundles, a shipping threshold 100 General develop Onboard to order two email series · cap 3 in 30 days 010 General retain Light touch, content not spend newsletter · cap 1 a month 000 Low value Deprioritise no contact · cap 0 a human call or a note email or DM at the checkout no contact The caps are illustrative and yours to set. What is not optional is that every corner has one, beside its action.
🔍 Click to zoom - most corners route to the letter, which is why the cap column matters more than the channel column
LiveThe action is the segment's definition read backwards3 min▶

Start from what the code says is missing. 111 High value is missing nothing, so the action is to protect: early access, a human touch at the moments that matter, and never a discount they did not need, because a discount to someone who was going to buy anyway is margin given away. 101 Key develop is recent and big but only once, so the whole job is the second order: a category cross-sell, a reason to return before the memory of the first order fades. 011 Key win-back is frequent and big and gone quiet, so win back with a relevant new arrival rather than a blanket coupon, and ask why they left, because the answer is worth more than the order.

001 Key potential, one big order long ago, gets one cheap reactivation attempt and then release; the corner is 12.1 percent of Olist customers and 26.6 percent of revenue, which is exactly why a blanket effort is tempting and exactly why it must be capped. 110 General potential, recent and frequent with small baskets, wants the basket raised at the checkout with bundles and a free-shipping threshold, not a message. 100 General develop, the biggest corner on Olist at 37.9 percent, is onboarding to the second order at low unit cost. 010 General retain gets content and no spend. 000 Low value is deprioritised, and "deprioritised" means the cap is zero, written down, so nobody adds them to a blast later.

Then extend every row with two more columns: the channel and the frequency cap. The channel follows the action (a human touch is a call or a handwritten note, onboarding is a short series, basket-raising is on-site). The cap is the promise you make to the customer's attention, and it is the column that turns an action table into an operating rule, because without it every corner drifts toward the email tool.

Self-studyPutler's action column, quoted3 min▶

The eleven-group map that most tutorials copy comes with its own action column, and it is worth reading beside the eight-corner table above because it says the same things at a finer grain. Quoted from putler.com/rfm-analysis, in short form:

Putler segmentSuggested action
ChampionsReward them; they can be early adopters of new products
Loyal customersUpsell higher-value products; ask for reviews
Potential loyalistOffer membership or a loyalty programme
New customersOnboarding support, early success
PromisingBrand awareness, free trials
Needs attentionLimited-time offers based on past purchases
About to sleepShare valuable resources, popular products
At riskPersonalised reconnect, offer renewals
Can't lose themWin back via renewals or newer products; talk to them
HibernatingOther relevant products and special discounts
LostRevive with a reach-out campaign, or ignore

Notice what is missing from both tables: a channel and a cap. Vendor action columns tell you what to say and never how often, because the vendor is paid per message. Add the two columns yourself.

Real world

A Douyin brand room and its old customers. Live commerce on Douyin is now mostly brands broadcasting from their own rooms: the 2025 white paper from 索象集团 (Suoxiang Group) and 中国直播产业研究院 (China Live Streaming Industry Research Institute) puts 2024 Douyin GMV above 3 trillion yuan, live streaming above 58 percent of sales, and 店播 (brand self-broadcast) at 41 percent against 达播 (creator-hosted) at 38 percent; CBNData's 2025 read puts 店播 (brand rooms) at about 70 percent of live revenue. An operator running a brand room therefore owns a returning audience and can act on it: a returning viewer who bought twice and has gone quiet is a win-back corner, and the room's own schedule is the channel. What no primary source publishes is the repeat-purchase rate of any of these rooms, so any 复购率 (repeat-purchase rate) figure you meet in a deck is a vendor or press claim and should be labelled as one. The mechanism transfers; the rate must be measured in your own room.

Part 2 · covers the tool decides your cut

Read the cut before the name

4 min live

If the segment came out of a marketing platform, somebody else chose the knife, and the name on the group tells you nothing about who is in it until you have read their rule. The rules are usually published. Klaviyo's help centre states its recency thresholds and its use of thirds; Putler publishes its score ranges. Read them, then check one thing on your own data before spending a cent on the top group: how many of its members bought exactly once.

THE TOOL'S DIAL Klaviyo's published recency thresholds, and what its top group holds on Olist 3 · under 180 days 2 · under 365 days 1 · older than that F and M: thirds by rank six groups from listed code sets read this before the name KLAVIYO'S SIX, ON OLIST 13.9% of customers called Champions 91.0% of those Champions bought exactly once PUTLER'S ELEVEN, ON OLIST 16.1% of customers called Champions 92.1% of those Champions bought exactly once 93,358 customers, ties broken with rank(method="first"), the convention every tutorial fix uses. A property of that convention on this base, not a claim about either product's live output.
🔍 Click to zoom - two published rules, two Champions groups of 13.9 and 16.1 percent, and nine in ten of both bought once
LiveRead the cut before the name3 min▶

Klaviyo's published rule (help centre article 17797937793179): recency scores 3 for a last order within 180 days, 2 within 365 days, otherwise 1; frequency and monetary are split into thirds; six named groups are assembled from listed sets of three-digit codes. Putler's rule (putler.com/rfm-analysis): quintile scores 1 to 5 on each of R, F and M, frequency and monetary averaged, and eleven segments defined by published score ranges.

Run both on Olist's 93,358 customers and the top groups are large. Klaviyo's map calls 13.9 percent of customers Champions, and 91.0 percent of those Champions bought exactly once. Putler's calls 16.1 percent Champions, of whom 92.1 percent bought once. Neither rule is broken. On a base where 97 percent of customers bought once, a frequency column split into thirds or fifths has to put most of the identical one-time buyers somewhere, and the pandas rank(method="first") convention that every tutorial uses to get past the duplicate-edge error assigns them by row order. One replication note applies to both: neither product documents its tie handling for a column that is 97 percent ties, so these are properties of that convention on this base, not claims about either product's live output.

The professional norm that follows: before funding anything aimed at a tool's top group, compute the share of that group who bought exactly once on your own data. If it is nine in ten, the group is not "your best customers", it is "recent single big baskets with a row-order score", and the right action is Key develop's second-order push rather than High value's protect. The name on the group did not tell you that. The cut did.

Self-studyWhat the six-group and eleven-group maps agree on: recency dominates2 min▶

Put the two vendor maps side by side and one structural agreement stands out: the recency score does most of the sorting. Klaviyo scores recency on calendar thresholds (180 and 365 days) and only then splits the other two columns into thirds, so a customer's recency band decides which third of the map she can land in at all. Putler averages F and M into a single second axis and reads the map as recency against that average, so recency is one of only two axes and the one not diluted by averaging. On Olist the effect is visible in the population shares: Klaviyo's "Recent" group is 21.6 percent, Putler's "Loyal" 31.6 percent and "Potential loyalist" 24.3 percent, all bands of recent buyers with modest F and M.

That is a defensible design choice for a message-sending tool, because recency is the one column that predicts whether a message will be opened. It is a poor design choice for deciding who is worth a discount, because a recent one-time buyer and a recent ten-time buyer land in the same band under both maps. The eight-corner taxonomy keeps frequency as its own digit for exactly this reason, and it is why the 101 and 111 corners get different actions even though most tools would put them in one group.

Part 3 · covers the holdout

The lift is a difference, never a rate

6 min live

Some of the people you message would have bought again anyway. A report that counts every reorder in the messaged group as the campaign's revenue is counting those people twice: once as customers, once as a result. The only way to know how many is to hold some of the audience back, message the rest, and measure both groups over the same window. Hughes's test mailing in 1994 was this step. Holding out a random tenth is the modern version.

THE HOLDOUT same audience, same window, one group never hears from you treated · nine in ten get the message held out · one in ten never messaged, same window REORDER RATE IN THE WINDOW illustrative heights, your numbers will differ THE LIFT, the only number that pays for the message rate(treated) minus rate(held out) treated held out the organic rate Attributed revenue counts every treated reorder. The held-out group says how many would have happened anyway.
🔍 Click to zoom - the hatched bar is what happens with no message, and everything below the tangerine gap was going to happen regardless
LiveThe lift is a difference, never a rate4 min▶

The procedure has four lines. One: take the audience the segment defines and mark a random tenth held out, with a seed, before anyone sees the list. Two: message the other nine tenths and never message the tenth, for the whole window. Three: after the window (say 90 days from the send), compute the reorder rate in each group. Four: the lift is the treated rate minus the held-out rate, and incremental orders are the lift times the treated count.

What the held-out rate exposes is the organic reorder: the share of this audience that buys again with no message at all. Every one of those, in the treated group, will show up in the platform's "attributed revenue" because the platform attributes any purchase after a send to the send. Revenue that would have happened anyway is the organic rate times the treated count, and it is not the campaign's. Vendor content occasionally concedes the point; one vendor-side blog puts the organic share of win-back revenue attributed under Klaviyo's methodology at 15 to 22 percent. That is a vendor or press claim, not checked at source here, and the honest reading is not the number but the admission: the attribution model itself says some of what it attributes was not caused.

The same sheet must carry contact cost and the frequency cap from Part 1. Net incremental is incremental revenue minus the cost of every message sent, including the ones to people who would have bought anyway, and the cap is what stops the cost line from being paid twice a week. A lift that does not clear the cost is a campaign that lost money while reporting a positive attributed number, and that is a common outcome, not a rare one.

Self-studyHow big a holdout: a tenth is a convention, the real answer is the smallest group that can still show a difference2 min▶

Ten percent is a habit inherited from the direct-mail test cell, and it is a reasonable habit on a large audience. It is the wrong habit on a small one. On the session 3 bench, the mean-split win-back corner is 33 people; a tenth of that is 3, and three people cannot show a difference in reorder rate against thirty. The right question is: what is the smallest held-out group that could still show the lift I am hoping for, at the reorder rates I expect? That is a sample-size calculation, and it depends on the organic rate, the lift you would act on, and how sure you need to be.

Two consequences for RFM practice. First, the holdout share should be decided per audience, and for a small corner it may need to be a third or a half, or the corner may need to be pooled with a neighbour before it can be tested at all. Second, an audience too small to test is an audience too small to campaign on with any confidence about the result; the honest move is to act on it anyway if the action is cheap, and to say plainly that the outcome will be an anecdote. The maths of the calculation belongs to Experimentation, which owns power, minimum detectable effect and the sequencing of tests.

Real world

"私域复购是公域的3-5倍 (private-domain repurchase is 3 to 5 times public-domain)." The claim that private-domain (WeChat community, mini-programme, owned-group) repurchase runs three to five times the public-domain rate is everywhere in Chinese growth writing, and its most recent home is a 2026 article on the 腾讯云开发者社区 (Tencent Cloud developer community). It is an industry claim, it names no dataset, and it is unverified. It is also a claim a single shop can settle for itself in one quarter: take the shop's lapsed repeat buyers, message half through the community and hold the other half out, and read the two reorder rates. Whatever the multiple turns out to be for that shop is worth more than any multiple in an article, and the operator will have it before the next quarter's budget is set.

Part 4 · covers the trap

Message everyone in win-back

3 min live

The sentence sounds like a plan. It is a plan whose cost was set in session 2, when the knife decided how many people "win-back" contains. On the bench in session 3, the same 4,000 real customers give a win-back audience of 33, 50, 767 or 2,555 depending on the cut, and under the stated response model those audiences pay back very differently per message and in total. Read both columns, because they do not agree on the winner.

TWO COLUMNS, TWO ORDERINGS the win-back audience under four cuts, 4,000 real customers, stated response model NET YIELD PER MESSAGE NET TOTAL Mean split · 33 people 100% repeat buyers Median split · 50 people 100% repeat buyers One total score · 2,555 2.5% repeat buyers Putler's eleven · 767 2.4% repeat buyers R$ 3.46 · 1st R$ 2.84 · 2nd R$ 0.02 · tied 3rd R$ 0.02 · tied 3rd R$ 103 · 2nd R$ 128 · 1st R$ 53 · 3rd R$ 13 · 4th Yields on the stated model: 1.0% and 4.0% organic reorder, lifts of 15% and 50%, R$ 0.30 a message, a tenth held out. The ordering transfers because it follows the measured repeat-buyer share; the reais do not.
🔍 Click to zoom - the two tangerine bars are on different rows, and that swap is the whole lesson of the honest dip
LiveYield per message and total pull in different directions3 min▶

From the bench in session 3, on the stated response model (lapsed one-time buyers reorder on their own 1.0 percent of the time and a message lifts that by 15 percent; lapsed repeat buyers reorder 4.0 percent of the time and a message lifts that by 50 percent; a message costs R$ 0.30; a tenth of every audience is held out):

CutWin-back audienceRepeat buyers in itNet yield per messageNet total
Mean split, eight (code 011)33100%R$ 3.46R$ 103
Median split, eight (code 011)50100%R$ 2.84R$ 128
Quintiles into Putler's eleven (At risk + Can't lose)7672.4%R$ 0.02R$ 13
One total score (the middle band)2,5552.5%R$ 0.02R$ 53

Two columns, two orderings. Per message, the mean split wins because every one of its 33 people is a proven repeat buyer. In total, the median split wins because it has 50 such people rather than 33, and the extra 17 are worth more than the small drop in yield each. Neither column is "the" answer. The per-message column is what a finance partner reads (what does each contact return?), the total column is what a growth lead reads (what does this campaign return?), and a spec that shows only one of them is hiding a decision.

What transfers and what does not. The reais are on a model and will not be your numbers. The ordering is not on the model: it follows from the measured repeat-buyer share of each audience, 100 percent under the two-way splits against 2.4 and 2.5 percent under the quintile and sum cuts, and that share is a fact about who the knife put in the group. Any response model in which repeat buyers reorder more readily than one-time buyers will reproduce the ordering. That is why "message everyone in win-back" costs what it costs: under a quintile or sum cut, "everyone" is 767 or 2,555 people of whom nineteen in twenty have never bought twice.

Build-along

The campaign spec with a holdout, on your rfm table

★ 22 min · everyone builds

Six steps in pandas on the rfm table from session 1, with its three-digit code column from the mean split. The campaign has not run, so nothing below prints a result; the measurement template carries placeholders and is left ready. Where a number is printed, it is from the Olist population and your table will differ.

THE SPEC SHEET, WRITTEN BEFORE THE SEND CAMPAIGN SPEC · win-back 011 · one page 1 · Segment 011 Key win-back 2 · Snapshot date 2018-09-01, in the code 3 · Cut mean split, eight groups 4 · Holdout share, seed 10% · rng(20180901) 5 · Audience size len(audience), from your table 6 · Metric and window reorder within 90 days of send 7 · Cost per message R$ 0.30 (the model; use yours) action: new arrival email, ask why they left, 1 send after 90 days RESULT TEMPLATE · blank today rate, treated rate, held out lift = difference incremental orders incremental revenue minus cost filled once, from both groups Write the spec before the send. A spec written after the result is a story, not a measurement.
🔍 Click to zoom - the tangerine row is the one that cannot be added afterwards, because once the list has gone out there is nobody left to hold back

The action table as a dictionary. Code to (action, channel, cap), joined onto the table so every customer carries her first move. Caps are yours to set; these are placeholders.

★ Step 1 · ACTIONSimport pandas as pd import numpy as np ACTIONS = { "111": ("protect, no unneeded discount", "human note", "1 per quarter"), "101": ("get the second order", "email", "2 per month"), "011": ("win back, ask why they left", "email", "1 send, then stop"), "001": ("reactivate cheaply, then let go", "email", "1 send"), "110": ("raise the basket", "checkout", "on-site only"), "100": ("onboard to order two", "email", "3 in 30 days"), "010": ("light touch, content not spend", "newsletter", "1 per month"), "000": ("deprioritise", "none", "0"), } acts = pd.DataFrame.from_dict(ACTIONS, orient="index", columns=["action", "channel", "cap"]) rfm = rfm.join(acts, on="code") print(rfm[["code", "action", "channel", "cap"]].drop_duplicates().sort_values("code"))

Pick the audience. One corner, by its code, from the table as of the snapshot. On Olist at the mean split this is 0.79 percent of 93,358 customers; yours will differ.

★ Step 2 · the audienceaudience = rfm[rfm["code"] == "011"].copy() print(len(audience)) # 0.79% of customers on Olist at the mean split; your table differs print(audience["action"].iloc[0], "·", audience["channel"].iloc[0], "·", audience["cap"].iloc[0])

Mark the holdout, seeded, before anyone sees the list. A tenth by convention tonight; Part 3's self-study says when a tenth is too small. The seed is what lets a second analyst reproduce exactly who was held back.

★ Step 3 · seeded holdoutrng = np.random.default_rng(20180901) audience["held_out"] = rng.random(len(audience)) < 0.10 n_treated = int((~audience["held_out"]).sum()) n_held_out = int(audience["held_out"].sum()) print(n_treated, n_held_out) # about nine to one; the exact split depends on your N

Write the spec as a dictionary and print it. Audience definition with the snapshot date, the cut, the holdout share and seed, the metric with its window, and the cost per message. This is the sheet in the figure above; it goes in the ticket before the send.

★ Step 4 · the campaign specSNAPSHOT = "2018-09-01" COST_PER_MESSAGE = 0.30 # the session 3 model's unit cost; replace with your real one spec = { "segment": "011 Key win-back", "audience": f"rfm[code == '011'] as of {SNAPSHOT}", "cut": "mean split on R, F, M (session 1, step 5)", "n_audience": int(len(audience)), "holdout_share": 0.10, "holdout_seed": 20180901, "metric": "reorder within 90 days of send", "cost_per_message": COST_PER_MESSAGE, "action": ACTIONS["011"], } for k, v in spec.items(): print(f"{k:>17}: {v}")

The measurement template, with placeholders. After the window closes you fill two rates from two groups; everything else is arithmetic on them. It is written now so that the definition of "lift" is fixed before anyone has a result to prefer.

★ Step 5 · measure, after the windowresult = { "rate_treated": None, # reorders in the treated group / n_treated, after 90 days "rate_held_out": None, # reorders in the held-out group / n_held_out, same window } def measure(result, n_treated, avg_order_value, cost=COST_PER_MESSAGE): lift = result["rate_treated"] - result["rate_held_out"] incremental_orders = lift * n_treated organic_orders = result["rate_held_out"] * n_treated # would have happened anyway incremental_revenue = incremental_orders * avg_order_value net = incremental_revenue - n_treated * cost return {"lift": lift, "incremental_orders": incremental_orders, "organic_orders_in_attributed": organic_orders, "incremental_revenue": incremental_revenue, "net": net} # do not call measure() until the window has closed; today it would raise on None, # and that is the correct behaviour: there is no result yet to report

The bought-once check on your tool's top group. Apply the tool's published rule to your table (session 2 built both vendor maps), then ask what share of its top group has F of 1. If the answer is nine in ten, the action is Key develop's, not High value's.

★ Step 6 · bought once, in the top group# rfm["segment"] holds whichever tool map you built in session 2 (Klaviyo's six or Putler's eleven) top = rfm[rfm["segment"] == "Champions"] # or whatever your tool calls its top group print(len(top) / len(rfm)) print((top["F"] == 1).mean().round(3)) # Olist, 93,358 customers, rank(method="first") on the 97% tie: # Klaviyo's six -> 0.139 of customers are Champions, 0.910 of them bought once # Putler's eleven -> 0.161 of customers are Champions, 0.921 of them bought once

Save the spec and the held-out ids together, dated. spec_011_2018-09-01.json and heldout_011_2018-09-01.csv. The result is only a measurement if the same file that says who was held back also says when and by what rule, and session 6 will ask for both when the model is re-run.

Homework

Before session 6

45 min
  1. Write the spec for one segment on your own table. Seven fields, one page, before any list is exported. If you cannot fill the metric field with a window, you are not ready to send.
  2. Find your tool's cut in its documentation. The recency thresholds or quantiles, how frequency and monetary are split, and how ties are handled. If tie handling is not documented, write "not documented" in the spec; that is a finding.
  3. Compute the bought-once share of its top group on your data, as in step 6. Decide whether the action for that group is protect or get-the-second-order, and say which fact decided it.
  4. Decide the holdout share using the Experimentation course's guidance on the smallest group that can show the lift you would act on. If your audience is too small to test, write that down too, and pool or proceed knowingly.
Check yourself

Three questions

Clear all three and session 5 gets stamped in your passport.

1 · A platform reports R$ 40,000 of revenue attributed to a win-back campaign. Why is the lift a difference rather than that number, and what does attributed revenue hide?

A platform attributes any purchase after a send to the send. Without a held-out group there is no way to know how many of those purchases were organic, and vendor content itself concedes the organic share is not zero. The lift is the only number that pays for the message, and it is a difference between two groups measured over the same window.

2 · Your marketing platform shows 14 percent of customers as "Champions". Why check the bought-once share of that group before funding a campaign aimed at them?

The rules are published and should be read. Splitting a frequency column that is 97 percent ties into thirds or fifths has to put most identical one-time buyers somewhere, and the common rank convention assigns them by row order. The bought-once share on your own data tells you whether the group is a core to protect or recent single big baskets to develop.

3 · On the bench, the mean split gives the best net yield per message (R$ 3.46) but the median split gives the largest net total (R$ 128). Why can the two columns rank the cuts differently, and which of the figures transfers to your data?

Neither column is the answer alone. A spec that shows only one of them is hiding a decision. The per-message ordering is robust because it is driven by who the knife put in the audience (100 percent repeat buyers under the two-way splits against 2.4 and 2.5 percent under the quintile and sum cuts), and that share is measured, not modelled.

Sources covered

What this session teaches, and where it came from

✓The eight-group action table with channel and capPractice, following the common English rendering of the 2x2x2 taxonomy's action list; the caps on this page are illustrative and labelled so
✓Putler's action columnQuoted in short form from putler.com/rfm-analysis, eleven segments and their suggested actions
✓Klaviyo's thresholdsQuoted from Klaviyo help centre article 17797937793179: recency 3 within 180 days, 2 within 365, else 1; frequency and monetary by thirds; six groups from listed code sets. Champions shares on Olist replicated with rank(method="first"), tie handling not documented by either vendor
◐The organic share of attributed win-back revenue (15 to 22 percent)A vendor or press claim from a vendor-side blog about Klaviyo's attribution methodology; not checked at source, quoted only for the admission that some attributed revenue is organic
◐"私域复购是公域的3-5倍 (private-domain repurchase is 3 to 5 times public-domain)"Industry claim from a 2026 腾讯云开发者社区 (Tencent Cloud developer community) article, no dataset named, unverified; the page says a holdout inside one shop settles it for that shop
✓The session 3 bench: win-back audiences, yields and totals under four cutsComputed on a seeded 4,000-customer sample of the Olist population, verified in node against an independent pandas run; yields on a stated response model, with the model printed beside every figure

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 5 cheat sheet · pin this

Action = definition read backwardsWhat the code says is missing is the first move. 011 wins back and asks why; 101 gets order two; 000 gets a cap of zero.
Three columns per cornerAction, channel, frequency cap. Vendor tables give you the first and never the third.
Read the cut before the nameKlaviyo: R 3/2/1 at 180 and 365 days, F and M by thirds. Then compute the bought-once share of the top group on your data.
The lift is a differencerate(treated) minus rate(held out), same window. Organic rate times treated count is revenue that was happening anyway.
Hold out before the list leavesA random tenth, seeded, saved with the spec. A tenth of 33 is 3: decide the share per audience, with the experimentation maths.
Two columns, two orderingsPer message: mean 3.46 beats median 2.84. Total: median 128 beats mean 103. The ordering transfers, the reais do not.