Why your three reports never agree
By now you know attribution is a credit-splitting rule, you can read the heuristics, and you know the data-driven models are still only correlational. This session zooms all the way out. In any serious marketing org, three separate measurement disciplines run at once - MMM (marketing mix modeling), MTA (multi-touch attribution), and incrementality (holdout experiments) - and they routinely produce different numbers for the same channel. The rookie move is to pick a favorite. The leader move is to know what each one is built to answer, and to wire them together so each covers the others' blind spots. Six sessions, one real brand, zero code.
One question, three tools, three honest answers 6 min live
Lumen's CMO asks a simple-sounding question: "Should we spend more on paid social?" Send that question to each of the three measurement systems and you get three different - and each individually correct - answers, because each is answering a subtly different version of the question.
LiveThe three questions, said plainly3 min▶
Memorize the questions, not the acronyms. Each tool exists because a leader needed to answer one specific thing:
- MMM answers "how do I split my TOTAL budget?" It looks at aggregate weekly spend and outcomes across every channel - including the ones you cannot track a user through, like CTV, brand, and offline. It is the strategic anchor for the whole media plan.
- MTA answers "which campaign or creative wins WITHIN a channel?" It follows individual user journeys touch by touch, so it is granular and fast - perfect for daily steering of the campaigns you can actually track.
- Incrementality answers "is this channel's ROAS actually real?" It runs a controlled experiment - turn a channel off in some geographies, leave it on in others, measure the gap. It is the only one of the three that establishes cause.
A brand's MTA dashboard credited branded search with a 14x ROAS - the highest of any channel. Then they ran a holdout: paused branded search bidding in half their regions. Sales barely moved. Those customers were going to type "the brand" and buy anyway; the ads were harvesting demand that already existed. MTA measured the correlation perfectly and drew exactly the wrong conclusion. Only the experiment revealed the truth.
LiveCorrelational vs causal - the line that matters2 min▶
Here is the single most important distinction in modern measurement, and it splits the three tools cleanly. MMM and MTA are correlational. They observe what happened alongside the sale and infer contribution. They can be fooled by demand they did not create. Incrementality is causal. It manufactures a genuine comparison - a world with the channel and a world without - so the difference it measures is the lift the channel actually caused.
This is why incrementality is called the gold standard. It is also why it is used sparingly: experiments cost real revenue (you deliberately turn off working channels in some regions) and take weeks to read. You cannot run one for every daily decision. The art is using the cheap, fast, correlational tools for most decisions and the expensive, slow, causal tool to keep them honest.
What each tool is actually good at 5 min live
Put the three side by side on the four dimensions a leader actually cares about: what data it eats, what question it answers, whether it survives privacy loss, and whether it can claim cause. The pattern jumps out.
| Dimension | MMM | MTA | Incrementality |
|---|---|---|---|
| Data it uses | Aggregate weekly spend + outcomes, 2-3 years | User-level touchpoint paths | A controlled experiment (geo on/off, holdout) |
| Question it answers | How to split the TOTAL budget across all channels | Which campaigns/creatives win within channels | Is this channel's ROAS causally real? |
| Sees offline / brand / CTV? | Yes - the only one that can | No - only trackable digital touches | Yes - whatever you can toggle by geo |
| Privacy resilience | High - uses no user-level data | Low - depends on tracking individuals | High - measures aggregate geo outcomes |
| Causal or correlational? | Correlational | Correlational | Causal (the gold standard) |
| Speed / cadence | Slow - quarterly refresh | Fast - daily | Slow - weeks per test |
LiveRead the grid like a leader3 min▶
The table and the grid say the same thing two ways. Three takeaways to carry out of the room:
- MTA is the only low-privacy-resilience tool - it needs to follow individuals, which is exactly the thing 2026 privacy rules took away. That is next session's whole story, and it is why MTA's share of trust has been shrinking.
- MMM is the only tool that sees your untrackable spend - CTV, brand, offline, influencer. If a big slice of Lumen's budget is invisible to user-level tracking, MMM is not optional; it is the only lens that sees the whole board.
- Incrementality is the only causal tool, so it is the tiebreaker - when MMM and MTA disagree, you do not argue, you run an experiment. The test settles it.
Self-studyWhy the numbers disagree - three real reasons3 min read▶
When your MMM, your MTA dashboard, and your last geo-test show different ROAS for the same channel, it is almost never because one is broken. It is because:
- Different data grains. MMM sees weekly aggregates; MTA sees individual clicks. A channel that looks weak per-click can look strong in aggregate once you fold in its untracked halo effect on other channels.
- Different questions. "How to split the total" and "which creative wins" have genuinely different answers. Comparing them directly is a category error.
- Correlational vs causal. MMM and MTA report what co-occurred with sales; incrementality reports what caused them. A channel can correlate beautifully with revenue while causing very little of it - branded search is the classic trap.
So the goal is never to force the three into one number. It is to know which one to believe for which move - and to let the causal one calibrate the other two.
Don't pick one - stack all three 5 min live
The modern answer is not a winner. It is a division of labor: MMM as the strategic anchor, incrementality tests to calibrate it, and MTA for daily tactical steering. Lift results become priors that make the MMM more trustworthy; the MMM sets the budget; MTA optimizes inside it. But you do not build the whole stack on day one - you climb a ladder as your budget grows.
LiveThe calibrated loop, in one breath2 min▶
Here is how the three fit together once you are at the top of the ladder:
- MMM is the anchor. Once or twice a year it tells you roughly how the total budget should split across every channel, including the untrackable ones.
- Incrementality calibrates the anchor. You run geo-lift tests on your biggest channels; the measured lift becomes a prior that corrects the MMM, so the model is grounded in real experiments instead of pure correlation.
- MTA steers day to day. Inside the budget the MMM set, MTA and modeled attribution optimize which campaigns and creatives get the money this week.
Mature growth teams now run this as a rhythm: MMM refreshed quarterly, two or three incrementality tests per quarter feeding priors back into it, and MTA-driven optimization running continuously underneath. The MMM says "put 30% into social"; the lift test confirms social genuinely causes that lift; MTA decides which social campaigns get the 30%. Three tools, one loop, no arguing about which is "right".
Self-studyWhere Lumen sits, and why2 min read▶
Lumen spends $4M/year across nine channels. On the ladder, that places it squarely in the "attribution + geo-lift" tier - it is past the point where a simple MTA model is enough, and it should already be running incrementality tests on its biggest bets (paid social, influencer). It is not yet at the "full MMM as standing anchor" tier, but it is close enough that MMM is the obvious next investment - especially because a real chunk of its budget (CTV at 8%, influencer at 14%) is exactly the untrackable spend that only MMM can see. That is the through-line into your first work-along.
Place Lumen on the tier ladder ★ 10 min · everyone
Use Lumen's real numbers - $4M budget, nine channels, CTV and influencer growing - to place it on the ladder and name its single next measurement investment. Then do the same for your own org.
Find Lumen's rung. $4M puts it in the "$1M-$5M: add geo-lift" tier. So the baseline expectation is: it already runs attribution, and it should be running incrementality tests. Ask - is it?
Check the untrackable slice. CTV is 8% and influencer is 14% of the $4M - that is over a fifth of the budget that user-level MTA literally cannot see. Any channel MTA can't track is a vote for MMM.
Name the next investment. Lumen is at the top of its rung and about to cross into MMM territory. The single next move: stand up a first MMM to get a strategic anchor across ALL nine channels, calibrated by the geo-lift tests it should already be running.
Now do yours. Where does your budget put you on the ladder? What is the one tool you are missing at your rung - and is any of your spend invisible to user-level tracking (a hidden MMM argument)?
Reconcile a disagreement ★ 10 min · discuss
The scenario every CMO faces: MMM says paid social is under-funded, last-touch says paid search is the hero. Two systems, opposite verdicts, one budget. Which do you trust - and what do you do next?
Name the questions each is answering. The MMM is answering "how should the total split?" - a strategic question. Last-touch is answering "who got the final click?" - a closing-credit question. They are not even in conflict; they measure different things.
Apply create-vs-capture. Paid search captures demand that already exists - it sits by the finish line, so last-touch always flatters it. Paid social creates demand earlier, so last-touch always under-credits it. This is a known, predictable bias, not new information.
Refuse to move budget on either alone. The MMM is correlational (it could be reading a halo it did not cause); last-touch is a heuristic that throws away 80% of the journey. Neither is proof.
Break the tie with an experiment. Run a geo-lift test on paid social: hold it back in a set of regions for four weeks, measure the sales gap. If social causes real incremental lift, the MMM was right and you fund it - now with causal proof, not a model's opinion.
Write the verdict. "I trust neither number for the budget yet. The MMM's story is more strategically sound than last-touch's, so I will test it: a four-week geo-holdout on paid social settles it causally before we move a dollar."
The wrong answer is "trust the one with the nicer dashboard". The disagreement between MMM and last-touch is not a problem to resolve on a spreadsheet - it is a hypothesis worth testing. Leaders who treat conflicting measurement as a prompt to experiment, rather than a debate to win, are the ones who reallocate budgets that actually hold up.
This week ◐ 25 min total
- Locate your org on the ladder. What is your annual media budget, and what measurement should you have at that rung? Write down the one tool you are missing.
- Find your untrackable slice. Add up the share of your budget in channels that user-level tracking cannot follow - CTV, offline, brand, some influencer. If it is over 15%, you have an MMM argument to make.
- Ask your team the reconciliation question. "The last time two of our measurement systems disagreed, how did we decide who to believe?" If the answer is "we picked the one we liked", you have found a process gap.
- Optional: read one public geo-lift case study (Meta's Conversion Lift docs are free) and notice how the "turn it off in some regions" logic works - that intuition is all you need as a leader.
Three questions before you go 🎯 ◐ 90 seconds
1 · Which of the three measurement approaches is the only one that establishes cause?
Only an experiment - a real comparison of a world with the channel and a world without - can prove causation. MMM and MTA are both correlational; they observe what co-occurred with the sale.
2 · Why does MMM survive the loss of user-level tracking when MTA does not?
MMM models channels in aggregate, so cookies and device IDs are irrelevant to it. MTA depends on stitching one person's touches together - exactly the thing privacy rules removed.
3 · Your MMM and your MTA dashboard report different ROAS for paid social. The best read is...
Disagreement is usually two tools answering different questions, not an error. The mature move is to let the causal tool - incrementality - break the tie, and to stack all three rather than crown one.
What this session covers
This session distills the "three measurement approaches and why they disagree" chapter of the leading courses into a leader-first framing. Certificates and full video courses stay with their official sources.