Confounder, collider, mediator

A variable sitting between the treatment and the outcome can be arranged in exactly three ways, and there is no fourth. It can be a common cause of both, in which case arrows point out of it into each. It can be a link in the chain, with an arrow in from the treatment and out to the outcome. Or it can be a common effect, with arrows pointing into it from both sides.

That exhaustiveness is what makes the classification practical. You are not searching an open-ended space of ways a variable could misbehave; you are asking three yes-or-no questions about arrow directions on a graph you already drew. The graph does the hard part, which is why session 1 came first.

• season - confounder. Winter sends more email and sells more. Adjust.
• tenure - confounder. Long accounts get more email and buy more anyway. Adjust.
• loyalty - collider, and a pre-treatment one. A common effect of two unobserved causes. Do not adjust.
• opened - mediator. Carries part of the real effect. Adjust only if the direct effect is the question.
• ticket - collider, post-treatment. A common effect of the email and the spend. Do not adjust.
• device - bystander. No arrows at all. Free either way, and worth knowing that it is free.

Two adjust, three do not, one does not matter. Nothing about the column names, the data types or the correlations with spend would have produced that split.

The file carries a seventh column, recency, and it is deliberately not on that list because it is not one of the three shapes. It is an effect modifier: it sets how much the email is worth to each customer and it shifts their baseline spend, but it does not cause the email, so it opens no back door and needs no adjustment either way. The back-door set is still season and tenure and every graph on this page stands. Sessions 7 and 8 are where it earns its hour.

The shape is X <- Z -> Y. Season causes the email to be sent and independently causes spend to rise, so emailed customers and non-emailed customers differ in more than the email. Association flows from the treatment up the incoming arrow, across the fork and down into the outcome, and that flow is not the effect of anything the campaign did.

• Left alone, the path is open and its contribution is added to whatever the treatment really does. At Kestrelmark the two open back doors add 4.31 on a true effect of 6.23.
• Conditioning closes it. Inside a single season the season cannot vary, so it cannot explain any of the remaining difference in spend. Same for a tenure band.
• This is the only shape where conditioning is unambiguously the right move, and it is the shape most people have in mind when they say "control for it". The trouble starts when the phrase is applied to the other two.

The shape is X -> M -> Y. Opening the email is not a nuisance sitting between the treatment and the outcome; it is one of the two routes by which the treatment does its work. Holding it fixed compares emailed and non-emailed customers who ended up with the same open status, which subtracts the mechanism from the answer.

• The path carries real effect, which makes this the opposite case from a confounder. Closing it is not cleaning, it is deleting.
• What remains is the direct effect: whatever the treatment does by routes that do not run through the mediator. At Kestrelmark that is the sender name in the inbox.
• So the verdict is conditional on the question. Total effect, leave it out. Direct effect, put it in and say so in the write-up. There is no default that is right for both.

The failure mode is silent. Nothing errors, the fit improves, and the number that comes back is a perfectly good estimate of a quantity nobody asked for.

The shape is X -> C <- Y, and it is the one that reverses every intuition built on the first two. Formally: in a structure i -> m <- j, conditioning on m makes i and j dependent when they were independent, a result known in epidemiology as Berkson's paradox.

• A collider path is closed to begin with. Two arrows meeting head to head do not transmit association. Nothing needs to be done to it.
• Conditioning opens it. Restricting to a value of the common effect forces its two causes to trade off against each other inside that group.
• The association it creates is entirely manufactured by the analyst. It was not in the population and it will not replicate under a different selection rule.

Take the ticket flag. Two things push a customer toward raising one: getting the email, which prompts questions, and spending a lot, which also prompts questions. Now look only at the customers who raised a ticket.

Among those customers, learning that somebody did not get the email tells you they probably spent a lot, because something had to have caused the ticket and the email was not it. Learning that somebody did get the email removes the need for the other explanation, so their spend looks ordinary. Inside the ticket group, email and spend now appear negatively related, and that appearance was created by the decision to look inside the group.

• Neither variable changed. The population is exactly as it was; only the window has narrowed.
• Adjustment does this by construction. Putting ticket on the right-hand side of a regression is the same operation as looking within ticket groups, done for every value at once.
• It also happens without a regression. Any filter that removes rows on a common effect does it: analysing only customers who contacted support, only accounts that survived, only applications that were approved.

Kestrelmark's loyalty score is computed from behaviour that all happened before the campaign went out. Every reviewer who has ever been burned by a leaked post-treatment feature will wave it through on that basis, and every one of them is wrong here.

The score is a common effect of two things nobody measured. Call them u1 and u2: u1 is the disposition that makes somebody both a campaign target and a certain kind of loyal, and u2 is whatever makes somebody both loyal and a spender. So u1 -> loyalty <- u2, while u1 -> email and u2 -> spend. The path email <- u1 -> loyalty <- u2 -> spend contains a collider in the middle and is therefore closed while nobody touches it.

• Conditioning on loyalty opens that path, connecting the email to spend through two variables that are not in the warehouse and cannot be adjusted away.
• It happens despite loyalty preceding the treatment. The shape is A <- U1 -> C <- U2 -> Y and the middle of an M has nothing to do with when C was recorded.
• The price is measured. Adding loyalty to the correct set moves the estimate from 6.56 to 5.26, a bias of -0.97 against a true effect of 6.23. Timing says nothing about the size either: the post-treatment collider on the same graph costs -3.64, the biggest single harm of the four additions.

The formal statement includes it explicitly: conditioning on m or any descendant of m makes i and j dependent. Anything caused by a collider inherits most of its information, and conditioning on the child partially conditions on the parent.

• "We used a proxy instead" is not a fix. A support-contact rate, a satisfaction survey sent to ticket raisers, a churn-risk flag built partly from ticket volume: all of them are downstream of the ticket and all of them leak the same association.
• Nor is a coarser version. Bucketing a collider into three bands weakens the effect rather than removing it, and the residual bias is unknown without the graph.
• Nor is a composite score. If any input to an engineered feature is a collider or a descendant of one, the feature is contaminated, and this is how the problem gets into models that never mention the offending column.

The practical consequence is that classification has to run on the lineage, not on the column name. A feature is safe when everything feeding it is safe.

Adding the open flag to the correct set moves the estimate from 6.56 to 5.48. That looks like the same kind of failure as the loyalty score, and at -0.75 against that column's -0.97 it is even about the same size. It is not the same thing at all.

The model that generated the data has a mean direct effect of 5.16 written into it: the part of the email's work that does not run through opening. Conditioning on the open removes the mediated part and leaves that direct effect, and 5.48 is a good estimate of 5.16. The arithmetic did what it was asked. The specification asked the wrong thing.

• Total effect, 6.23 - what happens if the email is sent. This answers "should we run the campaign", and it is the estimand for almost every business decision.
• Direct effect, mean 5.16 - what happens if the email is sent and nobody opens it. This answers "how much of the lift survives a bad subject line", which is a genuine question for a different team.
• The bench cannot tell which you wanted. Nor can a reviewer, unless the estimand is written down before the covariate list.

Device has no arrows in the Kestrelmark graph. Adding it to the correct set leaves the estimate at 6.56 and the bias at +0.33, matching the back-door set to 0.01.

That result is worth as much as the harmful ones, because it removes the escape route of treating every extra covariate as risky. If all additions cost something, "keep the list short" would be sufficient advice and no graph would be needed. They do not. One of the four additions is free, three are expensive, and the four look identical in a schema browser. The classification is what separates them and there is no cheaper substitute.

• In finite samples a bystander is not quite free - it spends a degree of freedom and can move the estimate. At 5,000 rows here it moves it by 0.01.
• A variable that predicts the outcome and not the treatment is also generally safe and can tighten things up. The recency segment is exactly that: it moves spend, it does not move who gets the email, and so it carries no adjustment duty. Precision is a different topic from bias, and the bench measures only bias.