Observational causal inference, when you cannot run the experiment

Lumen's email opt-in users convert far better than everyone else, and the growth team wants to declare email the hero channel. But opt-in was never randomized - users self-selected on intent. The confounders are obvious once you name them: prior engagement, account tenure, acquisition channel. All three drive both who opts in and who buys. Session 10 is about doing the best you honestly can when the coin flip was not available: draw the causal graph, adjust for the right variables (and never the wrong ones), estimate with propensity methods, and then try hard to break your own answer. It ends the course by mapping every method you have learned to the questions it actually answers.

Before you adjust for anything, you draw the assumed causal graph - a directed acyclic graph (DAG). Pearl's backdoor criterion tells you what to condition on: block every "backdoor" path from treatment T to outcome Y (paths that start with an arrow into T), and condition on no descendants of T. Confounders open backdoor paths, so you adjust for them. But two kinds of variable must be left alone - conditioning on them creates bias where there was none: colliders and mediators.

Once the DAG says which confounders X to adjust for, the propensity score gives you a clean way to do it. The propensity is e(X) = P(T=1 | X) - each unit's modelled probability of being treated given its covariates (Rosenbaum & Rubin 1983). Two ways to use it: matching (pair treated and control units with similar scores) and inverse-probability weighting (IPW - weight each unit by 1/e(X) so the treated and control groups become comparable in aggregate). Doubly robust estimators (AIPW, TMLE) combine a propensity model and an outcome model and stay consistent if either one is correct - two shots on goal.

Straight callback to b1, now on Lumen's email data. Opt-in users self-select on intent, so the raw opt-in-minus-non-opt-in conversion gap is confounded. Quantify how wrong the naive number is before we fix it.

import pandas as pd, numpy as np

df = pd.read_parquet("lumen_email_optin.parquet")
# columns: opt_in (T), converted (Y), prior_engagement, tenure_days, acq_channel
confounders = ["prior_engagement", "tenure_days", "acq_channel"]

naive = df.loc[df.opt_in==1, "converted"].mean() - df.loc[df.opt_in==0, "converted"].mean()
print(f"Naive opt-in 'effect' = {naive:.4f}")   # large - looks like email is magic

print(df.groupby("opt_in")[["prior_engagement", "tenure_days"]].mean())
# opt-in users start higher on engagement AND tenure -> confounded comparison
# this is the b1 lesson: naive diff = real effect + selection on intent

Fit a propensity model, use it to weight or match, and recover a credible effect. We drive it through DoWhy's identify-estimate-refute flow so the causal assumptions are explicit, not buried in a regression call.

from sklearn.linear_model import LogisticRegression
X = pd.get_dummies(df[confounders], drop_first=True)

e = LogisticRegression(max_iter=1000).fit(X, df.opt_in).predict_proba(X)[:, 1]
e = np.clip(e, 0.02, 0.98)                         # trim extremes -> stable weights

w = np.where(df.opt_in==1, 1/e, 1/(1-e))           # inverse-probability weights
ipw = (np.average(df.converted[df.opt_in==1], weights=w[df.opt_in==1])
       - np.average(df.converted[df.opt_in==0], weights=w[df.opt_in==0]))
print(f"IPW effect = {ipw:.4f}   vs naive {naive:.4f}")   # much smaller, credible

from dowhy import CausalModel

model = CausalModel(data=df, treatment="opt_in", outcome="converted",
                    common_causes=confounders)
est_and = model.identify_effect(proceed_when_unidentifiable=False)   # backdoor set
estimate = model.estimate_effect(
    est_and, method_name="backdoor.propensity_score_weighting")
print("DoWhy ATE:", estimate.value)
# EconML alt: LinearDML / DRLearner for a doubly robust estimate

A causal estimate you have not attacked is a guess. DoWhy's refuters stress-test whether your number survives deliberate sabotage - and then you state, out loud, what an unmeasured confounder could still do.

# placebo treatment: replace opt_in with random noise -> effect should collapse to ~0
placebo = model.refute_estimate(est_and, estimate,
            method_name="placebo_treatment_refuter", placebo_type="permute")
print(placebo)   # new effect ~0 => your pipeline is not inventing effects

# random common cause: add an irrelevant covariate -> estimate should be stable
rcc = model.refute_estimate(est_and, estimate,
            method_name="random_common_cause")
print(rcc)       # estimate unchanged => robust to a spurious added confounder

# subset (data) refuter: effect stable on random subsamples?
subset = model.refute_estimate(est_and, estimate,
            method_name="data_subset_refuter", subset_fraction=0.8)
print(subset)

# then the honest sentence: name a plausible unmeasured confounder (e.g. purchase
# intent we never logged) and state that unconfoundedness is UNTESTABLE - this
# estimate holds only if we captured the confounders that matter.

Ten sessions, one skill: match the method to the question and the constraints. Here is the whole course as a decision table for Lumen's real questions. The rule that ties it together is at the bottom.

• Add a hidden confounder and hide it. Simulate an unobserved driver of both opt-in and conversion, drop it from the adjustment set, and show the de-confounded estimate is still biased - the honest ceiling, demonstrated.
• Condition on a collider on purpose. Filter to shoppers who reached checkout and watch a spurious relationship appear. This is why "add more controls" is wrong.
• Compare estimators. Put IPW, matching, and an EconML doubly-robust learner side by side on the same data; explain where they agree and diverge.
• Capstone: take one real question from your own work, place it in the decision table, and write the one-paragraph analysis plan - method, assumptions, and the residual risk you would name.

The capstone teaches the working core of observational causal inference and the discipline of stating your assumptions. The deep identification theory and the graphical formalism live in the sources below.