Feature prep, and the Pipeline that makes leakage impossible

Every session so far quietly hand-picked numeric features or one-hot encoded on the whole frame. That is fine for a demo and dangerous in production: if you fit a scaler or an encoder on all the data before splitting, the test set has already influenced training, and your score lies. The fix is not more discipline - it is structure. A Pipeline binds the preparation to the model as one estimator, so the transforms are fit on train only and replayed identically on test and in production.

Mixed-type data needs two different treatments. Numbers want scaling so no single large-valued column dominates; categories want one-hot encoding so a model can use them as 0/1 flags. A ColumnTransformer runs both lanes in parallel on the right columns, and a Pipeline glues that prep to the model as one estimator.

We go back to the classification task - predict converted - because it uses every column type Lumen has. We build one Pipeline, cross-validate it cleanly, read what it leaned on, and then prove the noise column carries nothing.

import numpy as np, pandas as pd

df = pd.read_parquet("lumen_sessions.parquet")
y = df["converted"]
X = df.drop(columns=["converted", "order_value"])

num = ["prior_30d_spend", "pages_viewed", "session_hour"]
cat = ["channel", "device", "new_vs_returning", "product_category"]
print("numeric:", num)
print("categorical:", cat)

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

pre = ColumnTransformer([
    ("num", StandardScaler(), num),
    ("cat", OneHotEncoder(handle_unknown="ignore"), cat),
])
pipe = Pipeline([
    ("pre", pre),
    ("model", LogisticRegression(max_iter=1000, class_weight="balanced")),
])

auc = cross_val_score(pipe, X, y, cv=5, scoring="roc_auc")
print("ROC-AUC %.2f +/- %.2f" % (auc.mean(), auc.std()))   # ~ 0.71

pipe.fit(X, y)
names = pipe.named_steps["pre"].get_feature_names_out()
coefs = pipe.named_steps["model"].coef_[0]

rank = pd.Series(np.abs(coefs), index=names).sort_values(ascending=False)
print(rank.head(6).round(3))
# top: prior_30d_spend, new_vs_returning, channel_* ...
# session_hour ~ 0.00  (the planted noise, exposed)

num2 = ["prior_30d_spend", "pages_viewed"]     # session_hour removed

pre2 = ColumnTransformer([
    ("num", StandardScaler(), num2),
    ("cat", OneHotEncoder(handle_unknown="ignore"), cat),
])
pipe2 = Pipeline([("pre", pre2),
    ("model", LogisticRegression(max_iter=1000, class_weight="balanced"))])

auc2 = cross_val_score(pipe2, X, y, cv=5, scoring="roc_auc")
print("without session_hour: ROC-AUC %.2f" % auc2.mean())   # ~ 0.71, unchanged

• Engineer a feature. Add one you can compute at prediction time (for example spend-per-page, prior_30d_spend / pages_viewed), fold it into the numeric lane, and re-fit the pipeline. Did it help?
• Explain handle_unknown. Set handle_unknown='ignore' and write why it matters - what happens in production when a channel appears that was not in the training data?
• Rank and prune. Rank features by importance, remove the weakest one, and confirm the CV score does not fall. More columns is not more signal.
• Optional: swap LogisticRegression for a RandomForestClassifier in the same Pipeline - only the model step changes, and you read feature_importances_ instead of coef_.

This builder session covers correct feature preparation - the leak-proof way to feed mixed-type data to a model - on the Lumen classification task, and sets up the capstone in b10.