Capstone: the feature spec

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder, TargetEncoder
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split, cross_val_score

X_train, X_test, y_train, y_test = train_test_split(         # b5 rule 1: FIRST
    X, y, test_size=0.4, stratify=y, random_state=42)

numeric     = ["prior_30d_spend", "pages_viewed", "tenure_days"]   # b4 derived
categorical = ["channel", "product_category", "device", "new_vs_returning"]
high_card   = ["postal_code"]                                # ~1,200 levels

prep = ColumnTransformer([
    ("num", Pipeline([
        ("impute", SimpleImputer(strategy="median",          # b7 imputation
                                 add_indicator=True)),       # b7 missingness
        ("scale",  StandardScaler()),                        # b2 scaling
    ]), numeric),
    ("cat", OneHotEncoder(handle_unknown="ignore",           # b3 encoding
                          min_frequency=0.01), categorical),
    ("hc",  TargetEncoder(), high_card),                     # b3 cross-fitted
])

pipe = Pipeline([                                            # b5 the container
    ("prep",   prep),
    ("select", SelectKBest(f_classif, k=25)),                # b8 selection
    ("model",  LogisticRegression(max_iter=1000)),
])

pipe.fit(X_train, y_train)                                   # b5 rule 2
print(cross_val_score(pipe, X_train, y_train, cv=5).mean())

Walk the mapping out loud, because this is the moment the ten sessions become one object:

• b2 · scaling. StandardScaler sits after imputation, inside the numeric sub-pipeline, so its mean and variance are learned from training rows only.
• b3 · encoding. OneHotEncoder(handle_unknown="ignore") for the low-cardinality columns, with min_frequency grouping rare levels; TargetEncoder for postal_code, whose internal cross-fitting is the only reason a 1,200-level column is safe here at all.
• b4 · datetime. tenure_days and the weekday flags were derived from signup_date upstream and enter as ordinary numeric and categorical columns. Decomposition happens before the pipeline; the pipeline only transforms.
• b5 · the pipeline itself. Not a convenience. The structure is what makes the wrong thing hard to type, which is worth more than remembering to do the right thing.
• b7 · imputation. add_indicator=True keeps the missingness as its own feature, because "this was blank" is frequently the signal.
• b8 · selection. SelectKBest sits inside, so it re-fits within every cross-validation fold. Outside, it sees every row before the split - which is precisely the sklearn experiment that scored 0.76 accuracy on pure noise where the honest answer was 0.50.

• The b9 embedding block. Text encoding runs upstream and caches; the vectors join as columns. Add it only when it beats the hand-crafted flags, per Rule 17.
• Interactions (b6). Lumen's spend-times-returning term is derived before the pipeline for readability. Anything you can name explicitly, name explicitly.
• The leakage lever. post_purchase_flag is not in numeric, not in categorical, not anywhere. That absence is field 3 doing its job in code.

• learn-model-evaluation-with-phoebe judges what these features produced: metrics, thresholds, and the cost of being wrong in each direction. This course owns the inputs; that one owns the verdict.
• learn-intro-ml-with-phoebe is the modelling mechanics underneath - the estimator, the split, the fit. Worth a revisit now that you know what you were feeding it.
• The leader track (a1-a6) is for whoever funds and reviews this work. a5 covers hand-crafted versus learned features for decision-makers, and a6 is the feature review checklist that the seven-field spec is designed to survive.

Named, not taught, and you should know they exist:

• Automated feature engineering - featuretools, autoML feature synthesis. They generate candidate features at volume. They do not answer field 3, which means everything in this course still has to happen afterwards, on a much larger pile.
• Time-series features - lags, rolling windows, seasonality. That is learn-timeseries-forecasting-with-phoebe.
• Representation learning - training the encoder as part of your model rather than calling a frozen one. That is learn-deep-learning-with-phoebe; b9 was the bridge, not the destination.
• Feature stores - the concept and when it pays are in the leader track (a4); implementation and ops belong to the data-engineering bucket.