Why normalize at all 8 min live
Normalization is the least glamorous thing in this course and one of the highest-yield. Google's crash course gives four concrete reasons, and it is worth knowing all four rather than the vague "it helps" you usually hear. Models trained on normalized features:
The fourth is the one people underrate. If prior_30d_spend runs 0 to 4,000 and pages_viewed runs 1 to 20, a gradient-descent model does not see "spend matters 200 times more" - it sees a loss surface stretched 200 times along one axis, and it wobbles down that valley slowly. The weights it eventually learns are shaped by the units you happened to record in, not by the signal.
Which transform, though? Google ties the choice to the shape of the distribution, not to the column's name or business meaning. Learn this table and most of the guesswork disappears.
| Technique | Formula | Use when the feature... | Shape |
|---|---|---|---|
| Linear (min-max) scaling | x' = (x - min) / (max - min) | "is mostly uniformly distributed across a fixed range" | Flat |
| Z-score scaling | x' = (x - μ) / σ | "is normally distributed (peak close to the mean)" | Bell |
| Log scaling | x' = log(x) | "is heavy skewed on at least either side of the tail" | Long tail |
| Clipping | cap at a chosen max or min | "contains extreme outliers" | Spikes at the edge |
LiveThree numbers to carry in your head4 min▶
Google works each technique with a small example. Memorize these three and you can sanity-check any transform on a whiteboard.
- Min-max. A feature spanning 100 to 900. The raw value 300 becomes 0.25, because it sits a quarter of the way along the range. Every value lands in 0 to 1, and the shape of the distribution is unchanged - only the axis moves.
- Z-score. With μ = 100 and σ = 20, the raw value 130 becomes +1.5: one and a half standard deviations above the mean. The unit is now "how unusual is this", which is exactly what a bell-shaped feature is telling you.
- Log. Google's line is the one to quote when someone asks why a log helps at all: "the log of 1,000,000 is only about three times larger than the log of 100." A ratio of ten thousand to one becomes a ratio of three to one. The order survives, the tyranny of the tail does not.
Lumen's spend column is the textbook case. prior_30d_spend is gamma-shaped and right-skewed - most sessions cluster near zero, a thin tail runs into the thousands. Min-max scaling would squash 95% of your customers into the bottom few percent of the range. Log or a power transform is what the shape is asking for, and the tail customers stay distinguishable from each other instead of all mapping to "high".
Self-studyClipping is not cheating3 min read▶
Clipping caps a feature at a chosen ceiling or floor. A session with 4,000 dollars of prior spend becomes 1,000 if that is where you clip, and the row stays in the dataset. Two things worth being clear about:
- Clipping keeps the row. Deleting outlier rows throws away real customers and quietly biases the model against the segment that spends the most. Capping keeps the customer and only limits how far one number can drag the loss.
- Clipping destroys information above the cap. Everything past the ceiling becomes identical. If the difference between a 1,200-dollar customer and a 4,000-dollar customer matters to you, log-scale instead, or do both: log first, then clip the residual extremes.
The honest framing: clipping is a statement that values beyond a point are noise or data-entry error, not signal. Make that statement deliberately, and write down where you clipped - it belongs in the feature spec you build in b10.
sklearn's scaler family, and its own advice 9 min live
sklearn ships half a dozen scalers and its user guide is unusually opinionated about when each one applies. The guidance below is quoted from the preprocessing chapter itself - anchor by section name rather than number, because sklearn renumbered its user-guide chapters and any number you memorize will rot.
| Transformer | What sklearn says about it | Reach for it when |
|---|---|---|
| StandardScaler | Zero mean, unit variance. On sparse input it needs with_mean=False or it raises a ValueError, because "silently centering would break the sparsity" | Roughly bell-shaped, dense data |
| RobustScaler | A "drop-in replacement" that uses "more robust estimates for the center and range" of the data | The column has outliers you do not want steering the mean |
| MinMaxScaler / MaxAbsScaler | MaxAbsScaler is "specifically designed for scaling sparse data, and is the recommended way to go about this" | Bounded ranges; MaxAbs for sparse matrices |
| PowerTransformer | Two methods: yeo-johnson and box-cox. "Box-Cox can only be applied to strictly positive data." Applies zero-mean unit-variance to the output by default | You want the column mapped toward Gaussian |
| QuantileTransformer | Maps to a uniform or normal output distribution by rank. Non-linear, so it collapses distances between values | Shape is a mess and rank is what you trust |
| Normalizer | Scales each row to unit norm, not each column | Text or count vectors where direction matters, not magnitude |
method="box-cox" will fail on any column containing a zero or a negative number. Lumen's prior_30d_spend has plenty of zeros - customers who bought nothing in the last thirty days. Use yeo-johnson, which handles zero and negative values, unless you have already established your column is strictly positive.
Here is the same reasoning applied to Lumen's numerics. Notice that each column gets the transformer its shape asks for, and that everything lives inside a Pipeline so the fitting happens on training rows only - the discipline b5 makes into a rule.
LiveWhich models actually care4 min▶
Scaling is not universally required, and knowing where it is optional saves you arguing about it in a code review.
- Trees do not need it. Decision trees, random forests and gradient boosting split on thresholds within a single column. A monotonic rescaling moves the threshold and changes nothing else. If your whole stack is XGBoost, scaling is mostly wasted effort.
- Anything with gradient descent does. Logistic regression, linear regression, neural nets. Unequal feature ranges stretch the loss surface and slow convergence - Google's first reason, restated.
- Anything with a distance or a penalty does, badly. kNN, SVM with an RBF kernel, k-means, and any L1 or L2 penalized model. Here scaling is not a convenience: an unscaled feature with a wide range dominates the distance metric outright, and a penalty applied to unscaled coefficients is not penalizing what you think it is.
The nuance tonight's workbench makes visible. The in-browser model is a logistic regression trained by gradient descent, and its numerics arrive unscaled. It still learns - raw numerics take held-out AUC from 0.500 to 0.696. Scaling would have bought faster convergence and better-behaved coefficients, not a different ranking of customers. It is worth being honest about that: scaling is mandatory for penalized and distance-based models, and is mostly a convergence and interpretability tool for a plain logistic fit.
Self-studySparse data has its own rules3 min read▶
A sparse matrix stores only the non-zero entries. That is the entire point, and centering a column - subtracting the mean from every value - turns every stored zero into a non-zero. A 40,000-row one-hot matrix that fit comfortably in memory becomes a dense block that does not.
sklearn refuses to do this by accident. StandardScaler raises a ValueError on sparse input unless you pass with_mean=False, because in its own words "silently centering would break the sparsity structure of the data". The recommended path is MaxAbsScaler, which sklearn describes as "specifically designed for scaling sparse data, and is the recommended way to go about this" - it divides by the maximum absolute value, so zeros stay zeros.
This matters directly after b3, where one-hot encoding a high-cardinality column is exactly how you end up holding a sparse matrix.
Binning, splines, and buying curvature 7 min live
Scaling moves a feature. Binning changes what the model is allowed to learn from it. Google defines it simply - binning "groups different numerical subranges into bins or buckets" - and gives two criteria for when it is the right move. Both are about the same thing: the relationship is not a straight line.
The choice of bin edges then matters more than the choice to bin. Equal-width buckets on a skewed column put almost every row in bucket one. Quantile bucketing fixes this, and Google's description of why is the clearest sentence on the topic: it "gives extra information space to the large torso while compacting the long tail into a single bucket". The dense middle of your distribution, where most customers live, gets resolution. The tail, where a handful of rows live, gets one bucket and stops distorting everything.
LiveSplines versus polynomials, and why sklearn prefers splines5 min▶
Both PolynomialFeatures and SplineTransformer let a linear model fit a curve. sklearn's preprocessing guide lays out four reasons splines usually win:
- No wild edges. High-degree polynomials show "oscillatory behaviour at the boundaries" - the guide names it: Runge's phenomenon. Your fit looks reasonable in the middle of the data and swings violently at the ends, which is exactly where your unusual customers sit.
- Better extrapolation. Splines extrapolate more sensibly beyond the observed range, and
extrapolation="constant"makes the behaviour explicit instead of accidental. - Numerically better behaved. The spline basis produces a banded matrix with a low condition number; the polynomial basis produces a Vandermonde matrix, which is famously ill-conditioned at higher degrees.
- But no interactions. Straight from the docs: "The SplineTransformer treats each feature separately, i.e. it won't give you interaction terms." If you need spend-times-returning, splines will not produce it. That is b6's job.
And the tidy equivalence to file away: a degree=0 spline is exactly KBinsDiscretizer(encode='onehot-dense') with n_bins = n_knots - 1. Binning is the crudest spline. Raising the degree is how you buy smoothness back.
Self-studyThe polynomial caution nobody reads2 min read▶
Google's numerical-data module attaches a warning to polynomial transforms that applies to any derived numeric feature: "If you transform a feature in a way that changes its scale, you should consider experimenting with normalizing it as well."
Square a feature that ranges 0 to 4,000 and the new column ranges 0 to 16,000,000. You have just re-created, at a larger scale, the exact imbalance normalization existed to fix - and if you built the polynomial after your scaler in the pipeline, nothing is going to catch it. Order matters: derive, then scale, and check the ranges of what you actually handed the model rather than the ranges of what you started with.
Rung one, earned ★ 12 min · everyone runs it
The workbench opens with raw numerics already on - the first real rung of the ladder you climbed in b1. Toggle it off and back on, pressing Score on held-out each time, and watch what the plain numbers are worth before any shaping at all.
Levers off: held-out AUC 0.500. A coin flip. Nothing about the algorithm gets you above this line.
Raw numerics on: held-out 0.696. Spend, pages viewed and session hour, straight from the table, unscaled and untransformed. This is the ladder's first real rung and it is free - the columns were already there.
Note what the workbench is not doing: those numerics are unscaled, and the logistic regression still learns from them. Be honest about this rather than overselling normalization. Scaling matters most for convergence speed and for penalized or distance-based models; for a plain logistic fit on three well-behaved columns, it buys stability rather than a different answer.
Now ask the question this session is really about: which of those three numerics is shaped badly? prior_30d_spend is the skewed one. Everything in Part 1 and Part 2 is about what you would do to it before it reaches a model that cares.
Try it yourself - this week ◐ 25-35 min total
- Take five numeric columns from a table you own and histogram each one. Label each shape flat, bell, or heavy tail, then write the transform Google's table prescribes. You will be surprised how many are heavy-tailed.
- Pick your most skewed column and fit
PowerTransformer(method="yeo-johnson")on it. Plot before and after. Then trybox-coxand confirm for yourself that it fails the moment a zero appears. - Take one column where you suspect the relationship is not linear. Bin it with
strategy="quantile", then withstrategy="uniform", and compare how many rows land in each bucket. That difference is the "torso versus tail" sentence made concrete. - Find one place in your existing code where a scaler is fitted before the train/test split. Move it inside a
Pipeline. That is a b5 fix you can make today.
Official sources covered
Tonight leans on Google's numerical-data module for the when-to-use logic and sklearn's preprocessing chapter for the implementations and their caveats. Certificates and graded exercises stay with the official providers. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · A column has a handful of extreme values that you do not want dragging the center and the spread around. Which sklearn scaler does the documentation point you to?
RobustScaler is the documented answer for outliers. Normalizer works across a row rather than down a column, and MaxAbsScaler is sklearn's recommendation for sparse data, where it preserves zeros - a different problem entirely.
2 · You reach for PowerTransformer(method="box-cox") on Lumen's prior_30d_spend. What stops you?
Straight from the docs. Yeo-Johnson is the method that tolerates zero and negative values, which is why it is the default choice on any spend, balance or delta column.
3 · Google gives two criteria for binning a numeric feature. What are they?
Both criteria say the same thing from different angles: a straight line is the wrong shape for this relationship. Binning lets the model learn "separate weights for each bin" and draw a step response instead.