Logistic regression, from the inside

sigma(z) = 1 / (1 + e^-z). Three properties earn it the job. It is monotonic: a higher score always means a higher probability, so the linear ranking survives the squash. It is saturating: past z of about plus or minus 4 it flattens near 1 or 0, so absurd scores cannot produce probabilities above 1 or below 0. And it is symmetric around z = 0: sigma(0) = 0.5 exactly, which is what makes "score positive or negative" and "probability above or below a half" the same question.

So the model is two moving parts, not one mysterious blob: a linear scorer (all of session 2's machinery - weights, features, an intercept) and a fixed, parameter-free squash. Everything the model learns lives in the line. The sigmoid never changes.

Where does the model flip from "no" to "convert"? Where p = 0.5, which is where z = 0, which is the equation w·x + b = 0 - and in feature space that is a straight line (a flat plane in more dimensions). This is session 1's boundary explorer coming home: of the four boundary signatures you met, logistic regression owns the calm straight seam.

Same explorer, now locked to logistic on the real sample - 2 features (spend x pages, standardized), balanced converters vs non-converters. Notice how the seam stays a straight, unbothered line at ~59% test accuracy: two features of real checkout data carry a weak, honest signal. Real data whispers.

Because z is the log-odds, each coefficient says: "one unit of this feature adds this much to the log-odds". Adding on a log scale is multiplying on the raw scale - so exp(coef) tells you what one unit of the feature multiplies the odds by. That multiplier is the odds ratio (OR), and it is the currency logistic models are read in. Here is Lumen's near-unregularized fit, dummies vs the affiliate reference channel:

Notice the model's honesty: it found the strong channels, priced spend and pages sensibly, and gave the decoy feature a multiplier of one. You will verify all of this against the generator's true coefficients in the build.

"Returning customers have OR ~2.8" does not mean "returning customers are 2.8x as likely to convert". Odds and probability are different rungs of the ladder, and the ratio of one is not the ratio of the other - the gap widens as probabilities grow. At Lumen's ~3% base rate the two happen to be numerically close, which is exactly why sloppy phrasing survives unchallenged - until someone applies it at a 40% base rate and the claim becomes mathematically impossible.

Surprise for statisticians: LogisticRegression() is regularized out of the box - L2 penalty, C=1, where C is the inverse regularization strength (small C = strong penalty; the reciprocal of the alpha convention from session 2's ridge card). Great default for prediction; quietly biased for interpretation, because the penalty shrinks every coefficient toward zero and your odds ratios come out closer to 1 than the data says.

The convention this course uses: for clean, readable odds ratios, fit near-unregularized with C=1e6 - the statisticians' habit. For pure prediction pipelines, keep the default and tune C by cross-validation - the ML habit. Both are correct; they answer different questions. This is sklearn section 1.1.11's territory - read the whole section once, it is short and saves careers.

lbfgs is the default and the right answer most days: fast, handles L2, no drama. saga is the one to reach for when you need L1 or elastic-net penalties or have very large data - but it converges properly only on scaled features. liblinear is the old workhorse with a documented quirk worth knowing: it penalizes the intercept along with the weights, which subtly shifts fitted values - avoid it when you plan to read coefficients. If you ever see a convergence warning, raise max_iter first, check scaling second, switch solvers third.

Log-loss has a personality: it charges for wrongness in proportion to confidence. Predict p = 0.6 for a session that does not convert and you pay a modest fee. Predict p = 0.99 for that same session and the penalty is brutal - the loss grows without bound as your confidence approaches certainty. A model trained on log-loss learns not just to be right, but to be calibrated: to reserve its 0.99s for cases that deserve them.

That is exactly the incentive you want in a conversion model - a scorer that shouts only when it should. Compare naive Bayes in session 6, whose probabilities you will learn not to trust.