One question, seven ways of answering it
Every algorithm in this course answers the same two Lumen questions - will this session convert, and how big will the order be - but each answers in its own voice. A linear model draws a straight line. A tree asks yes/no questions. kNN just looks at the neighbors. Naive Bayes multiplies probabilities. An SVM hunts for the widest gap. None of them is "the best one"; each is a different bet about what shape the truth has. Learning the zoo means learning which bet fits which data - that judgement is what separates someone who calls .fit() from someone who models.
Seven algorithms, two families 8 min live
The zoo looks big until you see the family resemblance. Model-based algorithms compress the training data into a small set of learned numbers - weights, split rules, class statistics - then throw the data away. Instance-based algorithms keep the data and answer by looking things up. That one distinction predicts most of each algorithm's personality: training speed, prediction speed, memory, and how the boundary looks.
LiveWhat intro-ml already gave you (and what we will not repeat)3 min▶
This course assumes the intro-ml builder track. In one breath: you split data into train and test before looking at it, you fit models (and scalers!) on train only, you never trust raw accuracy on imbalanced data (Lumen converts ~3%, so "always no" is ~97% accurate and useless), you read classifiers with ROC-AUC / precision / recall and regressors with RMSE / MAE / R², and you tune with cross-validation, never on the test set.
We will use all of that constantly and re-teach none of it. If any sentence above felt foggy, take learn-intro-ml-with-phoebe ↗ first - this course starts where its b10 capstone ended.
LiveParametric vs non-parametric - the vocabulary that unlocks the docs4 min▶
Parametric models have a fixed number of learned parameters no matter how much data you feed them. Linear regression on 7 features learns 8 numbers whether you train on 400 rows or 40 million. That makes them fast, small, and stable - and caps how bendy they can get.
Non-parametric models grow with the data. kNN's "model" is the whole training set; a tree can keep splitting as long as rows remain. They can fit almost any shape - which is power and danger in one feature: the bendier the family, the easier it memorizes noise.
scikit-learn's user guide calls kNN "non-generalizing" for exactly this reason: it does not compress the data into a generalization, it remembers. You will feel the consequence in the live tool below - and measure it properly in session 5.
LiveThe two Lumen tasks, one last time3 min▶
Same brand, same generator, seed 42 - the dataset you built in intro-ml b1 and the numbers that reconcile with the statistics and experimentation courses:
| Task | Target | Shape | The floor to beat |
|---|---|---|---|
| Will this session convert? | converted (0/1, ~3% yes) | Classification | "always no" - ~97% accuracy, catches nobody |
| How big will the order be? | order_value ($, mean ~$73) | Regression | "predict the mean" - RMSE ~$40 on buyers |
Every algorithm in the zoo will face both tasks (where it can). By session 10 you will have a leaderboard for each - and the honest surprise of this course is which models top it.
The decision boundary - where a model changes its mind 10 min live
Take any classifier, feed it every point on a 2D map, and paint each point by the predicted class. The seam between the colors is the decision boundary - the set of places where the model flips from "no" to "yes". Every classifier in the zoo draws this seam in its own signature style: logistic regression draws one straight line, trees carve axis-parallel rectangles, kNN traces the local crowd, RBF kernels wrap curves around clusters. Learn to read this picture and you can identify an algorithm from its boundary like a naturalist reading footprints.
LivePlay: the boundary explorer6 min▶
These are real models fit live in your browser - real gradient descent, real CART splits, real neighbor votes. Two datasets: Lumen real is 300 actual generator points (150 converters, 150 non-converters, spend x pages standardized), and Lab: two moons is a stylized shape for geometry intuition - clearly not Lumen data. Try this script:
- Start on Lab + kNN. Drag k down to 1: train accuracy hits ~100% while test accuracy sinks. That gap IS overfitting - you will see it on every algorithm all course.
- Switch to tree and crank depth: watch it carve one rectangle per stray point.
- Switch to Lumen real: every algorithm gets humbler. Two features of real checkout data hold a weak, honest signal (~60% accuracy on a balanced sample) - real data whispers, it does not shout.
Self-studyWhy only two features in the picture?4 min▶
Real Lumen models use all seven features; boundaries live in 7-dimensional space and cannot be drawn. Two features are a teaching window, not a recommendation - and the window has a catch: with fewer features, the models above score lower than the ~0.69 ROC-AUC you will get with the full feature set in session 3.
The shapes, however, generalize perfectly: a logistic model in 7D draws a flat 6-dimensional plane exactly as it draws a line in 2D; a tree still cuts one feature at a time; kNN still votes among neighbors. Learn the geometry in 2D, trust it in 7D.
Rebuild Lumen, then meet your baselines ★ 18 min · everyone builds
Two jobs: regenerate the exact Lumen dataset (seed 42 - if you still have lumen_sessions.parquet from intro-ml, this run will produce the identical file), then fit one representative from three families with identical code and stand up the leaderboard this course exists to explain.
Recreate the dataset. Run the generator exactly as intro-ml b1 shipped it - same seed, same coefficients. Sanity-check the fingerprint: 40,000 rows, ~3% conversion, AOV mean ~$73.
Build the shared preprocessing. One ColumnTransformer: one-hot the four categoricals, scale the three numerics. Fit on train only - the b2 discipline, forever.
Fit three animals with one loop. Same Pipeline, three estimators swapped in - the estimator API is why this is three lines, not three scripts.
Read the leaderboard. ROC-AUC on the held-out test set. Do not explain the ranking yet - explaining it is the whole course.
Before session 2 ◐ 25 min
- Run
baselines.pyend to end and keep the printed leaderboard somewhere visible - it is the "before" photo for the whole course. - In the boundary explorer, find a k where kNN's train and test accuracy are closest on the Lab dataset. Write the number down; session 5 explains why that k and not 1.
- Add
GaussianNB()(fromsklearn.naive_bayes, needs no scaling but works fine with it) as a fourth zoo member. Where does it land on the leaderboard? - One paragraph: which family (model-based or instance-based) do you expect to win on Lumen, and why? No wrong answers yet - you will grade yourself in session 10.
Three questions before you go 🎯 ◐ 90 seconds
1 · What is the decision boundary of a classifier?
Paint every point in feature space by predicted class; the seam where the color changes is the boundary. Its shape is the algorithm's signature: line, rectangles, local wobble, or smooth curves.
2 · kNN is called "instance-based" (or "non-generalizing") because...
kNN's model IS the training data. That buys flexibility (any boundary shape) and costs memory and prediction speed - the family trade-off in one algorithm. It handles regression fine (session 5).
3 · In the explorer, kNN with k=1 shows train accuracy ~100% and much lower test accuracy. What is that gap called and why does it appear?
At k=1 each training point predicts itself perfectly, so the boundary wraps around every noisy point. Perfect train score, poor test score - the overfitting tell you will re-meet on trees (depth), polynomials (degree), and RBF (gamma).
What this session covers
Session 1 covers the orientation layer of the anchor sources - the supervised-learning map and the vocabulary - so each later session can go deep on one animal. Certificates, videos, and graded labs stay with the official providers.