Learn with Phoebe · Data Science

Learn Ensemble Methods with Phoebe

The tabular champions: bagging, random forests, gradient boosting, XGBoost, LightGBM, CatBoost - and the discipline that makes them win. One running project across all six sessions: predict Mango Lane's churn, climbing from a single overfit tree at 58 percent holdout to a tuned booster at 73. Every rung on that ladder is fitted live in your browser - real trees, real splits, real train-versus-holdout curves - because a number you measured yourself argues better than any slide.

6sessions
58% · one tree, holdout
73% · tuned booster
600real trees per click
45min per session

The six sessions 🌲 · one churn model, built across all of them

Everyone starts at session 1 with one hand-built split. By session 6 the leaderboard fits five whole ensembles live and the tuned booster takes the crown - with the receipts.

🌳 Session 1 · easy

Why trees win tables

The evidence read at quote level: three reasons from the research, the honest competition tally, and your first hand-built split on the course data.

▶ Start here 🌲 Session 2 · easy

Random forests

Bagging turns instability into strength: bootstrap samples, random feature subsets, out-of-bag error - and ten free holdout points, measured.

the Breiman recipe 📉 Session 3 · medium

The residual game

Boosting as gradient descent in function space - then drive it live: real trees, ten at a time, the 32-point overfit gap, and early stopping teaching itself.

🔬 live engine 🚀 Session 4 · medium

XGBoost

The regularized objective, the second-order trick and its real lineage, the four knobs that matter, and the 17-of-29 statistic quoted the way the paper wrote it.

the industrial engine ⚡ Session 5 · hands-on

LightGBM & CatBoost

Histograms, GOSS, exclusive feature bundling, leaf-wise growth - and CatBoost's ordered boosting against the target leakage hiding in every naive encoding.

the speed & the leak 🏆 Session 6 · capstone

Tune, explain, ship

Five configs fit live on the leaderboard, tuning wins by five points, SHAP explains every flag, and TabPFN gets its honest challenger slot.

🔒 the leaderboard
easy - everyone medium hands-on capstone

Your passport 🎫

Clear all three quiz questions in a session and it gets stamped. Stamps live in this browser only.

Choose your path 🗺️

Three doors into the same discipline: average what disagrees, chain what corrects, and never watch the train curve.

⚡ I need working XGBoost intuition, fast 3→ 4→ 6 the loop, the engine, the tuning strategy
📚 I want the foundations properly 1→ 2→ 3 evidence, bagging, boosting - the two recipes from first principles
🌲 The whole journey 1→ 2→ 3→ 4→ 5→ 6 from one hand-built split to a tuned, explained, shipped model

What is real here 🧾

A course whose title promises winners owes you a precise account of its own numbers.

ClaimStatus
The trees, forests and boosters in the widgetsReal. Genuine greedy split-search trees fitted to the embedded 300-customer sample in your browser, every click - up to 600 trees per run
The ladder: 58, 68, 68, 73, 70Measured on 100 held-out customers by the same engine, deterministic seed, reproducible on your machine right now
The datasetThe teaching artifice: a seeded 300-customer Mango Lane sample with 10 percent label noise by construction - stated wherever it appears
Every research claimVerified to source at quote level - including the two most misquoted facts in the field: the XGBoost 17-of-29 stat (blog-published winning solutions, not all competitions) and LightGBM's 20x (vs pre-sorted GBDT, not vs XGBoost)
Trees vs deep learningScoped honestly: typical, medium-sized tabular data per Grinsztajn et al.; the TabPFN challenger presented two-sided and dated
The in-browser engine vs the real librariesA teaching implementation - it demonstrates the mechanism; xgboost and lightgbm ship far more engineering, which sessions 4-5 cover honestly

The knowledge map 🧠

The whole course at a glance - hover a session to spotlight its concepts, click any node to jump in.