A different way to see the problem
ETS (b3), ARIMA (b4), and Prophet (b5) are all purpose-built time-series models. The ML approach throws that framing out and asks: what if forecasting is just supervised regression? Build a table whose target column is tomorrow's value and whose feature columns describe what we knew up to today - yesterday's value, last week's same day, a 7-day average, the day of week. Fit any regressor to that table. Gradient-boosted trees (XGBoost, LightGBM, sklearn's HistGradientBoostingRegressor) are the workhorse here, and they are what carried the top of the M5 competition. The power is that this scales to hundreds of related series and lets you feed in external drivers; the price is that you engineer the features, and one leak sinks the whole thing.
Turn the series into a supervised table 6 min live
A regressor needs rows of features -> target. We build them by sliding a window along the series: for each day t, the target is the value at t, and the features summarise the days before it. Slide the window forward one day and you get the next row. Do this for the whole history and the time series becomes an ordinary table any regressor can learn.
LiveThe three feature families3 min▶
Almost every ML forecast is built from three kinds of feature:
- Lags - the value at t-1, t-7, t-14.
s.shift(1)is yesterday;s.shift(7)is the same weekday last week. Lags carry the autocorrelation that ARIMA modelled explicitly. - Rolling statistics - a moving mean or std over a recent window, e.g.
s.shift(1).rolling(7).mean(). These summarise the recent level and volatility. - Calendar features - day-of-week, month, is-weekend, is-holiday, read straight off the datetime index. These carry seasonality the tree can split on.
s.shift(1).rolling(7).mean(), not s.rolling(7).mean(). The shift(1) pushes the window to end at yesterday, so the feature only uses data you would actually have when forecasting today. Drop the shift and the rolling mean includes today's target - a leak that flatters your score and fails live.
Self-studyWhy trees, and when this family wins2 min read▶
Any regressor can learn the table, but gradient-boosted trees dominate this space for good reasons:
- They capture non-linear interactions (a Saturday effect that only shows during a sale) without you specifying them.
- They shrug off feature scale and tolerate irrelevant features, so you can throw in many candidates.
- They scale to hundreds or thousands of related series stacked into one big table - the M5 setup, where a single model forecasts every product in every store.
M5 is why this session exists. The M5 competition (Walmart hierarchical sales, 2020) saw gradient-boosting and ML/hybrid methods rise to the top of a field long dominated by statistical models - especially where many related series and external drivers were in play. The lesson is not "ML always wins" - the M-competitions still show simple baselines are hard to beat on a single series - but that when you have many series plus drivers, the regression framing scales in a way one-series models do not.
Forecasting H days: direct vs recursive 5 min live
A regressor trained on "features -> next day" predicts one step. To reach H days out you need a strategy. There are two: recursive feeds each prediction back in as a lag to predict the next day with one model, and direct trains a separate model for each horizon (h=1, h=2, ... h=H). Recursive is simple and lets errors compound; direct avoids compounding but costs H models.
LiveLeakage: the one mistake that ruins ML forecasts3 min▶
The regression framing is powerful precisely because it is flexible - and that flexibility is where leakage sneaks in. Every feature must be computable using only information available at forecast time.
- Split by time, never at random. Same rule as b1. A random split scatters future rows into training and inflates the score.
- Lag your rolling windows.
shift(1)beforerolling, so the window ends yesterday, not today. - Only known-in-advance drivers. A "did it rain" feature is a leak unless you have a rain forecast; the calendar (day-of-week, holidays) is safe because it is known in advance.
- Drop the warm-up rows. The first few rows have no full lag/rolling history -
dropna()them rather than filling with something you would not have.
Build the supervised feature frame ★ 12 min · everyone builds
Turn the Lumen series into a table of features and a target, with every feature engineered to be known at forecast time.
Rebuild the Lumen daily series from b1 and put it in a dataframe with the target column y.
Add lag features with s.shift(1) and s.shift(7), and rolling stats with s.shift(1).rolling(7).
Add calendar features from the datetime index: day-of-week, month, is-weekend.
dropna() the warm-up rows and print the supervised table.
lag_1 and lag_7 carry the recent value and the weekly echo; roll_7 the level; the calendar columns the seasonality. This table, not the raw series, is what the model actually sees.
Fit gradient boosting and forecast H days ★ 10 min · build your own
Split by time, fit a HistGradientBoostingRegressor on the feature frame, then roll a recursive multi-step forecast forward H days and score it against the b1 seasonal-naive baseline. Finish by asking which features carried the prediction.
Split the table by time - the last H rows are the test set - and fit the regressor on the training rows.
Forecast recursively: predict the next day, append it to the history, rebuild that day's features, repeat H times.
Score MAPE on the holdout and set it beside the seasonal-naive baseline from b1.
Read permutation importance to see which features matter.
Feature importance is a sanity check, not a victory lap. On Lumen you should see lag_7 and dow near the top - the weekly rhythm is the real signal, exactly what seasonal-naive exploits. If a leaky feature ever tops the list with suspiciously perfect scores, that is your leak alarm. And the ruling is the same as every session: unless this MAPE beats the seasonal-naive baseline on a backtest (b8), the extra machinery has not earned its place. On one clean series, do not be surprised if the simple baseline is right there with you - that is the M-competition lesson, not a bug.
Try it yourself - this week ◐ 30-45 min total
- Build the supervised feature frame on your own daily series. Print it and confirm by eye that every feature uses only past information.
- Fit
HistGradientBoostingRegressor(or XGBoost/LightGBM if installed) and produce a recursive H-step forecast. - Score its MAPE and add it to your running table beside seasonal-naive, Holt-Winters (b3), and Prophet (b5). Where does it rank?
- Add one more feature - a longer lag (t-14) or a rolling std - and see whether it helps or hurts.
- Deliberately introduce a leak (drop the
shift(1)from a rolling feature), re-score, and watch the MAPE fall implausibly. Then fix it. - Optional: read the sklearn
HistGradientBoostingRegressordocs and skim the M5 competition summary.
Sources covered (cited)
Taught from the gradient-boosting documentation and the M-competition evidence. This session covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · How does ML forecasting reframe the problem?
The series becomes an ordinary table - features (lags, rolling stats, calendar) map to a target (the future value) - and any regressor, usually gradient boosting, learns it.
2 · Why write s.shift(1).rolling(7).mean() instead of s.rolling(7).mean()?
Without the shift, the rolling mean includes the value you are trying to predict. That is leakage: the score looks great in testing and collapses live.
3 · What is the difference between recursive and direct multi-step forecasting?
Recursive uses one model and feeds each prediction back as a lag (errors can compound); direct trains one model per horizon (no compounding, but H models to maintain).