Builder session 7 · Deep forecasting - learn timeseries forecasting with phoebe

Part 0

What this session is - and is not

Be clear before we start: this is a survey, not a build. We map the deep-forecasting landscape and cite the primary papers, and the Python here is illustrative sketch code - clearly labelled - that shows how these models are fed and called, never a model trained on real data. Deep forecasters need lots of data, tooling (GluonTS, PyTorch-Forecasting, Darts), and compute; full training belongs in a dedicated deep-learning course. And the honest headline from the M-competitions: on a single short series, deep models often do not beat a well-tuned ETS or ARIMA. Reach for them when the problem shape earns it.

Live - presented in session Self-study - read after class ★ Illustrative sketch (not trained) Sources covered (cited)

★ What you walk out with today A clear mental map of why deep forecasting exists (many related series, long horizons, exogenous inputs, one global model across series), the three landmark models and what each one is for, a sketch of the sliding-window dataset that feeds them, the API shape of a library call, and - most usefully - a sober sense of when a deep model is worth it and when seasonal-naive plus ETS already wins.

⚠ Re-verify - this area moves fast Deep forecasting changes quickly. The three papers below are the durable landmarks, but libraries, defaults, and new architectures shift often. Treat the sketch code as pseudocode for the current API, and check each library's live docs before you rely on it.

Part 1 · covers DeepAR / global-model motivation

Why a deep, global model at all 6 min live

Everything up the ladder so far - ETS, ARIMA, Prophet - fits one model per series. That is fine for Lumen's single daily-sessions line. But when you have thousands of related series (every product, every store, every region), deep models offer something classical methods cannot: a single global model that learns shared patterns across all of them, borrowing strength from short and new series. They also handle long horizons, many exogenous inputs (price, promotions, weather), and produce probabilistic outputs (quantiles), not just a point.

🔍 Click to zoom - one model per series vs one global model across all

LiveThe four things deep models are actually good at3 min▶

Deep forecasting is not "better ARIMA". It solves a different shape of problem:

Many related series. Train once, globally, and every series - including short and brand-new ones - benefits from patterns learned elsewhere.
Long horizons. Multi-horizon architectures forecast the whole path at once rather than recursing one step at a time.
Exogenous inputs. Known-future covariates (promotions, holidays, price) and static features (category, region) go straight in.
Probabilistic output. Quantiles or a full distribution, so the prediction interval is a first-class citizen, not an afterthought.

Real world

Lumen is the wrong-shaped problem - on purpose. One daily-sessions series is exactly where a global deep model has nothing to borrow from. If Lumen grew to forecast every SKU across every market daily, that fleet of related series is when DeepAR or TFT start to pay off. Match the model to the shape of the problem, not to the hype.

Part 2 · covers DeepAR, N-BEATS, TFT

Three landmarks worth knowing 5 min live

Three papers anchor the modern deep-forecasting map. You do not need to implement them tonight - you need to know what each is for, so you can pick the right one (or decide none of them beats your baseline).

Model	What it is	Why it matters
DeepAR Salinas et al. 2020	An autoregressive RNN trained globally across many series, predicting a distribution (e.g. quantiles), not a point.	The reference for probabilistic, at-scale forecasting when you have a large panel of related series.
N-BEATS Oreshkin et al. 2020	A pure deep residual stack of basis-expansion blocks - no RNN, no attention - with an interpretable trend/seasonality variant.	Strong accuracy from a simple, fast architecture; the interpretable blocks echo classical decomposition.
TFT Lim et al. 2021	Temporal Fusion Transformer: attention over past inputs plus known-future covariates, multi-horizon, with built-in interpretability.	The go-to when you have rich known-future inputs and want to see which ones the model leans on.

🔍 Click to zoom - past window + known-future inputs → model → multi-horizon quantile forecast

Self-studyThe tooling you would actually reach for2 min read▶

Nobody hand-codes these from scratch for a project. The ecosystem does the heavy lifting:

GluonTS - the home of DeepAR and a broad probabilistic-forecasting toolkit.
PyTorch-Forecasting - production-friendly TFT, N-BEATS, DeepAR on top of PyTorch Lightning.
Darts - a single, sklearn-like API across classical and deep models, easiest for a first sketch.

Baseline first, always Before any of these, you already have the seasonal-naive baseline (b1) and ETS/ARIMA/Prophet (b3-b5). A deep model only earns its complexity if it beats those on a backtest (b8) - not on one lucky holdout.

Demo 1 of 2 · illustrative

How a deep model is fed: the sliding-window dataset ★ 12 min · sketch, not a model

You do not train anything here - you build the dataset shape deep models consume. A window of past values becomes the input X, the next H values become the target y, and windows are pooled across many series into one global training set. This numpy sketch runs, but it is here to build intuition, not a forecaster.

Slide a fixed lookback window across a series; each position yields one (past window → next H) training row.

Repeat for many related series and stack the windows into one pool - this is what "global" means.

Inspect the shapes: thousands of rows, each mapping a past window to a multi-step target.

★ Illustrative sketch (Python) - how deep models are fed, NOT trainedimport numpy as np # ILLUSTRATIVE ONLY: this builds the DATASET SHAPE a deep/global model eats. # It trains nothing - it shows how past windows become (X, y) rows, pooled. def make_windows(series, lookback=28, horizon=14): X, y = [], [] for t in range(lookback, len(series) - horizon + 1): X.append(series[t - lookback:t]) # past window -> inputs y.append(series[t:t + horizon]) # next H values -> targets return np.array(X), np.array(y) # pool windows from MANY related series into ONE training set (global model) rng = np.random.default_rng(7) series_bank = [1000 + 3*np.arange(180) + rng.normal(0, 20, 180) for _ in range(50)] # 50 Lumen-like daily series Xs, ys = zip(*(make_windows(s) for s in series_bank)) X = np.concatenate(Xs); y = np.concatenate(ys) print("global training set:", X.shape, "->", y.shape) # (~many, 28) -> (~many, 14)

Real world

This is where leakage sneaks back in. Every window must respect time: X is strictly before y, and when you split for evaluation you split by date, not by shuffling windows. A shuffled window split leaks the future exactly like a shuffled row split (b1) - just harder to spot.

Demo 2 of 2 · API sketch

The shape of a deep fit: an N-BEATS call in Darts ★ 8 min · API sketch only

This is the shape of a deep-model fit, so the API is not a mystery when you meet it. It will not run in this notebook as-is - it needs the library installed, enough data, and compute (ideally a GPU). Read it as annotated pseudocode for the real thing.

Wrap the Lumen series in the library's TimeSeries object and split it by time.

Instantiate a model with a past window (input_chunk_length) and a multi-horizon output (output_chunk_length).

.fit then .predict - then, crucially, compare against the seasonal-naive baseline from b1.

★ API sketch (Python) - needs the library + data + compute; not run here# pip install darts # (or u8darts) - NOT run in this notebook # This is the SHAPE of a deep fit on the Lumen series, nothing more. from darts import TimeSeries from darts.models import NBEATSModel ts = TimeSeries.from_series(s) # the Lumen daily series from b1 train_ts, test_ts = ts[:-H], ts[-H:] # split by TIME, never shuffle model = NBEATSModel(input_chunk_length=28, # past window fed in output_chunk_length=H, # multi-horizon output n_epochs=50) # needs real compute + data model.fit(train_ts) # training: GPU/CPU time here forecast = model.predict(H) # H-step forecast # the ONLY verdict that matters: did it beat seasonal-naive on a backtest (b8)? print(forecast.values().ravel())

Complexity has to earn it A DeepAR, N-BEATS, or TFT that cannot beat your ETS or seasonal-naive on a rolling-origin backtest is not a forecaster - it is a liability: slower, harder to explain, more ways to break in production. Next session gives you the honest scoreboard to make that call.

Homework

Try it yourself - this week ◐ 30-45 min total

Build the sliding-window dataset (Demo 1) on your own daily series. Confirm every X window sits strictly before its y target - print a few index ranges to prove it.
Write one paragraph: does your problem have many related series, long horizons, or rich known-future inputs? If not, say honestly why a deep model is probably overkill.
Skim one paper abstract - DeepAR (arXiv:1704.04110), N-BEATS (arXiv:1905.10437), or TFT (arXiv:1912.09363) - and note the one problem it was built to solve.
Optional: install Darts and run the N-BEATS sketch on a longer synthetic series, then compare its holdout MAPE to your seasonal-naive baseline from b1.

Source material

Sources covered (cited)

A survey taught from the primary papers. Re-verify before relying on any library API - this area moves fast. This session covers:

✓Salinas et al. (2020), DeepAR - Int. J. Forecasting 36(3):1181-1191; arXiv:1704.04110Part 1-2 · global, probabilistic, autoregressive RNN

◐Oreshkin et al. (2020), N-BEATS - ICLR 2020; arXiv:1905.10437Part 2 + Demo 2 · deep residual basis expansion, interpretable

◐Lim et al. (2021), Temporal Fusion Transformer - Int. J. Forecasting 37(4):1748-1764; arXiv:1912.09363Part 2 · attention over past + known-future, multi-horizon

◐M-competitions (Makridakis, M4/M5)Framing · simple methods are hard to beat on a single series

Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · What is the main advantage of a global deep model?

Global models learn shared patterns across a whole panel of series. That is their edge - not raw accuracy on one short line, where classical methods usually win.

2 · Which model is built to use known-future covariates with attention, multi-horizon?

TFT (Lim et al. 2021) applies attention over past inputs plus known-future covariates and forecasts multiple horizons at once, with built-in interpretability.

3 · When is a deep model probably the wrong tool?

The M-competitions are blunt: on a single short series, complexity rarely pays. Deep earns its place with many related series, long horizons, and rich inputs.

Builder session 7 cheat sheet · pin this

Survey, not a buildwe map deep models + cite the papers; sketch code shows how they are fed/called, nothing trained.

Why deepmany related series, long horizons, exogenous inputs, probabilistic (quantile) output - a global model.

DeepARautoregressive RNN, trained globally, predicts a distribution - the at-scale probabilistic reference.

N-BEATSpure deep residual basis expansion; interpretable trend/seasonality blocks; fast, no RNN/attention.

TFTattention over past + known-future inputs, multi-horizon, interpretable.

ToolingGluonTS, PyTorch-Forecasting, Darts - never hand-code these for a project.

M-competition cautionon one short series, deep often loses to good ETS/ARIMA. Beat the baseline on a backtest or drop it.

Next sessionb8 - rolling-origin backtesting + MAPE/RMSE/MASE + interval coverage, the honest scoreboard.