The one idea: reconstruction error
Almost every deep anomaly detector rests on a single move. Train a model to compress and then rebuild normal data. Force the information through a narrow bottleneck and the model can only learn to reproduce the patterns it saw most - the normal ones. Feed it a normal point and it rebuilds it well; feed it an anomaly and it rebuilds it badly, because it never learned that shape. The size of that rebuild miss - the reconstruction error - is the anomaly score. That is the whole thesis, and it survives from linear PCA all the way to deep autoencoders.
truth), an API sketch of a Keras autoencoder you can carry into a real project, and an honest map of LSTM-autoencoders, deep one-class, and VAEs - with the frank caveat that on a single low-dimensional series they often do not beat an isolation forest or a good seasonal detector.
Compress, rebuild, measure the miss 6 min live
An autoencoder is two halves. The encoder squeezes an input into a small latent vector; the decoder expands that vector back toward the original. Train it only on normal data with a reconstruction loss, and the bottleneck forces it to keep just the structure that normal data shares. Anomalies do not share that structure, so they come back distorted - and that distortion is the score.
LiveThe deep family, in one breath each3 min▶
- Autoencoder - the base case above. Reconstruction error is the score.
- LSTM-autoencoder - the same idea for sequences. It reconstructs a whole window of a time series; a window it rebuilds poorly is an anomalous stretch. This is the natural deep cousin of the forecast-residual detector in b7.
- Deep one-class / VAE - a one-line nod: Deep SVDD learns a tight boundary around normal in a learned space, and a variational autoencoder scores by reconstruction probability. Same spirit, different machinery.
Deep is not a free upgrade. Autoencoders need a lot of clean normal data, careful architecture and threshold tuning, and compute - and on a single low-dimensional series they frequently do not beat an isolation forest (b5) or a good seasonal detector (b3, b7). Reach for deep methods when the data is genuinely high-dimensional or complex - images, logs, multivariate sensor streams - where the structure a shallow model cannot capture is exactly where anomalies hide.
Reconstruction error over time 5 min live
Once trained, the detector is simple: push every point (or window) through, measure how far the rebuild lands from the original, and threshold that error. Plot the error over time and the anomalies announce themselves - normal points hug a low baseline, anomalies spike above the line. The threshold is the usual precision/recall decision, and setting it on a clean stretch of normal errors is the whole game.
Self-studyThe three ways this bites you2 min read▶
Reconstruction-error detectors fail in predictable ways - know them before you ship one:
- Contaminated training - if anomalies leak into the "normal" training set, the model learns to rebuild them too, and stops flagging them. Curate the normal sample.
- Over-capacity - a bottleneck that is too wide (or a model too powerful) learns to rebuild everything, anomalies included, and the error signal collapses. The squeeze is the point.
- Threshold drift - as the normal regime shifts, yesterday's error threshold turns into today's false-alarm generator. Re-estimate it on fresh normal data.
PCA reconstruction error ★ 12 min · everyone builds
The concept made concrete, and actually runnable. PCA is a linear autoencoder: project each window onto a few components, then project back. Normal windows rebuild well; the anomalous ones rebuild badly, and the squared rebuild error is the anomaly score - no deep-learning library required.
Rebuild the labeled Lumen series (from b1) so the notebook runs standalone.
Turn the series into a feature matrix of overlapping 7-day windows.
Fit PCA, reconstruct, and take the per-window squared error as the score.
Threshold robustly and score precision, recall, and F1 against truth.
Autoencoder API sketch ○ read-through · not executed
What the deep version looks like in code. This is an API sketch - the shape of a Keras autoencoder so you recognise it in the wild - not a trained model. It needs the library, a lot of clean normal data, and compute, none of which we run here.
Build a symmetric encoder / decoder with a narrow bottleneck.
Train it to reconstruct normal-only data with a reconstruction loss.
Score by per-sample reconstruction error; threshold on a held-out normal set.
This area moves fast - treat any survey as a snapshot. The map here follows Pang, Shen, Cao & van den Hengel (2021), "Deep Learning for Anomaly Detection: A Review" (ACM Computing Surveys 54(2); arXiv:2007.02500). Before you commit to a deep method for a real system, re-verify the current state of the art and, just as importantly, benchmark it against the shallow detectors from this course - deep should have to earn its complexity on your data, not win by reputation.
Try it yourself - this week ◐ 30-45 min total
- Run the PCA-reconstruction detector and record its precision, recall, and F1. Which anomalies does it catch, and which slip through?
- Sweep the number of PCA components from 1 to 5. Watch the error signal collapse as the bottleneck widens toward "rebuild everything."
- Put PCA-recon head to head with your isolation forest (b5) and forecast-residual detector (b7) on the same series. Does the deeper idea actually win here?
- Read the abstract and taxonomy figure of Pang et al. (2021) and list which method families the paper flags as strongest for high-dimensional data.
- Optional: sketch how you would curate a clean "normal-only" training set for a real autoencoder - where would contamination sneak in?
Sources covered (cited)
A surveyed map, not a from-scratch build. This session covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · In a reconstruction-based detector, what is the anomaly score?
The model learns to rebuild normal data through a bottleneck. Anomalies were never learned, so they come back distorted; that distortion (the reconstruction error) is the score.
2 · When are deep methods most worth the extra cost?
On one low-dimensional series, an isolation forest or seasonal detector often matches or beats an autoencoder. Deep pays off where the structure is too complex for a shallow model.
3 · Why does PCA reconstruction error illustrate the deep idea?
Projecting onto a few components then back is exactly encode-then-decode. A deep autoencoder just replaces that linear projection with a nonlinear network.