learn-anomaly-detection-with-phoebe / Builder session 3 of 10
Learn Anomaly Detection with Phoebe · Builder track · Session 3 of 10

Rolling & seasonal detectors: flagging in context

Global thresholds are blind to time. A weekend low is normal; the same value on a Tuesday is an alarm. This session builds detectors that judge each point against its local and seasonal context - rolling mean and standard deviation, EWMA, and the STL-residual detector - on the Lumen series, and hands you a live detector where you switch method and sensitivity and watch precision and recall move.

🟡 Builder track Practitioners: analysts · data scientists · engineers Python 3.10+ · statsmodels 45 minutes
0-3 · Recap 3-18 · Rolling, EWMA, seasonal 18-42 · Build-along + the live detector 42-45 · Q&A
Part 0

From global to contextual

In b2 you built the statistical detectors - z-score, IQR, and robust MAD - and felt their limit: they judge every point against the whole dataset, so they miss anything that is only odd for its moment. Tonight you fix that. Rolling and seasonal detectors compare each point to what was normal right there - its recent neighbours, or its weekday - and that is what finally catches the contextual anomaly. At the end you drive a live detector that scores five methods in front of you.

Live - presented in session Self-study - read after class ★ Run it now (Python) Sources covered (cited)
★ What you walk out with today Rolling-window and EWMA detectors and an STL-residual detector built on Lumen, an intuition for why removing the seasonal context is the whole game, and a live in-browser detector that shows precision and recall trading off as you change method and sensitivity.
Part 1 · local context

Judge each point against its neighbours 6 min live

A rolling detector keeps a moving window of recent days and asks: is today far from the local mean, in units of the local standard deviation? Because the window travels with the series, it adapts to a drifting trend that would fool a global threshold. EWMA does the same with an exponentially-weighted memory instead of a hard window.

A local band travels with the series rolling window flagged The band adapts to the trend - but if seasonality lives inside the window, weekend dips still trip it.
🔍 Click to zoom - a rolling local band flags points that leave it
LiveRolling and EWMA, and where they still fail3 min▶
  • Rolling z-score - local mean and std over a trailing window; flags points beyond k local sigmas. Adapts to trend and slow drift.
  • EWMA - an exponentially-weighted moving average; recent points count more, no hard window edge. Smooth and responsive.
  • The catch - if the window spans the weekly cycle, the seasonal swing inflates the local std and the weekend dips look like anomalies. Rolling fixes trend, not seasonality. That is what the seasonal detector is for.
Real world

Rolling detectors are the backbone of ops monitoring. Most metric-alerting systems are, under the hood, a rolling mean and a band. They are cheap, streaming-friendly, and explainable - which is why they are everywhere. Their weakness is exactly the seasonal one you are about to fix.

Part 2 · seasonal context

Subtract the pattern, then flag the remainder 5 min live

The strongest simple detector on seasonal data does one clever thing: it removes the expected weekly pattern first. Decompose the series into trend, season, and residual (STL), and run your detector on the residual alone. Now a weekday-at-weekend-level jumps out, because against its own weekday it is far from normal.

Observed trend + season + noise (anomaly hidden) - Weekly season the expected pattern for each weekday = Residual flat and centered - now the contextual anomaly is the point that sticks out contextual anomaly Deseasonalize first, and a global threshold on the residual becomes a contextual detector.
🔍 Click to zoom - observed minus season leaves a residual you can threshold
Self-studyBuild the rolling and STL-residual detectors2 min read▶

Both are a few lines on the Lumen series from b1:

★ Run it now (Python) - rolling + STL-residualimport numpy as np, pandas as pd from statsmodels.tsa.seasonal import STL # 1) rolling z-score (local context, window = 10) mu = s.rolling(10, min_periods=4).mean() sd = s.rolling(10, min_periods=4).std() roll_z = (s - mu) / sd roll_flags = set(np.where(np.abs(roll_z) > 2.5)[0]) # 2) STL-residual detector (seasonal context) res = STL(s, period=7, robust=True).fit().resid med = res.median(); mad = (res - med).abs().median() or 1 mz = 0.6745 * (res - med) / mad # robust z on the residual stl_flags = set(np.where(np.abs(mz) > 3)[0]) for name, f in [("rolling", roll_flags), ("STL-residual", stl_flags)]: tp = len(f & set(truth)); fp = len(f - set(truth)); fn = len(set(truth) - f) p = tp/(tp+fp) if tp+fp else 0; r = tp/(tp+fn) if tp+fn else 0 print(f"{name:13s} P {p:.2f} R {r:.2f} (caught {tp}/{len(truth)})")

The STL-residual detector is the one that finally catches the contextual anomaly - because on its own weekday, it is far from normal.

Demo · drive it yourself

The live detector ★ real math in your browser

This is a genuine detector, not a mock-up. It runs z-score, IQR, robust MAD, rolling-window, and seasonal-residual detectors on a synthetic Lumen series with four injected anomalies, flags points in red, rings the true anomalies, and scores precision, recall, and F1 live. Switch the method and the sensitivity, and watch the trade-off. Note which anomalies each method can and cannot see - the global ones catch the obvious spikes and miss the contextual dip; the seasonal-residual detector catches them all.

Sensitivity is a business decision Turn the sensitivity up and recall rises but false alarms flood in; turn it down and precision rises but you miss real events. There is no free lunch - where you set it depends on the cost of a missed anomaly versus the cost of a false alarm. That is a conversation with the business, not just a hyperparameter.
Build-along

Compare all five on Lumen ★ 8 min · build your own

Score every detector so far side by side and see the seasonal one win.

Run the z-score, IQR, MAD, rolling, and STL-residual detectors on the same Lumen series.

Score precision, recall, and F1 for each against the labels.

Confirm the seasonal-residual detector has the best F1, and identify which anomaly each simpler method misses.

★ Run it now (Python) - the scoreboardimport numpy as np, pandas as pd def score(flags): f = set(flags); t = set(truth) tp = len(f & t); fp = len(f - t); fn = len(t - f) p = tp/(tp+fp) if tp+fp else 0; r = tp/(tp+fn) if tp+fn else 0 return p, r, (2*p*r/(p+r) if p+r else 0) zf = np.where(np.abs((s-s.mean())/s.std()) > 3)[0] res = __import__("statsmodels.tsa.seasonal", fromlist=["STL"]).STL(s, period=7, robust=True).fit().resid med = res.median(); mad = (res-med).abs().median() or 1 sf = np.where(np.abs(0.6745*(res-med)/mad) > 3)[0] for name, fl in [("z-score", zf), ("STL-residual", sf)]: print(f"{name:13s} P {score(fl)[0]:.2f} R {score(fl)[1]:.2f} F1 {score(fl)[2]:.2f}") # STL-residual beats the global z-score - it sees the contextual anomaly
Real world

Deseasonalizing is the highest-leverage move in time-series detection. Before reaching for isolation forests or autoencoders, remove the known pattern - weekday, holiday, campaign - and detect on what is left. Half of "our model is too noisy" turns out to be a seasonal pattern nobody subtracted.

Homework

Try it yourself - this week ◐ 30-45 min total

Source material

Sources covered (cited)

Taught from the detection canon. This session covers:

✓STL decomposition (Cleveland et al.) via statsmodels; robust residual thresholdingParts 1-2 · rolling, EWMA, and seasonal-residual detection
◐Chandola, Banerjee & Kumar (2009) - contextual anomaliesPart 2 · why context must be removed before thresholding
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · What does a rolling-window detector fix that a global z-score does not?

The window travels with the series, so it adapts to a drifting trend. It does NOT fix seasonality - a weekly swing inside the window still trips it.

2 · Why does the STL-residual detector catch the contextual anomaly?

Subtracting the seasonal component leaves a flat residual. Against its own weekday, the contextual point now stands out - which it never did globally.

3 · Turning sensitivity up does what?

Sensitivity is a precision/recall trade-off. More sensitive = catch more real anomalies but flag more false ones. Where you set it is a business decision.

Builder session 3 cheat sheet · pin this

Rolling z-scorelocal mean + std over a trailing window. Adapts to trend and drift.
EWMAexponentially-weighted memory; recent points count more, no hard window edge.
Rolling's limitseasonality inside the window inflates the local std - weekend dips look anomalous.
STL-residualdecompose, then threshold the residual. Removes the pattern, so context anomalies show.
Deseasonalize firstthe highest-leverage move: subtract the known pattern before detecting.
Robust thresholduse median + MAD on the residual, not mean + std - outliers do not inflate it.
Sensitivity = trade-offup = more recall + more false alarms; down = more precision + more misses.
Next sessionb4 - distance & density: kNN distance and the Local Outlier Factor.