learn-anomaly-detection-with-phoebe / Builder session 2 of 10
Learn Anomaly Detection with Phoebe · Builder track · Session 2 of 10

Statistical detectors: the threshold family

The oldest idea in the book: draw a line, flag whatever crosses it. This session builds the three classic threshold detectors on the Lumen series - the z-score / three-sigma rule and the fatal flaw hiding inside it, the IQR / Tukey fences that quartiles make robust, and the median-absolute-deviation (MAD) rule that resists outliers best of all - then proves, in code, why the plain z-score masks the very anomalies you care about.

🟢 Builder track Practitioners: analysts · data scientists · engineers Python 3.10+ · numpy / scipy 45 minutes
0-3 · Recap 3-18 · Three thresholds + masking 18-42 · Build-along: score + unmask 42-45 · Q&A
Part 0

Where we are

In b1 we built the labeled Lumen series and shipped a z-score baseline that caught the giant spike and drop but missed everything subtler. Tonight we stay in the world of global thresholds - a single cut line for the whole series - but we make it honest. The plain z-score has a self-inflicted wound: the big anomalies it is meant to catch inflate its own standard deviation, and that fattened band swallows the moderate anomalies whole. This is masking. The fix is to build thresholds on statistics the outliers cannot move - quartiles and the median - which gives us the IQR fences and the robust MAD rule. Three detectors, one line of defence, and a clear rule for when each one fits.

Live - presented in session Self-study - read after class ★ Run it now (Python) Sources covered (cited)
★ What you walk out with today Three global detectors coded and scored on Lumen - z-score, IQR / Tukey fences, and robust MAD - a working feel for which one to reach for, and a hands-on demonstration of masking: watch a moderate anomaly stay hidden until you strip the giant outliers that were inflating the threshold.
Part 1 · the threshold family

Three ways to draw the line 7 min live

All three detectors flag a point that sits too far from the centre. They differ in how they measure the centre and the spread - and that difference is everything, because one uses statistics the outliers can drag around, and two do not.

One outlier, three different lines giant outlier median mean, dragged right mean ± 3σ outlier inflates σ, so the band swallows it - flags nothing Q1 − 1.5·IQR ... Q3 + 1.5·IQR outlier falls outside → flagged median ± k·MAD tightest, most outlier-resistant red = anomaly Quartiles and the median barely move when one point explodes - so fences built on them stay tight while the mean±3σ band balloons.
🔍 Click to zoom - the same outlier seen by three thresholds
LiveThe three detectors, and when each fits3 min▶
  • z-score / three-sigma - flag |x − mean| / std > 3. Fast, assumes a roughly bell-shaped spread. Fatal flaw: the mean and std are computed from data that includes the anomalies, so big outliers inflate both and the rule masks the moderate ones. Use only when you are confident outliers are few and small, or as the baseline to beat.
  • IQR / Tukey fences - flag anything below Q1 − 1.5·IQR or above Q3 + 1.5·IQR, where IQR = Q3 − Q1. Quartiles ignore the extreme tails, so the fences do not widen when an outlier appears. A solid, distribution-light default.
  • robust MAD - modified z-score 0.6745·(x − median) / MAD, where MAD is the median of |x − median|; flag |modified z| > 3.5. The median and MAD have the highest breakdown point of the three - roughly half the data would have to be corrupt to move them - so this is the most outlier-resistant threshold.
Real world

Server latency dashboards learned this the hard way. A plain mean+3σ alert on p99 latency goes quiet during the exact incidents it should catch: one monster spike lifts the mean and fattens the band, so the sustained-but-smaller degradation that follows sits comfortably inside it. Teams that switched to median+MAD or IQR fences started catching the slow bleeds. Same data, robust statistic, very different alerts.

Part 2 · the fatal flaw

Masking and swamping 5 min live

The plain z-score does not just miss subtle anomalies by bad luck - it hides them by construction. Because a giant outlier inflates the standard deviation, the three-sigma band grows so wide that moderate anomalies fall inside it. Strip the giant outlier out and the band snaps back, and the hidden anomaly suddenly crosses the line. This is masking; its mirror image, swamping, is when outliers stretch the band so far that normal points get wrongly flagged.

Same day 71 - masked, then caught With the giant spike in the data +3σ threshold (high) spike day 71 hides below Spike removed, σ shrinks +3σ threshold (drops) day 71 caught The value of day 71 never changed. Only the giant outlier's grip on the standard deviation did - that grip is what was hiding it. Robust MAD and IQR never had this problem, because a lone spike cannot move a median or a quartile.
🔍 Click to zoom - masking: the outlier inflates the very band meant to catch its neighbours
Self-studyWhy 1.5, 3, and 3.5? The magic constants2 min read▶

The thresholds are conventions, not laws - tune them to your precision/recall need:

  • 3σ - for a normal distribution about 99.7% of data sits inside ±3σ, so ~0.3% is flagged. If your data has fat tails, expect more false alarms.
  • 1.5·IQR - Tukey's original choice for a "boxplot outlier". 3·IQR marks "far out" points. Lower it to flag more, raise it to flag less.
  • 0.6745 and 3.5 - 0.6745 rescales MAD so that, on normal data, the modified z-score matches an ordinary z-score. Iglewicz and Hoaglin recommend flagging above 3.5. All three constants are levers on the same alert-fatigue trade-off.
Demo 1 of 2

Three detectors, scored side by side ★ 12 min · everyone builds

Rebuild the Lumen series, run all three thresholds, and score each against the truth. Watch the robust pair beat the plain z-score on the same data.

Rebuild the labeled series from b1 so the notebook runs standalone.

Compute z-score flags, IQR / Tukey flags, and MAD modified-z flags.

Print each detector's flagged days and its precision, recall, and F1 against truth.

★ Run it now (Python) - z-score vs IQR vs MADimport pandas as pd, numpy as np # --- rebuild the labeled Lumen series (from b1) so this runs standalone --- idx = pd.date_range("2026-01-01", periods=90, freq="D") season = np.array([30,42,48,40,22,-95,-110]); i = np.arange(90) y = 900 + 1.2*i + season[i % 7] + 10*np.sin(i*1.7) + 7*np.cos(i*0.6) y = y.astype(float) truth = [20, 45, 63, 70, 71, 72] y[20] += 430; y[45] -= 380 # point spike, point drop y[63] = y[63] - season[63 % 7] + season[6] - 40 # contextual dip y[[70,71,72]] += [175,190,170] # collective run s = pd.Series(np.round(y), index=idx, name="sessions") def score(flags, name): f = set(int(k) for k in flags); t = set(truth) tp = len(f & t); fp = len(f - t); fn = len(t - f) p = tp/(tp+fp) if tp+fp else 0 r = tp/(tp+fn) if tp+fn else 0 f1 = 2*p*r/(p+r) if p+r else 0 print(f"{name:9s} caught {tp}/{len(t)} P {p:.2f} R {r:.2f} F1 {f1:.2f} {sorted(f)}") # 1) plain z-score / three-sigma z = (s - s.mean()) / s.std() score(np.where(np.abs(z) > 3)[0], "z-score") # 2) IQR / Tukey fences - quartiles are robust to outliers q1, q3 = s.quantile(0.25), s.quantile(0.75); iqr = q3 - q1 lo, hi = q1 - 1.5*iqr, q3 + 1.5*iqr score(np.where((s < lo) | (s > hi))[0], "IQR") # 3) robust MAD - modified z-score med = s.median(); mad = (s - med).abs().median() mz = 0.6745 * (s - med) / mad score(np.where(np.abs(mz) > 3.5)[0], "MAD") # z-score caught 2/6 P 1.00 R 0.33 F1 0.50 [20, 45] # IQR caught 5/6 P 1.00 R 0.83 F1 0.91 [20, 45, 70, 71, 72] # MAD caught 4/6 P 1.00 R 0.67 F1 0.80 [20, 45, 71, 72]
Read the scoreboard, not the vibe All three keep precision at 1.00 - nothing false-flagged. The story is in recall: the plain z-score catches only the two giant point anomalies (0.33), while IQR reaches 0.83 and MAD 0.67 by also flagging the moderate collective run. Same series, same three-line effort - the robust statistic is what buys the extra catches.
Demo 2 of 2

Watch masking happen ★ 8 min · build your own

Now prove the flaw directly. The collective-run days sit hidden inside the z-score band. Remove the two giant point anomalies that were inflating the standard deviation, recompute, and watch the hidden days climb across the threshold - value unchanged, only the band moved.

Print the z-score of the collective-run days with the giant spike and drop still in the series - all below 3, all masked.

Drop the two giant point anomalies and recompute the z-score.

See the standard deviation shrink and days 71 and 72 cross the threshold - the definition of masking.

★ Run it now (Python) - unmasking a hidden anomalyimport numpy as np # (continues from Demo 1: s and truth already defined) z = (s - s.mean()) / s.std() print("WITH the giant spike + drop:") print(f" std = {s.std():.1f}") for d in [70, 71, 72]: print(f" day {d}: z = {z.iloc[d]:.2f} {'< 3 (masked)' if abs(z.iloc[d]) < 3 else '> 3 (caught)'}") # strip the two giant point anomalies that inflate the std s2 = s.drop(s.index[[20, 45]]) z2 = (s2 - s2.mean()) / s2.std() print("\nWITHOUT the giant spike + drop:") print(f" std = {s2.std():.1f} (it shrank)") for d in [70, 71, 72]: zd = z2.loc[s.index[d]] print(f" day {d}: z = {zd:.2f} {'< 3 (masked)' if abs(zd) < 3 else '> 3 (caught)'}") # WITH: std 94.9 -> day 71 z 2.87, day 72 z 2.69 (all masked) # WITHOUT: std 83.1 -> day 71 z 3.27, day 72 z 3.07 (now caught) # the days never changed value - the giant outliers were hiding them
Real world

This is why "just use a z-score" quietly fails in production. The loudest anomaly in your window sets the threshold for every other point, so the biggest incident actively conceals the medium ones around it. You cannot see masking by staring at the output - a masked anomaly leaves no trace. You catch it by defaulting to robust statistics (MAD, IQR) or by removing known outliers before you fit the threshold. When in doubt, reach for the median.

Homework

Try it yourself - this week ◐ 30-45 min total

Source material

Sources covered (cited)

Taught from the statistical-detection canon. This session covers:

✓scikit-learn "Novelty and Outlier Detection" user guide + statsmodels / scipyParts 1-2 · z-score, IQR / Tukey fences, robust MAD in practice
◐Chandola, Banerjee & Kumar (2009), "Anomaly Detection: A Survey," ACM Computing SurveysPart 1 · statistical technique family, parametric vs non-parametric thresholds
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · What is the fatal flaw of the plain z-score / three-sigma rule?

Masking: the mean and std are computed from data that includes the anomalies, so a giant outlier fattens the ±3σ band until the moderate anomalies fall inside it.

2 · The Tukey / IQR fences flag a point outside which range?

The fences are built on quartiles, which ignore the extreme tails - so a lone outlier cannot widen them the way it widens a standard deviation.

3 · Why is the MAD-based modified z-score the most outlier-resistant of the three?

Median and MAD barely budge when a few points explode, so the threshold built on them stays honest - unlike the mean and std, which any single giant outlier can drag.

Builder session 2 cheat sheet · pin this

z-score / 3σflag |x − mean|/std > 3. Fast, but its own outliers inflate the band. Baseline only.
IQR / Tukeyflag below Q1 − 1.5·IQR or above Q3 + 1.5·IQR. Quartiles make it robust.
Robust MADflag |0.6745·(x − median)/MAD| > 3.5. Median + MAD resist outliers best.
Maskinga giant outlier inflates the std, widening the band so moderate anomalies hide inside.
Swampingthe mirror: outliers stretch the band so far that normal points get wrongly flagged.
On Lumenz-score F1 0.50, IQR F1 0.91, MAD F1 0.80 - robust statistics buy the extra catches.
Breakdown pointhow much of the data can be corrupt before a statistic lies. Median ≈ 50%, mean ≈ 0%.
Next sessionb3 - rolling & seasonal: judge each point against its local context, not the whole series.