How this track works
Ten hands-on sessions, one running series: Lumen, whose daily website sessions carry a weekly rhythm - and, hidden inside, a bot-driven spike, an outage-driven drop, a contextual dip, and a sustained odd run. You will hunt those with every family of detector in real Python - statistical, distance and density, isolation, one-class, time-series, and a look at deep methods - then learn to measure honestly, serve a detector, and keep it from crying wolf. Tonight lays the foundation: what an anomaly actually is, why rare events break your intuition, and the baseline detector every later method must beat. A running note: most real detection is unsupervised, labels are scarce, and accuracy is a liar on rare events - so we measure with precision and recall from day one.
Three kinds of anomaly 6 min live
Not every anomaly looks the same, and the type decides which detector can see it. The canonical survey (Chandola, Banerjee & Kumar, 2009) splits them into three: a point anomaly stands out on its own, a contextual anomaly is only odd for its context, and a collective anomaly is a run of points that together are wrong.
LiveWhy the type decides the method3 min▶
The type is not academic - it tells you which detector can even see the anomaly:
- Point - a single value far from the rest. A global z-score or IQR catches these easily. The bot spike in Lumen is a point anomaly.
- Contextual - a value that is normal in the whole dataset but wrong for its context (a weekday sitting at a weekend-low level). Global methods miss it; you must remove the seasonal context first.
- Collective - a sequence that is individually plausible but collectively abnormal (a sustained, unexplained run). Needs methods that look at windows, not single points.
Fraud is usually contextual or collective. A $500 purchase is not a point anomaly - plenty of people spend that. It is anomalous in context: this card, this hour, this location, after this sequence of small tests. That is why simple threshold rules miss real fraud, and why we spend this whole course past the global z-score.
Why accuracy lies 5 min live
Anomalies are rare by definition - often well under 1% of the data. That rarity breaks the metric everyone reaches for first. A detector that flags nothing at all is 99.5% accurate and completely useless. This is the base-rate problem, and it is why we never judge a detector by accuracy.
Self-studySupervised, semi-supervised, or unsupervised?2 min read▶
How much labelled truth you have decides the whole approach:
- Unsupervised - no labels; assume anomalies are rare and different, and score how much each point stands out. This is the common case, and most of this course.
- Semi-supervised - you have a clean sample of normal data only; learn its shape and flag departures (one-class methods, b6).
- Supervised - you have labelled anomalies. Rare and precious, and heavily imbalanced - a classification problem with special care (b9). Real labels usually arrive late, via investigation.
Our Lumen series is labelled only so we can measure honestly. In production you would rarely have those labels up front.
Build the labeled Lumen series ★ 12 min · everyone builds
Construct the running series and its ground-truth labels so every later detector can be scored.
Build a daily sessions series with a weekly rhythm on a datetime index.
Inject four anomalies: a point spike, a point drop, a contextual dip (a weekday at a weekend-low level), and a collective run.
Record the ground-truth anomaly indices in a label vector - this is what precision and recall are measured against.
Plot it. See how the point anomalies jump out and the contextual one hides in plain sight.
label - they run unsupervised on s. Keep that wall up, or you will fool yourself.
Ship the z-score baseline ★ 8 min · build your own
Before any clever detector, build the baseline it has to beat. A global z-score flags any day more than three standard deviations from the mean. It is three lines, it catches the obvious point anomalies, and it will completely miss the contextual one - which is exactly the lesson.
Compute the z-score of every day against the global mean and standard deviation.
Flag days with |z| above 3, and score precision, recall, and F1 against the labels.
Note what it misses - in b3 you will watch a seasonal detector catch the contextual anomaly this one cannot see.
Never skip the baseline. A three-sigma rule is the "just show the bestsellers" of anomaly detection - shockingly serviceable, and the honest bar. If your isolation forest or autoencoder cannot beat a z-score on F1 (b9), it is not earning its complexity. And note the trap you just felt: the z-score's own huge outliers inflate its standard deviation, blunting it against the smaller anomalies. Robust methods (b2) fix exactly that.
Try it yourself - this week ◐ 30-45 min total
- Build the labeled Lumen series and confirm its base rate. Feel how rare the anomalies are.
- Plot it and classify each injected anomaly by type (point, contextual, collective) by eye.
- Run the z-score baseline and record precision, recall, and F1. Which anomaly does it miss, and why?
- Compute the accuracy of a "flag nothing" detector on this series and see for yourself why accuracy is useless here.
- Optional: skim Chandola, Banerjee & Kumar (2009), "Anomaly Detection: A Survey" - the taxonomy this track follows.
Sources covered (cited)
Taught from the anomaly-detection canon. This session covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · A weekday whose traffic sits at a normal weekend-low level is which kind of anomaly?
The value is perfectly normal globally, but wrong for its context (a weekday). Global detectors miss it; you must remove the seasonal context first.
2 · Why is accuracy a bad metric for anomaly detection?
The base-rate problem: with anomalies under 1%, "flag nothing" is 99%+ accurate and useless. Precision and recall are the honest scoreboard.
3 · Why build the z-score baseline first?
A three-sigma rule is a serviceable baseline. If a complex model cannot beat it on F1, it is not earning its complexity - and it reveals why robust methods (b2) are needed.