Builder session 6 · One-class & covariance - learn anomaly detection with phoebe

Part 0

When you can trust your normal data

Every detector so far ran fully unsupervised on a mixed series - it assumed anomalies were rare and let them stand out. Tonight is different. When you have a clean window you can trust as normal - a quiet stretch with no incidents - you can learn its shape directly and flag any future point that does not fit. This is the semi-supervised, normal-only approach the survey calls novelty detection. Two classic methods lead: the One-Class SVM (Scholkopf et al., 2001) draws a flexible boundary around the normal region, and the Elliptic Envelope / Minimum Covariance Determinant (Rousseeuw & Van Driessen, 1999) fits a robust Gaussian and measures how far each point sits from its centre.

Live - presented in session Self-study - read after class ★ Run it now (Python) Sources covered (cited)

★ What you walk out with today A One-Class SVM fit on a clean normal slice of Lumen and an Elliptic Envelope reading Mahalanobis distances, both scored with precision and recall, and a firm sense of which to reach for: OC-SVM for flexible non-Gaussian normals, Elliptic Envelope when normal is a single roughly-elliptical blob.

Part 1 · covers Scholkopf et al. 2001

One-Class SVM: a boundary around normal 6 min live

The One-Class SVM learns from normal data only. It maps the points into a kernel space and finds the tightest boundary that encloses most of them, leaving a small fraction outside. Anything that lands outside that boundary at test time is flagged. Because it uses a kernel, the boundary can be wavy and non-Gaussian - it wraps whatever shape the normal cloud actually has. The nu parameter bounds the fraction of training points allowed outside, so it doubles as your tolerance for outliers.

🔍 Click to zoom - the learned boundary, and the points that fall outside it

LiveHow nu and gamma shape it3 min▶

nu - an upper bound on the fraction of training points that may sit outside the boundary, and a lower bound on the fraction of support vectors. Roughly, your assumed outlier rate. Higher nu = a tighter boundary that flags more.
gamma - the RBF kernel width. Large gamma makes a wiggly boundary that hugs every point (overfits); small gamma makes it smooth and loose.
Scale your features first - the RBF kernel is distance-based, so an unscaled feature dominates. Always standardize before fitting.

Real world

One-Class SVM shines when normal is complex but well-sampled. Think machine-health monitoring: you have months of clean sensor readings from a healthy machine and want to flag any departure. You do not know what failure looks like, only what health looks like - the textbook novelty-detection setup. Its cost is tuning nu and gamma, and it slows down on very large training sets.

Part 2 · covers Rousseeuw & Van Driessen 1999

Elliptic Envelope: a robust Gaussian 5 min live

If your normal data is roughly one Gaussian blob, you can do something cleaner. Fit a mean and a covariance to the normal cloud and you have described an ellipse; a point's Mahalanobis distance - its distance from the centre, scaled by the covariance - says how far it is in the ellipse's own units. Flag the far ones. The Minimum Covariance Determinant makes the fit robust: it estimates the mean and covariance from the tightest subset of points, so a few outliers cannot warp the ellipse. It is elegant and fast - but it lives or dies on the Gaussian assumption.

🔍 Click to zoom - a robust ellipse fits Gaussian data and mis-fits a two-cluster shape

Self-studyOC-SVM vs Elliptic Envelope - which when?2 min read▶

Both learn normal only, but they assume different things about its shape:

Elliptic Envelope - assumes normal is a single Gaussian blob. When that holds it is fast, needs almost no tuning, and gives an interpretable Mahalanobis distance. When normal is multi-modal, curved, or clustered, the single ellipse spans the empty middle and both false-flags real points and misses true gaps.
One-Class SVM - makes no shape assumption; the kernel boundary bends to whatever the normal cloud looks like. More flexible, but you pay in tuning (nu, gamma) and it scales worse.
Rule of thumb - plot your normal data first. One tidy blob? Elliptic Envelope. Anything weirder? One-Class SVM, or back to Isolation Forest.

And the shared caveat: both are only as good as the "normal" sample you trained on. If an anomaly sneaked into your clean window, you taught the detector that it is normal.

Demo 1 of 2

One-Class SVM on a clean normal slice ★ 12 min · everyone builds

Train on a normal-only slice of the Lumen features, then predict on every day and score the flags.

Rebuild the labeled Lumen series and its feature matrix, then standardize - the RBF kernel needs scaled features.

Select a clean normal window to learn from (here, the known-normal days - in practice, a quiet incident-free stretch).

Fit OneClassSVM, predict on all days, and score precision, recall, and F1.

★ Run it now (Python) - OneClassSVM novelty detectionimport pandas as pd, numpy as np from sklearn.svm import OneClassSVM from sklearn.preprocessing import StandardScaler # --- rebuild the labeled Lumen series (from b1) --- idx = pd.date_range("2026-01-01", periods=90, freq="D") season = np.array([30,42,48,40,22,-95,-110]); i = np.arange(90) y = 900 + 1.2*i + season[i % 7] + 10*np.sin(i*1.7) + 7*np.cos(i*0.6) truth = [20, 45, 63, 70, 71, 72] y[20] += 430; y[45] -= 380 y[63] = y[63] - season[63 % 7] + season[6] - 40 y[[70,71,72]] += [175,190,170] s = pd.Series(np.round(y), index=idx, name="sessions") label = pd.Series(0, index=idx); label.iloc[truth] = 1 # --- feature matrix + scaling (RBF kernel is distance-based) --- df = pd.DataFrame({"value": s}) df["lag7"] = s.shift(7) df["rmean"] = s.rolling(7, min_periods=1).mean() df["rstd"] = s.rolling(7, min_periods=1).std() X = df.bfill().ffill().values Xs = StandardScaler().fit_transform(X) # --- semi-supervised: learn the shape of a CLEAN normal sample --- normal = label.values == 0 # a known incident-free window in practice ocsvm = OneClassSVM(nu=0.08, gamma="scale").fit(Xs[normal]) pred = ocsvm.predict(Xs) # -1 = novelty/anomaly, on ALL days flags = set(np.where(pred == -1)[0]) t = set(truth) tp = len(flags & t); fp = len(flags - t); fn = len(t - flags) p = tp/(tp+fp) if tp+fp else 0; r = tp/(tp+fn) if tp+fn else 0 print(f"OC-SVM caught {tp}/{len(t)} P {p:.2f} R {r:.2f} F1 {2*p*r/(p+r) if p+r else 0:.2f}")

Your "normal" window is a promise you are making One-class methods only work if the training slice is genuinely clean. Here we used the labels to pick it, which is a shortcut for teaching - in production you would choose a quiet, incident-free stretch by hand and defend that choice. If an anomaly hides in your training window, the detector learns to call it normal, and you will never flag its cousins.

Demo 2 of 2

Elliptic Envelope and Mahalanobis distance ★ 10 min · build your own

Fit a robust Gaussian to the same features, read the Mahalanobis distances, flag the far points, and contrast it with the OC-SVM - noting the Gaussian assumption it rests on.

Fit EllipticEnvelope with a contamination rate on the scaled features.

Read mahalanobis distances - the far ones are the anomalies - and score the flags.

Rank days by Mahalanobis distance and compare to the OC-SVM result; note where the Gaussian assumption helps or hurts.

★ Run it now (Python) - EllipticEnvelope + Mahalanobisimport numpy as np, pandas as pd from sklearn.covariance import EllipticEnvelope # same scaled feature matrix Xs; contamination = assumed anomaly rate (the threshold) ee = EllipticEnvelope(contamination=0.07, support_fraction=0.9, random_state=0).fit(Xs) pred = ee.predict(Xs) # -1 = anomaly, 1 = normal maha = ee.mahalanobis(Xs) # squared distance to the robust centre flags = set(np.where(pred == -1)[0]) t = set(truth) tp = len(flags & t); fp = len(flags - t); fn = len(t - flags) p = tp/(tp+fp) if tp+fp else 0; r = tp/(tp+fn) if tp+fn else 0 print(f"EllipticEnvelope caught {tp}/{len(t)} P {p:.2f} R {r:.2f} " f"F1 {2*p*r/(p+r) if p+r else 0:.2f}") # rank days by Mahalanobis distance (large = far from the Gaussian centre) print("\nfarthest days:") print(pd.Series(maha, index=s.index).sort_values(ascending=False).head(6))

Real world

Elliptic Envelope is the honest default when your normal really is one blob. Sensor calibration, lab measurements, and many financial features cluster tightly around a mean, and there the Mahalanobis distance is interpretable and cheap. But always plot first: the moment normal splits into two regimes - weekday vs weekend, product A vs product B - a single ellipse straddles the gap, false-flags the honest points, and waves the real ones through. When you see multiple modes, reach for One-Class SVM or Isolation Forest instead.

Homework

Try it yourself - this week ◐ 30-45 min total

Fit One-Class SVM on a clean slice and sweep nu from 0.02 to 0.2. Watch the boundary tighten and recall climb as precision falls.
Change gamma from "scale" to a large value and see the boundary overfit - it wraps every training point and flags almost nothing new.
Fit Elliptic Envelope and plot the Mahalanobis distances. Do the true anomalies sit at the top?
Deliberately give both detectors a two-cluster normal (stack a weekday and a weekend feature) and watch the ellipse break where the SVM copes.
Optional: skim Scholkopf et al. (2001) for the OC-SVM formulation and Rousseeuw & Van Driessen (1999) for the MCD estimator.

Source material

Sources covered (cited)

Taught from the detection canon. This session covers:

✓Scholkopf, Platt, Shawe-Taylor, Smola & Williamson (2001), One-Class SVM, Neural Computation 13(7)Part 1 · a kernel boundary around the normal region; the nu parameter

✓Rousseeuw & Van Driessen (1999), Minimum Covariance Determinant (Elliptic Envelope)Part 2 · robust Gaussian fit and Mahalanobis distance

◐scikit-learn "Novelty and Outlier Detection" user guideDemos 1-2 · OneClassSVM, EllipticEnvelope, and feature scaling

Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · What makes One-Class SVM and Elliptic Envelope "semi-supervised"?

This is novelty detection: you train on normal-only data (no anomaly labels) and flag anything that does not fit the learned normal. The clean training window is the whole assumption.

2 · When does Elliptic Envelope break down?

Elliptic Envelope fits one robust Gaussian. If normal is multi-modal or curved, the ellipse straddles the gap, false-flags real points, and misses true anomalies.

3 · Why standardize features before a One-Class SVM?

The RBF kernel measures distances between points. A feature on a large scale swamps the others, so you standardize first to give every feature a fair say.

Builder session 6 cheat sheet · pin this

Novelty detectionlearn normal only, flag departures. Semi-supervised - needs a clean normal sample.

One-Class SVMa kernel boundary around the normal cloud; flexible, non-Gaussian shapes.

nubounds the outlier fraction - your tolerance. Higher = tighter boundary, more flags.

Scale firstthe RBF kernel is distance-based; standardize or one feature dominates.

Elliptic Envelopefits a robust Gaussian (MCD); flags large Mahalanobis distance.

Mahalanobisdistance from the centre in the ellipse's own units - interpretable when Gaussian.

Assumption checkplot first. One blob = envelope; multi-modal or curved = OC-SVM.

Next sessionb7 - time-series anomalies: change-points, CUSUM, and forecast residuals.