Learn with Phoebe ยท Data Ops & Security

Learn Data Observability with Phoebe

Pipelines that never fail still ship wrong numbers. A shard stops arriving and the row count is at 41 percent of a normal day's rows for two business days; a merge key changes and the duplicate rate triples while the count looks perfect; a column disappears and two dashboards quietly lose a filter. None of that raises an error, so the person who finds it is a stakeholder. This course builds the layer that finds it first: four cheap metadata queries, thresholds learned from a table's own history rather than guessed, and a bench that scores your monitor set over 30 nights of real load metadata with five real defects in it. Six leader sessions on what to fund and what to report; eight builder sessions that end with a monitor set you have actually measured.

14sessions, two tracks
5signals, four of them free
30nights of metadata on the bench
25false pages from tightening everything
45min per session

Leader track ๐ŸŒ™ ยท six sessions, no SQL

For whoever funds the work and reads the report. What data downtime costs, which signals are worth paying for, and the three numbers that say whether the programme is working.

Builder track ๐Ÿ”ง ยท eight sessions, one monitor set

One table, orders_daily at Bellwether, instrumented signal by signal. Session 8 runs the finished set against 30 nights of history and counts what it caught.

๐Ÿ”Ž Builder session 1 ยท start here

The metadata you already have

Four numbers every warehouse already stores, the SQL that pulls them, and the anatomy of a monitor that somebody actually answers at 07:00.

โ–ถ Start here
โฑ๏ธ Builder session 2

Freshness: on-time is a distribution

Why a fixed deadline is the wrong shape for the question, trailing percentiles that learn the table's real habits, and the full-rebuild trap.

๐Ÿงฎ Builder session 3

Volume and uniqueness

Same-weekday medians against seasonality, robust bands with MAD, and the duplicate rate that catches what a healthy row count hides.

๐Ÿงฌ Builder session 4

Schema change: the break you can see coming

Column-set diffs, type widening, the difference between an additive change and a contract break, and why the revert pages too.

๐Ÿ“Š Builder session 5

Distribution drift from metadata only

Null rates, cardinality, min and max, segment share. How far you get without reading a row, and the point where this becomes a different discipline.

โœ… Builder session 6

Expectations as code

Great Expectations Core end to end: Data Context, Batch Definition, Suite, Validation Definition, Checkpoint and Actions, wired to the metrics table.

๐Ÿ•ธ๏ธ Builder session 7

Lineage and blast radius

Column-level lineage with OpenLineage, the impact query that turns an alert into a triage decision, and the reverse-ETL job nobody documented.

๐ŸŽ›๏ธ Builder session 8

The monitor bench

Thirty nights, five real defects, seven monitors, three numbers. Hair-trigger everything and detection falls back to where it started.

๐Ÿ”’ the capstone
start here foundations hands-on advanced and capstone

Choose your path ๐Ÿ—บ๏ธ

Three doors into the same practice. Both tracks start at their own session 1, because the vocabulary is what makes the rest arguable.

๐Ÿ’ผ I fund this and read the report L1 โ†’ L4 โ†’ L5 โ†’ L6 the cost, the noise problem, tiering, and the three numbers to report
๐Ÿ”ง I own the table B1 โ†’ B2 โ†’ B3 โ†’ B4 โ†’ B8 the four numbers, three learned thresholds, then score the set
๐ŸŽฏ Everything, in order L1 โ†’ L2 โ†’ L3 โ†’ B1 โ†’ B5 โ†’ B6 โ†’ B7 โ†’ B8 from a table nobody watches to a set that catches five of five with two false pages

Your passport ๐ŸŽซ

Clear all three quiz questions in a session and it gets stamped. Fourteen stamps across both tracks, and they live in this browser only.

What is real here ๐Ÿงพ

A course about detecting quiet failures should be exact about which of its own numbers are measured.

ClaimStatus
The 30 nights of load metadataGenerated once from a fixed seed and identical for every learner. Row counts, load lateness, duplicate rates, null rates and column sets, with a real weekday and weekend pattern
The five defectsInjected into specific nights: a 230-minute late load, a load holding 41 percent of a normal count, a duplicate rate of 8.4 percent, a dropped column, and a null rate of 11.4 percent. On a defect night every other signal is held at a quiet baseline, so credit for a catch can only go to the monitor watching that pillar
The ladder: 0, 1, 2, 2, 3, 4, 5 of 5 caughtComputed by the bench in your browser, night by night. Trailing percentiles, same-weekday medians, duplicate rates, column-set diffs and standard deviations are all real arithmetic on that history
Eight false pages from a fixed volume bandArithmetic. Weekend volume is about 45 percent of a weekday, which sits outside a plus-or-minus 30 percent band on all eight weekend nights in the window
Tightening everything: 2 of 5, 25 false pages, 15.8 hoursThe counts and the thresholds are real. The consequence runs through one modelled rule, below
The muted channelModelled, not measured. Six or more false pages in a trailing week is treated as a channel nobody reads, so the page fires and the stakeholder still finds the problem first. Labelled as a model on the widget itself
Great Expectations component namesRead from the GX Core 1.x documentation: Data Context, Data Source, Data Asset, Batch Definition, Expectation, Expectation Suite, Validation Definition, Checkpoint, Actions, Data Docs
DAMA-DMBOK's six dimensions and ISO/IEC 25012Both named as published, including the point that uniqueness appears in DAMA and not in the ISO inherent set. The ISO standard itself is paywalled, so wording travels through close secondary sources and is flagged where it does
BellwetherA fictional grocery delivery company. The failure patterns are ordinary ones; the company is not real

The knowledge map ๐Ÿง 

Both tracks at a glance - hover a session to spotlight its concepts, click any node to jump in.