Where the registry sits on the track
Session 1 wrote an estate registry: every table, its domain, its grain, its owner. Tonight extends the same idea one layer up, from tables to the numbers people quote. Sessions 2 to 6 built the inputs (conformed orders, the identity gate, the live event spec, stage metrics, creator yield); every one of them ends in a number somebody will put on a slide. The metric registry is where that number gets a single name, a single formula and a history. The data is the real Olist file, Q2 2018; every value printed below is what the code printed on it.
★ What you walk out with tonight. A metrics.yaml with three GMV entries and two active-seller entries, each answering the three questions a serious reader asks of a key metric; a registry.py that is the only thing allowed to read it; and a test_registry.py that passes on the approved definitions and fails, loudly, when one line of SQL changes without a version bump.
Three GMVs from one real quarter
6 min liveQ2 2018 on Olist, April to June, order lines by purchase date: 19,947 orders. Filter the same rows three ways and you get three totals, each with a sound argument behind it. The registry does not choose the "true" one. It gives each one its own name, so that nobody can ever again say "GMV" and mean two of them.
LiveFive numbers, one quarter, all defensible4 min▶
| Definition | Q2 2018 on Olist | The argument for it |
|---|---|---|
| Placed, any status, items only | R$ 2,858,289.74 | The demand the stores generated; what the trading team steers |
| Placed, any status, items + shipping | R$ 3,332,156.97 | What the buyer was charged; matches a marketplace headline that counts shipping |
| Delivered only, items only | R$ 2,807,156.64 | Closest to the money that settles; what a GMV-linked fee should rest on |
| Delivered only, items + shipping | R$ 3,273,631.74 | Settled-ish, buyer-side view |
| Payments table, same orders | R$ 3,332,386.55 | What was actually collected; differs from items + shipping by R$ 229.58 |
Cancelled or unavailable orders in the quarter add up to only R$ 8,559.48 across 50 orders, so on this file the status filter moves the total far less than the shipping choice does. On an enabler's own estate, with returns and failed deliveries, the order flips. Either way, the size of the gap is not the point; the point is that a board pack and a brand statement built on two of these rows will disagree, and nobody will know why until somebody opens both queries.
Self-studyThe same spread, on Serai's constructed quarter3 min▶
The leader track uses one constructed Serai quarter to make the same point with fees on top. GMV placed including shipping $48.6M, placed excluding shipping $45.0M, settled net of cancellations and returns $39.2M. Divide the same $1,764,000 of retainers and GMV share by each and the take rate reads 3.63, 3.92 or 4.50 percent; drop the retainers and it is 2.00 percent. Every figure there is constructed, and your numbers will differ. The Olist quarter above is the real-rows version: the same spread, measured.
The definitions themselves (GMV, take rate, what counts as an order) are taught properly in the GMV equation session of the ecommerce metrics track. The enabler twist is that an enabler reports someone else's GMV, often on three marketplaces with three conventions, and bills against it.
A listed marketplace's GMV counts shipping and unsettled orders. Sea Limited's annual report for fiscal 2019 defines Shopee's GMV as the value of orders on the marketplace and says the calculation "includes shipping and other charges", and it counts each confirmed order regardless of whether the transaction is settled or the item returned. That is a clear, written choice, and it is the middle row of the table. An enabler whose own GMV is delivered-only will always look smaller than the marketplace's figure for the same stores, and that is fine, as long as both definitions are written down.
A registry entry is code
6 min liveA metric definition in a wiki page is a hope. The same definition as a YAML entry, read by the code that computes the number, is a contract: change the entry and every number built on it changes with it, in one commit, with a name and a date. Ten fields are enough, and each one has a reader.
LiveThe ten fields, and the three that answer the reader4 min▶
| Field | What goes in it | Who reads it |
|---|---|---|
description + sql | "A clear definition of the metric and how it is calculated" | Board pack, every analyst |
why_useful | "The reasons why the metric provides useful information" | Board pack, brand statements |
how_used | "How management uses the metric in managing or monitoring the performance of the business" | Board pack, the owner |
grain | The row the SQL sums over, and the date it keys on | Dashboard, anyone joining it |
sources | Table and column for every input: the lineage | Data engineering, when a source changes |
owner | One team, not a person's name | Whoever gets paged when a test fails |
version, valid_from, change_log | The history: what changed, when, and whether prior periods were restated | The CI test, auditors, next year's you |
The quoted wording in the first three rows is the SEC's, below. Put the three questions in the schema as required fields and the reader loads nothing that cannot answer them.
Self-studyOne reader, and the spec sheet it replaces3 min▶
The YAML is half of it. The other half is that exactly one module reads it. If the dashboard parses metrics.yaml with its own code and the board pack script with another, you have two readers and, the first time somebody adds a field, two behaviours. registry.py below is thirty lines, and every consumer imports it: load() refuses an entry with a missing field, compute() runs the entry's own SQL. Nobody writes SUM(price) anywhere else.
How to write a good definition in the first place (the one-page spec, the edge cases, the sign-off) is taught in the metric spec sheet session of the ecommerce metrics track, and governance around it in its metric governance session. The registry is the spec sheet made executable: same content, but now a machine checks it on every change.
A public checklist for a key metric. The US SEC's 2020 guidance on key performance indicators in MD&A says the Commission would generally expect, alongside such a metric, a clear definition and how it is calculated, the reasons it provides useful information, and how management uses it. If the calculation changes, it says to consider disclosing the differences, the reasons and the effects on amounts previously reported. Read it here as what it is: a public example of the three questions any serious reader asks of GMV or active sellers, and a good schema for three required fields. What any exchange requires of a listing is a question for counsel, not for a YAML file.
Tests that fail when a definition drifts
4 min liveDefinitions drift for good reasons. Somebody decides cancelled orders should not count, adds one line to the SQL, and the quarter's GMV moves by R$ 8,559.48 without a meeting. The registry cannot stop the decision, and should not. It can refuse to let it happen silently. Three tests do it.
LiveThree tests, three different failures3 min▶
Golden numbers. Freeze one past period and write down the approved value of each metric: gmv_placed for Q2 2018 is R$ 2,858,289.74, active_sellers_1000 is 552. The data for a closed quarter does not change, so if the computed value moves, the definition did. This is the test that catches the one-line SQL edit.
Reconciliation. Two independent routes to roughly the same money should agree within a tolerance you write down in advance. Items plus shipping for the quarter is R$ 3,332,156.97; the payments table for the same orders is R$ 3,332,386.55; the gap is R$ 229.58. The tolerance in the build-along is 0.01 percent of payments, R$ 333.24. When a join starts fanning out or a table is loaded twice, this test fails first.
Definition change. Store a fingerprint of each approved SQL next to its version. If the SQL changes and the version does not, or the change log has no entry that says prior periods were restated, fail. This is the test that turns "somebody tidied the query" into a reviewed decision.
Self-studyWhat a restatement is, in practice2 min▶
When gmv_placed moves to v2, three things happen in the same commit: the version goes up, the change log says what changed and why, and every prior period that anyone will compare against is recomputed on v2 and the golden value updated. Last year's board pack is not edited; next year's shows both, with a line saying which definition each column uses. A trend that crosses a definition change without restating is the most common way a chart lies with every number on it correct.
Constructed, and common: two take rates in one meeting. On Serai's constructed quarter, the finance deck divides fees by GMV placed excluding shipping and shows 3.92 percent; the brand team's deck divides by settled GMV and shows 4.50 percent. Both queries are correct and both are called "take rate". With a registry, the two numbers have two names, take_rate_placed and take_rate_settled, and the board sees one of them with its definition printed under it. Every figure in this story is constructed; your numbers will differ.
Write the registry and test it on Olist
★ 22 min · everyone buildsFive entries, one reader, three tests, then one deliberate drift to watch the gate close. The code below ran on the full Olist file with Python 3, pandas 2.1 and PyYAML 6.0; every output line is pasted exactly as it printed. If you do not have PyYAML, save the registry as metrics.json and swap yaml.safe_load for json.load: the reader and the tests do not change.
Point at the data and check the one dependency. Work in a scratch folder, not inside a repo. The CSVs are the kagglehub download from session 1; the path below is where kagglehub caches them.
Write the registry: three GMVs, two active-seller counts. Each entry answers the three reader questions (description, why_useful, how_used), carries its SQL, its grain, its lineage and its history. The dates are half-open, >= start and < end, so a quarter never double-counts its last second.
Write the one reader. load() refuses any entry missing a required field, so an entry that cannot answer the three questions never reaches a dashboard. compute() runs the entry's own SQL with named parameters; SQLite is in the standard library, so the SQL in the YAML is exactly what runs.
Compute every metric for Q2 2018. Five names, five numbers, and each one matches the table in Part 1. Note what the two seller counts say: 1,680 sellers traded at all, but only 552 sold R$ 1,000 of delivered goods. The file has 3,095 sellers, and 1,653 had a delivered order in the quarter. Four honest answers to "how many active sellers".
Lock the approved definitions. A short hash of each entry's SQL (whitespace normalised, so re-indenting is not a change) and its version, written to a file you commit next to the YAML. Re-run this only when a definition change has been reviewed.
Write the three tests, with plain asserts, and watch them pass. Golden values for the frozen quarter, the payments reconciliation with its tolerance stated in the code, and the definition-change check against the lock file. In CI this is the file that runs on every pull request touching metrics.yaml.
Now drift one definition, on purpose. Somebody decides cancelled orders should not count in gmv_placed, adds one reasonable line to its SQL, and does not touch the version. Run the tests again. The golden test catches the number moving by exactly the R$ 8,559.48 of cancelled and unavailable orders, and the version test names the missing bump. The script ends in an AssertionError, exit code 1, and CI stops the merge.
Decide, then make the decision visible. Two honest ways out, and neither is "update the golden number and move on". Either the new filter is a different metric (add gmv_placed_net as its own entry and leave gmv_placed alone), or it replaces the old one (bump to version 2, add a change log entry that says prior periods were restated, recompute Q2 2018 on v2, update the golden value and re-run lock.py, all in one reviewed commit). Revert the drift before homework: python3 test_registry.py should print ALL PASS again.
Extend the session 1 registry, do not start a second one. The estate registry already knows every table's grain and owner; the metric registry's sources should point at those same table names, so a grain fix in one place reaches every metric built on it. Two registries that each describe items will disagree the first week somebody edits one.
Before session 8
30 min- Find the three most-quoted numbers in your team's last board pack or brand review. For each, find every query that computes it. Write down how many distinct definitions you found, and which one the slide used.
- Write registry entries for those three, with all ten fields. Any field you cannot fill is a finding: an empty
how_usedmeans nobody decides anything with it, an emptyownermeans nobody will notice when it breaks. - Pick one closed period and record golden values for all three. Add one reconciliation between two independent routes to the same money, and write the tolerance down before you run it.
- Drift one definition on a branch, as in Step 7, and confirm your tests fail. A test suite you have never seen fail is not yet a test suite.
Three questions
Clear all three and Analyst session 7 gets stamped in your passport.
1 · On Olist Q2 2018, one query returns R$ 2,858,289.74 and another returns R$ 3,332,156.97, both labelled GMV. What should the registry do?
Both are defensible; the first is item value, the second is what the buyer was charged, which is the marketplace-headline convention. The failure is one word covering two numbers. Two names end it.
2 · Which three fields map onto the three questions the SEC's 2020 guidance lists for a key metric?
The guidance expects a clear definition and calculation, the reasons the metric is useful, and how management uses it. The other fields matter too, for lineage and history, but those three are the reader's questions, and making them required means no entry loads without them.
3 · After a one-line SQL edit, the golden test reports gmv_placed as 2,849,730.26, want 2,858,289.74. What is the right response?
The data for a closed quarter cannot move, so the definition did, by exactly the R$ 8,559.48 of cancelled and unavailable orders. The test is working. The fix is to make the change a visible decision, not to make the test quiet.
What this session teaches, and where it came from
Every fact on this page, and its verification tier, is recorded in the course's source map.
Analyst session 7 cheat sheet · pin this
You have a registry that names every number and a gate that refuses a silent change. Session 8 is the bench: the identity ladder runs on real Olist orders in your browser to show which join keys survive the marketplaces' masking, and the roadmap sequencer orders Serai's use cases by what the estate can actually obtain.