Where this sits in the practitioner track
Ten sessions, 45 minutes each, and a laptop from here on. Tonight is the one that comes before any model gets opened: the inventory that says what exists, and the tiering call that decides how much review each thing earns. Session 2 reads Brackwater's design before touching a number. Session 3 is the leak hunt. Sessions 4 to 7 are replication, challengers, outcomes analysis and out-of-time testing. Session 8 is monitoring, session 9 is writing findings, and session 10 runs the whole workflow against a model whose truth is known by construction. The worked example throughout is Brackwater's consumer scorecard, with 4,000 development applications at a bad rate of 0.292.
★ What you walk out with tonight. A column list you can defend, derived from the one clause that does the work: the inventory has to support management of model risk at the individual level and in aggregate, and an aggregate question cannot be answered by a column you did not record. A way to score materiality that separates the measurable half from the judged half, so two people scoring the same model disagree about something specific. And the failure mode that no field catches on its own: Brackwater's scorecard was built to rank applicants and by year two was setting prices, which is a change of materiality that nothing in the register notices unless a person re-tiers it.
LiveThe clause that decides the columns4 min▶
It is common industry practice to maintain a comprehensive set of information for models under development and models in use, and the level of detail may reasonably vary with a model's complexity and usage. That much sounds like a filing exercise. The clause that turns it into a design problem is the next one: an effective inventory holds enough information to understand model risks and support the management of them both at the level of the individual model and in aggregate.
Read that as a requirement on the columns rather than on the rows. Individual-level management is satisfied by a row: who owns this, what is it for, when was it last validated, what is still open. Aggregate management is a question asked across rows, and you can only ask it about things you wrote down. Nobody can see that eleven models share one data feed if the feed was never a field. Nobody can see that a whole portfolio rests on one assumption about bureau comparability if assumptions live in PDFs. A register of names, owners and dates is a list of models. It is not an inventory, because the second half of the job is structurally impossible with it.
Brackwater's spreadsheet had a column called "data sources" and every cell in it said "warehouse". True, useless, and the reason nobody could answer the only question that mattered when the collections table was re-platformed: which models would change behaviour on the Monday. The answer turned out to be found by asking around for three weeks.
Self-studyThe words tonight uses precisely3 min▶
- Inventory. The comprehensive record of models under development and in use, detailed enough to support management of model risk individually and in aggregate.
- Exposure. The measurable half of materiality. How much rides on the output: portfolio size, business impact, the volume of decisions the thing touches.
- Purpose. The judged half. What the output is for, with uses that meet a regulatory requirement or manage financial risk weighing more than uses that size a campaign.
- Materiality. Purpose together with exposure. In the current guidance these two are what determine a model's materiality, and overall model risk is inherent risk read in the context of it.
- Tier. An editorial device, not a regulatory object. The guidance asks that the nature and rigour of review align with the model's approach, use and materiality; a numbered tier is one house convention for writing that alignment down.
- Aggregate risk. Interactions and dependencies among models, and reliance on common assumptions, data or methodologies. The reason the columns look the way they do.
- Immaterial. A scored outcome, not an exemption. An immaterial model is still identified, and still monitored for the conditions that would make it material.
- Re-tier. The act a person performs when purpose or exposure moves. No field does it automatically, which is the subject of Part 2.
A register says a model exists. An inventory answers a question.
12 min liveEvery column in a good inventory earns its place by being the answer to a question somebody will ask under pressure. Drop the aggregate clause and you can justify four columns. Keep it and the list roughly doubles, because dependencies, shared assumptions and shared methodologies are all cross-row properties, and a cross-row property that nobody recorded does not exist as far as the portfolio is concerned.
LiveAggregate risk is a property of the portfolio, not of any model in it4 min▶
Aggregate model risk reflects interactions and dependencies among models, and reliance on common assumptions, data or methodologies. Every clause in that sentence describes something that is invisible inside a single model document, because a single model document has no reason to mention what the model next door is doing.
The practical consequence is that the inventory is the only artefact in the whole discipline that can see this. A validation report covers one model. A monitoring pack covers one model. The inventory is where twenty individually reasonable models turn out to share one vendor feed, one macro assumption and one preprocessing library, and where somebody can notice that a correction to any of the three moves the whole book at once. If your columns cannot express that, the aggregate view is not being done anywhere, no matter how many committees discuss it.
LiveWhat the documentation is actually for3 min▶
Documentation gets treated as a tax, which is why it gets written last and read never. It is worth being specific about the jobs it does, because each one implies a field. It supports continuity of operations, so the model survives the departure of the person who built it. It tracks recommendations, responses and exceptions, so an argument that was settled two years ago stays settled and the reasoning is recoverable. And it supports the management of remediation, so a finding has a state rather than a memory.
Read as column requirements, those three sentences produce most of the right-hand list. Continuity means owner, purpose, feeds and assumptions written where a stranger can find them. Tracking recommendations and exceptions means a findings register that the inventory points at, including the exceptions that were granted and by whom. Remediation management means each open finding carries an owner and a date, which is what separates a condition from a sentence.
Self-studyModels under development belong in it too3 min▶
The scope covers models under development as well as models in use, and teams routinely read this as pedantry. It is not. A model that is being built is a model that somebody is already planning around, and the gap between "we are building a pricing model" and "the pricing model exists" is precisely the window in which validation has to be scheduled, data access negotiated and a tier assigned. A model that appears in the inventory on the day it goes live has already skipped the part where the review could have changed anything.
There is a second reason, which is that the pipeline is where the aggregate view gets its warning. Three teams independently building on the same new feed is a concentration that has not happened yet, and the cheapest moment to notice it is while all three are still choosing.
Score the materiality, do not describe it
11 min liveModel purpose, together with model exposure, determines model materiality. Two halves, and they behave completely differently. Exposure is measurable, so two people who disagree about it are disagreeing about a quantity and can go and look. Purpose is judged, so two people who disagree about it are disagreeing about what the output is for, which is a conversation with the business rather than a query. Write both down separately and the tier becomes reproducible; blend them into a single adjective and the tier becomes whatever the loudest person in the meeting thinks.
LiveScoring without adjectives4 min▶
The failure pattern is a tiering sheet whose inputs are words: "high impact", "significant", "business critical". Every model owner believes their model is business critical, and they are not being dishonest, they are answering a question that has no shared unit. Replace the words with two recorded quantities and one recorded sentence and the argument gets much shorter.
For exposure, pick a measurable thing the model touches and record the number with its unit and its date: balances scored, decisions per year, provisions influenced, notional covered. Any of these is defensible; none of them is a synonym for the others, so the inventory has to say which one it holds. For purpose, record what the output decides, in a sentence naming the decision and who acts on it, and record whether that use meets a regulatory requirement or manages financial risk, because those uses weigh more. The tier is then a stated function of the two, applied the same way to every model, and if somebody wants a different tier they have to argue with the function rather than with you.
Brackwater's original tiering sheet had one column called "criticality" with the options low, medium and high, chosen by the model owner. Every model in the consumer book was high. The sheet had been in use for four years and had never once caused a review to be scheduled differently, which is the operational definition of a control that does not exist.
LiveImmaterial is a score, not an exemption3 min▶
Rigour is commensurate with risk, which means some models correctly get a paragraph rather than three weeks. The part that gets dropped is the second half: a model judged immaterial is still identified, and still monitored for the conditions under which it would stop being immaterial. Those conditions are the useful artefact, and they have to be written at tiering time, while the reasoning is fresh, not reconstructed afterwards by someone explaining a loss.
A condition is specific and checkable. "Re-tier if this model is extended beyond the one product line." "Re-tier if the balances it touches pass the threshold in the tiering function." "Re-tier if its output starts feeding a price rather than a ranking." Each of those is a sentence somebody can be given ownership of, and each of them is a column in the inventory rather than a paragraph in a PDF, because a paragraph in a PDF cannot be reported on.
Self-studyWhy purpose is the half that moves silently3 min▶
Exposure changes loudly. Balances grow, volumes appear in a monthly pack, and somebody somewhere is already reporting the number, so a threshold crossing can be automated. Purpose changes quietly, because a change of purpose usually looks like a good idea implemented by a competent person: the scorecard ranks well, pricing needs a risk input, the score is right there, and the integration takes an afternoon. No document is amended, because no document was wrong. The model still does what it always did.
What changed is what the output decides, which is one of the two inputs to materiality. This is also the seam with model use, which session 2 covers: using a model beyond its intended purpose introduces additional uncertainty and risk, and when a use is extended the sound practice is additional analysis of the new usage and its limitations, plus a review of the existing controls. For tonight the inventory consequence is narrow and practical. Put a "purpose last confirmed" date next to the purpose sentence, and make confirming it an act somebody performs, because an undated purpose field ages into fiction without ever being edited.
Design the register
17 min build-alongBuild the schema first, then prove it earns its keep by asking it a question no register could answer. The code below is deliberately plain: an inventory that needs a platform before it can exist will not exist. A dataframe with the right columns, kept honestly, beats a tool with the wrong ones.
Write the model identifier scheme before the first row. Stable, never reused, never encoding anything that can change. An id that embeds the owning team becomes wrong at the first reorganisation, and every downstream reference becomes wrong with it.
Give purpose a sentence and a date. What the output decides and who acts on it, plus the date somebody last confirmed that is still true. The sentence is what gets compared against reality; the date is what makes the comparison an event rather than a hope.
Record exposure as a quantity with a unit. One measure, stated, the same measure for every model of that kind. Two models whose exposure fields hold different units cannot be compared, and an inventory that cannot rank its own rows cannot allocate review.
Record every upstream feed as a list, not a sentence. This is the field that makes the aggregate view possible, and prose destroys it. A list can be exploded and grouped; "various warehouse tables" cannot.
Record shared assumptions the same way. Name the assumption in a form another team would recognise, so that two models holding the same assumption use the same string. This one needs a controlled vocabulary or it decays within a year into thirty phrasings of one idea.
Close the loop with validation dates and open findings. Last validation, next review implied by the tier, and the count of findings still open. These three make the inventory a work queue instead of an archive, which is the difference between a file that is maintained and a file that is recreated annually.
Then ask it an aggregate question immediately. Group by feed on day one, while the register has five rows. If the question is awkward to ask at five rows it will be impossible at two hundred, and the time to change the schema is now.
import pandas as pd
# One row per model. Every column is here because some question needs it,
# and the cross-row questions are the ones that decide the design.
COLUMNS = [
"model_id", # stable, never reused, encodes nothing that can change
"name",
"owner", # a named person, not a mailbox
"status", # "in development" or "in use" - both are in scope
"purpose", # what the output decides, in one sentence
"purpose_confirmed", # the date somebody last checked that sentence
"regulatory_or_risk", # True if the use meets a regulatory need or manages financial risk
"exposure_measure", # the unit, stated
"exposure_value", # the quantity, in that unit
"tier", # a stated function of purpose and exposure
"retier_triggers", # the conditions that would change the tier
"upstream_feeds", # a list, so it can be exploded and grouped
"shared_assumptions", # a list, from a controlled vocabulary
"last_validation",
"next_review",
"open_findings",
]
rows = [
{
"model_id": "MR-0141",
"name": "Consumer application scorecard",
"owner": "A. Okafor",
"status": "in use",
"purpose": "Rank consumer applications at origination and set the approval cutoff",
"purpose_confirmed": "2026-03-18",
"regulatory_or_risk": True,
"exposure_measure": "applications decided per year",
"exposure_value": 180000,
"tier": 1,
"retier_triggers": ["output feeds price", "extended beyond consumer book"],
"upstream_feeds": ["bureau_v3", "core_ledger", "collections_log"],
"shared_assumptions": ["bureau score comparable across vintages"],
"last_validation": "2026-03-18",
"next_review": "2027-03-31",
"open_findings": 4,
},
{
"model_id": "MR-0207",
"name": "Collections propensity",
"owner": "R. Villanueva",
"status": "in use",
"purpose": "Order the outbound contact queue for delinquent accounts",
"purpose_confirmed": "2025-11-04",
"regulatory_or_risk": False,
"exposure_measure": "accounts contacted per year",
"exposure_value": 41000,
"tier": 3,
"retier_triggers": ["output feeds a provision or a price"],
"upstream_feeds": ["core_ledger", "collections_log"],
"shared_assumptions": ["contact outcome recorded within 30 days"],
"last_validation": "2025-11-04",
"next_review": "2027-11-30",
"open_findings": 0,
},
]
inventory = pd.DataFrame(rows, columns=COLUMNS)
print(inventory[["model_id", "tier", "exposure_value", "open_findings"]])
# Explode the feed lists so one model becomes one row per feed it depends on.
pairs = inventory.explode("upstream_feeds")
concentration = (
pairs
.groupby("upstream_feeds")
.agg(
models=("model_id", "count"),
tier1_models=("tier", lambda t: int((t == 1).sum())),
exposure_behind_it=("exposure_value", "sum"),
)
.sort_values("exposure_behind_it", ascending=False)
)
print(concentration)
# collections_log appears under two models, one of them tier 1.
# Nothing in either model document says so, because neither document
# has any reason to mention the other model.
# The same shape answers the assumption question, which is the one
# that tends to be worse, because assumptions are shared silently.
shared = (
inventory.explode("shared_assumptions")
.groupby("shared_assumptions")["model_id"]
.apply(list)
)
print(shared)
Run the concentration query before you add the sixth row. An inventory is a schema decision pretending to be an administrative task, and schema decisions get expensive per row. Nine models is an afternoon of backfilling. Ninety is a project nobody funds, and the register that results is the one that has a "data sources" column where every cell says "warehouse".
Three questions
Your register holds model name, owner, build date and last review date for every model in the bank. Which part of the inventory expectation is it structurally unable to meet?
An effective inventory holds enough to understand model risks and support managing them at the individual level and in aggregate. The four columns serve the individual level adequately. No amount of rows lets you group by a feed you never wrote down, which is why the aggregate half is impossible rather than merely difficult.
A model is scored immaterial on its purpose and exposure. What is the correct treatment?
Rigour is commensurate with risk, so a light touch is correct. Immaterial is a score rather than an exemption: the model stays identified and gets monitored for the conditions under which it would stop being immaterial. Those conditions belong in a field, written at tiering time, with an owner.
Brackwater's scorecard was built to rank applicants. By year two its output is setting prices. Nothing in the model changed and no document is wrong. What happened to its materiality, and what in the inventory catches it?
Purpose together with exposure determines materiality, so a change of purpose is a change of materiality. Exposure moves loudly and can be automated against a threshold. Purpose moves quietly, usually as a sensible integration nobody documents, which is why the purpose sentence needs a confirmation date and a named re-tier trigger.
Before session 2
40 minTake whatever your organisation calls its model inventory and answer one question against it without asking a human: which models depend on the single most widely used data feed. Time yourself. If the answer takes more than a query, write down which column was missing, because that column is your finding. Then pick one model you own and fill in the two halves of materiality properly: the exposure measure with its unit and value, and the purpose sentence with the decision and the person who acts on it. Bring the purpose sentence to session 2, which reads a model's design before touching any number, and starts with exactly that sentence.