A condition is a control, or it is a sentence in a document

The most useful habit you can bring to one of these documents is to stop reading it as a report on a model and start reading it as an argument about a claim. Somebody asserted something, in writing, in the model document. Validation evaluates whether the model performs as expected, assesses its reliability and its limitations, and clarifies appropriate use and whether corrective actions may be warranted. Everything in the report exists to support or undermine that assertion, and a report that never states the assertion is asking you to grade an answer without showing you the question.

This matters because of what it does to the reading order. If the claim is stated up front, every finding underneath it has an obvious weight: it either survives the claim, narrows it, or breaks it. If the claim is missing, the findings become a list of technical observations of unknown importance, and the only person in the room who can weigh them is the person who wrote them. That is how a report ends up being summarised to a committee by its own author, which is a governance failure with a perfectly pleasant surface.

• The claim. What the model document asserts, in one sentence. The thing the report is about.
• Finding. A statement of what the evidence showed, written so that the model owner can agree with the fact even while disagreeing with the grade.
• Severity. The weight attached to a finding. A judgement about consequence, made by the validator and signed off by somebody, and the field most often quietly edited.
• Observation. A finding that has been graded as not worth a condition. Legitimate, and also the drawer that findings disappear into.
• Condition. Something that must happen before or during use. A control if it names an owner, a trigger or a date, and a consequence. Otherwise a sentence.
• Limitation. A boundary on the claim that no amount of remediation removes. It belongs in the report and then in the use.
• Scope. What was tested, and by omission what was not. An untested area is not a passed one.
• Recommendation. One of three answers: approve, approve with conditions, or decline to approve.

Session 3 gave the three components of validation in the order the guidance uses them: conceptual soundness, outcomes analysis, and ongoing model monitoring. A report organised around those three headings is one a non-specialist can follow, for a reason that has nothing to do with taste. The three headings exist independently of what anybody ran. If the outcomes analysis section is thin, you can see that it is thin. If there is no ongoing monitoring section, the absence is on the page.

A report organised around the work that happened has the opposite property. Its sections are the tests somebody chose to run, so every section is full, every section reports a result, and the document looks complete no matter how much was skipped. There is no empty heading, because the headings were written after the work. You are reading a list of things that were done, presented as an assessment of a model, and the only way to find the gap is to already know what should have been there.

The leader's move is not to redesign anybody's template. It is to ask, for each of the three components, which section covers it and what it concluded. Three questions, and a report that cannot answer one of them has told you what it is.

Severity is the field that decides whether anything happens, and it is graded by people who have just spent six weeks on the work. Two failure modes follow from that, and they pull in opposite directions.

Grading by difficulty. A finding that took three weeks of forensic effort to uncover feels heavier than one that was obvious on the first afternoon. It is not. Severity is a statement about what happens if the finding is correct and nothing is done about it, and the effort spent discovering it is information about the review, not about the model. The obvious finding that puts the whole claim in doubt outranks the ingenious one that narrows it slightly.

Grading by embarrassment. The mirror image, and the more common one. A finding is graded down because the consequence of grading it up is a delayed launch, an awkward conversation, or a colleague's reputation. This is rarely done dishonestly. It is done by adding qualifiers until the sentence is true and weightless, and the tell is that the finding text and the severity no longer describe the same event.

The discipline that fixes both is to write the consequence before choosing the grade. One sentence: if this finding is right and nothing changes, the following happens to the following decisions. Grade that sentence. If the sentence is hard to write, the finding is not yet understood well enough to grade, and that is itself worth saying out loud.

Four findings, in the order a reader can follow rather than the order they were discovered. None of them require you to evaluate a metric; each is a statement about whether the submitted evidence supports the submitted claim.

• A field that could not have been available at decision time. The collections field records contacts logged after the outcome window opens, so it is a consequence of default rather than a predictor of it. On its own it discriminates at 0.748 against the submitted 0.863. Refit without it, development performance falls to 0.772, so the leak was worth 0.091. Consequence if right: the headline number in the model document is not a number the model can reproduce in production. This is the finding that decides the report.
• Performance on a later vintage. The honest model, scored on the following year's 2,000 applications, discriminates at 0.682, a further drop of 0.090. Consequence if right: the approved performance and the lived performance are different numbers, and the cutoff calibrated on the first is being applied to the second.
• Calibration that is sound in aggregate and wrong in every band. Out of time, the model predicts a mean default rate of 0.330 against an actual 0.324, a gap of -0.006 and as close as anybody reports. By quintile of predicted risk the picture inverts: the safest fifth is under-predicted by 0.061, and the riskiest fifth is over-predicted by 0.142. Consequence if right: the aggregate figure that was reported would have passed this model, and the decisions are made in the bands.
• A population that moved. The share of thin-file applicants in the book went from 19.0 percent to 34.0 percent between the two vintages. Consequence if right: the growth is concentrated in the applicants the model has least evidence about, and no monitoring threshold was written that would have fired.

There is a fifth thing in the pack that is not a finding, and it is worth naming as an item in its own right. A three-feature benchmark, built from columns anybody could have written, scores 0.676 out of time against the champion's 0.682. That is not a finding about a defect. It is a finding about what the eleven features, the leak and the validation cycle actually bought, which is 0.006.

A condition is a promise that the model's use is bounded until something happens. Three things have to be in the text for that promise to mean anything, and the reason each one is necessary is worth having separately, because conditions usually fail on exactly one of them.

An owner, and a person rather than a role. "The modelling team will refit the scorecard" assigns the work to an abstraction. Teams reorganise, priorities move, and no abstraction has ever taken a phone call. A named individual can be asked in a corridor, can hand the item over explicitly when they move on, and can say no, which is information you want early. The test is whether you could send one email. If you would have to ask who owns this first, the condition has no owner.

A trigger or a date, written now. Time-bound conditions get a date. Event-bound ones get a trigger: when the book crosses a stated size, when the next vintage closes, when the feed definition changes, when the model is put to a new use. Either is acceptable and the mixture is common. What is not acceptable is "as soon as practicable", which is a date chosen later by whoever is least inconvenienced by the delay. A trigger written in advance also has a property a date does not: it survives the plan changing, because it describes the world rather than the calendar.

A consequence if it is not met, decided before it is needed. This is the part that is almost always missing, and it is the part that makes the other two load-bearing. If the condition is not met by the date, what happens: does the model stop, does a use narrow, does the approval lapse, does it escalate to a named committee. Decide it now, while the model is not yet in production and nobody is under pressure. Decided later, in the week the date is missed, the answer is always an extension, because by then the model is running the book and stopping it is a business event.

Two smaller tests catch most of what survives those three. Is the condition observable by somebody who is not thinking about model risk that day, and is it recorded somewhere other than the report. Documentation supports the tracking of recommendations, responses and exceptions, and a condition that lives only in a PDF in a shared drive is not being tracked, whatever the policy says. Session 4 made the same point about the register: the artefact that holds the item is the one that decides whether the item exists.

Approve. The evidence supports the claim, the limitations are stated, and the use described in the model document is the use being approved. This is not a rare answer and it should not be treated as suspicious. A well-built model with honest evidence and a clear statement of what it cannot do earns a clean approval, and a function that never gives one is teaching its model owners that the report is theatre.

Approve with conditions. The model can be used, and the use is bounded until stated things happen. This is the answer Brackwater's evidence supports, and it is worth being precise about why. Nothing in the four findings says the scorecard cannot rank applicants. What they say is that the number in the document is not the number the model produces honestly, that the later vintage performs worse than the approved figure, that the aggregate calibration conceals the band-level picture, and that the population has moved toward the applicants the model knows least about. Every one of those is addressable, and every one of them is a boundary on use until it is addressed. That combination is the definition of this recommendation.

Decline to approve. Reserved for a model that cannot be used safely at all in the use proposed. Not a model that disappoints, not a model whose author was careless, and not a model whose findings are numerous. The test is whether a bounded use exists: if you can describe a narrower use, a lower limit, or a closer monitoring regime under which the model is defensible, the answer is conditions, and the narrowing belongs in the conditions. If no such setting exists, conditions are dishonest, because you would be writing controls around a use that should not happen. Session 6 is about that answer, including how to give it without leaving the business stranded.

One structural point holds across all three. The recommendation is the last thing in the report and the first thing everybody reads, which means it routinely gets negotiated independently of the findings above it. When a recommendation is upgraded, the findings almost never change to match, and the document is left arguing with itself. A report whose recommendation does not follow from its own severity grades has recorded that a decision was made somewhere else, and the useful question is where.

There is a sentence everybody in the room reaches for once the findings are on the table, and it is wrong. It is some version of "the champion is worthless, the challenger is better, build the simple one." On this evidence that claim cannot be sustained. Out of time the champion scores 0.682 and the three-feature challenger 0.676, which is a difference of 0.006 and well inside the noise of a 2,000-application sample. At their respective cutoffs they win on different axes: the champion approves 69.3 percent of applicants with 24.7 percent bad among those approved, the challenger approves 72.2 percent with 25.3 percent bad. The champion buys 0.6 points of bad rate and gives up 2.9 points of volume. Which of those a lender prefers is a business question, not a validation finding.

The defensible finding is narrower and sharper: the evidence submitted did not support the claim made. That is a statement about the model document, not about the model, and it is the one you can hold in front of a supervisor, a committee or the model owner without retreating. It also survives the most likely counterattack, which is a rebuttal about the challenger's own weaknesses, because the finding was never about the challenger.

Notice what the narrower claim does to the conditions. If the finding is "the model is bad", the only condition available is "build a new one", which is not a control and will not happen on any timescale you can name. If the finding is "the evidence did not support the claim", the conditions write themselves: produce honest evidence for the claim you want to make, or make a smaller claim. That is achievable, datable and ownable, which is the whole point.

The last thing to carry out of the report is what it does not remove. Even with sound modelling practices and rigorous validation, material model risk can remain. That is a statement in the guidance, not a pessimistic aside, and it puts a short list of obligations on the people who use the output rather than on the people who validated it: understand and communicate the model's limitations, monitor performance, periodically review whether the model is still relevant, and supplement the output with complementary analysis and information. Session 3 covered that list from the validator's side. Tonight it arrives as the thing a signature does not buy: the conditions bound the use, the report bounds the claim, and the residual risk stays with the business.