What your name is actually going on

Start from the definition the track opened with. Model risk is the potential for adverse financial consequences associated with models, which may result from decisions made based on model output. Read that back slowly and notice what it is not about. It is not about whether the model is right. It is about consequences, which arrive through decisions, which are made by people. A signature that claims the model is correct is claiming something no signature can carry, and the moment it is tested it collapses into an argument about a coefficient.

A sign-off that holds asserts three things instead. Understood: the risk this model carries has been identified, including its limitations, and it has been written down in language the people making the decisions can act on. Bounded: the use is limited to what the evidence supports, and each limit is a condition with an owner, a trigger or a date, and a consequence if it is missed. Owned: for every one of those conditions and every residual limitation, a named person is accountable, and it is knowable who that is without a meeting.

Each of the three is checkable by somebody who cannot read a model, which is the point. If you can say what the risk is, what the limits on use are, and who owns each of them, you have signed something you can defend. If any of the three is missing, your signature is doing work you did not intend it to do.

• Accountability. Who answers for an activity, as distinct from who performs it. Delegating the second never moves the first.
• Conflict of interest. Named in the guidance, with misalignment of incentives between different reporting lines, such as model development and validation groups, given as the example.
• Internal audit. The third line. It evaluates whether model risk management practices are rigorous and effective, rather than performing them. Session 2 covers this in full.
• External resources. Work brought in from outside to help manage model risk. Oversight stays inside, and the work has to be integrated rather than filed.
• Vendor model. A product bought rather than built, which may include data, parameter values or a complete model, and whose components may be proprietary.
• Customisation. Anything your organisation changed on the way in. Documented, justified, and evaluated as part of validation.
• Residual risk. What remains after sound modelling and rigorous validation, which the guidance is explicit can still be material.
• The refusal. Declining to approve. Correct when no bounded use of the model is defensible, and wrong as a stronger way of saying that conditions are needed.

The guidance asks for clear roles with well-defined accountability, and it does something unusual immediately afterwards: it says that the accountability has to be clear specifically with respect to potential conflicts of interest, and it gives an example rather than leaving it abstract. The example is misalignment of incentives between different reporting lines, such as model development and validation groups. That is a supervisory document naming, in writing, the thing everybody in the industry already knows and rarely says at a committee table.

What makes it worth dwelling on is that it is a statement about incentives rather than about integrity. Nobody has to behave badly for this conflict to bite. A development group is measured on delivery and a validation group on the quality of challenge, and those two measures pull against each other at exactly the moment a launch date is close. If both report to the same person, that person resolves the tension privately, using judgement, with no record. The resolution may well be the right one. What is missing is the trace.

Session 2 covered the underlying argument in full, including the 2026 change that moved the test from organisational structure to the rigour of the review. Tonight only needs the consequence for the signature. If your validators and your developers share a reporting line, that is not disqualifying, and the burden of demonstrating rigour sits with you rather than with the guidance. Before signing, know two things: who resolved any disagreement between the two groups on this model, and whether that resolution is recorded anywhere you could show somebody.

The practical form of "clear roles" is unglamorous. For each of the four lifecycle stages, one name. Not a function, not a committee, a person, with a deputy if the model is material enough to need one. A governance map made of job titles reads well in a policy and produces a two-week delay the first time something fires, because every title has to be resolved into a person under pressure.

Internal audit. The line is in the guidance and it is worth having exactly: internal audit would generally not duplicate model risk management activities such as model development or validation. Instead, internal audit's role is generally to evaluate whether the model risk management practices are rigorous and effective and whether related policies are implemented accordingly. Session 2 taught why that distinction matters and what goes wrong when it is blurred in either direction. On the accountability map it simply sits above everything: audit is not a stage in the lifecycle, and its findings are about your framework rather than about Brackwater's scorecard. If audit comes back with a view on whether the scorecard is any good, something has gone wrong upstream of audit.

External resources. Consultants, contractors, outsourced validation, a third-party reviewer brought in because the internal team is at capacity. All legitimate, and widely used. Where an organisation uses external resources to help manage model risk, sound practice involves maintaining proper oversight of that work and integrating it into the broader model risk management framework, with roles clearly defined when activities are delegated. Read the three requirements separately, because organisations tend to satisfy one and assume the others.

Oversight means somebody inside the organisation is close enough to the work to know whether it was done well, which implies enough internal expertise to judge it. Integration means the output lands in the same inventory, the same findings log and the same remediation tracking as everything else. An external validation report that lives in the procurement folder has been bought rather than used. And defined roles means it is written down, before the engagement, which parts of the activity the external party owns and which parts remain internal, because the boundary is where things fall through.

What none of the three do is move the accountability. You can delegate the analysis. The answer is still yours, and the signature at the end of it is still yours.

Accountability maps have a reputation for producing org-chart projects, which is why they so often stall. The version that works is deliberately smaller: one model, four names, one page, produced in an afternoon. The exercise is not to design a target operating model. It is to find out whether the four names exist.

Three failures show up almost immediately and all three are findable without anybody's cooperation. An empty cell, usually ongoing monitoring, because monitoring is the stage with no launch event and therefore no natural moment at which somebody is handed it. One name in two cells, usually development and monitoring, which means the person who built the model is the person who watches it, and nobody will find out that it has drifted from a source with an interest in it not having drifted. A name that has left, which is the commonest of all and the reason the guidance ties documentation to continuity of operations.

None of those three require a restructure to fix. They require somebody to notice, put a name in the cell, and tell the person that their name is in it. The last clause is not a joke: a surprising share of accountability maps contain people who have never been told.

The argument that arrives with every bought model is that the usual expectations cannot apply, because the usual expectations assume you can open the thing. It is half true and it is used as though it were wholly true. The guidance is direct on this: the principles still apply even where components are proprietary. What changes is the evidence available to you, not the standard you are held to.

Sound practice for a vendor model has four parts, and it is worth noticing how little of it depends on seeing the code.

Develop an understanding of the vendor model's conceptual soundness, design, development data and performance. The vendor will not hand over the methodology in full, and you can still establish a great deal: what class of model it is, what it was built on, which population that data represented, what it was tested against, what the vendor claims it does and does not do. A vendor who will not answer any of that is itself a finding, and a commercially useful one, because it is a question you can ask before the contract is signed rather than after.

Conduct ongoing monitoring and outcomes analysis. This is the part that is fully in your hands and is most often skipped. The vendor's performance figures were produced on the vendor's data. Whether the model is accurate on your book, on your applicants, at your cutoffs, is a question only you can answer, and it is answered with the same outcomes analysis you would run on anything you built. A bought model is not exempt from the question session 5 kept asking, which is whether it performs by band rather than only in aggregate.

Assess whether it remains fit for purpose and continues to be reliable. Fit for purpose is about your use, which the vendor knows nothing about and cannot certify. A model sold for one decision and deployed against another is the same scope-creep problem session 4 described, with the added difficulty that the boundaries of intended use were written by somebody with an interest in them being broad.

Document, justify and evaluate every customisation. Covered in the next card, because it is the part your organisation genuinely owns.

Almost nothing is installed as shipped. Thresholds get retuned, a segment gets excluded, an input gets mapped from a field that means something slightly different in your systems, an override is added for a population the vendor did not anticipate. Each of those is a customisation, and customisations should be documented, justified and evaluated as part of validation.

Take the three verbs separately, because they fail separately. Documented is the easy one and still commonly missing, since customisation usually happens during implementation rather than during modelling, and implementation notes are not model documentation. Justified means there is a recorded reason, which matters most for the changes made under time pressure in the last fortnight of a project. Evaluated as part of validation is the one with teeth: the customised model, not the shipped model, is the thing in production, so it is the thing that has to be assessed. A validation of the vendor's product that was performed before your changes has validated something you are not running.

There is a governance sharp edge here worth saying plainly. Customisation is where a bought model quietly becomes a built one. At some point the accumulated changes mean your organisation is making the modelling choices, and the comfortable position of "we bought it, they built it" stops being true while nobody notices the transition. The signature is where that gets tested: if you are approving a model whose behaviour is substantially set by your own adjustments, you are approving a model you built, on evidence produced by somebody else.

Validation generally occurs prior to a model's first use. That is the expectation, and it is the right default. The guidance also recognises that an urgent business need may mean a model is used before validation is complete, and it says what sound practice looks like when that happens: greater attention to the model's limitations, informing the relevant stakeholders, and placing controls on the use, such as limits or closer monitoring.

Session 2 covered this as the legitimate route, and the reason to repeat it here is that it is the single most useful thing a leader can know on a bad night. The path exists. It is narrow, it is documented, and using it is a decision you can defend. Not using it is what produces the failure mode: a model goes live because the business need was genuinely urgent, nobody wanted to write down that validation was incomplete, and so no limitation was recorded, no stakeholder was told, and no control was placed. The model has then been deployed in exactly the same state as the defensible version, with none of the three things that would have made it defensible.

The discipline is to treat the early-use decision as its own sign-off with its own conditions. What is not yet known, who has been told that it is not known, what limit applies until it is known, and what date the validation completes by. Brackwater did not use this route. The scorecard went live with a validation that had been performed and whose central finding had been recorded as an observation, which is a different failure and a worse one, because on paper it looks finished.