The last mile: from "it works" to "we can trust it"
Every workflow in this course was built to a rail: enterprise tools and data boundaries, AI assists while the human decides and owns, and audit everything. This session takes those rails and turns them into a checklist you can run - the thing that stands between a clever prototype and a workflow finance, risk, and IT are willing to put their names on. It is the same governance a leader session a3 covers from the top down, done here from the builder's bench.
Demo to production: the risk stack 7 min live
A prototype only has to work once, for you, on a good day. A production workflow runs unattended, on other people's data, with decisions riding on it - and that opens a stack of risks a demo never had. Know the five that bite finance workflows before you build the controls that stop them.
LiveThe five risks a demo never had4 min▶
Each of these is invisible in a one-off demo and expensive in production. Name them out loud - you cannot control a risk you have not written down.
- Prompt injection. Hidden instructions inside a document, email, or web page tell the model to ignore your rules, leak data, or take an action. It bites hardest exactly where finance lives now: document Q&A / RAG (session b7) and any agent that reads untrusted content and can act.
- Data leakage / exfiltration. Confidential financials, client PII, or MNPI end up in a consumer tool, a log, a pasted prompt, or an output shared too widely. The more of a workflow you automate, the more places data can escape.
- Over-reliance. Once a workflow looks polished, people stop checking. A plausible wrong number ships because "the tool said so" - the exact failure the human gate exists to stop.
- Model drift / change. The vendor updates the model and your prompts behave differently overnight. A workflow validated in March can quietly regress in April with no code change on your side.
- No audit trail. Something goes wrong and you cannot reconstruct what the AI was asked, what it returned, or who approved it. Without a trail there is no investigation, no rollback, and no defensible answer for an examiner.
The controls 7 min live
Every risk maps to a control you already half-know from the rails in this course. Production is just applying them on purpose, writing them down, and being able to show they are on. Here is the risk-to-control map.
LiveMap each risk to its control4 min▶
| Risk | The control that stops it |
|---|---|
| Data leakage | Enterprise tools + hard data boundaries; least-data principle - send only the fields the task needs, never MNPI or client PII into consumer tools. |
| Over-reliance | Human-in-the-loop gate on every decision; AI drafts and flags, the licensed human reviews, decides, and owns the call. |
| No audit trail | Logging + audit trail: capture the prompt, source, model + version, output, and who approved - for every AI-assisted output. |
| Prompt injection | Injection awareness: treat retrieved documents and agent inputs as untrusted; constrain tool actions; keep a human gate before anything acts. |
| Model drift / change | Version pinning + change management: pin the model where you can, re-run your test set on any model change, log the version used. |
| Unmanaged sprawl | Model inventory (SR 11-7 discipline): list every AI-assisted workflow, its owner, purpose, tier, and review date. What is not inventoried is not governed. |
Self-studyReview cadence - governance is not one-and-done2 min read▶
Controls decay. Models change, prompts get edited, the business shifts, and a workflow signed off in Q1 may not be safe by Q3. Set a review cadence proportional to the stakes:
- High-stakes workflows (feed a decision, a filing, a control): quarterly review at minimum, and an immediate re-check on any model change.
- Low-stakes assists (draft narrative, formatting, summaries a human rewrites): annual review, or when the tool visibly changes.
- Every workflow re-runs its test set when the vendor announces a model update - drift is the risk you cannot see coming.
The production checklist + sign-off 5 min live
A control that no one owns is not a control. Going live means three functions sign off, monitoring is on, and there is a written plan for the day it goes wrong. This is exactly the gate leader session a3 asks the business to enforce - here is the builder's version of passing it.
LiveWhat "ready" actually requires3 min▶
- Three sign-offs. Finance owns accuracy and the business use; risk / model risk owns the tier and review cadence (SR 11-7); IT / security owns the data boundary and access. No single person clears a production workflow alone.
- Monitoring on. Someone watches outputs and usage after go-live - not to babysit, but to catch drift, misuse, or a spike that signals something changed. A workflow no one looks at is a workflow no one governs.
- Incident + rollback plan. Written before you need it: how you know something is wrong, who to tell, how to turn the workflow off, and how to revert to the manual process. If you cannot turn it off cleanly, it is not ready.
The forecast that shipped without a rollback path. A team put an AI-assisted rolling forecast into the monthly cycle - accurate, well-liked, no incident plan. A vendor model update shifted its scenario logic mid-quarter and the numbers drifted. Because there was no version log and no defined revert, the fix took a scramble of two days instead of a clean switch back to the prior spreadsheet. The lesson was cheap only because someone spot-checked. The rollback plan is the control you are gladdest to have never needed.
Run the readiness checklist against a workflow you built ★ 13 min · everyone builds
No new tool tonight - you audit your own work. Pick ONE workflow you built earlier this course (the b3 close copilot, the b4 forecast, or a b5 report) and score it against each control. The output is a one-page readiness scorecard with gaps and next actions.
Choose the workflow to audit. Write one line: what it does, who would use it, and what decision it feeds. That line sets the stakes - and how heavy the controls need to be.
Score each control Green / Amber / Red: enterprise tool + data boundary, least-data, human-in-loop gate, logging/audit trail, injection awareness, version pinning, inventory entry, review cadence, named owner.
For every Amber or Red, write the single next action that would make it Green - "add a version-logged line to the run", "name the finance sign-off", "stop pasting the full ledger, send only the reconciled lines".
Have the AI help you build the scorecard, then you set the honest scores - the tool does not know your data boundary or who signs off. This is rail two: AI drafts the frame, the human owns the judgement.
Read the Reds out loud. If any high-stakes control is Red, the workflow is not ready to turn on - and now you know exactly why, and what to fix first.
Write the incident note + audit-log spec ★ 8 min · build your own
Two short artefacts finish the readiness pack: a plain-language incident/rollback note for the day it breaks, and an audit-log spec that says exactly what to record for every AI-assisted output. Both are things you can hand to IT and risk tomorrow.
Draft the incident note for the same workflow: how you would notice something is wrong, who to alert, how to switch the workflow off, and how to revert to the manual process. Keep it to half a page.
Define the audit-log spec: the fields to capture for every AI-assisted output so the run is reconstructable months later. At minimum - timestamp, user, prompt, source/document, model + version, output, and who reviewed/approved.
Pressure-test the rollback: can you actually turn this off and go back to the prior process cleanly? If the answer is "not really", that is your top Red from Demo 1 - flag it.
Attach both to the scorecard from Demo 1. That three-part pack - scorecard, incident note, log spec - is your production-readiness artefact for this workflow.
The log line that answered the examiner. A controllership team logged model + version and the approver on every AI-assisted journal narrative - two extra fields, near-zero effort. Months later an internal audit asked "how do you know this commentary was reviewed, and by what model". They pulled the log and answered in minutes. The teams that could not answer that spent weeks reconstructing. The AI did the drafting; the log made it defensible.
Try it yourself - this week ◐ 30-45 min total
- Finish the three-part readiness pack for your chosen workflow if you did not complete it live: scorecard, incident note, audit-log spec.
- Run the same scorecard against a second workflow you built this course. Notice which controls are consistently your weak spot - that is your personal thing to fix by default.
- Take one Red and actually close it this week - name the sign-off, add the version log line, or tighten the data sent. Move it to Green for real, not on paper.
- Write your one-line model-inventory entry for the workflow: owner, purpose, tier, review date. That single line is where SR 11-7 governance starts.
- Optional: read leader session a3 to see the same governance from the top-down view - it is the conversation your scorecard is designed to survive.
Official sources covered
This session turns the model-risk and security canon into a builder's readiness checklist, mirroring leader session a3 from the practitioner's bench. Certifications and vendor governance tooling stay with the official programs - said honestly. This session covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · What is prompt injection, and where does it bite finance workflows hardest?
Prompt injection is malicious instructions buried in a document, email, or web page that hijack the model. It is most dangerous exactly where finance is heading: RAG over documents and agents that read untrusted content and can take actions.
2 · Why is an audit trail a non-negotiable control?
Logging the prompt, source, model + version, output, and approver is what lets you reconstruct what happened, roll back, and answer an examiner. Without a trail, a problem is unprovable and unfixable.
3 · The AI-assisted workflow produces a number that feeds a decision. Who decides?
The hard rail all course: AI assists, the licensed human decides and owns the decision. The human-in-the-loop gate is the control that stops over-reliance from shipping a plausible wrong answer.