Rollout and measuring quality

Four numbers, and each one only moves when the work gets better:

• Conformance - the share of screens that pass all five measurable checks the first time they hit review. The five checks are contrast, spacing rhythm, type scale, hierarchy and colour restraint, and two of them are formal standards you can cite: WCAG 2.2 SC 1.4.3 for text contrast at 4.5:1 normal and 3:1 large, and SC 1.4.11 for 3:1 on control boundaries and meaningful graphics.
• Rework - review rounds per shipped screen. This is the number a generator makes worse if direction is thin, because volume without a brief produces more things to reject. Falling rework is the clearest evidence a standard is working.
• Coverage - states designed per feature, not just the happy path. Count empty, loading, error, offline, permission-denied, long-content and the AI-uncertainty state. Generated work forgets these almost every time, and they are where real users live.
• Distinctiveness - can a stranger tell your work from a competitor's. This one is a judgement call, scored by humans, and pretending otherwise would be dishonest. The next card is how to score it repeatably anyway.

Keep asset count and cycle time on the dashboard if you like, but move them into a section labelled "context, not targets". They are useful for capacity planning and useless as evidence of quality.

A judgement metric earns its place on a dashboard only if the method is fixed. Write the protocol down, run it the same way every quarter, and it becomes comparable even though it is subjective:

• Blind the set. Take five of your screens or assets and five from a fixed competitor set chosen at the start of the year. Strip logos, product names and any wordmark.
• Ask one question. "Which of these came from the same company?" Grouping accuracy is your real signal. If reviewers cannot group your own work together, you do not have a visual point of view yet.
• Then score 1 to 5 on "this could only have come from one company". Three reviewers minimum, at least one from outside the design team, and take the median rather than the mean so one strong opinion cannot move the quarter.
• Keep the sheet. Same competitor set, same question wording, same reviewer mix. The absolute number matters far less than the direction over four quarters.

State the caveat in the report itself: this is a scored human judgement, not a measurement. That sentence is what stops a finance partner treating 3.1 as though it were revenue.

Three mechanisms, all of them mechanical rather than sinister. First, generation genuinely collapses the cost of a first draft, so drafts multiply whether or not anyone briefed them. Second, credit systems reward usage: when a tool bills by credits and the budget is already spent, the incentive is to use it, not to use it well. Third, review capacity does not scale with output, so a rising asset count arrives at a review queue of the same size and the queue starts approving faster.

The failure mode is easy to describe and hard to see from a dashboard: throughput up, conformance flat, rework up, distinctiveness down. That is a rollout getting worse while every headline number improves. Conformance and rework are the two numbers that catch it early.

Ships: a one-page written standard with the five measurable checks, each cited where it is a standard rather than a convention. Contrast is WCAG 2.2 SC 1.4.3, non-text contrast is SC 1.4.11, enhanced contrast is SC 1.4.6, text spacing is SC 1.4.12, minimum target size is SC 2.5.8 at 24 by 24 CSS pixels with exceptions. Type scale, spacing unit and "one loudest thing" are conventions, and the page must say so. Also ships: the review ritual, meaning a fixed thirty-minute slot where screens are scored on the five checks before anyone discusses feeling, plus the Q1 baseline for all four metrics.

What could go wrong: the standard becomes a twelve-page document nobody reads. Cap it at one page. Second risk: you write it alone, and then it is your standard rather than the team's. Draft it with two designers and one engineer in the room so the enforcement in Q3 is not a surprise.

Ships: at most two tool pilots, each with a success test written before the pilot starts and phrased in the Q1 metrics, for example "conformance on first review rises from the Q1 baseline on the same class of screen". Also ships: a metered budget per pilot. Credits are the real constraint in this space rather than seats, so state the monthly credit ceiling, who can spend it, and what happens when it runs out. Figma's credits do not roll over and are not shareable, and Figma Make alone is documented at 30 to 100 or more credits per task, so a small pilot can exhaust a plan quickly. Canva now meters a shared credit pool across quality tiers per member rather than counting features. Check today's numbers before you sign anything: this is the part of the market that changes monthly.

What could go wrong: the pilot gets judged on enthusiasm. Write the success test first and archive it, so the review reads the test rather than the room. Second risk: four tools instead of two, because every team wants its own. That is the purchase pattern the stop list exists to break.

Ships: the five checks moved out of human memory and into tokens, components and automated checks. Practically that means a token set, a component library that makes the compliant thing the easy thing, and a contrast and spacing check that runs without a designer present. Two constraints to plan around honestly. The W3C Design Tokens Community Group format's stable version is 2025.10, published as a Final Community Group Report on 2025-10-28, and it is explicitly not a W3C Standard, so treat it as a useful interchange format rather than a guarantee. And the Figma variables REST API requires a Full seat in an Enterprise organisation, which means below Enterprise your only documented token route is selection-scoped access through the Figma MCP server's get_variable_defs. If token automation is in your plan, that single line decides your budget.

What could go wrong: a beautiful token file that no product surface consumes. Tie the quarter's success to a shipped screen built from tokens, not to the file existing. Second risk: Code Connect's framework-specific parsers lose updates and support from 2026-08-17, with template files as the only maintained path, so verify which path your team is on before you build on it.

Ships: the ladder change a5 argued for, written into the actual levels document. Critique quality, direction quality and systems contribution become named expectations rather than folklore, and the junior role is redefined around briefing and critique rather than production volume alone. Hiring changes with it: portfolio review asks how a candidate got there and what they rejected, not only what the final frame looks like. Then re-measure all four metrics on the same class of screen you baselined in Q1, and publish the comparison whether or not it flatters the rollout.

What could go wrong: the re-measure quietly uses easier screens, and the year reports a win it did not earn. Freeze the screen class in Q1 and write it down. Second risk: the ladder changes on paper while promotion decisions carry on as before. Pick the first two promotions under the new wording and review them against it in public.

Three categories, and keep the list this short so people remember it:

• Anything carrying a brand mark. Reference parameters buy family resemblance, not brand compliance. Midjourney's own documentation says style references do not copy objects or people, only the overall style, and that references are used as inspiration rather than copied exactly. A generated logo lockup is a defect by construction.
• Anything customer-facing at scale. One person's plausible screen is a draft. Ten thousand impressions of it is a brand position you did not choose.
• Anything asserting a fact. Numbers, claims, comparisons, capabilities, availability. A generator produces the most likely sentence, and the most likely sentence is not a verified one.

Everything outside those three is fair game with normal review. That is deliberate: a guardrail list long enough to cover every case gets ignored in week three.

One rule, stated once: every generated asset that ships carries a record of how it was made. Model and version, the prompt or brief, any reference images and their licence, the tool, the date, and the human who accepted it. Keep it with the asset rather than in someone's notes, because the moment you need it is the moment that person is on leave.

On the standards side, C2PA is at spec version 2.4, and its Conformance Program launched mid-2025 with a Trust List whose interim list froze on 2026-01-01 with no new entries being added. Adobe applies Content Credentials automatically where 100% of the pixels are Firefly-generated and across the Firefly web app and APIs, and elsewhere it is opt-in, which in Photoshop lives under Window then Content Credentials. Teach the limit as clearly as the capability: Content Credentials prove a signed history of how an asset was made and edited. They do not prove an unsigned asset is fake, and they do not prove a signed asset is truthful.

Somebody in your organisation should be able to answer "is our unreleased design work training a model" without opening a browser. Today, that person usually cannot. The defaults are not neutral.

• Figma has two toggles, content training and AI features. Content training defaults on for Starter and Professional teams, and is off and not toggleable on Organization and Enterprise. The policy took effect 2024-08-15, and Education and Government accounts are excluded.
• Canva governs training use through Privacy Settings, and its rights language is worth reading before a commercial launch: you own your Output except for output that modifies or incorporates Licensed Content, outputs may not be unique, and Canva recommends independent advice before commercial use.
• Midjourney plan tiers matter for commercial terms in one specific way: terms are the same across tiers except that companies grossing over 1 million USD per year must be on Pro or Mega. Stealth mode, meaning your generations are not public, is Pro and Mega only.
• Adobe frames commercial safety as a design goal rather than a warranty, and for the 30-plus partner models now hosted in Firefly it states it cannot verify commercial safety and points to the partner's terms. Indemnification is an enterprise entitlement that may be purchased, scoped by published lists of eligible features and surfaces, not a default. We could not reach Adobe's enterprise legal FAQ to verify the exact conditions, so treat that scope as something your counsel confirms in writing.

Write your answer for each tool on the guardrail page, with the date you checked. All of this changes, and a dated answer is honest in a way an undated one is not.

Three habits, and stopping them costs nothing:

• Stop running volume-based creative reviews. A review that opens with forty options is a review that will pick the most familiar one. Bring three directions with the reasoning, not forty with none.
• Stop having taste debates without a scored baseline. Score the five checks first, in ninety seconds. Whatever survives that is a real disagreement about the judgement half, and worth your time.
• Stop buying a tool per team without a metering budget. Credits do not roll over, are not shareable, and vary by seat and plan. Four unmanaged pilots is how a design budget disappears without producing a single defensible number.

Add a fourth if you have the appetite: stop accepting "it just needs polish" as review feedback. Name the measurable failure or name the judgement failure, but name one.