Two success criteria, one letter apart, and the difference is everything
4 min liveYou have heard somebody say "we're accessible, we have transcripts". That sentence is true at one conformance level and false at the one every legal mandate actually points at, and almost nobody in the room knows which is which.
Here is the whole thing, in W3C's own words. "At Level A in Success Criterion 1.2.3, authors do have the choice of providing either an audio description or a full text alternative. If they wish to conform at Level AA, under Success Criterion 1.2.5 authors must provide an audio description."
Read it twice. At Level A, a transcript substitutes for audio description. At Level AA, that escape hatch closes. The DOJ's ADA Title II web rule adopts WCAG 2.1 Level AA. So does most of what anyone means when they say "the standard". Which means the transcript-on-the-page answer, the one your organisation almost certainly gives, is a Level A answer wearing an AA badge.
Notice what this figure is not saying. It is not saying transcripts are worthless. A transcript is a genuine conformance route at Level A, and it is genuinely useful to real people. It is saying that the sentence "we have transcripts, so we're compliant" is answering a question nobody asked, because the level being demanded is the one where that answer does not count.
Captions are a separate obligation and they sit lower still. SC 1.2.2 Captions (Prerecorded) is Level A, and it reads: "Captions are provided for all prerecorded audio content in synchronized media, except when the media is a media alternative for text and is clearly labeled as such." That is the floor. Everything in this session is about what sits above it.
LiveWhich version of WCAG you are actually being held to3 min▶
Three facts that stop most of the version confusion in this area.
- Guideline 1.2 is unchanged from WCAG 2.0. Nothing was added to it in 2.1 or in 2.2. So the 1.2.3 versus 1.2.5 fork you just read is not new, is not contested, and has been sitting in plain sight for well over a decade.
- WCAG 2.2 is a W3C Recommendation of 12 December 2024. That is the current finished version.
- WCAG 3.0 is still a Working Draft. Its own status text says "It is inappropriate to cite this document as other than a work in progress". If somebody hands you a compliance plan built on WCAG 3.0, they have built it on a draft that says not to.
And one more that catches federal work specifically: Section 508 still incorporates WCAG 2.0, so US federal procurement is anchored a full version behind the ADA Title II rule. Two different parts of the same government are pointing at two different documents. You need to know which one is pointing at you.
7.0 percent average, and a per-file range that runs to 53.8
7 min liveThere is a real peer-reviewed benchmark for this, and almost nobody quotes it because the vendor numbers are easier to find. Kuhn, Kersken, Reuter, Egger and Zimmermann, in ACM Transactions on Accessible Computing, put 11 commercial and open ASR services through 120 three-minute samples: 221 hours of audio, 3,840 transcriptions.
The averages are the headline and they are not the story. Per-file WER ranged from 0 percent to 53.8 percent. The paper says it plainly: "Providers that achieve a relatively low average WER can show a high error rate for an individual sample."
Sit with what that means operationally. You pick a vendor on its average. You upload two hundred videos. Some unknown subset of them come back at rates that make the captions actively misleading, and nothing in your pipeline flags which ones, because the average was fine. An average is a promise about a population. Your viewer is watching one file.
W3C's own position on this is not ambiguous: "Automatically-generated captions do not meet user needs or accessibility requirements, unless they are confirmed to be fully accurate." Its own worked examples are the kind of error that reads as perfectly fluent English: "4 to 5 minutes" becoming "45 minutes", and "should not preheat" becoming "should know to preheat". Neither looks like an error on the screen. Both invert the instruction.
Self-studyWhy the fluent errors are the dangerous ones3 min▶
A garbled caption is annoying and harmless, because the reader can see it is broken and discounts it. A grammatical caption that says the opposite of what was said is neither.
"Should not preheat" to "should know to preheat" costs one small word and reverses a safety instruction. There is no visual signal that anything went wrong. The reader who depends on captions has no second channel to check it against, which is precisely the point: the person least able to catch the error is the person the caption exists for.
This is also why "spot check a few" is a weaker control than it feels. Spot checking catches the obviously broken file. The fluent inversion survives a spot check, because it reads fine.
Before you compare two WER figures, ask what was done to them
6 min liveThe same study that produced the benchmark also produced four findings that change how you read every accuracy claim you will ever be shown. Three of them are about measurement. The fourth is about invention.
Normalisation roughly halves the reported number. The same audio scored 21.83 percent raw and 10.16 percent fully normalised. Normalisation strips things like punctuation and capitalisation before comparing, so a transcript that got every word right but every sentence boundary wrong scores beautifully.
Two consequences. First, vendor and research WER figures are not comparable unless the normalisation is stated, and vendors almost never state it. Second, the FCC requires correct punctuation and capitalisation, which is exactly what normalisation throws away before scoring. So a normalised WER can look excellent on a file that fails the regulatory requirement.
Streaming is significantly worse than batch. 10.9 percent against 9.37 percent on English in the same study. If your captions are generated live, you are on the worse curve by construction, and no amount of vendor selection moves you onto the better one.
Custom vocabularies did not significantly improve WER. 8.61 percent to 8.10 percent, not statistically significant. This is the feature every vendor sells you when you say "but our domain has specialist terms".
And yet supply the vocabulary anyway. The same study found that specialist terms and names only appeared at all when a vocabulary was supplied. So it does not move the aggregate score, and it is still the difference between your drug names and your speaker names being present or absent. Two true things that point in opposite directions, which is why you need both of them.
ASR does not only mis-hear, it invents. Koenecke and colleagues, "Careless Whisper" at ACM FAccT 2024, found that roughly 1 percent of transcriptions contained entire hallucinated phrases with no counterpart in the audio, and about 38 percent of those carried explicit harms: fabricated violence, invented personal attributes, made-up authority.
Run the last one against a real artefact. A forty-minute recorded lecture is not a small number of transcription units, and 1 percent of it is several confidently fabricated sentences that a viewer reads as authoritative because they are sitting in the same caption track as the real ones. There is no visual difference between a sentence the speaker said and a sentence the model produced.
Self-studyWhat to ask a caption vendor, in the order that gets answers4 min▶
Four questions. The first two are the ones that make sales calls go quiet.
- "Is that WER normalised, and how?" If they cannot answer, the number is not comparable to anything and you should treat it as marketing copy rather than a measurement.
- "What is your per-file distribution, not your average?" The published benchmark range is 0 to 53.8 percent. If a vendor will only give you a mean, you are being told about the population and sold a promise about the file.
- "Is this figure from streaming or batch?" Those are different systems with different error rates. A batch number quoted at you for a live use case is not wrong, it is just not about you.
- "What did you measure it on?" Clean studio audio with one native speaker is the easiest case in existence, and it is not the case you have.
One vendor example worth knowing as a hygiene lesson rather than a data point. ElevenLabs' own blog describes Scribe as having "the lowest automated transcription word error rate in Italian (98.7%), English (96.7%)". A 98.7 percent word error rate would mean almost every word is wrong. Those are accuracy figures mislabelled as WER in the vendor's own copy. Use it to calibrate how carefully these numbers are handled, and never as a WER number.
The gap is roughly 2x and it holds across all five vendors
6 min liveKoenecke and colleagues, in PNAS in 2020, tested five commercial ASR systems against 73 Black speakers and 42 white speakers, 19.8 hours of audio, matched on age and gender. Read the chart before the commentary, because the shape of it is the argument.
The obvious objection to a result like this is that the two groups were saying different things, so the systems were being asked harder questions on one side. The study anticipated it and closed it. Using 206 identical short phrases spoken by both groups, matched on gender and age, error rates for Black speakers remained roughly double. Same words. Same length. Same measurement. Double the errors.
The authors attribute the cause to the acoustic model, which points at insufficient training data rather than at the language model. And it is not a 2020 artefact that newer architectures cleaned up: a later study finds that wav2vec 2.0, HuBERT, WavLM and XLS-R "perpetuate the bias in performance against AAVE". Architecture changed. The gap did not.
The same shape appears with accent. ESL speakers averaged 10.2 percent WER against 7.0 percent for native speakers, a relative degradation of roughly 46 percent, with a worst case of 28.1 percent on Google. Whisper was the exception, holding at 3.3 percent on both. And this is an English-centric result before you leave English at all: German averaged 15.3 percent against English at 7.0.
Self-studyWhat this changes about how you deploy, in practical terms3 min▶
The finding is about systems, not about speakers, and the deployment consequence follows from that.
If you run auto-captions across a body of content with a diverse set of speakers, the caption quality your audience receives is not uniform, and it varies in a direction the measurements have already established. A single organisation-wide accuracy figure averages across that variation and reports a number that is true of nobody in particular.
Two things follow. Review budget should not be spread evenly, because the error is not spread evenly. And any accuracy figure you report internally should say what it was measured on, because a figure derived from your clearest native-speaker recordings will overstate what everyone else is getting.
Note also that the vendor ordering here is not the vendor ordering in the benchmark chart above. Different studies, different years, different audio. That is a reason to test on your own material rather than to pick a winner off either chart.
The 99 percent rule is industry convention, not law
5 min liveEveryone in this field quotes a 99 percent accuracy standard. It is worth knowing where it comes from, which is nowhere. It is not in WCAG. It is not in the FCC rules. It is not in DCMP's Captioning Key.
The National Deaf Center confirms the vacuum directly: "Currently, there is no standard threshold that would dictate when ASR captions are accurate enough to be used as a form of effective communication."
What the same source does say, and what is therefore defensible to quote: "When automatically generated captions average at 60%-70% accuracy, communication cannot be reasonably assumed to be as effective or equivalent to the audio", and poor auto-captions "can negatively impact students who are relying on captions."
So the honest position is narrower and more useful than the folklore. There is no numeric threshold in law. There is a documented level at which the claim of effective communication fails. Between those two facts sits a judgment you have to make and be able to defend, which is a different job from copying a number off a vendor page.
And there is a deeper problem with anchoring on a percentage at all. WER is a poor proxy for usability. Kafle and Huenerfauth, at ASSETS '17, gave 30 DHH participants ASR outputs of identical WER and found they systematically preferred the one favoured by a word-importance-weighted metric. Two caption files can score the same 95 percent and differ wildly in how usable they are, because WER treats "the" and the speaker's name as equally weighted errors.
Which means a caption pipeline tuned to maximise a WER score is optimising something the people using it do not experience. Losing a determiner and losing a proper noun cost the same in the metric and cost nothing like the same in the room.
Self-studyThree accuracy claims this course refuses to repeat3 min▶
Naming what has been left out is part of the method, so here are three that circulate constantly and are not on any page of this course.
- All 3Play Media accuracy figures, including the 99.6 percent measured claim and the 84.7 to 94.4 percent competitor range. These sit on marketing pages with no published methodology. They may well be right. There is no way to check, which is the same thing as unusable.
- "Comprehension drops sharply below 95 percent accuracy." No peer-reviewed study exists. The citation trail dead-ends at a marketing redirect. It is industry folklore repeated confidently enough to sound sourced.
- "32 characters per line, per DCMP." Covered in the next part, and it is simply not in the document.
One thing from 3Play is worth keeping, not as data but as a teaching device: "For a sentence of 8 words, a 95% accuracy rate means there will be an average of 2.5 errors in every sentence." That is arithmetic, and it does the useful work of making a percentage feel like a sentence.
Audit one real caption file against the numbers that exist
★ 18 min · everyone buildsTake one video you have already published with automatic captions. Not a sample, not a demo file, one of yours. The DCMP Captioning Key gives you concrete numbers to check it against, and the interesting part is which of them you fail.
The word-per-minute figures count every word, including speaker IDs and sound effects, which is the part people miss when they check. And background music under 5 seconds is not captioned at all. These are small, checkable, and almost never checked.
Export the caption file, do not read the burned-in version. You want the timings, and the burned-in version hides them. Any editor that produced the captions can give you an SRT or a VTT.
Find every caption longer than 6 seconds and every caption shorter than 40 frames. Both are DCMP violations, and automatic captioners produce both routinely because they segment on pauses rather than on reading time.
Count words per minute across the fastest thirty seconds. Include speaker IDs and sound-effect descriptions in the count. Compare against 130, 140 or 160 depending on who your audience is. A talking-head video recorded at a natural pace will frequently be over.
Now read the transcript against the audio, for one minute only. One minute. Count errors and classify each one: harmless, changes meaning, or fabricated. You are looking for the third category specifically, because roughly 1 percent of transcriptions contain phrases with no counterpart in the audio at all.
Check whether every proper noun survived. Names, products, drug names, place names. These are the errors that cost the reader the most and the metric the least, which is precisely the finding from the identical-WER study.
Finally, ask the question this session opened with. Does this video have an audio description, or does it have a transcript? If it has a transcript, you are conforming at Level A and you should stop describing it as AA.
Why one minute rather than the whole file. Because you will actually do one minute, and the error rate you find in it is the honest input to a decision about the other thirty-nine. A review plan that requires reading everything is a review plan nobody runs, and an unrun plan scores zero regardless of how rigorous it looked.
Four days before it hit, the date changed
5 min liveThe DOJ's ADA Title II web rule, 89 FR 31320 of 24 April 2024, adopts WCAG 2.1 Level AA. Everyone who prepared for it prepared for 24 April 2026. On 20 April 2026, four days out, DOJ published an Interim Final Rule extending both dates by a year.
| Entity size | Current deadline | Has it passed? |
|---|---|---|
| Public entities serving 50,000 or more persons | 26 April 2027 | No, roughly 8 months out |
| Under 50,000, and special district governments | 26 April 2028 | No |
Any material written before April 2026 that cites "April 24, 2026" is now wrong. That includes a large amount of training content, a large amount of consultancy deck, and probably something currently sitting in your own organisation's compliance folder. It is a live example of why compliance dates get re-verified rather than remembered.
Two caveats that must travel with this, every time you repeat it.
First, the extension delays only the WCAG technical conformance date. It does not suspend Title II's underlying effective-communication obligations, which have supported claims for years and are unaffected by any of this. "We have until 2027" is a statement about one rule, not about your exposure.
Second, the eCFR text of 28 CFR 35.200 still shows the original dates. If you look it up in the CFR you will read the superseded version and believe it. Cite ada.gov.
Self-studyWhy this course keeps making you re-check dates yourself2 min▶
Session 1 opened with a model retirement. This session closes with a regulatory date that moved with four days' notice. Session 6 will hand you an EU date that a widely reported "delay" did not actually change.
The pattern is the point. In this category, the thing most likely to make you wrong is not a hard concept, it is a fact you learned correctly two years ago. Nobody sends you a correction. The old number keeps circulating because it is in everyone's slides, and the corrected one is only on the primary source.
So the habit this course is really trying to build is small and slightly tedious: before you say a date out loud, open the page it came from. It takes ninety seconds and it is the difference between being current and being confidently out of date.
Before session 5
40 min- Find your organisation's accessibility statement and read what it claims. Look for the word "transcript" and the words "Level AA" in the same document. If both are there and audio description is not mentioned, you have found a Level A claim presented as AA compliance. This takes about five minutes and it is the single most valuable thing on this list.
- Run the one-minute audit from the build-along on a published video. Count errors, classify them into harmless, meaning-changing and fabricated. Write down the three numbers. That is your baseline and you did not have one before.
- Check the SC 1.2.5 wording yourself. Open W3C's Understanding page for Audio Description (Prerecorded) and find the sentence quoted at the top of this session. Read it in its own context. This is the fact you are most likely to be argued with about, so you want to have seen it.
- Verify the ADA dates on ada.gov, then look up 28 CFR 35.200 on the eCFR. Note that the two disagree. That disagreement is the exercise: knowing which source wins is worth more than knowing either date.
- Ask your caption vendor whether their published WER is normalised. Send the email. Whatever comes back, you will have learned something about the number, or about the vendor.
- Optional, and it will change how you read every vendor page: find three caption accuracy claims online and try to trace each one to a published methodology. Note how many attempts end at another marketing page.
Three questions
Clear all three and session 4 gets stamped in your passport.
1 · Your organisation publishes video with captions and a full transcript, and states that it conforms to WCAG Level AA. What is wrong with that?
W3C states it outright: at Level A authors "do have the choice"; at Level AA "authors must provide an audio description". Every US legal mandate points at Level AA, which is why this is the highest-value sentence in the course. Captions are a separate obligation under SC 1.2.2, and that one is Level A.
2 · A vendor shows you a 4 percent WER and a research paper shows you an 8 percent WER on the same task. What is the first thing to establish before comparing them?
Normalisation strips punctuation and capitalisation before scoring, so it can nearly halve the number. Vendor and research figures are not comparable unless the normalisation is stated, and the FCC requires exactly the punctuation and capitalisation that normalisation discards. Corpus size and recency matter, but neither can rescue two numbers measured on different rules.
3 · Where does the "99 percent caption accuracy standard" come from?
NDC is explicit: "there is no standard threshold that would dictate when ASR captions are accurate enough". What is defensible from the same source is that at 60-70 percent accuracy, communication cannot reasonably be assumed equivalent. And WER is a weak proxy anyway: given identical WER, DHH participants systematically preferred one output over another, because WER weights "the" and the speaker's name the same.
What this session teaches, and where it came from
Every fact on this page, and its verification tier, is recorded in the course's source map., including a quarantine list of what is deliberately not stated.
Session 4 cheat sheet · pin this
Tonight was about the obligations that already exist and the measurements that already exist to test them against. Session 5 turns the same instrument on the generated video itself: physics benchmarks where the best model scores 29.5 out of 100, identity and wardrobe drift as formally scored defects, the lip-sync metrics that make dubbing comparable, and one widely quoted iteration cost that nobody has actually audited.