Why specs are where AI is most dangerous
Sessions b2 to b4 did the work: an evidenced problem, verified research, a named job, a chosen branch of the opportunity tree. This session turns that into the artifact someone builds from - and it is the point in the process where AI helps most and hurts most, for the same reason. It writes a document that looks finished. A reviewer reads the shape, recognises the shape, and approves it. Nobody notices that four of the six decisions were never made, because the sections where those decisions should live are full of confident prose.
The six decisions a spec must contain 7 min live
Not sections - decisions. A spec can have all the right headings and none of these, which is exactly what rung 1 in the lab below demonstrates. Each of the six has a failure mode that is specific, common, and expensive, and each one gets decided by somebody whether or not you decide it.
LiveThe two that are almost always missing4 min▶
In the lab below, non-goals and the trade-off survive four of the five rungs. That is not an accident of the example - it is the pattern, and both have the same cause: they are the two checks that require somebody to give something up.
- Non-goals. A drafting tool has no reason to exclude anything, because exclusion is not a language problem. You have to hand it the scope decision, which means the scope decision has to exist first. If you cannot write the non-goals, you have not finished deciding - you have finished describing.
- The trade-off. The hardest line in any spec is "we chose X at the cost of Y, and we are accepting that". AI will not write it unsolicited, because the trade-off is not in the input unless you put it there. A spec where everything wins is a spec where nothing was chosen, and it will read beautifully right up until the review where two people discover they agreed to different things.
Self-studyWhat the six checks cannot tell you3 min read▶
The lab is a completeness check, not a quality check, and the difference matters:
- It cannot tell you the metric is the right metric. A spec targeting a vanity number with a baseline and a target scores full marks on check 3. The scorer sees that a metric exists; it cannot see whether moving it means anything.
- It cannot tell you the evidence is good evidence. Eight interviews from your friendliest customers pass check 1 as easily as eight representative ones. Session b3 is where that judgement lives.
- It cannot tell you the trade-off was the right trade-off. It only sees that one was stated.
- And it cannot tell you the problem is worth solving at all. A perfectly complete spec for the wrong problem scores 100.
Which is the whole reason this is a lab in the middle of a course rather than a tool you install. The six checks catch absence, reliably and cheaply, and absence is what an AI-drafted spec suffers from. Judgement is still yours, and it is the part nobody can score.
Run the ladder, then score your own ★ the scoring is real
Five rungs. Each one is the spec a drafting tool produces from that much input about Cadence's summaries feature and no more. Read the failing checks at each rung before you move up, and read the gained column when you run the whole ladder - that column is the actual lesson.
Start at rung 1 - one sentence in. Read the spec: it has every section a PRD should have, and 15 out of 100. Five of the six decisions were never made.
Add research (rung 2). The problem becomes checkable and the user gets named - two checks, from one input. Success is still a mood.
Add the metric definitions (rung 3), then constraints (rung 4). Watch success become measurable, then watch the edges get decided. Non-goals and the trade-off are still missing at 70 out of 100.
Add the non-goals (rung 5). 100 out of 100, signable - and note that one input fixed two checks, because writing down what you are not doing is what forces the trade-off into the open.
Then press "Score your own spec" and paste something real. This is the part that matters after today.
The rung-5 spec is not longer because it is padded. It is longer because it contains four decisions the rung-1 version left to somebody else: who this is not for, what happens when a meeting has no clear decisions, whether summaries survive the 30-day retention setting, and the fact that this ships without the live-notes feature sales asked for. Those four sentences are the spec. Everything else is context around them - and a drafting tool will produce the context all day without ever producing the four sentences, because they are not writing, they are deciding.
Your turn: three specs, three verdicts ★ 8 min
Use the "Score your own spec" box in the lab above for each of these. Predict the failing check before you press Score.
LiveQ1 · The one-liner everyone has written3 min▶
Paste this in and score it:
It scores 0 out of 100 - all six checks fail. Worth sitting with for a second, because most teams have shipped from something very close to this, and the reason it scores zero is not that it is short. It is that there is nothing in it anyone could disagree with on the evidence, no way to know afterwards whether it worked, and no boundary at all.
LiveQ2 · Your own worst spec4 min▶
Find a spec you wrote when you were busy - ideally one that shipped and went sideways. Score it. Then look only at the failing checks and ask which of them explains what actually went wrong.
In practice it is usually one of two: the trade-off was missing, so two stakeholders agreed to different things; or success was not measurable, so the argument afterwards was about whether it worked rather than what to do next. Both are visible in the spec before a line of code exists, which is the uncomfortable part.
Self-studyQ3 · Score a spec you did not write3 min▶
No paste needed if you would rather not - a thought exercise. Take the last spec somebody handed you and run the six checks by eye. Then, and this is the actual skill, work out how to say the failing one out loud without it landing as criticism.
"What are we not doing here?" and "how will we know this worked?" are the two most useful questions in product review, and neither reads as an attack. Both are just checks 4 and 3, asked politely. A team that gets asked those two questions every time starts writing them into the spec before the review, which is the entire goal.
Try it yourself - this week ◐ 25-35 min total
- Score your carried backlog item's spec, if it has one. If it does not have one, that is the finding - write the four sentences (user, metric with a baseline, non-goals, trade-off) and you have most of a spec.
- Add the six checks to your team's spec template as a checklist at the top, not the bottom. At the bottom it is a review step; at the top it is a prompt.
- Take one spec currently in review and ask only the two polite questions: what are we not doing, and how will we know it worked. Note how long the answer takes.
- Next time you draft with AI, give it the metric definitions before asking for the spec, and see whether check 3 passes first time. This is the cheapest habit change in the course.
- Write your own non-goals for the Cadence summaries feature before b6 - prioritisation is much easier when the scope already has an edge.
Sources covered
Full source map in materials/official-course-map.md. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · Rung 1 of the lab scores 15 out of 100 but has a problem section, a solution section, a success section and requirements. What is missing?
A spec is a set of decisions that happen to be written down. Sections are where decisions live, not the decisions themselves - and a reviewer who reads the shape and recognises the shape will approve a document containing none of them.
2 · Adding the non-goals at rung 5 fixed two checks at once. Which, and why?
The two checks that survive longest are the two that require somebody to give something up. Once the not-doing list exists, the trade-off is one sentence away; without it, no amount of drafting produces one.
3 · A spec scores 100 out of 100. What does that guarantee?
The scorer catches absence, cheaply and reliably, which is what an AI-drafted spec suffers from. Whether the decisions were good ones is judgement, it is yours, and nothing on this page can score it.