learn-ai-pm-with-phoebe / Leader session 1 of 6
Learn AI Product Management with Phoebe · Leader track · Session 1 of 6

What AI changes, and what it does not

Half of product management looks like writing. That half just collapsed in cost. The other half is choosing, cutting, and being the name on the outcome, and it has not moved by a single hour. Almost every disappointment leaders are currently having with AI in product work comes from expecting the second half to move because the first one did. This session draws the line precisely, then changes what you ask your PMs for.

🟢 Leader track Heads of product · founders · stakeholders No code to write 45 min ▶ Leaders start here
0-3 · Welcome 3-15 · The honest split 15-27 · The fluent artifact, and the new bottleneck 27-40 · Group exercise: count the decisions 40-45 · Q&A
Part 0

How this track works

Six sessions, no code, no tool tour, nothing to install. You approve roadmaps, you fund teams, and you referee the meeting where two competent people disagree about what to build. This track is about doing that well now that the documents arriving on your desk cost almost nothing to produce: what AI changes in product work and what it cannot touch (this session), what evidence has to mean when a fluent research summary is free (a2), what a spec must contain before you sign it (a3), prioritisation without theatre (a4), metrics and the readout that admits the bet failed (a5), and what to fund in an AI-assisted product org without hollowing out the craft (a6).

The examples all use Cadence, the AI note-taker running through this whole series: self-serve plus team and enterprise, focused on team admins with paid workspaces of 5 to 50 seats. Three people have asked for three different things and nobody has decided. Your PMs work that decision in detail on the practitioner track; you need to be able to referee it, which is a different skill and a shorter course.

This course ends at the decision. Once the call is made, charters, milestones, critical path and status reporting are a different discipline and a different course: learn-ai-project-management. If your instinct in a product review is to ask when it ships, that instinct is correct and it belongs there.

Live - presented in session Self-study - read after class ▶ Discussion exercise - run it in the room Sources covered
★ What you walk out with today A written line between the half of your PMs' job that got cheap and the half that did not, one thing you will stop asking for, one thing you will start asking for, and the ability to spot a fluent document containing no decisions in under two minutes.
Part 1 · covers AI-assisted knowledge work, the PM craft literature on decision ownership

The honest split 10 min live

Product management divides into production and commitment. Production is drafting, summarising, restructuring, adapting one message for three audiences, and arguing the other side of your own case. Commitment is choosing, cutting, setting the number you will be judged on, saying no, and owning it publicly when it turns out you were wrong. AI is genuinely excellent at the first and completely absent from the second, and the gap between those two facts is where every current disappointment lives.

Two halves of the same job. AI moved exactly one of them. Production - the cost collapses drafting the PRD, the update, the one-pager summarising 30 interviews into themes restructuring a messy doc into an argument one message rewritten for three audiences arguing the other side of your own case Was costing: hours of typing per week Commitment - the cost has not moved choosing which problem gets the quarter cutting the other two asks, out loud setting the number you are judged on saying no to somebody more senior owning it when the call was wrong Still costs: the same as it always did The half that got cheaper was never your bottleneck. You were not short of documents. You were short of decisions, and the price of a decision did not change at all. Cheap production beside unchanged commitment does not improve the ratio; it worsens it.
🔍 Click to zoom - one half collapsed, one half did not, and the ratio between them got worse

Which changes what you should be asking for. The instinct when production gets cheap is to ask for more of it: more documents, more options, faster turnaround. That instinct is exactly backwards, because volume was never the constraint and it is now free enough to become a nuisance.

The workCost to produce nowStop asking forStart asking for
First-draft documents: PRD, update, one-pagernear zeromore of them, fasterone of them, with the decision on page one
Research summaries and theme listsnear zerothe themesthe themes with sources, and one verbatim quote
Options and alternativesnear zeroa longer options listthe two you rejected, and the reason
Choosing which problem gets the quarterunchangedconsensus in the rooma named choice with a name attached to it
Setting what success meansunchanged"improve engagement"a baseline, a target, and a date
Saying no to a senior askunchangeda document that explains it awaythe sentence, said out loud, with a reason
LiveWhat actually got faster, stated narrowly4 min

Vague claims about productivity are how teams end up disappointed, so be precise. Three categories, and only one of them is a real gain:

  • Genuinely faster. The blank page. First drafts, restructuring, summarising, adapting one message for three audiences, and producing the strongest argument against your own proposal. That last one is underrated and the cheapest quality upgrade available to a PM: it used to require a colleague with time and courage, and now it requires four minutes.
  • Not faster, and never was. Getting five stakeholders to agree. Knowing which of two genuinely good options is right for this company this quarter. The conversation where somebody is told no. None of these are writing problems, which is why none of them moved.
  • Actively slower if you are careless. Anything where verifying the output costs more than writing it would have. Research synthesis is the classic case, and it is a2's whole subject: one invented theme costs more to find than the synthesis saved.
The honest one-line summary AI removed most of the typing and none of the deciding. If your PMs' weeks were mostly typing, their weeks changed. If their weeks were mostly deciding, their documents got better and their calendars are identical - and if you were hoping for the second thing, that hope was the error, not their performance.
Self-studyWhy the commitment half cannot move3 min read

It is worth understanding why this is structural rather than a limitation that gets fixed next year. Commitment work has three properties that are not language problems:

  • It requires a stake. A decision means somebody bears the consequence. A tool bears none, so it has no basis for preferring one option over another and no reason to refuse a bad idea. Ask it to choose and it will produce something that reads like a choice, which is worse than a refusal.
  • It requires knowing what your company cannot afford. Not the public facts about your market, the private ones: which team is already at capacity, which customer is one bad quarter from leaving, which promise you made in a board meeting. That context lives in people and mostly is not written down anywhere.
  • It requires being wrong in public later. Accountability is the actual product of the commitment half. A decision nobody can be blamed for is not a decision, it is a description of a preference - and a document is very good at producing exactly that.

Which is the reassuring part of this session, if you want one. The scarce, valuable, hard-to-hire half of the PM job is the half AI does not touch. The threat to your PMs is not that AI does their job; it is that the cheap half of their job becomes the visible half, and they get measured on it.

Part 2 · the failure mode you have to learn to see

A fluent artifact with no decisions in it 6 min live

Here is the new thing a leader has to be able to recognise, and it is genuinely new. Six well-structured pages arrive. Problem, solution, success criteria, requirements, all present, all confidently written. It reads like progress. Nothing in it can be checked, nothing is excluded, nothing is measurable, and no alternative was rejected. It is not a bad document. It is a document with the shape of a decision and none of the substance, and the reason you approve it is that you have spent twenty years using fluency as a proxy for thought.

The same six pages, read two ways What the review sees a Problem section, four paragraphs a Solution section, well structured a Success section, confident prose Requirements, numbered and thorough six pages, delivered on Thursday Verdict in the room: looks thorough What is actually inside it no evidence anybody could check no named segment, so no boundary no baseline, so no way to be wrong nothing excluded from scope no trade-off, so nothing was chosen Decisions contained: zero Fluency arrives free now. Progress does not. A well-written six-page document used to be weak evidence that somebody had thought hard about the problem. It is no longer evidence of anything at all. Count the decisions in it, never the pages.
🔍 Click to zoom - the shape of a decision, with the decision removed
LiveThree tells, spottable without reading the document4 min

You are busy and the document is six pages. You do not need to read it to know whether it contains anything. Three tells, in order of reliability:

  • There is no not-doing list. Scan for a section that names what is out of scope. If it is absent, no boundary was set, and every adjacent request will arrive later as a small ticket. A document without exclusions has not finished deciding; it has finished describing.
  • Every stakeholder wins. Read the summary paragraph. If sales, support and engineering all get what they wanted, nothing was chosen and you are looking at a document three people will approve while agreeing to three different things. That specific failure is a3's subject.
  • Success is a direction, not a number. "Improve engagement", "increase satisfaction", "drive adoption". A direction cannot be wrong in twelve weeks, which sounds safe and is the opposite of safe: it guarantees the argument afterwards is about whether it worked rather than what to do next.

Three tells, roughly ninety seconds, no reading. If all three fire, the honest response is not a comment in the margin. It is handing it back with one sentence: this is a description, come back with the decision.

Self-studyWhy this fools good reviewers, including you3 min read

This is not a story about careless leaders. Three things conspire:

  • Fluency used to be costly. A well-argued document required someone to have argued it with themselves first. The correlation between polish and thought was real for your entire career, and correlations you have relied on for twenty years are not switched off by knowing they broke.
  • Recognition is faster than reading. A reviewer under time pressure checks that the expected sections are present and that the prose does not stumble. Both pass. Neither has anything to do with whether a decision was made, and both feel like reviewing.
  • Confident prose suppresses questions. Hedged writing invites challenge. A document that states everything evenly and without qualification reads as settled, and people do not interrogate settled things in a meeting with eleven items on the agenda.

The fix is not more scepticism, which does not scale and makes you tiring to work with. It is a fixed, boring, repeated set of questions asked identically every time, so nobody has to decide whether today is a suspicious day. The last section of this page is that set.

Part 3 · the org consequence

If documents are cheap, judgement is scarce 6 min live

Follow the split to its organisational conclusion. Writing capacity went up several times over. Reviewing capacity is your calendar and your attention, and neither changed. Throughput in any pipeline is set by its narrowest step, and the narrow step just moved from the person writing to the person deciding whether what was written is any good. That person is usually you.

Writing capacity went up. Review capacity did not. Production capacity up several times over no longer the constraint Review capacity unchanged: your calendar and your attention Decision throughput set by the narrowest step which is now the middle one What the new bottleneck looks like on a Monday morning eleven documents in your review queue, four of which contain a decision a spec approved because there was no time to read it properly, and it looked fine the same argument reopened for the third time, because nobody wrote down what was given up Review capacity is now something you fund, not admin you absorb. Fewer artifacts per PM, each carrying a decision, costs you less than more artifacts carrying none. The funding version of this argument is a6. This session is about what you ask for on Monday.
🔍 Click to zoom - the narrow step moved from writing to deciding
LiveThe two levers you have, and only two4 min

If review is the constraint, there are exactly two honest responses, and most organisations reach for a third that does not work.

  • Reduce what needs reviewing. Fewer artifacts, each carrying a decision. This is almost entirely in your control and costs nothing: it is a change to what you ask for. One document per decision, not one document per week.
  • Make each review cheaper. A fixed set of questions asked in the same order every time, and a shared standard for what a document must contain before it is worth your attention. a3 sets that standard; the questions at the bottom of this page are the cheap version you can use on Monday.
  • The one that does not work: reviewing faster. Skimming a fluent document does not detect a missing decision, because the missing decision is invisible at skim speed. That is the entire mechanism of the failure mode in Part 2. Speed is what causes it.
The uncomfortable implication If your PMs are producing more documents than you can read properly, some of them are being approved on their formatting. That is a fact about your review process rather than about your PMs, and it is your problem to fix. Nobody will raise it with you, because raising it means volunteering that their work is not being read.
Self-studyWhat this means for hiring and levelling3 min read

a6 handles the org and funding question properly. Two things are worth knowing now, because they affect decisions you might make this quarter:

  • Writing samples have stopped discriminating. A take-home document tells you almost nothing about a candidate now. What still discriminates is asking someone to defend a decision they actually made: what they chose, what they gave up, what they would have to have believed for the other option to be right, and what happened. That conversation cannot be produced by a tool because it is about a specific past.
  • The junior rung got stranger, not simpler. The tasks juniors used to learn on were production tasks, and those are now nearly free. That does not remove the need for juniors; it removes the ladder they used to climb. If you want PMs who can make decisions in three years, they have to be making small ones now, with real consequences, in public.

Neither of these is a reason to slow hiring, and both are reasons to change what you interview for. Do not settle either question from this page; a6 is where the argument is made with numbers.

Group exercise · 13 min · everyone in the room

Count the decisions ★ run this in your next product review

No slides, no framework, no artifact to produce. Everybody in the room opens a document they were actually sent in the last two weeks and counts two things. The exercise works because the ratio is always worse than people expect, and because it is impossible to argue with your own count.

Everyone opens the most recent product document they were sent. A spec, a brief, an update, a research summary. Not a new one, not a good one, the most recent one. No preparation and no selecting.

Count the pages. Say the number. This takes four seconds and it is the number everybody currently uses as a proxy for effort.

Now count the decisions. A decision is a sentence where something was chosen and something else was given up. "We will build summaries" is not a decision. "We will build summaries and not live notes this quarter" is one. Mark each one you find.

Say both numbers out loud, around the table. Pages, then decisions. Do not explain or defend. The room will get quiet on its own, and the silence is the lesson rather than anything you say next.

For every document that scored zero, write the one question you will ask its author. One sentence, and you have to actually ask it this week. If you cannot think of a question, the document had no content to question, which is the finding.

LiveThe prompt, word for worduse this exact wording
Say this "Open the most recent product document you were sent. Count the pages. Now count the decisions, where a decision is a sentence in which we chose something and gave something else up. Say both numbers."

What a good answer sounds like. "Four pages, two decisions: we picked the support problem over the sales one, and we cut the enterprise variant to hit the date." Specific, and both decisions have a loser. Somebody with a good document will name the loser without being asked, because they had to argue with that person to write the sentence.

The failure mode to listen for. Padding the count. "Well, deciding to write the spec at all was a decision" and "we decided the scope was right" are not decisions, they are activities with the word decision attached. Also listen for the plural with no loser: "we decided to do summaries, live notes and the agent" is a list, and a list is what you get when nobody wanted to lose an argument. When you hear either, ask what was given up. If the answer is nothing, the count is zero.

Do not let it become a performance review. The point is that this is systemic and mostly your own fault for asking for volume. Say that out loud in the first minute, or people will spend the exercise defending documents instead of counting them.

LiveSecond round: one stop, one start4 min
Say this "Name one thing you are going to stop asking your team for, and one thing you are going to start asking for instead. Both have to be things you asked for in the last month."

What a good answer sounds like. "I am going to stop asking for a weekly written update, and start asking for one line on what changed in the decision. And I will stop asking for options and start asking which one you would pick and what you would be giving up." Concrete, recent, and it costs the person saying it something, because they liked the thing they are giving up.

The failure mode to listen for. The stop with no cost: "I will stop asking for documents nobody reads" is not a commitment, because nobody asked for those. Push for a specific artifact with a specific name and a specific author. If the whole room names the same one, you have found a genuine ritual to kill and you should kill it before the meeting ends.

Real world

Cadence, three asks, zero decisions. Sales wants live in-meeting notes, and offers "every demo asks for it" plus three lost deals. Support wants post-meeting summaries, and offers 214 tickets in three months tagged "cannot find what was decided", the second most common tag. The CEO wants an agent, and offers nothing. A drafting tool handed those three asks will produce an excellent document about all three: a roadmap where live notes come first, summaries come second, agents are explored in a discovery spike, and every stakeholder reads their own name and relaxes. It is well structured, it is honest about nothing, and it contains zero decisions because none of the three asks lost. The real decision - summaries over live notes, at the cost of the demo moment sales wanted, because the pain is located after the meeting and live notes need a streaming pipeline that does not exist - is one paragraph, and it took a person with a stake to write it.

Take this with you

Questions to ask your PMs the whole point of the session

Six questions. Ask them in the same words every time, so that nobody has to work out whether today is a day you are paying attention. Within about a month your PMs will start answering them in the document before the review, which is the entire objective.

This week

Five things to do before a2 ◐ conversations, not documents

Source material

Sources covered

Full source map in materials/official-course-map.md. This page covers:

AI-assisted knowledge work: where drafting help is real, verification cost, automation biasParts 1 to 3 · the production and commitment split, and the review bottleneck
The PM craft literature on the role: decision ownership, saying no, accountabilityPart 1 · why the commitment half is structurally immovable
Research quality, representativeness, and provenanceNamed as the second tell only - the full treatment is a2
PRD and spec practice: what a signable spec must containThree tells in Part 2 · the six checks and who signs are a3
Org design, funding, and levelling in an AI-assisted product teamPart 3 self-study raises it and stops - a6 makes the argument
Delivery: charters, milestones, critical path, status reportingOut of scope by design - learn-ai-project-management
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · Your PMs are producing three times as many documents as last year and you are no happier with the decisions. What is the most likely explanation?

Volume was never the constraint. The half that collapsed in cost was drafting; choosing, cutting and owning cost exactly what they always did. More output from the cheap half does not produce more decisions, and asking for it faster makes the ratio worse.

2 · A six-page spec arrives: every section present, confident prose, sales and support and engineering all satisfied by the summary. What have you most likely been handed?

Every real decision has a loser. When all three asks survive in the summary, the trade-off was never made, and the document will be approved by three people who each read the version they wanted. The failure surfaces in the retro, or in a3, whichever comes first.

3 · Review is now the narrowest step in your product pipeline. Which response does not work?

A missing decision is invisible at skim speed - recognising that the expected sections are present is exactly what a fluent empty document passes. Speed is the mechanism of the failure, not a defence against it. The two that work are reducing volume and making each review cheap and repeatable.

Leader session 1 cheat sheet · pin this

The splitProduction (drafting, summarising, adapting, arguing the other side) collapsed in cost. Commitment (choosing, cutting, the number, saying no, owning it) did not move.
The error to stop makingExpecting the commitment half to speed up because the production half did. That hope is the disappointment, not your PMs.
Stop asking forMore documents, faster. Longer options lists. Consensus in the room.
Start asking forFewer, better decisions, with the trade-off written down and a loser named.
The new failure modeA fluent artifact containing no decisions. It reads as progress and is not.
Three tells, no readingNo not-doing list · every stakeholder wins · success is a direction, not a number.
The bottleneck movedWriting capacity rose, review capacity did not. Judgement is the scarce thing now, and review is what you fund.
The two questions"What are we not doing?" and "how will we know it worked?" Next: a2, what evidence has to mean now.