learn-product-thinking-with-phoebe / Session 2 of 6
Learn Product Thinking with Phoebe · Session 2 of 6

What people ask for is not what they need

Last time you turned one request into four lines. Tonight you have a pile of them. A request is a strange object: it is reliable evidence that somebody has a problem, and it is a terrible instruction for what to build. Learn to hold both of those at once and the pile stops being a to-do list you are behind on and starts being the cheapest research anybody will ever hand you. Along the way you will take apart the number everybody in this field quotes, by reading the slide it came from.

🟢 Everyone Bring the requests you collected No code, nothing to design 45 min
0-3 · Welcome 3-18 · Requests as evidence 18-40 · Build-along: read your pile 40-45 · Q&A
Part 0

Where we start

Somebody forwards you a message: "can we get a print button on the report screen?" You have two obvious moves and both of them are wrong. You can add the print button, which takes a week and is the single most popular way products turn into landfill. Or you can explain why the print button is not a priority, which takes four minutes and costs you the next twelve things that person would have told you. There is a third move, it is not a compromise, and it is what tonight is about.

🎫 Clear all three quiz questions to stamp your passport Live = presented in the 45 minutes Self-study = the full depth, after class

★ What you walk out with tonight. Your own pile of requests, regrouped by the problem each one implies rather than by the feature it names, with the people who asked marked, and at least one request left over that serves no problem you can name.

Part 1 · covers requests as evidence

A request is evidence, not a specification

6 min live

Somebody went to the trouble of asking. That is a real signal and it is worth a lot. What it is not is a description of what should exist.

WHAT A REQUEST RELIABLY TELLS YOU treat all three as true until shown otherwise WHAT IT DOES NOT TELL YOU and never has, no matter how confidently it is phrased A problem exists Somebody hit friction on a real working day Roughly where it lives Which screen, which week, which part of the job It hurts enough to say out loud Most friction is absorbed in silence. This one was not What to build The named feature is the nearest thing they have seen How common the problem is One loud voice and forty quiet ones look identical here Whether the asker holds the job Plenty of requests are relayed on somebody else's behalf Take the left column seriously. Never let the right column be filled in by assumption.
🔍 Click to zoom - the two halves of a request, and only one of them is information
LiveTaking a request seriously without building it5 min▶

There is a move that keeps the evidence and drops the instruction, and it fits in one short conversation. It has three parts and they go in this order.

Restate the problem back. Not the feature, the problem. "So when the Monday numbers come out, you need to get them in front of four people who cannot log in, and right now that means screenshots." If you have it wrong, they will correct you immediately and enthusiastically, which is the point. If you have it right, you have just done something almost nobody does for them: you listened past the sentence.

Ask what they would do if the feature did not exist. This is the gentlest possible way to find the workaround, because it does not sound like a refusal. Sometimes the answer is "nothing, I would just keep screenshotting", and now you know the cost. Sometimes it is "oh, I would just ask Sam", and you have learned the problem already has a solution, which is a very different conversation.

Ask what happened the last time. One incident, with a date attached. "Last Monday I sent four screenshots and one of them was the wrong week, and the board meeting used it." That single sentence outranks a page of general description, and it hands you what better would have to mean.

Notice what has not happened. You have not agreed to build the print button, and you have also not said no to it. You have taken the request more seriously than the person expected, which is why this rarely feels like a brush-off.

Self-studyThe two failure modes, and why one is much worse4 min▶

Build everything requested. This is the popular failure because every individual decision looks reasonable and helpful. The effect is cumulative rather than immediate: you become an order-taker, which means the queue now decides what the product is, and whoever asks most often wins. The product becomes incoherent in a specific way, where each part makes sense to the person who asked for it and no part makes sense to anybody arriving fresh. Nobody can point at the meeting where that decision was taken, because it never was one.

Dismiss requests. This looks like the disciplined alternative and it is far worse, for a reason that is easy to miss. The requests were your cheapest source of evidence about where the product hurts. They arrive unprompted, they cost nothing, and they come from people doing the actual work. Dismiss enough of them and they stop arriving. The problems do not stop, you just stop hearing about them, and the silence is indistinguishable from things going well right up until somebody leaves.

The asymmetry matters. Over-building wastes effort you can see. Dismissing destroys an information channel you cannot see, and it does not grow back quickly once people have decided you are not worth telling.

The third move avoids both because it separates two things that got glued together: whether to take the request seriously, and whether to build the named thing. Those were never the same question.

Part 2 · covers the most-quoted number in product

45% of features are never used. Where that came from.

7 min live

You have heard this number. It is on slides, in books, in the opening minute of conference talks. The figure is real and it was read here from the original. What is behind it is not what almost anybody quoting it believes.

"FEATURES & FUNCTION USAGE", AS PRINTED ON THE SLIDE a 2002 conference keynote deck, surviving only in a web archive capture 45% 19% 16% 13% 7% Never Rarely Sometimes Often Always 45 + 19 = 64%. The other number everybody quotes is these two segments of this one slide, added together. It is not a second study, and it is not corroboration. The slide carries no sample size, no year, and no methodology at all. Two later accounts of what was measured exist, and they contradict each other.
🔍 Click to zoom - every figure here is as printed; everything missing is missing from the source too
LiveWhat the slide says, and what it does not4 min▶

The primary source was tracked down and read for this session. It is a keynote slide deck from a software conference in 2002, titled ROI - It's Your Job!, by Jim Johnson, copyright The Standish Group International. It is not hosted anywhere current. It survives in a Wayback Machine capture of the conference host, and that is the only reason any of this can be checked at all.

One slide is titled "Features & Function Usage". It is a pie chart with five slices: never 45%, rarely 19%, sometimes 16%, often 13%, always 7%. Those five numbers are what the slide says. The famous 45% is the first slice. The other famous number, "64% of features are rarely or never used", is 45 plus 19, from the same pie, on the same slide. Two figures, one source, and they have been travelling separately for twenty years as if they were independent confirmations of each other.

Here is the finding. The slide carries no sample size, no year, and no methodology. Not a footnote, not a caption, nothing. The deck does have a general methodology slide elsewhere, mentioning tens of thousands of projects since 1994, but that describes a separate body of success-and-failure research, not this pie. The pie itself follows another slide that is equally uncited.

Two later accounts of what was actually measured exist, and they disagree. One says Standish stated in 2015 that it came from four internal applications. Another has a 2010 Standish report citing an unpublished 1996 study of a hundred custom applications. Four applications or a hundred, 2002 or 1996: neither account could be corroborated against a primary document. The honest position is to state that the two accounts disagree and not to pick one.

So never call it a study. It is one unsourced pie chart in a conference deck. The figures are real in the sense that they are really on the slide. Nothing further is known.

Why this matters more than it looks. You will meet this number in a meeting, quoted at you as settled fact, usually to win an argument you were already going to win on the merits. Knowing what it is lets you do something better than contradicting the person: you can say where it came from, agree that the direction is probably right, and move the conversation onto the mechanism, which is where the actual decision lives.

There is a much better number. It measures something different, it does not say what people think the first one says, and it is worth knowing precisely because it does not tidily agree.

TWO NUMBERS THAT ARE NOT THE SAME NUMBER never used rare used more than that The 2002 slide unsourced pie, no sample, no year, no method 45% 19% 36% Pendo 2019 615 subscriptions, customers of over a year, three months of aggregated usage 24% 56% 20% Rarely or never: 64% on the slide, 80% in the telemetry. Different buckets, decades apart. Neither confirms the other. Pendo is real telemetry from one vendor's own B2B SaaS customers, not a representative sample of software. No academic replication of either figure exists.
🔍 Click to zoom - two figures that get quoted together and measure different things
Self-studyThe honest counterweight, and its limits4 min▶

In February 2019 Pendo published a Feature Adoption Report. Unlike the slide, it says what it measured: 615 subscriptions, all customers of more than a year, with usage aggregated over a three-month period, across banking, HR technology, education, logistics, healthcare and e-commerce. That is telemetry. Software reporting what people actually clicked, not a survey and not a recollection.

Its split is frequent 12%, moderate 8%, rare 56%, never used 24%. It also reports that 12% of features generate 80% of average daily usage volume, which is the part most worth carrying around.

Resist the urge to say this confirms the 45%. It does not, in both directions at once. Pendo's never bucket is 24%, which is a lot lower than 45. Its rarely-or-never total is 80%, which is considerably worse than 64. Different definitions of the buckets, different populations, seventeen years apart. Two numbers that disagree in opposite directions are not corroboration, they are two numbers.

And state the limit whenever you quote it. Pendo measured Pendo's own customer base: B2B software companies that bought a product analytics tool. Real telemetry, honestly described, and not a representative sample of software in general. No peer-reviewed replication of any of this exists, and no comparable published figure from another analytics vendor could be found. The absence is part of what you know.

The lesson underneath the numbers. The tidy famous figure travels and the careful one does not. One unsourced pie chart from a 2002 keynote has outrun a properly described telemetry study by a wide margin, and it will still be quoted at you next year. Here is the part that actually helps: you do not need any of these figures to justify cutting. The argument for cutting stands on mechanism, which is what session 3 measures in front of you. What you need the numbers for is narrower and more practical: knowing which ones you are allowed to say out loud, and what you have to say alongside them.

Part 3 · covers reading a pile

Group by the problem, never by the feature

5 min live

A pile of requests sorted by feature name tells you what words people use. Sorted by the problem underneath, it tells you where the product hurts. Those are not the same list and they are frequently in a different order.

What arrived, word for wordWho said itThe problem it implies
"Can we get a weekly CSV export?"The finance analystThe Monday numbers have to leave the system to be checked and shared, so she rebuilds them by hand every week
"Can you add a print button?"Her managerThe same problem. Different feature name, different person, one underlying thing: the Monday numbers cannot be shared with people who do not log in
"We need notifications"The support leadHe finds out about an angry customer three days late, usually from the customer
"We need notifications"The overnight ops schedulerA different problem entirely. She cannot tell whether last night's job finished before the morning shift arrives, and the wording is identical
"Can we have a search box?"A new starter, week twoShe cannot find records she knows exist, because nobody told her the account names were entered four different ways
LiveVolume is not priority4 min▶

Once the pile is grouped by problem, the tempting next move is to count. Resist it for one minute, because a raw count is almost always a measure of who is comfortable asking rather than where the pain is.

Three things to record next to each group, in this order.

Who asked. By name, not by department. A group of four requests that all came from one enthusiastic person is a different object from a group of four requests from four people, and a plain tally cannot tell them apart.

How many distinct people. This is the count that means something. Two separate people arriving independently at the same problem is a stronger signal than eight requests from one.

Whether they hold the job. The most important and most skipped. Is the person asking the one who actually does the work, or are they relaying, guessing, or advocating on behalf of somebody they last spoke to in March? A request from the person with the problem and a request about somebody else's problem look identical in a ticket queue and are worth wildly different amounts.

The quiet outcome of doing this properly: some groups shrink to nothing, and one group you had not noticed turns out to have three different people circling it with three different feature names.

Self-studyThe request that serves no problem3 min▶

Every pile has at least one. A request where, after honest effort, you cannot write down what problem it solves for whom. Not a hard one, not a vague one: one where the problem line comes out empty.

There are three usual explanations and they need different handling. Sometimes you simply have not asked yet, and the problem is real and obvious the moment you do. Sometimes it was copied from another product, because somebody saw it there and it looked professional. And sometimes it exists because it was on a list, and the list came from a meeting, and nobody in the meeting had the problem either.

The first is fixed with a ten minute conversation. The other two are the interesting ones, because they are how a backlog silently fills with work that will be built, shipped, maintained for years, and never used. Finding one of these in your own pile tonight is a genuinely useful outcome, and it is why the last stage of the build-along asks for exactly that.

Build-along

Read your own pile

★ 22 min · use what you collected

The requests you gathered over the last month. Five stages. You should end with fewer groups than you had requests, and one request sitting on its own with nothing written next to it.

List every request exactly as it arrived, in the words that were used, including the ones you already decided against. Do not tidy the phrasing and do not merge anything yet. Ten to twenty is a good pile. Five is enough.

Next to each, write the problem it implies in one line. Not the feature restated. If you genuinely cannot write one, leave it blank rather than inventing it, because the blanks are the point of stage five.

Now group by the problem line, ignoring the feature names entirely. Expect two surprises: requests that sound nothing alike collapsing into one group, and two identically worded requests landing in different groups.

Mark each request with who asked and whether they hold the job they are describing. Then count distinct people per group, not requests. Compare that order with the order you would have guessed.

Find the request whose problem line is still empty. Decide which of the three it is: not yet asked, copied from another product, or inherited from a list. Write your decision down next to it.

If your pile is small. Five requests is plenty for this. The exercise gets its value from the regrouping, not the volume, and a small pile makes stage four honest because you can actually name every person. If you have no pile at all, use the last five things anybody asked you for in any context, including the ones that were not about software.

Homework

Before session 3

25 min
  1. Take one grouped problem back to the people who asked. Restate it to them the way you wrote it, and watch what they correct. The correction is the finding, and it usually arrives in the first ten seconds.
  2. Ask the "what would you do if it did not exist" question about one request you were quietly planning not to build. Write down the answer word for word. Sometimes it changes your mind, which is the useful case.
  3. Track down where one number you repeat came from. Any number, from any part of your work. Find the primary source, not the article citing it. Note what the source does not say, because that is the part that travels badly.
  4. Bring your grouped list to session 3. You will be cutting a backlog next time, and it goes much better when you already know which of your own items serve a problem you can name.
Check yourself

Three questions

Clear all three and session 2 gets stamped in your passport.

1 · Two people ask for "notifications" in the same week. What does that tell you?

Grouping by the feature name would put these two together and produce one build that serves neither properly. Grouping by the problem underneath splits them, which is the whole reason the second grouping is worth doing.

2 · Somebody quotes "45% of features are never used" at you. What is the accurate thing to say?

A overstates: Pendo measured different buckets on a different population, so it neither confirms nor disproves. B borrows the methodology of separate research described elsewhere in the same deck. C is the opposite of true, since no academic replication exists at all.

3 · Why is dismissing feature requests worse than building too many of them?

Over-building wastes effort you can see and account for. Dismissing quietly destroys an information channel, and the silence that follows looks exactly like everything being fine. Note that taking a request seriously and building the named feature were never the same decision.

Sources covered

What this session teaches, and where it came from

✓The five figures: never 45, rarely 19, sometimes 16, often 13, always 7Read from the primary source, a 2002 conference keynote deck that survives only in a web archive capture
✓That "64% rarely or never" is the same slide45 plus 19 from the same pie. Established from the primary, not inferred from secondary retellings
◐What was measured, on how many, and whenNot stated, because the slide gives none of it and the two later accounts disagree with each other. Never described here as a study
✓The 2019 feature adoption telemetryRead in full: 615 subscriptions, customers of over a year, three months of aggregated usage, and the four buckets as published
◐That the telemetry confirms the older figureNot claimed, and the page says why. Different buckets, different decades, disagreeing in both directions, with no academic replication of either

Every fact on this page, and its verification tier, is recorded in the course's source map.

Session 2 cheat sheet · pin this

Evidence, not specificationA request proves a problem exists. It never says what to build.
Restate the problem backIf you have it wrong they will correct you at once. That is the point.
"What would you do if it did not exist?"Finds the workaround without sounding like a refusal.
Dismissing beats over-building for damageThe requests stop. The problems do not.
Group by problem, never by feature nameSame words, two problems. Five names, one problem.
Count distinct people, not requestsAnd ask whether the asker holds the job.