The retrieval bench

Every model has numbers baked into it, and a bench that hides them is asking to be believed rather than checked. These four are on the widget.

• Two hops. Navigation starts at the places you actually open each week, not at the top of the tree, and runs two steps. Nobody clicks further than that for a note they half-remember.
• Twelve notes is a scannable place. Past about a dozen you are not looking any more, you are reading a list, and lists are where notes go to be forgotten. This is the single most consequential threshold in the bench: it is why one big folder does not count as somewhere you can find things.
• A useful tag points at eight notes or fewer. A tag sitting on twenty notes returns twenty notes, which is a list, not an answer.
• A title is searchable if a query word is in it. A note called "Notes" is not findable by search, whatever is inside it, because there is no word to match.

Disagree with any of them and the widget still works; the numbers move and the ordering of the five filings does not. That is worth knowing, because it means the lesson does not depend on the thresholds being exactly right.

Because that is what people do. Bergman and colleagues compared users of better and worse desktop search tools and found people estimated they used navigation, meaning browsing through folders, for 56 to 68 percent of file retrievals, and search for only 4 to 15 percent. Improving the quality of the search engine had a "limited and inconsistent" effect on how often people actually searched, and search was used "mainly as a last resort when users could not remember file location". These are self-reported estimates from questionnaires rather than logged behaviour, which is a real limitation and worth carrying with the number.

The consequence for a collection is direct. If most retrieval is navigation, then a note that navigation cannot reach is already relying on the fallback, and if its title has no searchable word the fallback fails too. That is not two independent problems. It is one problem with two locks, and session 2 unlocked one of them.

• One big folder finds nothing by looking. Zero of sixteen. Every question that gets answered is answered by search, and the four that search cannot answer are gone. The folder holds all forty-eight notes, which is why it reaches nothing: you cannot look in a place that size, you can only scroll it.
• Sorting into topic folders is the honest disappointment. Reachability triples, from 4 percent to 23 percent, and the lost count does not move at all. Still four. This is the weekend everybody spends: the collection becomes tidy, it photographs well, and the questions you could not answer on Friday you still cannot answer on Monday.
• PARA halves the lost count, from four to two, because filing by what you are doing puts the project material next to the two notes you actually open. Nine of sixteen now answer by looking. Note what did it: not better categories in the abstract, but categories that sit where your attention already is.
• Links are the move. Adding the links a person would naturally make, from the project notes to the things they depend on, takes it to fourteen of sixteen by looking and 73 percent reachable. Thirty-seven links. That is an afternoon, and it does more than the entire filing exercise before it.
• Tagging everything is the trap. One hundred and ninety-two filing decisions, four tags on every note, and the buckets are identical: fourteen, one, one. It is not that tags are harmful. It is that this work bought nothing, while feeling more like organising than anything else in the list.

Read the panel under the numbers on the best filing. One question survives every improvement: "the competitor thing somebody sent me."

Its note is titled "Interesting". Nothing links to it. It sits in a perfectly correct folder under a perfectly sensible category, and there is no route to it and no word to search for. No amount of further filing recovers it, because filing was never the thing that was broken. It was captured in four seconds by a rule that said keep what resonates, given a title that meant something for about a day, and then left alone.

That single row is the argument of the whole course. A note is recoverable if something points at it or its title carries the word you will look for. A note with neither is a note you have already lost, and you lost it at the moment of capture, not at the moment of filing.

Open your own collection beside the bench and answer four questions honestly.

How big is your biggest place? If one folder, notebook or tag holds more than about a dozen notes, that place is not somewhere you find things. Count it. Most people find their biggest place holds more than half of everything they own.

What do you open every week? Two or three notes, usually. Those are your entry points, and everything you can reach in two steps from them is your real collection. Everything else is in storage.

How many links are there? Not tags, not folders: notes that point at other notes. For most people the answer is zero, which puts them on the third row at best.

Ask yourself four questions you failed to answer this year, and trace each one. Reached from an entry point? Matched by a title? Or gone? That ratio is your own bench result, and you can run it without any tooling at all.

Be careful with this one, because it is the easiest result on the page to overstate. The bench shows that in this collection, applying four tags to every note bought no additional answers. That is a measurement of a cost and a return, in one constructed corpus, under stated thresholds.

It is not a finding that tagging harms retrieval. The personal-information-management literature compared folders and tags on the same material and reports no clean winner: each model has strengths and weaknesses for keeping, organising and re-finding. And no study was found that quantifies a point at which adding tags starts to cost you, which is why no such number appears anywhere here. The defensible sentence is the narrow one: tagging is work, work should buy something, and in this collection it did not.