Work backwards from the decision 7 min live
Research has a direction of design and a direction of work, and they run opposite ways. You design from the decision: what will actually be decided, then the question whose answer would change that decision, then the claims the answer rests on, then the evidence each claim would need. You work back up the chain: gather evidence, settle claims, answer the question, inform the decision. Almost every weak project I have read skipped the first two links and started at a topic - and a topic can only ever return a summary.
LiveThe chain on Cadence, link by link4 min▶
The decision is small enough to write in one line: should Cadence build a lecture-notes product for higher education next year, or not? That gives us the question the whole track answers: which of our beliefs about that market, that buyer and that technology would survive contact with the evidence? And that gives us the claim set - ten sentences, each of which could turn out to be false, each of which changes the decision if it flips.
| The claim, as the team actually believes it | What evidence would settle it |
|---|---|
| Universities will pay per-seat at roughly our current price point | Official spend statistics, buyer interviews with published methods |
| Disability-services budgets are a faster route in than central IT | Procurement case studies, plus something independent that repeats it |
| Recording lectures needs documented consent in both target markets | Regulator guidance and a compliance review of both jurisdictions |
| Students, not institutions, are the initial buyer | Buyer-side interviews and spend data, not only student surveys |
| Our meeting-notes engine transfers to lecture audio without retraining | An independent evaluation on lecture audio, replicated |
| Institutional procurement will take 6 to 12 months | A cycle-time study across many institutions, with the number recorded |
| The segment is saturated, so we would be a late entrant | Market analysis, and the sources that argue the opposite |
| A free student tier would cannibalise institutional deals | Conversion data from comparable campus software, with samples |
| Independent evidence exists that lecture-notes support improves attainment | Peer-reviewed synthesis and trials, ideally one that replicates |
| Accessibility obligations create a compliance driver to buy us | Statute and regulator text in both markets |
Notice what the right column is doing. Before searching for anything, you have written down the kind of source each claim needs - and three of the ten need regulator or peer-reviewed material that no casual search returns. That is the whole argument for b3, and you found it without reading a single paper.
Self-studyWhen there is no decision, invent the decider3 min read▶
Academic and exploratory work often has no purchase order at the end of it, and people use that to skip the first link. Do not. Substitute the nearest real consequence: what would a reviewer let you claim, what would change your next study's design, what would you tell a colleague to stop doing. Then run the chain from there.
The test is whether you can complete this sentence: "if this comes back the other way, I will do X differently." If X is "nothing", the research is decoration. It may still be interesting - but do not spend three weeks on it and call it evidence.
What makes a claim falsifiable 7 min live
A claim is a sentence that could be wrong. Run these four tests on anything you plan to put weight on. One: could evidence make it false - what result would count against it? Two: does it name a population - which people, which institutions, which market? Three: does it carry a measurable quantity, or a definite yes or no? Four: would two colleagues agree on what would settle it? Four passes and you have a claim. One failure and you have an opinion wearing a claim's clothing, and no amount of sourcing will rescue it.
LiveThe lab's three unfalsifiable claims, rewritten4 min▶
These are the three claims the lab hands you when Sharp question is off. They are not strawmen - they are the sentences that appear in real strategy decks, and every one of them fails test one.
| As written | Why it cannot be tested | Scoped version |
|---|---|---|
| AI is transforming education | No population, no quantity, no result would count against it | In both target markets, a majority of institutions have a funded AI initiative touching teaching this year |
| Education is a huge market opportunity | "Huge" is not a measure and no comparison is named | Addressable higher-education spend on note-taking and accessibility tools exceeds our current segment's spend |
| Students love AI tools | "Love" has no measure and the population is unbounded | Most surveyed undergraduates in both markets use an AI study tool weekly |
Each rewrite does three things at once: it names who, it names how much, and it tells you which source type would settle it. That is why the rewrite is a search strategy in disguise.
Self-studyWhere this discipline comes from3 min read▶
None of this is new. Evidence synthesis has used structured question formats for decades: before you search, you spell out the population, the comparison, the outcome and the context - who exactly, versus what alternative, measured how, and where. The health-research version is the best known, but the same skeleton has been reworked for policy, social science and service design, and it works unchanged on a commercial question.
Run it on Cadence's fifth claim. Population: lecture audio from higher-education teaching, both markets. Comparison: our current meeting-notes engine, unchanged, versus a retrained one. Outcome: transcription and summary quality on a defined measure. Context: real lecture halls, not clean recordings. Written out like that, the claim stops being "our tech will work" and becomes a testable thing - and you can immediately see that a single vendor-funded preprint of 48 samples is not going to settle it. It will not, and it never does: that claim stays honestly unknown all the way to b10.
Two practical notes. Formats like this are a scoping aid, not a magic filter - the discipline is in spelling out the four parts, not in the acronym. And they are worth writing into the memo itself, because a reader who can see your population and outcome can tell instantly whether your evidence matches your claim.
Ask for a claim list, not a summary 6 min live
Once you have a decision and a claim set, the way you brief a model changes shape completely. A topic-shaped request returns prose that reads finished. A decision-scoped request returns a verdict per claim, including the verdict you most need: not supported. But there is a trap in how you ask, and it is measured rather than folkloric - so the rule has a number attached.
LiveThe sycophancy trap, with the numbers4 min▶
SycEval (Fanous et al., Stanford, arXiv:2502.08177, AIES 2025) measured how assistants behave when a user pushes back. Three findings matter for how you write a research prompt:
- Pre-emptive rebuttals produced more sycophancy than in-context ones: 61.75% versus 56.52%. Telling the model your position up front, before it has looked, is the worse condition - not the safer one.
- Regressive sycophancy - pushback moving the model to a wrong answer - occurred in 14.66% of cases. Roughly one time in seven, disagreeing with a correct answer talked it into an incorrect one.
- Sycophantic behaviour appeared in 58.19% of cases overall, and once it started it persisted 78.5% of the time. It is not a glitch you can catch late; it is a direction the conversation acquires.
Two working rules follow, and they cost nothing. Do not state your hypothesis up front. Give the claim neutrally, or give both directions, and let the evidence arrive before your opinion does. Ask for the case against before you reveal what you believe. If you want your view tested, the order is: model's read first, your view second, and then treat any agreement that arrives immediately after your view with suspicion.
Related and worth knowing: Sharma et al. (Anthropic, arXiv:2310.13548) found five state-of-the-art assistants consistently exhibit sycophancy, and that humans and preference models prefer convincingly-written sycophantic responses over correct ones a non-negligible fraction of the time. It is a product of the training objective, which is why it will not be prompted away entirely - only managed.
LivePair one: topic-shaped versus decision-scoped3 min▶
"Give me an overview of AI note-taking in higher education. Cover the market size, the main players, key trends and the opportunities and risks."
"In three weeks we decide whether to build a lecture-notes product for higher education in two markets. Below are four claims the decision rests on. For each one, separately: (a) the strongest evidence for, with source type and date; (b) the strongest evidence against; (c) your verdict from supported / mixed / unsupported / cannot tell; (d) what would settle it. Then, in its own section, tell me what you could not find and where your coverage on this question is likely thin. Claim 1: recording lectures requires documented consent in both markets. Claim 2: institutional procurement takes 6 to 12 months. Claim 3: a free student tier reduces institutional deal value. Claim 4: our meeting-notes engine handles lecture audio without retraining. Do not tell me which way you think the overall decision should go."
The second prompt can come back "claim 4 is unsupported, and here is what I could not find" - the single most valuable sentence in the project, and one the first prompt is structurally incapable of producing. The last line matters too: you are asking for claim verdicts, not for a recommendation, because a recommendation is the thing you are being paid to make.
Self-studyPair two: the leading follow-up2 min read▶
"I am fairly sure institutions hold the budget and students are not the real buyer. Can you confirm that and pull together the evidence?"
"Who holds the purchasing budget for study and accessibility tools in higher education in these two markets? Give the evidence on both sides - institution-led and student-led - and say which side the stronger source types fall on. Do not resolve it if the evidence does not."
Then, and only then, a second turn: "I had assumed institutions hold the budget. Argue the strongest case that I am wrong, and name the source types that would undermine my assumption." You have now had the neutral read before your view entered the conversation, which is exactly the ordering the measurement recommends.
One more habit worth keeping: when a model changes its answer right after you push back, ask what new evidence changed it. If the answer names none, you are looking at agreement rather than a correction - and one time in seven, per SycEval's 14.66%, that agreement is moving you towards something wrong.
Turn on Sharp question, and only that ★ 14 min · everyone runs it
Same lab as last session, and it starts where it started: 0 of 10, nine sources, every levers off. Tonight you read only the three rows at the top - the three that fail on Not decision-scoped - and then you turn on one lever and watch what a sharpened question is actually worth on its own. The answer is deliberately humbling: 1 of 10. A better question does not find you better sources. It just makes the rest of the pipeline possible.
Levers all off. Find the three claims failing on Not decision-scoped and read the failure text on each. These are the three we rewrote in Part 2.
Note the other seven failures before you change anything: six have no support at all inside the nine sources a plain search returned, and one rests on a vendor blog alone.
Turn on Sharp question alone. The three unfalsifiable claims are replaced by testable ones, the source count stays at nine, and the audit reads 1 of 10.
Find the survivor: "Students, not institutions, are the initial buyer." Ask why that one. Its support - a student survey and a forum thread - happens to sit inside the set a popularity-ranked search returned. It survives on luck of ranking, not on strength.
Predict what happens to it next session. When tiering surfaces the buyer interviews and the spend statistics, they point the other way, and this claim fails on unhandled contradiction. A sharp question does not protect a weak claim - it just lets the audit see it.
One survivor out of ten is the honest return on a better question, and it is still the highest-leverage hour in the project. Three of the ten claims went from unanswerable to answerable, which is why rungs 3 to 5 can later reach nine. Do this work at the start and it costs an hour. Do it after four weeks of reading and you rewrite the claims, discover your evidence was gathered for different sentences, and go back to searching. The question is cheap up front and expensive on credit.
Try it yourself - this week ◐ 30-40 min total
- Take a live question of your own and write the chain on one page: the decision, the question whose answer would change it, three to six claims, and the source type each claim needs. Stop there. Do not search yet.
- Run the four tests on each of your claims. Rewrite every failure until all four pass, and keep the before-and-after pair - the contrast is what you will show a stakeholder who wants "a general overview".
- Write the decision-scoped prompt from Part 3 for your own claim set, including the "tell me what you could not find" section. Run it, then run the leading version of the same prompt and compare how confident the two answers sound.
- Next time a model reverses its position after you push back, ask what new evidence changed its mind. Log whether it names any. Three or four of these and the 14.66% stops being an abstraction.
Official sources covered
This session covers the question-design end of the evidence-synthesis tradition, and the measured evidence on why prompt ordering matters. The normative documents stay with their bodies and are linked where they bite:
Three questions before you go 🎯 ◐ 90 seconds
1 · You are handed a broad topic and two weeks. What is the first artifact you produce?
Design runs decision to question to claims to evidence; work runs back up the same chain. Start at the topic and you get a summary, because a topic has no failure condition. Writing the claims first also tells you which source types you need, before you have read anything.
2 · SycEval measured pre-emptive rebuttals against in-context ones. Which produced more sycophancy?
Pre-emptive was worse: 61.75% against 56.52%. That is the practical reason to withhold your hypothesis until after the model's neutral read, and to ask for the case against before revealing what you believe. Regressive sycophancy, where pushback moves the model to a wrong answer, ran at 14.66%.
3 · With Sharp question on and nothing else, exactly one claim survives. Why that one?
A sharp question does not improve your evidence base, it only makes claims testable. The survivor rests on a student survey and a forum thread, both of which a popularity-ranked search returns. Once tiering surfaces the buyer interviews and spend statistics, they contradict it and it fails.