learn-ai-research-with-phoebe / Researcher session 10 of 10
Learn AI Research with Phoebe · Researcher track · Session 10 of 10

Capstone: the research memo

Nine sessions ago the Cadence question had ten claims and none of them survived. Tonight you assemble what the track produced into the one artifact anybody outside your team will ever read: a memo short enough to be read, honest enough to be trusted, and built so that the weakest claim in it is the one you point at first. Nine claims survive. The tenth stays unknown, priced, with its test named - and that unknown will very likely be the most useful sentence in the document. This is a closing session, not another lesson. Assemble, review, present, and go and run it on something real.

🔴 Researcher track Analysts · academics · UX & market researchers Ship it at 9 of 10 plus one honest unknown 45 min
0-3 · What we built 3-18 · The memo and the pre-send review 18-41 · Build-along: the capstone gate 41-45 · Close
Part 0

What the track built 4 min live

Every session in this track left an artifact rather than a feeling, and that was deliberate: a memo assembled from artifacts can be defended line by line, and a memo assembled from memory cannot. If you have been doing the homework, you already own the whole document. Tonight is assembly and review.

Live - presented in session Self-study - read after class ▶ Live evidence lab - toggle levers, watch claims survive or die Official sources covered
LiveThe eight artifacts, and what each one is for4 min▶

Read this as an inventory check. Anything missing is tonight's homework, not a reason to skip the memo:

  • b2 - the decision-scoped claim list. Claims that could be false, tied to a decision with a date. This becomes the memo's spine.
  • b3 - the documented search and tier tags. Where you looked, with what queries, and what tier every source carries. This becomes the first half of the method appendix.
  • b4 - the screening log with exclusion reasons. What you excluded and why. The reasons are the valuable part, because they are what a reviewer cannot otherwise reconstruct.
  • b5 - the extraction table. One row per source with sample, measure, number, location, funder and population. Every figure in the memo traces to a row here.
  • b6 - the audit verdicts and the triage. Which claims survive, which failed, and whether each failure was fatal, work or honesty.
  • b7 - the contradiction and replication record. Who disagrees, how strong the disagreement is, and whether the finding has held up more than once.
  • b8 - the calibrated claim statements. Claim, weight, basis, and what would change your mind. These are the sentences that actually go in the memo.
  • b9 - the qualitative audit trail. Codebook version, coder agreement, saturation evidence, verbatim quotes, consent and data handling.

Notice what is not on the list: a literature summary. Nobody is going to read a summary of the literature. They are going to read your claims and then check one of them.

★ What you walk out with today A memo structure with a source for every section, the two gates a memo has to pass before it goes anywhere, a six-check pre-send review you can run in twenty minutes, the order to present in so the method appendix stops sinking the meeting, and the move that turns your weakest claim into a credibility gain.
Part 1 · the document

The memo structure, section by section 7 min live

Six sections, in this order, and the order is doing work. The decision comes first because it is the only reason the document exists. The method appendix comes last because it is evidence you are prepared, not the argument itself. Nothing here needs to be long: the most useful research memo any of us has written was two pages plus an appendix.

SectionWhat goes in itWhere it came from
1 · The decision and the questionThe decision being made, who makes it, by when, and the question scoped tightly to it. Two or three sentences.b2
2 · The claim listEach load-bearing claim with a verdict and one calibrated confidence sentence: claim, weight, basis, what would change it.b2 claims, b6 verdicts, b8 wording
3 · What we could not findSearches that returned nothing usable, and claims only vendor or promotional material would support. Stated as findings, not apologies.b3 search record, b4 screening log
4 · The honest unknownThe claim that does not survive, what not knowing it costs, and the specific test that would settle it.b6 triage, b7 replication check
5 · Method appendixSearch record, screening log with exclusion reasons, extraction table, tier tags, and the qualitative trail if you ran interviews.b3, b4, b5, b9
6 · AI-use disclosureWhich tools did what, at which stage, and where human oversight sat. Written to the rule of the venue you are sending it to.b8 wording, venue rule
LiveThe two gates - and they are not negotiable4 min▶

A memo ships when it passes two gates. Everything else in this session is machinery for getting through them.

  • Gate one: every load-bearing claim either survives the audit or is explicitly marked unknown. There is no third state. A claim that failed and got softened into "early evidence suggests" has not passed the gate - it has been smuggled through it wearing research clothes. Load-bearing means the decision changes if the claim is false; anything that is not load-bearing can simply be cut.
  • Gate two: the memo names what would change its mind. Per claim, in the confidence sentence, and once at the top for the recommendation as a whole. This is the difference between a position and a preference, and it is the line experienced readers look for first.

Both gates are cheap if you built the artifacts and expensive if you did not, which is the entire argument for building them as you go.

Self-studyWriting the honest unknown so it survives editing3 min read▶

The unknown section is the one most likely to be edited out by someone senior who wants a cleaner document. Write it so that removing it obviously costs something:

Deletable

"Further research is needed on whether the engine transfers to lecture audio."

Load-bearing

"Unknown: whether our meeting-notes engine transfers to lecture audio without retraining. Support is one vendor-funded preprint, n=48, with nothing independent behind it. If it does not transfer, the pricing and the launch date both move, so this is the single largest open risk in the plan. A two-week internal benchmark on 20 real lecture recordings would settle it. Until it does, we should not price on transfer working."

Three ingredients make it survive: the price of not knowing, a test that is small enough to actually be commissioned, and a decision rule for the interim. An unknown with a cost, a test and a rule is a plan. An unknown without them is a caveat, and caveats get deleted.

Part 2 · run this as a script

The pre-send review, six checks 8 min live

Run this on the finished document, not the draft you audited in b6, and run it as a script rather than a vibe. It takes about twenty minutes on a two-page memo and it catches exactly the failures a hostile reader recognises on sight. Do it the day before you send, not the hour before.

1 Re-run the seven audit rules on the final claim set, not the draft you audited in b6 2 Re-check every number against the location you recorded, never from memory 3 Confirm nothing is retracted check every citation in Crossref - free, no sign-up needed 4 Quotes and confidentiality every quote verbatim; nothing confidential left the room 5 Disclosure matches the venue ICMJE, COPE, Springer Nature, Science and Elsevier differ 6 Read the confidence words aloud say each sentence against the evidence actually behind it Two gates: every load-bearing claim survives or is marked unknown, and the memo says what would change it.
🔍 Click to zoom - the six checks, in the order to run them
LiveChecks 1 to 3: the audit, the numbers, the retractions4 min▶
  • Re-run the seven audit rules on the final claim set. Claims move during writing. A claim you audited as "institutions most likely hold the budget" has a habit of becoming "institutions hold the budget" by the third draft, and that edit changes the verdict. Audit what you are actually sending.
  • Re-check every number against its recorded location. Not against the draft, not against your memory of the source, and not by asking a model to confirm it. Open the extraction row, open the source at the recorded location, read the figure. This is the check that catches the denominator that quietly disappeared.
  • Confirm no citation has been retracted. Crossref carries retraction and post-publication update information, and it needs no sign-up. Run the final reference list through it. A withdrawn citation is a credibility loss you cannot argue your way out of, and it is the cheapest possible thing to prevent.
LiveChecks 4 to 6: quotes, disclosure, and the words themselves4 min▶
  • Quotes and confidentiality. Every participant quote verbatim from the transcript, with its reference. And a hard look at what went where: which tools saw the transcripts, which saw confidential client material, and whether any of that breached what the participants agreed to or what your contracts allow.
  • The disclosure matches the venue's rule, and the rules genuinely differ. ICMJE asks for it in both the cover letter and the submitted work, splitting writing assistance into acknowledgements and data or analysis work into methods. COPE puts it in Materials and Methods. Springer Nature documents it in Methods. Science asks for the cover letter plus methods or acknowledgements. Elsevier requires a separate declaration statement with prescribed wording. For commercial insights work, ICC/ESOMAR Article 9(b) requires disclosing whether AI or synthetic data played a significant role in sampling, deployment, analysis or interpretation, and to what extent human oversight was involved. Write to your venue, not to a generic template.
  • Read the confidence words out loud. Out loud, genuinely. "Institutions hold the budget" and "institutions most likely hold the budget, based on national IT-spend statistics and twelve buyer interviews" are different sentences, and your ear catches the mismatch between the word and the evidence faster than your eye does.
Part 3 · the meeting

Presenting it without sinking it 6 min live

Most good research memos are lost in the first four minutes of the meeting, and almost always the same way: the researcher opens with the method. Everyone in the room hears "this is going to take a while" and starts reading their phone. The fix is an order, and it is the reverse of how you did the work.

LiveThe order that survives the room3 min▶

Four moves, then stop talking:

  • The decision. "You are deciding by the 30th whether we build lecture notes for higher education." One sentence, and it makes everyone in the room the same kind of listener.
  • The recommendation with its confidence. "We recommend a limited pilot rather than a full build. Moderate confidence." The confidence word is not hedging, it is calibration, and stating it up front buys you enormous credit later.
  • The two or three claims that carry it. Not all nine. The ones the recommendation would break without, each with its basis in half a sentence.
  • The unknown and its test. "One thing we do not know, here is what it costs us, here is the two-week test." Ending on this rather than on your evidence base is what makes the room trust the rest.

The method appendix goes behind the recommendation, not in front of it. It exists so that when someone asks how you know, the answer is a page reference rather than a story. Bring it, refer to it, do not present it.

LiveWhen someone attacks your weakest claim3 min▶

They will, and they will find the right one, because weak claims read as weak even to people who have not done the work. There is exactly one good response: agree with them, immediately, and point at the audit row.

"You are right, and that is the one I am least confident in. It rests on a single study of 48, it is in the memo marked as untested, and the two-week benchmark on page four is how we would settle it."

Watch what that does. The person who came to find a hole discovers you found it first, documented it, and priced it. Everything else in your memo just became more believable, because you have demonstrated that you say so when something is thin. A pre-empted weakness is a credibility gain. A defended weakness is a credibility loss, and it costs you the strong claims too.

The version to avoid, and it is tempting under pressure: adding evidence in real time from memory. Any number you produce in a meeting without its recorded location is a number you may have to retract in writing afterwards.

Self-studyThe three questions you should be able to answer instantly2 min read▶
  • "Where did that number come from?" Source, sample, and the location in the source. If the answer takes longer than ten seconds, your extraction table is not doing its job.
  • "Did you look at anything that disagreed?" Name the disagreeing source and say how you weighted it. This is the b7 record, and having it is the difference between synthesis and advocacy.
  • "What would change your mind?" You wrote it into every confidence sentence, so read it back. The room is not testing your evidence at this point, it is testing whether you have a position or a preference.
Build-along

The capstone gate ★ 18 min · everyone runs it

All five levers on: 9 of 10, 22 sources. This is the state your work should be in, and it is the state the memo gets written from. Do not read the lab this time - transcribe it. Write the memo's claim table straight from the survivor rows, one line per claim, then write the tenth as the honest unknown. When the table is done, press Add 40 sources exactly once, so the last thing you do in this track is remind yourself what unscreened thoroughness costs.

Five levers on: 9 of 10. Do not touch anything yet. Read the nine survivor rows and the one failure.

Write the claim table. One row per surviving claim: the claim, the verdict, the tier and source ids carrying it, and a confidence word you can defend. Nine rows.

Write the tenth as the unknown: the claim, what the support actually is, what not knowing costs, and the test. Use the b8 wording, not a softened version.

Now press Add 40 sources once. 62 read, 8 surviving. Read the new failure row, then turn it off and leave it off. That was the whole lesson in one click.

Last step: write the two-sentence answer to "should Cadence build this?" with its confidence word, and the one line saying what would change it.

What is real here and what is a teaching corpus The 22 sources are a teaching corpus built for one decision - they are plausible, not real publications. The audit is genuinely a rule engine: each source carries a type, tier, year, sample size and funder, and every verdict is derived from those attributes plus what the source supports or contradicts. What transfers to your own work is the rules, the triage and the memo structure, not the corpus.

And the scoreboard for the whole track, one last time. 0 → 1 → 2 → 4 → 7 → 9, then the anti-lever at 8 with 62 sources read. Five practices, forty-one minutes of lever-turning across ten sessions, and the single most reflexive move in research priced at minus one claim.

1 QUESTION 2 GATHER 3 SCREEN 4 EXTRACT 5 CHALLENGE 6 SYNTHESISE b2 claim list, decision-scoped b3 search record + tier tags b4 screening log with reasons b5 extraction table one row/source b7 contradiction + replication log b8 calibrated claim statements b1 - the loop and the seven audit rules the standard every artifact is measured against b9 - the qualitative audit trail codebook, kappa, saturation, verbatim quotes b6 - THE AUDIT GATE: seven rules, in a reviewer's order fatal / work / honesty. A claim that fails does not ship. b10 - THE MEMO claims with verdicts, the honest unknown, method appendix THE DECISION - should Cadence build a lecture-notes product? Nine claims survive. One is marked unknown, with its price and its test both named. The gate sits before the decision, never after the memo has already gone out.
🔍 Click to zoom - the whole track, from the loop to the decision
Real world

The thing that will actually change your working life is not the audit, it is the order. Screening rules before searching. Tier tags at capture, not at writing time. The number's location recorded when you read it. The disagreeing source logged the day you find it, not the week you defend it. None of these are hard, all of them are boring, and every one of them is cheap when done in sequence and brutally expensive when reconstructed under time pressure. AI made gathering and extraction fast. Spend the time it gave you back on the order.

After the course

Four things to do with this ◐ your next month

Source material

Official sources covered

This session assembles the track rather than introducing new standards, and it applies the disclosure and retraction-checking practice on the finished document. Normative documents stay with their bodies and are linked in the track's source map. This page covers:

✓The memo structure, the two gates, and the honest unknown with its price and testParts 0-1 · assembled from every session's artifact
✓The six-check pre-send review, run as a scriptPart 2 · including the Crossref retraction check
✓AI-use disclosure, and how the venues differPart 2 · ICMJE, COPE, Springer Nature, Science, Elsevier, ICC/ESOMAR Art 9(b)
✓Presenting order, and handling the attack on the weakest claimPart 3 · the method appendix goes behind the recommendation
◐Setting an org-wide evidence standard, and the governance railsNamed here as what comes next · leader sessions a2 and a4
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · What are the two gates a memo has to pass before it ships?

There is no third state between surviving and unknown - a failed claim softened into "early evidence suggests" has not passed the gate. And naming what would change your mind is what separates a position from a preference. Gate C is the opposite of the standard: the unknown belongs in the memo, priced.

2 · From a clean 9 of 10 you press "Add 40 sources". What happens, and why?

Nearly three times the reading, one claim worse off. Breadth without screening is not more evidence, it is more chances to import something retracted, duplicated, promotional or off-population. That is why b4 exists and why the answer to "can we look wider" mentions inclusion rules rather than effort.

3 · Which of these is part of the pre-send review rather than something you can leave to the meeting?

The review is six specific checks on the finished document: re-run the seven rules on the final claim set, verify each number at its recorded location, run the reference list through Crossref, confirm quotes are verbatim and nothing confidential leaked, match the disclosure to the venue, and read the confidence words out loud.

Researcher track cheat sheet · pin this

The loopQuestion → gather → screen → extract → challenge → synthesise, with the audit gate before the decision. AI compresses gather and extract only.
The seven rulesScoped → support exists → support good enough → nothing retracted → number recorded → disagreement handled → holds up twice.
TriageFatal (rewrite or abandon) · Work (extract, screen, match population) · Honesty (qualify, or state as hypothesis). Fatal first.
The ladder0 → 1 → 2 → 4 → 7 → 9 claims surviving. Anti-lever: 62 sources read, 8 surviving. Ship at 9 plus one honest unknown.
Eight artifactsb2 claims · b3 search + tiers · b4 screening log · b5 extraction table · b6 verdicts · b7 contradiction and replication · b8 confidence sentences · b9 qualitative trail.
Memo sectionsDecision and question · claim list with verdicts · what we could not find · the honest unknown · method appendix · AI-use disclosure.
Pre-send reviewRe-run the rules · re-check numbers at their location · Crossref retraction check · quotes and confidentiality · venue disclosure · read confidence words aloud.
The meetingDecision → recommendation with confidence → the two or three claims carrying it → the unknown and its test. Appendix behind, never in front. Agree with the attack on your weakest claim and point at the audit row.