Searching on purpose 7 min live
If you accept what ranked first, your evidence base is a popularity sample. Ranking optimises for links, freshness and clicks; it does not optimise for the peer-reviewed study on your population or the regulator's own text. Neither of those tends to rank, which is precisely why our claims about consent and attainment had no support at all in b2. A search you did on purpose has four things a ranked scroll never has: named sources, dates, the actual strings, and a record of what you excluded and why.
LiveWhy one database is never enough4 min▶
This is measured, not asserted. Bramer et al. (Systematic Reviews 2017;6:245) took 58 systematic reviews and asked what proportion of the included studies each database would have found on its own:
| Searched alone | Recall of the included studies |
|---|---|
| Embase | 85.9% |
| MEDLINE | 78.8% |
| Web of Science | 68.1% |
| Google Scholar | 34.4% |
| All four combined | 98.3% |
Read the last two rows together. The most familiar tool in the list found about a third of the relevant literature on its own, and the combination found nearly all of it. The paper's own conclusion is a floor, not an aspiration: searches "should search at least Embase, MEDLINE, Web of Science, and Google Scholar as a minimum requirement".
Now the honest translation for commercial work, because most of us are not doing health reviews. There is no equivalent set of four corpora for a market question. Institutional spend statistics, regulator guidance, procurement case studies, analyst coverage and trade press live in five different places, and no index covers them all. That does not excuse you from the discipline - it raises the bar. If you cannot name the canonical databases, you must name the sources you actually searched, one by one, so a reader can see the shape of the hole. "I searched the web" describes nothing and defends nothing.
Self-studyBuilding strings that do not lie to you3 min read▶
Take the scoped claim from b2 and lift the parts straight out of it. Population becomes one bracket of synonyms, outcome becomes another, and you join them with AND. Everything inside a bracket is an OR list, because vocabulary varies by field: lecture capture, lecture recording, recorded lectures, note-taking support. Miss a synonym and you will believe a literature is thin when it is only differently named.
Two habits keep you honest. First, run the string in more than one place and note where the results diverge - a term that works in one index often fails in another, and the divergence is information about coverage. Second, when a search returns almost nothing, treat that as a hypothesis about your string before you treat it as a finding about the world. Absence of evidence and absence of searching look identical in a results page.
Three services worth knowing by name 6 min live
Underneath every AI research tool sit a handful of open metadata services. Knowing them directly is useful for two reasons: you can check what a tool claims to have searched, and you can answer questions no chat interface will answer honestly, such as "has this paper been retracted?" All figures below were verified live on 2026-08-08 - treat them as a snapshot, and re-check before you quote them.
| Service | Scale, verified 2026-08-08 | What it is for | The catch |
|---|---|---|---|
| OpenAlex | 323,171,764 works · 123,255,918 authors · 255,204 sources · CC0 | Open, licence-clean coverage of the scholarly record. The default backbone when you want breadth without a subscription. | Documents a metered credit model: $1/day free with an API key, $0.01/day without, search $0.001 per query, PDF download $0.01 per item. An unauthenticated search still returned 200 on test, but treat metering as the forward contract and get a key. |
| Crossref | 185,277,841 records · no sign-up needed | The registry behind DOIs, and the place you check retractions and post-publication updates. If a citation has been withdrawn, this is where it shows. | Adding a mailto to your request moves you into the polite pool and triples your rate limit from 1 per second to 3 per second - verified from the response headers. It costs one parameter and it is rude not to. |
| Semantic Scholar | 214M papers · 2.49B citations · 79M authors | Citation graph plus useful per-paper fields, including openAccessPdf - the practical bridge from a hit to a paper you can actually read. |
Unauthenticated access is "rate-limited to 1000 requests per second shared among all unauthenticated users", while an API key's introductory limit is 1 RPS. The key buys stability, not throughput. And the tldr field is machine-generated: never quote it as the authors' words. |
The retraction check is the one that saves a career, and it takes ten seconds. Before a number goes in your memo, look the DOI up and see whether the record carries a retraction or correction notice. In b1 you watched the anti-lever drop a clean audit from nine claims to eight because one bulk hit was retracted and landed in a previously clean claim's support. That is not an exotic scenario - it is what happens when volume arrives without screening. b4 turns this into a routine step in the pipeline; tonight, just know where the answer lives.
Self-studyWhy the machine-written summary field matters2 min read▶
The tldr field is a genuine convenience and a genuine trap. It is generated, not written by the authors, so a sentence lifted from it and quoted in your memo attributes a claim to researchers who never made it. That is the exact failure this whole course is built around: a real source that does not support the sentence written on top of it.
The rule is simple and it applies far beyond one field. Any text a system generated about a source is a pointer, not a quotation. Machine summaries, AI answer boxes, tool-generated abstracts - use them to decide what to open, never as the thing you cite. What you quote is what you read.
Four tiers, and the rules that make them work 7 min live
Tiering is how you stop treating a press release and a randomised trial as two data points. Four levels, assigned by what kind of thing the source is, not by whether you agree with it. The tier does not tell you the source is right - it tells you how much weight a claim can put on it before a reviewer objects.
LiveThe three rules that make tiering survive contact with a deadline4 min▶
- Tag at capture, never retro-tag. Assign the tier the moment the source enters your list, before you know which claim it will end up supporting. Tier a source after you have written the argument and you will tier it generously, because you now need it to be good. This is the single rule people drop first and regret most.
- A claim resting only on tier 4 fails outright. Not "is weaker" - fails. A vendor blog is evidence that a vendor said something, which is occasionally the claim you are making and usually not. In the lab, one claim fails at rung zero for exactly this reason.
- Tier is not enough on its own. Check the funder and what they sell. Check the date against what has actually changed in the mechanism - a 2019 procurement study is probably still fine, a 2019 study of AI tool adoption is not, and the question is not the age but whether the thing being measured moved. And check the population: a well-run trial on school-age students is a tier-1 source and the wrong source for a claim about undergraduates.
On that last point, the honest caution from the standards literature applies: a design label is not an appraisal. A credible synthesis can summarise biased studies, and a poor synthesis can summarise good ones. Tier gets you to the right shelf. Reading gets you the answer.
LiveWhat the AI search tools can and cannot see3 min▶
These tools are genuinely useful and they all have a documented boundary that decides whether their output can carry a tier-1 or tier-2 claim. Learn the boundary, not the interface:
- Elicit works from title and abstract only unless the source is open access, or you supply your own subscription plus their browser extension. So an extracted number from a paywalled paper came from an abstract, and abstracts routinely omit the sample and the caveat.
- Consensus and its Meter analyse the top 20 papers for a query. That is a sample of the top of one ranking, not a synthesis of a literature - and after Bramer, you know what one ranking's top slice is worth.
- Undermind has the most interesting published evaluation of the group, including a convergence estimator that judges comprehensiveness by fitting the decay of the discovery curve. Its published evaluation covers arXiv only. Excellent for machine learning literature, silent on higher-education policy.
None of this makes them unusable. It makes them gathering tools whose coverage you have to state. If a claim in your memo rests on a tool that read abstracts, that belongs in the record, in the same row as the tier.
Turn on Source tiering ★ 14 min · everyone runs it
The lab opens where b2 left it: Sharp question on, 1 of 10, nine sources. Tonight you turn on Source tiering and the source count goes from 9 to 22. Watch what arrives: the meta-analysis, the two trials, the replication, the regulator guidance, the compliance review, the spend statistics, the buyer interviews - not one of which a popularity-ranked search returned. Two claims revive on that material. And one claim dies, which is the more interesting half of the lesson.
Start from Sharp question only: 1 of 10, nine sources. Note the survivor and note how many claims fail with "no support in your set".
Turn on Source tiering. Sources go 9 to 22. Read the new arrivals before you read the verdicts - they are all tier 1 and tier 2, and they are all things ranking never showed you.
Two claims revive: the attainment claim, now resting on a meta-analysis, a trial and a replication, and the accessibility-obligations claim, now resting on regulator text and a compliance review. That is what tiering buys.
Now the survivor from b2 - "students, not institutions, are the initial buyer" - fails. Tiering surfaced the buyer interviews and the spend statistics, and they point the other way. It was never strong; it was unopposed.
Net: 2 of 10. Say the lesson out loud: tiering does not add claims, it adds the sources that decide them - in both directions. Better evidence sometimes takes a claim away from you.
Try it yourself - this week ◐ 30-40 min total
- Fill in the six-row search record for a real question of your own, including the exclusions row. If you cannot name the sources you searched, that is the finding - write it down and then go and name them.
- Take the last ten sources you cited in anything and tier them retrospectively. Count how many tier 3s and 4s were doing load-bearing work, then notice how tempted you are to grade them up. That temptation is the argument for tagging at capture.
- Pick one number you have quoted from a paper in the past year, look its DOI up in Crossref, and check for a retraction or correction notice. Add the
mailtoparameter while you are there and see the polite-pool limit for yourself. - For any AI research tool you use, find and write down one sentence of its documented coverage boundary - what corpus, full text or abstract, how many results considered. Keep the sentence. It belongs in your method section.
Official sources covered
This session covers the gathering half of the evidence-synthesis standards, plus the current verified state of the open metadata layer. Figures were fetched from primary sources on 2026-08-08:
Three questions before you go 🎯 ◐ 90 seconds
1 · In Bramer et al.'s 58 reviews, what did Google Scholar alone recall, and what did the four databases combined recall?
Google Scholar recalled 34.4% on its own; Embase was the strongest single database at 85.9%, then MEDLINE 78.8% and Web of Science 68.1%. All four together reached 98.3%, which is why the paper names those four as a minimum requirement rather than a nice-to-have.
2 · Why does Crossref belong in a researcher's toolkit even if you never build anything?
Crossref is where you check whether a citation was withdrawn or corrected - the ten-second check that stops a retracted source carrying a claim. It needs no sign-up, and the polite pool costs one parameter. It does not rank quality and it is not a full-text service.
3 · Why must you tag a source's tier at capture rather than later?
Retro-tagging is motivated reasoning with a spreadsheet. Assign the tier before you know what the source will be needed for, and the label stays honest. This also makes the tier-4-only failure rule enforceable, because nothing has quietly been promoted to tier 3 on its way into the argument.
Researcher session 3 cheat sheet · pin this
mailto for 3/s. Semantic Scholar 214M papers, 2.49B citations, openAccessPdf.tldr field is machine-written. Any system-generated text about a source is a pointer, not a quotation.