The evidence standard you share

Tiering is trivially easy to write down and surprisingly hard to run, and the failure always happens in the same place: the tag gets applied after the conclusion exists. Once a researcher believes the answer, a vendor case study starts to feel like tier 3, a preprint starts to feel like tier 2, and nobody is lying - the tier just quietly drifts toward whatever the argument needs.

So make it mechanical:

• Tier is recorded in the same action as the link. If the row has a URL and no tier, the row is not finished. This is a two-second habit that removes an entire class of argument later.
• Tier is a property of the source, not of the claim. A vendor blog is tier 4 when it agrees with you and tier 4 when it does not.
• Never retro-tag a whole batch. Bulk tagging at the end of a project is indistinguishable, on the file, from having decided the answer first.
• Publishing venue is not authorship. A vendor-funded study in a peer-reviewed journal is tier 2 with the funder recorded, not tier 4 - and the funder column is why you record funders.

One more leader-grade nuance. Tier 3 is not a rubbish bin. Analyst reports and preprints are frequently the only source of a market number that exists, and the honest handling is to use them for direction and say so in the sentence: "analyst estimates put this in the region of X; no peer-reviewed figure exists". That sentence is defensible. "The market is X" over the same source is not.

Tiering matters more now than it did five years ago, for a specific structural reason: the ranking that surfaces sources is not sorted by tier. Vendor content is written to be found - clean headings, direct answers, generous keyword coverage - and regulator text and peer-reviewed synthesis are written to be correct. When a tool retrieves and summarises in one motion, the tier distribution of what comes back is a property of the index, not of your question.

The practical consequence for a leader is that an untiered AI search tends to produce a tier-3-and-4 evidence base with tier-1 confidence in the prose. The fluency is uniform; the underlying weight is not. Tiering is the cheapest available correction, because it makes the distribution visible: count the rows by tier before you read the conclusion, and you will know in ten seconds whether this is a decision-grade base or a market-narrative one.

In the researcher track's live audit, turning source tiering on is the rung that takes the source count from 9 to 22 - not because 22 is a magic number, but because that is when the peer-reviewed and regulator sources finally arrive. They were always there. Nothing had gone looking for them.

GRADE rates the certainty of a body of evidence, not of a single study, at four levels. Two of the definitions are worth having verbatim, because the wording is the point:

Notice what the High definition is about: the distance between the estimate and the truth. It is not about how impressive the study was, or how many sources agreed, or how confident the author sounded. That is the shift to teach your team.

Then the starting points, which are the part people find genuinely surprising: a body of randomised-trial evidence starts at High. A body of observational evidence starts at Low. You do not begin neutral and argue your way up. You begin at the level the study design earns and then move for named reasons. For a commercial insights team the translation is immediate: almost everything you work with - surveys, panels, analytics, interviews, market data - is observational, so your default starting point is Low, and every step above it needs to be argued. That single sentence changes how a lot of decks get written.

Movement between levels is not vibes. GRADE names five downgrading factors:

• Risk of bias - how the studies were run and reported.
• Inconsistency - the studies disagree with each other more than chance explains.
• Indirectness - the population, intervention or outcome is not the one you actually care about. This is the one that bites hardest in commercial research, where the available study is nearly always about a slightly different population.
• Imprecision - the interval is wide enough that opposite decisions sit inside it.
• Publication bias - what got published is not what got studied.

And three upgrading factors, which apply to non-randomised evidence only:

• Large effect - the magnitude is big enough that plausible confounding cannot explain it.
• Residual confounding that would reduce the effect - the biases you can identify all point the other way, so the true effect is probably at least this big.
• Dose-response gradient - more of the input, more of the output, in an orderly way.

Run this as a leader by asking one question in reviews: "which factor moved this, and in which direction?" If the answer is a factor name, the certainty word is earned. If the answer is a tone of voice, it is not.

Risk of bias has its own instrument, Cochrane's RoB 2, and two of its design choices are worth borrowing even if you never fill in the tool itself.

First: it is assessed per result, not per study. The same paper can be low risk for its primary outcome and high risk for a secondary one, so "this study is good" is not a statement RoB 2 can make. For your team, that kills the most common shortcut in AI-assisted synthesis - treating a paper as a unit of credibility and then citing whichever number inside it is convenient.

The five domains it assesses, in order: the randomisation process; deviations from intended interventions; missing outcome data; measurement of the outcome; selection of the reported result. Each gets one of three judgements: low risk, some concerns, or high risk.

Second: the overall judgement is effectively the least favourable domain. Four clean domains and one high-risk domain does not average to "some concerns". This is the rule to import wholesale into your own standard, because it is the exact opposite of how most decks are built. A claim is as strong as its weakest load-bearing link, not as strong as the average impression of its evidence - and the hostile reader from a1 will find the weakest link, not the average.

Murad and colleagues' core argument is one sentence long, and it is the most useful sentence in this session: "credible systematic review can summarize biased evidence and poorly done systematic reviews can summarize well done trials". Both halves matter. A rigorous review of a weak literature is still a weak evidence base - the rigour is in the method, not in the underlying studies. And a sloppy review of excellent trials wastes the trials.

So the review is not a higher grade of evidence. It is an instrument for looking at evidence - a lens, in their word - and the quality of the picture depends on both the lens and the thing being looked at. That is why they moved it off the top and drew it as something you view the pyramid through.

The wavy lines make the companion point. A design label sets a prior, not a verdict. A large, well-run cohort study with a pre-registered analysis can carry more weight than a small trial with high dropout and a swapped primary outcome, and the straight-lined pyramid has no way to express that. Waviness is how you draw "appraisal beats label".

The leader takeaway, and the line to say out loud in your next review: a design label is not an appraisal. "It is a meta-analysis" is the beginning of the conversation. What is inside it, how it was run, and whether its included studies would survive their own appraisal is the conversation.

Three habits follow directly, and all three are cheap:

• Stop accepting design labels as quality claims. When a slide says "based on a systematic review", the follow-up is "of what, and how many of the included studies were any good?" This is not scepticism for its own sake - it is the question the source's own field says you should ask.
• Separate the tier from the appraisal in your own standard. Tier says what a source may carry structurally. Appraisal says what this particular source, on this particular result, actually earned. Two columns, not one, and the second is where the RoB 2 least-favourable-domain rule lives.
• Treat "peer-reviewed" as a floor, not a ceiling. It clears tier 1 or 2 and tells you nothing about risk of bias, indirectness or imprecision. A funder-conflicted study of the wrong population is peer-reviewed and still nearly useless for your decision.

This is also the answer to a question leaders ask about AI-assisted synthesis: why is a fluent, well-cited summary still not enough? Because a summary reports labels. Labels are cheap to produce and cheap to read, and appraisal is neither.

Four tiers as in Part 1, written out with two or three examples each so nobody has to interpret them. Then three lines of policy:

• The tier is recorded when the source is captured. A row with a link and no tier is unfinished.
• The tier belongs to the source, not to the argument, and it does not move.
• A claim resting only on tier 4 does not ship. Not softened - not shipped.

Add a funder column while you are there. It costs nothing at capture and it is the difference between "a peer-reviewed study found" and "a vendor-funded peer-reviewed study found", which are two different sentences.

Pick four words, define each in one line, and forbid everything else. A plain-language version that maps cleanly onto GRADE:

Two rules make the list real rather than decorative. Every certainty word is accompanied by the reason it moved - a factor name from Part 2, not a tone. And "Unknown" is a permitted, respectable output. If your culture punishes it, your team will write "Limited" over nothing, which is the failure this whole standard exists to prevent. In the researcher track's audit, the tenth claim never survives at any setting, and the correct answer for it is exactly this word.

Every claim that carries a number needs a row, and the row has eight columns:

And then the rule that makes the table worth keeping: a number without a recorded row does not go in a deck. No exceptions for numbers that feel obvious, and none for numbers under time pressure - those are precisely the ones that turn out to be from a blog post quoting a webinar quoting nothing.

The standard also governs how work is commissioned, because a vague brief cannot produce a defensible answer no matter how good the standard is downstream. Compare the two requests a leader might send about the Cadence decision:

The first brief will come back looking excellent and will not be able to tell you no. The second can come back "claim 4 is unsupported" - the most valuable sentence in the whole project, and one the first brief structurally cannot produce. Note the last two clauses especially: asking what could not be found turns silence into information, and asking which claim is weakest forces someone to read the work the way the hostile reader from a1 will.