The manual arrives
Session 4 gave TriageBot a router: small model for the easy 85%, reasoning model for the hard 15%. Then Meridian's legal team shipped a 50-page refund and returns policy that every triage decision must now respect. Suddenly TriageBot is a long-context system, and everything you learned about short prompts needs a supplement: document placement, grounding the answer in the actual policy, and caching so the manual does not bankrupt the ticket queue.
Where things go: three vendors, three answers 6 min live
Ask the three big vendors where instructions and documents belong in a long prompt and you get three different official answers. Claude's docs: put long documents at the top and the query AFTER the docs - worth up to +30% response quality on long-document tasks. The GPT-4.1 guide: put instructions at BOTH the beginning and the end of long context. Gemini 3: data first, question LAST. These are not rumors; they are each vendor's published guidance.
LiveHow to hold three contradicting truths4 min▶
The disagreement is not sloppiness - it reflects how each model family was trained to attend over long sequences. The honest takeaways:
- Placement matters. Claude's +30% figure alone should end any "it doesn't matter where things go" argument. Position is one of the cheapest quality levers you have on long prompts.
- Know your vendor's guidance and start there. Claude: docs top, query after. GPT-4.1: instructions at both ends. Gemini 3: data first, question last. Defaults first, opinions later.
- Then test on YOUR task. Vendor guidance is a prior, not a law. A 20-minute A/B with your real documents beats any blog post, including this one. Sessions 8-9 make that A/B a repeatable eval.
The cross-vendor migration trap. Meridian's team drafted TriageBot v3 for Claude (manual at top, query after), then trialed the same string on a GPT model and quality dipped. Nothing was "broken" - the layout was tuned for one vendor's attention pattern. Multi-vendor systems need per-vendor layout variants, not one blessed string.
Making the answer come from the documents 8 min live
A long document in the prompt does not guarantee the answer comes from it. Models happily blend the document with their training data and their imagination. Grounding is the set of patterns that force the answer back onto the page - and it is mostly three moves.
LiveMove 1: document tags with metadata3 min▶
Wrap each document in tags that carry its identity, so instructions can refer to documents by name and the model can cite its source:
The metadata is not decoration. "Quote the source you used" only works if the source has a name, and multi-document prompts collapse into soup without per-document boundaries. This is the same data/instruction separation from session 2, scaled up.
LiveMove 2: extract quotes first, then answer3 min▶
The single most effective anti-hallucination move on long documents (tutorial Ch8, and the consolidated best-practices docs): make the model extract the relevant quotes before it answers. Two steps in one prompt: "First, pull the exact lines from the policy that bear on this ticket into quote tags. Then answer using only those lines."
- The quotes act as a checkable intermediate: if the quote step comes back empty, the honest answer is "the document does not cover this" - and the model is far more likely to say so.
- You get an audit trail for free. When TriageBot approves a refund, the quoted policy line is right there in the output.
- It composes with everything else: schemas, routing, reasoning models. Quote-grounding is a layer, not a fork.
Self-studyMove 3: tune context reliance2 min read▶
The GPT-4.1 guide makes explicit a dial most people never set: how much the model should rely on the provided context vs its own internal knowledge.
| Mode | Instruction shape | When |
|---|---|---|
| Docs only | "Answer ONLY from the provided documents. If they do not contain the answer, say so." | Policy, legal, compliance, anything auditable - TriageBot v3 lives here |
| Blend | "Use the documents first; supplement with general knowledge and flag which is which." | Research assistants, drafting, exploratory work |
Leaving the dial unset means the model picks per answer, silently. Set it in writing, every long-context prompt.
Self-studyPaste, RAG, or "grounding"? Plus two pointers3 min read▶
| Approach | What it is | When it fits |
|---|---|---|
| Just paste it | Whole document in the prompt, patterns from this session | Corpus fits in context and is stable - a 50-page manual qualifies easily |
| RAG | Retrieve relevant chunks per query, inject only those | Corpus too big for context, or per-query freshness needed |
| Vendor "grounding" | Google's term for retrieval-backed answers (e.g. grounding with Search); RAG by another name | You want the vendor's managed retrieval instead of building your own |
Two pointers to file away. First, when you do build RAG, the eval vocabulary is the RAG Triad: context relevance (did we retrieve the right chunks), groundedness (does the answer stick to them), answer relevance (does it answer the question). Sessions 8-9 put numbers on all three. Second, prompt compression - trimming long context algorithmically to save tokens - exists, and the honest read is that it is marginal: a dedicated course spends 17 minutes on it. Know the term, reach for caching and retrieval first.
Structure for speed and cost 6 min live
TriageBot v3 will send the same 50-page manual with every single ticket. Unoptimized, that is the same tokens billed thousands of times a day. Prompt caching fixes it - but only if the prompt is shaped for it, and the shape rule is the one thing every vendor agrees on: static content first, variable content last.
LiveThe shape rule, and why v1 already had it4 min▶
Caching works on prefixes: the provider reuses the processed form of the leading tokens if they are byte-identical to a previous call. Anthropic prompt caching lets you mark cache breakpoints explicitly; OpenAI caches static prefixes automatically and its prompt-engineering guide tells you to put static content first for exactly this reason. Either way the design rule is identical: static content first, variable content last - a timestamp or ticket ID at the top of the prompt silently kills the whole cache.
Here is the satisfying part: TriageBot v1 was already cache-shaped. Back in session 2 we put the fixed instructions first and the ticket last purely for clarity - data/instruction separation. That same layout is what caching rewards. Good structure keeps paying dividends you did not plan for.
TriageBot v3: the policy manual arrives ★ 22 min · laptops open
Meridian ships the 50-page refund and returns policy, and every triage decision must now respect it. We rebuild TriageBot with everything from today: document tags, vendor-aware placement, quote-grounding, a context-reliance rule with a POLICY_GAP escape hatch, and a cache-shaped layout.
Wrap the policy in document tags. Source name, version, date, page count in the metadata - v4 of the policy will ship eventually, and you will want to know which version each decision quoted.
Place it per your vendor's guidance. On Claude: manual at the top, instructions and ticket after (the skeleton below). On GPT-4.1: repeat the key instructions above AND below the manual. On Gemini: data first, question last. Note in a comment where you would A/B - placement is the first eval you will run in session 8.
Add quote-grounding. "Before answering, extract the exact policy lines that apply into policy_quotes tags, then decide." The quotes become your audit trail and your hallucination brake in one move.
Add the context-reliance rule. Docs-only mode, with an escape hatch: "If the policy does not cover this case, output POLICY_GAP - do not improvise." Every POLICY_GAP is routed to a human and logged; Meridian's policy team gets a weekly list of holes in their own manual. Everyone wins.
Keep it cache-shaped. Instructions and manual and examples first, byte-identical on every call; the ticket dead last. Mark the cache breakpoint if your vendor uses explicit ones.
Test both sides of the boundary. Run one ticket the policy clearly covers (expect a decision with quoted lines) and one it does not (expect POLICY_GAP, not a confident improvisation). The second test is the one that matters - it is the difference between a grounded system and a fluent one.
Three things before session 6 ◐ 30 min total
- Run the placement A/B. Take one long-document task you actually have, build two layouts (your vendor's official one, plus one rival layout), run 5 inputs through each, and score which grounded better. Keep the numbers - session 8 turns this into a real eval.
- Add quote-grounding to one existing prompt. Any prompt of yours that answers from a document. Watch what happens on a question the document does not answer - before and after.
- Cache-audit your most-called prompt. Is anything variable (timestamp, user name, ID) sitting above the static content? Move it to the end and estimate the token savings at your call volume. Bring the number - it is usually the easiest cost win of the quarter.
What this session covers
Builder-level coverage of the sources below, including the honest cross-vendor disagreement on placement. Official vendor docs and course videos stay with the vendors; links live in the course map.
Three questions before you go 🎯 ◐ 90 seconds
1 · You are building a long-document prompt for Claude. Per the vendor's own guidance, the layout is...
Claude: docs top, query after (+30%). Both-ends is GPT-4.1's guidance; data-first-question-last is Gemini's. Know your vendor's default, then A/B on your own task - the three officially disagree.
2 · The point of "extract the exact policy lines first, then answer" is...
Quote-grounding is the anti-hallucination move for long docs: a checkable intermediate step, an audit trail, and a natural path to "the document does not cover this" instead of a confident improvisation.
3 · To make TriageBot v3 cache-friendly on any vendor, you put...
Caching works on identical prefixes - Anthropic via cache breakpoints, OpenAI automatically. One early variable token (like that timestamp) breaks the prefix and silently kills the entire cache.