Where TriageBot stands
By the end of session 3, TriageBot - Meridian Retail's ticket-triage prompt - emits schema-valid JSON, carries few-shot examples, and separates data from instructions with delimiters. It handles routine tickets beautifully and faceplants on the weird ones: multi-issue tickets, ambiguous refund requests. Today we fix the hard tail by understanding how models learned to reason - and why the prompt that made v2 great is the wrong prompt for a thinking model.
Four papers that became model features 8 min live
Between 2022 and 2023, four papers defined how practitioners squeezed reasoning out of models. All four still matter - but as history and architecture, not as phrases you type. Here is each one with its real 2026 status, so you never cargo-cult a dead trick or dismiss a live one.
LiveChain-of-thought: the trick that became a feature3 min▶
Wei et al. (2022) showed that giving a model exemplars containing intermediate reasoning steps - not just question and answer - unlocked multi-step reasoning that the same model failed at cold. The insight was real and it stuck: labs trained the behaviour in with reinforcement learning, and "reasoning models" were born.
That is why the status is partial, not dead. The concept is foundational - reasoning models are chain-of-thought internalized. But the prompt phrase is now redundant on models that already think, and still genuinely useful on small ones that do not. Same paper, two opposite instructions, depending on which model class you are holding.
Meridian's data team hit this exactly. Their v2 TriageBot prompt says "reason through each issue before deciding". On the cheap small model handling routine tickets: helps. Ported unchanged to the reasoning model they trialed for hard tickets: longer, costlier answers, no accuracy gain, and occasionally worse. Nobody had told them the advice had inverted.
LiveSelf-consistency and ToT: early test-time compute3 min▶
Self-consistency (Wang et al.): sample the same prompt many times at nonzero temperature, let the reasoning paths diverge, take the majority-vote answer. Look at what that is - spending more inference compute to buy more reliability. It was test-time compute before the term existed, and the labs absorbed the idea into the models themselves. It survives as a cheap reliability trick you can still run from the outside when you need one.
Tree of Thoughts (Yao et al. 2023) went further: generate candidate "thoughts", evaluate them, expand the promising branches, backtrack from dead ends. As a prompting pattern it is largely historical for practitioners - but it is the direct ancestor of the search that thinking models now run internally. Teach it as lineage, not as a recipe.
Self-studyReAct: the loop moved into the harness2 min read▶
ReAct (Yao et al. 2022) interleaved reasoning traces with actions - call a tool, observe, reason, act again. You used to hand-write this loop into the prompt: Thought / Action / Observation, round and round.
The honest status is architecture yes, prompt no. Native tool calling killed the hand-rolled version: you define tools, the model decides when to call them. But peel open any agent harness - Claude Code, an OpenAI agent, a LangChain agent - and a ReAct loop is running inside. The paper won so completely that it disappeared into infrastructure. Session 6 (agentic prompting) picks this thread up properly.
Prompting a model that already thinks 9 min live
Here is the part that breaks 2023 habits: on reasoning models, do NOT say "think step by step". This is OpenAI's official guidance, not folklore - it can hurt performance, and few-shot examples make o1-class models measurably worse. The whole 2023 playbook inverts: keep prompts simple and direct, state explicit success criteria, start zero-shot and only add examples if outputs miss the format.
LiveThe four principles, and the anti-pattern4 min▶
DeepLearning.AI's Reasoning with o1 course distills reasoning-model prompting into four principles that generalize across vendors:
- Simple and direct. Say what you want. The model plans its own method.
- No explicit CoT. Do not instruct step-by-step thinking; the model already does it, and the instruction can interfere.
- Structure and delimiters. Clean sections and delimiters still help - clarity never inverted, only method-coaching did.
- Show, don't tell. State criteria and constraints rather than narrating a procedure.
Here is the anti-pattern in the wild - TriageBot v2 habits ported straight onto a reasoning model:
Every line of that was best practice in 2023. On a reasoning model it micro-manages a planner that plans better than the script, and the worked examples drag accuracy down. Delete the method, keep the goal, state the bar.
LiveSame task, two model classes - side by side3 min▶
| Prompt element | Standard / small model | Reasoning model |
|---|---|---|
| Method | "Reason through each issue before deciding, inside <thinking> tags" | Omit entirely - state the goal, let it plan |
| Examples | 3-5 few-shot examples with reasoning shown | Zero-shot first; examples only if the format keeps missing |
| Criteria | Implicit in the examples | Explicit: "a multi-issue ticket takes the category blocking the customer's money" |
| Structure | Delimiters + schema (keep) | Delimiters + schema (keep - this never changed) |
| Compute knob | None - it answers immediately | Effort / thinking parameter, not prompt words |
Self-studyThe knobs: effort, thinking, and "Think very hard"3 min read▶
How much a model thinks is now a parameter, not a phrase. Vendor by vendor:
- Claude: adaptive thinking replaced the old extended-thinking
budget_tokenscontrol (deprecated, then removed). The budget control is now the effort parameter: low / medium / high / xhigh / max. Migrating old code? Delete the token budget, pick an effort level. - OpenAI: reasoning models take a reasoning-effort parameter; the prompt stays simple and the knob carries the load.
- Gemini: thinking is built in. The documented lever is the phrase "Think very hard" for the deepest setting - and the guidance explicitly says do not request redundant planning the model already performs.
The pattern across all three: depth of thought moved from prompt vocabulary to an API surface. Budget it like you budget tokens - deliberately, per task, and measured (sessions 8-9 give you the eval harness to measure with).
Manual CoT still earns its keep 3 min live
None of the above means CoT prompting is dead. On small and non-reasoning models - the cheap workhorses handling your high-volume traffic, or anything at the edge - induced reasoning is still one of the highest-leverage tricks you have. You just have to hand-roll it properly.
LiveThe recipe: think first, answer after3 min▶
Three rules, all boring, all load-bearing:
- Reasoning before the answer, always. An answer emitted first is an answer the reasoning then rationalizes. Boonstra's whitepaper is explicit: put the answer after the reasoning, and run CoT at temperature 0 - reasoning chains should be reproducible, not creative.
- Show the shape in your few-shot examples. Put the reasoning inside
<thinking>tags in the exemplars, and the model will follow the pattern - which also lets you strip the tags before the output reaches anything downstream. - Add a self-check when stakes justify it. "Before finalizing, verify your category against the ticket text; if they conflict, redo the reasoning." Crude, effective, costs a few tokens.
When you would still hand-roll it: cheap small models on high-volume routes, edge deployments where a thinking model does not fit the latency or hardware budget, and anywhere you need the reasoning trace visible for audit. Everywhere else, reach for the effort knob instead.
TriageBot learns to escalate ★ 22 min · laptops open
The insight of the day, applied: not every ticket deserves the same brain. We split TriageBot into a routing decision - easy tickets keep the small fast model with the full engineered v2 prompt; hard tickets escalate to a reasoning model with a deliberately simpler prompt. Criteria, not procedure.
Identify the hard tickets. Pull 10 real (or realistic) tickets and sort them: easy = one clear issue, obvious category. Hard = multi-issue tickets, ambiguous refund requests, anything where two reviewers might disagree. Meridian's data: roughly 85% easy, 15% hard - and the 15% caused nearly all the mis-routes.
Write the escalation rule. Easy tickets keep the small fast model with the full engineered v2 prompt from sessions 2-3. Hard tickets go to a reasoning model with a simpler prompt. Write the rule as criteria a router can check: more than one issue detected, refund requested without an order number, sentiment above a threshold.
Write the reasoning-model variant. Strip the method, keep the goal and the bar. Steal this shape:
Compare on two hard tickets. Run the same two hard tickets through both setups: v2 prompt on the small model vs the simple variant on the reasoning model. Read the disagreements out loud - typically the reasoning model untangles the multi-issue ticket the small model flattened.
Note the cost and latency tradeoff. The reasoning call costs more and takes longer - that is the price of the judgment. Write down both numbers per ticket. The router exists precisely so you pay it on 15% of traffic, not 100%. Sessions 8-9 turn this eyeball comparison into an eval you can rerun.
Three things before session 5 ◐ 30 min total
- De-CoT one of your own prompts. Find a prompt you run on a reasoning model that still says "think step by step" or narrates a method. Strip it to goal + criteria, run both versions on 3 inputs, and keep the outputs side by side.
- Play the effort knob. Take one genuinely hard task and run it at a low and a high effort/thinking setting. Note quality, latency, and cost per run - you are building the intuition the router needs.
- Write your own escalation rule. For one workflow you own, define in 3 bullet points which inputs deserve the expensive brain. Bring it - session 5 adds a 50-page policy manual to TriageBot and your rule will meet real context pressure.
What this session covers
Builder-level coverage of the sources below. Papers are taught as lineage with their honest 2026 status, per the course map. Official vendor docs and course videos stay with the vendors; links live in the course map.
Three questions before you go 🎯 ◐ 90 seconds
1 · You port TriageBot's five worked few-shot examples to an o1-class reasoning model and accuracy drops. The most likely cause is...
OpenAI's reasoning guidance is explicit: few-shot can degrade o1-class models. The 2023 workhorse inverts - start zero-shot, state criteria, and only add examples to pin a stubborn output format.
2 · On current Claude models, the control for how much the model thinks is...
Adaptive thinking replaced extended thinking, and the effort parameter is the budget control now. Depth of thought moved from prompt vocabulary to an API surface - budget it deliberately, per task.
3 · Hand-rolled chain-of-thought (thinking tags, answer after reasoning, temperature 0) is still the right tool when...
Manual CoT is alive and well on models that do not think by themselves. On reasoning models, reach for the effort knob instead - prompting the method there can interfere with a planner that plans better than your script.