How this track works
Ten hands-on sessions for people who ship things: analysts, data engineers, data scientists, software engineers. Everything is tool-agnostic - every prompt here runs in any chat UI or any API. And everything is one project: across the whole track we build and harden TriageBot, a support-ticket triage system for Meridian Retail that classifies, summarizes, and routes customer tickets. It starts naive today and ends production-grade in session 10.
Success criteria before prompts 5 min live
Anthropic's prompt engineering docs open with a funnel, not a technique: define success criteria, build empirical evals, draft a prompt - and only then start engineering it. Their official Skilljar API course makes the same move: the evals module comes before the techniques module. That ordering is deliberate, and it is the single biggest habit difference between hobbyists and practitioners.
LiveWhy criteria come first - the practitioner's reason3 min▶
Without written criteria, "better prompt" means "the output I happened to like this afternoon". You will tweak forever because there is no finish line. With criteria, prompt engineering becomes normal engineering:
- You know when to stop. "Category accuracy at 90% on the test set" is done or not done.
- You can compare fairly. Two prompt versions, same 10 tickets, count the wins. No memory bias.
- You can change models safely. When a new model ships, rerun the set. Criteria outlive any one model.
Self-studyWhere fine-tuning fits - the decision boundary2 min read▶
One boundary mention, once, so it never distracts again: prompting is almost always the first tool, because it is instant, cheap, and reversible. Fine-tuning enters only when you have strong evals proving that prompting has plateaued, plus thousands of examples and a stable task. If you cannot measure the gap, you are not ready to fine-tune - which is another reason the funnel starts with criteria. This track does not cover fine-tuning further; the funnel is the point.
Prompting inside context engineering 5 min live
Anthropic's 2025 framing renamed the wider game: context engineering is curating everything that lands in the model's context window - system prompt, retrieved documents, tool outputs, conversation history - aiming for the smallest set of high-signal tokens that gets the job done. The prompt did not get less important. It got a precise job title: it is the one layer of the context you fully control.
LiveWhy the industry renamed the game3 min▶
Three practical consequences of the context-engineering frame, before we zoom back into the prompt layer for the next few sessions:
- Signal over volume. Stuffing the window hurts. Every irrelevant token competes with the relevant ones. Curation - choosing what NOT to include - is real work.
- The prompt is the constant. Retrieval varies per query, history grows per turn, tools return whatever they return. Your prompt is the stable contract holding it together - which is why it deserves engineering discipline.
- Techniques still work. What died was the standalone "prompt engineer" job title, not the skill. The skill got absorbed into every builder role, the way SQL did.
Self-studyThree words you will hear later: compaction, just-in-time retrieval, subagents2 min read▶
Anthropic's context-engineering post names three techniques for keeping windows lean. Today, names only - each gets real treatment later in the track:
- Compaction: summarizing older history into a short note so long conversations do not drown the window. Session 5.
- Just-in-time retrieval: fetching documents at the moment they are needed instead of preloading everything. Session 5.
- Subagents: handing a subtask its own fresh context window and getting back only the conclusion. Session 6.
Which model class am I prompting? 6 min live
This is the question that decides which advice applies. Reasoning models plan internally before answering; classic prompting tricks that induce planning can now actively hurt them. Standard models still respond to those tricks exactly as the 2023 playbook says. Ask this question first, every time, or you will apply the right technique to the wrong model.
Live2023 advice vs 2026 advice, side by side3 min▶
| 2023 said | 2026 says (reasoning models) |
|---|---|
| Always add "think step by step" | Never add it - the model plans internally; the phrase can degrade output |
| More examples = better | Zero-shot first; OpenAI measured reasoning models doing worse WITH examples |
| Spell out the method in detail | State the goal, constraints, and success bar; let it choose the method |
| Tune temperature and top_p | Use effort / reasoning parameters; some current models reject sampling params outright |
| Long elaborate role-play personas | Short domain framing; heavy personas demoted by Anthropic's own guidance |
The classic failure: a team upgrades to a reasoning model, keeps their lovingly tuned 2023 mega-prompt full of "think carefully, step 1, step 2" scaffolding - and quality drops. They blame the model. The prompt was fighting the model's own planning. Deleting half the prompt fixed it.
The six shifts that reshaped this craft 4 min live
If you learned prompting from a 2023 blog post, here is the diff. One line each - every shift gets its full session later in the track.
| # | Shift | One line | Where |
|---|---|---|---|
| 1 | Reasoning models invert 2023 advice | No induced CoT, few-shot can hurt, goal over procedure, effort params are the new lever | Session 4 |
| 2 | Prompt engineering → context engineering | Curate the whole token ecosystem; the job title died, the skill spread everywhere | Today + session 5 |
| 3 | Agentic prompting replaced scaffold prompting | Native tool calling killed hand-rolled ReAct loops; the skill is now tool descriptions and harness prompts | Session 6 |
| 4 | Security is architecture, not wording | "Ignore malicious instructions" is not a defense; injection needs design-level answers | Session 10 |
| 5 | Evals are the core practitioner skill | Prompt changes are code changes and need regression tests; tips without evals are obsolete | Sessions 8-9 |
| 6 | Automated prompt optimization is emerging | Metaprompting descendants can draft and improve prompts; a tool to know, not a pillar | Session 10 |
Meet TriageBot: criteria first, then v0 ★ 22 min · laptops open
Meridian Retail's support inbox gets hundreds of tickets a day. Our job across this track: a prompt system that classifies each ticket, summarizes it, and routes it - reliably enough to trust. Today we practice the funnel from Part 1 on it: criteria before prompt.
LiveThe raw material: 10 Meridian tickets3 min▶
Your working test set - real inbox energy, typos and rage included. Keep these; they follow us to session 10.
- "Charged twice for order #48210, refund one please"
- "Where is my parcel?? Ordered 9 days ago, tracking frozen since Tuesday"
- "The kettle arrived dented, I want to send it back"
- "Your website charged my card but shows no order confirmation???"
- "Jacket is lovely but too small - how do I exchange for a size up?"
- "Driver left the box in the rain, everything inside is soaked"
- "I keep getting someone else's invoices by email, this is the third time"
- "Cancel my order if it has not shipped, otherwise ignore this"
- "hi"
- "Store staff were great but the app deleted my loyalty points AND my last order shows twice on my statement"
Notice tickets 8, 9, and 10: conditional, empty, and genuinely two-category. Every real inbox has them. They exist to punish naive prompts.
Read the 10 raw tickets. Before writing anything, decide what categories even exist. We settle on BILLING, SHIPPING, RETURNS, and UNSURE for everything else - small on purpose.
Write success criteria FIRST. Ours: category accuracy 9/10 or better on the test set · summary 25 words or fewer · neutral tone even when the customer is furious · zero invented order numbers or amounts. Write yours down before touching the prompt.
Draft v0, the naive one. Run the bad box above on tickets 2, 9, and 10. Watch it invent categories you never defined, guess on "hi", and pick one category for ticket 10 without telling you it dropped the other.
Write v0.1 with criteria baked in. Role, enumerated categories, output format, and the UNSURE escape hatch. Run the same three tickets and compare against your written criteria - not against your mood.
Save both versions and the criteria. A file, a gist, anywhere durable. In session 8 those criteria become a real eval, and v0 becomes your baseline to beat on a scoreboard.
Three things before session 2 ◐ 25 min total
- Label the full test set. Assign your own expected category to all 10 tickets and note the two you found hardest to label - if a human hesitates, the criteria need a rule for it.
- Run v0.1 on all 10 tickets in your chat UI or API of choice. Score it against your criteria: how many categories correct? Any invented details? Bring the number.
- Classify your models. List the 2-3 models you use most at work and decide which side of the Part 3 decision tree each sits on. If you are not sure, check the vendor's docs for a thinking or reasoning mode - that is your answer.
What this session covers
Builder-level coverage of the sources below. Official certificates, videos, and quizzes stay with the vendors - links live in the course map.
Three questions before you go 🎯 ◐ 90 seconds
1 · Anthropic's official working order for prompt development is...
The funnel runs criteria → evals → draft → engineer. Their Skilljar course even teaches evals before techniques - the ordering is the method, not an accident.
2 · You are prompting a reasoning model. Which habit should you drop?
Reasoning models plan internally: induced CoT is redundant-to-harmful, and few-shot examples can measurably hurt. Go zero-shot first and steer depth with effort or reasoning parameters.
3 · Context engineering is best described as...
It is the wider discipline of managing the whole token ecosystem. The prompt remains the one layer you fully control - which is exactly why this course exists inside that bigger frame.