From understanding to using
Most people use GenAI at 10% of its value because they treat it like a search box. This session is the craft layer: how to write a prompt that gets useful output every time, how to ground the model in your company's real data so it stops guessing, when to reach for the expensive option (fine-tuning) versus the cheap ones, and how to prove a GenAI feature is good enough to ship. Leaders leave able to judge a GenAI project; practitioners leave able to shape one.
Prompting is specification 5 min live
"Summarize this" and a full spec get wildly different results from the same model. A good prompt tells the model who to be, what to do, what constraints to respect, and what format to return. It is not a secret incantation - it is a clear brief, the kind you would give a sharp new hire. Give examples (few-shot) and you steer even harder.
LiveWrite the brief, not the wish3 min▶
- Role sets the voice and expertise. "You are a pediatric nurse explaining to a worried parent" changes tone, vocabulary, and what gets emphasized - for free.
- Constraints do the heavy lifting. Length, audience, tone, format, and things to avoid. Vague prompts get vague output because you left every choice to the model.
- Examples beat adjectives. One or two worked examples (few-shot) steer harder than any pile of describing words. Show the shape you want.
- Iterate. First output is a draft of your prompt, not just the answer. Tighten the brief and rerun - that loop is the skill.
Teams that write a shared prompt library - a vetted, reusable prompt per common task - get consistent output across everyone, instead of each person reinventing "summarize this" badly. The prompt becomes an asset, like a document template. That is the difference between dabbling and operating.
Self-studySystem vs user prompts, and prompts as assets2 min read▶
In real applications there are two layers: the system prompt (set once by the builder - the persistent role, rules, and guardrails) and the user prompt (what the end user types each time). The system prompt is where you bake in "always refuse X", "answer only from provided context", "use this format" - so end users get safe, consistent behavior without having to prompt well themselves. Practitioner takeaway: treat prompts as versioned assets. Store them, test them (evals, Part 3), and change them deliberately - a prompt edit is a product change, and an untested one can quietly break output for everyone.
Prompt, RAG, or fine-tune? 5 min live
Three ways to make a general model fit your specific need, from cheapest to most expensive. Most teams reach for the expensive one first and waste a quarter. The rule is simple: prompt first, add RAG when it needs your facts, fine-tune only for consistent style or format at scale - and rarely.
LiveThe three tools, and when each earns its cost3 min▶
- Prompting - free, instant, no infrastructure. Your first resort for nearly everything. Most "we need a custom model" problems are actually "we need a better prompt".
- RAG (Session 5) - retrieve your own documents and feed them into the context so the model answers from your facts, not its training. The enterprise default for "answer from our knowledge". Grounding beats guessing.
- Fine-tuning - actually retrain the model on your examples. Expensive, slow, and worth it mainly for a consistent style or output format at high volume (not for teaching it facts - RAG does that better and stays current).
- The trap: jumping to fine-tuning for a facts problem. Facts change; a fine-tuned model goes stale, while RAG just retrieves the latest. Match the tool to the need.
Self-studyCost and latency literacy2 min read▶
Two numbers govern any GenAI feature in production. Cost: you pay per token (in and out), so a chatty prompt or a huge retrieved context costs real money at scale - a feature that is fine in a demo can be ruinous at a million calls a day. Latency: bigger models are slower; a user waiting 8 seconds for a reply may churn. The practitioner move is to match model size to task - a small fast model for routing and simple replies, a large one only for the genuinely hard step - and to trim context to what is needed. "Which model?" is a cost/latency/quality tradeoff, not "always the biggest".
Agents, tools, and knowing it works 5 min live
An agent is an LLM that can call tools (search, a calculator, your APIs) and take several steps toward a goal instead of answering in one shot. Powerful - and it multiplies the failure modes, so it needs guardrails. And underneath all of it sits the discipline nobody demos but everybody needs: evals. You cannot ship what you cannot measure.
LiveAgents add power and risk; evals add trust3 min▶
- Agents can do multi-step work - search, then read, then draft, then send - by calling tools between reasoning steps. This is how GenAI reaches beyond one reply into real tasks.
- They also fail in new ways: looping without finishing, calling the wrong tool, or compounding an early mistake across steps. Give them limited tools, step caps, and human approval on anything irreversible.
- Evals are how you know it works. A set of test cases with known-good answers, run every time you change the prompt or model, plus human review on a sample. "It looked great in the demo" is not an eval.
- Measure before and after every change. A prompt tweak that helps one case can quietly break ten others. Without an eval set you are flying blind.
The teams shipping reliable GenAI features are not the ones with the cleverest prompts - they are the ones with an eval set of 50-200 real cases they run on every change, tracking a quality score over time. It is the unglamorous habit (straight out of the data-literacy lifecycle) that turns a flashy demo into a dependable product.
Self-studyWhat an eval actually looks like2 min read▶
An eval is a small, curated test set: real inputs paired with what a good output looks like (or a rule for judging one). You run the whole set on every prompt or model change and get a score - percent correct, or a rubric rating from a human or a second model acting as judge. For a support-reply bot, that might be 100 real tickets with model answers rated on accuracy, tone, and whether they leaked anything they should not. The point is a number that moves when quality moves, so decisions ("ship the new prompt?", "switch models?") stop being vibes. This is the same act as train/test from the data-literacy course, applied to GenAI.
Weak prompt, strong prompt ★ 10 min · everyone compares
Same model, same task - only the prompt changes. Flip between the weak and strong version and watch the output transform. Then rewrite one of your own real prompts as a spec.
For each task, read the weak output, then flip to the strong prompt and its output. Name what the strong prompt added (role, length, audience, constraints, banned words, format).
Extract the pattern into your own words - the five parts from Part 1.
Take a prompt you actually used this week and rewrite it as a spec. Share one before/after.
Pick the approach ★ 6 min · everyone decides
Four scenarios: (a) answer employee questions from our internal wiki, (b) always write in our exact brand voice, (c) a one-off summary of a report, (d) analyze this quarter's numbers.
For each, choose prompt / RAG / fine-tune and justify with the Part 2 rule (current-or-private facts -> RAG; consistent style at scale -> fine-tune; else -> prompt).
Expected: (a) RAG, (b) fine-tune candidate, (c) prompt, (d) prompt with the numbers pasted in (grounding). Debate the close calls.
Note which of your own real use cases were secretly "just prompt it better" all along.
Design an eval ★ 6 min · everyone specs
Pick one GenAI use case from your world (a drafting bot, a summarizer, a classifier).
Define its eval: what test cases (how many, drawn from where), who or what judges them, and what bar it must clear to ship.
Name the failure you most need to catch (wrong facts? wrong tone? leaked data?) and make sure a test case covers it.
Write one sentence: "We ship this when ___ % of ___ cases pass ___ review." That sentence is the difference between a demo and a product.
"It seemed fine when I tried it" is how bad GenAI features reach customers. A 50-case eval run before every change catches the regressions your spot-check never will - the same act as testing any other software, just newer.
This week ◐ 25 min total
- Rewrite three real prompts you use as full specs (role, task, constraints, format). Keep the good ones - you are starting a prompt library.
- Run the approach test on one AI idea from your Session 1 list: is it really prompt, RAG, or fine-tune? Be honest about defaulting to the cheapest that works.
- Read the self-study cards above: system vs user prompts, cost/latency literacy, and what an eval looks like.
- Optional deep end: sketch a 10-case eval for one use case - real inputs plus what a good answer looks like. You now have the seed of a real quality process.
Three questions before you go 🎯 ◐ 90 seconds
1 · What makes a prompt strong?
A prompt is a brief, not an incantation. Naming the role, constraints and format does the work; examples steer hardest. Length for its own sake does not help.
2 · You need an assistant that answers from your company's current internal docs. Best approach?
Facts that change belong in RAG, not fine-tuning. Retrieval always uses the latest docs; a fine-tuned model goes stale the moment a doc updates.
3 · What is an eval and why do you need one?
Evals turn "seemed fine" into a measured score that catches regressions when you tweak a prompt or swap a model. You cannot ship what you cannot measure.
Concepts covered & where they come from
Leaders get the judgment; practitioners get the Self-study lane and the patterns to apply Monday.