learn-ai-agents-with-phoebe / Builder session 5 of 10
Learn AI Agents with Phoebe · Builder track · Session 5 of 10

Planning and reasoning

Ask an unplanned agent to diagnose the March dip and it fetches one number, wraps it in confident prose, and stops. A number is not a diagnosis. Tonight you learn why one-shot answers happen - the model optimizes for "answered something", not "answered everything" - and build the fix: plans the agent writes down before it acts. ReAct, plan-then-execute, reflection, and the decomposition heuristics that turn a vague ops question into four crisp tool calls and a real synthesis. Rung 3 of the ladder falls tonight.

🟡 Medium Builder track Python · Claude API + Ollama path 45 min
0-3 · Welcome 3-18 · Why one shot fails 18-42 · Build-along: plans that hold 42-45 · Q&A
Part 1 · covers Anthropic transparency principle, Ng M2, HF Unit 1 (T-A-O)

A number is not a diagnosis 7 min live

The planning-off signature, straight from the scorecard: one tool call, a partial answer, and a stop. Why does a capable model do this? Because without an explicit plan, nothing in the loop rewards completeness - the model reaches the first answer-shaped thing and declares victory. It answered something. Anthropic's second agent principle is the antidote: transparency - show the agent's planning steps explicitly. A written plan is both a completeness contract and a debugging trail.

Same task: "diagnose the March dip" ONE-SHOT lane TASK why the dip? 1 call query_sales("2026-03") "March was $150." a number, then it stops ✗ shallow no cause found PLANNED lane TASK why the dip? PLAN first 4 written steps 1 · Feb: $271 2 · Mar: $150 3 · refunds: 1 4 · subs: 1 cancel SYNTHESIS -44.6%, causes named ✓ diagnosis complete + checkable Same model, same tools. The only difference: a plan written before acting - and a synthesis step it must reach.
🔍 Click to zoom - one task, two lanes: one-shot stops at a number; the planned lane decomposes, executes four calls, and synthesizes
LiveWhy "answered something" wins by default4 min▶

Nothing in a bare loop forces completeness. The model sees a question, sees a tool, makes the obvious call, and the result LOOKS like an answer - so it answers. Three forces push toward the shallow exit:

  • No completeness contract - "diagnose" quietly requires compare, decompose, and explain; the bare prompt never spells that out, so one number satisfies it.
  • Every extra step has a cost - more calls mean more latency and more chances to err; stopping early is locally safe and globally wrong.
  • No visible trail - with no written plan, neither the agent nor you can see which sub-questions were never asked. Anthropic's transparency principle exists precisely to make the gap visible.
Real world

The board slide with one query behind it. A revenue "analysis" agent shipped weekly summaries an exec loved - until someone audited the trace and found every report was a single aggregate query plus adjectives. The fix was not a bigger model. It was requiring an explicit plan with a compare step and a cause step - and rejecting any run whose trace skipped one.

Self-studyReasoning is not planning3 min read▶

Two words that get blurred:

  • Reasoning is the model thinking inside one turn - b1's Thought in the T-A-O cycle. Every capable model does some of it unprompted.
  • Planning is structure ACROSS turns - a declared set of steps the loop then executes and checks off. It lives in your code and prompts, not just in the model's head.

That is why planning is a lever you engineer, not a capability you hope for: the same model that one-shots a diagnosis will follow a four-step plan faithfully once the loop demands one.

Part 2 · covers Ng M2+M5 (reflection, planning), HF Unit 1, S1 patterns recap

The planning toolkit 8 min live

Four techniques, from lightest to heaviest. They compose: a plan-then-execute agent can run ReAct inside each step and reflect on the final draft.

LiveReAct / T-A-O - reason before every action3 min▶

ReAct is b1's Thought → Action → Observation cycle made explicit: the agent must WRITE a thought before every tool call - "I have March, now I need February to compare" - instead of silently jumping to the next action. The lightest planning there is, and it is already in your b2 loop if your system prompt demands a reasoning line before each call. The payoff is the trail: when a run goes wrong, the written thoughts show you exactly where it stopped thinking.

LivePlan-then-execute - the workhorse4 min▶

Ask for the whole plan up front - a numbered list of steps - then execute the steps one by one, re-planning only when a result surprises you. The plan is the completeness contract: the run is not done until every step is. This is tonight's build (code below), and it is the technique that unlocks rung 3.

LiveReflection - generate, critique, improve3 min▶

Ng's Module 2 pattern: the agent drafts an answer, then critiques its own draft against the task ("does this name a CAUSE for the dip, or just restate the number?"), then revises. You already met its inner move in b3 - reflection is exactly what the evaluator-optimizer workflow does, with the evaluator turned inward. Cheap to add: one extra model call after the synthesis step.

Self-studyDecomposition heuristics - what makes a good subtask3 min read▶

Plans fail at the seams, so cut carefully:

  • A good subtask = one tool call or one judgment. "Get February revenue" is one call. "Explain the gap" is one judgment. "Investigate revenue" is neither - split it.
  • Each step should produce a checkable artifact - a number, a list, a yes/no - so the loop can tell done from not-done.
  • End with an explicit synthesis step that consumes the artifacts. The one-shot failure is really a missing synthesis step.
  • Prefer 3-6 steps. Ten-step plans go stale by step four; re-planning on surprise beats over-planning up front.
LiveBuild it: plan-then-execute in ~25 lines6 min▶

Two phases. The first messages.create call asks ONLY for a JSON plan - no tools, no answers. Then a loop feeds each step to your b2 agent loop, appending results as it goes, and closes with a synthesis:

A good plan step passes two checks One call or one judgment? Checkable artifact? yes no no yes Split it further Add a checkable result Keep as a plan step Ten-step plans go stale by step four - prefer three to six checkable steps, then synthesize.
🔍 Click to zoom - the one-shot failure is really a missing synthesis step
import json
from anthropic import Anthropic

client = Anthropic()

def plan_then_execute(goal, tools):
    # Phase 1: plan only - no tools, no answering
    plan = client.messages.create(
        model="claude-opus-5",
        max_tokens=800,
        system=("Return ONLY a JSON plan for the task: "
                '{"steps": ["..."]}. Each step must be one '
                "tool call or one judgment. End with a synthesis step."),
        messages=[{"role": "user", "content": goal}],
    )
    steps = json.loads(plan.content[0].text)["steps"]

    # Phase 2: execute each step through the b2 loop
    messages = [{"role": "user", "content": goal}]
    for i, step in enumerate(steps, 1):
        messages.append({"role": "user",
            "content": f"Plan step {i}/{len(steps)}: {step}"})
        result = run_loop(messages, tools)  # your b2 loop, unchanged
        messages.append({"role": "assistant", "content": result})

    messages.append({"role": "user",
        "content": "All steps done. Synthesize the final answer "
                   "using only the results above."})
    return run_loop(messages, tools)

Re-planning on surprise is one more branch: if a step's result contradicts the plan's assumption, feed the plan plus the surprise back to phase 1 and ask for a revised remainder. Print the plan before executing - that single print IS Anthropic's transparency principle in practice.

Demo 1 of 2

Watch planning turn a number into a diagnosis ★ 10 min · everyone runs it

Same founder question as b4: "Diagnose the March dip." This time tools and memory are on but planning is OFF. Watch the one-call shallow answer land - then toggle planning on and watch the same agent write a four-step decomposition and follow it to a synthesis with real causes.

Step through with planning off: one query_sales("2026-03") call, a fluent sentence around $150, stop. A number, not a diagnosis.

Toggle Planning on. First new thing in the trace: a written plan, before any tool runs.

Follow the four steps land their artifacts: Feb $271, Mar $150, the March refund, the subscription cancellation.

Read the synthesis: $271 → $150, -44.6%, with the contributing causes named. Every claim traces back to a step above it.

Diagnose by signature, as always One call + partial answer + stop = planning off. Invented numbers = tools off. Contradicting its own earlier finding = memory off (b4). The trace tells you which lever to reach for before you touch the prompt.
Demo 2 of 2 · rung moment

Rung 3 on the ladder: 7/12 becomes 10/12 ★ 6 min · explore

Turn planning on alongside tools and memory and the scorecard jumps to exactly 10/12 - rung 3 of the ladder (1 → 5 → 7 → 10 → 12), the biggest single-lever jump in the course. Read which three tasks flipped, and which two are still red.

Four levers, one ladder: 1 → 5 → 7 → 10 of 12 tasks No levers (b1) 1/12 + tools (b2) 5/12 + memory (b4) 7/12 + planning (b5) 10/12 Rung 3: planning is the biggest single-lever jump in the course - 7/12 to 10/12.
🔍 Click to zoom - planning is the biggest single jump yet: 7/12 to 10/12

Check the score reads 10/12 with tools + memory + planning on. Three tasks flipped versus b4's rung 2.

The three unlocked tasks are the decomposition-shaped ones: the diagnosis, the report, and the join-shaped task that needs results from multiple tools combined.

Read the failure text on the two still red: they touch the outside world - sending email, reading untrusted tickets. More planning will never fix them.

Name what is missing out loud: guardrails. That is b7, and it is the dangerous rung - the levers so far only made the agent smarter, not safer.

★ What you walk out with today The planning-off signature (one call, partial answer, stop) and why it happens - the model optimizes for "answered something". Four techniques you can rank by weight: ReAct thoughts, plan-then-execute, reflection, and the subtask heuristic (one tool call or one judgment). A working ~25-line plan-then-execute loop on top of b2. And rung 3, banked: 10/12 - with a clear-eyed view of the two tasks only guardrails can save.
Homework

Try it yourself - this week ◐ 25-35 min total

Source material

Official sources covered

This session teaches the planning and reasoning core shared by the canonical agent sources. Certificates, videos, and graded assessments stay on the official sites (linked each session). This page covers:

✓Anthropic "Building Effective Agents" - transparency principlePrinciple 2: show the agent's planning steps explicitly · practiced in every trace tonight
✓DeepLearning.AI "Agentic AI" (Ng) - Modules 2 + 5Reflection pattern · planning workflows · built here on Daybreak
✓Hugging Face Agents course - Unit 1 (T-A-O)Thought-Action-Observation as the lightest planning form (ReAct)
◐OpenAI "A Practical Guide to Building Agents" - single-agent maximization"Maximize a single agent's capabilities first" - planning before you reach for multi-agent (b6)
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · The agent makes exactly one tool call, gives a fluent partial answer, and stops. Which lever was off?

One call + partial answer + stop is the planning-off signature. The tool call worked (so tools were on) and nothing was forgotten (the task simply was never decomposed). A written plan is the completeness contract that forces the remaining steps.

2 · What is the real difference between plan-then-execute and ReAct?

Both reason and both use tools. The difference is when structure appears: ReAct plans locally (one thought per action), plan-then-execute plans globally (all steps declared first, re-planning on surprise). They compose - ReAct can run inside each plan step.

3 · When does turning planning ON make an agent worse?

"What was March revenue?" needs one call; a planner turns it into several. Planning pays on decomposition-shaped tasks - diagnoses, reports, joins - and taxes simple ones. Planning is a lever, not a religion: pick per task.

Builder session 5 cheat sheet · pin this

Planning-off signatureOne tool call, partial answer, stop. The model optimized for "answered something" - a number is not a diagnosis.
The principleAnthropic principle 2: transparency - show the agent's planning steps explicitly. A printed plan is a completeness contract plus a debug trail.
ReAct / T-A-OReason before every action - b1's cycle made explicit. Lightest planning; lives in the system prompt.
Plan-then-executeFirst call emits ONLY a JSON plan; a loop runs each step through the b2 loop; re-plan on surprise; synthesize at the end.
ReflectionGenerate → critique → improve (Ng M2) - the evaluator-optimizer's inner move, turned on the agent's own draft.
Good subtaskOne tool call or one judgment, producing a checkable artifact. 3-6 steps; always end with an explicit synthesis step.
Rung 3Tools + memory + planning = 10/12. Unlocked: diagnosis, report, and the join-shaped task. Still red: the two that act on the world.
When to skip itTrivial lookups get slower and costlier with a planner. Planning is a lever, not a religion - choose it per task.