learn-ai-agents-with-phoebe / Builder session 9 of 10
Learn AI Agents with Phoebe · Builder track · Session 9 of 10

Ship and observe

Your raw-Python agent works - now decide what to ship it ON. Tonight: the honest framework landscape (when raw Python stops being enough), tracing so you can see every loop turn in production, deployment and cost control, and where MCP ends and A2A begins.

🔴 Builder track Practitioners: DE · DS · AI engineers · builders Python · Claude API + Ollama path 45 min
0-3 · Welcome 3-18 · The framework landscape 18-42 · Build-along: trace it, cap it, ship it 42-45 · Q&A
Part 1 · covers Claude Agent SDK docs, OpenAI Agents SDK, LangGraph, DLAI short courses

The framework landscape 12 min live

Full honesty up front: this is a positioning lesson, not a build track. Framework deep-dives are deliberately out of this course's scope - each framework below gets a mental model and a pointer, so that when you open its quickstart you already know what shelf it sits on and what it would replace in the loop you built in b2. That loop, remember, is production-legitimate: Anthropic's own guidance is simple, composable patterns first.

multi-agent single agent code-first config-first LangGraph graphs + checkpoints AutoGen conversation patterns crewAI role-based crews Claude Agent SDK query() + subagents OpenAI Agents SDK handoffs + tracing raw Python your b2 loop Start bottom-left. Move up or right only when a named trigger fires: persistence, interrupts, or a fleet. Docs: code.claude.com · openai.github.io/openai-agents-python · docs.langchain.com/oss/python/langgraph
🔍 Click to zoom - the landscape map: code-first to config-first, single agent to multi-agent
LiveThe four shelves, one at a time6 min▶

Four serious options above raw Python, each with a distinct mental model:

  • Claude Agent SDK - Python and TypeScript only, and it bundles the Claude Code binary: you inherit a full agent harness rather than assembling one. Its primitives are built-in tools, hooks, subagents, MCP, permissions, sessions, skills, and plugins. Entry point is query() plus ClaudeAgentOptions; permission modes include plan, acceptEdits, and bypassPermissions. It was renamed from "Claude Code SDK" in late 2025 - older tutorials use the old name for the same thing.
  • OpenAI Agents SDK - five primitives, and you have met most of them: Agents, Handoffs, Guardrails, Sessions, Tracing. The b6 shapes and b7 guardrails map 1:1; tracing is the new gift, covered in Part 2.
  • LangGraph - your agent as a StateGraph of nodes and edges over a MessagesState. What it buys you: checkpointing and persistence, human-in-the-loop interrupts, subgraphs, a long-term memory Store, and a deployment story. Docs now live at docs.langchain.com/oss/python/langgraph - older links are stale.
  • crewAI - you describe a crew of role-based agents (researcher, writer, reviewer) and the framework runs the collaboration. Fastest path from idea to multi-agent demo; the two b6 shapes are still what runs underneath.
  • AutoGen - agents as conversation participants; the DLAI "Agentic Design Patterns" course teaches the four-pattern canon (Reflection · Tool Use · Planning · Multi-Agent) in it. Strong for research-style back-and-forth between agents.
FrameworkMental modelStrongest whenWatch out
raw Pythona loop you own end to endone agent, clear tools, full controlyou build persistence + tracing yourself
Claude Agent SDKinherit the Claude Code harnessyou want permissions, sessions, subagents ready-madePython/TS only; harness has opinions
OpenAI Agents SDKfive primitives, guide-shapedhandoffs + built-in tracing matterprimitives mirror one vendor's guide
LangGraphagent as a state graphpersistence, interrupts, complex flowsgraph thinking is a real learning curve
crewAIrole-based crew, declaredquick multi-agent prototypesconfig hides the loop you learned to debug
AutoGenagents in conversationpattern experiments, research flowsconversations are hard to make deterministic
The decision rule Stay on raw Python until you need persistence (durable state across restarts), interrupts (pause mid-run for a human, resume later), or fleets (many specialists with managed handoffs). None of the three? Your b2 loop IS production-legitimate - simplicity is an official principle, not a beginner's crutch.
Self-studyWhy "which framework" is the wrong first question3 min read▶

Every framework above ships the same four levers under different names - tools, memory, planning, guardrails. You learned the levers on a loop you can read in one screen; a framework is just someone else's opinionated packaging of them. So the productive question is never "which framework is best" - it is "which lever do I need packaged that I do not want to maintain myself". Checkpointing is a memory lever. Interrupts are a guardrail lever. Handoffs are b6's decentralized shape. Read any quickstart with the lever map in hand and it stops being magic in about ten minutes - that is your homework tonight.

Part 2 · covers OpenAI SDK tracing, LangSmith trajectory traces

Tracing and observability ★ 12 min · everyone builds

The eval vocabulary from b8 carries straight over: in production, the transcript becomes the trace. Log every loop turn - the model call, each tool call with its arguments, each tool result, and every guardrail firing. An agent without a trace is an agent you can only debug by vibes.

LiveA JSONL trace writer in ~20 lines8 min▶

The OpenAI Agents SDK ships tracing built in - it is one of the five primitives. LangSmith does trajectory-level traces for LangGraph and friends. On raw Python, this wrapper around your b2 loop covers 80% of what those give you:

# trace.py - one JSONL line per loop step
import json, time, hashlib

def trace(kind: str, name: str = "", args=None, tokens: int = 0):
    line = {
        "ts": time.time(),
        "kind": kind,          # "model" | "tool_call" | "tool_result" | "guardrail"
        "name": name,          # e.g. query_sales
        "args_hash": hashlib.sha256(
            json.dumps(args or {}, sort_keys=True).encode()).hexdigest()[:12],
        "tokens": tokens,
    }
    with open("trace.jsonl", "a") as f:
        f.write(json.dumps(line) + "\n")

# inside the b2 loop, one call per step:
r = client.messages.create(model="claude-opus-5", max_tokens=2048,
                           tools=DAYBREAK_TOOLS, messages=messages)
trace("model", tokens=r.usage.output_tokens)
for b in r.content:
    if b.type == "tool_use":
        trace("tool_call", b.name, b.input)
        result = TOOLBOX[b.name](**b.input)
        trace("tool_result", b.name)

Hash the arguments instead of storing them raw - customer emails and ticket text do not belong in a log file, but you still want to know whether two runs made the same call. Token counts per step are your cost meter for Part 3.

Self-studyTrajectory traces, formalized3 min read▶

What you just wrote by hand is what LangSmith calls a trajectory: the ordered sequence of steps an agent took, not just its final answer. b8's trajectory match modes - strict, unordered, subset, superset - grade exactly this object. That is the payoff of using one vocabulary across b8 and b9: your golden set graded transcripts offline; production grades the same shape of data live. When a trace platform demo looks impressive, check it against your JSONL fields - ts, kind, name, args_hash, tokens - and ask what it adds beyond search and pretty rendering. (Often plenty. But now you can name it.)

Real world

The March dip, in production. Back in b3 you diagnosed why Daybreak's March revenue fell from $271 to $150. Now imagine the shipped agent gets that question and answers wrongly. Without a trace you cannot tell whether it was a bad plan (never queried February), a lost memory (queried it, then dropped the finding), or a failed tool call (query errored and the model improvised). Three different levers, three different fixes, one identical wrong answer. The trace is how you know WHICH lever slipped - the b1 habit, kept alive in production.

Part 3 · covers deployment options, Managed Agents, cost levers

Deploy and cost control ★ 12 min · everyone builds

Three places your agent can live. Your own server - the raw loop or an SDK app behind an endpoint you run; maximum control, you own uptime. LangGraph deployment - the graph platform hosts and scales your graphs, with persistence wired in. Anthropic Managed Agents - a hosted sandbox product: Anthropic runs the agent's environment for you. Same loop in all three; what changes is who carries the pager.

LiveSessions and the cost levers5 min▶

State first. In production, "memory" (b4) becomes a storage decision: server-side session state (the OpenAI SDK's Sessions primitive, or your own store keyed by conversation id) versus LangGraph-style checkpointing, where every step is durably saved and a run can resume - or be interrupted for approval - mid-task. If your b7 approval gates need a human who answers hours later, checkpointing is the shape you want.

Then cost. Three levers that do not touch quality where it matters:

  • Model tier per step - the planner runs on a big model (claude-opus-5 in our path); the tool-result summarizer runs on a small one. Most steps in a long run are small-model steps.
  • Prompt caching - the system prompt and tool schemas are identical every turn; cache them and stop paying full price to resend your own instructions.
  • Budgets from b7, promoted to config - max turns and spend ceilings stop being demo code and become deploy-time settings someone can audit.

Pin model versions. A silent model upgrade is a silent behavior change. Deploys name exact versions; upgrades go through the b8 eval gate like any other change.

Trace every turn. The JSONL writer (or SDK tracing) is on from the first production request. No trace, no deploy.

Budgets and stop conditions on. Max turns, spend ceiling, tool-error limit - the b7 rails, now in config, not in a notebook.

Draft-and-hold on outward tools. send_email and refund_order draft; a human releases. The default outward-action pattern from b7 ships as-is.

Eval gate in CI. The b8 golden set runs on every change - prompt, tool, model version - and a failing set blocks the deploy. 12/12 or it does not ship.

Rollback plan. Keep the previous prompt + model + config bundle one command away. Agents fail weird; reverting fast beats debugging live.

Part 4 · covers A2A Protocol short course, MCP positioning

The protocol map: MCP vs A2A 6 min live

Two protocols, two directions. MCP standardizes the model-to-tools-and-context connection - it is how your Daybreak server plugs into any MCP-speaking agent without custom glue. A2A standardizes agent-to-agent communication - cross-framework interop, so an agent built on Google's ADK can delegate to one on LangGraph, BeeAI, or the Microsoft Agent Framework. MCP points down at the toolbox; A2A points sideways at the peers.

YOUR AGENT the b2 loop, shipped PEER AGENT LangGraph / BeeAI PEER AGENT ADK / MS Agent Fwk A2A A2A Daybreak MCP server query_sales · tickets web search server tool email server draft-and-hold MCP MCP MCP MCP: one standard plug between a model and its tools/context. Your Daybreak server ships unchanged. A2A: agent-to-agent interop across frameworks (ADK, LangGraph, BeeAI, MS Agent Framework).
🔍 Click to zoom - MCP plugs down into the toolbox; A2A reaches sideways to peer agents
LiveWhat each protocol buys you3 min▶
  • MCP - write the Daybreak server once, and any MCP-speaking host can use it: your raw loop, Claude Code, an Agent SDK app. Tools stop being framework-locked. You have lived this since the MCP course; b2 made it your agent's toolbox.
  • A2A - the same idea one level up: a standard way for one agent to discover another's capabilities and delegate to it, regardless of which framework either was built on. b6's manager and handoff shapes, but across organizational and framework boundaries.
  • One sentence to keep: MCP is how an agent talks to its tools; A2A is how an agent talks to other agents.
Self-studyHonest scope note1 min read▶

A2A here is positioning only - implementing an A2A server is out of this course's scope, by design. If you want the hands-on version, the DLAI "A2A Protocol" short course (Google + IBM, ~1.5h) builds one. Take the lever map and the two-protocol sentence with you; they make that course a fast read.

Homework

Try it yourself - this week ◐ 30-40 min total

Source material

Official sources covered

Certificates, videos, and graded assessments stay with the official providers - and tonight's framework coverage is deliberately pointers, not builds. This page covers:

✓Claude Agent SDK docs (code.claude.com)Primitives, query() + ClaudeAgentOptions, permission modes, the late-2025 rename
✓OpenAI Agents SDK - five primitivesAgents, Handoffs, Guardrails, Sessions, Tracing - tracing taught hands-on here
✓LangGraph docs + Academy - positioning + deploymentStateGraph, checkpointing, interrupts, Store, deployment; graph builds stay official
✓DLAI short courses - A2A Protocol + crewAI + AutoGenA2A protocol split, role-based crews, conversation-pattern canon - as positioning
◐Anthropic platform docs - Managed AgentsHosted sandbox product as a deployment option; tool docs covered in b2
◐OpenAI "A Practical Guide to Building Agents"Guide-to-SDK mapping recap; the guide's core landed in b1/b6/b7
◐LangSmith trajectory evalsTrajectory-level traces in production; grading modes were b8's territory
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · When does raw Python stop being enough for a production agent?

The b2 loop is production-legitimate - simplicity is an official Anthropic principle. Frameworks earn their complexity when a named trigger fires: durable state, pause-and-resume approvals, or managed multi-agent coordination. Tool count is not a trigger.

2 · What must a production trace capture, at minimum?

The transcript becomes the trace. With only the final answer you cannot tell a bad plan from a lost memory from a failed tool call - three levers, one identical wrong output. Log the trajectory, then diagnose by lever.

3 · MCP vs A2A - which line is right?

They point in different directions and coexist: MCP plugs the agent into its toolbox (your Daybreak server), A2A lets agents on ADK, LangGraph, BeeAI, or the MS Agent Framework delegate to each other. Neither replaces the other.

Builder session 9 cheat sheet · pin this

Claude Agent SDKPython/TS, bundles the Claude Code binary. query() + ClaudeAgentOptions; subagents, hooks, MCP, permission modes (plan, acceptEdits, bypassPermissions).
OpenAI Agents SDKFive primitives: Agents, Handoffs, Guardrails, Sessions, Tracing. Maps 1:1 to the OpenAI guide.
LangGraphStateGraph nodes + edges over MessagesState. Buys checkpointing, interrupts, subgraphs, Store, deployment.
crewAI / AutoGencrewAI: declared role-based crews. AutoGen: agents as conversation participants. Same b6 shapes underneath.
The decision ruleRaw Python until you need persistence, interrupts, or fleets. The b2 loop is production-legitimate.
Trace lineOne JSONL line per step: ts · kind (model / tool_call / tool_result / guardrail) · name · args_hash · tokens.
Ship checklistPin model versions · trace every turn · budgets + stop conditions · draft-and-hold outward · b8 eval gate in CI · rollback plan.
MCP vs A2AMCP: agent to its tools/context (Daybreak server). A2A: agent to other agents, across frameworks. They coexist.