Learn AI Red Team with Phoebe

Three thousand hours found nothing. That was not the same as nothing being there.

A bug bounty ran 183 participants over three thousand hours against a new safety system and found no universal jailbreak. A later public exercise with 339 people and a few hundred more hours found one. Nothing about the first result was wrong; it simply did not mean what "we red-teamed it and found nothing" is usually taken to mean. This is a course about testing AI systems you own, honestly, and about what you may claim afterwards. It covers scoping and rules of engagement, the attack classes named by OWASP 2026, MITRE ATLAS and NIST, the defences that hold and the measured evidence that most of them do not, how to score a finding when the same input gives a different answer every time, and how findings become a standing suite. At its centre, a live attack-surface board where an untested class counts at full risk rather than zero, because a coverage meter that treats "we did not look" as "we are fine" is worse than no meter at all. Every exercise runs against a fictional company, inside a scope document you write in the first session.

2tracks
16sessions
8attack classes
7controls compared
Authorized testing only, and where this sits Everything here is about systems you own or are contracted in writing to test. The course names attack classes so you can build test cases and pairs each with its defence; it contains no exploit payloads. Agent Launch has a safety gate that requires an adversarial slice and does not teach you to build one - that is this course. AI Observability owns the telemetry findings surface in. AI Agents owns what an agent is.

The leader track 🎯 · for the people who commission and read the report · 6 x 45 min · no tools required

Judgement mode: what a red team can and cannot establish, the threat model in two heuristics, how to read a report without being fooled by a number, how to score severity by what a finding chains to, which defences the measured evidence actually supports, and when the right answer is that this workflow should not ship as an agent at all.

🧭 Leader 1 · easy

What a red team is for

Three thousand hours found no universal jailbreak. A later exercise found one. What a report may claim, and the four sentences it may not.

▶ Leaders start here 🕸️ Leader 2 · easy

The threat model

Two heuristics do most of the work: the lethal trifecta, and the rule of two. Remove any one leg and the risk goes with it.

The map 🔬 Leader 3 · medium

Reading a red-team report

A near-zero attack success rate is now the most dangerous number in the field. Twelve defences reported one; most were broken above ninety percent.

The grilling kit ⚖️ Leader 4 · medium

Severity and triage

The same jailbreak is informational in one system and critical in another. Severity is what it chains to, not how clever the prompt was.

What it chains to 🛡️ Leader 5 · deciding

What defences actually hold

Every individual defence was beaten by a third to sixty percent of teams that tried. Stacking cut it to sixteen percent, not to zero.

The evidence 🛑 Leader 6 · deciding

When to stop

The regulator says it plainly: if the residual risk is intolerable, it may not be a good use case for an LLM. That is a finding.

The hard call

The builder track 🛠️ · for security engineers, AI engineers & platform leads · 10 x 45 min · hands-on

Craft mode: write the rules of engagement, map the attack surface to the current framework IDs, work through injection, refusal, extraction and tool abuse as classes to test rather than tricks to perform, build the suite, run the engagement, harden and honestly measure the delta, and leave behind a standing suite that keeps holding after you go.

📋 Builder 1 · easy

Scope and rules of engagement

Authorization is not a sentence you type. A real campaign bypassed guardrails by claiming to be authorized testing.

▶ Builders start here 🗺️ Builder 2 · easy

The attack surface

The eight classes and the frameworks that name them. Four OWASP IDs moved in the 2026 edition, and one was renamed entirely.

The taxonomy 💉 Builder 3 · medium

Prompt injection

The class with no parameterised query. Three official bodies now say plainly that no reliable prevention mechanism exists.

The unsolved one 🔓 Builder 4 · medium

Jailbreaks and refusal

Measured as a rate, never pass or fail. And the benign set that stops you scoring well by refusing everything.

Rate not verdict 🗄️ Builder 5 · medium

Data extraction

Assume the context is discoverable, then design so that discovering it buys an attacker nothing.

Assume discovered 🔧 Builder 6 · hands-on

Tool abuse and agency

Severity is what the credential can do. Most documented MCP threats turn out to be OAuth, SSRF and supply chain problems.

Blast radius 🎯 Builder 7 · hands-on

Build the suite

The live board. Probe the whole surface and the total barely moves while the unknown goes to zero. That is the job.

★ The live board 🏃 Builder 8 · hands-on

Run the engagement

Open-ended first, then guided, then adaptive. Baselines got eleven percent; attacks tailored to the target got eighty-one.

The two stages 🔨 Builder 9 · hardest

Harden and retest

Prompt-level controls reduce rates; architecture removes capabilities. The gap between them is fifty-five points.

The delta ♻️ Builder 10 · hardest

The standing suite

Green is not safe and never was. Regression-test the architecture, not only the prompts, and re-test on triggers.

After you go
easy - everyone medium hands-on / deciding hardest

Choose your path 🗺️

Four doors into the same conviction: you cannot make the model refuse to be fooled, so bound what being fooled gets an attacker.

🎯 I commission the testing a1→ a2→ a3→ a4→ a5→ a6
🛠️ I run the engagement b1→ b2→ b3→ b4→ b5→ b6→ b7→ b8→ b9→ b10
⚡ We ship an agent next month a2→ b6→ b7
🚀 Full journey (test AND decide) a1-a6→ b1-b10

The knowledge map 🧠

Both tracks at a glance - hover a session to spotlight its concepts, click any node to jump in.