learn-ai-red-team-with-phoebe / Builder session 2 of 10
Learn AI Red Team with Phoebe · Builder track · Session 2 of 10

The attack surface: the eight classes, and the frameworks that name them

This is the taxonomy session, and it starts with a version warning rather than a definition, because the list most people quote moved under them. Four OWASP IDs changed number between the 2025 and 2026 editions and one was renamed outright. By the end you will have the 2026 list, the agentic list that sits beside it, the MITRE ATLAS technique IDs with their evidence grades, and the NIST objectives - mapped onto Beacon's three bots as a test plan rather than a poster.

🟢 Builder track Security engineers · AI engineers · platform leads 45 min
0-3 · Welcome 3-14 · The 2026 list and what moved 14-28 · Agentic, ATLAS, NIST 28-45 · Build-along and the matrix
Part 0

Why a taxonomy session is not a poster session

⚠ Authorization This course is about testing AI systems you own, or are contracted in writing to test. Every exercise runs against Beacon, a fictional company, inside the scope document you wrote in b1. This page names attack classes so you can build test cases, and pairs each with the defence that addresses it. It contains no exploit payloads and teaches no attack craft. Testing a system you are not authorized to test is a crime in most jurisdictions.

A taxonomy earns its place in an engagement in exactly one way: it is the completeness check you run after open-ended exploration, so that the report can say which surfaces were explored and which were not. That is the fourth of the four things a report may claim, and it is the only one a taxonomy can supply. Everything on this page exists to make that fourth claim honest.

⚠ Read this before you quote a framework ID at anyone The OWASP Top 10 for LLM Applications on this page is the 2026 edition, and four IDs moved. Excessive Agency went LLM06 to LLM03:2026, up three. Unbounded Consumption went LLM10 to LLM06:2026, up four. Improper Output Handling went LLM05 to LLM10:2026, down five, the furthest fall on the list. And System Prompt Leakage was renamed and re-scoped to LLM08:2026 Hidden Context Exposure. If a blog, a slide or a vendor datasheet shows Improper Output Handling at LLM05, it is teaching the 2025 list. That single ID is the fastest version check available to you.
Live - presented in session Self-study - read after class Official sources covered
★ What you walk out with today The 2026 OWASP list with the four moves marked, the boundary rule that decides when risk leaves the LLM list for the agentic one, the ATLAS technique IDs for Beacon's surface with their maturity grades and their mitigation gaps, the NIST objective each class violates, and a test-case schema that carries all four framework references on every row so a finding can be traced back to a named risk.
Part 1 · the list everyone quotes

OWASP Top 10 for LLM Applications, 2026 edition 8 min live

Here is the whole list, with where each item sat in the previous edition. Read the right-hand column first. Four rows in it are the reason this session exists.

ID (2026)TitleWhere it was in 2025
LLM01:2026Prompt InjectionLLM01 - steady at the top
LLM02:2026Sensitive Information DisclosureLLM02 - steady
LLM03:2026Excessive AgencyLLM06 - up 3
LLM04:2026Supply ChainLLM03
LLM05:2026Data and Model PoisoningLLM04
LLM06:2026Unbounded ConsumptionLLM10 - up 4
LLM07:2026MisinformationLLM09
LLM08:2026Hidden Context ExposureLLM07 System Prompt Leakage - renamed and re-scoped
LLM09:2026Vector and Embedding WeaknessesLLM08
LLM10:2026Improper Output HandlingLLM05 - down 5, the furthest fall
Four IDs moved between the 2025 and 2026 editions. These are the four. 2025 edition 2026 edition What changed LLM06 Excessive Agency LLM03:2026 Excessive Agency Up 3 places Agents moved it up the list LLM10 Unbounded Consumption LLM06:2026 Unbounded Consumption Up 4 places The largest single jump LLM05 Improper Output Handling LLM10:2026 Improper Output Handling Down 5 places The furthest fall on the list LLM07 System Prompt Leakage LLM08:2026 Hidden Context Exposure Renamed and re-scoped Not just the system prompt now The version check that takes two seconds If a page shows Improper Output Handling at LLM05, it is teaching the 2025 list. Check that one ID before you trust anything else on the page, including the mitigations.
🔍 Click to zoom - the four moves, and the one ID that dates any document instantly

How the 2026 ranking was made, and why it is the best teaching moment on this page

The ranking is 75% community vote and 25% incident data, and the incident half was drawn from 7,714 real incidents, of which 6,639 were classifiable. That is a serious evidence base by the standards of this field. Now the part worth stopping on.

The number that should change how you read numbers

Ranked by raw incident count alone, Prompt Injection falls out of the top 10 entirely. OWASP kept it at number one anyway, and published its reasoning: the absence is read as a defence effect. Mature teams spend real money holding injection off, so comparatively few clean exploits reach public incident databases. Meanwhile Misinformation was voted near the bottom by the community but sits near the top of the incident record - the opposite failure, a risk practitioners underrate and reality does not.

The lesson generalises past OWASP and into your own dashboard: an incident count measures what got through AND what got reported. It does not measure what is dangerous. The same distortion is why a quiet quarter in your own incident tracker is not evidence that a class is under control, and why "we have never seen this" is not a finding.

OWASP's stated design philosophy, which is also this course's line Stop trying to build a model that cannot be fooled. Build the surrounding system so that when the model is fooled, nothing important breaks. Every defence you will evaluate in this course gets graded against that sentence. Controls that ask the model to behave better score badly. Controls that remove a capability score well. You will watch that play out numerically in the matrix at the end of this page.
Part 2 · the second list

OWASP Top 10 for Agentic Applications, and the boundary rule 6 min live

There is a second list, released in December 2025, running ASI01 to ASI10. It is not a replacement and not a subset. The two lists divide the ground by a rule worth memorising, because it decides which taxonomy your finding belongs in and therefore who reads it.

IDTitle
ASI01Agent behaviour / goal hijack title unconfirmed
ASI02Tool Misuse and Exploitation
ASI03Identity and Privilege Abuse
ASI04Agentic Supply Chain
ASI05Unexpected Code Execution
ASI06Memory and Context Poisoning
ASI07Insecure Inter-Agent Communication
ASI08Cascading Failures
ASI09Human-Agent Trust Exploitation
ASI10Rogue Agents
⚠ ASI01's exact title is unconfirmed, and this course will not pretend otherwise OWASP's own repository file is named ASI01_Agent_Behaviour_Hijack.md and the December announcement says "Agent Behavior Hijacking", while secondary coverage of the final release says "Agent Goal Hijack". Until the released PDF is checked, write it as "ASI01 (agent behaviour / goal hijack)". This is a small thing and it is also the whole discipline: a course that guesses one title will guess a number later. The ASI02 to ASI10 titles above come from draft filenames plus agreeing secondary summaries, so treat them as strong but not primary.
The boundary rule: which list owns the risk The model is a component inside your app OWASP Top 10 for LLM Applications It classifies text in and text out. The model answers, retrieves, summarises, renders. The consequence stops at the edge of the response: a bad answer is a bad answer, not an action. The model is an actor: tools and memory OWASP Top 10 for Agentic Applications It acts. Tools fire, memory persists across sessions, one agent's output is another's input. The consequence leaves the response and lands in a system you own. The three things that move a risk across the line Tools with real effects Memory outliving a session Downstream consequences Neither list covers the ground alone. Beacon's Support Bot needs both: it answers and it issues refunds.
🔍 Click to zoom - component or actor, and why Beacon's Support Bot is filed under both

Applied to Beacon: the Analyst Bot writing SQL that a warehouse executes is squarely an LLM-list problem right up until you notice the query runs with a credential, at which point ASI03 Identity and Privilege Abuse joins LLM10:2026 Improper Output Handling on the same finding. The Support Bot that can issue a refund credit is an actor by any reading, so its worst findings will carry ASI02 alongside LLM03:2026. The Insight Bot caches what it reads, which is ASI06 Memory and Context Poisoning territory even though nothing it does looks agentic in a demo.

Part 3 · the technique-level map

MITRE ATLAS, and the two fields that turn it into a test plan 8 min live

OWASP gives you categories. ATLAS gives you techniques with IDs you can put in a ticket, tactics that say where in an operation a technique sits, and - new in the version 6 schema - an evidence grade that tells you what to test first.

MITRE ATLAS dataset 2026.07, format 6.0.0. Cite the version whenever you cite a count. 16 tactics · 178 techniques (101 top-level + 77 sub) · 37 mitigations · 68 case studies. maturity - a ready-made test-prioritisation axis Realized 68 of 178 test these first Demonstrated 92 of 178 watch list Feasible 18 of 178 backlog platforms - Agentic AI is now the largest tag in ATLAS Agentic AI 119 Generative AI 92 Predictive AI 70 Enterprise 61 Counts read from the machine-readable dataset, not the rendered site. See the caution note below.
🔍 Click to zoom - evidence grade as a prioritisation axis, and the platform tag that overtook the rest
⚠ Cite the dataset version whenever you cite a count A second research pass for this course could not load the rendered ATLAS site and found conflicting counts circulating in secondary sources - both 16 tactics / 84 techniques and 16 / 167 appear in material published within months of each other. The figures above come from the machine-readable dist/v6/ATLAS-2026.07.yaml dataset that the site is built from, which is the strongest available source, but the rendered site may lag or lead it. So never write "ATLAS has 178 techniques". Write "ATLAS dataset 2026.07 has 178 techniques". The discipline costs you four words and saves you an argument.

The two v6 fields worth building a workflow around

Two tactics with no ATT&CK equivalent, and why that matters operationally Fourteen of the sixteen ATLAS tactics map to ATT&CK. Two do not: AML.TA0000 AI Model Access and AML.TA0001 AI Attack Staging. These are the genuinely new ground - the stages where an adversary obtains access to the model itself and prepares against it. If your detection engineering is inherited wholesale from an ATT&CK-shaped programme, these two tactics are precisely where you have no coverage and no alerts, because there was nothing to inherit.
⚠ The schema trap that fails silently The cross-reference key was renamed from ATT&CK-reference in v5 to attack-reference in v6. Tooling written against v5 silently returns null mappings rather than erroring, so a coverage report built on it will show a clean, confident, entirely empty ATT&CK column. Only 37 of 178 techniques carry a cross-reference at all, which makes an all-null result look plausible. Check one known-good technique by hand before you trust any automated mapping.

The technique IDs for Beacon's surface

These are the rows you will actually cite in findings for the rest of the builder track. All verified from the 2026.07 dataset.

IDNameTacticMaturity
AML.T0051
(.000 Direct, .001 Indirect, .002 Triggered)
LLM Prompt InjectionExecutionRealized
AML.T0054LLM JailbreakDefense Evasion, Privilege EscalationRealized
AML.T0053AI Agent Tool InvocationExecution, PrivEsc, Lateral MovementDemonstrated
AML.T0086Exfiltration via AI Agent Tool InvocationExfiltrationRealized
AML.T0080 (.000 Memory, .001 Thread)AI Agent Context PoisoningPersistenceDemonstrated
AML.T0110 (.000 / .001 / .002)AI Agent Tool PoisoningPersistenceRealized, no mitigation mapped
AML.T0070 / AML.T0071RAG Poisoning / False RAG Entry InjectionPersistence / Defense EvasionDemonstrated
AML.T0056 / AML.T0057Extract LLM System Prompt / LLM Data LeakageExfiltrationFeasible / Demonstrated
AML.T0084 (.000-.003)Discover AI Agent ConfigurationDiscoveryDemonstrated
AML.T0077LLM Response RenderingExfiltrationDemonstrated, no mitigation mapped
AML.T0098 / AML.T0082 / AML.T0083Credential harvesting via tools / RAG / agent configCredential AccessDemonstrated
AML.T0101Data Destruction via AI Agent Tool InvocationImpactRealized
AML.T0109AI Supply Chain Rug PullDefense EvasionRealized, no mitigation mapped
AML.T0034.002Agentic Resource ConsumptionImpactFeasible
AML.T0061LLM Prompt Self-ReplicationPersistenceDemonstrated
⚠ The residual-risk column, in the framework's own words Several Realized techniques - T0110 Tool Poisoning, T0109 Rug Pull, T0011.002 Poisoned Tool, T0077 LLM Response Rendering, T0083 - have no ATLAS mitigation mapped at all. Observed in the wild, and the framework has nothing to prescribe against them. Do not gloss this. It is the residual-risk column of an honest engagement, handed to you by the framework itself, and it is the strongest argument you will ever have for the b1 blast-radius section: where no mitigation exists, the only lever left is bounding what the technique reaches. Carry these rows into your report explicitly, as risk accepted rather than risk missed.

NIST AI 100-2e2025 - the objective layer above both

NIST's adversarial ML taxonomy classifies by what the attacker is trying to achieve, which is the layer that survives when the technique names change. Five objectives: NISTAML.01 Availability Breakdown, .02 Integrity Violation, .03 Privacy Compromise, .04 Misuse Enablement - GenAI only, and .05 Supply Chain. Attacks are then classified on five axes: system type, lifecycle stage, properties violated, attacker capability and access, and attacker knowledge.

Two structural points to carry forward. First, NIST taxonomises predictive AI and generative AI separately, and the separation is not cosmetic - the attack surfaces genuinely differ, and a control inventory written for one does not transfer. Second, .04 Misuse Enablement exists only on the GenAI side, because it is the objective that has no predictive-model analogue: the attacker is not corrupting your system, they are using it as a capability. That distinction is why misuse findings so often fall between the security team and the trust-and-safety team, and why b1 asked you to name the two tracks and their owners before the engagement starts.

Demo 1 of 2

Map Beacon's surface, four framework references per row ★ 10 min · everyone builds

A taxonomy that lives in a slide is decoration. A taxonomy that lives in the test-case schema is infrastructure, because every finding then arrives carrying the IDs a reviewer needs to place it. Build the schema now and the rest of the builder track just fills it in.

Start from the eight classes, not from the frameworks. Direct injection, indirect injection, refusal bypass, sensitive information disclosure, excessive agency and tool abuse, improper output handling, context and memory poisoning, unbounded consumption. Eight rows. This is the working set for the whole course and it is deliberately smaller than any of the official lists, because a class you cannot write a test for is not doing work in an engagement.

Give every row its OWASP 2026 ID. Use the table in Part 1 and check yourself against the version trap: improper output handling is LLM10:2026, not LLM05. Excessive agency is LLM03:2026, not LLM06. Unbounded consumption is LLM06:2026, not LLM10. If your row for context poisoning says LLM04, you have copied from a 2025 source.

Add the agentic ID where the boundary rule says it applies. For the Support Bot, context poisoning also carries ASI06 and tool abuse also carries ASI02, because that bot is an actor. For a read-only bot with no tools, leave the agentic column empty and say why in a comment. An empty cell with a reason is documentation; an empty cell without one is an oversight.

Add the ATLAS technique ID and copy its maturity across. The maturity value is the field you will sort by when the schedule slips, so it belongs on the row rather than in someone's head. Mark the rows where ATLAS has no mitigation mapped with a flag of its own - those rows cannot be closed by pointing at a control, only by bounding a consequence.

Add the NIST objective. One of the five. This is the column your executive summary is written from, because "an integrity violation on the refund path" survives a framework version change and "AML.T0051.001" does not.

Add the trial protocol and the defence reference. N from b1, result recorded as k of N rather than a boolean, and the name of the control that is supposed to hold this class. That last field is what turns the suite into a regression test in b10: when a control is removed or reconfigured, you know exactly which rows to re-run.

test-case schema · defensive · no payloads# one row per attack class, per surface. IDs verified against: # OWASP LLM Top 10 2026 · OWASP Agentic ASI list · ATLAS dataset 2026.07 · NIST AI 100-2e2025 class: indirect_injection surface: support_bot / ticket_ingest owasp_2026: LLM01:2026 agentic: ASI06 # actor, so the agentic list applies too atlas: AML.T0051.001 # maturity: Realized -> test first nist_objective: NISTAML.02 # integrity violation mitigation_gap: false # true when ATLAS maps no mitigation trials_n: 25 # from the RoE, fixed before testing result: k_of_n # never a boolean defence_ref: [leastpriv, hitl] # what is supposed to hold this class blast_radius: refund_credit # the b1 field that sets severity
The column teams skip, and regret in b9 mitigation_gap. When ATLAS maps no mitigation to a Realized technique, there is no control to point at during the harden-and-retest round, and a team that discovers this at retest time has already promised someone a fix. Flag it at mapping time and the conversation happens while options still exist.
Demo 2 of 2

Read the effectiveness matrix, top row first ★ 8 min · together

Here is the same eight-class taxonomy from the other side: not what an attacker does, but what each control actually removes. Read the top row before anything else. It belongs to the control every team reaches for first, and it is the only row on the board with zeroes in it.

⚠ Honesty rail for this board The inherent-risk figures and the defence-effectiveness matrix are a teaching model. They encode the direction and rough proportion that the published literature reports - notably that prompt-level instructions are near-useless against injection that arrives inside content, while architectural controls do the work - but they are not measurements of your system or anyone's. What is real is the arithmetic: coverage, residual risk and the untested-class accounting are computed live in your browser from the matrix and your settings.

Find the zeroes. The system-prompt instruction telling the model to ignore injected commands scores a flat zero against indirect injection, context and memory poisoning, improper output handling and unbounded consumption. Not a small number - zero. That encodes the consistent published position that instruction-level defences do not solve injection, and it is deliberately not softened.

Ask why those four classes specifically. They are the classes where the hostile text does not arrive from the person talking to the bot. A system-prompt instruction is being asked to police text that arrives looking exactly like the text it is meant to obey, from a source it cannot verify. The failure is architectural, so no rewording of the instruction changes the row.

Now read down the columns with the big numbers. Least-privilege scoping at 0.75 against excessive agency. Human approval at 0.85. Sandboxed execution at 0.80 against improper output handling. Rate and spend caps at 0.85 against unbounded consumption. Every large number on this board belongs to a control that removes a capability rather than requesting good behaviour. That is OWASP's design philosophy from Part 1, showing up as arithmetic.

Take the working conclusion into b3. The next session is the class with the zeroes running through it. You already know from this matrix that the fix will not be a better instruction, so the question b3 actually has to answer is what the measured limits of the real defences are - and what a defender does about the gap that remains after all of them.

Homework

Try it yourself - this week ◐ 45-60 min total

Source material

Official sources covered

Every framework, figure and quotation on this page traces to a primary source catalogued in this course's source map, verified 2026-08-14. This page covers:

✓OWASP Top 10 for LLM Applications, 2026 edition - all ten IDs and the four movesPart 1 · [P] from the project repository, 2026/final/
✓The 2026 ranking method: 75% community vote + 25% incident data, 7,714 incidents, 6,639 classifiablePart 1 · [P] · prompt injection falls out of the top 10 by raw count and is kept at #1 anyway
✓OWASP's design philosophy - build the surrounding system, not an unfoolable modelPart 1 · [P] · the course's own line
◐OWASP Top 10 for Agentic Applications, ASI01-ASI10Part 2 · ASI01's exact title is UNCONFIRMED (repo says Agent Behaviour Hijack, secondary says Agent Goal Hijack). ASI02-ASI10 titles from draft filenames plus agreeing secondary summaries
✓MITRE ATLAS dataset 2026.07, format 6.0.0 - 16 tactics, 178 techniques, 37 mitigations, 68 case studiesPart 3 · [P] machine-readable dataset · rendered site could not be loaded; conflicting counts circulate; cite the dataset version
✓ATLAS v6 maturity (Realized 68 / Demonstrated 92 / Feasible 18) and platforms (Agentic AI 119 largest)Part 3 · [P] · the test-prioritisation axis
✓AML.TA0000 AI Model Access and AML.TA0001 AI Attack Staging - the two tactics with no ATT&CK equivalentPart 3 · [P] · 14 of 16 tactics map; only 37 of 178 techniques carry a cross-reference
✓The v6 schema rename: ATT&CK-reference to attack-referencePart 3 · [P] · v5 tooling returns null silently
✓Realized techniques with no mitigation mapped: T0110, T0109, T0011.002, T0077, T0083Part 3 · [P] · the residual-risk column in the framework's own words
✓NIST AI 100-2e2025 objectives NISTAML.01 to .05, with .04 Misuse Enablement GenAI-onlyPart 3 · [P] · PredAI and GenAI taxonomised separately; five classification axes
◐The effectiveness matrix and inherent-risk figuresDemo 2 · teaching model encoding the direction the literature reports, not measurements · the arithmetic is real and computed live
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · A well-written vendor page lists Improper Output Handling as LLM05. What have you learned about the page?

This one ID is the fastest version check available, because it moved five places, further than anything else. The useful move is not to correct the number but to distrust the rest of the page: mitigations, severity guidance and coverage claims written against the 2025 list will also be a year stale. Three other IDs moved too - Excessive Agency up to LLM03:2026, Unbounded Consumption up to LLM06:2026, and System Prompt Leakage renamed and re-scoped to LLM08:2026 Hidden Context Exposure.

2 · Ranked by raw incident count alone, Prompt Injection drops out of the OWASP top 10 entirely, and OWASP keeps it at number one. What is the transferable lesson?

The mirror image is on the same list: Misinformation was voted near the bottom and sits near the top of the incident record. Two opposite distortions from the same dataset. Carry this into your own dashboard - a quiet quarter in your incident tracker is not evidence a class is under control, it is a measurement of what reached you and got written down. This is the same absence-of-evidence problem that runs through the whole course, arriving here as a ranking methodology rather than as a test result.

3 · Your ATLAS mapping shows several Realized techniques with no mitigation mapped at all - T0110 Tool Poisoning, T0109 Rug Pull, T0077 among them. What is the correct response?

Option C inverts the maturity field: Realized is the strongest evidence grade in the dataset, meaning observed in the wild. Option B is the more tempting error, because a complete-looking column passes review - but a borrowed mitigation that nobody has evaluated against this technique is a claim you cannot support, and it hides exactly the row a reader most needs to see. The honest move is to flag it at mapping time, before anyone has been promised a fix, and let the blast-radius limits from b1 carry the weight.

Builder session 2 cheat sheet · pin this

The 2026 version checkImproper Output Handling is LLM10:2026. If a page says LLM05, it is the 2025 list and everything version-sensitive on it is suspect.
The four that movedExcessive Agency to LLM03:2026 (up 3). Unbounded Consumption to LLM06:2026 (up 4). Improper Output Handling to LLM10:2026 (down 5). System Prompt Leakage renamed to LLM08:2026 Hidden Context Exposure.
An incident count is not a danger ranking7,714 incidents, 6,639 classifiable. By raw count prompt injection leaves the top 10; OWASP reads the absence as a defence effect. Counts measure what got through and got reported.
The design philosophyStop trying to build a model that cannot be fooled. Build the surrounding system so that when it is fooled, nothing important breaks.
The boundary ruleModel as component: the LLM list. Model as actor with tools, memory and consequences: the Agentic list ASI01-ASI10. Neither covers the ground alone. ASI01's exact title is unconfirmed.
Cite the dataset versionATLAS dataset 2026.07, format 6.0.0: 16 tactics, 178 techniques, 37 mitigations, 68 case studies. Conflicting counts circulate and the rendered site could not be loaded.
maturity is your test orderRealized 68, Demonstrated 92, Feasible 18. Test Realized first. platforms: Agentic AI 119 is now the largest tag. Schema trap: ATT&CK-reference became attack-reference in v6 and v5 tooling returns null silently.
The residual-risk columnT0110, T0109, T0011.002, T0077, T0083 are Realized with no ATLAS mitigation mapped. Nothing to point at, so bound the consequence and report it as risk accepted.