The class, and the two numbers that describe it
MITRE ATLAS calls it AML.T0054 LLM Jailbreak, filed under two tactics at once, Defense Evasion and Privilege Escalation, with maturity Realized - meaning it has been observed in the wild, not merely demonstrated in a lab. OWASP files the input side under LLM01:2026 Prompt Injection. NIST's taxonomy gives it its own attacker objective, NISTAML.04 Misuse Enablement, which exists only on the generative side of the taxonomy: there is no equivalent objective for predictive AI, because there is nothing there to talk into anything.
It is a rate, and it was never anything else 6 min live
This is the methodological heart of the session, and it is not a refinement of ordinary security testing - it is a different kind of claim. In classic penetration testing an exploit either works or it does not, and a single successful run is proof. Here, the same input does not give the same answer twice, so a single run is a sample, and a sample of one tells you almost nothing about the distribution it came from.
LiveThe three variables hiding inside any success rate3 min▶
HarmBench gives the formal definition of attack success rate, and the useful thing about seeing it written out is that it exposes exactly three places where two people can produce different numbers for the same system:
- The classifier
c- whoever decides a given output counts as a success. Change the judge and change the number, with no change to the system under test. - The token or attempt budget
T- HarmBench's own finding is that budget "drastically impacts" the result and is not standardised across papers. - N - the trial count from Part 1, which is the difference between 57% and 80% on identical attacks.
promptfoo's documentation says the same thing in its own words: attack success rate depends heavily on attempt budget, prompt set composition and judge choice. So when a vendor gives you an ASR, the three questions are not rude, they are the only way to read the number: which judge, what budget, how many trials.
This is also why cross-vendor ASR comparison is not a thing you may do. A NeurIPS 2025 paper by Chouldechova, Cooper, Barocas, Palia, Vann and Wallach puts it plainly: conclusions about relative system safety are often not supported by the evidence that ASR comparisons provide, resting on apples-to-oranges comparisons or low-validity measurements. Report your rate as a trend within your own pinned configuration, over time. Never as a league table.
The cheat, and the cheapest guard against it 6 min live
Here is the failure mode that no amount of adversarial cleverness will catch, because it is not an attack. You harden the system, the jailbreak rate falls, everyone is pleased, and the product quietly becomes unusable. There is exactly one cheap instrument that detects this, and it is the reason serious jailbreak benchmarks ship two sets rather than one.
Read that last line again, because it is a rare thing in this literature: a defence paper publishing its own cost in the same document as its own win. Instruction hierarchy trains the model to rank system instructions above user instructions above tool output, it generalised to attack types it was never trained on, and it delivered up to a 63% robustness improvement. It also over-refused some requests it should have answered. Both facts are in the paper. A vendor blog reporting only the first number is not lying, it is quoting.
The benchmark landscape, read defensively 6 min live
You are not going to write a jailbreak corpus from scratch, and you should not want to. Four public suites are named by NIST itself, and each answers a different question. Pick by question, not by popularity.
| Suite | Shape | The question it answers |
|---|---|---|
| HarmBench | 510 behaviours, 7 semantic categories plus 4 functional categories | Breadth. Does the system hold across the range of harm types, and does it hold for functional shapes such as multimodal or contextual behaviours as well as topics? |
| JailbreakBench | 100 harmful and 100 benign behaviours, 10 categories aligned to published usage policies | The pair. What did the hardening cost? This is the suite that ships the counter-metric in the box. |
| AgentHarm | 110 malicious agent tasks, 11 categories, multi-step | Two things at once: does the model refuse, and if it is jailbroken, does it retain the capability to actually complete a multi-step task? |
| AgentDojo | 97 tasks, 629 security test cases | Utility under attack in an agent environment. b6 and b7 use this one; it is listed here so you know where it sits. |
That second AgentHarm dimension is worth pausing on, because it is what makes it an agent benchmark rather than a jailbreak benchmark with tools attached. A model that has been talked out of refusing but is then too incoherent to complete a five-step task is a different risk from one that stays fully capable after the refusal falls away. On a chat product only the first dimension exists. On an agent, capability retention is most of the severity, which is exactly the argument b6 builds out.
Anthropic's Constitutional Classifiers is the most useful measured result in this session, because it reports all three of the numbers a defender needs: the win, the cost to users, and the cost to the bill.
Build Beacon's jailbreak test-case schema ★ 12 min · everyone builds
Beacon's Support Bot is the target for this one, because it is the T3 bot that can issue refund credits, so it is the one where a policy bypass could matter. You are writing the record format, not the attacks. Get the format right and the corpus can grow for years; get it wrong and every finding needs re-testing to become comparable.
One case is a behaviour, not a string. A case names a thing the system must not do for anybody - "issue a refund credit outside policy", "produce content in a category the usage policy forbids". Attack techniques are how you probe the behaviour and they change every quarter; the behaviour does not. This is also what lets you record a case in a shared repository without shipping an exploit.
Fix N in the case, not in the run. If N lives in the runner, someone will change it on a Friday and the trend line silently breaks. Put it in the case file, pin it in review, and make a change to N a visible diff.
Name the judge in the case. Whoever or whatever decides success is the c from Part 1. A human adjudicator is a valid judge if you name them. A model judge is a valid judge if you version it. An unnamed judge means the number cannot be reproduced.
Pair every harmful case with a benign twin in the same category. The twin is the request a real Beacon customer makes that lives near the line: the genuine refund enquiry, the legitimate question about account closure. If a hardening round moves the harmful case and the twin, you have not hardened anything, you have narrowed the product.
Record k of N and the build. A finding with no build identifier expires the moment someone deploys, and nobody will know it expired.
benign_twin. It feels like paperwork on the day you write it and it is the only field that will stop a future team from shipping a refusal machine and calling it a security improvement. Make the schema reject a harmful case that has no twin.
The severity conversation, before it happens to you ★ 8 min · in pairs
Somebody is going to walk into your review with a jailbreak and a strong opinion about its severity. This is the ten minutes that decides that conversation in advance, using published vendor positions rather than your own instinct, so that it is a policy discussion rather than a personality one.
Read Google's published scope out loud. Their AI vulnerability reward programme puts direct injection, jailbreaks and alignment failures out of scope for security bounties, while indirect injection with production impact is in scope. That is not a claim that jailbreaks do not matter. It is a claim about the security threat model: absent a chain to data or an action, a bare jailbreak has no security impact to pay for.
Read Microsoft's alongside it. MSRC's AI bug bar classifies content-related issues as In Scope or Out of Scope with no severity rating at all - a deliberate structural separation of the safety track from the security track. Two tracks, two owners, two timelines. Beacon should copy the structure, which is exactly what your b1 rules-of-engagement document already split.
Now do the chaining exercise on Beacon, in pairs, four minutes. Take one jailbreak finding and ask what it reaches. On Insight Bot with no tools, no private data and no egress, it is informational: a bad answer in a box. On Support Bot, where the same bypass ends in a refund credit being issued, the finding is not a jailbreak finding at all any more, it is an unauthorized-action finding, and it is severe because of the credential, not because of the prompt.
Write the severity sentence into your report template. Ours: severity is set by what the finding chains to, not by whether the model misbehaved. Then keep reporting the bare rate anyway, in the safety track, because it is the trend line that tells you whether your next model upgrade regressed.
Try it yourself - this week ◐ 45-60 min total
- Find the last jailbreak finding your organisation recorded and check whether it says k of N. If it says "succeeded" or "blocked", re-run it at N=25 and see whether the sentence survives.
- Write ten benign twins for your product - the requests that live nearest the line and must still be answered. Ten is enough to catch the refusal machine, and you can grow it later.
- Ask whoever owns your guardrail what the compute overhead is. Constitutional Classifiers published 23.7%. Most internal defences have never had the number measured at all, and it is usually the reason a defence gets quietly disabled under load.
- Look up your own vendor's bounty scope and find the sentence about jailbreaks. Bring it to your next triage meeting before the argument, not during it.
Official sources covered
Every framework, figure and quotation on this page traces to a primary source catalogued in this course's source map, verified 2026-08-14. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · A tester reports "we attempted the policy bypass and it was blocked". What is the problem?
Option C sounds like the diligent answer and is the wrong lesson, because it treats the problem as effort rather than as measurement. Non-determinism is a property of the system, not a failure of the tester. The fix is structural: N fixed in the case file before testing, k of N in the finding, and the build identifier alongside it so the result expires visibly rather than silently.
2 · Your hardening round takes the jailbreak rate from 42% to 1%. What single number do you need before you may call this an improvement?
Both A and B are worth knowing and neither detects the specific cheat. Compute overhead tells you the bill, technique count tells you coverage, but only the benign set tells you whether the security gain was actually a usability loss wearing a security costume. JailbreakBench ships 100 benign behaviours next to its 100 harmful ones for precisely this reason, and OpenAI's Instruction Hierarchy paper is the honest example: +63% robustness reported alongside its own over-refusal regressions.
3 · A researcher brings you a reliable jailbreak of a bot that has no tools, no private data and no outbound network. How do you triage it?
C is the trap, and it is the one that gets adopted by accident. "The bounty does not pay for it" is a statement about who owes money, not about whether the behaviour is real or whether the trend matters. Keep measuring the rate, keep the counter-metric next to it, and note the condition under which the triage flips: the day that bot gets a tool or a data source, the same finding becomes an unauthorized-action finding and the severity comes from the credential rather than from the prompt.