What the bench measures
Access control is usually reported as a single number, and the number is usually leaks, which is why over-blocks are invisible in every access review anybody has ever read. The bench prints leaks, over-blocks and standing privileges together, because each one has a one-line fix that ruins another and a policy set is only good when all three are low at the same time.
★ What you walk out with tonight. A policy set you have scored request by request instead of asserted. The measured cost of the two reflexes: stripping every PII column takes leaks to zero and leaves the support team unable to answer a ticket in 8 places, while one ACCOUNTADMIN grant restores all 109 original leaks and all 30 standing privileges in a single statement. And the honest finding that role splitting does most of the work, which is not where most teams spend their effort.
LiveWhy an over-block is not a safe failure3 min▶
The comfortable assumption is that denying too much is the conservative error. It is not, because the work does not stop. A support agent who cannot see a phone number still has a ticket to close, so the number arrives by another route: a colleague pastes it into chat, somebody exports a spreadsheet, or the team shares one login that still works.
Every one of those routes has less logging, less masking and a wider audience than the warehouse you just locked down. So an over-block does not remove exposure, it moves it somewhere you cannot see, which is why it belongs on the scoreboard next to leaks rather than being treated as the safe direction to fail in.
After an audit, Saltbox removed contact columns from every role. Within a fortnight the support team had a shared spreadsheet, refreshed weekly by an analyst with broad access, sitting in a folder shared with a contractor. The warehouse reported zero leaks throughout.
Self-studyWhat is real here and what the ground truth is3 min▶
Nothing on this bench is modelled or scripted, which is worth being precise about.
- The ground truth is the written access design from session 1: for each of the five jobs, which columns are needed in clear, which are needed masked, and which regions the job covers. It is a table, not an opinion, and it is visible in the widget.
- The requests are every combination of 5 personas, 10 columns and 3 regions, which is 150. Of those, 41 legitimately need a value in clear, 3 need it masked, and 106 have no business being answered at all.
- The evaluation is a real policy engine. Row policies are applied before masking policies, exactly as a warehouse does it. Column grants come from the role split. Masking policies transform what the grants allowed. The three counters are counted from what came back.
- Standing privileges are counted as columns a role can read in clear that the design does not mark clear for that role. That is the definition an access review uses, and it is why the number survives long after the person who asked for the grant has left.
The ladder, with the numbers it actually produces
10 min liveEvery row below is computed by the bench on this page. Read the third column as carefully as the first: the row where standing privileges collapse from 17 to 1 is the row where the real work happened, and it is a role-modelling exercise rather than a policy one.
| Policies on | Leaks | Over-blocks | Standing privileges |
|---|---|---|---|
| One shared analyst role | 109 | 0 | 30 |
| + mask national_id for everybody | 94 | 0 | 25 |
| + mask email and phone, except support | 70 | 0 | 17 |
| + split the shared role five ways | 22 | 0 | 1 |
| + row access policy on region | 3 | 0 | 1 |
| + pseudonymise the modelling view | 0 | 0 | 0 |
| The whole design, plus strip every PII column | 0 | 8 | 0 |
| The whole design, plus grant ACCOUNTADMIN | 109 | 0 | 30 |
Read the last two rows together. They are the two things teams actually do, and they fail in opposite directions. Stripping PII satisfies an auditor and breaks the support desk in 8 places. Granting the top role satisfies everybody in the room on Friday and returns all three counters to their opening values, because a role above the policies is a role the policies do not apply to.
Run the ladder yourself
★ 25 min · everyone buildsThe bench is live below. The grid shows every job against every column; hover a cell for the verdict in each region. Work the steps in order and argue about step six.
Start from one shared role. 109 leaks, no over-blocks, 30 standing privileges. Note what a leaks-only report would say about this: nothing good, but also note that a report on over-blocks alone would call it perfect.
Add the two masking policies. national_id first, then email and phone with the support exception. Leaks fall to 70 and the support agent's row stays clean, because the exception role is part of the policy rather than a hole punched through it afterwards.
Now split the role. Leaks fall from 70 to 22 and standing privileges from 17 to 1 in one move. This is the step worth taking back to your own estate first, and it is the one that looks like admin work rather than security work.
Add the row policy. 22 leaks to 3. Everything remaining is one cell: the data scientist holding a real customer_id where the design asked for a pseudonym.
Pseudonymise the modelling view. Zero, zero, zero across 150 requests. Everybody can do their job and nobody holds a column they cannot justify.
Now tick "strip every PII column". Leaks stay at zero and 8 over-blocks appear, all of them the support agent, across name, email, phone and support_notes in both of their regions. Decide out loud how that team closes its tickets tomorrow, and where the phone numbers end up.
Untick it and tick "grant ACCOUNTADMIN" instead. 109 leaks, 30 standing privileges, and every policy you built still defined and still deployed. Read the grid: it is identical to step one. A role above the policy layer is a role the policy layer does not reach.
Write down the three numbers for your own estate and which of the two over-corrections your team reached for last time. Both are normal. Neither is a fix.
The number that is missing from this bench, deliberately. Every request counts the same, so a leaked national ID scores exactly like a leaked plan name. A real scoreboard weights by classification tier, and adding that weighting is the first change worth making to your own version. It moves the ranking of the middle rungs and does not rescue either over-correction.
Closing the track
45 min- Score your own table the way the bench does. You already have the design from session 1 and the grants from session 7, so this is a join rather than a project.
- Count your over-blocks by asking, not by querying. Three questions to support and operations: what can you not see, what do you do instead, and where does that copy live.
- List every human holding a role above the policy layer, and the date each one got it. Any grant older than the incident that caused it is a standing privilege with a nicer name.
- Add classification weighting to your own scoreboard and see which of your rungs change order.
- Put the three numbers in the same monthly report as everything else. A security number reported only when somebody asks is a number that gets worse quietly.
Three questions
Granting ACCOUNTADMIN on top of the finished design returns leaks to 109 and standing privileges to 30. Why does the policy set stop working?
The policies are all still defined and deployed. They simply do not apply to a role that sits above them, which is what makes this the most expensive one-line fix available.
Stripping every PII column holds leaks at zero and produces 8 over-blocks. Why is that not the conservative choice?
Exposure moves rather than disappearing. The warehouse reports zero leaks while the phone numbers live in a shared spreadsheet, which is the outcome the audit was trying to prevent.
Which single step on the ladder moves both leaks and standing privileges the most?
Role modelling does most of the work and gets the least attention, because it reads as administration rather than security. The masking and row policies are what make the remainder correct.