The question a leader actually has to answer
Not "should we use AI in performance management" - you already are, the moment a review tool suggests wording or a dashboard summarises feedback. The real question is narrower and sharper: what are we letting AI decide, and what are we only letting it help with? Get that boundary right and AI gives managers back the hours they were spending on admin, and makes feedback fairer by catching bias a tired human misses. Get it wrong and you have built surveillance that the EU classes as high-risk, that New York already audits, and that the UK regulates through data-protection duties - and that quietly destroys the trust every other session of this course was trying to build. Tonight is about drawing that boundary on purpose.
Support augments the manager; surveillance replaces judgment 7 min live
The same technology sits on both sides of the line. The difference is not the model - it is whether the output informs a human who then decides, or becomes the decision about a person. Everything on the left frees a manager to spend more time coaching. Everything on the right measures activity and calls it performance.
LiveWhy the "helper" list is genuinely worth having3 min▶
The left column is not a consolation prize - done well, it makes measurement fairer than a human alone:
- De-biasing language - a model can flag that a draft review calls a woman "abrasive" and a man "assertive" for the same behaviour, or that feedback is vague ("great attitude") rather than specific. It catches what a rushed manager misses.
- Fighting recency bias - by summarising notes gathered all quarter, AI stops the whole review collapsing into "what happened last week".
- Giving time back - drafting and summarising is the admin that eats the hours managers should spend actually talking to people. Automate the paperwork, not the judgment.
The test that sorts the two lists. Ask of any feature: "if this output is wrong, who catches it before it affects a person?" If the answer is "a manager reviews and can overrule it," it is support. If the answer is "the system acts on it directly," it is surveillance wearing an efficiency badge - and it belongs on the right.
The governance a leader must know 6 min live
You do not need to be a lawyer, but you do need to know that measuring workers with AI is now one of the most regulated things software can do. Three rules matter most. Treat them as gates: an AI measurement tool that cannot clear all three should not go near your people.
Self-studyWhat each gate is really protecting3 min read▶
- EU AI Act - lists AI used for recruitment, task allocation, and monitoring or evaluating workers as "high-risk" (Annex III, section 4). High-risk does not mean banned; it means obligations: risk management, data governance, logging, transparency, and meaningful human oversight. If a vendor cannot show you these, that is your answer.
- GDPR Article 22 - a person has the right not to be subject to a decision based solely on automated processing that has a legal or similarly significant effect (like firing or denying a promotion). The safeguard is a genuine human-in-the-loop who can review, explain, and reverse - plus the person's right to contest.
- NYC Local Law 144 - automated employment decision tools must pass an independent annual bias audit, publish a summary of the results, and give candidates at least ten business days' notice before the tool is used on them. It exists because these tools were quietly encoding discrimination.
Regulation moves, so hold these loosely and check them before you rely on them. EU: the Annex III classification stands, but the AI Omnibus (in force 27 July 2026) pushed the compliance duties for standalone Annex III systems from 2 August 2026 to 2 December 2027. You have design time, not a pass. New York City: Local Law 144 is already live, in effect since January 2023 and enforced since July 2023. UK: there is no AI Act and no high-risk classification - worker-management AI is governed through data-protection duties (UK GDPR, ICO guidance), and the Data (Use and Access) Act 2025 replaced the near-prohibition on solely automated significant decisions with a safeguard-led regime. So if your organisation spans the EU and the UK, the same tool sits under two different rulebooks. None of this is legal advice; it is what to ask counsel about.
Two cautionary cases, read as a leader ★ 10 min · think it through
Both companies were sophisticated and well-resourced. Both crossed the line anyway - which is exactly why you cannot delegate this decision to a procurement checklist. Walk each one back to the boundary from Part 1.
Microsoft Productivity Score (2020). A feature let managers see per-employee activity - emails sent, chats, document edits, meeting attendance. Critics called it workplace surveillance; after public backlash Microsoft removed the individual-level view and apologised.
Amazon warehouse "time off task" (reported by MIT Technology Review). Systems tracked idle time and, per reporting, could generate warnings and terminations largely automatically - people managed by an algorithm measuring seconds.
Locate each on the Part 1 line. Both sit firmly on the right: per-person activity scores, and automated action on a proxy - with the human either absent or reduced to a rubber stamp.
Ask the diagnostic question: did the metric measure value delivered, or just activity observed? In both cases, activity - which is precisely what Goodhart's Law tells us people will then game.
| Design choice | Surveillance framing | Support framing |
|---|---|---|
| Unit of measure | the individual, ranked | the team + the workflow |
| Who acts on it | the system, automatically | a manager, accountably |
| What is measured | activity (keystrokes, idle time) | outcomes + blockers |
| The person can... | nothing - it is opaque | see it, question it, contest it |
Audit your own AI touchpoints against six principles ★ 8 min · your org
You almost certainly have more AI in your performance stack than you think - in the HRIS, the review tool, the recruiting funnel, the productivity dashboard. Take each place AI touches how people are measured and run it through the principle set.
List every point where AI touches performance or measurement - drafting reviews, summarising feedback, scoring candidates, ranking, flagging activity, monitoring.
For each, test the six principles: augment not replace, transparency (people know it is used and how), contestability (they can challenge it), human accountability (a named person owns the decision), measure outcomes not surveillance, and proportionality (the least intrusive method that works).
Flag any touchpoint that fails augment-not-replace, transparency, or contestability - those are the three that also map to the legal gates.
Proportionality is the quiet one. Even if a monitoring tool were legal and transparent, ask whether you need it at all. If a weekly outcome conversation would tell you the same thing as continuous keystroke logging, the logging is disproportionate - and every hour of it is a withdrawal from the trust account. The most defensible answer is often to measure less, and talk more.
Try it yourself - this week ◐ 30-45 min total
- List every place AI currently touches performance or measurement in your organisation - reviews, feedback, recruiting, dashboards, monitoring.
- Check each one against three tests: is it transparent to the people affected, is there a real human-in-the-loop, and is it contestable?
- Circle any touchpoint that fails a test. Decide whether to fix it (add the human, make it transparent) or retire it.
- Pick one vendor tool and ask them the single question: "show me where a named human reviews and can overrule the output, and how a person contests it."
- Optional: read the summary of EU AI Act Annex III and the ICO guidance on GDPR Article 22 before a6.
Sources covered (cited)
Taught from the AI-governance and worker-measurement canon. This session covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · What separates AI as support from AI as surveillance?
Same technology, different design. If a manager reviews and can overrule the output, it is support. If the system acts on a person directly, it is surveillance - and the law now treats that as high-risk.
2 · What does GDPR Article 22 require for a decision with significant effect on a person?
Article 22 protects people from purely automated decisions like firing or denying promotion. The safeguard is a genuine human who can review, explain, and reverse, plus the person's right to contest.
3 · The Microsoft Productivity Score and Amazon "time off task" cases both went wrong because they...
Both measured per-person activity rather than value delivered, and removed or minimised the accountable human. That is the surveillance side of the line, and exactly what Goodhart's Law predicts people will game.