learn-ai-consulting-with-phoebe / Leader session 1 of 6
Learn AI Consulting with Phoebe · Leader track · Session 1 of 6

The jagged frontier

Every opinion you have heard about AI in consulting - the euphoric ones and the dismissive ones - is downstream of one field experiment. In 2023, Harvard researchers ran 758 BCG consultants through realistic consulting tasks with and without GPT-4, and the results point in two directions at once. This session gives you the primary source, properly read, so that when your partners debate AI policy you are arguing from evidence instead of anecdotes.

🤝 Leader track Partners + practice leads No tools required Start here
0-3 · Welcome 3-22 · The study 22-40 · What it means for the firm 40-45 · Q&A
Part 0

Why we start with one paper

This is the leader track: six sessions for partners, practice leads, and independent consultants who own the P&L, the risk, and the bench. It runs firm-level while the consultant track runs engagement-level - over there, your teams work a six-week Northbridge Advisory engagement for Cadence, an AI note-taker client. Here, session 1 does one thing: it puts the largest field experiment ever run on AI-assisted consulting work in front of you, whole. Almost every claim a vendor, a client, or a junior will make to you this year traces back to it - usually with half the numbers missing.

The discipline for the next 45 minutes: no opinions until the evidence is on the table. Then three moves - what the study found, what the two second-order findings mean for staffing and what the firm sells, and how to turn the frontier from a metaphor into a management object your practice actually governs. The six leader sessions build on each other:

Live - presented in session Self-study - read after class Sources covered
★ What you walk out with today The four headline numbers from the BCG field study (three flattering, one frightening), the two second-order findings most leaders miss, and a first frontier map of your own practice - which task classes are delegated, supervised, or forbidden.
The house rule for this track Every session names its evidence. When a claim is a study result you get the study; when it is a company's own press-reported statement you will see it labeled that way. Model the same standard in your partner meetings - it is cheaper than learning it from a client.
Part 1 · covers the evidence base

The study, properly 9 min live

Dell'Acqua et al., 2023: 758 BCG consultants - about 7% of the firm's individual contributor staff - worked 18 realistic consulting tasks, randomized into a GPT-4 arm and a no-AI control. Inside the frontier of AI capability, the AI arm completed 12.2% more tasks, worked 25.1% faster, and produced quality rated 40% higher than control. Then the researchers added one task deliberately designed to sit outside AI's capability - and on that task, AI users were 19 percentage points less likely to reach the correct answer than consultants working unassisted.

758 BCG consultants about 7% of the firm's IC staff GPT-4 arm Control arm - no AI both arms, compared on two task regimes Regime 1 · 18 realistic tasks INSIDE the frontier +12.2% · 25.1% faster · +40% tasks completed · speed · quality, GPT-4 arm vs control Bottom-half performers gained +43%; the top half +17%. Regime 2 · 1 task designed OUTSIDE -19 points AI users were 19 percentage points less likely to reach the correct answer. Four headline numbers, two directions: +12.2% · 25.1% faster · +40% inside, -19 points outside.
🔍 Click to zoom - the study design and its four headline numbers
LiveCentaur, cyborg, and the pattern that fails4 min▶

The authors describe two working patterns among consultants who navigated the frontier well. Centaurs divide the work: discrete subtasks go to the AI - draft this, summarize that - while the human keeps every decision point. Cyborgs weave AI through the whole workflow continuously. Both patterns work. What fails is the third, unnamed one: pasting the task in and shipping whatever comes back. On the outside-frontier task, paste-and-ship is exactly how consultants ended up 19 percentage points behind colleagues with no AI at all - the wrong answers read just as confident and just as polished as the right ones.

For a leader, the takeaway is not "train people to prompt better." It is that the returns come from a division of labor, and the losses come from its absence. Your firm's exposure is not the consultant who refuses AI; it is the one who trusts it evenly across tasks whose difficulty is anything but even.

Self-studyWhy "19 percentage points" is the number to memorize2 min▶

Say it precisely: 19 percentage points, not 19 percent - the gap between the two arms' correctness rates. And note what the researchers did: they designed that task on purpose, as a trap, because they knew clients would eventually set one by accident. The honest reading is not "AI fails sometimes." It is: on tasks beyond its capability, AI-assisted professionals did worse than unassisted ones, because a fluent wrong answer is more persuasive than no answer. That asymmetry is the entire reason this track exists.

Part 2 · covers the firm-level consequences

The two second-order findings leaders miss 7 min live

The headline numbers get quoted everywhere. The two findings that actually change how you staff and what you sell sit a layer down - and most coverage skips them.

LiveThe floor rises, the output converges5 min▶

Finding one: AI is a floor-raiser. Bottom-half performers gained +43% with GPT-4; the top half gained +17%. The gap between your weakest and strongest producers narrows dramatically the moment both have the tool. Consequence for staffing: baseline production - a competent first draft, a clean summary, a serviceable deck - stops differentiating juniors. The qualities that still separate people are the ones the tool does not supply: judgment about what matters, knowing the client, owning a number.

Finding two: output homogenized. AI-assisted answers converged on similar ideas - the study found less variety across consultants who used the tool. Consequence for what the firm sells: if your deliverables and your competitor's deliverables are drafted by the same model, the draft is no longer the product. Differentiation moves to judgment, framing, and proprietary knowledge - the things a client cannot get by asking the model themselves.

Put the two together and the strategic sentence writes itself: the work AI does well is becoming table stakes, so the firm's margin must live in the work it cannot do. Every later session in this track - pricing (a2), quality control (a4), operating model (a5), the bench (a6) - is a consequence of that sentence.

Real worldThe firms in the study did not wait to act on it. BCG has stated - in press-reported figures, label them as such - that roughly 90% of employees use AI, with internal tools like Deckster for slide drafting. When 90% of a competitor's staff has the floor-raiser, "our juniors write good first drafts" is no longer a differentiator you can bill for.
Self-studyHow to read a field experiment without a stats degree3 min▶

Three words carry most of the weight. n - how many people: 758 here, and about 7% of BCG's IC staff, which is why this study outranks every 12-person survey in your inbox. Control - who they were compared against: consultants doing the same tasks without AI, which is what separates "AI users did well" from "AI caused them to do well." Effect size - how big, in plain words: not "significant" but "25.1% faster" and "19 percentage points less likely correct." When a vendor shows you a chart, ask for those three things in that order. If any answer is missing, you are reading marketing, not evidence.

Part 3 · covers frontier governance

The frontier as a management object 6 min live

The study's authors called the frontier jagged for a reason: AI capability is uneven across tasks in ways that do not follow intuition. It is also invisible - work from beyond the frontier looks exactly as confident as work from inside it - and it is moving, because models change under your feet. A boundary with those three properties cannot be left to individual judgment call by call. It has to be mapped, and the map has to be owned.

OUTSIDE the frontier - fluent, confident, and wrong INSIDE - AI drafts well, humans review the -19 pt zone: tasks that look inside but are not summarize draft analyze decide Jagged: uneven per task. Invisible: wrong output looks as polished as right. Moving: models change. A boundary like this is not a training topic - it is a map the practice owns and reviews.
🔍 Click to zoom - the frontier is jagged, invisible, and moving; the map is the management tool
LiveFrontier governance: the three-column map4 min▶

A leader's job here is not prompting skill - it is frontier governance: deciding, explicitly and per task class, which work is delegated to AI with light review, which is supervised with a human checking every load-bearing fact, and which is forbidden to delegate at all. The instrument is a three-column map:

ColumnWhat lives thereThe rule
INSIDEsummaries, restructures, first drafts of routine materialdelegate, review before it ships
EDGEanalysis, anything needing facts the model must fetchsupervise - a human owns every number
OUTSIDEjudgment calls, client politics, commitments, the signatureforbidden to delegate, in writing

The EDGE column is where the -19 points live: work that looks inside but is not. The consultant track builds this map individually in its first session - your version today is for the practice, and it is a standing document, not a workshop artifact. The frontier moves; the map gets a review date.

Real worldFrontier governance at firm scale already exists. McKinsey's internal platform Lilli - launched 2023, used by roughly three quarters of its ~43,000 people at over half a million prompts a month - is, structurally, a governance decision: one approved tool, firm knowledge inside it, and the firm's claim of about 30% time saved on information gathering attached. The tool choice is the policy. Session a5 takes the operating-model question in full.
Everyone works

Map your practice 🗺 ★ 18 min · everyone works

No tools, no slides - a sheet of paper and honesty. By the end you have a first frontier map of your practice and the note that makes it real.

List the 8 most billed activity types in your practice. Not job titles - activities: competitor scans, interview synthesis, first-draft decks, due-diligence memos, workshop design, the readout itself.

Sort them into the three columns honestly. INSIDE / EDGE / OUTSIDE. If you catch yourself putting everything INSIDE, remember the study built its trap task precisely because that instinct is common - and wrong by 19 percentage points.

Mark every EDGE item with the last time AI failed you there. A date, or "never checked." Items marked "never checked" are not EDGE items - they are unexamined risk sitting in the delegated pile.

Pick ONE forbidden-list item and write why in a single sentence. If the sentence is convincing, it becomes policy language. If you cannot write it, the item may not belong on the forbidden list - move it and say what supervision it needs instead.

Draft the one-paragraph note to your team announcing the map exists. What is delegated, what is supervised, what is forbidden, and who owns the next review. Send it this week - an unannounced policy is an opinion.

✦ If you finish early Ask your preferred AI tool a question from your own practice area whose answer you know cold, and grade it. You just took one sounding of where the frontier runs in your domain. Repeat weekly; the map stays honest only if someone keeps measuring.
Before next session

Homework 📝 ◐ 20 minutes

Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · The four headline numbers from the 758-consultant BCG field study are...

Both directions are the finding. Quoting only the gains is how firms end up trusting AI evenly across an uneven frontier - and the outside-frontier work reads exactly as confident as the inside-frontier work.

2 · Who gained most from GPT-4 in the study?

The floor rises and output converges, so baseline production stops differentiating juniors - and the firm's margin has to move to judgment, framing, and proprietary knowledge.

3 · Operationally, "the frontier is jagged" means...

Uneven, invisible, and moving - which is why it is a management object, not a training topic. The three-column map (INSIDE / EDGE / OUTSIDE) is the governance instrument, and the EDGE column is where the -19 points live.

Coverage, honestly

What this session covered from the official universe

✓Dell'Acqua et al., "Navigating the Jagged Technological Frontier" (HBS/BCG, SSRN 2023)the four headline numbers both directions, centaur/cyborg, plus both second-order findings
◐Mollick-style commentary on the studythe homogenization theme and where differentiation moves; read the primary source first
◐DeepLearning.AI "Generative AI for Everyone" - W3 task-analysis lensthe frontier map borrows its task-level (not job-level) decomposition framing

Certificates, videos, and graded assessments stay with the official providers - this track teaches the working core.

Leader session 1 cheat sheet

The studyDell'Acqua et al. 2023: 758 BCG consultants (~7% of IC staff), 18 realistic tasks, GPT-4 vs control.
Inside the frontier+12.2% tasks completed · 25.1% faster · quality +40% vs control.
Outside the frontierOn the deliberately designed task: 19 percentage points less likely to reach the correct answer.
Centaur vs cyborgDivide subtasks, keep decision points vs weave throughout - both work. Paste-and-ship fails.
Floor-raiserBottom half gained +43% vs +17% top half - baseline production stops differentiating juniors.
HomogenizationAI-assisted answers converge - differentiation moves to judgment, framing, proprietary knowledge.
Frontier propertiesJagged, invisible, and moving - so it is mapped per task class, never assumed or set once.
The leader's instrumentThree-column map: INSIDE delegated · EDGE supervised · OUTSIDE forbidden. Owner + review date.