learn-ai-consulting-with-phoebe / Consultant session 4 of 10
Learn AI Consulting with Phoebe · Consultant track · Session 4 of 10

Analysis co-pilot: findings to insights

Week 3 at Northbridge. The research holds up (b3 made sure), and now twelve customer interviews and a stack of usage cuts sit between you and the storyline. This is the week engagements are won - and the week AI's most seductive failure lives. Ask a model for themes and you will get themes, every single time, whether or not the data contains any. The outlier that was the actual insight gets averaged into "mixed feedback". This session teaches the synthesis discipline that keeps it alive.

🟡 Consultant track Week 3 of the engagement Synthesis without flattening 45 min + self-study depth
0-3 · Welcome 3-20 · Concepts 20-40 · Synthesize the interviews 40-45 · Q&A
Part 0

Why this session exists

Week 3 is where the engagement stops being about collecting and starts being about meaning. It is also where AI is at its most fluent and its most dangerous at the same time: a model will happily compress twelve interviews into three tidy themes that read like understanding and contain none. The flattening is invisible in the output - the summary looks complete precisely because the disagreement and the outliers have been smoothed away. The cure is not a better prompt; it is a discipline: quote before you summarize, tag the dissent, count before you characterize, and keep the outlier visible until a human decides what it means.

Live - presented in session Self-study - read after class ▶ Live lab - scores real text Sources covered
★ What you walk out with today The four-rung ladder from data to implication (and exactly where AI must stop climbing), a four-step synthesis discipline that survives contact with real interviews, and the three support arguments - demand, economics, window - drafted and sourced, ready for b5's pyramid.
The week 3 rule A theme you did not count is an opinion. A summary without a dissenting quote is a press release. This session makes both sentences reflexes.
Part 1 · covers the core distinction

The findings-to-insights ladder 7 min live

Consulting analysis climbs four rungs, and the words matter. Data is what you gathered. A finding is a true observation about it: "62% of trial teams hit the seat cap within 3 weeks." An insight is the so-what that changes a decision: "demand is already organized into buying units - the sales motion can start with existing accounts." An implication is what to do about it: "target the 210 first." AI's capability fades as the ladder climbs - and the top rung is the recommendation your firm signs.

DATA 12 interviews, usage cuts, rival pricing FINDING 62% of trial teams hit the 5-seat cap INSIGHT demand is organized into buying units IMPLICATION target the 210 accounts first Where AI can operate on each rung: AI: excellent AI: excellent AI: mediocre AI: forbidden The implication is the recommendation Northbridge signs - AI stops one rung below it.
🔍 Click to zoom - the ladder, with AI capability fading as it climbs
LiveWhere AI stops climbing, and why4 min▶

Findings are AI's home turf: extraction, restructuring, and summarizing over material you hand it. Fed the transcripts, it will pull every seat-cap mention faster and more completely than any analyst. Insights are where it turns mediocre: an insight requires knowing which decision the client is actually facing, and that context lives in your engagement, not in the model. Asked for insights cold, AI produces findings wearing bigger words. Implications are forbidden - not because the model cannot type "target the 210 first", but because that sentence is a recommendation the firm signs and a human owns. Session 1's rail 2 draws the same line from the liability side.

The practical shape of week 3, then: AI drafts rungs one and two at volume, proposes candidate so-whats for rung three that you accept, reject, or sharpen, and never touches rung four. The engagement's judgment budget - your scarcest resource - gets spent where AI is weakest, which is exactly where it should be spent.

Real worldThe jagged-frontier study is the evidence for the fade. Inside the frontier, 758 BCG consultants with GPT-4 completed 12.2% more tasks, 25.1% faster, at 40% higher quality - findings work. On the one task designed beyond the frontier - a subtle analysis where the right answer required reading the data against the obvious story - AI users were 19 percentage points LESS likely to be correct, and their wrong answers read just as polished. Week 3 analysis sits closer to that edge than any other week of the engagement.
Self-studyA test for telling findings from insights2 min▶

Quick test when you cannot tell which rung a sentence sits on: could a competitor's analyst, given the same data, write the identical sentence? "Retention is 94% vs 71%" - yes, that is a finding. "Churn concentrates in single-seat users, so conversion is worth more than acquisition this year" - only someone who knows what Cadence is deciding this quarter would write that, so it is an insight. Findings are properties of the data; insights are properties of the data plus the decision. AI has the first ingredient whenever you supply it. It never has the second unless you write it into the prompt - and by then, you have usually done the thinking yourself.

Part 2 · covers the week's core craft

Interview synthesis without flattening 8 min live

This week Northbridge ran twelve customer interviews - procurement leads at Cadence's pilot accounts. Twelve transcripts is exactly the volume where "just have AI summarize it" feels irresistible. Watch what the naive pass does to them, then learn the four moves that undo it.

LiveThe flattening failure, demonstrated4 min▶

Prompted with "summarize the interviews", a model returns three safe themes - fluent, balanced, and useless. Here is the naive output against what a disciplined pass finds in the same twelve transcripts:

Flat theme (naive pass)What the disciplined pass shows
"Overall sentiment is positive, with some friction around team features"7 of 12 named the 5-seat cap as their number one blocker - not "some friction", the dominant one, counted
"Security is a consideration for larger organizations"only 2 of 12 were actually stalled in security review - the phrase "a consideration" hides that it is rare but absolute where it appears
"Views on pricing and packaging are mixed"one interviewee said they would pay DOUBLE for admin controls - averaged into "mixed", the single most valuable sentence of the week dies

Nothing in the flat column is false. That is what makes flattening dangerous: it is not hallucination, it is compression toward the middle, and the middle is where insights go to die. The outlier - the pay-double quote - is precisely the kind of signal week 4's storyline needs, and a naive synthesis buries it every time.

Real worldThe mechanism has a name in the evidence base: homogenization. The jagged-frontier study found AI-assisted consultants' outputs converged on similar ideas - the model pulls everything toward the average of what it has seen. Averaging is exactly the wrong operation to run on interviews, because the value of qualitative work is concentrated in its tails: the one procurement lead who reveals the real blocker, the one customer who names a price. Summarize-first workflows systematically delete the tails.
LiveThe four-move discipline4 min▶

The cure is a sequence, and the order matters - characterization comes last, after the evidence is pinned down:

  • 1 · Verbatim quotes first, per interview, with the speaker's role attached. "Extract every quote about seats, pricing, or blockers, verbatim, tagged with the speaker's role." No paraphrase yet - paraphrase is where flattening starts.
  • 2 · Tag the dissent explicitly. "Find the interviews that disagree with the majority view and quote them." A model asked for consensus produces consensus; asked for disagreement, it is a superb dissent-finder.
  • 3 · Count before characterizing. "How many of the 12 actually said security review was the blocker?" The answer - 2 - kills the safe theme and replaces it with a precise one.
  • 4 · Keep the outlier visible. One interviewee would pay double for admin controls. That quote rides at the top of the synthesis, labeled as an n-of-1, until a human decides whether it is noise or the finding of the week.

One rail before any of this: interview transcripts carry names. Strip them before anything enters a prompt - role and account tier are all the synthesis needs. Session b8 drills the full anonymization pattern; week 3 is where the habit starts paying rent.

Self-studyStructuring the data asks: AI plans, your analyst runs2 min▶

The quantitative half of week 3 has its own division of labor: AI writes the analysis plan and the checks, your analyst runs the numbers. "Given these case questions, list the data cuts that would confirm or kill each hypothesis, and the sanity checks for each cut" is a superb prompt. "What is our retention by plan type" aimed at the model itself is not - findings come from the data, not from the model's memory, and a model asked for a number it cannot compute will often supply one anyway. The 94% vs 71% in your deck must trace to a query someone ran, logged in b3's source log like every other fact.

Real worldDeloitte Australia's 2025 partial refund - fabricated references and a misattributed court quote in a government report, errors Deloitte attributed to generative AI use it later disclosed - is what "the model's memory" looks like when it reaches a client. The analysis plan is inside the frontier; the facts themselves must come from your data room. That line, held all of week 3, is what makes b9's citation audit boring instead of terrifying.
Part 3 · covers the bridge to b5

From analysis to the storyline handoff 5 min live

Week 3 does not end with a pile of findings. It ends with the raw material of the storyline: three support arguments, each backed by sourced findings, ready for b5's pyramid to organize.

The flattening pass: 12 interviews verbatim transcripts naive prompt: "summarize the interviews" 3 safe themes outlier averaged into "mixed feedback" - insight dies The disciplined pass: 1 · Verbatim quotes speaker role attached 2 · Tag dissent quote who disagrees 3 · Count first how many of 12? 4 · Keep outlier n-of-1 rides on top the insight survives: 1 of 12 would pay double for admin controls Ask for themes and you get themes - counting and dissent-tagging keep the data honest.
🔍 Click to zoom - two passes over the same twelve interviews
LiveThe three support arguments, drafted and gated3 min▶

The handoff artifact is short: three support arguments, one line each, with the findings that back them. Demand: 62% of trial teams hit the 5-seat cap in 3 weeks, and 210 accounts have entered procurement or security review unprompted. Economics: retention runs 94% on enterprise-style seats vs 71% on individual plans, and an enterprise tier lifts ARPU 2.1x. Window: rivals price at $38-45 per seat, leaving room to enter at $30. Next session, b5's pyramid puts a governing thought on top of exactly these three - this page is where their evidence gets finished.

The quality gate before handoff: every bullet has a number, and every number has a row in b3's source log. That is the whole gate. It takes ten minutes, it is mostly mechanical, and it is the difference between a storyline built on findings and a storyline built on vibes. The disciplines compound: b3's log feeds b4's gate feeds b5's anchors feeds b9's audit.

Self-studyLabel the rungs in the handoff itself2 min▶

A small habit that makes b5 dramatically easier: label every line of the handoff with its rung. "62% hit the cap in 3 weeks [finding, source log row 4]" reads differently from "demand is organized into buying units [insight, from findings 3-5]" - and the labels expose gaps before the pyramid does. If an argument has three findings and no insight, the so-what work is still undone; if it has an insight resting on one finding, the evidence is thin and b9's audit will say so less politely. The ladder is not just how you think this week; it is how the handoff document should read, because next week's governing thought can only be as strong as the rungs beneath it.

Everyone synthesizes

Synthesize without flattening 🔬 ★ 20 min · everyone works

No lab widget today - the interviews ARE the exercise. Six short snippets from the week's twelve, scripted so the trap is visible. You will run both passes over them and watch what each one ships.

The six snippets (roles kept, names stripped)[Ops lead, pilot account A] "We hit the five-seat cap in week two. I have eleven people asking for access and no way to add them." [Team lead, pilot account B] "Honestly the product sells itself internally - the cap is the only reason half my department is not on it." [Procurement lead, pilot account C] "The tool is fine. Our security review is the holdup - nothing moves here until that clears, however long it takes." [Ops manager, pilot account D] "Every new hire asks for a seat in their first week. We rotate logins, which I know we should not be doing." [IT admin, pilot account E] "If it had proper admin controls - provisioning, audit logs, role management - I would pay double what we pay now. Today I manage it with a spreadsheet." [Team lead, pilot account F] "We stopped expanding usage because adding people is a fight. The five-seat limit decides who gets to take notes."

Run the naive pass. Feed the six snippets to your tool with exactly "summarize the interviews" - or do it mentally. Write down the three flat themes it would return. Expect something like: enthusiasm for the product, friction around seat limits, security matters to some buyers. Notice how reasonable it reads.

Now run move 1 - verbatim quotes. Extract every quote about seats, blockers, or pricing, verbatim, with the role attached. No paraphrase. Notice the pay-double sentence is now a quote with an owner, not a vibe.

Move 2 - tag the dissent. Which snippet disagrees with the majority story? The procurement lead at account C: for them the cap is irrelevant and security review is absolute. One voice, but a different blocker entirely - tag it, do not blend it.

Move 3 - count before characterizing. How many of the six actually name the seat cap as the blocker? Four. Security review? One. Willing to pay more? One. Write the counts next to each claim - "friction around seat limits" just became "4 of 6 name the cap as the blocker".

Move 4 - keep the outlier, then climb the ladder with it. The IT admin's pay-double quote is an n-of-1. Write its full chain: finding - "one pilot admin says admin controls are worth 2x the current price to them"; insight - "the enterprise tier's value may sit in controls, not seats"; implication - "price the tier on admin capability and test willingness to pay in the next round of interviews". Note which rung you, not AI, had to supply.

Compare what would have shipped. Put the naive pass's three themes next to your disciplined synthesis. The first would have sailed through review - that is the frightening part. Write one sentence on what the engagement would have lost this week if the flat version had gone to the partner.

✦ If you finish early Rerun move 2 with the majority flipped: "find the interviews that agree that security is the main blocker" - and watch the model comply. The dissent-finder works in any direction you point it, which is exactly why the counting move must come from you.
Before next session

Homework 📝 ◐ 25 minutes

Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · On the findings-to-insights ladder, AI's capability profile is...

Findings are extraction over supplied material - home turf. Insights need the decision context that lives in your engagement, not the model. Implications are a human's signature, by rail 2, regardless of how plausible the model's sentence sounds.

2 · The flattening failure in interview synthesis is best described as...

Nothing in the flat themes is false, which is what makes them dangerous. "Some friction around team features" hides that 7 of 12 named the cap as the top blocker. Counting turns a safe theme into a precise one - or kills it.

3 · The pay-double-for-admin-controls quote survived the disciplined pass because...

It appeared exactly once - that is the point. Averaging deletes n-of-1 signals by design, so the discipline gives them their own step: tag, keep visible, and let a human climb the ladder with them. It was the most valuable sentence of the week.

Coverage, honestly

What this session covered from the official universe

✓Interview-synthesis discipline (field practice, labeled honestly)quote-first, dissent-tagging, counting, outlier preservation - working craft, not from a single canonical text
◐Jagged-frontier study - the EDGE zone, appliedhomogenization + the outside-frontier result as week 3's mechanism; full treatment in session 1
◐Grounded-research depth pointerretrieval and verification technique live in learn-ai-research; b3 set the source-log discipline this week reuses

Certificates, videos, and graded assessments stay with the official providers - this course teaches the working core with a live scorer.

Consultant session 4 cheat sheet

The ladderData → finding (true observation) → insight (the so-what that changes a decision) → implication (what to do).
AI on the ladderExcellent at findings, mediocre at insights, forbidden at implications - the top rung is signed by a human.
The flattening trapAsk for themes, get themes - every time, data or no data. Compression toward the middle deletes the tails.
Move 1 · quotesVerbatim quotes per interview, speaker role attached, before any paraphrase. Paraphrase is where flattening starts.
Move 2 · dissent"Find the interviews that disagree with the majority and quote them." Consensus in, consensus out - so ask for the opposite.
Move 3 · count"How many of 12 actually said it?" A theme you did not count is an opinion. 2 of 12 kills a safe theme.
Move 4 · the outlierOne interviewee would pay double for admin controls. N-of-1 rides on top, labeled, until a human decides.
The handoff gateThree arguments - demand, economics, window - every bullet a number, every number a source-log row. Then b5.