learn-ship-platform-diagnosis-with-phoebe / Chairman session 3 of 3
Learn Ship Platform Diagnosis with Phoebe · Chairman track · Session 3 of 3

Decisions and the data we owe ourselves

Two sessions of findings end here, in a table. Every row has a recommendation, the number that will say whether it worked, the role that owns it, the date it is first reviewed, and one line written from 2027 explaining why it failed. Before the table, the harder part: the questions the data could not answer at all - why nothing changed on the platform when the numbers moved, who the guest customers were, what fulfilment actually cost - and the discovery that each of those gaps has a name on it. Not a system. A person who was never asked.

🟡 Chairman track Chairman · C-level · board · no code Decision focus 45 min
0-3 · Welcome 3-15 · What we cannot see 15-40 · Decisions with owners 40-45 · Q&A
Part 0

From findings to decisions with owners

Session c1 made the case for the whole history and handed you the map. Session c2 walked the break, the lag and the turn. This session is act six: decide and collect. Everything you heard becomes a row in a table with an owner, a number and a review date, because a recommendation without those three is a wish with a chart attached. The analyst track builds the same tables in code from the same notebooks, so when your team presents them, you will already know which row each chart belongs to.

Live - presented in session Self-study - read after class Sources covered
★ What you walk out with today The ownership map: every table the diagnosis needed, the domain that should own it, the contract it should honour, and the gap today. Three decision tables - business, tech, data to collect - each row with outcome metric, owner role, first review and pre-mortem. And a 90-day plan that starts with a field in the release gate, not a wiki page.
Part 1 · covers availability + identity + the login snapshot + the missing event log

What we cannot see, and who owns that 12 min live

Before any analysis ran, the analysts drew one chart: which table covers which month. Five years of order lines, sixty-nine days of logins, a promotion table that starts in 2025 while promo codes have been stamped on orders since 2020, and no record at all of what changed on the platform when. Each of those is usually filed as a technology gap. It is an ownership gap first: a table with no owner starts when a module launches and keeps whatever the default retention setting says.

Every gap has a name on it: the ownership map Table Owning domain Contract it should expose Gap today fact_transaction platform commerce order line with true cost, daily cost is a unit constant fact_login product analytics sessions kept five years 60-day retention, from 2026-07 dim_promotion two marketing teams promo_id resolvable for its life 2020-2024 ids orphaned dim_customer each merchant an id that survives guest checkout 6 to 12% of lines have no id dim_store/merchant merchant success open/close as events, not fields dead stores never closed event log eng + leadership dated, scoped, owned events nine remembered rows, no log Read left to right. A gap with no domain in column two is not a tech ticket; it is a hiring or a naming decision.
🔍 Click to zoom - table, domain, contract, gap. Name the domain and the gap stops being a technology problem.
Live The availability heatmap 3 min▶

One chart to draw before any other: tables down the side, months across the top, a filled cell wherever a table has rows. It takes an hour and it saves months, because it turns "we cannot answer that" from an argument into a picture. Read this one and you know immediately that any question about logins is a two-month question, any question about promotion types is a 2025 question, and any question about what changed on the platform has no table at all.

Timeline of what each table covers 2020 to 2026: orders and customers span the full period, promotions only from 2025, logins only 69 days, and the event log is nine remembered rows.
🔍 Click to zoom - full rows can answer five-year questions. The short rows can only answer the months they cover.
QuestionNeedsAnswerable spanVerdict
Revenue, cost, margin by server, merchant, productfact_transaction2020-01 to 2026-09yes, full history
Customer cohort retentionfact_transaction, customer idfull history minus guestsyes, with a growing guest blind spot
Login-to-purchase funnel, device, session depthfact_login2026-07 to 2026-09two months only - no trend, no season
Did a promotion type lift baskets or retention?fact_transaction x dim_promotion2025-01 to 2026-0920 months; earlier promo ids unresolvable
What changed on the product or infra when the numbers moved?known_events, 9 rows from memorysparseno - there is no event log
Fulfilment cost per ordernothing existsnoneno - cost is a unit constant per SKU
Product views, cart adds before 2026-07nothing existsnoneno
The rule behind the table A question a table cannot answer is filed as a data-collection recommendation, never silently dropped. The bottom three rows of this table are the first three rows of the collection list in Part 2. That is the whole mechanism: an honest "no" today becomes a "yes" in eighteen months only if somebody writes it down now.
Live Two identity gaps 3 min▶

Two kinds of row have no name on them. The first is the guest checkout: 6.3 percent of order lines in 2020 carried no customer id, 11.9 percent in 2026. Every retention number computed from this data excludes them, and the blind spot grows every year as more customers choose not to sign in. The second is the orphaned promotion: promo codes were stamped on orders from 2020, but the table that says what each code was starts in January 2025, so four years of promotions are ids that point at nothing.

Two panels. Left: share of order lines with no customer id rising from 6.3 percent in 2020 to 11.9 percent in 2026. Right: share of transactions whose promo id resolves to the promotion table, near zero before 2025 and near complete after.
🔍 Click to zoom - the guest line climbs, the promo line jumps in 2025. Both are ownership decisions nobody made.

Neither gap needs new technology. A hashed email or phone at guest checkout gives a customer an identity that survives not signing in, and it is a decision each merchant has to take for their own customers. A promotion table that keeps every code for its whole life is a decision the two marketing teams have to take between them. Ask who owns it before asking what it costs.

Live A funnel with a date on it 3 min▶

Sixty-nine days of login data, from July to September 2026, is all the behavioural history the platform has. Read it for what it is: a snapshot. Login to purchase converts at 16.8 percent on the key servers against 26.9 percent on the rest - consistent with everything c2 found, and a useful place to point product effort. What it cannot tell you is whether that gap is new, seasonal or permanent, because there is no earlier summer to compare with. A chart like this carries its date in the title so nobody quotes it as a trend.

Funnel bars for key servers and other servers over the 69 days with login data: login, viewed products, added to cart, purchased. Login to purchase conversion is 16.8 percent on key servers and 26.9 percent on the others. The title carries the date range.
🔍 Click to zoom - a snapshot with its date on it. Ten points of conversion gap is a finding; the trend is a question for 2027.
Real world A consumer app once reported a conversion collapse to its board from six weeks of analytics data. The collapse was the summer holiday, which the previous year's data would have shown in one glance - except the analytics tool had been switched in spring and the old data was gone. The board learned to ask how long the series was before asking what it said.
Self-study The event log that does not exist 4 min▶

Every long chart of the platform carries the same nine vertical lines: the fulfilment change, the merchant-success cut, a checkout release, a few pricing changes. Those nine rows were written down from memory during the diagnosis, because nowhere in the company is there a record of what changed on the platform and when. Releases live in a deployment tool that keeps ninety days. Incidents live in chat. Business decisions live in slide decks. None of it joins to a month on a revenue chart.

This is the highest-leverage gap in the whole diagnosis, and it is not a technology gap. An event log is a table with five columns - date, what changed, scope (which server, which merchant), owner, link to the decision - and it fails for one reason only: writing to it is nobody's job. Treat it as a data product with an owner and a contract. Engineering owns release and incident events. The leadership team owns decision events. The metric layer joins both to every chart automatically.

  • A gap is an ownership gap before it is a tech gap. Ask "whose table is this" before "which tool".
  • Memory is data too. The nine remembered rows are the first nine rows of the log. Start from them.
  • The log earns its keep on the next diagnosis. Every future "why did the number move" starts by reading it. Without it, every diagnosis starts from interviews.
Part 2 · covers business moves + technical fixes + data to collect, each with owner, number, date, pre-mortem

Decisions with owners 20 min live

Three tables. Every row has five columns and a row missing any one of them does not ship. The recommendation says what to do. The outcome metric says how you will know. The owner is a role, not a name, so it survives a reorganisation. The first review is a date in the diary before the work starts. And the pre-mortem is one line written from 2027 beginning "this failed because" - which, more often than not, turns out to be the real recommendation.

Impact against effort: where the first ninety days go Do first Fund and schedule Quick hygiene Collect when the pipe exists Event log field in the release gate Per-server margin wall, full history Guest identity by hashed email or phone Reverse or reprice S07/S08 fulfilment Restore VN merchant-success team Metric layer with frozen cohorts Duplicate guard on ingestion Store status as events Promo id kept for its whole life Delivery time and returns capture Competitor price references per line Support tickets joined to customers low effort high effort low impact high impact
🔍 Click to zoom - amber is where the first thirty days go. Navy needs a budget line. The bottom row is hygiene and patience.
Live Business: nine moves 7 min▶

The order matters. c2 established that the March fulfilment change reached the customer and the June merchant-success cut compounded it, so the first two rows are the two levers in that order. The next seven are the growth findings, each traceable to a chart you have already seen.

RecommendationOutcome metricOwner roleFirst reviewPre-mortem: it is 2027 and this failed because
Reverse or re-price the S07/S08 fulfilment decisionKey-server margin back toward 42 percent from 29.3; monthly gap to pre-shift margin from 8.4k toward zeroCOO, with the fulfilment lead60 dayswe renegotiated cost without repricing and the volume term moved instead
Restore Vietnam merchant success, with upskillingKey-server merchant churn hazard from 2.39 back toward 1.26 percent a monthHead of merchant success90 dayswe rehired but gave the team no leading-indicator list, so it measured funerals again
Merchant acquisition by server tier and vertical where retention is earned12-month merchant survival of new vintages by tierHead of merchant acquisitionQuarterlyquota stayed on onboarded count, so the team acquired everywhere
Customisation push: tooling and margin share for merchant SKUsMerchant-SKU share of revenue; customer repeat rate from 28.1 toward 33.9 percentHead of product, with merchant success90 daysthe tooling shipped but the margin share never did, so merchants had no reason to switch
Bundle and cross-sell programme from measured liftAttach rate of mug + t-shirt, poster + cushion, notebook set against today's baseHead of catalogue60 daysbundles were priced as discounts and cannibalised the second item
Upsell ladder inside each lineShare of orders on the top rung of the price ladder per lineHead of catalogueQuarterlythe ladder existed in the catalogue but not on the product page
Product diversification: bags up, stationery downCategory revenue share, bags and stationery, year on yearHead of catalogueQuarterlystationery stayed because a founder liked it
Per-merchant customer preference profiles handed to merchantsRepeat rate of merchants using the profile against those who do notHead of merchant success90 daysprofiles were a PDF once a quarter; nobody opened them
Promo policy: acquisition promos buy baskets, not customersEver-repeat of promo-acquired customers from 63.4 toward 71.5 percent; first basket held near 91.8CMOQuarterlythe promo budget was measured on first-order revenue and nothing else
Read the last column first Every pre-mortem in this table names a measurement or an incentive, not a technology. That is the pattern to expect in your own table. When the failure you can imagine is "nobody measured it", the fix is to move the metric into the owner's review, not to build anything.
Live Tech: eight fixes, one of them first 6 min▶

None of these needs a platform migration. Each is a table, a field or a rule, and each is owned by a domain from the ownership map. The first row is the one that pays for all the others.

RecommendationOutcome metricOwner roleFirst reviewPre-mortem: it is 2027 and this failed because
Release and incident event log as a data productShare of releases and incidents with a log row; target 100 percent within a quarterHead of engineering30 dayswriting to it was nobody's job; the fix was a required field in the release gate, not a wiki
Business decision log, same shape, same joinEvery pricing, fulfilment and staffing decision dated and scoped within a weekChief of staff30 daysdecisions were logged after the fact from slide decks and the dates were guesses
Guest-checkout identity by hashed email or phoneShare of order lines with no id from 11.9 percent toward the 2020 level of 6.3Head of product, per merchant60 daysmerchants saw it as friction and turned it off
Login history retained beyond 60 daysMonths of login history available; target the full five years going forwardHead of product analytics30 daysthe retention setting reset when the analytics tool was upgraded
promo_id resolvable for its whole lifeShare of stamped promo ids that resolve to a promotion rowPlatform and merchant marketing, jointly60 daystwo teams each assumed the other owned the table
Duplicate guard on order ingestionExact-duplicate lines per month; S02 2021 was 3.4 percentHead of data engineering30 daysthe guard ran on the warehouse, not the source, and a reload slipped past it
Metric layer computing cohorts with frozen historyZero restated retention numbers between two board packsHead of data90 daysteams kept their own spreadsheets because the layer was slower than a pivot
Store and merchant status as events, not overwritten fieldsActive-store count equals stores with an order in 90 days, within a few percentHead of merchant success60 daysthe dimension was fixed once and drifted again because closure was still manual
★ Why the event log is the highest-leverage row Every diagnosis you will ever run starts with one question: what changed when the number moved? Today the answer is nine remembered rows and a round of interviews. With a log, it is a join. The break in c2 took a team weeks to date; with an event log joined to the margin chart, the March 2024 line would have been drawn on the chart the month it happened, and the thirteen-point step would have had a name beside it before anyone asked.
Live Data to collect: eight things the business does not record 5 min▶

These are the questions the availability table marked "no". Each becomes a row with the same five columns, because collecting data nobody owns produces a table nobody trusts. The first row is the one that would have turned the cost story from an allocation into a fact.

CollectOutcome metricOwner roleFirst reviewPre-mortem: it is 2027 and this failed because
Fulfilment cost per order: shipping, supplier, returnsCost is a live per-order value, not a unit constant per SKUCOO, with finance90 dayssuppliers invoiced monthly in bulk and nobody allocated it back to orders
Product views and cart eventsFunnel available for every month, not 69 daysHead of product analytics60 daysthe events fired but were never joined to a customer or a server
Marketing spend by channel and serverAcquisition cost per customer and per merchant, per server - unknowable todayCMOQuarterlyspend lived in agency reports with a country column that did not match a server
Merchant acquisition funnel: pitched, trialled, onboardedConversion at each step by tier and verticalHead of merchant acquisitionQuarterlythe sales team logged onboarded merchants only, because that was the bonus
Merchant health signals: logins, catalogue edits, campaignsWeekly list of merchants below the month-three thresholds: 1.3 months to first sale, 399 in 90 daysHead of merchant success30 daysthe list existed and nobody was paid to call it
Delivery time and returnsDelivery days and return rate per server and per lineCOOQuarterlycarriers reported by country and the platform could not split it by server
Support tickets joined to customers and merchantsTicket rate per hundred orders, per serverHead of supportQuarterlythe ticket tool had no customer id field
Competitor price references per linePrice index per line against the market, refreshed quarterlyHead of catalogueQuarterlycollected once for a board deck and never again
Real world Marketplaces that grew across several countries routinely discover that cost of delivery is the one number nobody can produce per order. Suppliers invoice in bulk, carriers bill by lane, returns land in a different system. Margin per unit becomes an accounting allocation that stays constant for years while the real cost moves - which is exactly how a 22 percent cost shift can sit in the books as a decision nobody priced.
Self-study The promotion double edge 4 min▶

Promotions do two things at once and the company only measured one. Customers acquired through a promotion place a first basket of 91.8 against 84.7 for organic customers - the promotion works, on the day. But 63.4 percent of them ever order again against 71.5 percent of organic customers. Acquisition promos buy baskets, not customers. That does not make them wrong; it makes them a basket tool that should be measured as one, with the repeat cost written next to the basket gain.

Two pairs of bars comparing promo-acquired and organic customers: first basket 91.8 against 84.7, ever-repeat rate 63.4 against 71.5 percent. The promotion wins the first order and loses the second.
🔍 Click to zoom - bigger on the left, smaller on the right. Both bars belong on the same slide or the policy will be wrong.
First basket and repeat rate by promotion type for the twenty months since January 2025 when the promotion table exists, with the caveat that earlier promotions cannot be typed because their ids do not resolve.
🔍 Click to zoom - by type, but only from 2025. Four years of promotions are ids that point at nothing.

The promotion-type view is honest about its own limit: it covers twenty months, because the table that names each promotion did not exist before. That is the orphaned-id gap from Part 1 showing up as a shorter chart. Fix the ownership and the chart grows backwards.

Self-study Growth accounting as the monthly opener 3 min▶

If the monthly review opens with one chart, this is it. Active customers this month equal new plus retained plus resurrected; the churned bar below the line is what left. It cannot be gamed by a growth number, because growth is the top of the stack and churn is the bottom, on the same axis. Had it been on the wall in 2024, the key-server churned bar would have grown two quarters after the cost decision and someone would have asked why.

Monthly customer growth accounting for the whole platform 2020 to 2026: new, retained and resurrected customers stacked above zero, churned customers below. The churned bar deepens from late 2024.
🔍 Click to zoom - the identity every monthly review should open with. Read the bar below the line first.
  • Per server, then platform. The blended version is the one that hid the break. Ten small versions, then the total.
  • Merchants get the same chart. c2 showed the merchant version; both belong on the same wall.
  • Frozen history. Last month's bars do not change when this month's data lands.
Part 3 · covers the 30/60/90 strip + the pre-mortem

The 90-day plan and the pre-mortem 5 min live

Twenty-five rows is a plan for a year. Ninety days needs nine, and the first three need to be things a single person can finish without a budget cycle. Then, before any of it ships, the top three go through a pre-mortem: sit in 2027, assume it failed, and write the sentence.

Nine things in ninety days 30 days · no budget needed 60 days · the two levers 90 days · growth and cost 1. Event log field in the release gate 2. Per-server margin wall, full history 3. Guest identity by hashed email/phone 4. S07/S08 fulfilment decision reviewed 5. VN merchant-success team restored 6. Metric layer with frozen cohorts 7. First bundle live: mug + t-shirt 8. Merchant leading-indicator list 9. Fulfilment cost per order captured Each item carries an owner, a number and a review date before it goes on this strip.
🔍 Click to zoom - the first column costs nothing but a decision. The second is where the money moves. The third is the turn.
Live The pre-mortem on the top three 3 min▶

It is 2027. The event log exists. Nobody writes to it. Why? Because writing to it was nobody's job: it was a page in a wiki, engineers were asked nicely, and the third week of a busy quarter was the last time anyone opened it. The fix that survives the pre-mortem is a required field in the release gate - a deployment does not go out without a log row, and a decision is not minuted without one. The log fills itself because the thing people already have to do now produces it.

Top-three itemIt is 2027 and it failed becauseSo the recommendation becomes
Event logwriting was nobody's joba required field in the release gate and the decision minute, not a wiki page
Per-server margin wallit showed twelve months and the step scrolled offfull history by default; the window is a zoom control, never the default view
Guest identitymerchants saw friction and switched it offhash at checkout, invisible to the customer, with the merchant's own repeat rate as the reason
The pattern Every pre-mortem answer above is about incentive or default, not technology. That is what the exercise is for: it finds the recommendation hiding inside the recommendation. Run it on your own top three before the table leaves the room.
Build-along

Build the owner table for your business ★ 8 min · everyone builds

Take the three recommendations you would most like to make in your own company this quarter. Give each of them the five columns, in this order, and notice which column is hardest to fill - that is where the recommendation is weakest.

Write the recommendation as a verb. Reverse, restore, retain, hash, log. Not "review" or "consider" - those are ways of deciding later.

Name the outcome metric with today's value. "Repeat rate from 17.2 toward 28.8 percent", not "improve retention". If you do not know today's value, the first task is to measure it.

Name the owner as a role. Head of merchant success, not a person's name. Roles survive reorganisations; names leave and take the recommendation with them.

Put the first review in the diary now. Thirty, sixty or ninety days, with the people who can act in the room. A review booked after the work starts gets moved.

Write the pre-mortem. "It is 2027 and this failed because ..." One sentence. If the sentence names an incentive or a default, rewrite the recommendation to fix that instead.

Strike the row you cannot complete. A row with no metric is a wish; a row with no owner is a meeting. Keep the finding in the appendix and come back when you can fill all five.

The table in one line "Do X, measured by Y from today's value, owned by role Z, reviewed on date D, and it will fail because W - so we are also doing V."
Homework

Try it yourself - this week ◐ 30 min

Sources covered

What this session draws on

Turning findings into decisions borrows from decision science, data-product ownership, sales and growth operations, and one very useful exercise from product management. This page teaches the working core of:

✓Cassie Kozyrkov - Decision Intelligence: the decision column, the default action, the outcome metricPart 2 · every row's second column
✓Zhamak Dehghani - data as a product with a domain owner and a contractPart 1 · the ownership map and the event log as a product
✓Ayesha Khanna - owner and number on every recommendation, or it is pilot purgatoryPart 2 · owner role and first review on every row
✓Shreyas Doshi - the pre-mortem: it is 2027 and it failed, whyPart 3 · the fifth column and the top-three exercise
✓Mark Roberge - leading indicators over lagging ones; measure readiness, not the funeralPart 2 · merchant health signals in the collection table
◐Availability heatmap and question-availability tablePart 1 · read here; built in analyst session a1
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · A team proposes "launch a bundle programme" as a recommendation. What is missing before it can go in the table?

The finding is real and the decision is real. But a recommendation with no number cannot be judged, with no owner cannot be chased, and with no date will be chased never. Attach rate of mug + t-shirt against today's base, owned by the head of catalogue, reviewed in sixty days - now it is a row.

2 · The promotion table starts in 2025 while promo codes have been stamped on orders since 2020. What kind of gap is this, first?

Two marketing teams each assumed the other owned the promotion table, so it had no contract and no retention rule. Every gap in the ownership map has the same shape: the table did whatever its default did because no domain was accountable for it. Name the domain first; the technology is the easy part.

3 · The pre-mortem on the event log reads "it is 2027 and nobody writes to it". What does that sentence change?

The pre-mortem finds the recommendation hiding inside the recommendation. A log that depends on goodwill fails in the first busy quarter; a log that fills itself because a deployment cannot ship without a row survives. The answer to "why did it fail" is almost always an incentive or a default, and that is what gets rewritten.

Chairman session 3 cheat sheet · pin this

Availability before analysisTables down the side, months across the top. A question a table cannot answer becomes a collection row, never a silent drop.
Ownership mapTable, owning domain, contract, gap. A gap with no domain is a naming decision, not a tech ticket.
Five columns or no rowRecommendation as a verb, outcome metric with today's value, owner role, first review date, one-line pre-mortem.
The two levers, in orderReverse or re-price the S07/S08 fulfilment decision first; restore Vietnam merchant success second. c2 proved the order.
The event log firstEvery future diagnosis starts by asking what changed. Make it a required field in the release gate and the decision minute.
Identity gapsGuest lines 6.3 to 11.9 percent; promo ids orphaned before 2025. Hash at checkout; keep every promo for its whole life.
Snapshots carry their date69 days of logins: 16.8 vs 26.9 percent conversion is a finding; the trend is a 2027 question. Say so in the title.
The pre-mortem"It is 2027 and this failed because ..." The answer names an incentive or a default. Rewrite the recommendation to fix that.