The rule for tonight
Only two kinds of evidence get on these slides. Numbers a company published on its own engineering blog about its own platform, and findings from an independent audit. Everything else - the vendor case study, the analyst statistic, the conference keynote - is labelled as what it is. That rule is restrictive enough that it removes most of the material usually shown in this session, and what survives is worth far more than what it replaces.
What actually shipped, with numbers 10 min live
Airbnb is the spine of this session because it is the only company that published the whole arc: three named artifacts, six years apart, on its own engineering blog, each one solving the problem the previous one left open. Read as a roadmap it is the most honest thing in this course, because it shows how long each step took and in which order the pain arrived.
LiveReading the arc as a plan, not a story4 min▶
Three things to take from the timeline, in the order a sponsor cares about them.
- The order was forced, not chosen. Discovery came first because people could not find tables. Metrics came second because once everyone could find tables, everyone reported a different number for the same question. Lineage and governance came third because once 12,000 metrics existed, somebody had to answer where each one came from.
- Minerva's scale is the argument for funding metric work. 12,000+ metrics, 4,000+ dimensions and 200+ data producers is not a YAML file somebody writes in a sprint. It is a standing function.
- The payoff line is Minerva's COVID dashboard: built in days, 11,000 views. That speed was only possible because the definitions already existed. The value showed up in a crisis, four years after the first investment - which is precisely the funding problem this course exists to name.
Three more builds, all with numbers from the companies themselves. Read the middle column as evidence of what these platforms are asked to carry, and the right column as the decision it should inform.
| Platform | Published numbers | What a leader should take |
|---|---|---|
| Uber Hudi 2026-01-16, own blog | 19,500 datasets · 350 logical PB · 6 trillion rows/day · 10 PB/day ingested · 4 million analytical queries/week | Open table formats are not an experiment. This is the format layer under a production business at the top of the scale range. |
| Uber Michelangelo own blog | ~400 active ML projects · 5,000+ models in production · 10M real-time predictions/sec at peak | 5,000 models in production is a registry problem, not a modelling problem. The governance layer is the constraint at scale. |
| LinkedIn OpenHouse own blog | 3,500+ managed tables · 550+ daily active users · 6+ months reduction in time-to-market for dbt · 50% less end-user operational overhead | The clearest published statement of what a platform layer buys: time, and someone else's operational load. |
Self-studyTwo contrasts worth knowing3 min read▶
Netflix does not publish an Airbnb-style portal. They publish Maestro for orchestration and Metaflow for the ML surface, with 3,000+ AI/ML projects on Metaflow, and no single unified all-persona front door. Two very capable companies, two different answers about whether one portal is the goal. If somebody tells you the unified portal is settled best practice, that is one company's choice, not an industry consensus.
Spotify Backstage is a category error if you cite it as a data platform. It is a software and service catalog, a developer portal for services and ownership. It is genuinely the canonical "one internal portal" pattern - but for engineering assets, not for data. The mistake is common enough in board decks that it is worth naming out loud before someone builds a data strategy on it.
The one case that publishes its cost 7 min live
Almost every migration story you will be shown publishes the benefit and hides the effort. GetYourGuide published both, on their own blog, which is why it is the case in this course. It is not the largest migration and not the most impressive saving. It is the only one you can plan against.
The move: Snowflake to Databricks, 750 tables.
The benefit: 20% operational cost reduction. A real number, and a modest one - note that it is not the 60-80% a vendor deck would headline.
The effort, published: proof of concept 1.5 months at 1 FTE, then migration 4.5 months at roughly 2 FTE. Six months of elapsed time and about ten person-months of work.
The quality bar they reported: 98%+ of queries working after migration, 72% completing under 10 seconds. They told you what did not work as well as what did.
LiveDo the arithmetic in front of your board4 min▶
This is the only case in the session where a sponsor can compute a payback period without inventing an input. Take your own platform's annual operational spend, apply 20%, and set it against six months of elapsed time plus about ten person-months of engineering. If the resulting payback does not clear your organization's hurdle, that is a finding, and it is a finding you reached with published numbers rather than a vendor's model.
Two cautions before it becomes a benchmark:
- 750 tables is a specific size. Effort does not scale linearly with table count, but it does not stay flat either. Your estimate should move with your estimate of your own sprawl.
- 20% is their saving on their workload. A migration between the same two products at a different company, with different query patterns and a different contract, has no reason to land on the same number.
What genuinely transfers is the shape: a defensible platform migration is a two-quarter project with a couple of engineers on it, and the honest headline is a fifth off operations rather than a transformation.
LiveThe genre test: read a vendor case study for what is missing4 min▶
Set GetYourGuide against the standard artifact. JPMorgan's data mesh case study, co-authored with AWS, is headlined "drive significant value" and contains zero cost figures, zero timelines, zero scale numbers and zero adoption numbers. Not weak numbers. No numbers.
That is not an accusation of dishonesty. It is a description of a genre. A co-authored case study is a marketing artifact with a customer's logo on it, approved by two communications teams, and the absence of numbers is the predictable output of that process. Treat it as evidence that a large regulated organization attempted a data mesh, and as evidence of nothing else.
The four-question test to run on any case study handed to you in a procurement cycle. Who published it - the customer, the vendor, or both? Does it state effort as well as benefit? Does it carry a date? Does any single number in it have a denominator? GetYourGuide passes all four. Most of what you will be sent passes none.
What actually failed 14 min live
This part started as a search for cancelled data platforms and returned a structural finding instead. No named company has publicly cancelled a data-platform program with disclosed losses in 2024-2026. Not one. That vacuum is not evidence that these programs succeed - it is evidence about who is forced to disclose.
LiveBirmingham City Council and Oracle Fusion: the audited one6 min▶
This is the strongest case available because it is independently audited, still unfolding, and documented in public council papers rather than in anyone's marketing.
- The cost trajectory: original estimate £19M in 2018 with a December 2020 go-live target. Approved budget £131M by 2024. Total forecast £144.4M through 2027/28, with a separate report putting the cumulative cost at £216.5M by 2026.
- The decision that did the damage: the system went live in April 2022 knowing it was unfit. Testing was incomplete because the build was too unstable to test.
- What broke: custom bank reconciliation failed, leaving the council unable to produce auditable accounts. The fraud-detection audit trail was disabled for 18+ months. And £2 billion in transactions were misallocated to the wrong fiscal years.
- The audit finding, which is the whole point: Grant Thornton's 66-page report concluded the failures were process failures - governance, technical oversight and vendor management - not resourcing failures and not technical failures.
Sit with that last line, because it is the one that applies to you. The technology was not the thing that failed. The governance around the technology was. Every layer of the stack you are funding has an owner question attached to it, and Birmingham is what "we will sort the ownership out later" costs when the number is audited.
LiveLidl and SAP: a data model, not a system6 min▶
Abandoned in 2018 after seven years. Roughly €201M budgeted, about €500M spent, and Lidl reverted to its legacy system.
The root cause is the reason this case is in a data course rather than an ERP course. Lidl runs inventory on purchase price. SAP for Retail assumes retail price. That is a conflict between one company's data model and one product's data model - a single modelling assumption, sitting under everything.
Lidl refused to change the process, so the software was customized to fit the model instead. The customization grew until it consumed the project. Seven years, half a billion euros, and a return to the starting point, over which price a stock figure is held at.
This maps exactly onto the choice every semantic layer forces on you. When you adopt a platform, you either conform your definitions to its model or you bend the platform to yours. Both are legitimate. What is not legitimate is choosing to bend without pricing it, because bending has no natural stopping point - each customization makes the next upgrade harder, and the project ends when the bending cost overtakes the benefit. Lidl found that point at roughly €500M. Ask which of your metric definitions are genuinely non-negotiable, and be strict, because everything on that list is a customization you have agreed to fund forever.
Side by side, the two cases fail differently and transfer differently. Neither root cause is a product choice, which is the reason both are in a course about platform funding.
| Birmingham City Council / Oracle | Lidl / SAP | |
|---|---|---|
| Money | £19M estimated (2018) → £131M approved by 2024 → £144.4M forecast through 2027/28; a separate report puts it at £216.5M by 2026 | ~€201M budgeted → ~€500M spent |
| Duration | 2018 to at least 2027/28, still unfolding | Seven years, abandoned 2018, reverted to legacy |
| Root cause | Process: governance, technical oversight, vendor management, per Grant Thornton's 66-page audit | Data model: inventory on purchase price versus a product assuming retail price |
| Transfers to you as | Every layer needs a named owner and an oversight cadence, decided before go-live rather than after | Conform to the platform's semantic layer or bend it - and price the bending, because it has no natural stopping point |
Self-studyWhat the vacuum means for your own reporting3 min read▶
If nobody publishes data-platform failures, two things follow for how you run yours.
- You cannot benchmark your risk externally. Any confidence interval you have been shown about data-platform success rates is built on surveys and predictions, not on disclosed outcomes. Treat the absence of a denominator as the default state of this evidence base.
- Your internal reporting has to do the job disclosure does elsewhere. That means publishing adoption per persona, cost per layer and the definition backlog to your own board on a fixed cadence - so that a failing program surfaces while it is still cheap, rather than at renewal.
Put plainly: the discipline auditors impose on ERP is the discipline you have to impose on yourself, because nobody is going to impose it on you.
Bring these to the roadmap workshop ◐ 30 min
- Write your own version of the Airbnb arc: which of the three problems - discovery, definitions, lineage - is currently hurting most in your organization? That is where the sequence starts, and it is rarely where the budget is pointed.
- Take the last vendor case study anyone sent you and run the four-question test. Note which of the four it fails.
- Estimate your GetYourGuide arithmetic: 20% of your platform's annual operational spend, against six months and about ten person-months. Decide whether that clears your hurdle before anyone demos anything.
- List the metric definitions your organization genuinely cannot change to fit a platform. Keep the list short and be honest about it - each entry is a Lidl-shaped commitment you are agreeing to fund indefinitely.
Official sources covered
Every number tonight came from a company's own engineering blog or from an independent audit, with the date attached. Vendor-authored material appears only as an example of its genre, labelled as such. This page covers:
Three questions before you go 🎯 ◐ 90 seconds
1 · Airbnb shipped Dataportal in 2017, Minerva in 2021 and Metis in 2023. What does that arc prove about sequencing?
Metis did not replace Dataportal, it absorbed its UI and added a metadata and lineage service. Each stage answered the question the previous stage made unavoidable, and Minerva's 12,000+ metrics and 4,000+ dimensions are what made governance a necessity rather than a preference.
2 · Why is GetYourGuide's migration the case study used in this course rather than a larger one?
20% is a modest saving, deliberately so. The unusual part is the cost side: almost nobody publishes it. Compare with JPMorgan's AWS-co-authored data mesh case study, headlined "drive significant value" with no cost, time, scale or adoption numbers at all.
3 · Grant Thornton's 66-page audit of Birmingham City Council's Oracle program found the failures were primarily what?
The program ran from a £19M estimate in 2018 to a £144.4M forecast through 2027/28, went live in April 2022 knowing the system was unfit, left the fraud-detection audit trail disabled 18+ months and misallocated £2 billion in transactions to the wrong fiscal years. The audit still landed on governance, not technology.