learn-ai-infra-with-phoebe / Leader session 3 of 6
Learn AI Infra with Phoebe · Leader track · Session 3 of 6

Build, buy, or rent - and who wrote the answer

You are about to be handed a total-cost-of-ownership model. Before you read a single row of it, read the byline. Seven of the nine top-ranking public GPU TCO analyses are published by GPU cloud providers or infrastructure vendors, and every one of the nine concludes "rent." That is not a conspiracy and it does not make them wrong. It makes them a captured literature, and knowing that changes how you read every number in this session - including the depreciation assumption that quietly decides the whole answer.

🟡 Leader track CDAIOs · heads of data · platform sponsors No code, decisions only 45 min
0-3 · Welcome 3-18 · The captured literature 18-42 · Depreciation and the honest counterweight 42-45 · Q&A
Part 0

What this session is really about

Sessions a1 and a2 gave you the eight layers and the persona map. This one is the money decision that sits underneath all of them: do you own the metal, buy a managed platform, or rent capacity by the hour. The honest answer for your organization depends on numbers nobody has published neutrally, on an accounting assumption two of the largest technology companies on earth publicly disagreed about in the same year, and on a base rate for internal builds that is worse than most sponsors think. Those three things are the session. There is no calculator at the end, because a calculator would be the fourth vendor model.

Live - presented in session Self-study - read after class Official sources covered
★ What you walk out with today Three things you can use in a steering committee this week: the reason every TCO deck you have been shown reaches the same conclusion, the one assumption to interrogate first in any ownership model, and the published base rate for internal builds versus vendor purchases - with its sample size, so you quote it honestly.
Part 1 · read the byline before the spreadsheet

The finding is about the literature 15 min live

We went looking for a defensible build-versus-rent break-even point. What we found instead was a fact about who is allowed to publish one. Of the nine top-ranking public GPU TCO analyses, seven are published by GPU cloud providers or infrastructure vendors, and all nine conclude "rent." No peer-reviewed model and no neutral-consultancy public model exists. Teach that as the finding, because it is more useful than any break-even number those nine papers contain.

Nine top-ranking public GPU TCO analyses. Seven vendor-published. Nine conclude rent. 1 · GPU cloud provider · concludes rent 2 · GPU cloud provider · concludes rent 3 · GPU cloud provider · concludes rent 4 · GPU cloud provider · concludes rent 5 · infrastructure vendor · concludes rent 6 · infrastructure vendor · concludes rent 7 · infrastructure vendor · concludes rent 8 · unaffiliated write-up · concludes rent 9 · unaffiliated write-up · concludes rent Conclusion: RENT 9 of 9 · and 7 of them are vendor-published Independent or peer-reviewed model none found in the public literature Break-even claims of 50-70% sustained utilization are all vendor-sourced: directional, not measured.
🔍 Click to zoom - who published the nine analyses, and the box that stayed empty
LiveThe number everyone quotes, and where it comes from5 min▶

The commonly cited break-even for owning GPUs rather than renting them is 50-70% sustained utilization. If your fleet runs above that band year-round, ownership wins; below it, rent. It is a clean rule, it is easy to remember, and every source we could find for it is a vendor.

That does not make it false. Vendors have the best telemetry in the industry and a real interest in being right about capacity. But it means the number is directional, not measurement. Treat it the way you would treat a supplier's own estimate of how much you should buy from them: useful input, not evidence.

Two practical consequences for you in a steering meeting:

  • When someone puts a break-even threshold on a slide, ask for the source. If the answer is a vendor blog, the slide is a starting hypothesis, not a finding.
  • Ask what utilization your organization actually sustains today, measured, not planned. Almost nobody knows, and that ignorance is usually the real decision blocker rather than the threshold itself.
LiveThe critique that matters: on-demand is the wrong baseline5 min▶

Here is the single most useful thing in this session. Vendor break-even thresholds are almost always quoted against on-demand rental rates - the most expensive way anyone actually buys GPU capacity. Nobody running a sustained fleet pays on-demand for a year.

Re-run the same comparison against a committed rate and the ownership case degrades sharply. AWS's own Savings Plan feed (fetched 2026-08-07) prices p5.48xlarge at 3-year all-upfront as $2.587 per GPU-hour, against an on-demand rate of $6.88. That is a 62% reduction, and it applies to exactly the steady, high-utilization workload that ownership models are built to justify.

The point is not that reserved instances always win. The point is that almost no vendor analysis runs that comparison at all, and the omission is the tell. If someone shows you a build case and the rental column says on-demand, they have compared your best-case ownership scenario against the buyer's worst-case rental scenario.

The one question to ask "Which rental rate is in this model - on-demand, or a committed three-year rate?" If the answer is on-demand, the model has not been run yet. Ask for it again with the committed rate and see whether the recommendation survives.
Self-studyWhat a neutral-ish capex number looks like3 min read▶

The closest thing to an independent build-side number in circulation comes from SemiAnalysis, which prices a 1,024-GPU H100 cluster at roughly $35,375 to $39,750 of capex per GPU, all-in including network. Treat it as the best available anchor with a stated caveat: SemiAnalysis sells research to the industry it covers, which makes it independent-ish rather than independent.

Use it the way an auditor would use a comparable: as a sanity check on whatever number your own procurement process produces. If a build proposal lands far under that range, ask what has been left out of it - network fabric, power, cooling, and facility work are the usual omissions, and layer six of the stack is where they hide.

Two structural costs that rarely appear in a first-pass build case, both covered in detail in a4: per-installed-GPU software licensing, and the physical envelope. A DGX B300 draws roughly 14 kW in 10U. The facility constrains you long before the budget does, and facility work is a capital programme with its own timeline.

Part 2 · the assumption that decides the answer

Depreciation is contested, publicly 12 min live

Every ownership model rests on one input: how many years the hardware is useful. Change that number from three to six and you halve the annual cost of ownership without touching a single price. It is the most sensitive assumption in the model and the least discussed in the room, and the companies with the most GPUs on earth are publicly disagreeing about it right now.

Useful life is management judgement, not a physical constant operator running the fleet interested party bracketing the range 2 yrs 3 yrs 4 yrs 5 yrs 6 yrs Michael Burry ~$1.1B in puts 2 to 3 Nebius GPU cloud operator 4 years Amazon $700M income cut 5 years, cut from 6 CoreWeave GPU cloud operator 6 years, raised from 5 Nvidia sells the hardware 4 to 6 Meta extended its estimate in 2025 - the opposite direction, the same year, the same technology. The two interested parties bracket the range; the three operators sit between 4 and 6 years. Move 3 years to 6 and annual ownership cost halves without touching a single price.
🔍 Click to zoom - run your model at both ends of this range before you believe it
WhoUseful life assumedPosition and interest
CoreWeave6 years (raised from 5)GPU cloud operator · longer life flatters its own economics
Nebius4 yearsGPU cloud operator · same business, different judgement
Amazon6 to 5 years on a subset, effective 2025-01-01Shortened it, a $700M cut to operating income
MetaExtended its estimate in 2025Moved the opposite way, same year, same technology
Nvidia4 to 6 yearsLong-interested · sells the hardware being depreciated
Michael Burry2 to 3 yearsShort-interested · held ~$1.1B notional in puts
LiveAmazon and Meta moved in opposite directions in the same year5 min▶

Read the middle two rows of that table together, because they are the whole lesson. In 2025, under the same generation of accelerators, in the same market, with roughly comparable engineering sophistication, Amazon shortened its useful-life estimate on a subset of equipment from six years to five, taking a $700M cut to operating income - and Meta extended its estimate.

Both disclosures are public and both are defensible. That is exactly why it matters. It is proof, from the two organizations best placed to know, that useful life is management judgement, not a physical constant. There is no correct answer waiting to be discovered by better analysis.

So the question in your own model is not "what is the right depreciation schedule?" It is:

  • What schedule is this model using, and who chose it?
  • What does the recommendation look like at both ends of the published range - three years and six?
  • If the answer flips somewhere inside that range, the decision is being made by an accounting assumption rather than by an operational fact, and it should be escalated as such.

We are deliberately not settling this. A course that told you "use five years" would be adding a tenth captured source to a literature of nine.

Self-studyReading interested parties without dismissing them3 min read▶

Two of the six positions in the table come from parties with an obvious financial stake in the direction of their own argument. Nvidia, arguing four to six years, sells the hardware. Michael Burry, arguing two to three, held around $1.1B notional in put options while making the case.

The lazy move is to discard both. The better move is to notice that they bracket the range, and that the operators sitting inside that bracket - Nebius at four, Amazon at five, CoreWeave at six - land closer to each other than to either advocate. When interested parties on opposite sides define the outer bounds and disinterested-ish operators cluster in the middle, the middle is where your sensitivity analysis should live.

Carry the habit across to vendor selection generally. "Who benefits if I believe this?" is not cynicism, it is the same diligence you already apply to a supplier's delivery estimate.

Part 3 · the base rate you are betting against

The honest counterweight to building 12 min live

You came into this session ready to be sceptical of vendor TCO models, and you should be. Now here is the discipline applied in the other direction, because a leader planning to build a platform is also making a claim that needs evidence. The most-quoted study in enterprise AI right now has a headline finding everyone repeats and a second finding almost nobody does. The second one is about you.

Real world

MIT Project NANDA, August 2025. The headline: 95% of enterprise generative-AI pilots deliver no measurable profit-and-loss return. The finding that rarely travels with it: buying from specialist vendors succeeded around 67% of the time, while internal builds succeeded at roughly one third of that rate. The methodology is thin and must be stated every time it is quoted: 52 executive interviews, 153 leader surveys, and 300 public deployments. That is a small, non-random sample carrying an enormous amount of citation weight.

The finding that rarely travels with the headline 0% 50% 100% specialist vendor buy ~67% succeed internal build ~1/3 of that rate Quote it with the sample attached, every time: 52 executive interviews · 153 leader surveys · 300 public deployments The headline: 95% of enterprise generative-AI pilots show no measurable P&L return. A base rate is not a prohibition. It is the thing your build case has to name a reason to beat.
🔍 Click to zoom - a small non-random sample carrying enormous citation weight
LiveHow to quote this without becoming part of the problem5 min▶

The 95% number has been laundered into a permanent statistic in roughly twelve months, which is exactly how "85% of big data projects fail" happened a decade ago. This course refuses to cite that one at all, and the way to avoid repeating the mistake with NANDA is mechanical: state the sample size in the same breath as the number, every single time.

Say it like this: "MIT's NANDA study, which is 52 executive interviews, 153 leader surveys and 300 public deployments, found 95% of pilots showed no measurable P&L return." It takes four extra seconds and it does two things at once. It keeps you honest, and it signals to everyone in the room that you read the methodology - which is a more useful reputation than being the person with the scariest slide.

The buy-versus-build split deserves the same treatment, and it is the more actionable of the two findings. A roughly 67% success rate for specialist-vendor purchases against roughly one third of that for internal builds is a base rate. It is not a prohibition.

LiveWhat "know your base rate" means in a funding conversation4 min▶

Build proposals fail in review for one of two reasons: the sponsor has never heard the base rate, or the sponsor has heard it and cannot articulate why this build is different. The first is fixable in a meeting. The second is the actual work.

Specific reasons a build can beat the base rate, each of which is checkable:

  • No vendor fills the layer. Genuinely novel requirements exist. Most are not, but some are, and this is the strongest ground.
  • The chokepoint must be yours. If a layer's narrow waist determines your regulatory position or your negotiating power, owning it is a strategic decision rather than a cost decision, and it should be argued that way.
  • The economics genuinely invert at your volume. Provable with your own billing data, not with a vendor model.
  • You already run and staff the capability. A team that has operated this class of system before is the single strongest predictor available.

If none of those four apply, you are betting against a published base rate with nothing specific to point at. That is worth knowing before the money is committed rather than eighteen months after.

The honest read The lesson is not "never build." It is: know the base rate you are betting against, and be specific about why you differ. A build case that names its reason survives a change of sponsor. One that rests on preference does not.
Self-studyWhy "rent" and "buy" are not the same recommendation3 min read▶

The nine TCO analyses conclude "rent," meaning rent compute. The NANDA finding is about "buy," meaning purchase a finished capability from a specialist vendor. These get collapsed in conversation and they are different decisions on different layers.

Renting compute is a decision about layer six: capacity, utilization, and depreciation. Buying a capability is a decision about layers five, seven and eight: whether your organization builds the semantic layer, the gateway and the self-serve experience itself, or purchases products that fill those jobs. It is entirely coherent to rent all your compute and build your own semantic layer, or to own a GPU fleet and buy every application on top of it.

Session a6 turns this into a layer-by-layer sequencing decision. For now, hold the distinction: rent-versus-own is about the metal, buy-versus-build is about the capability, and the evidence bases for the two are completely separate.

Homework

Try it yourself - this week ◐ 30 min total

Source material

Official sources covered

This session is deliberately built on primary disclosures and first-party billing feeds rather than analyst commentary, and every source is labelled with its interest where it has one. All prices on this page were fetched on 2026-08-06/07 and go stale faster than anything else on this shelf. This page covers:

◐SemiAnalysis GPU cluster capex analysis (~$35,375-39,750 per GPU all-in)Independent-ish anchor · sells research to the industry it covers
✓Public depreciation disclosures: CoreWeave, Nebius, Amazon, Meta (2025)The contested assumption · Amazon 6→5yr, a $700M operating-income cut
✓MIT Project NANDA, August 2025 - 52 executive interviews, 153 leader surveys, 300 public deployments95% no measurable P&L return · ~67% vendor-buy success vs ~1/3 of that for builds
✓AWS Savings Plan rates, first-party feed (fetched 2026-08-07)p5.48xlarge 3yr all-upfront $2.587/GPU-hr vs $6.88 on-demand · full cost model in a4
Check yourself

Three questions before you go 🎯 ◐ 90 seconds

1 · Nine public GPU TCO analyses all conclude "rent." What is the finding a leader should take away?

The agreement is not independent confirmation, because the sources are not independent. That does not make them wrong - vendors have the best telemetry available - but it means the widely quoted 50-70% sustained-utilization break-even is directional, not measurement. The critique that matters is that those thresholds are quoted against on-demand rates rather than committed ones.

2 · Why does the depreciation schedule decide the build-versus-rent answer more than the hardware price does?

Doubling assumed useful life halves annual ownership cost without changing a single price. The published range runs from Burry at 2-3 years to CoreWeave at 6, with operators clustering between 4 and 6. Run your model at both ends of that range - if the recommendation flips inside it, an accounting assumption is making your decision.

3 · What did MIT's NANDA study find about buying from specialist vendors versus building internally?

This is the least-quoted finding of the most-quoted study, and it is the one that applies to a leader planning to build a platform. Always state the sample size with it - the same discipline that keeps the headline 95%-no-P&L-return figure from becoming a permanent laundered statistic. The honest read is not "never build," it is know the base rate you are betting against and be specific about why you differ.

Leader session 3 cheat sheet · pin this

The literature finding7 of 9 top-ranking GPU TCO analyses are vendor-published. All 9 conclude rent. No peer-reviewed or neutral-consultancy model exists.
The break-even number50-70% sustained utilization. All vendor-sourced, so directional, not measurement.
The critique that mattersThresholds are quoted against on-demand. Re-run at AWS 3yr reserved $2.587/GPU-hr vs $6.88 on-demand and ownership degrades sharply.
The omission is the tellAlmost no vendor analysis runs the committed-rate comparison. Ask which rental rate is in the model.
Depreciation rangeBurry 2-3 · Nebius 4 · Amazon 5 · CoreWeave 6 · Nvidia 4-6. Interested parties bracket, operators cluster in the middle.
The proof it is judgementAmazon shortened 6→5 in 2025 ($700M operating-income cut); Meta extended. Same year, same technology, opposite directions.
The NANDA base rateVendor buys ~67% success; internal builds ~1/3 of that. Sample: 52 interviews, 153 surveys, 300 deployments. Say the sample size.
The honest readNot "never build." Know the base rate you are betting against, and name the specific reason you differ.