GPU hours, and the narrow waist

The neocloud narrative says specialist GPU providers undercut the hyperscalers by two to three times. Check it against the rate card, all fetched 2026-08-07:

CoreWeave sits 11% below AWS, not 2-3x below. The real discounters on list are RunPod and Lambda. That does not make CoreWeave a bad buy - its value is in committed contracts and InfiniBand fabric, neither of which shows up in an on-demand rate. But if someone in your organization is proposing a neocloud on the grounds that it is dramatically cheaper, that claim does not survive the list price, and you should ask what the committed tiers actually are.

Both of these have put real numbers into real board decks. Neither is hidden - they are simply units nobody checks.

The general rule: before comparing any two GPU prices, write down the denominator of each one. Per GPU or per instance. Per hour or per month. With or without the interconnect. Two of the five providers above use a different denominator from the others, and nothing on their pages shouts about it.

List price is the worst rate available. Two commitments change the picture entirely, both fetched 2026-08-07:

• AWS 3-year All Upfront: $2.587/GPU-hour - 62% off on-demand. The 1-year All Upfront rate is $4.045/GPU-hour, 41% off.
• Azure H100 spot: $2.27/GPU-hour - which is below AWS's three-year reserved rate, from the provider with the most expensive list price on the chart.

Sit with that second one. Azure is 79% more expensive than AWS on list and cheaper than a three-year AWS commitment on spot. For interruptible batch work - backfills, embedding regeneration, offline scoring - spot capacity inside a contract you already have is frequently the cheapest compute available to you, and it does not require leaving your security perimeter or signing anything new.

The practical shape of a compute plan is therefore three tiers, not one: reserved for the steady-state baseline, on-demand for interactive work that cannot be interrupted, and spot for everything restartable. Teams that pick one tier and apply it to all workloads overpay in one direction or lose jobs in the other.

Two facts that should change how long you let a GPU cost model stand without review.

• AWS cut P5 pricing by 44%, effective 2025-06-01. Not a promotion - a rate-card change.
• Azure's $98.32/hour for 8xH100 is exactly AWS's pre-cut price. Azure never matched the cut. The most expensive bar on the chart is not a premium product; it is a stale number that a competitor moved away from.

And the market clearing price is not the list price at all. The SemiAnalysis GPU Price Index composite puts H100 at $2.82/GPU-hour (April 2026), down 57% from $6.62 in the second half of 2023. That is the dashed line on the chart, and every list bar except RunPod's sits above it.

The most-linked GPU pricing aggregator in this category labels AWS's 3-year no-upfront rate as "1-Year Reserved." AWS's actual 1-year rates are materially higher. The error is roughly 32%, and it sits on the single page most likely to be pasted into a planning document.

Nobody was being dishonest. Someone built a scraper, a label drifted, and thousands of people have since built cost models on top of it. That is how bad numbers propagate in this field: not through fabrication, but through a convenient intermediary that nobody re-checks against the source.

The rule that follows is short and worth writing on a wall: use first-party billing feeds. AWS Price List API, Azure Retail Prices API, the vendor's own published card. If a number in your business case came from an aggregator, it is unverified until you have fetched it yourself.

NVIDIA AI Enterprise is the one software rate card in this space published to the dollar. Prices from NVIDIA's own documentation page, updated 2026-06-08, re-checked 2026-08-07:

Two consequences worth more than the table itself.

• The break-even is a utilization number. $1.00/GPU-hour is $8,760 per GPU-year at full use. Against the $4,500 self-managed annual price, the two cross at roughly 51% annual utilization. Below that, the hourly cloud entitlement wins. Above it, buy the term. Now ask what your actual GPU utilization is - most organizations do not know, which means most organizations cannot answer this question.
• It is licensed per INSTALLED GPU, not per GPU used, with one licence per physical GPU on a multi-GPU card. An 8-GPU DGX B300 therefore carries $36,000 per year in software licensing before a single dollar of hardware. This is the line item first-pass budgets miss, and it does not scale down when the machines are idle.

One more planning number, because it turns a vendor conversation into arithmetic. The storage sizing rule is approximately 12.5 Gb/s of storage bandwidth per GPU, and it is linear - 16 GPUs need around 200 Gb/s. When a vendor tells you their array is fast enough for your cluster, that claim is now checkable in one line of multiplication.

Look at the shape of the traffic through it. Upstream is plural and messy - many experiments, many notebooks, many people, most of whose work will never ship. Downstream is singular and consequential - one artifact, serving real traffic, making real decisions about real people.

Anywhere a stack goes from plural to singular, you have a narrow waist. And the registry is the only point in the whole MLOps layer where all four governance questions can be answered at the same time:

• What is in production right now?
• Where did it come from?
• Who approved it?
• How do I undo it?

Ask those four questions of a training pipeline and you get two answers. Ask them of a serving endpoint and you get a different two. Only the registry holds all four.

MLflow deprecated Model Stages as of version 2.9.0, in favour of aliases plus tags. This is not a cosmetic API change - it is a design correction that any registry you build or buy has to get right.

What was wrong with stages. A stage encoded the environment into the model object itself. So "Production" meant one thing in one cluster and something else in another, and the word travelled with the artifact into contexts where it was false. Worse, a model could occupy exactly one stage, so you could not have two candidates under evaluation at once - which is the normal state of a healthy team.

What aliases fix. An alias is a pointer, decoupled from the artifact:

models:/MyModel@champion     # the version serving traffic today
models:/MyModel@challenger   # the candidate under evaluation

# promotion  = reassign @champion to the challenger version
# rollback   = reassign @champion back to the previous version
# the artifacts never move, and never change

Three things fall out of that. @champion and @challenger coexist, so a real evaluation is expressible. The pointer belongs to the environment, not to the model, so the same artifact can be champion in one place and unused in another without lying about itself. And rollback becomes a pointer reassignment rather than a redeployment - seconds instead of a release cycle, at the moment you least want a release cycle.

Tags carry the rest: approval status, evaluation results, ticket references, whatever your reviewers need. The general principle transfers well beyond MLflow: never encode an environment into an artifact. Point at artifacts from environments instead.

Most monitoring tools give you one bucket called "drift," and most teams therefore respond to two completely different problems with the same reflex, which is to retrain. Google's split is better, and it is the most immediately useful thing in this section:

Blurring these wastes weeks. The classic sequence: a model underperforms, the team retrains, it underperforms again, they retrain on more data, it underperforms again - and eight weeks later someone notices that a feature is scaled in the training pipeline and not in the serving path. Skew was present from the first deployment. No amount of retraining addresses a preprocessing mismatch.

The diagnostic is simple. Was it ever right in production? If performance was poor from day one, suspect skew and go read the serving code. If it was good and decayed, suspect drift and retrain.

Before you design around any managed feature, open its documentation page and look for exactly this class of sentence. It takes two minutes per component and it is the cheapest risk control in this entire course.

• "No longer open to new customers" - the component is in maintenance. Existing customers are usually safe for a while; new architectures should not depend on it.
• A rename or a repositioning - expect documentation churn and a possible change of strategic direction.
• No release notes in eighteen months - not an announcement, but the same signal arriving silently.

Then weight the finding by where the component sits. A stalled visualization widget is an inconvenience. A stalled governance component is a compliance gap that appears at your next audit, not at your next sprint review. Governance components deserve the most scrutiny and usually get the least, because they are the boring part of the demo.

Kubeflow's own 2023 community survey - 90 responses, so directional rather than authoritative, and say so whenever you quote it - reports:

• 49% in production. Roughly half of adopters had it running in production at the time.
• Pipelines used by 90% of respondents, by a wide margin the most-used component.
• Top gaps: documentation at 55% and installation at 39%.

The gap list is the interesting half. Documentation and installation are not model-quality problems or scale problems - they are platform operations problems, and they are exactly what you inherit when you adopt a Kubernetes-native MLOps stack.

There is a lighter path hiding in that 90% Pipelines number: most Kubeflow adoptions are, in practice, orchestration adoptions. If pipelines are what you actually want, ask whether you want them badly enough to run the cluster underneath them, or whether the orchestrator from b4 already covers the need.