AI Infrastructure Strategy: Hyperscaler vs Neocloud vs On-Prem
Mark aistrategy.guide as a preferred source Our AI strategy guides show up more often, with a Preferred badgeA decision framework for where AI workloads should run in 2026: hyperscaler, neocloud (GPU as a service) or your own hardware. How to place each workload, the contract terms that create lock-in, how to hedge capacity risk, and what to decide this quarter.
Most AI infrastructure decisions are framed as a vendor choice: AWS or Azure, CoreWeave or Lambda, buy the servers or not. That framing leads to bad contracts, because the real decision is per workload. A company running a customer-facing assistant, a fine-tuning program and an internal document search has three different infrastructure problems, and putting all three on the same provider under the same contract is how organisations end up overpaying for some workloads and under-protected on others.
This page gives the decision framework. The procurement read on individual GPU and inference platforms is in GPU and inference platforms; the run-it-yourself case is in private AI.
What are the three options, really?
Hyperscalers (AWS, Microsoft Azure, Google Cloud, Oracle) sell GPU capacity inside a full cloud: identity, storage, networking, security tooling, managed model APIs, and an existing enterprise agreement. You pay for that integration in price and, at times of scarcity, in access to the newest GPUs.
Neoclouds are providers built specifically for AI and high-performance workloads. Gartner, in a June 2026 forecast, expects them to capture 20% of a $267 billion AI cloud market by 2030. They sell GPU as a service with fast interconnects and fewer managed services. Prices are visible: Lambda’s public page on 20 October 2026 listed on-demand NVIDIA H100 SXM capacity at $3.99 to $4.29 per GPU-hour depending on instance size. The trade-off is that you build or buy the surrounding platform yourself.
On-prem or colocation means you own or lease the hardware and run it in your own facility or a colocation hall. You carry the capital, the utilisation risk, the power and cooling, and the operations team. In return you get full control over data, predictable unit cost at high utilisation, and no queue for capacity you already own.
How should each workload be placed?
Score each workload on four questions. The answers usually place it without much debate.
| Question | Points to hyperscaler | Points to neocloud | Points to on-prem |
|---|---|---|---|
| How predictable is demand? | Spiky, early | Large bursts (training runs, batch jobs) | Steady, high utilisation |
| How sensitive is the data? | Covered by existing cloud agreements | Contractually manageable, region-pinned | Cannot leave the perimeter |
| How much platform do you need around the GPUs? | A lot: managed APIs, data services, identity | Little: your team runs the stack | You run everything |
| How close must compute sit to other systems? | Data already lives in that cloud | Data can be moved for the job | Latency to plants, branches or devices |
A typical result for a mid-sized enterprise: model APIs and early pilots on the hyperscaler where the data already lives; fine-tuning and large batch jobs on a neocloud under a short reservation; a small owned inference footprint for the regulated workloads and the steady-state internal assistants.
Where does lock-in actually come from?
Lock-in in AI infrastructure rarely comes from the GPU itself. It comes from four contract and architecture terms, and all four can be negotiated before signing.
- Committed spend. Enterprise cloud agreements reward committed spend with discounts. Moving AI workloads off-platform can break the commitment math, so the discount quietly becomes the reason to stay. Model the commitment with and without the AI workload before you sign.
- Reserved-capacity length. Multi-year GPU reservations lock in today’s hardware generation and price. NVIDIA reported on 26 August 2026 that its next platform, Vera Rubin, was in full production; a three-year H100 or B200 reservation signed today will run alongside newer, cheaper-per-token hardware for most of its term. Ask for a refresh or swap clause.
- Data gravity and egress. Training data, embeddings and logs accumulate where the compute runs. Egress fees and re-indexing effort make the second move more expensive than the first. Keep a copy of training data and evaluation sets in storage you control.
- Managed services above the GPU. Proprietary model-serving, vector and orchestration services are convenient and sticky. Where possible, standardise on open serving stacks and open model formats so the workload can move.
The broader case for paying a small premium to keep a second supplier viable is argued in vendor optionality.
How big is the capacity risk?
Capacity risk has two sides. The first is your provider’s dependence on its biggest customers. CoreWeave’s IPO filing showed Microsoft accounted for 62% of its 2024 revenue; CoreWeave said in September 2025 that its contracts with OpenAI totalled up to about $22.4 billion. A provider that size will prioritise its anchor tenants when capacity is short, and a smaller customer’s allocation is what moves.
The second side is the provider’s own supply: GPUs, and increasingly power. A neocloud’s roadmap depends on data center space with an energisation date. Before signing, ask which facility your capacity sits in, whether it is already energised, and what happens contractually if it slips. The power question is explored further in Valar Atomics vs Radiant.
Hedges that work in practice:
- Two providers for anything customer-facing, with the workload tested on both. Price the second as insurance.
- Short terms for growth capacity, long terms only for proven base load.
- Service credits that matter. A credit of a few percent of monthly fees does not compensate for a launch slip. Negotiate capacity-delivery dates with remedies, including the right to terminate.
- Financial diligence on smaller providers, the same way you would on any critical supplier: funding, customer concentration, debt maturity.
What does the cost comparison actually look like?
The useful comparison is cost per unit of work (per million tokens served, per fine-tuning run, per document processed) at your expected utilisation, not the hourly rate. Three adjustments change most comparisons:
- Utilisation. Owned hardware costs the same whether it is busy or idle. At low utilisation, renting wins almost regardless of rate; at high, steady utilisation, owning can win.
- Price movement. Rental prices for a given GPU generation fall as newer hardware ships. AWS cut P5 (H100) instance prices by up to 45% in June 2025. A purchase or long reservation locks in the price on the day you sign.
- People. On-prem and neocloud setups need engineers to run the platform. Include them in the cost, or the comparison is misleading.
What should you decide this quarter?
- Inventory AI workloads and score each on the four placement questions. Most organisations have more distinct workloads than they think.
- Review every GPU commitment coming up for renewal in the next 12 months against the newest hardware generation and current list prices.
- Add three clauses to the next infrastructure contract: capacity-delivery dates with remedies, hardware refresh, and data egress terms.
- Name an owner for capacity risk, usually the CIO or CTO, with a quarterly view of which roadmap items depend on capacity you do not control.
For the engineering detail of cluster design, networking and serving stacks, ctaio.dev’s AI infrastructure hub covers the CTO view.
The verdict
There is no single right provider. Place each workload where its demand pattern, data sensitivity and platform needs fit, keep terms short where the hardware curve is steep, and make capacity delivery a contract term rather than a promise.
Sources & methodology
- Gartner: neocloud providers will capture 20% of the $267 billion AI cloud market by 2030 (23 Jun 2026)
- Lambda: pricing (checked 20 Oct 2026)
- AWS: up to 45% price reduction for Amazon EC2 NVIDIA GPU-accelerated instances (Jun 2025)
- The Register: CoreWeave rides AI wave with IPO filing, but its fate hinges on Microsoft (Mar 2025)
- CoreWeave: CoreWeave expands agreement with OpenAI by up to $6.5B (25 Sep 2025)
- NVIDIA Q2 fiscal 2027 results, Form 8-K exhibit (26 Aug 2026)
- Methodology: prices and figures from vendor pages, filings and named reports, checked 20 October 2026. Prices change often; check the vendor page before you sign. No provider paid for placement.
