Buying GPUs outright is a capital decision most teams cannot justify and cannot execute — the hardware is expensive, the lead times are long, and the depreciation curve is brutal. So almost everyone leases. What surprises people is how much the same physical card costs depending on the contract wrapped around it.
Understanding that spread is the difference between an AI infrastructure budget that works and one that quietly consumes the runway.
What changed in 2026
- Neoclouds matured into a real category. GPU-specialist providers moved past being cheap capacity resellers and started competing on orchestration, storage throughput, and support quality.
- Reservation terms got shorter. Pressure from specialists pushed the large clouds toward more flexible commitment windows, which reduced the penalty for guessing wrong.
- Inference and training capacity diverged. Inference workloads increasingly run on different hardware and different contract shapes than training, and buying one plan for both became visibly wasteful.
- Interconnect became a headline spec. Buyers learned to ask about fabric bandwidth per node rather than only counting cards, after enough teams discovered their expensive cluster was network-bound.
Contract shapes compared
| Contract |
Relative price |
Interruption risk |
Right for |
| On-demand |
Highest |
None |
Bursty experiments, short jobs |
| Reserved, 1 year |
Roughly half on-demand |
None |
Steady baseline inference |
| Reserved, 3 year |
Lowest committed rate |
None |
Proven, stable, large workloads |
| Spot or preemptible |
Lowest overall |
High; can be reclaimed |
Checkpointed training, batch jobs |
| Serverless per-token |
Usage-based |
None |
Low or unpredictable volume |
The pattern most well-run teams settle on is a layered one: a modest reservation covering the baseline they are confident about, on-demand for the variable middle, and spot for anything that checkpoints cleanly. Reserving to cover peak is how budgets get destroyed, because peak capacity sits idle most of the month while billing continuously.
What actually determines your effective cost
Sticker price per GPU-hour is the least interesting number. Three others matter more.
Utilization. A reserved cluster at thirty percent utilization costs three times its headline rate per unit of useful work. Most organizations discover their real utilization is far below what they assumed, and fixing scheduling is cheaper than renegotiating the contract.
Interconnect. For distributed training, the fabric connecting nodes frequently determines throughput more than the accelerators do. A cluster with excellent cards and commodity networking will train slower than a cheaper cluster with high-bandwidth interconnect, and you will pay for the difference in wall-clock hours.
Data gravity. Egress fees and storage throughput are the line items that surprise people. If training data lives in one provider and compute in another, the transfer costs can erase the price advantage that motivated the split.
For most teams building products rather than models, the honest answer is that leasing compute is the wrong layer entirely — API access is cheaper until volume is very large. Our AI API cost comparison covers where that crossover sits, and AI cost optimization covers reducing spend before you consider infrastructure.
Common mistakes
- Reserving for peak. Reserve the floor. Burst above it with on-demand.
- Ignoring utilization before renegotiating. A better rate on capacity you are not using is still waste.
- Comparing providers on GPU model alone. Same card, different fabric, very different throughput.
- No checkpointing on spot. Spot only saves money if a reclaim costs you minutes rather than the whole job.
- Forgetting egress. Model artifacts and datasets move more than teams plan for, and egress is billed on every move.
FAQ
Is a neocloud safe to depend on?
Financially, do the diligence you would for any vendor holding your production workload. Technically, they are frequently excellent at raw GPU delivery and thinner on managed services, regions, and compliance certifications.
When does owning hardware beat leasing?
Roughly when utilization is high and sustained over multiple years, and you have the operations capability to run it. Below constant high utilization, leasing usually wins on total cost.
Do I need the newest accelerator generation?
For training frontier-scale models, generation matters enormously. For inference and fine-tuning, previous-generation cards are frequently the better value once the price gap is factored in.
How do I forecast capacity honestly?
Instrument first. Six months of real utilization data beats any projection, which is why short commitments early and longer ones later is the safer sequence.
Where to go next
For what the silicon is actually doing, read AI chips explained and HBM4 memory explained. If your goal is lower spend rather than more capacity, start with AI model routers explained.