Cloud GPU Prices for H100 Drop Below $3/Hour
The cost of cloud-based GPU computing for AI model training continues to shift, with recent benchmarks showing significant price differences. A market comparison reveals Lambda Labs offering NVIDIA H100 access at $2.99/hr, while the Vast.ai marketplace has A100s near $1.27/hr. AWS is also expected to slash its H100 prices by 44% in mid-2025.
- The NVIDIA H100 GPU, built on the "Hopper" architecture, offers significant performance gains over its predecessor, the A100 ("Ampere"). For training large language models, the H100 can be 2-3 times faster and provides up to 30 times faster inference performance. This is largely due to its dedicated Transformer Engine with FP8 precision, which the A100 lacks. - A single NVIDIA H100 GPU unit costs between $25,000 and $40,000 to purchase outright, depending on the specific configuration and vendor. This high upfront cost is a major driver for the growth of the GPU-as-a-Service (GPUaaS) market, which was valued at $6.4 billion in 2023 and is projected to grow at a CAGR of over 30%. - Cloud GPU pricing is influenced by several factors beyond the hardware itself, including the geographical location of the data center, local energy costs, and market demand. For instance, a 10% spike in H100 rental prices in early 2026 was isolated and not seen in other models like the A100, indicating specific supply-demand pressures. - The cost of training state-of-the-art AI models has escalated dramatically, making GPU rental costs a critical factor. While training the original Transformer model in 2017 cost under $1,000, training GPT-4 is estimated to have cost between $78 million and $100 million in compute alone, and Google's Gemini Ultra cost an estimated $191 million. - The competitive landscape for cloud GPUs is led by the "Big Three"—AWS, Azure, and Google Cloud—which collectively hold over 60% of the market. However, an influx of over 300 new, smaller providers in 2025 has intensified price competition, contributing to the overall drop in rental costs. - NVIDIA's next-generation "Blackwell" architecture is expected to influence the market further. The upcoming B200 and other GPUs may lead to further stabilization or slight declines in H100 prices as it transitions from a premium offering to a mainstream workhorse. - While the H100 offers superior performance, it also has a higher power consumption, peaking around 700W compared to the A100's 400W. However, because it completes tasks faster, the total energy consumed per workload can be more efficient, a key consideration for data center operations.
