Hyperscaler hardware bets

Amazon is trying to change a basic rule of cloud computing: instead of renting the same general-purpose servers to everyone, it wants customers to pick chips built for one job at a time. Its own Trainium chips now handle about 40% of inference compute on Amazon Web Services, the company said this week, which means a large share of the work of generating answers and predictions is already running on Amazon-designed silicon. (aboutamazon.com) Inference is the part of artificial intelligence that happens after training, when a model actually answers a question, ranks a product, or matches a rider with a driver. It is the always-on part of the system, so shaving cost and power there matters more than winning a one-time benchmark. (press.aboutamazon.com) Amazon’s pitch is that custom chips can do that job more cheaply than the graphics processing units that became the default during the artificial intelligence boom. Amazon Web Services says its Trainium2 instances deliver 30% to 40% better price performance than its graphics processing unit-based P5e and P5en instances for training and inference workloads. (aws.amazon.com) The other chip in this strategy is Graviton, Amazon’s central processor for everyday cloud work like moving data, running application logic, and handling web requests. Amazon said Graviton4 delivers up to 30% better compute performance than Graviton3, which is why customers can pair Graviton for the general plumbing with Trainium for the artificial intelligence-heavy parts. (press.aboutamazon.com) Uber is a clean example of that split. Amazon said on April 7 that Uber is moving more of its Trip Serving Zones system onto Graviton and has started pilot training some artificial intelligence models on Trainium, using one chip family for real-time production traffic and another for model building. (aboutamazon.com) Trip Serving Zones is the software layer that decides, in milliseconds, which driver sees which ride or delivery request in which area. Uber says that system handles millions of predictions and location updates fast enough to affect rider matching, delivery timing, and demand balancing in real time. (aboutamazon.com) That is the shift in this story: custom hardware is moving out of the lab and into live consumer services. Amazon’s Trainium page now lists customers using the chips for workloads like recommendation, search, language models, and real-time video, not just giant one-off training runs. (aws.amazon.com) Once cloud providers convince customers to use specialized chips, they need somewhere to put a lot more of them. Amazon said on April 9 that it will invest another $12 billion in Mississippi data centers, bringing its total planned investment in the state to $25 billion across Madison County, Warren County, and a new project in Clinton. (aboutamazon.com) Those buildings are not just warehouses full of servers. Amazon said the Mississippi build-out includes $300 million in grid improvements and a plan for the Canton campus to use 100% recycled wastewater for cooling by 2027, which shows how power supply and water systems are now part of the cloud business itself. (aboutamazon.com) Amazon also said the Mississippi projects are expected to support about 2,000 jobs, while the newest phase alone adds 800 jobs in Madison County and Clinton. The company is effectively building regional infrastructure for a future in which artificial intelligence demand is steady enough to justify chip design, power upgrades, and water recycling in the same package. (aboutamazon.com)