TorchTPU speeds PyTorch on TPUs

A Tensor Processing Unit is Google’s custom chip for the kind of math that trains and runs large language models, and a PyTorch program is the Python code many researchers already write for that job. Until now, getting those PyTorch programs onto TPUs usually meant going through a separate layer called PyTorch/XLA, with extra APIs and different execution rules. (developers.googleblog.com) That extra layer mattered because PyTorch is built around eager execution, which means operations run right away like pressing keys on a piano and hearing each note immediately. Older PyTorch/XLA often used lazy execution instead, which is more like writing a whole sheet of music first and only then sending it to the orchestra. (cloud.google.com) (github.com) Google’s new answer is TorchTPU, announced on April 7, 2026, as a stack meant to make PyTorch feel native on TPU hardware. Google says the basic promise is that a developer can point an existing script at “tpu” and keep the core training loop unchanged. (developers.googleblog.com) Under the hood, TorchTPU still uses the XLA compiler, which is the software that turns model code into instructions the chip can execute efficiently. The change is that TorchTPU tries to hide more of that compiler machinery instead of forcing users to manage it directly with TPU-specific habits. (github.com) (developers.googleblog.com) Google built this around what it calls an “Eager First” design, where operations can run in the normal PyTorch style and then get compiled in the background when that helps. In the October 2025 design proposal, the team described compiling individual operations, short sequences, or larger fused clusters asynchronously and caching the results so compile work can overlap with execution. (developers.googleblog.com) (github.com) The speed claim attached to the launch is the part hardware teams will stare at: Google says a “Fused Eager” mode can deliver 50 percent to more than 100 percent performance gains on some workloads. That means the same PyTorch code can move from “works on TPU” to “works fast enough to change the budget” without a ground-up rewrite. (developers.googleblog.com) This is not just about one chip running one model faster. Google says modern training jobs now span on the order of 100,000 accelerators, and TorchTPU is being built for that scale rather than only for single-machine experiments. (developers.googleblog.com) The hardware itself is one reason Google thinks this is worth the software work. In Google’s TPU systems, each host connects to multiple chips, and the chips are linked by an inter-chip interconnect in a two-dimensional or three-dimensional torus, which is a grid-like network built to move data across large clusters without ordinary data-center bottlenecks. (developers.googleblog.com) Google is also pitching TPUs as a full PyTorch destination, not just a training niche. Its TPU developer hub now highlights PyTorch for training and vLLM for serving, and Google’s current TPU docs still show the older PyTorch/XLA path for Cloud TPU virtual machines, which makes TorchTPU look like the next step in a longer migration away from TPU-specific friction. (cloud.google.com) (docs.cloud.google.com) The competitive angle is simple: Nvidia’s grip has never just been about chips, because developers stay where the software is easiest. If TorchTPU really lets PyTorch users switch to TPUs with a device change instead of a rewrite, Google is no longer asking teams to learn a new world before they can rent its hardware. (github.com) (developers.googleblog.com)