PyTorch 2.13 speeds Apple Silicon

PyTorch released version 2.13 on July 8 with a set of performance and platform updates that put unusual emphasis on hardware breadth. The most eye-catching change is on Apple Silicon, where the project said FlexAttention now runs on the Metal Performance Shaders backend and can deliver up to about 12x speedups over scaled dot-product attention, or SDPA, on sparse patterns. The release also includes updates tied to ROCm, Arm and Intel’s XPU stack, alongside changes for distributed training, memory efficiency and Python support. PyTorch said maintainers and contributors will walk through the release in a live Q&A on July 22 at 11 a.m. Pacific time. ### Why is the Apple Silicon change getting the most attention? Apple Silicon is the first place in the 2.13 notes where PyTorch is claiming a large, concrete inference-related jump tied to a specific workload. In the official release blog and GitHub release notes, the project said FlexAttention “lands on Apple Silicon (MPS)” with up to roughly 12x speedup over SDPA on sparse patterns. (pytorch.org) FlexAttention is PyTorch’s programmable attention interface, introduced earlier as a way to express different attention variants in ordinary PyTorch code rather than requiring a separate custom kernel for each pattern. The new 2.13 change matters because sparse attention patterns show up in long-context and structured model designs, where avoiding unnecessary attention computation can cut latency and memory pressure. (pytorch.org) ### What does “up to 12x” actually describe? The “up to 12x” figure is not a blanket claim about all model workloads on all Macs. PyTorch tied that number specifically to sparse patterns, and specifically to performance against SDPA on Apple Silicon through MPS. That makes it a workload-specific benchmark claim, not a general statement that every transformer model will suddenly run an order of magnitude faster on Mac hardware. (pytorch.org) Sparse attention matters because it changes which token pairs are computed, instead of treating every token as attending to every other token. When that structure matches the model’s workload, the compute savings can be substantial. PyTorch’s framing suggests the gain comes from making those sparse patterns practical on Apple’s backend rather than forcing developers onto more generic attention paths. (pytorch.org) ### What else shipped in 2.13 besides the Apple work? PyTorch 2.13 also added a deterministic backward path for FlexAttention on CUDA, which the project said is meant to support reproducible gradient computation. The release introduced a CuTeDSL “Native DSL” backend for Inductor, added nn.LinearCrossEntropyLoss to reduce peak GPU memory by up to 4x for large-vocabulary language-model training, and launched torchcomms as a new communications backend for distributed workloads. (pytorch.org) The same release notes point to broader platform coverage. PyTorch highlighted updates spanning ROCm, Arm and XPU, underscoring that 2.13 was not just a Mac-focused release but a cross-platform one aimed at training and inference users across AMD, Arm-based systems and Intel accelerators. ### Why does this matter for teams working on-device? (pytorch.org) Apple Silicon has become a common development target for local inference and prototyping because Macs are widely available inside engineering teams and share architectural traits with other Apple hardware. Faster sparse-attention paths can make it easier to test long-context or structured-attention models locally before moving them into a mobile or embedded deployment workflow. (pytorch.org) That is an inference from the release materials, not a statement PyTorch made directly. PyTorch has also been putting more attention on on-device AI more broadly through its ecosystem work, including recent ExecuTorch-related outreach on mobile builders. In that context, better Apple Silicon performance fits a larger push to make PyTorch tooling more usable beyond traditional data-center GPUs. ### Where can developers get the details on migration and tuning? (pytorch.org) PyTorch said in release materials that a live PyTorch 2.13 Q&A is scheduled for Wednesday, July 22, 2026, at 11 a.m. PT. The session is expected to cover the release and answer community questions, which is where developers are most likely to get practical guidance on migration choices, benchmark setup and performance tuning for the new features. (pytorch.org) (openworld.news)