AMD MI350P outpaces H200 40%

AMD’s MI350P matters because it is not just another giant AI chip for hyperscalers. It is a PCIe card — basically the kind of accelerator enterprises can slot into more conventional servers — and AMD is pitching it as a way to run modern inference workloads without redesigning the whole data center. (amd.com) That is the gap here. Nvidia has dominated the high end, but a lot of companies do not want NVL racks, exotic cooling, or a full infrastructure migration. On May 7, AMD launched the Instinct MI350P directly into that opening. (amd.com) ### What is the MI350P, exactly? The MI350P is AMD’s new Instinct accelerator in a PCIe add-in card form factor. AMD says it is built for standard air-cooled servers, supports up to eight cards per node, and is aimed at inference, RAG pipelines, and what it calls agentic AI deployments inside existing enterprise racks. (amd.com) That framing is the point — this is less “build a new AI factory” and more “upgrade the servers you already own.” ### Why is everyone talking about 40%? Because the social post is pointing at a real spec-sheet gap, but it is still a spec-sheet gap. AMD’s launch materials say MI350P delivers an estimated 2,299 TFLOPS and up to 4,600 peak TFLOPS at MXFP4, with native support for low-precision formats like MXFP6 and MXFP4 plus sparsity support for mainstream 8- and 16-bit precisions. (amd.com) Separate coverage comparing published specs put MI350P roughly 39% to 43% ahead of Nvidia H200 on FP8 and FP16 theoretical throughput. That is where the “about 40% faster” claim comes from — not from broad independent application benchmarks. ### So is it actually faster than H200? (amd.com) Maybe on some inference math, but that is not proven yet in the way buyers usually care about. The clean claim is narrower: MI350P appears stronger on theoretical low-precision compute in a PCIe card. Real-world inference depends on software kernels, memory behavior, batching, quantization path, and whether the model stack is tuned for ROCm or CUDA. A 40% paper win can shrink, disappear, or occasionally grow once you run an actual serving workload. ### Why does the memory number matter so much? Because inference is often memory-bound before it is math-bound. AMD gives MI350P 144GB of HBM3E and says up to 4TB/s bandwidth. (amd.com) Nvidia’s H200 is built around 141GB of HBM3E and 4.8TB/s. So AMD is basically matching H200 on capacity while fitting the product into the enterprise PCIe story. That means larger models, longer context windows, or fewer painful compromises on quantization and sharding for on-prem deployments. ### Why is PCIe the real angle? Because PCIe is the boring form factor that companies can actually buy around. The catch with top-end AI hardware is not just chip price — it is power delivery, thermals, rack design, and interconnect topology. (amd.com) AMD is saying MI350P can run as a dual-slot, air-cooled card in existing infrastructure, with a 600W envelope and even a 450W configurable mode for tighter deployments. That changes the conversation from peak prestige to practical rollout. ### Does this threaten Nvidia? In the broad market, not overnight. Nvidia still has the software lead, the ecosystem lead, and stronger mindshare for production inference. (amd.com) But MI350P gives AMD something it badly needed — a modern PCIe product that looks credible for enterprises that want big memory and strong inference throughput without buying into rack-scale Nvidia systems. If those customers care more about fit, cost, and control than absolute ecosystem comfort, AMD suddenly has a sharper pitch. ### What should buyers watch next? Independent benchmarks. That is the whole ballgame now. Buyers need apples-to-apples tests on real LLM serving, not just peak FP8 or FP16 tables. (amd.com) They also need to see software maturity, model support, and total system economics. The MI350P launch makes the claim plausible. Validation is the next step. ### Bottom line? The interesting part is not that AMD posted a bigger number. It is that AMD finally wrapped competitive AI silicon in a form factor enterprises can realistically deploy. If the real-world inference results hold up, MI350P could matter less as a benchmark trophy and more as the card that made on-prem AI upgrades feel normal again. (amd.com)