Datacenter NVLink touted for large‑model training
A social post this week argued datacenter NVLink remains superior for training large models compared with distributed edge compute — a hot‑take in ongoing infra debates. The claim reinforces interest in tightly‑coupled racks for model parallelism and high‑throughput jobs. (x.com)
Vadim (X handle @zacodil) posted an X thread arguing datacenter NVLink remains the better substrate for training very large models (x.com) NVIDIA documents that fifth‑generation NVLink plus NVSwitch can enable 72‑GPU all‑to‑all fabrics with roughly 1,800 GB/s per‑GPU links and about 130 TB/s aggregate bandwidth in NVL72 configurations (developer.nvidia.com) NVIDIA’s GB200 NVL72 rack was described as containing 72 Blackwell GPUs, 36 Grace CPUs and nine NVLink switch trays, and NVIDIA presented aggregate system figures of about 720 petaflops for training and 1.4 exaflops for inference for that design (datacenterdynamics.com) An independent technical write‑up comparing generations estimated a prior multi‑trillion‑parameter training run that used ~8,000 H100 GPUs and ~15 MW could be completed on ~2,000 GB200 GPUs using an NVL72‑style fabric at roughly 4 MW, illustrating NVLink fabric efficiency at scale (nextplatform.com) Surveys and recent IEEE/ACM papers on distributed edge training enumerate concrete limits for edge‑distributed model training—restricted memory, power budgets, heterogeneous accelerators, and high communication overhead that undermines synchronous model‑parallel scaling across geographically dispersed nodes ( ) Open fabrics are evolving—CXL and UALink advertise memory pooling and large fabric scale (CXL 4.0 increases link rates and UALink specifications target thousands of accelerators), but technical reviews warn they currently lack NVLink‑class low hop counts, topology‑aware all‑to‑all bandwidth, and a mature software stack for tightly synchronized LLM training ( ) NVIDIA’s NVLink Fusion and OCP MGX rack messaging targets hyperscalers with semi‑custom NVLink racks as the alternative to spreading model parallelism across edge sites, positioning tightly coupled rack fabrics as the practical path for high‑throughput, low‑latency LLM training today ( )
