RDMA delivers sub-10µs latency

RDMA, short for remote direct memory access, is being discussed by datacenter engineers as a way to move data with very low latency inside tightly controlled networks. NVIDIA says its RoCE stack is designed to deliver “ultra-low latency” for performance-critical applications, while Microsoft says RDMA-capable adapters can run at full speed with lower latency and without the same CPU cost as conventional networking paths. (docs.nvidia.com) The technology is not a new consumer networking feature. (nvidianews.nvidia.com) The InfiniBand Trade Association maintains the InfiniBand and RoCE specifications, and RDMA has long been used in high-performance computing, storage and database environments where direct memory operations matter more than broad compatibility. (infinibandta.org) NVIDIA’s April 27, 2020 completion of its $7 billion Mellanox acquisition is part of the history behind that discussion. NVIDIA said at the time that Mellanox’s networking technology would be combined with its computing portfolio for next-generation data centers. ### Why does RDMA get talked about in microseconds instead of just bandwidth? (docs.nvidia.com) RDMA is designed around direct placement of data into application memory, which reduces software overhead from kernel networking and repeated copying. Microsoft’s SMB Direct documentation says RDMA-capable adapters can deliver lower latency while preserving throughput and reducing CPU use, and that behavior is why engineers often describe the benefit in microseconds rather than only gigabits per second. (infinibandta.org) NVIDIA’s RoCE documentation makes the same case from the adapter side. The company says ConnectX Ethernet adapters use hardware offload to provide low-latency transport for financial, database, storage and content-delivery workloads. (nvidianews.nvidia.com) ### What exactly is the fabric here — InfiniBand, RoCE, or something else? The InfiniBand Trade Association says it maintains both the InfiniBand and RoCE specifications. In practice, RDMA is the programming and transport model, while InfiniBand and RoCE are two of the main ways operators deploy it inside datacenters. RoCE runs RDMA semantics over Ethernet, and NVIDIA’s documentation says it does not require an InfiniBand subnet manager in the fabric. (learn.microsoft.com) InfiniBand uses its own fabric and is commonly associated with tightly tuned high-performance clusters. That distinction matters because operators choosing between them are usually deciding how much they want to preserve Ethernet tooling versus adopt a purpose-built interconnect. (docs.nvidia.com) ### Why is Mellanox part of this story? Mellanox was one of the most recognized names in high-performance interconnects before NVIDIA bought it. NVIDIA said in 2020 that the acquisition would join Mellanox networking with NVIDIA compute to improve performance and resource utilization in data centers. (infinibandta.org) ConnectX adapter cards remain part of that lineage. NVIDIA’s ConnectX-7 datasheet says the cards target HPC, AI and hyperscale cloud datacenters with ultra-low latency and up to 400 gigabits per second of throughput, while ConnectX-6 materials describe RDMA, in-network computing and in-network memory features. (docs.nvidia.com) ### Where does RDMA show up in real deployments? Microsoft names Hyper-V, SQL Server and remote file servers as examples for SMB Direct, its RDMA-enabled SMB implementation. The company says the feature can make a remote file server resemble local storage for those workloads. (nvidianews.nvidia.com) NVIDIA names financial trading, databases, storage and content delivery as examples for RoCE. Those are all settings where operators care about tail latency, CPU overhead and predictable packet handling inside a private network. That last point is an inference from the vendor workload lists and protocol design, not a separate vendor claim. (nvidia.com) ### Why isn’t this the default everywhere? Ethernet is broader and easier to buy in commodity form, but RDMA deployments usually require more deliberate hardware, software and network configuration. Microsoft’s documentation lists RDMA-capable adapters as a requirement for SMB Direct, and NVIDIA’s materials position RoCE for performance-critical applications rather than general-purpose office or home networking. (learn.microsoft.com) The result is that RDMA is usually chosen when memory access semantics, low jitter and low CPU overhead are design constraints inside an on-premises or tightly managed datacenter. That framing is an inference from the official specifications and deployment documents rather than a formal standard definition. (docs.nvidia.com) ### What should readers watch next? The InfiniBand Trade Association’s specifications and integrators’ lists are the clearest place to track which InfiniBand and RoCE implementations are current. Microsoft’s SMB Direct documentation and NVIDIA’s current ConnectX product pages are the practical references for how RDMA is being packaged for Windows storage stacks and modern datacenter adapters. (infinibandta.org) (learn.microsoft.com)