NVIDIA Overhauls Edge AI Stack with Dynamo v0.9.0

- The removal of NATS and ETCD is part of a larger infrastructure simplification that replaces them with a new Event Plane built on ZeroMQ (ZMQ) for high-performance transport and MessagePack for data serialization. For Kubernetes-based deployments, Dynamo now uses Kubernetes-native service discovery, reducing the "operational tax" of managing separate clusters. - The enhanced multi-modal support introduces "Encoder Disaggregation," which splits the Encode/Prefill/Decode (E/P/D) pipeline. This allows the computationally intensive encoder stage for image or video data to run on a separate pool of GPUs from the prefill and decode workers, optimizing hardware use and preventing bottlenecks. - Dynamo is positioned as the successor to NVIDIA's widely adopted Triton Inference Server, designed to orchestrate and accelerate inference communication across thousands of GPUs in distributed environments. - The FlashIndexer component, included as a sneak preview in this release, is engineered to reduce latency in distributed Key-Value (KV) cache management. This is critical for performance when serving models with large context windows, as moving KV cache data between GPUs can be a slow process. - Multi-modal capabilities were expanded across three primary backends: vLLM, SGLang, and TensorRT-LLM. The update also adds first-class support for diffusion-based language models, allowing them to be served in the same deployment as autoregressive models. - This version introduces more intelligent scheduling and routing capabilities, including predictive load estimation using fractional decay and support for routing hints from external orchestrators like the Kubernetes Gateway API Inference Extension (GAIE).