vLLM-Omni Gets Major Multimodal Update

The vLLM-Omni update is built on a fully disaggregated architecture, separating the components of a multimodal pipeline (like encoders, the LLM core, and generators) into a graph of interconnected stages. This design allows for independent resource allocation and optimization for each part of the process, a significant departure from traditional, more rigid serving systems. This approach is particularly effective for complex "any-to-any" models that handle text, images, video, and audio. This disaggregated execution model has been shown to dramatically reduce job completion time (JCT). For instance, with the Qwen3-Omni model, vLLM-Omni can cut JCT by up to 91.4% compared to baseline Transformer implementations. The system employs per-stage request batching to maximize the utilization of underlying hardware resources. The project extends the core capabilities of vLLM, which is known for its PagedAttention algorithm for efficient memory management, to non-autoregressive models like diffusion transformers. This allows vLLM-Omni to efficiently serve not just text generation but also complex media generation tasks, such as those performed by models like Stable Diffusion 3.5 and the Bagel model, which now have tensor parallelism support. The rebase on vLLM v0.16.0 brings a host of upstream improvements, including better support for pipeline parallelism and asynchronous scheduling, which together can boost end-to-end throughput by over 30%. This version also enhances support for a wider range of hardware and quantization formats, including FP8 for KV caches, which is critical for managing the memory footprint of large multimodal models.