vLLM v0.20.1 cuts CPU bottlenecks, adds DeepSeek V4 support and faster GEMM/FlashInfer

LLM serving is usually sold as a GPU problem. But a lot of the pain is actually in the plumbing around the GPU — scheduling, communication, kernel choices, and the ugly edge cases that only show up in production. That’s why vLLM v0.20.1 matters. It is not a flashy rewrite. It is a patch release, published this week, that tightens the hot path for DeepSeek V4 and fixes bugs that can quietly wreck throughput or reliability under load. (github.com) ### What actually shipped? vLLM says v0.20.1 is a patch on top of v0.20.0, and the center of gravity is clear: DeepSeek V4 stabilization, performance improvements, and bug fixes. The release adds DeepSeek V4 Base support, multi-stream pre-attention GEMM, a configurable knob for that GEMM path, a tuned default threshold for when it kicks in, and BF16 plus MXFP8 all-to-all support for FlashIn(github.com)4 conversion path and optimized tile kernels for head computation. (github.com) ### Why do those changes matter? Because serving speed is often death by a thousand tiny stalls. Pre-attention GEMM is one of those spots where CPU coordination and launch overhead can eat into the gains you thought you were getting from big accelerators. Multi-stream pre-attention GEMM is basically vLLM trying to keep more work in flight instead of letting the front end dribble commands in(github.com)kernel library built for inference workloads, including attention, GEMM, and MoE paths, so better integration there can move real latency numbers, not just benchmark vanity metrics. (github.com) ### What was broken before? The most concrete production fix is a persistent TopK cooperative deadlock at TopK=1024, plus an inter-CTA initialization race in RadixRowState. vLLM temporarily disables persistent TopK as a workaround while keeping the fix set in the patch. There are also fixes for AOT compile cache import errors, a torch inductor error, repeated RoPE cache initialization, and (github.com)eek V3.2 and V4. None of that is glamorous. All of it is exactly the kind of thing operators care about. (github.com) ### Why is DeepSeek V4 the focus? Because DeepSeek V4 is a weirdly demanding target. vLLM’s own DeepSeek V4 write-up says the family includes a 1.6T-parameter Pro model and a 285B Flash model, both with up to 1 million tokens of context. Supporting that cleanly is not just “add a model name to a list.” It means getting new attention behavior, memory movement, and long-context scaling to beh(github.com) v0.20.1 looks like the cleanup pass that makes that support less fragile. (vllm.ai) ### Where does DFlash fit in? This is the adjacent story, not the same release. DFlash is a speculative decoding system from UCSD’s Z Lab that uses block diffusion to draft an entire block of tokens in one forward pass instead of predicting them one by one. That matters because normal speculative decoding still has a serial bottleneck in the drafter. Google highlighted a TPU implementation on May 4, saying(vllm.ai)n-source vLLM TPU ecosystem and got 3.13× average tokens-per-second on TPU v5p, with peaks near 6× on harder tasks. (developers.googleblog.com) ### Is DFlash already a standard vLLM feature? Not really — at least not in the clean “pip install stable release and use it” sense. The DFlash repo still describes vLLM support through a modified installation path tied to a pull request, while saying broader integration is in progress. So the signal here is strong performance potential, especially at lower batch sizes where parallel hardware gets underused, but the packaging still looks early. (github.com) ### Why should anyone outside infra care? Because this is where user-visible latency gets won. If you run a single-user assistant, these optimizations can make replies start faster. If you run a multi-tenant service, they can keep throughput from collapsing when workloads get messy — long context here, MoE there, weird sampling settings somewhere else. Basically, v0.20.1 is the kind of release that makes(github.com)Flash hints at a bigger jump if block-diffusion speculation becomes easier to deploy. (github.com) ### Bottom line The news is not that vLLM reinvented serving this week. The news is that it tightened the boring, expensive parts — and those are usually the parts that matter most. DFlash is the more dramatic speedup story, but v0.20.1 is the thing teams can use right now. (github.com)