vLLM benchmark peaks at 178 tokens/sec on Nemotron‑3‑Nano‑Omni‑30B setup

- vLLM benchmarks on a Nemotron‑3‑Nano‑Omni‑30B setup showed a peak throughput of 178 tokens/sec at concurrency eight during a 50K‑context test on GB10 GPUs. - That peak was measured with a long 50,000‑token context and concurrency 8, highlighting decode/KV‑cache coordination as the bottleneck. - The results underline why production teams push speculative decoding, quantization, and prompt caching to raise real‑world LLM serving throughput. (x.com 1) (x.com 2)