vLLM‑Metal brings LLM serving to Apple
- The vLLM project’s Apple Silicon plugin, vllm-metal, shipped a v0.2.0 update in April that makes its unified paged Metal attention kernel the default backend. - The repository lists 974 GitHub stars and says v0.2.0 cuts time-to-first-token by 83x and lifts throughput 3.6x versus v0.1.0 on Apple Macs. - The plugin keeps vLLM’s API server and scheduler while using Apple’s MLX and unified memory underneath, extending vLLM beyond Nvidia-heavy setups. (docs.vllm.ai)
Large language model serving is the software layer that keeps a model loaded, schedules requests, and streams tokens back to apps. vllm-metal brings that server stack to Apple Silicon Macs. (docs.vllm.ai) The project sits under the vLLM organization as a community-maintained hardware plugin for Apple Silicon. Its GitHub page says the April 2026 v0.2.0 release made a unified paged variable-length Metal kernel the default attention backend. (github.com) In plain terms, attention is the part of a model that decides which earlier words still matter, and it is one of the expensive steps in generation. The new default kernel is the low-level code path that runs that step on Apple’s Metal graphics stack. (github.com) The repository says v0.2.0 improves time to first token by 83x and throughput by 3.6x compared with v0.1.0. The docs also say the plugin uses MLX as the primary compute backend and keeps PyTorch for model loading and interoperability. (github.com) (pypi.org) That split matters because vLLM already provides the pieces developers use in production: an engine, a scheduler, tokenizers, and an OpenAI-compatible API server. vllm-metal keeps those layers and swaps in Apple-specific execution underneath. (docker.com) (pypi.org) Apple’s hardware uses unified memory, which means the central processor and graphics processor can work from the same memory pool instead of copying data back and forth. The project says it uses that design for zero-copy operations on supported paths. (docs.vllm.ai) (pypi.org) The current scope is narrower than the headline suggests. The supported-models page says vllm-metal is focused on text-only language models, and multimodal models with image or audio input are not yet supported. (github.com) The support table names Qwen3, Qwen3.5, Qwen3.6, Qwen3-Next, Gemma 4, Mistral-Small-24B, GPT-OSS, and GLM-4.7-Flash among the models being supported or tested. Llama 3 and Gemma 3 are listed as not yet verified rather than fully supported. (github.com) The project is also starting to show up in developer tooling beyond GitHub. Docker said on February 26, 2026 that Docker Model Runner added vllm-metal support on macOS, letting M-series Macs expose the same OpenAI-compatible and Anthropic-compatible interfaces through Docker workflows. (docker.com) So the story is less “LLMs now run on Macs” than “the same serving layer used in data-center workflows is being adapted to Macs.” vllm-metal does that by keeping vLLM’s server machinery intact and pushing Apple-specific inference work down into MLX and Metal. (docs.vllm.ai) (docker.com)
