vllm-metal

Community maintained hardware plugin for vLLM on Apple Silicon

What is vllm-metal?

Community maintained hardware plugin for vLLM on Apple Silicon

Overview of vllm-metal

What it is

vLLM Metal is a community maintained hardware plugin that enables vLLM to run on Apple Silicon Macs. It uses MLX as the primary compute backend and unifies MLX and PyTorch under a single lowering path. The plugin is hosted under the vLLM project on GitHub and is licensed under Apache-2.0. It is categorized as an AI tool and is open source with no stated pricing.

Key features

  • Runs vLLM on Apple Silicon using MLX as the compute backend.
  • Unifies MLX and PyTorch under a single lowering path.
  • Provides a paged varlen kernel for request-aware attention.
  • Supports M5 NAX tensor units to accelerate MHA, GQA, and MQA prefill.
  • Includes speculative decoding support.
  • Offers a sleep mode that lets an idle model server return memory to other applications.

Use cases

vLLM Metal is intended for running large language model inference on Apple Silicon Macs. It can be used for local model serving, concurrent duplex audio/video conversations (via vLLM-Omni), text-to-image generation (via vLLM-Omni), and structured decisions (e.g., text-only choice, yes/no, score requests). It is also used in the SiliconBench evaluation of serving engines on speed, memory use, and output fidelity.

Getting started

Stable release via Homebrew: bash brew tap vllm-project/vllm-metal https://github.com/vllm-project/vllm-metal brew install vllm-project/vllm-metal/vllm-metal Latest development build via curl installer: bash curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh | bash source ~/.venv-vllm-metal/bin/activate For contributing Python or kernel code, see the repository README for source setup instructions.

License and pricing

The license is Apache-2.0. The project is open source with no stated pricing.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/vllm-projectvllm-metal.svg)](https://yunodb.com/tools/vllm-projectvllm-metal?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/vllm-projectvllm-metal?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/vllm-projectvllm-metal.svg" alt="Featured on YunoDB" height="20"></a>