FastFlowLM
Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.
What is FastFlowLM?
Run LLMs on AMD Ryzen™ AI NPUs in minutes; purpose-built and deeply optimized for the AMD NPUs.
Overview of FastFlowLM
What it is
FastFlowLM (FLM) is a runtime for running large language models on AMD Ryzen™ AI NPUs. It is purpose-built and deeply optimized for AMD NPUs. The project is hosted under the AMD ROCm organization on GitHub. It supports Vision, Audio, Embedding and MoE models. It does not require a GPU and claims to be faster and over 10× more power-efficient than GPU-based alternatives. It supports context lengths up to 256k tokens. The runtime is described as ultra-lightweight (17 MB) and installs within 20 seconds. It supports all Ryzen™ AI Series chips with XDNA2 NPUs (Strix, Strix Halo, Kraken, and Gorgon Point).
Key features
- Runs fully on AMD Ryzen™ AI NPU — no GPU or CPU load
- Lightweight runtime (17 MB) — installs within 20 seconds
- Developer-first flow — a familiar single-command CLI, optimized for NPU
- Support for long context windows — up to 256k tokens (e.g., Qwen3-4B-Thinking-2507)
- No low-level tuning required
- Supports LLMs, VLMs, Audio, Embedding and MoE models
- Provides a local server mode with REST and OpenAI API compatibility
Use cases
FastFlowLM is designed for running large language models locally on AMD Ryzen™ AI NPUs. Use cases include running LLMs and VLMs on laptops or edge devices without a dedicated GPU, enabling private and offline AI inference, and integrating NPU-accelerated AI into applications via a simple CLI or API. It is also used for robotics policies (e.g., SmolVLA model) and can be used with AMD's Lemonade Server.
Getting started
See the repository README for detailed installation instructions. A packaged Windows installer (flm-setup.msi) is available. After installation, open PowerShell and run a model in CLI mode:powershellflm run llama3.2:1bTo start the local server (Server Mode):powershellflm serve llama3.2:1bOn Linux, see the Linux Getting Started Guide in the repository. The latest AMD NPU driver (32.0.203.311 or above) is required.
License and pricing
The project is open source. All orchestration code and CLI tools are under the MIT License. The NPU-accelerated binary kernels are not open source but are free for any use (non-commercial and commercial) under the FastFlowLM NPU Binary License. The developer kit (flm-dev-kit/) is open source under the MIT License. Third-party components and their licenses are listed in THIRD_PARTY_NOTICES.txt. The pricing type is listed as open source.
Based on the project's public README and repository metadata.