OfflineLLM
Private on-device AI chat for Android - runs any GGUF model locally via llama.cpp with ARM-optimised SIMD. Zero network permissions, encrypted settings, biome
What is OfflineLLM?
Private on-device AI chat for Android - runs any GGUF model locally via llama.cpp with ARM-optimised SIMD. Zero network permissions, encrypted settings, biometric lock, tamper detection. + GPU Acceleration - PQC ML-DSA-87 signed releases
Overview of OfflineLLM
What it is
OfflineLLM is a private, on-device AI chat application for Android. It runs any GGUF model locally using llama.cpp, with no network permissions in the app manifest. The app is designed for users who want a fully offline AI assistant, with all inference performed on the device. It includes optional GPU acceleration via Vulkan and a range of performance and security features.
Key features
- 100% offline: no INTERNET permission in the manifest, cannot phone home - On-device inference: GGUF models via llama.cpp, with runtime CPU dispatch for optimal kernel selection (dotprod, fp16, i8mm, SVE)
- GPU acceleration (Vulkan): opt-in toggle with per-layer offload control and automatic CPU fallback - Fast multi-turn chat: incremental prompt processing, only new messages are fed into the KV cache - Performance controls: CPU thread count, prompt-phase threading, memory-mapped loading, RAM lock, quantized KV cache - Streaming responses: token-by-token output - Import any model: bring your own GGUF at runtime via file picker - Multiple conversations: auto-titled, renameable, searchable - Translator: 75+ languages - Advanced sampling: Temperature, Top-P, Top-K, Min-P, Repeat Penalty - System prompts: General, Coder, Creative Writer, Tutor, Translator - Markdown and TTS: formatted responses, read aloud via system TTS - Thinking tag stripping: hides reasoning blocks - Theming: 11 Ptyxis terminal palettes, Catppuccin, Dracula, System/Light/Dark/AMOLED with Material You accents - 13 bundled fonts, context bar, tamper detection, encrypted settings, biometric lock, chat backup
Use cases
- Users who require complete privacy and no network connectivity for AI chat - Running LLMs on devices with limited RAM (2-8 GB) using small models - Developers and enthusiasts who want to experiment with local AI on Android - Users who need translation, coding assistance, or creative writing without cloud services
Getting started
Install from Releases or via ADB: bash adb install OfflineLLM-5.1.3-PQC-signed.apk Requires Android 13+ and arm64-v8a. After installation, import a GGUF model from Settings.
License and pricing
The license is listed as NOASSERTION, and the pricing type is open source. The exact license terms are not stated in the provided metadata or README.
Based on the project's public README and repository metadata.