OpenVision
Open-source iOS app connecting Meta Ray-Ban smart glasses to AI — 5 backends (on-device MLX models, Apple Intelligence, OpenAI, Gemini Live, OpenClaw), on-devic
What is OpenVision?
Open-source iOS app connecting Meta Ray-Ban smart glasses to AI — 5 backends (on-device MLX models, Apple Intelligence, OpenAI, Gemini Live, OpenClaw), on-device neural voice, face recognition & live web search. Private and offline-capable.
Overview of OpenVision
What it is
OpenVision is an open-source iOS app that connects Meta Ray-Ban smart glasses to AI assistants. It supports five backends: on-device MLX models, Apple Intelligence, OpenAI, Gemini Live, and OpenClaw. It offers on-device neural voice, face recognition, and live web search. It is private and offline-capable.
Key features
- Seven AI backends: Local (MLX) with choice of on-device models, Apple Intelligence, OpenClaw, Gemini Live, OpenAI, Grok, and Hermes.
- Live video mode: real-time voice + vision with Gemini Live, OpenAI Realtime API, or fully on-device with SmolVLM2.
- On-device photos with SmolVLM2: capture and analyze images entirely on-device.
- Natural voices: Kokoro on-device voice, Grok and OpenAI cloud voices, with fallback to Apple system voice.
- Agentic web search: models search the web using Tavily or DuckDuckGo when unsure.
- Conversation memory: multi-turn context on on-device, OpenAI, and Grok backends.
- Hands-free productivity tools: timers, Pomodoro, reminders, calendar, notes, clipboard via voice.
- On-device face recognition using Apple Vision.
- Smart voice control: wake word 'Ok Vision', barge-in, conversation mode, stop command.
- On-device model management: download and delete models from Settings.
- Glasses integration: photo capture, live video streaming, seamless registration.
- Production-ready: auto-reconnect, network monitoring, app lifecycle handling.
- Zero hardcoding: all API keys configurable.
Use cases
- Instant translation: point at a menu or sign abroad for real-time translation.
- Hands-free cooking: ask for next step while hands are busy.
- Smart shopping: get price comparisons and reviews.
- Travel guide: instant history and facts about buildings.
- Accessibility: describe surroundings, read text aloud, identify objects.
- Meeting assistant: remember people.
- Learning: identify flora, fauna, landmarks.
- DIY helper: show problem, get step-by-step help.
- Fitness coach: real-time posture feedback.
- With OpenClaw's 56+ tools: send emails, control smart home, set reminders, search the web hands-free.
Getting started
See the repository README for installation and quick-start instructions.
License and pricing
The project is open source under the MIT license.
Based on the project's public README and repository metadata.