harness
AI-driven user testing for iOS Simulator, macOS apps, and web apps. Write a goal in plain language; an LLM agent drives the UI and reports friction. macOS 14+,
What is harness?
AI-driven user testing for iOS Simulator, macOS apps, and web apps. Write a goal in plain language; an LLM agent drives the UI and reports friction. macOS 14+, Swift 6.
Overview of harness
What it is
Harness is a native macOS developer tool that drives an iOS Simulator, a macOS app, or a web app with an AI agent to run user tests. It simulates real-user interaction rather than scripted UI tests. The user writes a goal in plain language and optionally a persona; an LLM agent reads screenshots, clicks/types/scrolls, and pursues the goal, narrating what it sees and flagging UX friction. The tool outputs three artifacts: whether the goal completed, the path taken (replayable sequence of screens and actions), and timestamped friction events. It requires macOS 14+, Swift 6, and is available for Apple Silicon and Intel.
Key features
- AI-driven user testing for iOS Simulator, macOS apps, and web apps (embedded WKWebView).
- Per-platform input methods: iOS via WebDriverAgent; macOS via AX actions and CGEvent (contained input, no global-HID fallback by default); web via JS-synthesised events.
- Set-of-Mark targeting: numbered overlays on interactive elements; agent clicks by id, not pixel.
- Supports multiple LLM providers: Anthropic (Opus, Sonnet, Haiku), OpenAI (GPT-5 Mini, GPT-4.1 Nano), Google Gemini (2.5 Flash, Flash Lite), and local Mac inference via Ollama (Qwen3-VL 8B, Gemma 4 Vision 9B, Llama 3.2 Vision 11B, plus custom model).
- MCP server (harness-mcp) for autonomous runs or step-level UI sessions; standalone and relocatable binary.
- Authenticated web sessions: inject and export session state (cookies + localStorage) for SSO/MFA workflows.
- Sparkle auto-update built in; per-Application credential storage; per-provider Keychain storage; configurable per-model token budgets; unlimited-step option.
Use cases
- Running user tests on iOS Simulator, macOS apps, or web apps without writing scripted UI tests.
- Identifying UX friction such as dead ends, ambiguous labels, or unresponsive controls.
- Testing authentication flows (SSO, MFA) via visible web sessions that a human can log into, then exporting session state for headless replay.
- Replaying and reviewing the exact path a user would take through an application.
Getting started
bashgit clone https://github.com/awizemann/harness.gitcd harnessgit submodule update --init --recursivebrew install xcodegenxcodegen generateopen Harness.xcodeprojFull setup instructions are available in the Build-and-Run section of the project Wiki.
License and pricing
MIT license. Open source.
Based on the project's public README and repository metadata.