MCP server runs on Android

The accessibility tree on Android is surprisingly powerful — enough to let an LLM navigate real apps like a human would, no root required. That's the core idea behind Android Remote Control MCP, an MCP server that runs directly on your phone and exposes tap, type, scroll, and screenshot actions to any compatible agent. I've been testing it for a few weeks now, and the practical implications for AI workflow automation are genuinely interesting.

How it works in practice

The app registers as an accessibility service, which gives it read access to the UI hierarchy across all installed apps. When an agent sends a tool call — say, "open Gmail and search for 'invoice'" — the server parses the accessibility tree, finds the relevant elements, and executes the sequence. Token usage stays low because the server returns structured summaries instead of raw XML dumps. A typical interaction consumes roughly 200-400 tokens per step depending on screen complexity.

Privacy mode is the standout feature

After early feedback made it clear nobody wants their full screen context shipped to an LLM provider, the author added on-device redaction. A small local model combined with deterministic regex detectors strips emails, phone numbers, credit cards, IBANs, national IDs, and English names before anything leaves the device. Benchmarked detection sits around 87% — non-English names remain the weak spot, but a custom model is in development. For anyone building LLM agent pipelines that touch personal data, this approach deserves attention.

Integration with Claude and ChatGPT

The app acts as its own OAuth server. You approve connections via a code displayed on the phone, then point Claude.ai, Claude Desktop, or chatgpt.com at the local endpoint. No ngrok, no Cloudflare tunnel — though a free reverse tunnel with Let's Encrypt certificates and true end-to-end encryption is on the roadmap to replace those workarounds entirely.

Tradeoffs worth knowing

• Google Play distribution is impossible because accessibility services trigger policy restrictions. The APK lives on GitHub; a FOSS build without Play Services is heading to F-Droid soon.
• Prompt injection mitigation: every response from the server is prefixed with a strong untrusted-input warning. Not bulletproof, but it raises the bar significantly.
• App coverage varies. Mainstream apps (Chrome, Gmail, Maps, Spotify) work reliably. Custom or heavily obfuscated UIs sometimes break the tree traversal.

What's coming next

• Skills database per app so agents can plan faster without trial-and-error exploration
• Custom privacy model for better non-English name detection
• Guided setup flow and revised UI to lower the barrier for non-technical users

If you're experimenting with AI workflow automation on mobile, this is worth a hands-on guide session. The repo includes a practical tutorial for getting started from scratch, and even smaller models like Haiku can drive meaningful multi-step tasks when given enough context.