Claude Code AI Agent Controls Physical iPhone via Accessibility APIs

An AI agent is controlling a physical iPhone to write and post content to Reddit without human typing. The demonstration uses Claude Code (Anthropic's AI agent) running inside Blitz, a Mac app that connects AI agents to physical iPhones.

Technical Implementation

Blitz provides access to a real iPhone via WebDriverAgent, with navigation handled entirely through accessibility APIs. The developer encountered a specific technical limitation: normal tap commands don't work on physical devices, returning a 404 "unknown command" error.

The workaround discovered: a zero-distance swipe (same start AND end coordinates) registers as a tap. Every button press in the demonstration uses this hack.

Step-by-Step Process

• Called describe_screen to find Reddit's icon coordinates on the home screen (scan_ui is simulator-only, so the accessibility tree was used instead)
• Zero-distance-swiped the icon to open Reddit
• Tapped the search button, saw r/ClaudeAI in recent searches, tapped it
• Hit Create, tapped the title field, typed the title
• Tapped the post body field and started writing

Every action follows the pattern: describe_screen → parse coordinates → swipe(x, y, x, y). The agent operates the phone "blind," reading the world through an accessibility JSON tree.

The developer notes they also posted to r/vibecoding about the same session, where they got feedback about their initial title choice.