Local-first Android AI agent runtime for safe, observable phone control.

TouchPilot is inspired by OpenClaw and PicoClaw, but it is scoped around one hard problem first: letting an AI agent operate an Android device through explicit, permissioned, inspectable tools.

• Run as a native Android app, not only as a desktop companion.
• Expose Android actions through typed tools with clear risk levels.
• Use AccessibilityService for semantic UI observation and control.
• Prefer local-first routing and inference.
• Keep OpenAI-compatible model providers available as an experimental fallback.
• Run the first local LiteRT command-routing model, with ExecuTorch and llama.cpp documented as future runtime paths.
• Keep user approval and audit logs central to the runtime.

The first milestone is a local Android app that can:

• show a chat interface,
• run simple actions through the default local router without an API key,
• optionally connect to an OpenAI-compatible model endpoint as an experimental fallback,
• observe the current screen through AccessibilityService,
• tap, type, scroll, open apps, and press back/home through approved tools,
• record every model decision and tool execution in local logs.

Build the Android debug APK:

./gradlew assembleDebug

Build an installable development release APK:

./gradlew assembleRelease

During early development, the release build is signed with the local Android debug key so it can be installed on emulators such as LDPlayer. Replace this with a real release signing key before publishing.

The current agent MVP defaults to the local router for simple Android actions and includes a LiteRT local model mode for command routing. An experimental cloud fallback can be configured with an OpenAI-compatible chat completions URL, model name, and API key. Fallback URL and model are stored in app preferences. API keys are encrypted with an Android Keystore-backed key before being stored.

When the model selects a medium- or high-risk Android tool, TouchPilot asks the user for approval in the chat before executing it. Low-risk observation and wait tools can run without a prompt.

If building outside Android Studio, make sure either ANDROID_HOME is set or local.properties contains the local Android SDK path:

sdk.dir=/path/to/Android/Sdk

• OpenClaw for sessions, skills, channels, and gateway design.
• PicoClaw for lightweight runtime discipline and small-device deployment mindset.
• MobileClaw for mobile GUI automation and chat-channel agent ideas.
• ClawMobile for smartphone-native agent architecture and semantic Android control.
• ExecuTorch for future on-device model inference.

app/              Android app source
agent/            Agent runtime, provider clients, planner loop
tools/            Tool specs and tool router
androidcontrol/   Accessibility and Android action execution
memory/           Local session, logs, and skill storage
security/         Approvals, policy, risk levels, secrets handling
skills/           Markdown skills for task-specific behavior
docs/             Architecture, security, and tool documentation
examples/         Provider and MCP integration examples