16 KiB
summary, read_when, title
| summary | read_when | title | ||||
|---|---|---|---|---|---|---|
| Android app (node): connection runbook + Connect/Chat/Voice/Canvas command surface |
|
Android app |
Support snapshot
- Role: companion node app (Android does not host the Gateway).
- Gateway required: yes (run it on macOS, Linux, or Windows via WSL2).
- Install: Google Play for the app, Getting Started for the Gateway, then Pairing.
- Gateway: Runbook + Configuration.
- Protocols: Gateway protocol (nodes + control plane).
System control (launchd/systemd) lives on the Gateway host — see Gateway.
Mirror and control Android from a remote Mac
scrcpy mirrors an Android screen in a macOS window and forwards keyboard and pointer input through Android Debug Bridge (ADB). This is an operator-side workflow, separate from the OpenClaw node connection. It is useful when the Android device and the Mac are in different locations but share a private Tailscale network.
Before you begin
-
Install Tailscale on the Android device and the Mac, and connect both to the same tailnet.
-
On Android, enable Developer options and USB debugging. Android 16 places Wireless debugging under Settings > System > Developer options. See Android developer options.
-
Install scrcpy and ADB on the Mac:
brew install scrcpy brew install --cask android-platform-tools -
Keep the Android device available for the first connection. Android must approve each Mac's ADB key before that Mac can control the device.
Enable ADB over TCP
For the initial setup, connect the Android device by USB to a trusted computer and approve its debugging prompt. Then run:
adb devices
adb tcpip 5555
You can now disconnect USB. If port 5555 stops listening after a device reboot or debugging reset,
repeat this local setup step. Android 11 and later can also establish the initial trust with
Wireless debugging > Pair device with pairing code and adb pair.
Allow only the controller Mac
Tailnets with restrictive grants must explicitly allow the controller Mac to reach TCP port 5555 on the Android device. Add a narrow rule to the tailnet policy, replacing the example addresses with the two devices' stable Tailscale IPs:
{
grants: [
{
src: ["<remote-mac-tailnet-ip>"],
dst: ["<android-tailnet-ip>"],
ip: ["tcp:5555"],
},
],
}
See Tailscale grants for host aliases and other selectors. Do not grant this port to the public internet or expose it with Funnel: an authorized ADB client has broad control of the device.
Connect and start mirroring
On the remote Mac:
adb connect <android-tailnet-ip>:5555
adb devices
scrcpy --serial <android-tailnet-ip>:5555
The first adb connect from this Mac shows an authorization dialog on Android. Unlock the device,
confirm the key fingerprint, and select Always allow from this computer only when the Mac is
trusted. A successful adb devices entry ends in device; unauthorized means the on-device prompt
has not been approved.
Once the scrcpy window opens, use it directly or target it with a macOS screen-automation tool such as Peekaboo. scrcpy carries the display and input; Tailscale provides only the private network path.
Troubleshooting
Connection timed out: verify the tailnet grant for TCP 5555. A successfultailscale pingproves peer reachability, not that policy permits this TCP port. Test withnc -vz <android-tailnet-ip> 5555from the Mac.unauthorized: unlock Android and approve the remote Mac's ADB key, or remove the stale workstation under Wireless debugging > Paired devices and pair it again.Connection refused: reconnect locally and runadb tcpip 5555again.- More than one device listed: keep the explicit
--serial <android-tailnet-ip>:5555argument.
When finished, close scrcpy and disconnect ADB:
adb disconnect <android-tailnet-ip>:5555
Connection runbook
Android node app ⇄ (mDNS/NSD + WebSocket) ⇄ Gateway
Android connects directly to the Gateway WebSocket and uses device pairing (role: node).
For Tailscale or public hosts, Android requires a secure endpoint:
- Preferred: Tailscale Serve / Funnel with
https://<magicdns>/wss://<magicdns> - Also supported: any other
wss://Gateway URL with a real TLS endpoint - Cleartext
ws://remains supported on private LAN addresses /.localhosts, pluslocalhost,127.0.0.1, and the Android emulator bridge (10.0.2.2)
Prerequisites
- Gateway running on another machine (or reachable via SSH).
- Android device/emulator can reach the gateway WebSocket:
- Same LAN with mDNS/NSD, or
- Same Tailscale tailnet using Wide-Area Bonjour / unicast DNS-SD (see below), or
- Manual gateway host/port (fallback)
- Tailnet/public mobile pairing does not use raw tailnet IP
ws://endpoints. Use Tailscale Serve or anotherwss://URL instead. - The
openclawCLI available on the gateway machine (or via SSH), to approve pairing requests.
1. Start the Gateway
openclaw gateway --port 18789 --verbose
Confirm in logs you see something like:
listening on ws://0.0.0.0:18789
For remote Android access over Tailscale, prefer Serve/Funnel instead of a raw tailnet bind:
openclaw gateway --tailscale serve
This gives Android a secure wss:// / https:// endpoint. A plain gateway.bind: "tailnet" setup is not enough for first-time remote Android pairing unless you also terminate TLS separately.
2. Verify discovery (optional)
From the gateway machine:
dns-sd -B _openclaw-gw._tcp local.
More debugging notes: Bonjour.
If you also configured a wide-area discovery domain, compare against:
openclaw gateway discover --json
That shows local. plus the configured wide-area domain in one pass, using the resolved service endpoint instead of TXT-only hints.
Cross-network discovery via unicast DNS-SD
Android NSD/mDNS discovery does not cross networks. If the Android node and the gateway are on different networks but connected via Tailscale, use Wide-Area Bonjour / unicast DNS-SD instead. Discovery alone is not sufficient for tailnet/public Android pairing — the discovered route still needs a secure endpoint (wss:// or Tailscale Serve):
- Set up a DNS-SD zone (example
openclaw.internal.) on the gateway host and publish_openclaw-gw._tcprecords. - Configure Tailscale split DNS for your chosen domain pointing at that DNS server.
Details and example CoreDNS config: Bonjour.
3. Connect from Android
In the Android app:
- The app keeps its gateway connection alive via a foreground service (persistent notification).
- Open the Connect tab.
- Use Setup Code or Manual mode.
- If discovery is blocked, use manual host/port in Advanced controls. For private LAN hosts,
ws://still works. For Tailscale/public hosts, turn on TLS and use awss:/// Tailscale Serve endpoint.
After the first successful pairing, Android auto-reconnects on launch: the manual endpoint (if enabled), otherwise the last discovered gateway (best-effort).
Presence alive beacons
After the authenticated node session connects, and when the app moves to the background while the foreground service is still connected, Android calls node.event with event: "node.presence.alive". The gateway records this as lastSeenAtMs/lastSeenReason on the paired node/device metadata only after the authenticated node device identity is known.
The app counts the beacon as successfully recorded only when the gateway response includes handled: true. Older gateways may acknowledge node.event with { "ok": true }; that response is compatible but does not count as a durable last-seen update.
4. Approve pairing (CLI)
On the gateway machine:
openclaw devices list
openclaw devices approve <requestId>
openclaw devices reject <requestId>
Pairing details: Pairing.
Optional: if the Android node always connects from a tightly controlled subnet, you can opt in to first-time node auto-approval with explicit CIDRs or exact IPs:
{
gateway: {
nodes: {
pairing: {
autoApproveCidrs: ["192.168.1.0/24"],
},
},
},
}
This is disabled by default. It applies only to fresh role: node pairing with no requested scopes. Operator/browser pairing and any role, scope, metadata, or public-key change still require manual approval.
5. Verify the node is connected
openclaw nodes status
openclaw gateway call node.list --params "{}"
6. Chat + history
The Android Chat tab supports session selection (default main, plus other existing sessions):
- History:
chat.history(display-normalized — inline directive tags, plain-text tool-call XML payloads (<tool_call>,<function_call>,<tool_calls>,<function_calls>, and truncated variants), and leaked ASCII/full-width model control tokens are stripped; silent-token assistant rows such as exactNO_REPLY/no_replyare omitted; oversized rows can be replaced with placeholders) - Send:
chat.send - Push updates (best-effort):
chat.subscribe->event:"chat"
7. Canvas + camera
Gateway Canvas Host (recommended for web content)
To have the node show real HTML/CSS/JS that the agent can edit on disk, point the node at the Gateway canvas host.
Nodes load canvas from the Gateway HTTP server (same port as `gateway.port`, default `18789`).- Create
~/.openclaw/workspace/canvas/index.htmlon the gateway host. - Navigate the node to it (LAN):
openclaw nodes invoke --node "<Android Node>" --command canvas.navigate --params '{"url":"http://<gateway-hostname>.local:18789/__openclaw__/canvas/"}'
Tailnet (optional): if both devices are on Tailscale, use a MagicDNS name or tailnet IP instead of .local, e.g. http://<gateway-magicdns>:18789/__openclaw__/canvas/.
This server injects a live-reload client into HTML and reloads on file changes. The Gateway also serves /__openclaw__/a2ui/, but the Android app treats remote A2UI pages as render-only. Action-capable A2UI commands use the bundled app-owned A2UI page.
Canvas commands (foreground only):
canvas.eval,canvas.snapshot,canvas.navigate(use{"url":""}or{"url":"/"}to return to the default scaffold).canvas.snapshotreturns{ format, base64 }(defaultformat="jpeg").- A2UI:
canvas.a2ui.push,canvas.a2ui.reset(canvas.a2ui.pushJSONLlegacy alias). These use the bundled app-owned A2UI page for action-capable rendering.
Camera commands (foreground only; permission-gated): camera.snap (jpg), camera.clip (mp4). See Camera node for parameters and CLI helpers.
8. Voice + expanded Android command surface
- Voice tab: Android has two explicit capture modes. Mic is a manual Voice-tab session that sends each pause as a chat turn and stops when the app leaves the foreground or the user leaves the Voice tab. Talk is continuous Talk Mode and keeps listening until toggled off or the node disconnects.
- Talk Mode promotes the existing foreground service from
connectedDevicetoconnectedDevice|microphonebefore capture starts, then demotes it when Talk Mode stops. The node service declaresFOREGROUND_SERVICE_CONNECTED_DEVICEwithCHANGE_NETWORK_STATE; Android 14+ also requires theFOREGROUND_SERVICE_MICROPHONEdeclaration, theRECORD_AUDIOruntime grant, and the microphone service type at runtime. - By default, Android Talk uses native speech recognition, Gateway chat, and
talk.speakthrough the configured gateway Talk provider. Local system TTS is used only whentalk.speakis unavailable. - Android Talk uses realtime Gateway relay only when
talk.realtime.modeisrealtimeandtalk.realtime.transportisgateway-relay. - Voice wake is implemented in source (
VoiceWakeMode) but the shipping app runtime always forces it tooffon connect — there is no user-facing toggle today. - Additional Android command families (availability depends on device, permissions, and user settings):
device.status,device.info,device.permissions,device.healthdevice.appsonly when Settings > Phone Capabilities > Installed Apps is enabled; it lists launcher-visible apps by default (passincludeNonLaunchablefor the full list).notifications.list,notifications.actions(see Notification forwarding below)photos.latestcontacts.search,contacts.addcalendar.events,calendar.addcallLog.searchsms.searchmotion.activity,motion.pedometer
Assistant entrypoints
Android supports launching OpenClaw from the system assistant trigger (Google Assistant). Holding the home button (or another ACTION_ASSIST trigger) opens the app; saying "Hey Google, ask OpenClaw <prompt>" matches the app's declared App Actions query pattern and hands the prompt into the chat composer without auto-sending it.
This uses Android App Actions (shortcuts.xml capability) declared in the app manifest. No gateway-side configuration is needed — the assistant intent is handled entirely by the Android app.
Notification forwarding
Android can forward device notifications to the gateway as node.event items. This is configured on the device, in the app's Settings sheet — not in gateway/openclaw.json config.
| Setting | Description |
|---|---|
| Forward Notification Events | Master toggle. Off by default; requires Notification Listener Access to be granted first. |
| Package Filter | Allowlist (only listed package IDs forwarded) or Blocklist (default: all packages except listed IDs). OpenClaw's own package is always excluded in Blocklist mode to prevent forwarding loops. |
| Quiet Hours | Local HH:mm start/end window that suppresses forwarding. Disabled by default; defaults to 22:00-07:00 once enabled. |
| Max Events / Minute | Per-device rate limit on forwarded notifications. Default 20. |
| Route Session Key | Optional. Pins forwarded notification events into a specific session instead of the device's default notification route. |