list_devices, or get_device when you already know the id, and read the
capabilities block before you plan anything.
What a device declares
Actions
Each action field says whether the device supports something and, where it matters, by which mechanism. The mechanism changes how you should use it.pressKey is a list rather than a boolean for a reason. The tool’s parameter
enumerates every key the platform has a meaningful equivalent for, and that is
an upper bound across all backends. A given device supports a subset and
refuses the rest.
Observations
A device with no visual observation at all, such as a hardware input dongle,
declares
framesAreReferenceable: false and is never forced into frame
discipline. Passing observedFrame to one is refused as
capability_unsupported rather than quietly ignored.
Fidelity tier
fidelityTier says how input reaches the device. The MCP tools list two
values, software and hid. Every device you can reach today reports
software, and no device reports hid. See Device tiers
for what each one means.
The HTTP API reference still lists a third value, physical, with an extra
physical block. It belonged to the robot hand + iPhone, which is not part of
the API, MCP or CLI, and no device reports it.
Coordinates
Every device speaks the same grid: both axes run 0 to 1000 whatever the real screen size is, with the origin at the top left, in the current orientation.coordSpace states it explicitly.
Backends convert to device pixels internally. You never see pixels, except as
the informational pixelSize on a screenshot, which is there to describe the
image rather than to aim with.
Bounds inside a ui_snapshot tree are already normalized, so a coordinate you
read there can be tapped directly.
Backend kinds
backendKind names the mechanism that reaches a device. It is a closed set:
Every kind except
adb_cloud is declared with no adapter behind it. A device
row naming one of them is accepted, and a call against it is refused with
backend_not_implemented rather than failing in some less obvious way.
adb_cloud is not one of them: its adapter is live. A cloud row is reached
over the cloud phone platform’s HTTPS API, and its capability block is narrower than the
kind’s bound, so read capabilities on the device rather than assuming it
from the kind.
This is how the cloud phones you can buy on the spot from the
/devices/add screen are reached. The frame is captured inside the guest OS
(screenshot: "framebuffer"), live video comes from the cloud phone
platform’s own channel (stream: "webrtc", handed out by POST .../stream as
transport: "provider-rtc"), and wake is supported. There is no element
tree (uiTree: false), so ui_snapshot is refused with
capability_unsupported; read the screen from screenshots instead. Every key
in pressKey is available, and a key press is reported once the cloud phone
platform accepts it, without waiting for a confirmation from the device.
A cloud row reports its reachability in status, which is an answer about
that device and not a statement about the adapter. It reports offline when
the cloud phone platform named it and said it was not running, and
unknown when the platform’s fleet listing failed or never mentioned it at
all, or when the row names no cloud phone instance.
Three separate vocabularies describe a device and it is worth keeping them
apart. backendKind, in this table, is the mechanism. fidelityTier is how
input reaches the screen, and it is the coarser of the two: three backend kinds
collapse into software. The phone names on
Billing modes are a third vocabulary, for what a device
costs, and that page carries the crosswalk between them and backendKind.
Use backendKind to understand what you are talking to. Do not use it to
infer abilities: the capability block is the authoritative answer, and a
backend narrows it per device rather than widening it.
A device’s backend is resolved from that device’s own record. There is no
global switch, because a fleet mixes backends and a process wide setting would
always be wrong for somebody’s device.
Refusal, never silence
An adapter that claims an ability it does not implement, or implements one it did not declare, fails at startup rather than at your call site. At your call site, an unsupported ability is refused withcapability_unsupported. It is never accepted and quietly dropped, which
would leave you believing something happened on a real device when nothing
did.
What the device says it is
The capability block says what the control plane can DO to a device. The optionalinfo block beside it says what the device IS: the four facts a
person reads off Settings, About phone.
info can be absent on a
device that will report it a minute later, and the acquire that triggers the
reading can itself answer before the reading lands.
resolution is the screen in device pixels. It is not the grid you aim
actions at. Taps and swipes are sent in capabilities.coordSpace, a
normalised 1000 by 1000 space, and converting a coordinate through the pixel
rectangle instead will land it somewhere else. It is also not a screenshot’s
pixelSize, which is one observation’s dimensions in whatever orientation it
was taken in.
Three absences, three meanings
Missing is not one state here, and code that treats it as one will describe a phone wrongly.
Which fields arrive depends on how the device is reached, not on what the
hardware has. A cloud phone driven over the cloud phone platform’s HTTPS API has no route
that returns a shell command’s output, so nothing here comes from
getprop
or wm size: the model, the Android version and the operator are read from
the cloud phone platform’s own property API, and the screen comes from the same
framebuffer measurement the tap path already uses to place a coordinate.
A field is absent when that particular device did not report it, not because
of the transport. The reading is taken the first time the device is acquired,
and a reading that is missing a field is retaken once after a deployment, so
a field can appear later without anything being asked of you.
Every field is optional and the block only ever grows, so read it
defensively: branch on the field you want being present, never on info
having a particular shape.