A valid request URL is required to generate request examples{
"ok": true,
"others": {
"actions": 123,
"lastAt": "2023-11-07T05:31:56Z",
"by": [
"person"
]
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}{
"error": {
"code": "payload_too_large",
"message": "<string>"
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}{
"error": {
"code": "rate_limited",
"message": "<string>"
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}Act on the device
Look before you touch: an action is refused until this run has fetched at least one frame, because coordinates with no frame behind them have no meaning.
The action is bound to the frame it was decided on. Matching geometry is not enough on its own, since two different screens at the same resolution compare equal, so a superseded frame is refused and you are told to observe again. Retrying a request whose response you lost replays the recorded outcome instead of touching the screen twice.
A person may drive the same device beside this run, and their gestures go ahead of the run’s waiting ones. Every response carries others, what principals other than this run did since the frame this action was bound to, and by default an action is refused STALE_FRAME when that count is above zero: re-observe as you already do. Send refuseIfOthersActed: false to be informed instead of stopped. Gestures on one device run one at a time in arrival order, and a device with too many waiting answers ACTION_UNAVAILABLE; wait and retry, the run is untouched.
This operation reads no Authorization header. The run token in the path IS the credential, so the whole url is secret: never log it, paste it, or commit it. Every example writes {runToken} for that reason.
A valid request URL is required to generate request examples{
"ok": true,
"others": {
"actions": 123,
"lastAt": "2023-11-07T05:31:56Z",
"by": [
"person"
]
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}{
"error": {
"code": "payload_too_large",
"message": "<string>"
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}{
"error": {
"code": "rate_limited",
"message": "<string>"
}
}{
"ok": false,
"error": {
"code": "DEVICE_NOT_FOUND",
"message": "<string>",
"details": {}
}
}Path Parameters
The run token. Write {runToken}; never write a real one down.
Must be the device this run is bound to. Anything else is refused as not found.
Body
One action.
- tap
- swipe
- type_text
- press_button
- open_app
One action, discriminated by action. Coordinates are SCREENSHOT PIXELS of the frame you last fetched, and screen must echo that frame's size: a mismatch means every coordinate aims at the wrong thing, so it is refused rather than scaled.
Every variant also takes refuseIfOthersActed, this service's one addition to the dialect: the action is refused STALE_FRAME if another principal (a person driving the same device beside this run, for instance) acted after the frame it is bound to. On by default for a run, which always binds a frame; send false to be told through others instead. A worker that already re-observes on STALE_FRAME needs no new code.
"tap""screenshot_pixels"Show child attributes
Show child attributes
Refuse this gesture (409 stale_frame, reason others_acted) if another principal acted on the device after observedFrame was taken. Sending true needs observedFrame; without it the body is refused as invalid_argument. Left out, it defaults by who is acting: on for an API key, a copilot or a run that sends observedFrame, off for a person acting directly, and off whenever no observedFrame is sent. You are told through others either way.
Response
The action was applied.
The action was applied. others says what principals other than this run did on the device since the frame the action was bound to; the driver ignores this body, and a worker that reads it looks again before its next decision when actions is above zero.
What everybody else did on this device since you last looked. On a gesture the anchor is the frame you bound it to, else your previous gesture that reached the device (a refused one never anchors), else the last sixty seconds. A person may drive the same device beside an agent's run; each is informed of the other through this and neither is stopped.
Show child attributes
Show child attributes