> ## Documentation Index > Fetch the complete documentation index at: https://docs.phoneuse.com/llms.txt > Use this file to discover all available pages before exploring further. # Act on a device > One gesture per request: tap, swipe, long press, type text, press a key, open an app. Coordinates are in the normalized 0-1000 grid, so nothing has to know the device's resolution. WHOSE LEASE. Any caller may leave `leaseId` out. The gesture then runs under the lease your organisation currently holds on the device, whoever inside it took that lease (an agent's run, a colleague, you), and the response says so in `coDriving`. When your organisation holds none, a lease is taken for you and the gesture runs under it; if that gesture then fails, the lease taken for it is given back before the error is answered, and your next gesture takes one again. Two people taking an idle device at the same moment both drive it: one took the lease, the other is co-driving under it. When another organisation holds the device the answer is the 409 the acquire route gives. Taking needs `devices:lease`: a key with only `devices:act` acts on a device its organisation already holds. A signed-in person's gesture slides a lease THEY hold when it is running low; it never extends anybody else's. DRIVING BESIDE AN AGENT. A person's tap does not interrupt a run, and does not wait behind it either: gestures on one device run one at a time, a signed-in person's go ahead of every agent gesture still waiting, and the rest keep their arrival order. The lease is checked again when a gesture's turn comes (one that ended while it waited answers `lease_required`), each one is receipted with who took it, and every result carries `others`: what principals other than you did since your anchor. The agent reads the same field on its own results, so it is informed rather than stopped. The only interruptions are explicit and yours: `POST /v1/runs/{runId}/pause` and `POST /v1/runs/{runId}/cancel`. The response resolves only once the DEVICE applied the gesture. `status` has one value, and an action that did not commit is an error rather than a different status. Every attempt is on the receipt ledger before it is dispatched, so `GET /v1/receipts` can answer what happened even when this response is lost. Send your own `idempotencyKey` and a retry replays that outcome instead of touching the phone twice. A device may refuse a gesture it cannot perform; read its `capabilities` first. A robot arm, for instance, can tap and swipe and cannot type. Hosted deployments only. A local checkout does not mount this route, so calling it there is a 404. ## OpenAPI ````yaml /api/openapi.json post /v1/devices/{deviceId}/actions openapi: 3.1.0 info: title: Phonebase HTTP API version: 0.1.0 summary: >- Every HTTP operation the control plane mounts, generated from the app's own route table. description: >- The control plane exposes two HTTP surfaces, and reading the boundary between them matters more than reading any single endpoint. The CONTROL surface is what your backend and your operators call. It takes a bearer token in the `Authorization` header, and two credential channels resolve to the same authorization context: an OAuth 2.1 access token for interactive clients, or an org scoped API key for headless ones. The DEVICE surface, under `/vrx/{runToken}/v1`, is what a worker calls while executing one run. Its credential is the run token in the url path, and it reads no `Authorization` header at all. Device control itself is not REST. An agent taps, swipes and reads the screen over the Model Context Protocol at `POST /mcp`, one tool call at a time. There is no `POST /v1/devices/{id}/tap`, and if you are looking for one, start with the tool reference instead. Operations marked `"x-phonebase-modes": ["hosted"]` exist only on a hosted deployment; a local checkout does not mount them. servers: - url: '{origin}' description: Your control plane origin. variables: origin: default: https://mcp.phonebase.co enum: - https://mcp.phonebase.co - http://127.0.0.1:8788 description: >- The hosted deployment serves the public origin its metadata document advertises, and that is the default here. Select the loopback entry instead when you are running a local checkout, which serves the port `PORT` names. security: [] tags: - name: Apps description: Save immutable application builds and manage explicit device operations. - name: Managed Proxy description: >- Organization-owned fixed egress resources, independent one-time orders and verified network relationships. - name: Health description: Liveness, unauthenticated. - name: Gateway enrollment description: >- How a new gateway computer trades a one-time enrollment code for its tunnel credential. Unauthenticated: the code is the credential. - name: Gateway releases description: >- What a gateway computer installs: the current edge release of a channel. Called with the gateway's own credential. - name: MCP description: The JSON-RPC endpoint an agent drives devices through. - name: Copilot description: >- Drive a device by chatting. A signed in person's own conversations, streamed. Hosted deployments only. - name: Discovery description: Where to authenticate. Served on hosted deployments only. - name: Runs description: Start, watch and steer a run. - name: API keys description: Mint and revoke headless credentials. Hosted deployments only. - name: Device surface description: What a worker calls with its run token to observe and act. - name: Usage description: Read your org's own receipts and consumption. Hosted deployments only. - name: Devices description: Read the fleet your credential can see. Hosted deployments only. - name: Notifications description: >- What a person needs to know when they are not watching, and which emails they receive. Hosted deployments only. - name: Webhooks description: >- Subscribe your own system to what happens here, and read what was delivered. Hosted deployments only. - name: Provisioning description: >- Whether an organisation is provisioned, and how it becomes so. Hosted deployments only, and both operations answer a declared 503 on a deployment that cannot provision yet. paths: /v1/devices/{deviceId}/actions: post: tags: - Devices summary: Act on a device description: >- One gesture per request: tap, swipe, long press, type text, press a key, open an app. Coordinates are in the normalized 0-1000 grid, so nothing has to know the device's resolution. WHOSE LEASE. Any caller may leave `leaseId` out. The gesture then runs under the lease your organisation currently holds on the device, whoever inside it took that lease (an agent's run, a colleague, you), and the response says so in `coDriving`. When your organisation holds none, a lease is taken for you and the gesture runs under it; if that gesture then fails, the lease taken for it is given back before the error is answered, and your next gesture takes one again. Two people taking an idle device at the same moment both drive it: one took the lease, the other is co-driving under it. When another organisation holds the device the answer is the 409 the acquire route gives. Taking needs `devices:lease`: a key with only `devices:act` acts on a device its organisation already holds. A signed-in person's gesture slides a lease THEY hold when it is running low; it never extends anybody else's. DRIVING BESIDE AN AGENT. A person's tap does not interrupt a run, and does not wait behind it either: gestures on one device run one at a time, a signed-in person's go ahead of every agent gesture still waiting, and the rest keep their arrival order. The lease is checked again when a gesture's turn comes (one that ended while it waited answers `lease_required`), each one is receipted with who took it, and every result carries `others`: what principals other than you did since your anchor. The agent reads the same field on its own results, so it is informed rather than stopped. The only interruptions are explicit and yours: `POST /v1/runs/{runId}/pause` and `POST /v1/runs/{runId}/cancel`. The response resolves only once the DEVICE applied the gesture. `status` has one value, and an action that did not commit is an error rather than a different status. Every attempt is on the receipt ledger before it is dispatched, so `GET /v1/receipts` can answer what happened even when this response is lost. Send your own `idempotencyKey` and a retry replays that outcome instead of touching the phone twice. A device may refuse a gesture it cannot perform; read its `capabilities` first. A robot arm, for instance, can tap and swipe and cannot type. Hosted deployments only. A local checkout does not mount this route, so calling it there is a 404. operationId: postV1DevicesByDeviceIdActions parameters: - name: deviceId in: path required: true description: The device to act on. schema: type: string requestBody: description: One gesture. required: true content: application/json: schema: $ref: '#/components/schemas/BrowserDeviceAction' responses: '200': description: The device applied it. content: application/json: schema: $ref: '#/components/schemas/ActionResult' '400': description: >- The body failed validation: not a JSON object, an unknown `action`, a missing or mistyped parameter, a coordinate outside the grid, a duration outside its bounds, or `refuseIfOthersActed` without `observedFrame`. content: application/json: schema: $ref: '#/components/schemas/Error' '401': description: No credential, or one that was refused. headers: WWW-Authenticate: description: >- A bearer challenge. On a hosted deployment it names the metadata document to discover the authorization server from, and it adds `error="invalid_token"` when a credential was presented and rejected. schema: type: string content: application/json: schema: $ref: '#/components/schemas/UnauthorizedError' '403': description: Acting on a device needs `devices:act`. content: application/json: schema: $ref: '#/components/schemas/Error' '404': description: No such device that this credential can see. content: application/json: schema: $ref: '#/components/schemas/Error' '409': description: >- Five refusals, all about the device's state right now rather than about your credential, each with a remedy. `lease_required`: your organisation does not hold the device and this gesture may not take it (you named a lease that is gone, your key lacks `devices:lease`, or the lease ended while the gesture waited its turn); take it and retry. `device_unavailable` with reason `leased`: another organisation holds the device. `stale_frame`: the frame you sent is no longer current, or somebody else acted after it and the refusal was on (reason `others_acted`); look again. `device_busy`: too many gestures are already waiting on this device; retry after `retryAfterMs`. `action_outcome_unknown`: the gesture was sent and nobody knows whether the screen took it; take a new screenshot before acting again and do not resend it. content: application/json: schema: $ref: '#/components/schemas/Error' '413': description: >- The request body exceeds the server's limit (1 MiB). Refused before the body is parsed, so nothing was partially applied. Retrying the same body will fail the same way: send less. Per-field limits are tighter than this and are documented on the fields themselves; this is the backstop underneath them. The shape is the same on ordinary JSON surfaces, /vrx included. The explicitly configured APK content stream documents its own byte ceiling. content: application/json: schema: $ref: '#/components/schemas/PayloadTooLargeError' '429': description: >- Over the request budget for this caller. Hosted deployments only. Which budget applied depends on whether this request carried an `Authorization` header, not on which endpoint it was aimed at. Wait `Retry-After` seconds and retry the same request: the refusal happens before any work, so nothing was partially applied. See /api/rate-limits. headers: Retry-After: description: >- Whole seconds until one request is available again. Computed from the caller's own remaining allowance, not a fixed constant. schema: type: string RateLimit-Limit: description: >- The budget in effect for this caller class, in requests per minute. schema: type: string RateLimit-Remaining: description: >- Whole requests that were still available when this request was admitted. Zero on a 429. schema: type: string RateLimit-Reset: description: >- Seconds until the allowance is back to full. Longer than `Retry-After`, which is the wait for a single request. schema: type: string RateLimit-Policy: description: The budget and its window, as `;w=60`. schema: type: string content: application/json: schema: $ref: '#/components/schemas/RateLimitedError' '503': description: >- Either the authentication backend could not be reached (the service fails closed), or `device_disconnected`: the device dropped off mid-gesture. The body's `retryable` is true for the second; retry once the device is back. content: application/json: schema: $ref: '#/components/schemas/Error' security: - bearerAuth: [] components: schemas: BrowserDeviceAction: description: >- One gesture, discriminated by `action`. Coordinates are in the NORMALIZED 0-1000 grid with the origin at the top left of the screen in its current orientation, never in screenshot pixels: the same grid the tools use, so a coordinate read off a UI snapshot can be tapped directly and nothing has to know the device's resolution. `leaseId` is OPTIONAL for every caller: the gesture runs under the lease your organisation currently holds on the device, whoever inside it took that lease, and when it holds none one is taken for you, provided your credential may take devices (`devices:lease`). A `leaseId` you send is only an address: your organisation holding the device is what authorizes the gesture. Watching a screen needs no lease; moving it still does, and the response tells you which one it ran under. `refuseIfOthersActed` refuses the gesture if anybody else acted on the device after `observedFrame` was taken. It is on by default for an API key that sends `observedFrame`, off for a person, and off without a frame; send it explicitly to choose. oneOf: - type: object properties: action: type: string const: tap leaseId: $ref: '#/components/schemas/ActionLeaseId' refuseIfOthersActed: $ref: '#/components/schemas/RefuseIfOthersActed' x: type: number minimum: 0 maximum: 1000 'y': type: number minimum: 0 maximum: 1000 idempotencyKey: $ref: '#/components/schemas/ActionIdempotencyKey' observedFrame: $ref: '#/components/schemas/ObservedFrame' required: - action - x - 'y' additionalProperties: false title: tap - type: object properties: action: type: string const: swipe leaseId: $ref: '#/components/schemas/ActionLeaseId' refuseIfOthersActed: $ref: '#/components/schemas/RefuseIfOthersActed' fromX: type: number minimum: 0 maximum: 1000 fromY: type: number minimum: 0 maximum: 1000 toX: type: number minimum: 0 maximum: 1000 toY: type: number minimum: 0 maximum: 1000 durationMs: type: integer minimum: 1 maximum: 10000 description: >- Gesture duration in milliseconds. Defaults to 300; a value of the wrong type is refused, never read as absent. idempotencyKey: $ref: '#/components/schemas/ActionIdempotencyKey' observedFrame: $ref: '#/components/schemas/ObservedFrame' required: - action - fromX - fromY - toX - toY additionalProperties: false title: swipe - type: object properties: action: type: string const: long_press leaseId: $ref: '#/components/schemas/ActionLeaseId' refuseIfOthersActed: $ref: '#/components/schemas/RefuseIfOthersActed' x: type: number minimum: 0 maximum: 1000 'y': type: number minimum: 0 maximum: 1000 durationMs: type: integer minimum: 200 maximum: 10000 description: >- Hold duration in milliseconds. Defaults to 800; below 200 is a tap, not a hold, and is refused. Dwell time is best effort on a physical device, where the arm prioritises landing accurately over holding for exactly this long. idempotencyKey: $ref: '#/components/schemas/ActionIdempotencyKey' observedFrame: $ref: '#/components/schemas/ObservedFrame' required: - action - x - 'y' additionalProperties: false title: long_press - type: object properties: action: type: string const: type_text leaseId: $ref: '#/components/schemas/ActionLeaseId' refuseIfOthersActed: $ref: '#/components/schemas/RefuseIfOthersActed' text: type: string description: >- Text for the focused input, at most 4096 characters. A device may still refuse characters it cannot inject. maxLength: 4096 idempotencyKey: $ref: '#/components/schemas/ActionIdempotencyKey' observedFrame: $ref: '#/components/schemas/ObservedFrame' required: - action - text additionalProperties: false title: type_text - type: object properties: action: type: string const: press_key leaseId: $ref: '#/components/schemas/ActionLeaseId' refuseIfOthersActed: $ref: '#/components/schemas/RefuseIfOthersActed' key: type: string enum: - home - back - enter - delete - app_switch - power - volume_up - volume_down description: >- The key to press. A device supports a subset; read its capabilities. idempotencyKey: $ref: '#/components/schemas/ActionIdempotencyKey' observedFrame: $ref: '#/components/schemas/ObservedFrame' required: - action - key additionalProperties: false title: press_key - type: object properties: action: type: string const: open_app leaseId: $ref: '#/components/schemas/ActionLeaseId' refuseIfOthersActed: $ref: '#/components/schemas/RefuseIfOthersActed' appId: type: string description: 'OS level app id: an Android package name or an iOS bundle id.' idempotencyKey: $ref: '#/components/schemas/ActionIdempotencyKey' observedFrame: $ref: '#/components/schemas/ObservedFrame' required: - action - appId additionalProperties: false title: open_app ActionResult: type: object properties: deviceId: type: string description: The device the gesture was applied to. leaseId: type: string description: >- The lease the gesture ran under: the one you named, your organisation's current one, or the one taken for you. Keep it if you want to release a lease this call took for you; nothing else needs it. coDriving: type: boolean description: >- True when the lease belongs to somebody else inside your organisation, an agent's run most often: you acted beside them, and they were informed through their own `others`. False when it is yours. status: type: string const: committed description: The device applied it. There is no other success. committedAt: type: string description: ISO-8601 instant the device confirmed it, on this service's clock. idempotencyKey: type: string description: >- The key this action was deduplicated under, whether you sent it or the service generated it. deduplicated: type: boolean description: >- True when this outcome was replayed from the ledger and the device was NOT touched again. Absent means the gesture really happened just now. others: $ref: '#/components/schemas/Others' frameNotes: $ref: '#/components/schemas/FrameNotes' required: - deviceId - leaseId - coDriving - status - committedAt - others additionalProperties: false Error: type: object properties: error: type: object properties: code: type: string enum: - backend_resolution_failed - backend_not_implemented - device_unavailable - device_disconnected - lease_mismatch - lease_required - capability_unsupported - invalid_argument - backend_command_failed - idempotency_conflict - action_outcome_unknown - stale_frame - internal_error - unauthorized - permission_denied - run_not_found - frame_not_found - device_busy - admission_unavailable - checkout_unavailable - checkout_customer_balance - checkout_in_progress - quote_changed - payment_in_progress - out_of_stock - wrong_state - instance_not_served - reverification_required - not_found - receipt_not_found - rate_limited - payload_too_large - origin_not_allowed - api_key_from_browser_origin - key_predates_instance_attribution - org_not_served_by_instance - tenant_binding_unavailable - tenant_binding_unsettled - fleet_at_capacity - forbidden - service_unavailable - invalid_signature - invalid_event - not_settled description: >- The closed set of codes this service answers with on purpose, on MCP tools and REST alike (D152). What each one means and when you get it is on the errors page. message: type: string description: A human readable summary. Do not branch on it, branch on `code`. retryable: type: boolean description: >- Whether re-issuing the same call unchanged is meaningful. It is a property of the occurrence, not of the code, so read the field rather than inferring it. required: - code - message additionalProperties: true description: >- Specific errors add their own fields, such as `deviceId`, `reason` or `capability`. Read the ones you branch on and ignore the rest. required: - error additionalProperties: false description: >- The control surface error envelope. Every non 2xx JSON body on this surface has this shape. UnauthorizedError: type: object properties: error: type: object properties: code: type: string const: unauthorized description: Always `unauthorized` on this response. reason: type: string enum: - missing_credentials - malformed_credentials - invalid_token - token_expired - wrong_audience - missing_org - missing_org_role - unknown_key - revoked_key - expired_key description: The closed set of ways authentication can fail. hint: type: string description: >- A next step in prose, on the reasons where there is one to give: `revoked_key`, `expired_key`, `missing_org_role`, and `invalid_token` when the account is not a member of the named organization. Absent otherwise, so branch on `reason` and show this if it is there. required: - code - reason additionalProperties: false required: - error additionalProperties: false description: >- A refused credential. `unknown_key` covers both a keyId nobody has and a secret that does not match, because telling those apart would enumerate keys for you. Revocation and expiry are distinguishable, since you owned that key. PayloadTooLargeError: type: object properties: error: type: object properties: code: type: string const: payload_too_large description: Always `payload_too_large` on this response. message: type: string description: A human readable summary naming the limit in bytes. required: - code - message additionalProperties: false required: - error additionalProperties: false description: >- Returned when the request body exceeds the server limit. Identical on every surface, including /vrx. RateLimitedError: type: object properties: error: type: object properties: code: type: string const: rate_limited description: Always `rate_limited` on this response. message: type: string description: >- A human readable summary. The numbers are in the headers, not in here. required: - code - message additionalProperties: false required: - error additionalProperties: false description: >- Over the request budget. Raised by the rate limiter before any handler runs, which is why its code is not one of the action codes: nothing was attempted, so no action failed. ActionLeaseId: type: string description: >- Optional, for every caller. The gesture runs under the lease your organisation currently holds on this device, or has one taken for it when it holds none (a credential without `devices:lease` is refused `lease_required` instead of taking). A lease id you send is an address, not a credential: your organisation holding the device is what authorizes the gesture, and sending one while your organisation holds nothing is refused `lease_required` rather than turned into a new take. RefuseIfOthersActed: type: boolean description: >- Refuse this gesture (409 `stale_frame`, reason `others_acted`) if another principal acted on the device after `observedFrame` was taken. Sending `true` needs `observedFrame`; without it the body is refused as `invalid_argument`. Left out, it defaults by who is acting: on for an API key, a copilot or a run that sends `observedFrame`, off for a person acting directly, and off whenever no `observedFrame` is sent. You are told through `others` either way. ActionIdempotencyKey: type: string description: >- Your own id for this one action: 1 to 128 printable ASCII characters, and it must not start with "agent:", which is reserved for keys this service issues itself. Send the SAME value when retrying the SAME intended gesture and the recorded outcome is replayed instead of the phone being touched twice. Omit it and the service generates one, which cannot deduplicate your retries. ObservedFrame: type: string description: >- The frame your decision came from. Optional here, unlike on the worker surface: a person driving a live view is looking at the screen as they touch it. When you do send it, the action is refused if that frame is no longer the device's current one, so re-observe and decide again. Others: type: object properties: actions: type: integer description: >- How many actions principals other than you took on this device since your anchor. Pending and committed receipts count; failed ones do not. lastAt: type: - string - 'null' format: date-time description: >- When the newest of them was requested, on this service's clock. Null when `actions` is 0. by: type: array items: type: string enum: - person - agent description: Who they were. description: >- `person` is a signed-in person acting directly; `agent` is an API key, a copilot, or a run. Each appears at most once, person first. required: - actions - lastAt - by additionalProperties: false description: >- What everybody else did on this device since you last looked. On a gesture the anchor is the frame you bound it to, else your previous gesture that reached the device (a refused one never anchors), else the last sixty seconds. A person may drive the same device beside an agent's run; each is informed of the other through this and neither is stopped. FrameNotes: type: object properties: ageMs: type: number minimum: 0 description: >- Milliseconds between the bound frame's capture and the action reaching the arm. newerFrames: type: integer minimum: 0 description: How many frames the arm's camera captured after the bound one. edgeMovedSince: type: boolean description: >- True when the arm moved after the bound frame for a reason that did not come through this API (a calibration or maintenance job). The screen may no longer match what you decided on; look again. settled: type: boolean description: >- False when the bound frame was taken before the screen had settled after the arm's last move. required: [] additionalProperties: false description: >- What the robot arm noticed about the frame this action was bound to. The arm no longer refuses an old or overtaken frame: it performs the gesture and reports here, and you decide whether to take a new screenshot before the next one. Present only on a robot-driven phone whose controller reports it, and only on an action that was bound to a frame; any field may be missing. securitySchemes: bearerAuth: type: http scheme: bearer description: >- The control surface credential. Send `Authorization: Bearer `. Two kinds of token are accepted and they are told apart by shape, not by a separate header. A token beginning `pbk_` is an org scoped API key, whose public half and secret half are generated together and of which only a hash of the secret is ever stored; anything else is treated as an OAuth 2.1 access token and verified against the authorization server's keys. Both resolve to the same context: an org, a principal and a set of scopes. Nothing downstream branches on which channel you used, with one deliberate exception, key management, which requires a signed-in person so that a key can never mint another key. Scopes are enforced when MCP tools are REGISTERED rather than when they are called, so a tool your credential cannot use is absent from `tools/list` rather than refused mid gesture. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.