Intent commands

An intent is a goal-level step such as "fill the email field" or "submit the form and confirm the order page loaded". You describe the control by purpose and accessible hints. bobby finds it, acts, and verifies the result before returning evidence. Use intents instead of raw clicks when you want a step that fails loudly if it did not work.

Intents need the intent:execute capability.

Run an intent

Over MCP, call the matching tool:

{"name": "intent_fill", "arguments": {
  "workflowHandle": "wf_0123456789abcdef0123456789abcdef",
  "purpose": "enter the applicant email",
  "hints": {"role": "textbox", "accessibleName": "Email address"},
  "value": {"kind": "setText", "value": "ada@example.com"}
}}

Over HTTP, wrap the intent in a command envelope and send it to POST /v1/commands:

{"kind": "intent", "input": {"kind": "fill", "input": {"purpose": "enter the applicant email", "hints": {"role": "textbox", "accessibleName": "Email address"}, "value": {"kind": "setText", "value": "ada@example.com"}}}}

The TypeScript SDK builds envelopes with locateEnvelope, fillEnvelope, submitAndVerifyEnvelope, waitForStateEnvelope, followEnvelope, dismissObstructionEnvelope, extractEnvelope, detectChallengeEnvelope and solveChallengeEnvelope. For forms, use completeFormRuntimeCommand with intentEnvelope. The Rust client has the same builders in snake case.

import { fillEnvelope } from "@cavi-ai/bobby-browser";

await client.submit(
  fillEnvelope(meta, "enter the applicant email",
    { kind: "setText", value: "ada@example.com" },
    { role: "textbox", nearText: { kind: "exact", value: "Email address" } }),
  { idempotencyKey: crypto.randomUUID() },
);

meta holds commandId, workflowId, attemptId, sessionId, pageId and deadline.

The intents

The class tells you how to retry. A Replayable intent is safe to repeat. A Reconciliable intent may need a page check before repeating. A Boundary intent changes the world and takes a checkpoint first; after a failure with needsReconciliation, inspect the page instead of retrying. See Events and recovery.

Hints

hints narrow which control matches:

Copy a snapshot node's target into hints so ordinal is kept. In TypeScript, use intentHintsFromAccessibilityTarget(node.target). With role and exact nearText, nearText is the accessible name and purpose stays a free-text task description. A purpose is required and bounded. With no hints, submitAndVerify targets the button named by purpose. With no hints and no control matching purpose, dismissObstruction fails obstructionSuspected naming the open dialog; with visionAssist on, vision closes it and the dialog must close for the intent to complete.

Fill values

fill and each completeForm field take a value with a kind:

A fill succeeds only with postcondition evidence. It also reads the browser's constraint validity. A committed value that violates required, pattern, a range or a type fails with verificationFailed. The evidence has formControlValid and formControlValidationMessage; fix that field and retry.

Fill a whole form

completeForm resolves each field just before filling it, so a conditional field can follow the field that reveals it.

{"name": "intent_complete_form", "arguments": {
  "workflowHandle": "wf_0123456789abcdef0123456789abcdef",
  "purpose": "applicant contact form",
  "fields": [
    {"name": "email", "purpose": "enter the applicant email",
     "hints": {"role": "textbox", "accessibleName": "Email address"},
     "value": {"kind": "setText", "value": "ada@example.com"}},
    {"name": "terms", "purpose": "accept terms",
     "hints": {"role": "checkbox", "accessibleName": "I agree"},
     "value": {"kind": "setChecked", "checked": true}}
  ]
}}

• fields has 1 to 128 entries. Each name is unique and labels the evidence. If name is a controlId from form_snapshot and a field has no hints, bobby uses that control.
• Execution stops at the first failed field. The evidence keeps the fields already filled, so retry only the rest.
• Success evidence is compact by default and lists revealedControls with usable targets. Pass evidenceDetail: "full" to debug.
• The MCP tool accepts top-level hints only when fields has exactly one entry without its own hints. Otherwise it fails with hintsPerField.

Submit and verify

{"name": "intent_submit_and_verify", "arguments": {
  "workflowHandle": "wf_0123456789abcdef0123456789abcdef",
  "purpose": "Place order",
  "expectedState": {"condition": {"kind": "url", "matcher": {"kind": "contains", "value": "/confirmed"}}, "timeoutMs": 30000}
}}

expectedState is required. Use a text or url condition when you know the success state, or networkQuiet when you do not. With networkQuiet the result has submitSettlement:

• settled: the page went quiet with no invalid controls.
• validationRejected: fix the fields listed in formValidation (control, kind, name, target, validity, never values) and do not resubmit blindly.

A second submit in the same workflow fails with boundaryAlreadyExecuted unless you pass reSubmit: true.

Wait conditions

waitForState and the wait_for tool share one condition shape. See the table in MCP tools. state is one of attached, detached, visible, hidden, enabled, disabled. hidden holds once the element is invisible or gone; detached only once it is gone. ready is one of commit, domContentLoaded, interactive, networkIdle.

Follow

intent_follow replaces a click followed by a wait. Give it expectedState (alias expectedDestination; send exactly one). Set boundary: true when the activation changes state, such as signing out. Evidence is compact by default.

Extract

{"name": "intent_extract", "arguments": {
  "workflowHandle": "wf_0123456789abcdef0123456789abcdef",
  "purpose": "read the product details",
  "fields": [
    {"name": "title", "purpose": "product title", "value": {"kind": "text"}},
    {"name": "link", "purpose": "product link", "value": {"kind": "href"}}
  ]
}}

Value kinds are text, attribute (with an attribute name) and href. A field that does not resolve is reported on that field, not as a call failure. Fields target actionable elements. To read plain page text, use inspect or extract_structured.

Vision assist

When deterministic matching is stuck, an intent can ask a vision provider to pick a control from a screenshot. This is off unless all three hold:

1. The caller holds vision:assist.
2. The session was created with executionPolicy.visionAssist = true.
3. A vision backend is configured. See Configuration.

The provider chooses among up to five near-miss candidates, and bobby accepts a proposal only above a 0.75 confidence floor and after verifying it. With [vision].prefill = true (the default), intent_complete_form resolves unresolved fields from one screenshot before it starts. Evidence marks the path as deterministic, visionPrefill or visionFallback.

intent_detect_challenge classifies a captcha and never acts: its challengeDetection evidence holds the type, confidence and whether it blocks, or null for a clean page. intent_solve_challenge runs the solve loop until the challenge clears or timeoutMs passes. bobby does not scan pages for challenges on its own.