Configuration

bobby reads config.toml from the scope's config directory (the OS config directory under bobby-browser/ for the personal scope). --config or BOBBY_BROWSER_CONFIG selects another file. The working directory never selects the config, so agents started anywhere reach the same runtime. Relative paths in the file resolve next to it. A missing file uses the defaults below. A malformed file stops startup and names the path.

Credentials never go in config.toml. See Authentication.

[server]

[browser]

Engine selection

The engine is not a TOML key. The runtime resolves it in this order:

1. AUTOMATION_RUNTIME_BROWSER_SELECTION (JSON), when set.
2. The paired profile in browser-selection.json, written by Firefox pairing.
3. Firefox. With no paired profile, startup fails with an actionable error.

A selection that is present but malformed is an error. bobby doctor reports which source won.

A managed Chromium selection can use a durable profile with {"mode": "exact", "engine": "chromium", "profileId": "<name>"}. Its data lives in <profiles_dir>/chromium/<name>. Without profileId, each session gets a disposable profile.

Every session on a durable profile shares one Chrome. A Chrome already running on the profile with remote debugging is attached to; otherwise one starts and keeps running after the runtime exits. Extensions installed in the profile, such as a password manager, run in it. To use an existing profile directory, such as a signed-in Chrome profile, list it:

{
  "preference": {"mode": "exact", "engine": "chromium", "profileId": "signed-in"},
  "chromium": [{"profileId": "signed-in", "profileDir": "/path/to/chrome-profile"}]
}

Start that Chrome with --remote-debugging-port=0 to share it while it runs.

[storage]

[context]

Remembered form structure per site. See Context graph. It opens for profiles with a durable identity: a paired Firefox profile, a Chromium selection with profileId, or managed Chromium (stored under managed-chromium).

Every limit must be positive. See Limits.

[mcp]

A smaller starting set shrinks the tools/list an agent downloads at connect. It only changes what is advertised; hidden tools stay callable. An invalid startup_toolset stops startup, while an invalid BOBBY_MCP_TOOLSET is ignored with a warning. See MCP tools.

[mcp]
startup_toolset = "intent"

[http]

Limits on outbound requests the runtime makes, such as downloads. These are not the server's own listener.

[interface]

[cdp]

See Authenticated CDP.

[observability]

Command logs include a command span with command_id, workflow_id, attempt_id, session_id, command family and class, and runtime_version. command.finished records the returned outcome status and elapsed_ms. If the command future is dropped before returning, the status is interrupted; use the durable journal and recovery APIs to reconcile effects before retrying.

Polled command submissions reaching the authenticated runtime also emit command.request.finished in a command_request span, including authorization rejections and idempotency replays. The span includes correlation, command, workflow, attempt, and session IDs, plus request_mode (submit, oneShotVision, or autoCheckpoint) and whether an idempotency key was present. Keys and bearer credentials are omitted.

Request disposition is runtimeInvoked, replayed, rejected, or interrupted. The event includes outcome, a bounded error_code, elapsed_ms, and runtime_invoked, replayed, and interrupted flags. outcome_command_id links a replay to the original command even when the retry has a new command ID; it is none if no command outcome returned. Error messages are omitted.

Group request events by workflow ID for request, replay, and rejection summaries. Count command.finished separately for execution attempts: invoking the runtime may fail during checkpoint preparation before command execution begins. Neither count proves a browser effect. A dropped waiter is interrupted even if it never invoked the runtime. Transport authentication and parsing failures that never reach the authenticated runtime are outside this accounting boundary. Log filtering and retention can omit events; these are diagnostics, not durable audit records or permission to retry.

Vision calls emit vision.provider.started and vision.provider.finished within the command span. The latter includes provider_mode, elapsed_ms, and proposal_returned. A returned proposal still requires validation and does not mean an action succeeded. These events omit command payloads, URLs, page content, screenshots, and provider error text. Logging filters and operator retention apply; these diagnostics do not replace durable history or enforce provider health.

bobby doctor can check vision health against objectives under [observability.slo]. Both are optional, and an unset one is not evaluated.

Vision

Vision assist lets an intent ask a model to choose a control from a screenshot. It is off unless the caller holds vision:assist, the session sets executionPolicy.visionAssist, and a backend is configured here. See Intent commands.

Set up a provider

bobby vision connect --yes --provider openai
export BOBBY_VISION_TOKEN=...
export OPENAI_API_KEY=...
bobby serve --vision

connect writes the settings below. With a loopback endpoint_url and a selected provider, each runtime starts its own vision proxy on a free loopback port and sends vision requests there. The proxy exits with the runtime. A bobby vision start you run yourself serves endpoint_url only when no provider is selected.

Add --activate to readiness-test the provider right away, and --download-model to let an MLX setup fetch a missing model. Ollama and LM Studio manage their own servers. bobby doctor --fix repeats the readiness test and normalizes bobby-owned settings.

Direct backend

Each [vision.providers.<name>] profile has base_url (required, OpenAI-compatible), model (required) and api_key_env (optional; omit for local servers).

[vision]
endpoint_url = "http://127.0.0.1:9100/vision"
token_env = "BOBBY_VISION_TOKEN"
provider = "myhost"

[vision.providers.myhost]
base_url = "https://vision.example.com/v1"
model = "my-vision-model"
api_key_env = "MY_VISION_API_KEY"

Presets for bobby vision connect --provider:

Override with --base-url and --model. For LM Studio, use the server URL the app shows.

Interactive install and bobby vision connect list available models after you choose a provider. Type to search, select the model, or choose Refresh or Back. Ollama and LM Studio report image support, so known text-only models are excluded. OpenAI-compatible providers that omit image metadata show image support not reported beside their models; check the provider's model documentation before choosing one. MLX lists cached image models and offers download choices with confirmation.

Model lists are cached in memory for 60 seconds, separately for each provider, endpoint and credential. Refresh queries the provider again; a selected list entry is checked again before it is saved. Provider requests time out after 5 seconds. If discovery is unavailable, you can enter an exact model ID manually.

For unattended setup, use --yes --model <id> with vision connect, or --yes --vision-model <id> with install. An explicit model is retained. With no explicit model or existing selection, Ollama uses the default only if it is available, or the sole installed image model; multiple alternatives require a selection. When Ollama cannot be reached, setup can write the tagged default llava:7b; readiness is checked when activation is requested. Ollama reads an untagged model name as :latest; bobby doctor --fix can repair an untagged selection to an installed tag.

The proxy reads at most 1 MiB from an upstream response, counted after decompression, and rejects larger replies. Extracted values are capped at 64 KiB. Errors report the provider and HTTP status without the upstream body.

ACP backend

An ACP harness that already holds the model login (Codex, Claude, OpenCode, Hermes, OpenClaw) can serve vision. bobby never receives that provider token. Each task runs in a fresh child session with bounded text and image content and a strict JSON result.

bobby vision connect --yes --backend acp --provider codex --command codex --arg acp --auth advertised

[vision]
backend = "acp"
profile = "codex"

[vision.acp_profiles.codex]
command = "codex"
args = ["acp"]
auth = "advertised"

auth is one of advertised, oauth-authorization-code, oauth-device-code, environment, existing-session or none. bobby calls the harness's authenticate method that matches the strategy and fails closed if none is advertised. Log in through the harness first. bobby does not read keychains.

Corpus and training data

Before capture, bobby covers editable controls, credential-marked elements and inaccessible frames, and strips URLs of credentials, query strings and fragments. Records are skipped if masking fails. Directories use mode 0700 and files 0600 on Unix. Other page content can still appear in screenshots, so keep these directories local and access-controlled.

[nodes.<name>]

Named vision nodes. A session picks one with executionPolicy.visionNode.

[nodes.local-vision]
kind = "vision"
endpoint_url = "http://127.0.0.1:8080/propose"

A session that names no node escalates to none. A session that names an unknown node is declined; bobby never substitutes another or falls back to a remote default. A session bound to a loopback node cannot send screenshots or page text off the machine. When [nodes] and [vision] are both set, [nodes] wins with a startup warning. With only [vision], its endpoint is a node named vision.