Troubleshooting

No findings appear

Run mcpeval index after new capture data, then run mcpeval promote before mcpeval findings. Promotion requires evidence in two distinct sessions regardless of the selected threshold.

A probe will not start

MCP Eval validates the manifest, case selection, mutation authorization, server target, and required tool declarations before running cases. Check the error and then confirm:

• the manifest has "version": 1 and no unknown fields;
• the selected case or probe kind exists;
• a stdio command or --url is provided, but not both;
• every required tool is returned by the server's tools/list;
• every mutating case names a declared sandbox and the command includes --allow-mutation.

Common errors:

• A missing manifest prints manifest <path> not found; run mcpeval init to scaffold one and exits 2. --manifest defaults to mcp-eval.manifest.json in the working directory.
• A case that fails validation is named in the error, as in probe case slow-read: latency-budget attempts must be between 2 and 20.
• A server command that cannot start prints spawning MCP server <program> with the operating-system cause.

An HTTP endpoint is rejected

Loopback HTTP is accepted by default. A remote endpoint must use HTTPS and requires --allow-remote-http. Remove credentials, query strings, and fragments from the URL. Responses are limited to 8 MiB and use five-second connect, read, and write timeouts.

Check the store boundary

Run the hygiene scan:

mcpeval doctor --check-redaction

The command exits non-zero when a non-note JSONL field looks unredacted, and it prints the salt path as a must-not-share reminder.

Annotation notes are exempt from the automated detectors because they are deliberate user prose. When notes exist, doctor prints a non-failing review warning and can still exit zero, so a successful check does not prove the notes are safe. Review or remove every annotation note before sharing store records. Never include the sibling .salt, manifests, configuration, or index.db.

Generation is refused

generate accepts only a current promoted finding with a valid tool whose recorded arguments are a shape written by capture (finding arguments are not a recorded argument shape otherwise). It also requires --confirm-read-only. By default it creates a new file; use --force to replace an existing output.