How agents run
Each agent CLI's flags, what every run is told, and the scorecard.
One program does the rounds.
- The daemon It plans the day, runs each account's agent headless on tickets, supervises leads, reads CI, deploys and errors, and serves the dashboard.
- One at a time per project One ticket run per project at a time, in parallel across projects.
- Secrets In the macOS Keychain or the Linux keyring, or the file you opted in to with
OVERHAND_SECRETS_FILE.
How each agent CLI is run.
- Verified A real run went end to end through Overhand's own arguments, environment, output parser and schema check: a ticket answer validated against the ticket schema, and an edit (create
hello.txt) found on disk afterwards. - Flags checked Checked against the CLI's
--helpand docs only. Recorded output is inpackages/core/test/fixtures/runtimes. - Claude Code Verified; the default and the reference.
claude -p --output-format stream-json --json-schema … --permission-mode auto. Model: the account's backend (see Agents and accounts). - Codex Verified on 0.163.0 with Ollama
qwen3:1.7b.codex exec --json --sandbox workspace-write -c sandbox_workspace_write.network_access=true --output-schema <file>. Model:-m; a gateway account adds--oss --local-provider ollama. - Cursor Agent Verified on 2026.01.23 with model
auto.cursor-agent -p … --output-format json -f. Model:--model. - OpenCode Verified on 1.19.3 with Ollama
qwen3:1.7b.opencode run --format json --auto. Model:-m provider/model, or a gateway account's model. - Aider Verified on 0.86.2 with Ollama (edits on
llama3.1:8b, answers onqwen2.5-coder:7b).aider --message … --yes-always --no-pretty --no-gitignore. Model:--model(ollama_chat/…, oropenai/…through--openai-api-base). - Gemini CLI Flags checked.
gemini -p … -o json --approval-mode yolo --skip-trust. Model:-m.
What the real runs showed.
- Codex's sandbox In
workspace-writeit blocks the network (a sandboxedgit pushfails with "Operation not permitted") and keeps.gitread-only (a sandboxedgit commitfails on.git/index.lock). Overhand turns the network on and makes the repo's git folder writable; a Codex run then committed and pushed to agit://127.0.0.1remote. - Codex and the network Per Codex's config reference, the setting allows outbound network in general, so GitHub is reachable too (not tried).
- Codex's schemas They go to
--output-schemain OpenAI's strict form: closed objects, every property required, optional ones nullable. The nulls are dropped before validation. - Codex on a gateway It runs
--ossagainst the gateway's URL. There Ollama enforces--output-schemaon every model call, so the model can't call tools and never edits (it answered "done" without creating the file). Local runs are asked for JSON in the prompt instead. - Small local models are flaky
qwen2.5-coder:7bwrites tool calls as text, so it can't edit through Codex or OpenCode.qwen3:1.7bcan, but sometimes mistypes a long path or wraps its answer, which Overhand's one retry usually fixes. Use the biggest coding model your machine runs well. - Aider Its edits and the JSON answer come in one reply. An edit it couldn't apply ("did not conform to the edit format") fails the run, even though Aider exits 0.
- Aider leaves no trace No history files land in the repo, and
.aider*goes to.git/info/excludeinstead of the repo's.gitignore. - Cursor Agent On a free plan it only runs model
auto("Named models unavailable" otherwise). - Gemini CLI Not run: its cached login had expired, and an expired login opens a browser prompt and waits until the task timeout instead of failing. In an untrusted folder it also drops
yolo("Approval mode overridden to default"), so Overhand passes--skip-trust.
Answers and output.
- Checked answers Every answer is validated against its schema. A wrong one is retried once, with what was wrong.
- Claude Code only Only Claude Code reports usage windows and takes the Slack inbox's MCP flags; the inbox skips accounts on other agents.
- The live line "What is it doing" and a lead's log read each agent's own output: Claude Code's transcripts, Codex's and OpenCode's JSON events (commands, edits, what it said), Aider's last lines.
- One object at the end Cursor Agent and Gemini CLI print one JSON object when they finish, so their line shows only the result.
- Installed?
overhand doctorchecks each agent CLI in use is installed and shows its version.
What every run is told.
- Overhand's agent instructions The ticket workflow, when to ask instead of guess, and the hard limits. They live in Overhand's own folder (
~/.overhand/agent-home/claude-code/CLAUDE.md), never in your repo. - How each agent gets them Claude Code reads them from its config folder. Codex gets
-c developer_instructions=…, OpenCodeinstructionsin its inline config, and Aider--read. Cursor Agent and Gemini CLI read rules only from the project, so they get them at the head of the prompt. - Your project's own instructions Each project's row in Crew (or the command bar) shows its
CLAUDE.md,AGENTS.md,GEMINI.mdand rules files as written, which agents read each one, and any gaps and contradictions. - A rule from review When review asks for the same thing on two pull requests within 30 days, and no instruction file says it yet, Needs you offers it as a rule. Add rule opens a small draft pull request adding it to the file the project's agent reads.
The scorecard.
- Every run is logged Its project, ticket, kind, account, agent, backend, model, cost at API prices (known costs only; a local model is $0) and outcome.
- Each pull request's fate Merged or closed, fix rounds, change requests and time to merge, joined in from
outcomes.jsonl. Older days are read back from the activity log. - The ranking Health's Scorecard ranks each agent × model over 7 or 30 days, overall and per project: runs, pull requests opened, merge rate, first-try merges, median time to merge, and why the rest didn't land.
- Cost per merge Billed cost per merged pull request ("included in plan" on a plan), beside the API-price cost for comparison. Only pay-per-use and cloud runs count as billed.
- Too few Under 5 runs reads "too few runs to tell".
Routing: what the scorecard changes.
routing.modeSet in the config; a new install hassuggest.suggestWhen another agent and model you've set up merges clearly more often on a project (both with 5+ runs, at least 1.5× and 15 points better), Needs you says so and opens that project's accounts. Nothing changes by itself.autoEach new ticket runs on the project's allowed account (accounts, else all) that scores clearly best. While samples are thin it keeps today's order, and it moves to the next allowed account when one is cooling down or its agent or gateway doesn't answer.- Never spread It never spreads work over several accounts of the same agent and backend (your subscriptions), and leaves projects on Plan and warn only alone.
offToday's order.