Verification loops

Verified against Claude Code 2.1.311, Codex CLI 0.164.0 · Updated

I do not accept “done” from an agent. I accept a green test run, a passing build, or a browser check I can repeat. Prompts are suggestions. Hooks are gates.

The rule I put in this repo’s AGENTS.md is the same one I want in yours:

Any tool version, command, flag, file path, default, or model name you write or keep must be confirmed against a live source during this run: open the changelog, release notes, or docs page. Put that URL in the page’s sources frontmatter. If you cannot verify a claim, delete it. Do not hedge it.

Swap the content-specific sentence for your stack. The shape stays: name the proof, name the command, refuse the work if the command fails.

What counts as proof

  • Tests. npm test, cargo test, go test ./.... Tell the agent the exact command. If you skip this, it will invent one.
  • Build gates. npm run build on this site. A typecheck. A lint that fails the CI job.
  • Browser checks. Click the path a user hits. Screenshot is not enough if the form never submitted.
  • Review passes. A second agent or a hook that re-reads the diff after the first agent stops.

I use the cheapest proof that would have caught the last mistake. For this repo that is the production build.

In Cursor

Cursor hooks are scripts in .cursor/hooks.json. They receive JSON on stdin and print JSON on stdout. The stop hook runs when the agent finishes. If you return followup_message, Cursor submits that text as the next user message. loop_limit caps those automatic follow-ups. The default is 5. This repo sets loop_limit to 2.

Cloud agents run the command-based project hooks in .cursor/hooks.json. They do not load ~/.cursor/hooks.json. Hooks do not run during an early read-only turn. They start once the agent has a writable environment. stop is supported in cloud agents.

The worked example is .cursor/hooks/build-check.sh. On status: completed it runs npm run build. On failure it returns a followup_message with the last 2000 characters of the log. The hook itself always exits 0 so a broken build becomes a follow-up, not a dead hook.

{
  "version": 1,
  "hooks": {
    "stop": [
      {
        "command": ".cursor/hooks/build-check.sh",
        "timeout": 300,
        "loop_limit": 2
      }
    ]
  }
}

I keep the real check in a script, not in the JSON. The JSON only names the event, the command, and the loop cap.

In Claude Code

Claude Code hooks live in .claude/settings.json (project), ~/.claude/settings.json (user), or managed policy settings. PreToolUse can block a tool call before it runs: exit 2, or return JSON with permissionDecision: "deny" in hookSpecificOutput. Stop runs when Claude finishes responding. It does not run on a user interrupt. Exit 2 from Stop, or JSON decision: "block", keeps Claude working. After eight consecutive continuations with no tool call in between, Claude Code ends the turn. Set CLAUDE_CODE_STOP_HOOK_BLOCK_CAP to raise that cap. PostToolUse is too late to undo a write. Use it to format, lint, or report.

A PreToolUse deny blocks the tool even in bypassPermissions and with --dangerously-skip-permissions. PreToolUse fires before the permission-mode check, including dontAsk. An "allow" from that hook does not bypass a deny rule. That is the point. A deny hook is the only control I trust in an unattended run.

I did not copy this repo’s Cursor hook into Claude settings. The same idea maps to a Stop hook that runs npm run build and refuses the stop when the build fails.

In Codex

Codex has a Stop hook. Put it in hooks.json, or in an inline [hooks] table in config.toml, under ~/.codex/ or the repo’s .codex/. Project hooks load only when that .codex/ layer is trusted. A non-managed command hook stays skipped until you trust its current hash with /hooks.

On Stop, exit 0 and print {"decision":"block","reason":"..."}, or exit 2 and write the reason to stderr. That does not reject the turn. Codex continues and submits reason as the next user prompt. If any matching Stop hook returns continue: false, that wins. stop_hook_active is true when this turn was already continued. For codex exec, pass --dangerously-bypass-hook-trust only when the job already vets the hook source.

The loop

  1. Write the command in AGENTS.md. One row in the commands table.
  2. Wire a hook or a CI job that runs that command when the agent stops or the job ends.
  3. Send the failure output back to the agent. Do not summarize it. Paste the tail.
  4. Cap the retries. Two automatic follow-ups is enough to fix a real break. After that, a human looks.

If you cannot name the command, you do not have a loop. You have a hope.

Sources