The miss
I had Claude Code putting together a proposal. It told me, plainly and with full confidence, something about how one of my templates worked. It was wrong. It hadn’t read the template or the spec; it guessed from the name and handed me the guess as fact.
Nothing dramatic happened. I caught it when the finished file didn’t match what it had told me. But the pattern is the real problem: an agent will state something it guessed with the same confidence it uses for something it checked, and from the outside you can’t tell the two apart.
What is a Stop hook?
A Stop hook is a check Claude Code runs the moment the agent finishes a response. It hands the finished message to a small model, which decides whether to let the turn end or send it back for more work. I point one at the agent’s output to catch claims it stated as fact without actually checking them.
If that model returns a block, the reason gets handed back and the agent keeps working instead of stopping. That’s the lever: a verification pass that runs on every response, before I ever see it.
The hook
I added a prompt-type Stop hook to my Claude Code settings. It runs the default fast model, Claude Haiku, over the agent’s final message and looks for one thing: a concrete checkable fact stated with no sign it was actually read, run, or fetched. It watches for version numbers, prices, config defaults, file contents, dates, and specific identifiers.
"Stop": [
{
"hooks": [
{
"type": "prompt",
"prompt": "Review ONLY the assistant's final message this turn. Flag it ONLY if it states, as fact, a concrete verifiable claim in one of these categories: a software or library version number; a price, quote, or dollar figure; a config, API, flag, tool, or template default or behavior; a file's contents, path, or structure; or a specific named identifier or date -- AND gives no sign it was read, run, or fetched this session (phrases like 'I read', 'the file shows', 'per the schema', 'the tool returned', 'ran and got') AND does not hedge it as unverified, inferred, or from memory. Be conservative: flag only concrete checkable facts asserted with certainty and no visible source basis. Never flag opinions, plans, recommendations, questions, hedged statements, or general knowledge. Return {\"ok\": true} if nothing qualifies. Otherwise return {\"ok\": false, \"reason\": \"Unverified claim: <quote it>. Verify it against the authoritative source or mark it unverified before finishing.\"}. Hook input: $ARGUMENTS",
"timeout": 30,
"statusMessage": "Checking for unverified claims..."
}
]
}
]When it flags something, the agent has to verify the claim or mark it unverified before it can finish.
The config above has no model line, so the check runs on Claude Code’s default fast model. Per its hook docs, you can add a model field to run the check on a stronger model when you want more scrutiny.
What it catches, and what it doesn’t
It fired within the first day. The agent told me a GitHub Action in one of my workflows was pinned to version 2.7.0. The workflow actually pinned a full commit SHA, with 2.7.0 sitting beside it as a comment. That gap matters for a supply-chain pin: a tag can be repointed at new code, a pinned SHA can’t. The hook made it cite the SHA the file actually contained.
The limits are worth being honest about.
A prompt hook can’t open files. It doesn’t know whether the claim is true. It only knows whether the claim looks sourced. It would have caught the template mistake, because there was no “I read the template” anywhere near it, but it can’t confirm the right answer. For that you need an agent-type hook, which spawns a subagent that can actually read files and run commands. That’s slower and costs more per turn.
It also costs a model call on every response, and it produces false positives. Real, sourced numbers get flagged when the agent states them without showing its work. Claude Code overrides the hook after it blocks eight times in a row, so it can’t trap the agent in a loop.
Anthropic’s own guidance for reducing this is to give the model explicit permission to say it doesn’t know. The hook makes that a rule instead of a suggestion.
It doesn’t make the agent correct. What it does is force the agent to show where a fact came from, which makes admitting “I haven’t checked this” cheaper than a confident guess. That trade has been worth it for me.
