All the bugs that are fit to print
My AI coding agent was only allowed to commit code once the code passed my checks. One night the checks failed, the agent committed anyway, and the guard I’d built to stop exactly that did nothing. Here’s why, in plain words, and the rule that fixes it.
My AI coding agent, Claude Code, committed code that had just failed my checks. The guard I’d put in front of it, a small script that has to approve every command first, let it through without a word. To spot a commit, the guard split each command into words with a tool that treats every apostrophe as a quote mark, even inside a commit message, where the shell ignores them. The commit command had five apostrophes. With an odd number, one quote never closed, so the guard couldn’t read the command, and it decided the command wasn’t a commit. The fix is one rule: when the guard can’t tell, it blocks.
Anyone who lets an AI coding agent run commands in their code, and anyone who writes a script whose job is to block things.
Level: beginner to intermediate. The story needs no code. The fix has shell and Python you can copy.
One night in September, my AI coding agent did exactly what I’d built a guard to stop. It ran my checks, the checks failed, and it committed the code anyway. The guard didn’t make a sound.
Here’s the setup, because most of the names in this story are mine, not standards. I work with Claude Code, an AI agent that edits files and runs commands in my repo. Before any change is committed, saved into git’s history, it has to pass the same checks my CI server runs: tests, a linter, a formatter. I keep them all in one script called preflight. It’s my own script, not a git or GitHub feature.
The agent could forget to run preflight, or misread its result. So I added a guard. Claude Code can run a script of yours before each command the agent runs, and that script can block the command. Claude Code calls this a hook. My guard is a hook that asks: is this command a git commit? If it is, did preflight pass for exactly these files? To tell, preflight saves a fingerprint of the files when every check passes, a short code that changes if any file changes, and the guard compares it with a fresh one. If they don’t match, the guard blocks the commit.
That night the guard let the commit through. A fix went in the same night. It closed a real hole, but not the one this commit used. I found the real hole three weeks later, and it came down to five apostrophes.
So how does a guard built to stop exactly this let it through?
git add -A && ./scripts/preflight.sh > <log> 2>&1; echo "preflight_exit=$?" (in the background)Stage all changes and run preflight. Claude Code runs this in the background and saves what it prints, which is one line: preflight_exit=0 if every check passed, preflight_exit=1 if any check failed. $? is the shell’s name for the exit code of the last command.grep "preflight_exit" <task output> && git commit -q -F - <<'EOF'==> ruff format --checkTwo things went wrong here. First, the agent’s command. grep, a tool that finds lines containing some text, only answers “is the line there?”. The line was there. It just said preflight_exit=1, so the commit went ahead. The agent spotted that twelve seconds later. Second, my guard. It existed for exactly this moment, and it let the commit through without a word. That’s the part this post is about. The bad commit itself was harmless: one line a formatter wanted rearranged. A guard that fails silently is not.
incidentI’d copied my guard from an older one. When the original couldn’t read a command that contained git commit, it treated it as a commit. My copy treated it as not a commit.LessonWhen you copy a guard, compare it with the original. The safe line is the easiest one to lose.
incidentThe agent’s command finds the preflight_exit line and commits. The guard can’t read the command, decides it isn’t a commit, and allows it.LessonCheck what a result says, not just that there is one.
dead endThe agent’s first guess is right: the apostrophes. A minute later it tests the guard with shorter commit messages. Three tries, three blocks. It drops the idea: “I cannot pin the bypass on a parsing bug.”LessonTest with the exact input. Its closest test had one apostrophe fewer than the real command.
progressIt finds a different hole: when the fingerprint script fails, the guard also allows the commit.LessonA real bug can still be the wrong suspect.
fixedThat hole is fixed. The agent’s own commit command now refuses anything but preflight_exit=0.
the turnI replay the exact command from the session log against the guard as it was that night. The guard allows it.
Before the agent runs a command, Claude Code hands it to the guard. The guard answers by ending with a number, called an exit code. Exit code 0 means allow. Exit code 2 means block. To choose, my guard went through three steps, drawn below.
First: is this a git commit? To answer, it used shlex, a library that comes with Python and splits a command into words the way a shell would. Then it took a fresh fingerprint of the files and compared it with the one preflight saved.
The guard allowed things when unsure, on purpose. Most commands have nothing to do with git, and a guard that blocks them by mistake gets in the way. That’s fine until the command might be a commit. Two holes turned “I can’t tell” into allowed. That night’s fix closed the second. My commit had gone through the first.
In a shell, an apostrophe starts a quoted piece of text, and the next apostrophe ends it. shlex follows the same rule. But my agent writes long commit messages with a heredoc, a shell way to pass several lines of text to a command. It looks like this: git commit -q -F - <<'EOF', then the message, then a line that says EOF. The shell treats everything in between as plain text, so apostrophes in the message don’t matter. shlex doesn’t understand heredocs. To shlex, every apostrophe in the message is a quote mark.
The real command had five apostrophes: two around 'EOF', and three in words like someone’s. shlex pairs them up, and the fifth is left without a partner, a quote that never closes. shlex stops with an error: No closing quotation. My guard took that error to mean “not a commit”, and allowed the command.
Try it yourself, no guard needed. With four apostrophes, shlex splits the command into words. With three, it stops with an error.
python3 - <<'PY' import shlex even = "git commit -F - <<'EOF'\nCI's and the job's\nEOF" odd = "git commit -F - <<'EOF'\nit's done\nEOF" print(shlex.split(even)[:2]) # ['git', 'commit'] shlex.split(odd) # ValueError: No closing quotation PY
“When a guard can’t tell, it must block.”The moral of incident No. 004
The fix is a rule more than a line of code. Until the guard is sure a command is not a commit, it treats it as one and checks it. If it can’t read a command, it assumes a commit. And once it’s checking, anything that goes wrong blocks the command and says why. Engineers call this failing closed.
There’s one more trap: which exit code blocks. Git has hooks of its own, scripts git runs just before a commit. Those stop the commit on any exit code other than 0. Claude Code’s hooks block only on exit code 2. Exit code 1, or a script that crashes, lets the command run. So a guard that’s safe as a git hook can let things through as a Claude Code hook.
| How the script ends | As a git hook (pre-commit) | As a Claude Code hook |
|---|---|---|
| exit code 0 | commit goes ahead | command runs |
| exit code 2 | commit stopped | command blocked and the reason goes to the agent |
| exit code 1, or a crash | commit stopped | command runs |
Here’s the version I use now, in five steps: a script that fingerprints your files, two lines that save the fingerprint when preflight passes, the guard that compares the saved fingerprint with a fresh one, the setting that tells Claude Code to run the guard, and four tests. If you only want the rule, you can skip the code.
Preflight and the guard must agree on which files were checked. This script turns your last commit, your changes and any new files into one short code, the fingerprint. Change any file and the fingerprint changes. The || echo gone line keeps it going if a temporary file disappears while it runs. Save it as scripts/tree-hash.sh and make it runnable with chmod +x.
#!/usr/bin/env bash
# One hash for HEAD, the diff and untracked files.
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT"
{
git rev-parse HEAD 2>/dev/null || echo no-head
git diff HEAD 2>/dev/null || git diff --cached
git ls-files -o --exclude-standard | sort |
while IFS= read -r f; do
# A file can vanish mid-run: hash it as
# "gone" instead of dying under set -e.
printf '%s ' "$f"
{ shasum -a 256 "$f" 2>/dev/null |
cut -d' ' -f1; } || echo gone
done
} | shasum -a 256 | cut -d' ' -f1Add these lines at the very end of preflight. If any check failed, preflight stops before it gets here, so nothing is saved. The guard’s messages call the saved fingerprint the stamp.
# The last lines of scripts/preflight.sh "$ROOT/scripts/tree-hash.sh" > "$ROOT/.git/preflight-pass" echo "preflight passed: safe to commit"
First, if the command doesn’t contain the word commit, allow it. Then read it with shlex: if it’s clearly not a git commit, allow it, and if shlex can’t read it, treat it as a commit. Last, compare the saved fingerprint with a fresh one, and block with a reason if anything is missing, fails or doesn’t match. Save it as scripts/hooks/commit-guard.sh and make it runnable with chmod +x.
It protects against mistakes, not against someone trying to sneak past. A commit hidden inside bash -c "…" is just text to it.
#!/usr/bin/env bash
# Claude Code PreToolUse hook, matcher "Bash".
# Blocks `git commit` until preflight passed for
# this exact tree. Exit 2 blocks; ANY other exit
# allows, so every unproven path ends in block().
set -uo pipefail
ROOT="${CLAUDE_PROJECT_DIR:-$PWD}"
block() {
echo "BLOCKED: $1" >&2
echo "Run ./scripts/preflight.sh, then commit." >&2
exit 2
}
input="$(cat)"
# 1. Cheap filter: only a sure "no" exits 0.
case "$input" in *commit*) ;; *) exit 0 ;; esac
# 2. Is it a commit? Only a clear "other" escapes.
kind="$(printf '%s' "$input" | python3 -c '
import json, shlex, sys
def kind(cmd):
try:
w = shlex.split(cmd)
except ValueError: # unbalanced quotes
return "commit"
opt = {"-C", "-c", "--git-dir", "--work-tree"}
for i, t in enumerate(w):
if t.rsplit("/", 1)[-1] != "git":
continue
j = i + 1
while j < len(w) and w[j].startswith("-"):
j += 2 if w[j] in opt else 1
if j < len(w) and w[j] == "commit":
return "commit"
return "other"
print(kind(json.load(sys.stdin)["tool_input"]["command"]))
')"
[ "$kind" = "other" ] && exit 0
# 3. In scope now. Every failure below blocks.
stamp="$ROOT/.git/preflight-pass"
[ -s "$stamp" ] || block "no preflight stamp"
now="$("$ROOT/scripts/tree-hash.sh")" \
|| block "could not hash the working tree"
[ -n "$now" ] || block "empty tree hash"
[ "$now" = "$(cat "$stamp")" ] \
|| block "preflight has not passed for this tree"
exit 0Add this to .claude/settings.json in your repo. PreToolUse means “before the agent uses a tool”, and "matcher": "Bash" narrows that to shell commands, which is how the agent commits. "onFailure": "block" (Claude Code v2.1.295 or later) also blocks when the guard itself crashes, won’t start or runs too long. A broken guard then blocks every command until you fix it. That’s on purpose: you’ll notice.
{
"hooks": {
"PreToolUse": [
{
"matcher": "Bash",
"hooks": [
{
"type": "command",
"command": "\"$CLAUDE_PROJECT_DIR\"/scripts/hooks/commit-guard.sh",
"onFailure": "block"
}
]
}
]
}
}Run this from your repo root. It works in a temporary folder, so your real files aren’t touched. Each test prints the guard’s exit code. You want 0 for the first test and 2 for the other three. The third test is the shape that got past my old guard. The fourth uses a fingerprint script that always fails. My old guard printed 0 for all four: the third went through hole 1, and the second and fourth through hole 2, because the temporary folder has no working fingerprint script.
g=scripts/hooks/commit-guard.sh # or your hook
t=$(mktemp -d); mkdir -p "$t/.git" "$t/scripts"
run() { CLAUDE_PROJECT_DIR="$t" "$g" >/dev/null 2>&1
echo "$1: exit $?"; }
echo '{"tool_input":{"command":"ls"}}' | run "ls (want 0)"
echo '{"tool_input":{"command":"git commit -m x"}}' \
| run "commit, no stamp (want 2)"
run "odd apostrophes (want 2)" <<'JSON'
{"tool_input":{"command":"git commit -F - <<'EOF'\nit's done\nEOF"}}
JSON
echo stamp > "$t/.git/preflight-pass"
printf '#!/bin/sh\nexit 1\n' > "$t/scripts/tree-hash.sh"
chmod +x "$t/scripts/tree-hash.sh"
echo '{"tool_input":{"command":"git commit -m x"}}' \
| run "hasher fails (want 2)"Eight minutes after the bad commit, the agent ran its next commit command. This one didn’t stop at finding the line. It read the number in it, and if the number wasn’t 0, it printed REFUSING TO COMMIT and stopped. That’s the rule for any check: read the answer, and when you can’t read it, refuse.
In a Claude Code hook, any ending other than exit code 2 allows, crashes included. Each one should mean “certainly not a commit” or “preflight passed”. If one means “I couldn’t tell”, that’s your hole. Start with the explicit ones:
grep -n 'exit 0' scripts/hooks/*.sh
Copy the real command from the session log into a file, not a shorter version. Feed it to the guard as JSON, the way Claude Code does. Then run the four tests.
python3 -c 'import json,sys
print(json.dumps({"tool_input":
{"command": open(sys.argv[1]).read()}}))' cmd.txt \
| scripts/hooks/commit-guard.sh; echo "exit $?"Print a reason whenever the guard blocks. When it allows a command, the agent never sees what it printed, so while you debug, write each allow to a log file.
log() { printf '%s %s\n' "$(date +%T)" "$*" \
>> "$ROOT/.git/guard.log"; }
log "allow: not a commit"This goes in the agent’s own commit command, before git commit, not in the guard. Take the number out of the line, and refuse anything that isn’t exactly 0, including a missing line.
ec=$(grep -o 'preflight_exit=[0-9]*' pf.log | cut -d= -f2)
[ "$ec" = "0" ] || { echo "REFUSING TO COMMIT"; exit 1; }If the guard lives in several repos, fix the template first, then compare each copy with it.
diff ../template/scripts/hooks/commit-guard.sh \
scripts/hooks/commit-guard.shThe agent put the check and the commit in one command, so nobody, the agent included, looked at the result before the commit ran. A person could write that command too. An agent writes it fast, at any hour, with no one watching.
A guard that blocks by mistake gets noticed in minutes, because you can’t work. A guard that allows by mistake gets noticed only when the thing it guards against actually happens.
Seeing a guard handle normal commands proves little. Feed it strange input and a broken fingerprint script, as the four tests above do.
preflight_exit=0 means every check passed; preflight_exit=1 means at least one failed."block" means a hook that crashes or runs too long also blocks.<<'EOF' … EOF: a way to pass several lines of text to a command.