Vol. 1 · No. 004   Developer tools edition

Today I Learned

All the bugs that are fit to print

Incident report · AI coding agents · Git

The Guard That Couldn’t Read, So It Waved the Commit Through

My AI coding agent was only allowed to commit code once the code passed my checks. One night the checks failed, the agent committed anyway, and the guard I’d built to stop exactly that did nothing. Here’s why, in plain words, and the rule that fixes it.

TL;DR

My AI coding agent, Claude Code, committed code that had just failed my checks. The guard I’d put in front of it, a small script that has to approve every command first, let it through without a word. To spot a commit, the guard split each command into words with a tool that treats every apostrophe as a quote mark, even inside a commit message, where the shell ignores them. The commit command had five apostrophes. With an odd number, one quote never closed, so the guard couldn’t read the command, and it decided the command wasn’t a commit. The fix is one rule: when the guard can’t tell, it blocks.

Related to

  • AI coding agents
  • Claude Code hooks
  • Git
  • Shell scripts
  • Python
  • Incident review

What you’ll learn

  1. How to make Claude Code run your own script before every command an AI agent runs, so the script can block it.
  2. Why that script has to end with exit code 2, the number it hands back to Claude Code, to block a command. Exit code 1 lets the command run.
  3. How five apostrophes stopped my guard from reading a command, and the rule that fixes it, with code and four tests.
  4. Why you test with the exact command that failed, not a shorter copy.

Who it’s for

Anyone who lets an AI coding agent run commands in their code, and anyone who writes a script whose job is to block things.

Level: beginner to intermediate. The story needs no code. The fix has shell and Python you can copy.

One night in September, my AI coding agent did exactly what I’d built a guard to stop. It ran my checks, the checks failed, and it committed the code anyway. The guard didn’t make a sound.

Here’s the setup, because most of the names in this story are mine, not standards. I work with Claude Code, an AI agent that edits files and runs commands in my repo. Before any change is committed, saved into git’s history, it has to pass the same checks my CI server runs: tests, a linter, a formatter. I keep them all in one script called preflight. It’s my own script, not a git or GitHub feature.

The agent could forget to run preflight, or misread its result. So I added a guard. Claude Code can run a script of yours before each command the agent runs, and that script can block the command. Claude Code calls this a hook. My guard is a hook that asks: is this command a git commit? If it is, did preflight pass for exactly these files? To tell, preflight saves a fingerprint of the files when every check passes, a short code that changes if any file changes, and the guard compares it with a fresh one. If they don’t match, the guard blocks the commit.

That night the guard let the commit through. A fix went in the same night. It closed a real hole, but not the one this commit used. I found the real hole three weeks later, and it came down to five apostrophes.

So how does a guard built to stop exactly this let it through?

Flight recorder · from the session log

What I saw: the checks failed, then a commit

Session log · September 2026 · seconds from the commit · each line explained under it
  1. agentgit add -A && ./scripts/preflight.sh > <log> 2>&1; echo "preflight_exit=$?" (in the background)Stage all changes and run preflight. Claude Code runs this in the background and saves what it prints, which is one line: preflight_exit=0 if every check passed, preflight_exit=1 if any check failed. $? is the shell’s name for the exit code of the last command.
  2. agentgrep "preflight_exit" <task output> && git commit -q -F - <<'EOF'
    … a message a few dozen lines longSearch that saved output for preflight_exit. If the line is there, commit, with the message written below it.
  3. shellpreflight_exit=1
    <the new commit’s ID and subject line>The line said preflight had failed. The commit happened anyway.
  4. agent“I made an error there: my shell check only confirmed the grep found a line, not that it said zero. Preflight exited 1 and I committed anyway.”
  5. log==> ruff format --check
    1 file would be reformattedThe check that failed: ruff, a Python formatter, wanted one line of a test file laid out differently.

Two things went wrong here. First, the agent’s command. grep, a tool that finds lines containing some text, only answers “is the line there?”. The line was there. It just said preflight_exit=1, so the commit went ahead. The agent spotted that twelve seconds later. Second, my guard. It existed for exactly this moment, and it let the commit through without a word. That’s the part this post is about. The bad commit itself was harmless: one line a formatter wanted rearranged. A guard that fails silently is not.

How I worked it out

Eight minutes to a fix, three weeks to the real cause

  1. incidentI’d copied my guard from an older one. When the original couldn’t read a command that contained git commit, it treated it as a commit. My copy treated it as not a commit.LessonWhen you copy a guard, compare it with the original. The safe line is the easiest one to lose.

  2. incidentThe agent’s command finds the preflight_exit line and commits. The guard can’t read the command, decides it isn’t a commit, and allows it.LessonCheck what a result says, not just that there is one.

  3. dead endThe agent’s first guess is right: the apostrophes. A minute later it tests the guard with shorter commit messages. Three tries, three blocks. It drops the idea: “I cannot pin the bypass on a parsing bug.”LessonTest with the exact input. Its closest test had one apostrophe fewer than the real command.

  4. progressIt finds a different hole: when the fingerprint script fails, the guard also allows the commit.LessonA real bug can still be the wrong suspect.

  5. fixedThat hole is fixed. The agent’s own commit command now refuses anything but preflight_exit=0.

  6. the turnI replay the exact command from the session log against the guard as it was that night. The guard allows it.

Analysis · the flight path

How my guard decided, and the hole my commit used

Before the agent runs a command, Claude Code hands it to the guard. The guard answers by ending with a number, called an exit code. Exit code 0 means allow. Exit code 2 means block. To choose, my guard went through three steps, drawn below.

First: is this a git commit? To answer, it used shlex, a library that comes with Python and splits a command into words the way a shell would. Then it took a fresh fingerprint of the files and compared it with the one preflight saved.

The guard allowed things when unsure, on purpose. Most commands have nothing to do with git, and a guard that blocks them by mistake gets in the way. That’s fine until the command might be a commit. Two holes turned “I can’t tell” into allowed. That night’s fix closed the second. My commit had gone through the first.

Graphic · How my guard decided, the night it failed
the bad commit’s path can’t tell, so allowed allowed (exit code 0) blocked (exit code 2) Agent’s command goes to the guard 1 · A git commit? read it with shlex 2 · Fingerprint files changes if a file does 3 · Preflight passed? matches the saved one? yes ok no can’t read it fails no match allowed not a commit allowed hole 1: can’t read allowed hole 2: no fingerprint blocked the only block the hole my commit went through found and fixed that night
Two amber holes, one blue path. Amber turns I can’t tell into allowed. My commit went through the first amber hole, so it never reached the second, the one fixed that night.
The turn · three weeks later

Why five apostrophes broke it

In a shell, an apostrophe starts a quoted piece of text, and the next apostrophe ends it. shlex follows the same rule. But my agent writes long commit messages with a heredoc, a shell way to pass several lines of text to a command. It looks like this: git commit -q -F - <<'EOF', then the message, then a line that says EOF. The shell treats everything in between as plain text, so apostrophes in the message don’t matter. shlex doesn’t understand heredocs. To shlex, every apostrophe in the message is a quote mark.

The real command had five apostrophes: two around 'EOF', and three in words like someone’s. shlex pairs them up, and the fifth is left without a partner, a quote that never closes. shlex stops with an error: No closing quotation. My guard took that error to mean “not a commit”, and allowed the command.

Graphic · How five apostrophes broke the reader
an example with the same shape as the real command apostrophes numbered in reading order 1 2 3 4 5 line 1, the end git commit -q -F - <<'EOF' message the job's retry further down In CI's log further down the user's file last line EOF inside a quote 'EOF' opens, closes one quoted word, across lines still open at the end No closing quotation shlex gives up with an error so guard: can’t read it so not a commit so allowed, silently exit code 0 so the commit happens though preflight failed
Quote marks the shell ignores and shlex counts. With five apostrophes, the last one is still open when the command ends. That night’s closest test had four, an even number, so shlex could read it and the guard blocked it.

Try it yourself, no guard needed. With four apostrophes, shlex splits the command into words. With three, it stops with an error.

Even splits, odd failsAny terminal with python3
python3 - <<'PY'
import shlex
even = "git commit -F - <<'EOF'\nCI's and the job's\nEOF"
odd = "git commit -F - <<'EOF'\nit's done\nEOF"
print(shlex.split(even)[:2])  # ['git', 'commit']
shlex.split(odd)  # ValueError: No closing quotation
PY
“When a guard can’t tell, it must block.”The moral of incident No. 004
Recommendations · the fix

Make “I can’t tell” mean block

The fix is a rule more than a line of code. Until the guard is sure a command is not a commit, it treats it as one and checks it. If it can’t read a command, it assumes a commit. And once it’s checking, anything that goes wrong blocks the command and says why. Engineers call this failing closed.

There’s one more trap: which exit code blocks. Git has hooks of its own, scripts git runs just before a commit. Those stop the commit on any exit code other than 0. Claude Code’s hooks block only on exit code 2. Exit code 1, or a script that crashes, lets the command run. So a guard that’s safe as a git hook can let things through as a Claude Code hook.

How the script endsAs a git hook (pre-commit)As a Claude Code hook
exit code 0commit goes aheadcommand runs
exit code 2commit stoppedcommand blocked and the reason goes to the agent
exit code 1, or a crashcommit stoppedcommand runs
Graphic · A guard that blocks when unsure
allowed certainly not a commit allowed checks passed Agent’s command goes to the guard Certainly not a commit? allowed only if certain Preflight passed? for these exact files blocked and says why certain not certain can’t read passed not passed script fails And if the guard itself breaks? If it crashes, can’t start or runs too long, Claude Code runs the command anyway, unless the setting "onFailure" is "block" (step 4). if onFailure is block
Two questions, and when unsure, block. Allowed means the guard ends with exit code 0; blocked means exit code 2. The band underneath is Claude Code’s own hole, which only the onFailure setting closes.

Here’s the version I use now, in five steps: a script that fingerprints your files, two lines that save the fingerprint when preflight passes, the guard that compares the saved fingerprint with a fresh one, the setting that tells Claude Code to run the guard, and four tests. If you only want the rule, you can skip the code.

  1. Take a fingerprint of your files

    Preflight and the guard must agree on which files were checked. This script turns your last commit, your changes and any new files into one short code, the fingerprint. Change any file and the fingerprint changes. The || echo gone line keeps it going if a temporary file disappears while it runs. Save it as scripts/tree-hash.sh and make it runnable with chmod +x.

    #!/usr/bin/env bash
    # One hash for HEAD, the diff and untracked files.
    set -euo pipefail
    ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
    cd "$ROOT"
    {
      git rev-parse HEAD 2>/dev/null || echo no-head
      git diff HEAD 2>/dev/null || git diff --cached
      git ls-files -o --exclude-standard | sort |
        while IFS= read -r f; do
          # A file can vanish mid-run: hash it as
          # "gone" instead of dying under set -e.
          printf '%s ' "$f"
          { shasum -a 256 "$f" 2>/dev/null |
              cut -d' ' -f1; } || echo gone
        done
    } | shasum -a 256 | cut -d' ' -f1
  2. Save the fingerprint when preflight passes

    Add these lines at the very end of preflight. If any check failed, preflight stops before it gets here, so nothing is saved. The guard’s messages call the saved fingerprint the stamp.

    # The last lines of scripts/preflight.sh
    "$ROOT/scripts/tree-hash.sh" > "$ROOT/.git/preflight-pass"
    echo "preflight passed: safe to commit"
  3. Write the guard: when unsure, it’s a commit

    First, if the command doesn’t contain the word commit, allow it. Then read it with shlex: if it’s clearly not a git commit, allow it, and if shlex can’t read it, treat it as a commit. Last, compare the saved fingerprint with a fresh one, and block with a reason if anything is missing, fails or doesn’t match. Save it as scripts/hooks/commit-guard.sh and make it runnable with chmod +x.

    It protects against mistakes, not against someone trying to sneak past. A commit hidden inside bash -c "…" is just text to it.

    #!/usr/bin/env bash
    # Claude Code PreToolUse hook, matcher "Bash".
    # Blocks `git commit` until preflight passed for
    # this exact tree. Exit 2 blocks; ANY other exit
    # allows, so every unproven path ends in block().
    set -uo pipefail
    ROOT="${CLAUDE_PROJECT_DIR:-$PWD}"
    
    block() {
      echo "BLOCKED: $1" >&2
      echo "Run ./scripts/preflight.sh, then commit." >&2
      exit 2
    }
    
    input="$(cat)"
    
    # 1. Cheap filter: only a sure "no" exits 0.
    case "$input" in *commit*) ;; *) exit 0 ;; esac
    
    # 2. Is it a commit? Only a clear "other" escapes.
    kind="$(printf '%s' "$input" | python3 -c '
    import json, shlex, sys
    
    def kind(cmd):
        try:
            w = shlex.split(cmd)
        except ValueError:  # unbalanced quotes
            return "commit"
        opt = {"-C", "-c", "--git-dir", "--work-tree"}
        for i, t in enumerate(w):
            if t.rsplit("/", 1)[-1] != "git":
                continue
            j = i + 1
            while j < len(w) and w[j].startswith("-"):
                j += 2 if w[j] in opt else 1
            if j < len(w) and w[j] == "commit":
                return "commit"
        return "other"
    
    print(kind(json.load(sys.stdin)["tool_input"]["command"]))
    ')"
    [ "$kind" = "other" ] && exit 0
    
    # 3. In scope now. Every failure below blocks.
    stamp="$ROOT/.git/preflight-pass"
    [ -s "$stamp" ] || block "no preflight stamp"
    now="$("$ROOT/scripts/tree-hash.sh")" \
      || block "could not hash the working tree"
    [ -n "$now" ] || block "empty tree hash"
    [ "$now" = "$(cat "$stamp")" ] \
      || block "preflight has not passed for this tree"
    exit 0
  4. Tell Claude Code to run it

    Add this to .claude/settings.json in your repo. PreToolUse means “before the agent uses a tool”, and "matcher": "Bash" narrows that to shell commands, which is how the agent commits. "onFailure": "block" (Claude Code v2.1.295 or later) also blocks when the guard itself crashes, won’t start or runs too long. A broken guard then blocks every command until you fix it. That’s on purpose: you’ll notice.

    {
      "hooks": {
        "PreToolUse": [
          {
            "matcher": "Bash",
            "hooks": [
              {
                "type": "command",
                "command": "\"$CLAUDE_PROJECT_DIR\"/scripts/hooks/commit-guard.sh",
                "onFailure": "block"
              }
            ]
          }
        ]
      }
    }
  5. Prove it blocks: try to break it

    Run this from your repo root. It works in a temporary folder, so your real files aren’t touched. Each test prints the guard’s exit code. You want 0 for the first test and 2 for the other three. The third test is the shape that got past my old guard. The fourth uses a fingerprint script that always fails. My old guard printed 0 for all four: the third went through hole 1, and the second and fourth through hole 2, because the temporary folder has no working fingerprint script.

    Four tests: want 0, 2, 2, 2Repo root, bash or zsh
    g=scripts/hooks/commit-guard.sh   # or your hook
    t=$(mktemp -d); mkdir -p "$t/.git" "$t/scripts"
    run() { CLAUDE_PROJECT_DIR="$t" "$g" >/dev/null 2>&1
            echo "$1: exit $?"; }
    
    echo '{"tool_input":{"command":"ls"}}' | run "ls (want 0)"
    echo '{"tool_input":{"command":"git commit -m x"}}' \
      | run "commit, no stamp (want 2)"
    run "odd apostrophes (want 2)" <<'JSON'
    {"tool_input":{"command":"git commit -F - <<'EOF'\nit's done\nEOF"}}
    JSON
    echo stamp > "$t/.git/preflight-pass"
    printf '#!/bin/sh\nexit 1\n' > "$t/scripts/tree-hash.sh"
    chmod +x "$t/scripts/tree-hash.sh"
    echo '{"tool_input":{"command":"git commit -m x"}}' \
      | run "hasher fails (want 2)"

Eight minutes after the bad commit, the agent ran its next commit command. This one didn’t stop at finding the line. It read the number in it, and if the number wasn’t 0, it printed REFUSING TO COMMIT and stopped. That’s the rule for any check: read the answer, and when you can’t read it, refuse.

Finding revised
If this happens to you

A guard let something through: check these in order

  1. Find every place it allows

    In a Claude Code hook, any ending other than exit code 2 allows, crashes included. Each one should mean “certainly not a commit” or “preflight passed”. If one means “I couldn’t tell”, that’s your hole. Start with the explicit ones:

    grep -n 'exit 0' scripts/hooks/*.sh
  2. Replay the exact command, then try to break it

    Copy the real command from the session log into a file, not a shorter version. Feed it to the guard as JSON, the way Claude Code does. Then run the four tests.

    python3 -c 'import json,sys
    print(json.dumps({"tool_input":
      {"command": open(sys.argv[1]).read()}}))' cmd.txt \
      | scripts/hooks/commit-guard.sh; echo "exit $?"
  3. Make every exit explain itself

    Print a reason whenever the guard blocks. When it allows a command, the agent never sees what it printed, so while you debug, write each allow to a log file.

    log() { printf '%s %s\n' "$(date +%T)" "$*" \
      >> "$ROOT/.git/guard.log"; }
    log "allow: not a commit"
  4. Read the number, not the line

    This goes in the agent’s own commit command, before git commit, not in the guard. Take the number out of the line, and refuse anything that isn’t exactly 0, including a missing line.

    ec=$(grep -o 'preflight_exit=[0-9]*' pf.log | cut -d= -f2)
    [ "$ec" = "0" ] || { echo "REFUSING TO COMMIT"; exit 1; }
  5. Fix every copy

    If the guard lives in several repos, fix the template first, then compare each copy with it.

    diff ../template/scripts/hooks/commit-guard.sh \
         scripts/hooks/commit-guard.sh
Key takeaways
  1. When a guard can’t tell, it must block. Allow only when certain: certainly not a commit, or preflight passed. Everything else is blocked, with a reason.
  2. Test with the exact input. A shorter test message made the real cause look innocent. The real command exposed it on the first run.
  3. A fix isn’t done until every copy has it. Copies drift. Mine had lost the one safe line its original had.
Field guide · if you let an AI agent commit

What this taught me about guarding an AI agent

Its mistakes look like checks

The agent put the check and the commit in one command, so nobody, the agent included, looked at the result before the commit ran. A person could write that command too. An agent writes it fast, at any hour, with no one watching.

Silent failure is the worst kind

A guard that blocks by mistake gets noticed in minutes, because you can’t work. A guard that allows by mistake gets noticed only when the thing it guards against actually happens.

Attack your own guards

Seeing a guard handle normal commands proves little. Feed it strange input and a broken fingerprint script, as the four tests above do.

Glossary

Words from this issue, and where each comes from

Commitgit
Saving a snapshot of your changes into git’s history.
CIgeneral
Checks a server runs automatically on every change you push.
Preflightmy own script
Runs CI’s checks on my machine before a commit.
preflight_exitmy own label
The line printed when preflight finished. preflight_exit=0 means every check passed; preflight_exit=1 means at least one failed.
Guardmy own name
My script that allows or blocks each command before the agent runs it. Claude Code calls it a hook.
HookClaude Code
A script of yours that Claude Code runs at set moments, for example before each command (a PreToolUse hook). It blocks only by ending with exit code 2. Git has hooks too, and those stop a commit on any exit code other than 0.
Fingerprintgeneral (a hash)
A short code computed from files. Change one character and the code changes.
Stampmy own name
The saved fingerprint: what preflight writes when every check passes.
onFailureClaude Code
A hook setting. "block" means a hook that crashes or runs too long also blocks.
Exit codeshell
The number a program ends with. 0 means success; any other number means some kind of failure.
grepshell tool
Finds lines that contain some text.
Heredocshell
<<'EOF' … EOF: a way to pass several lines of text to a command.
shlexPython
Splits a command into words the way a shell would. A quote that never closes is an error.