Frank
04f93133fc
The gate scanned only `*.md *.yml *.yaml *.json *.toml *.sh *.ts *.js`. A `postgres.env`, a `.container` quadlet, an `.ini`, or an extension-less dotfile — exactly the shapes the dotfiles tier is made of — was never read at all. Discovered while deciding what else belongs in `dotfiles/`: `~/.config` carries `containers/systemd/postgres.env` with a real `POSTGRES_PASSWORD`, which the ignore list did not cover either (`git check-ignore` says so: `.env` does not match `postgres.env`). Token SHAPES now scan the entire staged diff, every file type; bcrypt hashes and `Authorization: Basic` headers joined the shape list. Credential ASSIGNMENTS are scanned in non-prose files only — the auto-learned memory tier is prose *about* credentials (7 such hits measured on the tracked tree), and a gate that fires there gets overridden until it means nothing. Markdown keeps shape detection, which is what it had before. Three invariants, each pinned by a test in tests/secret-gate.bats (13 tests, black-box: the hook is copied into a scratch repo and git itself runs it): - Fail closed. The first version of this rewrite passed `-I` to `git diff`, which is not valid there; the empty file list made every secret commit sail through, and a blind gate looks exactly like a clean tree. If the hook cannot read the staged diff it now refuses the commit. - Only ADDED lines can leak. The gate refused the redaction commit in this branch because the diff still carried the removed password — a gate that blocks the removal of a secret holds the leak hostage. - Path-valued variables (`…_FILE=`, `…_PATH=`) and placeholder/indirected values are not secrets, so the gate stays worth listening to. `.gitignore` quarantines `*.env` at any name, and gains the theme-media rule whose untracking lands in the next commit. `AGENTS.md` states the new contract. Refs #40
104 lines
5.3 KiB
Bash
Executable File
104 lines
5.3 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Secret gate: block any commit whose staged diff contains live-secret shapes or
|
|
# credential assignments. This repo IS the deployment and every workstation
|
|
# pulls it, so a committed secret is instantly in every agent's context window
|
|
# on the fleet. Deliberate override: ALLOW_SECRETS=1 git commit ...
|
|
#
|
|
# Two rules, deliberately different in scope:
|
|
#
|
|
# 1. Token SHAPES are scanned across the ENTIRE staged diff, whatever the file
|
|
# type. The first version of this gate whitelisted extensions
|
|
# (*.md *.yml *.yaml *.json *.toml *.sh *.ts *.js), so a `postgres.env`, a
|
|
# `.container` quadlet, an `.ini`, or an extension-less dotfile — exactly
|
|
# the shapes the dotfiles tier is made of — was never read at all.
|
|
# 2. Credential ASSIGNMENTS (PASSWORD=…, secret_key: …) are scanned only in
|
|
# non-prose files. The auto-learned memory tier is prose ABOUT credentials,
|
|
# and a gate that fires there gets overridden until it means nothing; the
|
|
# 2026-10-20 audit measured 7 such prose hits on the tracked tree. Markdown
|
|
# keeps shape detection only, which is what it had before.
|
|
#
|
|
# Bcrypt hashes and Basic-auth headers joined the shape list on 2026-10-20
|
|
# (issue #40) after a live Caddy basicauth hash plus its plaintext password —
|
|
# and the Pi-hole internal password — were found in a tracked memory file.
|
|
set -uo pipefail
|
|
|
|
shapes='sk-[A-Za-z0-9]{18,}|ghp_[A-Za-z0-9]{36}|github_pat_[A-Za-z0-9_]{22,}|glpat-[A-Za-z0-9_-]{20}|xox[baprs]-[A-Za-z0-9-]{10,}|AKIA[A-Z0-9]{16}|ASIA[A-Z0-9]{16}|BEGIN [A-Z ]*PRIVATE KEY|aes_[A-Za-z0-9/+]{40}|eyJ[A-Za-z0-9_-]{20,}\.ey|\$2[aby]\$[0-9]{2}\$[./A-Za-z0-9]{20,}|Authorization:[[:space:]]*[Bb]asic[[:space:]]+[A-Za-z0-9+/=]{24,}'
|
|
|
|
# `newpass1234`, `secret_key: "rotated-value"` — an assignment with a literal
|
|
# value. `$VAR`, `${VAULT}`, <placeholders> and empty values cannot match.
|
|
assign='[A-Za-z0-9_]*(PASSW|SECRET|TOKEN|API_?KEY|ACCESS_KEY|PRIVATE_KEY|CREDENTIAL)[A-Za-z0-9_]*[[:space:]]*[:=][[:space:]]*["'"'"']?[A-Za-z0-9/+._@-]{8,}'
|
|
placeholders='changeme|change[-_]|placeholder|redacted|example|sample|dummy|nokey|no[-_]key|not[-_]needed|your[-_]|localhost|xxxx|yyyy|\$\{|<[a-z]|\*\*\*|%s|null|false|true'
|
|
|
|
# Exemptions: a variable naming a location (`…_FILE=`, `…_PATH=`) or a value
|
|
# that is a filesystem path is a pointer, not a credential. Flagging those is
|
|
# how a gate gets permanently overridden.
|
|
assign_exempt='(_FILE|_PATH|_DIR|_URL|_ENDPOINT|_SOCKET)[[:space:]]*[:=]|[:=][[:space:]]*["'"'"']?(/|\.\./)'
|
|
# Fail closed: a gate that cannot read the staged diff must not report success.
|
|
if ! files=$(git diff --cached --diff-filter=ACM --name-only 2>&1); then
|
|
{
|
|
echo "SECRET GATE: cannot read the staged diff — refusing to commit."
|
|
printf ' %s\n' "$files"
|
|
echo "Override deliberately: ALLOW_SECRETS=1 git commit ..."
|
|
} >&2
|
|
[ -n "${ALLOW_SECRETS:-}" ] && exit 0
|
|
exit 1
|
|
fi
|
|
# stderr is captured so the failure branch can quote it; on success it must not
|
|
# leak a `warning:` line into the file list and have it mistaken for a path.
|
|
files=$(printf '%s\n' "$files" | grep -vE '^(warning|fatal|error):' || true)
|
|
[ -z "${files//[[:space:]]/}" ] && exit 0
|
|
|
|
report=""
|
|
exempt=""
|
|
count=0
|
|
while IFS= read -r f; do
|
|
[ -z "$f" ] && continue
|
|
# Self-exemption, deliberately an exact filename list and not `tests/`: the
|
|
# gate's fixtures must contain credential-shaped strings or there is nothing to
|
|
# test against, and this hook documents the shapes it looks for. A secret parked
|
|
# in any other test file is still caught.
|
|
case "$f" in
|
|
.githooks/pre-commit|tests/secret-gate.bats) exempt="$exempt $f"; continue ;;
|
|
esac
|
|
diff=$(git diff --cached -U0 -- "$f")
|
|
[ -z "$diff" ] && continue
|
|
# Only ADDED lines can leak. Matching removals too blocks the very commit that
|
|
# deletes a credential — which is exactly what happened on 2026-10-20, when the
|
|
# gate refused the redaction of the password it exists to catch.
|
|
added=$(printf '%s\n' "$diff" | grep -E '^\+' | grep -vE '^\+\+\+' | sed 's/^+//' || true)
|
|
[ -z "$added" ] && continue
|
|
hits=$(printf '%s\n' "$added" | grep -Ei "$shapes" || true)
|
|
if [ "${f##*.}" != "md" ]; then
|
|
assigns=$(printf '%s\n' "$added" | grep -Ei "$assign" | grep -Ev "$assign_exempt" | grep -Eiv "$placeholders" || true)
|
|
hits=$(printf '%s\n%s\n' "$hits" "$assigns" | sed '/^[[:space:]]*$/d')
|
|
fi
|
|
if [ -n "$hits" ]; then
|
|
n=$(printf '%s\n' "$hits" | wc -l)
|
|
count=$((count + n))
|
|
report+=" $f"$'\n'"$(printf '%s\n' "$hits" | cut -c1-100 | sed 's/^/ /')"
|
|
fi
|
|
done <<< "$files"
|
|
|
|
# Say so out loud: an exemption that prints nothing is how a carve-out becomes
|
|
# invisible, and invisible is the property a secret wants.
|
|
if [ -n "$exempt" ]; then
|
|
printf 'secret-gate: self-test fixtures exempt:%s\n' "$exempt" >&2
|
|
fi
|
|
|
|
if [ -n "${ALLOW_SECRETS:-}" ]; then
|
|
[ -n "$report" ] && printf 'secret-gate: ALLOW_SECRETS set, allowing %s secret-shaped string(s) through.\n' "$count"
|
|
exit 0
|
|
fi
|
|
|
|
if [ -n "$report" ]; then
|
|
{
|
|
echo "SECRET GATE: staged diff contains $count secret-shaped string(s):"
|
|
printf '%s\n' "$report"
|
|
echo "Fix it, or override deliberately: ALLOW_SECRETS=1 git commit ..."
|
|
echo "A credential that reaches a tracked file is already in history on every"
|
|
echo "machine that pulls this repo — deleting it later does not un-leak it."
|
|
} >&2
|
|
exit 1
|
|
fi
|
|
exit 0
|