91 lines
5.5 KiB
Markdown
91 lines
5.5 KiB
Markdown
---
|
|
name: git-repo-secret-audit
|
|
description: "Audit a git repo for leaked secrets across history, unreachable objects, LFS payloads, git config and host configs — with triage of every hit."
|
|
---
|
|
|
|
# Git repo secret audit (history + unreachable objects, not just the worktree)
|
|
|
|
Worktree greps prove nothing: the leak you care about is in a commit that was amended
|
|
away, in a branch nobody checks out, in an LFS payload, or in `.git/config`.
|
|
|
|
## 0. Build the known-value set from the live secrets, never from literals
|
|
Read the values from the secret store at runtime so no key enters the transcript:
|
|
```python
|
|
known = {}
|
|
for p in (home/".config/secrets").iterdir():
|
|
t = p.read_text().strip()
|
|
vals = {t}
|
|
m = re.search(r":([^:@/]+)@", t) # credential-store URL form
|
|
if m: vals.add(m.group(1))
|
|
known[p.name] = {v for v in vals if len(v) >= 12}
|
|
```
|
|
Also add every secret the repo itself embeds (e.g. credentials living in a *test helper*)
|
|
by concatenating its quoted fragments, then search for the joined value.
|
|
Report ONLY `value[:4] + len + sha256[:10]`. Never print a value, and never quote a
|
|
secret-shaped literal in a doc/commit message — the pre-commit gate will (correctly) block it.
|
|
|
|
## 1. Shapes to match (superset of the Dank gate)
|
|
`sk-[A-Za-z0-9]{18,}`, `sk-or-v1-`, `sk-ant-`, `hf_[A-Za-z0-9]{30,}`, `gh[pousr]_[A-Za-z0-9]{30,}`,
|
|
`github_pat_`, `glpat-`, `xox[baprs]-`, `(AKIA|ASIA)[0-9A-Z]{16}`, `npm_`, `pypi-`, `AIza[0-9A-Za-z_-]{30,}`,
|
|
`(sk|pk)_(live|test)_`, Telegram `\d{8,10}:AA`, JWT `eyJ…\.eyJ…`, `-----BEGIN [A-Z ]*PRIVATE KEY-----`,
|
|
`AGE-SECRET-KEY-1`, `ssh-(ed25519|rsa)\s+AAAA`, `scheme://user:pass@host` basic-auth URLs,
|
|
`Bearer <24+>`, and the noisy `(password|secret|api_key|token)\s*[:=]\s*<12+>` assignment shape.
|
|
|
|
## 2. Scope, in this order
|
|
```bash
|
|
git rev-list --all | wc -l # revs incl. every branch + tag
|
|
git ls-tree -r --format='%(objectname) %(path)' <rev> # sha -> path, per rev
|
|
git cat-file --batch # stream reachable blob content
|
|
git cat-file --batch-all-objects --batch-check # EVERY object, incl. dangling
|
|
git rev-list --objects --all # the reachable set; difference = dangling
|
|
git log --all --oneline --find-object=<sha> # empty => never committed on any ref
|
|
git status --porcelain -uall --ignored=matching # !! lines: on-disk ignored files
|
|
git config --list --show-origin # local/global/system credentials
|
|
git ls-files | grep -E '(auth\.json|trust\.json|\.env|\.pem|\.key|identity)' # quarantine
|
|
```
|
|
**Invocation forms that actually work** (each of these cost a wasted round-trip):
|
|
- `git grep --all -F '<fragment>'` — the options MUST precede the pattern; `git grep -l -F '<frag>' --all`
|
|
treats `--all` as a pathspec and silently returns 0 lines. Grepping a *fragment* (not the full
|
|
value) is what catches split-concatenation disguises like `"Assf.""1234"`.
|
|
- `git ls-tree -r --format=... <rev>` works for ONE rev; pass several revs and it emits nothing,
|
|
no error. For sha→path across all refs use `git rev-list --objects --all`, which yields
|
|
`"<sha> <path>"` lines directly.
|
|
- `git cat-file --batch` fed with `rev-list --objects` lines echoes `sha path` headers, so naive
|
|
header parsing misaligns and content is lost. Feed bare SHAs in chunks, or use
|
|
`git cat-file --batch-all-objects --batch` and parse `"<sha> blob <size>\n<content>\n"` records.
|
|
- Check every worktree/checkout: `git -C <other-checkout> rev-parse HEAD` often differs from the
|
|
audited worktree, so the leak may exist on `main` while the audited tree looks clean.
|
|
|
|
LFS: scan `.git/lfs/objects/**` as BYTES (`decode("latin-1")`) — a screenshot is a classic
|
|
secret carrier and the tracked tree holds only pointers.
|
|
|
|
## 3. Entropy layer (catches issuer-specific keys you did not list)
|
|
Tokens `[A-Za-z0-9+/_-]{32,}` with Shannon entropy >= 4.3 containing both digits and letters.
|
|
Flag every one whose value is in `known` as a LIVE leak; list the rest for triage.
|
|
|
|
## 4. Triage — the shapes that always fire and are fine
|
|
- `api_key="sk-no-key-required"` and other doc placeholders
|
|
- prose: `Set password: user.set_password(...)`, `authenticate(user, password)`
|
|
- 40-hex hits that are git object ids / packfile names (`pack-<40hex>.pack`)
|
|
- memory/KB records that *describe* a secret operation
|
|
- length-threshold gates that miss short passwords: a check like
|
|
`len(val) >= 24 and re.fullmatch(...)` clears a real 12-char password, so a passing
|
|
secret-scan test proves nothing about short credentials
|
|
A hit is a leak only when the value matches a live secret, decodes to a real credential, or
|
|
sits in a field a program reads.
|
|
|
|
## 5. Adjacent host scan (the audit's real yield)
|
|
Check file modes of the secret dir (`0600`, dir `0700`) — world-readable secret files are the
|
|
most common real finding. Then grep the live values across shell rc files, `~/.gitconfig`,
|
|
`~/.npmrc`, `~/.netrc`, `~/.curlrc`, `~/.docker/config.json`, `~/.cache/huggingface/token`,
|
|
`~/.aws/credentials`, `~/.ssh/config` and report filenames only.
|
|
Also check files the deployer *generates* on hosts: a script that writes a credential file
|
|
without `chmod 600` (while sibling writes in the same script do chmod) is the same class.
|
|
|
|
## 6. Report
|
|
Per layer: scope numbers + verdict. Every hit: path:line, redacted value, verdict
|
|
(leak / benign + why). Name the gate's coverage gap if the project's own hook misses shapes
|
|
you matched. Fix modes/chmod on the spot; changes to a security hook are the operator's call.
|
|
A secret in git history is only removable by rotation or history rewrite — say so, don't
|
|
imply an edit fixes it.
|