--- name: html-report-ground-truth-audit description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix wrong values, delete filler that restates git, verify the served artifact byte-identical, and flag unverifiable content to the operator." --- Ground-truth audit of an HTML report (generated by an agent, for a human). Three defects hide in a plausible report: **wrong numbers, unverifiable claims, and filler that restates git for a reader who has git.** ## Law Every numeric claim must be re-measured against the repo, not the narrative that produced it. ## Procedure 1. **Dump claims with file:line anchors.** `sed 's/<[^>]*>//g' report.html | grep -vE '^\s*$' | grep -nE 'commit|file|line|fix|refactor' > /tmp/claims.txt` Then enumerate the ground truth: `git log --oneline origin/main..HEAD` for commits; `git diff --stat origin/main..HEAD` for the per-file census — **sort it**: `--sort=-deletions` and `--sort=-creations` give the top movers, so the "largest single change" and "largest net code change" are computed facts. 2. **Check code blocks compile.** Write every snippet to a temp file and `bash -n` each. Common breakages from HTML generation: `"`/`&` entities leaking into code, and `
` line breaks that split a line mid-pipe. 3. **Check the diagrams carry information.** A mermaid `flowchart LR` whose nodes merely restate the bullet list above it is decoration — delete it. 4. **Check counts.** Commit and file counts must match `git rev-list --count` and `--name-only | sort -u | wc -l`. A report whose sections were written by different subagents drifts: each counts its own slice ("4 commits / 14 files" vs the true 11 / 25). 5. **Flag what cannot be verified.** Test pass claims, external tool output, runtime behaviour — if the repo can't confirm it, say so to the operator instead of asserting it. ## Filler test (delete on failure) - A `file → line-count → description` table restating `git diff --stat`? Delete — the reader has git. - A "what this means" / "conclusion" / "key takeaways" section? Delete — it is the same facts in softer prose. (Same rule as vault pages: point of use or gone.) - Keep only what a reader cannot rebuild from `git log` + `git show`. ## Serve and verify the artifact **Verify the file that is served, not the source.** If a `public/` copy is byte-identical, edit both or the served artifact stays wrong. Then re-fetch: ```bash base=http://host:port/reports/x.html for i in $(seq 1 30); do curl -fsS -o /tmp/got.html "$base" && break; sleep 1; done # service may still be activating for i in 1 2; do curl -fsS "$base" | md5sum; done # stable across refetches diff <(curl -fsS "$base") /path/to/source.html && echo "served == source" ``` ## Voice drift If the operator asked for a tone ("no filler", "terse"), the audit must enforce it too — marketing phrasing ("ships the", "This means:") is a style defect, fix it in the same pass, not a later one. ## Report Fixes grouped by defect class (wrong number / unverifiable / filler / style), then what stays unverifiable and why, with the measured ground truth quoted.