live: heartbeat sync 2026-10-21T02:32:51-07:00 (Refs #11)

agent/managed-skills/html-report-ground-truth-audit/SKILL.md

@@ -1,48 +1,49 @@
 ---
 name: html-report-ground-truth-audit
-description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix, and prove structure + render"
+description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix wrong values, delete filler that restates git, verify the served artifact byte-identical, and flag unverifiable content to the operator."
 ---
 
-Use after generating a large HTML report (e.g. via `repo-change-html-report`) whose claims were partly reconstructed from session memory. Fabricated numbers, invented code blocks, and silently-swallowed sections are the failure modes; the audit makes each traceable to the repo, a live command, or a recorded run.
+Ground-truth audit of an HTML report (generated by an agent, for a human). Three defects hide in a plausible report: **wrong numbers, unverifiable claims, and filler that restates git for a reader who has git.**
+
+## Law
+
+Every numeric claim must be re-measured against the repo, not the narrative that produced it.
 
 ## Procedure
 
-### 1. Enumerate every checkable claim
-Grep the report for numerics, file paths, symbol names, and code blocks:
-- line counts (`wc -l` the real file)
-- test counts (`grep -c '@test' tests/<f>.bats` — NOT suite-run totals: conditional skips make raw counts differ; run-observed totals are legitimate run history, keep them labeled as such)
-- diff stats (`git diff --shortstat main..HEAD` — the footer's files/insertions/deletions)
-- commit hashes (`git log --oneline`)
-- "verbatim" code blocks (see 3)
-- diagram node lists and captions (see 4)
+1. **Dump claims with file:line anchors.**
+   `sed 's/<[^>]*>//g' report.html | grep -vE '^\s*$' | grep -nE 'commit|file|line|fix|refactor' > /tmp/claims.txt`
+   Then enumerate the ground truth: `git log --oneline origin/main..HEAD` for commits; `git diff --stat origin/main..HEAD` for the per-file census — **sort it**: `--sort=-deletions` and `--sort=-creations` give the top movers, so the "largest single change" and "largest net code change" are computed facts.
 
-### 2. Ground numeric claims with one command each
-Fix every mismatch in place. Record the was/now pair — the audit table IS the evidence.
+2. **Check code blocks compile.** Write every snippet to a temp file and `bash -n` each. Common breakages from HTML generation: `&quot;`/`&amp;` entities leaking into code, and `<br>` line breaks that split a line mid-pipe.
 
-### 3. Verbatim code blocks: prove them
-For a block labeled verbatim, the exact string must exist:
-- live code: `grep -n '<distinctive line>' <file>`
-- historical: `git show <sha> -- <file> | grep -E '^[+-].*(<token1>|<token2>)'`
-If the block was fabricated (plausible-looking pseudo-code), replace it with the real excerpt. If condensed, the block header must say "condensed" and every shown line must be real.
+3. **Check the diagrams carry information.** A mermaid `flowchart LR` whose nodes merely restate the bullet list above it is decoration — delete it.
 
-### 4. Diagram claims: run the engine
-Behavioral diagrams (plan traces, closure graphs) must be re-derived by running the actual CLI:
-`python3 scripts/service-map.py plan --host <h> <path>` — then edit node labels, captions, and the prose count ("four services") to match output. Cross-check derived-set diagrams against the source relation (e.g. `grep -rl 'Network=gluetun.container' */**/*.container` for who rides a tunnel).
+4. **Check counts.** Commit and file counts must match `git rev-list --count` and `--name-only | sort -u | wc -l`. A report whose sections were written by different subagents drifts: each counts its own slice ("4 commits / 14 files" vs the true 11 / 25).
 
-### 5. Check for swallowed sections
-Large prior-session PUT ranges silently overwrite neighbors. Grep for headings/captions the plan promised; re-insert any that vanished, anchored on a fresh `read` of the region.
+5. **Flag what cannot be verified.** Test pass claims, external tool output, runtime behaviour — if the repo can't confirm it, say so to the operator instead of asserting it.
 
-### 6. Structural balance
-Python tag-balance pass over the file:
-- per-tag open/close counts (`div`, `pre`, `figure`, `svg`, `table`, `span`, `figcaption`)
-- running-div-depth scan printing per-section balance to localize stray closers
-Common defect: `.ln`/`.nt` pair emitted without its `.cg` wrapper → stray `</div>`; fix by opening `<div class="cg">` before the pair.
+## Filler test (delete on failure)
 
-### 7. Render proof
-`file://` is blocked on the shared browser — serve the dir: `bash` service `python3 -m http.server <port> --bind 127.0.0.1 --directory <dir>`, ready on that port. Then:
-- `browser.open({name, url: 'http://127.0.0.1:<port>/<file>'})`
-- DOM assertions via `tab.evaluate(() => …)` — NOT `tab.run` (that runs in Bun; no `document`). Sections are `<h2 id>` + siblings: scope queries to `document.body`/`document.querySelectorAll`, not `#ch5 …`.
-- screenshot the repaired regions (scroll via `tab.evaluate(el.scrollIntoView())`), then `tab.close()` and kill the server proc (`write proc://<name>/kill`).
+- A `file → line-count → description` table restating `git diff --stat`? Delete — the reader has git.
+- A "what this means" / "conclusion" / "key takeaways" section? Delete — it is the same facts in softer prose. (Same rule as vault pages: point of use or gone.)
+- Keep only what a reader cannot rebuild from `git log` + `git show`.
 
-## Deliver
-A was/now table of every fix with its ground command, plus the list of claims verified intact. State honestly which numbers are run-history and not recomputable.
+## Serve and verify the artifact
+
+**Verify the file that is served, not the source.** If a `public/` copy is byte-identical, edit both or the served artifact stays wrong. Then re-fetch:
+
+```bash
+base=http://host:port/reports/x.html
+for i in $(seq 1 30); do curl -fsS -o /tmp/got.html "$base" && break; sleep 1; done   # service may still be activating
+for i in 1 2; do curl -fsS "$base" | md5sum; done    # stable across refetches
+diff <(curl -fsS "$base") /path/to/source.html && echo "served == source"
+```
+
+## Voice drift
+
+If the operator asked for a tone ("no filler", "terse"), the audit must enforce it too — marketing phrasing ("ships the", "This means:") is a style defect, fix it in the same pass, not a later one.
+
+## Report
+
+Fixes grouped by defect class (wrong number / unverifiable / filler / style), then what stays unverifiable and why, with the measured ground truth quoted.