3.4 KiB
name, description
| name | description |
|---|---|
| html-report-ground-truth-audit | Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix, and prove structure + render |
Use after generating a large HTML report (e.g. via repo-change-html-report) whose claims were partly reconstructed from session memory. Fabricated numbers, invented code blocks, and silently-swallowed sections are the failure modes; the audit makes each traceable to the repo, a live command, or a recorded run.
Procedure
1. Enumerate every checkable claim
Grep the report for numerics, file paths, symbol names, and code blocks:
- line counts (
wc -lthe real file) - test counts (
grep -c '@test' tests/<f>.bats— NOT suite-run totals: conditional skips make raw counts differ; run-observed totals are legitimate run history, keep them labeled as such) - diff stats (
git diff --shortstat main..HEAD— the footer's files/insertions/deletions) - commit hashes (
git log --oneline) - "verbatim" code blocks (see 3)
- diagram node lists and captions (see 4)
2. Ground numeric claims with one command each
Fix every mismatch in place. Record the was/now pair — the audit table IS the evidence.
3. Verbatim code blocks: prove them
For a block labeled verbatim, the exact string must exist:
- live code:
grep -n '<distinctive line>' <file> - historical:
git show <sha> -- <file> | grep -E '^[+-].*(<token1>|<token2>)'If the block was fabricated (plausible-looking pseudo-code), replace it with the real excerpt. If condensed, the block header must say "condensed" and every shown line must be real.
4. Diagram claims: run the engine
Behavioral diagrams (plan traces, closure graphs) must be re-derived by running the actual CLI:
python3 scripts/service-map.py plan --host <h> <path> — then edit node labels, captions, and the prose count ("four services") to match output. Cross-check derived-set diagrams against the source relation (e.g. grep -rl 'Network=gluetun.container' */**/*.container for who rides a tunnel).
5. Check for swallowed sections
Large prior-session PUT ranges silently overwrite neighbors. Grep for headings/captions the plan promised; re-insert any that vanished, anchored on a fresh read of the region.
6. Structural balance
Python tag-balance pass over the file:
- per-tag open/close counts (
div,pre,figure,svg,table,span,figcaption) - running-div-depth scan printing per-section balance to localize stray closers
Common defect:
.ln/.ntpair emitted without its.cgwrapper → stray</div>; fix by opening<div class="cg">before the pair.
7. Render proof
file:// is blocked on the shared browser — serve the dir: bash service python3 -m http.server <port> --bind 127.0.0.1 --directory <dir>, ready on that port. Then:
browser.open({name, url: 'http://127.0.0.1:<port>/<file>'})- DOM assertions via
tab.evaluate(() => …)— NOTtab.run(that runs in Bun; nodocument). Sections are<h2 id>+ siblings: scope queries todocument.body/document.querySelectorAll, not#ch5 …. - screenshot the repaired regions (scroll via
tab.evaluate(el.scrollIntoView())), thentab.close()and kill the server proc (write proc://<name>/kill).
Deliver
A was/now table of every fix with its ground command, plus the list of claims verified intact. State honestly which numbers are run-history and not recomputable.