49 lines
3.4 KiB
Markdown
49 lines
3.4 KiB
Markdown
---
|
|
name: html-report-ground-truth-audit
|
|
description: "Audit a self-generated HTML report: ground every numeric/code/diagram claim against the repo, fix, and prove structure + render"
|
|
---
|
|
|
|
Use after generating a large HTML report (e.g. via `repo-change-html-report`) whose claims were partly reconstructed from session memory. Fabricated numbers, invented code blocks, and silently-swallowed sections are the failure modes; the audit makes each traceable to the repo, a live command, or a recorded run.
|
|
|
|
## Procedure
|
|
|
|
### 1. Enumerate every checkable claim
|
|
Grep the report for numerics, file paths, symbol names, and code blocks:
|
|
- line counts (`wc -l` the real file)
|
|
- test counts (`grep -c '@test' tests/<f>.bats` — NOT suite-run totals: conditional skips make raw counts differ; run-observed totals are legitimate run history, keep them labeled as such)
|
|
- diff stats (`git diff --shortstat main..HEAD` — the footer's files/insertions/deletions)
|
|
- commit hashes (`git log --oneline`)
|
|
- "verbatim" code blocks (see 3)
|
|
- diagram node lists and captions (see 4)
|
|
|
|
### 2. Ground numeric claims with one command each
|
|
Fix every mismatch in place. Record the was/now pair — the audit table IS the evidence.
|
|
|
|
### 3. Verbatim code blocks: prove them
|
|
For a block labeled verbatim, the exact string must exist:
|
|
- live code: `grep -n '<distinctive line>' <file>`
|
|
- historical: `git show <sha> -- <file> | grep -E '^[+-].*(<token1>|<token2>)'`
|
|
If the block was fabricated (plausible-looking pseudo-code), replace it with the real excerpt. If condensed, the block header must say "condensed" and every shown line must be real.
|
|
|
|
### 4. Diagram claims: run the engine
|
|
Behavioral diagrams (plan traces, closure graphs) must be re-derived by running the actual CLI:
|
|
`python3 scripts/service-map.py plan --host <h> <path>` — then edit node labels, captions, and the prose count ("four services") to match output. Cross-check derived-set diagrams against the source relation (e.g. `grep -rl 'Network=gluetun.container' */**/*.container` for who rides a tunnel).
|
|
|
|
### 5. Check for swallowed sections
|
|
Large prior-session PUT ranges silently overwrite neighbors. Grep for headings/captions the plan promised; re-insert any that vanished, anchored on a fresh `read` of the region.
|
|
|
|
### 6. Structural balance
|
|
Python tag-balance pass over the file:
|
|
- per-tag open/close counts (`div`, `pre`, `figure`, `svg`, `table`, `span`, `figcaption`)
|
|
- running-div-depth scan printing per-section balance to localize stray closers
|
|
Common defect: `.ln`/`.nt` pair emitted without its `.cg` wrapper → stray `</div>`; fix by opening `<div class="cg">` before the pair.
|
|
|
|
### 7. Render proof
|
|
`file://` is blocked on the shared browser — serve the dir: `bash` service `python3 -m http.server <port> --bind 127.0.0.1 --directory <dir>`, ready on that port. Then:
|
|
- `browser.open({name, url: 'http://127.0.0.1:<port>/<file>'})`
|
|
- DOM assertions via `tab.evaluate(() => …)` — NOT `tab.run` (that runs in Bun; no `document`). Sections are `<h2 id>` + siblings: scope queries to `document.body`/`document.querySelectorAll`, not `#ch5 …`.
|
|
- screenshot the repaired regions (scroll via `tab.evaluate(el.scrollIntoView())`), then `tab.close()` and kill the server proc (`write proc://<name>/kill`).
|
|
|
|
## Deliver
|
|
A was/now table of every fix with its ground command, plus the list of claims verified intact. State honestly which numbers are run-history and not recomputable.
|