docs(vault): documentation exists for point of use — destroy the ledgers (−13.7k lines) #31

Merged
dank merged 13 commits from docs/vault-destruction-20261019 into main 2026-10-19 10:59:51 -07:00
13 Commits
Author SHA1 Message Date
Frank a3bce7d123 docs(agents): documentation exists for point of use
*Documentation Policy* rewritten: a page earns its bytes only if it changes what
the reader does next. Measurements are disposable — a rate for anything not
running and not being compared is destroyed, not filed in an Eras row or a
technique page. "It is the only copy" is explicitly NOT a reason to keep a fact.
`log.md` retired from the policy; the Runbook Index no longer promises a page per
retired endpoint (rollback is a commit, not a document).

*Key Lessons Learned* purged of the dead: the dual-R9700 board-swap narrative and
the layer-split/x4-pair experiment laws (no live endpoint uses either), the QSA
patch note and the retired Kairic/FastMTP traps, and the throughput figures inside
the laws that survive — the law stays, the number goes. Two lessons added, both
from this campaign's own failures: a verdict is a lead, the scout's and the tool's
alike; and never let a census decide what to keep, because a tool can say where a
fact lives but not whether anything needs it.

997 → 820 lines.
2026-10-19 10:56:37 -07:00
Frank 327e78b950 docs(vault): destroy the dead weight — 53 pages and the change diary
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for
payloads that no longer run. A throughput figure for a runtime nobody operates is
not a record: AI moves faster than the vault, so the number is wrong before
anyone reads it, and a wrong number in a runbook gets believed. Per the new
purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed.

Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest):
- the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM,
  the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch
  campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two
  "Running Qwen3.6 …" pages;
- the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag)
  and the multi-device/x4-pair/hybrid/offload experiment pages;
- every vLLM page — no vLLM endpoint exists on this fleet;
- `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced
  owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff,
  self-hosted app census, the TTS survey) that were obsolete by construction;
- `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the
  page that owns the thing it governs, never to a diary.

Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras
tables keep the window, payload, runtime, reason and rollback pointer; the rate
columns are blanked. The live shapes stay verbatim — those numbers are what the
deployer and `models.yml` consume.
2026-10-19 10:56:37 -07:00
Frank 8775e64ed0 feat(scripts): tools to destroy documentation, not preserve it
Point-of-use enforcement needs hands, not advice:

- `destroy-vault.py` — moves the pages on a kill list to `.AGENTTRASH/` and then
  deletes every reference to them from the survivors, including the whole line
  when the dead link was the line's only content. Stems are seeded from the trash
  so the pass is re-runnable; `pathlib.Path.stem` is NOT used to compare link
  targets, because it strips the real suffix off names like `sd.cpp`.
- `axe-measurements.py` — deletes bench/throughput ledger sections and rate rows,
  blanks any table cell that reports a rate, and folds Eras older than the current
  one into a single row that says the numbers were destroyed and that rollback is
  a commit. Never touches a shape number or a Law/Wire/Rollback section.
- `repair-damage.py` — fixes the two artifacts those passes can leave (a list
  marker that lost its space, a table separator left without its header).
- `restore-punctuation.py` — restores a line whose only difference from HEAD is
  stripped trailing punctuation. Written after `repair-damage.py`'s first version
  matched every wrapped prose line ending in a comma and deleted 660 of them: the
  rule is now `… see .` only, and the incident is recorded in its docstring.
- `gen-vault-index.py` — never walks into `.AGENTTRASH/`, and sweeps rows whose
  page cell is empty (a description left nameless by a destruction).

Full suite: 451 tests pass.
2026-10-19 10:56:37 -07:00
Frank 52dbe18486 docs(vault): the census was lying; corrected, and the honest result is flat docs
`scripts/port-census.py` built its corpus from the new directory, so each page
under test was held by itself: every era page reported 0 orphan lines. With the
corpus excluding the file under test, each candidate holds 14-63 sole-record
lines, so no era page is padding and none was trashed.

Deletions of restatement only:
- Multi-Node Runtimes.md (-257): restated launch material owned by
  Qwen3.8 PP Deployment.md; index rows and two Related bullets repointed.
- log.md: 66 headings folded into 35 dated entries (one entry per date, newest
  first), titled headings kept as `date — Title` per the spec's wording, and 78
  lines of transient deploy chatter dropped where every fact token in the line
  is held by an owner page (preserved in .AGENTTRASH/20261019-log-transient/).
- Authentic: the 5-page split reverted (-93); the parent carries
  `okf-exempt: length` and the three stub pages stay deleted.

check-okf.py accepts `## YYYY-MM-DD — Title`; port-census.py excludes the file
under test and gained a docstring note about the trap. One-off compaction tool
moved to ~/.AGENTTRASH. Full suite: 451 tests pass, 0 OKF violations,
155 pages, 0 unresolved links.
2026-10-19 10:17:52 -07:00
Frank 2a12487653 fix(vault): delete the restated consolidation page, not the vault
`Multi-Node Runtimes.md` (+257) restated launch material that
[[Qwen3.8 PP Deployment]] (build target, two-process RPC table, cache-prune
cron), [[Sample Script Set]] (runnable scripts), [[Ethernet Bandwidth Math]]
and [[Expected Performance]] already own, and mixed-vendor numbers owned by
[[x4-pair-experiment]]. Consolidation means the facts end up in ONE place, not
that a new page is written beside the old owners: the four stubs it replaced
were frontmatter-only (9 lines each) and needed no successor. Page trashed,
three inbound links repointed at the real owners.
2026-10-19 09:58:04 -07:00
Frank a483e51804 Merge origin/main: reconcile the frank-zealot reslice with the dedupe pass
main moved under this campaign — PR #25/#26 resliced the RTX 3090 endpoint.
The frontmatter conflict is resolved in favour of the live shape, confirmed
by re-probe the same day: `--ctx-size 602112 --parallel 3` (3 × 200,704),
`--cache-type-k f16 --cache-type-v q8_0`, `--spec-type draft-mtp`, `--jinja
--reasoning auto`, `--metrics`. The 262,144 figures that remain on the page
are era rows and the checkpoint's trained window, not the live shape.

log.md's 2026-10-19 fleet-state bullet now records both probes and the law
this taught: a docs pass that reports live shapes must re-probe before
writing the number, because main moving mid-campaign makes the first probe
stale.
2026-10-19 05:12:41 -07:00
Frank c860fb3c8c docs(vault): the frank-medium locator's two wrong cells
The ROCmFPX row said "single card, pinned at 36e9acd": `36e9acd` is the
SHA that `scripts/build-rocmfpx.sh` pins for the *build*, and the era page
records the deployed commit as `c49ebdb` running a 2× R9700 `-sm layer`
split over `ROCm0,ROCm1` at 131,072 × 1 slot with draft-MTP pinned to
`ROCm1`. The 2026-09-20 row skipped a payload generation: the same day the
endpoint swapped MXFP4 → FP8 E4M3 128×128 ([[frank-medium-model-swap]])
before MXFP6 arrived on 09-21. Both cells now quote the era pages.
2026-10-19 05:09:48 -07:00
Frank fb8862b24c docs(vault): verified corrections with sources, not assertions
- frank-archon KV dtype law: `bf16|fp8` is a **radiance** restriction. Stock
  vLLM advertises `auto|fp8|fp8_e5m2|fp8_e4m3` (docs.vllm.ai engine args), and
  even there the advertised set is not the usable set — several accepted
  values crash engine init at the attention layer's own dtype check
  (vllm-project/vllm#42587). The page stated the restriction as if upstream
  had it.
- Open Notebook: re-censused 2026-10-19 — no unit file, no installed quadlet,
  no container. The page already said NOT deployed; it now says what was
  checked, so the claim is re-checkable instead of remembered.
2026-10-19 05:08:47 -07:00
Frank 293c25c50b docs(agents): compress the load-bearing lessons, repoint their owner links
Four bullets in Key Lessons Learned carried narrative a vault page owns: the
2026-09-20 board swap ([[Dual-R9700-Cooperation]]), the `--override-tensor`
gate ([[Llama Cpp Multi-Device Splits]]), the x4 link budget
([[x4-pair-experiment]]), and the silent-RAM incident plus the Prometheus
`--metrics` law ([[frank-archon]], [[frank-zealot]]). Each keeps its law, its
numbers and its error strings; the causal narration that duplicated a page is
what went. The RAM bullet linked to [[log]] — the incident is owned by
[[frank-archon]], so the link now names the owner.
2026-10-19 05:08:29 -07:00
Frank 483fd843b7 fix(scripts): port-census must not call prose words facts
A hyphen or a trailing period inside a word was enough to make it a tracked
token, so `long-standing`, `deployed:` and `cleaned.` were reported as lost
facts while real identifiers hid behind them. Require a digit, dot, slash,
colon, `--`, two hyphens (`frank-tts-engine`) or ALL-CAPS in the stripped
word. The census of the log rewrite went from 979 noisy lines to a set whose
unheld tokens are all fact-shaped.
2026-10-19 05:08:22 -07:00
Frank a5fe045b58 docs(vault): one fact, one owner — repair the doubled pages, single-source the facts
Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
2026-10-19 05:00:43 -07:00
Frank 3880503057 feat(scripts): OKF conformance linter, trash-aware vault seams
Add scripts/check-okf.py: enforces the OKF v0.1 hard rules the repo had
been carrying by convention — parseable frontmatter with type/title/
description/timestamp, description shape (length, not-equals-title, no
restated quantity), the reserved-name and index.md rules, log.md's date
heading shape (bare ISO date, one entry per date, newest first), the
500-line hint, and the doubled-page signature (a second `type:` key in
the body). Citations are reported as a soft tally, not a failure. The
500-line ceiling is escapable two ways, both explainable: `okf-exempt:
length` in a page's own frontmatter, and pages derived as one-per-
endpoint — live endpoints from scripts/, retired-endpoint era pages from
their tag (AGENTS.md lets a campaign ledger live on the page that
measured it).

scripts/gen-vault-index.py: sibling-presence was tested as a bare stem
substring, so a page whose name is a prefix of another page's name
(`frank-medium` inside `frank-medium-rocmfpx`) silently never got an
index row. Test the wikilink form instead.

scripts/check-vault-links.py: `.AGENTTRASH` is deleted-but-held state,
not vault content. It is now excluded from both the scan set and the
target index, so a live link into the trash reads as unresolved instead
of resolved.

tests/docs_completeness.bats: the same exclusion in both page walks —
trashed pages need no index row, and a trashed page must not satisfy the
docs requirement for a live compose service. Adds the OKF conformance
and index-freshness tests.

scripts/port-census.py: the port-before-cut orphan census used to prove
a merge lost no fact, kept so the next collapse can re-run it.
2026-10-19 04:54:06 -07:00
Frankandomp 1ed7f0ddf2 docs(agents): compress Key Lessons to one law per bullet, correct the FA-quants law
Key Lessons Learned 483 -> 340 lines. Every law survives; only the essays
went. Port-before-cut census over the diff: 454 deleted lines, 0 orphaned
facts (every number still has a holder in the vault).

- Each lesson is now one bullet with the page that owns the detail.
- De-duplicated the page's own internal repeats: the layer-split law was
  told twice (PCIe and ROCm sections), the FA-quants law twice more.
- CORRECTION, verified on dankclaw 2026-10-19: `GGML_CUDA_FA_ALL_QUANTS` is
  deprecated upstream (master's ggml/cmake/common.cmake tells you to use
  GGML_CUDA_FA_QUANTS=all), the successor's default pair list
  `q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16` compiles the quantized-KV kernels
  with the old flag OFF (master build-cuda cache: FA_ALL_QUANTS=OFF), and
  neither llama tree references the old macro under ggml/src/ggml-cuda/.
  The "required or prefill collapses ~12x" law is obsolete; the new law is
  to read the CMakeCache pair list.
- frank-zealot: ported the three facts AGENTS.md alone carried, and verified
  them against the host first — the 2026-10-18 13:16 cutover boot really did
  load PTQ1_0 at `new slot, n_ctx = 133888` x3 (journal), the mmproj fit
  check really printed 873.10 MiB against a 21,439 MiB projection, and the
  warmup spread is 2.6 -> 1,258 t/s prefill.
- New tooling: scripts/check-okf.py (OKF v0.1 conformance linter) and
  scripts/gen-vault-index.py (root index regenerated from frontmatter).

Co-Authored-By: omp <omq@openclaw.sh>
2026-10-19 02:51:32 -07:00