docs(vault): fleet-wide fact re-verification pass (67 pages)

The docs-fact census wave re-read the live hosts and re-keyed every page
it could: runtime/payload identities, ports, sizes, ledger figures, unit
state, and the retirement banners that drifted as the fleet moved. The
wave was interrupted mid-run, so this commit carries only what passed the
integrity gates: every touched page parses (front matter or H1+status
convention intact, balanced fences, no truncated tables, trailing
newline), the docs_completeness and link-integrity seams are green, and
bats is 384/384.

Numbers that this session proved on live hosts are keyed to the measured
value, not the page's earlier claim.
This commit is contained in:
Frank committed 2026-10-18 13:15:43 -07:00
1 parent 466fd477d8
commit e7f0d93c86
67 files changed
+2619 -1152

No files matched your search

@@ -55,7 +55,7 @@ First decide which kind of memory is in question:
|---|---|---|
| RTX 3090 (Dankclaw) | discrete GDDR6X | `nvidia-smi --query-gpu=memory.total,memory.used,memory.free` |
| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` (114 GiB) vs `mem_info_vram_*` (2 GiB carve-out) |
| R9700s (Dankserver) | discrete | `rocm-smi` figure (64 GiB, SVM-backed) |
| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram` → 34,208,743,424 B = 32,624 MiB (31.86 GiB) per card, read 2026-10-18 — **not** the 64 GiB "SVM-backed" figure this row once carried ([[Dankserver System Specs]]) |
Per-process truth on NVIDIA:
`nvidia-smi --query-compute-apps=pid,process_name,used_memory`. Never add the
@@ -66,8 +66,12 @@ co-tenants.
contiguous blocks — the honest fragmentation signal: 240 blocks/12.3 GiB
after reboot vs 475/14.7 GiB before means the model did not come back into
the same hole), device budget, KV pool/working split, "left for everything
else". llama.cpp prints at `llama-model.cpp:1827` (per-backend model buffer)
and `llama-context.cpp:692` (compute buffer) — but **`llama-bench`
else". llama.cpp prints them at `llama-model.cpp:1827` (per-backend model buffer)
and `llama-context.cpp:692` (compute buffer) on the era's build; on today's
`master` the same `LLAMA_LOG_INFO` lines sit at `llama-model.cpp:2009`
("model buffer size =") and `llama-context.cpp:842` ("compute buffer size =")
([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/src/llama-context.cpp),
re-read 2026-10-18) — but **`llama-bench`
suppresses loader INFO and a systemd unit did not put them in the journal**
(verified `grep -c` = 0). Recipe: one launch through `llama-cli`/`llama-server`
keeping the first ~200 lines.
@@ -80,7 +84,10 @@ co-tenants.
- **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
"move layers to the card" converts reclaimable cache into device memory and
frees nothing for co-tenants. A served model's footprint ≠ weight size
(frank-zealot: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity
(a retired frank-zealot shape: 22.6 of 24.1 GiB for an 11.29 GiB model; the card
today reads 24,576 MiB = 24.0 GiB total, `nvidia-smi --query-gpu=memory.total`,
2026-10-18, and the deployed payload is the PQ2_0 pair
[[frank-zealot-llama-prism-bonsai2-pq20]]). Do the capacity
subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
- **Context lengths: derive, never remember.** `llama-bench` computes
@@ -94,9 +101,13 @@ co-tenants.
## 3. Prompt cache and state sizing
`--cache-ram` (MiB, default 8192) is HOST RAM for saved prompt states —
unpinned, on demand, moves no VRAM. `--ctx-checkpoints` (default 32) is a
different mechanism: per-slot KV snapshots; `0` disables them.
`--cache-ram` (alias `-cram`, `common/arg.cpp:1719`) is HOST RAM for saved
prompt states — unpinned, on demand, moves no VRAM. `--ctx-checkpoints` (alias
`-ctxcp`/`--swa-checkpoints`, `common/arg.cpp:1701`) is a different mechanism:
per-slot KV snapshots; `0` disables them. Both flags are current upstream; the
two defaults quoted here (8192 MiB and 32) are the era's values and were not
re-readable in the upstream sources checked *(unverified 2026-10-18)* — print
them from the binary (`llama-server --help`) before sizing on them.
**State law** (compute before trusting any cap): a state is the serialized KV
of the whole sequence + constant recurrent state + the draft context's KV:
@@ -157,7 +168,7 @@ in the timings. Context cap for full-model residency on that shape stayed
## Related
- [[GGUF Compatibility]] — what a payload is before it is priced
- [[frank-zealot-llama-mimo-q6k-2x200704]] — the current two-boot priced shape
- [[frank-zealot-llama-mimo-q6k-2x200704]] — the two-boot priced shape of the retired MiMo era (live shape today: [[frank-zealot-llama-prism-bonsai2-pq20]])
- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
- [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side