docs(vault): fleet-wide fact re-verification pass (67 pages)
The docs-fact census wave re-read the live hosts and re-keyed every page it could: runtime/payload identities, ports, sizes, ledger figures, unit state, and the retirement banners that drifted as the fleet moved. The wave was interrupted mid-run, so this commit carries only what passed the integrity gates: every touched page parses (front matter or H1+status convention intact, balanced fences, no truncated tables, trailing newline), the docs_completeness and link-integrity seams are green, and bats is 384/384. Numbers that this session proved on live hosts are keyed to the measured value, not the page's earlier claim.
This commit is contained in:
1 parent
466fd477d8
commit
e7f0d93c86
67 files changed
+2619
-1152
No files matched your search
@@ -55,7 +55,7 @@ First decide which kind of memory is in question:
|
||||
|---|---|---|
|
||||
| RTX 3090 (Dankclaw) | discrete GDDR6X | `nvidia-smi --query-gpu=memory.total,memory.used,memory.free` |
|
||||
| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` (114 GiB) vs `mem_info_vram_*` (2 GiB carve-out) |
|
||||
| R9700s (Dankserver) | discrete | `rocm-smi` figure (64 GiB, SVM-backed) |
|
||||
| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram` → 34,208,743,424 B = 32,624 MiB (31.86 GiB) per card, read 2026-10-18 — **not** the 64 GiB "SVM-backed" figure this row once carried ([[Dankserver System Specs]]) |
|
||||
|
||||
Per-process truth on NVIDIA:
|
||||
`nvidia-smi --query-compute-apps=pid,process_name,used_memory`. Never add the
|
||||
@@ -66,8 +66,12 @@ co-tenants.
|
||||
contiguous blocks — the honest fragmentation signal: 240 blocks/12.3 GiB
|
||||
after reboot vs 475/14.7 GiB before means the model did not come back into
|
||||
the same hole), device budget, KV pool/working split, "left for everything
|
||||
else". llama.cpp prints at `llama-model.cpp:1827` (per-backend model buffer)
|
||||
and `llama-context.cpp:692` (compute buffer) — but **`llama-bench`
|
||||
else". llama.cpp prints them at `llama-model.cpp:1827` (per-backend model buffer)
|
||||
and `llama-context.cpp:692` (compute buffer) on the era's build; on today's
|
||||
`master` the same `LLAMA_LOG_INFO` lines sit at `llama-model.cpp:2009`
|
||||
("model buffer size =") and `llama-context.cpp:842` ("compute buffer size =")
|
||||
([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/src/llama-context.cpp),
|
||||
re-read 2026-10-18) — but **`llama-bench`
|
||||
suppresses loader INFO and a systemd unit did not put them in the journal**
|
||||
(verified `grep -c` = 0). Recipe: one launch through `llama-cli`/`llama-server`
|
||||
keeping the first ~200 lines.
|
||||
@@ -80,7 +84,10 @@ co-tenants.
|
||||
- **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
|
||||
"move layers to the card" converts reclaimable cache into device memory and
|
||||
frees nothing for co-tenants. A served model's footprint ≠ weight size
|
||||
(frank-zealot: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity
|
||||
(a retired frank-zealot shape: 22.6 of 24.1 GiB for an 11.29 GiB model; the card
|
||||
today reads 24,576 MiB = 24.0 GiB total, `nvidia-smi --query-gpu=memory.total`,
|
||||
2026-10-18, and the deployed payload is the PQ2_0 pair
|
||||
[[frank-zealot-llama-prism-bonsai2-pq20]]). Do the capacity
|
||||
subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
|
||||
28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
|
||||
- **Context lengths: derive, never remember.** `llama-bench` computes
|
||||
@@ -94,9 +101,13 @@ co-tenants.
|
||||
|
||||
## 3. Prompt cache and state sizing
|
||||
|
||||
`--cache-ram` (MiB, default 8192) is HOST RAM for saved prompt states —
|
||||
unpinned, on demand, moves no VRAM. `--ctx-checkpoints` (default 32) is a
|
||||
different mechanism: per-slot KV snapshots; `0` disables them.
|
||||
`--cache-ram` (alias `-cram`, `common/arg.cpp:1719`) is HOST RAM for saved
|
||||
prompt states — unpinned, on demand, moves no VRAM. `--ctx-checkpoints` (alias
|
||||
`-ctxcp`/`--swa-checkpoints`, `common/arg.cpp:1701`) is a different mechanism:
|
||||
per-slot KV snapshots; `0` disables them. Both flags are current upstream; the
|
||||
two defaults quoted here (8192 MiB and 32) are the era's values and were not
|
||||
re-readable in the upstream sources checked *(unverified 2026-10-18)* — print
|
||||
them from the binary (`llama-server --help`) before sizing on them.
|
||||
|
||||
**State law** (compute before trusting any cap): a state is the serialized KV
|
||||
of the whole sequence + constant recurrent state + the draft context's KV:
|
||||
@@ -157,7 +168,7 @@ in the timings. Context cap for full-model residency on that shape stayed
|
||||
## Related
|
||||
|
||||
- [[GGUF Compatibility]] — what a payload is before it is priced
|
||||
- [[frank-zealot-llama-mimo-q6k-2x200704]] — the current two-boot priced shape
|
||||
- [[frank-zealot-llama-mimo-q6k-2x200704]] — the two-boot priced shape of the retired MiMo era (live shape today: [[frank-zealot-llama-prism-bonsai2-pq20]])
|
||||
- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
|
||||
- [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
|
||||
- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side
|
||||
Reference in new issue
Block a user