docs(vault): strip restated fleet facts from 17 technique pages

Enforces "one fact, one owner": technique pages keep mechanism, laws, traps
and verdicts; per-endpoint shapes, digests and era measurements are cited to
the endpoint page that owns them.

Also repairs the four pointers the pass left empty (each verified against the
cited owner before landing):
- EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]]
- 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]]
- bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]]
- llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]]

Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the
owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a
hard-wrapped [[Dankserver System Specs]] link that failed link integrity.

Orphan census over the diff: 0 numbers lost (comma-insensitive match against
the whole vault plus AGENTS.md).
This commit is contained in:
Frank committed 2026-10-19 01:33:47 -07:00
1 parent eb21dbc705
commit 04f4e2c6d4
18 files changed
+348 -425

No files matched your search

@@ -8,8 +8,8 @@ timestamp: 2026-10-05T00:00:00-07:00
# RDNA4 Container Serving
Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201, 31.86 GiB each =
**63.7 GiB real VRAM**, 246 GiB RAM, 7700X) with
Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201 — card count,
per-card VRAM and host RAM are [[Dankserver System Specs]]'s) with
`docker.io/tcclaviger/vllm` (vLLM 0.29.0.dev, GLM/Qwen-specialised RDNA4 build
with native MoE quant kernels and a first-class expert offload planner). The
tuning history and current deployed shape live in
@@ -155,7 +155,9 @@ podman run -d --name <ctr> --ipc=host --ulimit memlock=-1:-1 \
## 5. Memory budgeting: the levers are not where they look
The planner builds `budget = total × util`, subtracts a KV reserve, and gives
Every figure in this section is a tcclaviger-era boot on the port that is
[[frank-archon]]'s today; the laws they teach are general. The planner builds
`budget = total × util`, subtracts a KV reserve, and gives
the REMAINDER to the expert arena. Consequences, all measured:
- **`--gpu-memory-utilization` grows the ARENA, not the KV pool** (KV stayed
@@ -163,11 +165,14 @@ the REMAINDER to the expert arena. Consequences, all measured:
- **Lowering `--max-model-len` is the WRONG lever and backfires:** a smaller
context shrinks the reserve, handing the space to the arena — measured 65536
→ 0.64 GiB KV, 32768 → 0.34 GiB KV.
- **Hybrid-attention undercount:** the planner's per-token KV figure (9190
B/token for Qwen4Exp) misses the large fixed per-sequence Mamba state
- **Hybrid-attention undercount:** the planner's per-token KV figure for
Qwen4Exp is an undercount — the printed figure and the true one are both on
[[frank-archon]] — because it misses the large fixed per-sequence Mamba state
(~2x), so the reserve is always short and the arena silently covers it.
Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (2 GiB →
113,595 tokens / 3.47x concurrency at 32k). For long context,
Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (era reading: a
2 GiB pin = 113,595 tokens ≈ 3.47x concurrency at 32k — that pair is this
page's record of the tcclaviger boots, not [[frank-archon]]'s). For long
context,
`--kv-offloading-size` (GiB) spills KV to host.
- **`--expert-offload-mem` caps the HOST side, not the VRAM arena**
(`cap = int(x * GIB)` against `host_memory_available()`). Do not reach for
@@ -201,18 +206,21 @@ host RAM = `host-backed set + layers still loading`, not the whole model. At
runtime `apply_hot_cold` sends arena-resident experts down a hot GEMM and the
rest straight over PCIe via UVA views — no staging copy in the critical path.
**Emulation check:** a dequantize-to-compute path doubles real host RAM (118
GiB offload request → ~238 GiB) and is invisible in the weight count. Read the
log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".
**Emulation check:** a dequantize-to-compute path doubles real host RAM — the
measured pair (offload request vs host footprint) is on [[frank-archon]] — and
the doubling is invisible in the weight count. Read the log for `Using
'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".
## 6. Performance profile and honest measurement
Decode on this pair is **PCIe-gather-bound, context-independent**: 12.11
tok/s FLAT from 304 to 16,095 context tokens — KV attention is not the cost.
With ~29 % of experts arena-resident, ~0.84 GB crosses PCIe per token at the
engine's effective read rate; cold-vs-warm differing ~10 % confirms the LRU
works; growth is linear in hit rate, and only speculative decode cuts
sequential STEPS rather than making each cheaper.
Decode on this pair is **PCIe-gather-bound, context-independent**: the rate is
flat from a few hundred to sixteen thousand context tokens, so KV attention is
not the cost (the flat figure and its MTP/gather deltas are
[[Tcclaviger Fork Ops]]'s profile of the same shape). With ~29 % of experts
arena-resident, under a gigabyte crosses PCIe per token at the engine's
effective read rate; cold-vs-warm differing ~10 % confirms the LRU works;
growth is linear in hit rate, and only speculative decode cuts sequential STEPS
rather than making each cheaper.
**Benchmark traps that produced fake numbers here:**