docs(vault): strip restated fleet facts from 17 technique pages
Enforces "one fact, one owner": technique pages keep mechanism, laws, traps and verdicts; per-endpoint shapes, digests and era measurements are cited to the endpoint page that owns them. Also repairs the four pointers the pass left empty (each verified against the cited owner before landing): - EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]] - 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]] - bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]] - llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]] Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a hard-wrapped [[Dankserver System Specs]] link that failed link integrity. Orphan census over the diff: 0 numbers lost (comma-insensitive match against the whole vault plus AGENTS.md).
This commit is contained in:
Frank committed
1 parent
eb21dbc705
commit
04f4e2c6d4
18 files changed
+348
-425
No files matched your search
@@ -23,8 +23,9 @@ guesswork.
|
||||
each near the target; record `nvidia-smi` used/total plus the boot log's
|
||||
`n_slots`/`n_ctx_slot`: shape A = current/cheap, shape B = new demand.
|
||||
`non-KV = used − (tokens × KiB/token)`; the two rows separate
|
||||
weights/buffers/graphs from the new demand (a 918 MB projector bills
|
||||
~1.4 GiB).
|
||||
weights/buffers/graphs from the new demand; a projector is billed after the
|
||||
fit at roughly 1.5x its file size ([[frank-zealot]] Contract has that
|
||||
measurement).
|
||||
3. **Solve the band:** `KV_tokens ≤ (card − non-KV − headroom_low) /
|
||||
KiB/token`, round the pool DOWN to a multiple of `PARALLEL × 4096` so every
|
||||
slot gets an equal integer window. Boot and measure; if outside the band,
|
||||
@@ -54,8 +55,8 @@ First decide which kind of memory is in question:
|
||||
| device | kind | read it with |
|
||||
|---|---|---|
|
||||
| RTX 3090 (Dankclaw) | discrete GDDR6X | `nvidia-smi --query-gpu=memory.total,memory.used,memory.free` |
|
||||
| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` (114 GiB) vs `mem_info_vram_*` (2 GiB carve-out) |
|
||||
| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram` → 34,208,743,424 B = 32,624 MiB (31.86 GiB) per card, read 2026-10-18 — **not** the 64 GiB "SVM-backed" figure this row once carried ([[Dankserver System Specs]]) |
|
||||
| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` vs `mem_info_vram_*`; the aperture and carve-out values are [[Dankclaw System Specs]]'s |
|
||||
| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram`; the per-card byte/MiB value (and the stale "64 GiB SVM-backed" figure it replaced) is [[Dankserver System Specs]]'s |
|
||||
|
||||
Per-process truth on NVIDIA:
|
||||
`nvidia-smi --query-compute-apps=pid,process_name,used_memory`. Never add the
|
||||
@@ -63,9 +64,10 @@ iGPU's GTT to host RAM as a separate pool; `free -g` is the arbiter for
|
||||
co-tenants.
|
||||
|
||||
- **Engine ledger first.** Halogen prints weights pinned (+ remaining 2 MiB
|
||||
contiguous blocks — the honest fragmentation signal: 240 blocks/12.3 GiB
|
||||
after reboot vs 475/14.7 GiB before means the model did not come back into
|
||||
the same hole), device budget, KV pool/working split, "left for everything
|
||||
contiguous blocks — the honest fragmentation signal: a block census that drops
|
||||
across a restart means the model did not come back into the same hole; the
|
||||
before/after census this was learned on is that era's, on [[frank-dragoon]]),
|
||||
device budget, KV pool/working split, "left for everything
|
||||
else". llama.cpp prints them at `llama-model.cpp:1827` (per-backend model buffer)
|
||||
and `llama-context.cpp:692` (compute buffer) on the era's build; on today's
|
||||
`master` the same `LLAMA_LOG_INFO` lines sit at `llama-model.cpp:2009`
|
||||
@@ -83,13 +85,13 @@ co-tenants.
|
||||
graph/compute pool).
|
||||
- **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
|
||||
"move layers to the card" converts reclaimable cache into device memory and
|
||||
frees nothing for co-tenants. A served model's footprint ≠ weight size
|
||||
(a retired frank-zealot shape: 22.6 of 24.1 GiB for an 11.29 GiB model; the card
|
||||
today reads 24,576 MiB = 24.0 GiB total, `nvidia-smi --query-gpu=memory.total`,
|
||||
2026-10-18, and the deployed payload is the PQ2_0 pair
|
||||
[[frank-zealot]]). Do the capacity
|
||||
subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
|
||||
28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
|
||||
frees nothing for co-tenants. A served model's footprint is not its weight
|
||||
size: the MiMo Q5_K_M era booted a 6.40 GiB payload to **20,530 MiB used** of the
|
||||
3090's addressable 24,576 — the Eras row and its addressable ≠ advertised law are on
|
||||
[[frank-zealot]]. What the card reports used is weights + KV + compute buffers +
|
||||
projector, never weights alone. Do
|
||||
the capacity subtraction before designing, on the card number the owner page
|
||||
quotes, not the advertised one.
|
||||
- **Context lengths: derive, never remember.** `llama-bench` computes
|
||||
`n_ctx = n_prompt + n_gen + n_depth` (defaults 512/128/0, type_k/v F16,
|
||||
flash_attn AUTO, split_mode LAYER). `-p 512,16384 -n 64` = contexts
|
||||
@@ -112,10 +114,10 @@ them from the binary (`llama-server --help`) before sizing on them.
|
||||
**State law** (compute before trusting any cap): a state is the serialized KV
|
||||
of the whole sequence + constant recurrent state + the draft context's KV:
|
||||
`per_token = 2 × n_full_attn_layers × n_kv_heads × head_dim × bytes (K and V) +
|
||||
draft_per_token`. Measured on frank-medium (qwen3.5 hybrid, f16, MTP draft):
|
||||
16×4×256×2×2 = 64 KiB/token + 4 KiB draft = **68 KiB/token**; a full
|
||||
196,608-token slot state ≈ 12.8 GiB. Cross-check against the journal's
|
||||
`prompt state size` lines — the two must agree.
|
||||
draft_per_token`. The worked qwen35-hybrid instance of that formula (full layer
|
||||
count, f16 KV, MTP draft) and the slot-size table it prices are on
|
||||
[[frank-medium-llamacpp-dflash2]]; cross-check any such arithmetic against the
|
||||
journal's `prompt state size` lines — the two must agree.
|
||||
|
||||
**The journal is the instrument:** `exceeds cache size limit … skipping` =
|
||||
cap binding in production; `making room … removing oldest` / `cache size
|
||||
@@ -162,13 +164,13 @@ Only the charlie12345/ROCmFPX build-qwen38 (PR #27742 qwen4exp merged) takes
|
||||
it; CIRU, upstream and stock builds do not. Validate live: load with
|
||||
`--flash-attn on`, health 200, a real completion finishing `"stop"`, journal
|
||||
clean of `GGML_ASSERT`/`cudaMalloc`/`missing tensor`, and flash-attn engaged
|
||||
in the timings. Context cap for full-model residency on that shape stayed
|
||||
32,768.
|
||||
in the timings. The context cap that build reached for full-model residency is
|
||||
part of that era's record on [[frank-dragoon]].
|
||||
|
||||
## Related
|
||||
|
||||
- [[GGUF Compatibility]] — what a payload is before it is priced
|
||||
- [[frank-zealot]] — the two-boot priced shape of the retired MiMo era (live shape today: [[frank-zealot]])
|
||||
- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
|
||||
- [[frank-zealot]] — the endpoint whose Eras rows are the two-boot priced shapes this method produced
|
||||
- [[frank-medium-llamacpp-dflash2]] — the state-law worked example behind section 3
|
||||
- [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
|
||||
- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side
|
||||
Reference in new issue
Block a user