docs(vault): strip restated fleet facts from 17 technique pages
Enforces "one fact, one owner": technique pages keep mechanism, laws, traps and verdicts; per-endpoint shapes, digests and era measurements are cited to the endpoint page that owns them. Also repairs the four pointers the pass left empty (each verified against the cited owner before landing): - EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]] - 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]] - bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]] - llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]] Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a hard-wrapped [[Dankserver System Specs]] link that failed link integrity. Orphan census over the diff: 0 numbers lost (comma-insensitive match against the whole vault plus AGENTS.md).
This commit is contained in:
1 parent
eb21dbc705
commit
04f4e2c6d4
18 files changed
+348
-425
No files matched your search
@@ -41,7 +41,7 @@ From the 111 GiB GGUF (`mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`,
|
||||
- 48 layers, of which **12 are full attention** (indices `[3,7,11,15,19,23,27,31,35,39,43,47]`,
|
||||
every 4th — `full_attention_interval = 4`) and 36 are GDN/linear.
|
||||
- Only the 12 attention layers carry KV. At the engine's measured 28 KiB per
|
||||
position, a 200k-token conversation reads **5.5 GiB of KV per token**.
|
||||
position, a 200k-token conversation holds **5.5 GiB of KV**.
|
||||
|
||||
- A layer split moves **whole layers**, so it moves ~1.62 GiB of experts with
|
||||
every layer (75 GiB / 48). Attention layers are spread across the whole depth
|
||||
@@ -82,10 +82,12 @@ Real placement in the split runs, sampled from `nvidia-smi` during the run:
|
||||
split is genuine, not a mislabelled single-device run (see Finding 5 for why
|
||||
that check is mandatory here).
|
||||
|
||||
Reference points on the same silicon: halogen (deployed) ≈ **25 tok/s** at real
|
||||
contexts; the retired ik hybrid 3.8–4.6 tok/s; mainline with the trunk on the
|
||||
3090 and CPU experts 1.01 tok/s (2026-09-15); the CPU expert path's prefill
|
||||
ceiling 192 tok/s.
|
||||
Reference points, all era readings rather than measurements of this run: the
|
||||
deployed halogen shape at ≈ **25 tok/s** at real contexts and the retired ik
|
||||
hybrid at 3.8–4.6 tok/s (both are [[frank-dragoon]] eras, and that page's Eras
|
||||
rows are where those shapes live); mainline with the trunk on the 3090 and CPU
|
||||
experts at 1.01 tok/s (2026-09-15, this page's own earlier pass); the CPU expert
|
||||
path's prefill ceiling at 192 tok/s.
|
||||
|
||||
## Finding 1 — per-tensor placement across devices is killed by one flag: `-ot`
|
||||
|
||||
@@ -248,15 +250,16 @@ distinguishes a real split from a mislabelled sweep.
|
||||
## Conclusion
|
||||
|
||||
The fleet's answer is unchanged — **two independent endpoints**
|
||||
([[frank-dragoon]] on the iGPU via halogen,
|
||||
[[frank-zealot]] holding a whole 27B in the 3090's 24 GB) — but two
|
||||
claims from the first round of this experiment are now corrected:
|
||||
([[frank-dragoon]] on the iGPU, [[frank-zealot]] alone on the 3090; each page
|
||||
owns what its card holds) — but two claims from the first round of this
|
||||
experiment are now corrected:
|
||||
|
||||
1. **Cross-vendor layer split is possible** (measured: CUDA0 + Vulkan1, 27.23
|
||||
tok/s and 268.5 t/s prefill at 64k). It is not a regime change (+7 % decode),
|
||||
and it costs the 3090 ~15 GiB — which this fleet spends on the vision endpoint
|
||||
(`frank-zealot` holds 22.4 of the card's 24.6 GiB). An operator who wants the
|
||||
7 % must give up the fleet's only image-capable endpoint.
|
||||
and it costs the 3090 ~15 GiB — memory this fleet instead spends on the
|
||||
vision endpoint, whose settled footprint is an Eras row on
|
||||
[[frank-zealot]]. An operator who wants the 7 % must give up the fleet's only
|
||||
image-capable endpoint.
|
||||
2. **Per-tensor placement across two devices is impossible while `-ot` is used**,
|
||||
for a scheduler-mechanical reason, regardless of driver or vendor mix.
|
||||
|
||||
@@ -316,11 +319,13 @@ Operational notes for the next run:
|
||||
- Restarting frank-dragoon needs the quadlet source back in
|
||||
`~/.config/containers/systemd/` plus `daemon-reload`; the generator re-creates
|
||||
the unit from that file only. After the restore (and the host reboot, see the
|
||||
landmines) the ledger read `400384 positions, ~27.2 GiB` against a 33.5 GiB
|
||||
budget — no clamp — with 12.3 GiB left for everything else (14.7 GiB before
|
||||
the reboot: the same model, a more fragmented host afterwards — the engine
|
||||
reported 240 free contiguous 2 MiB blocks against 475 earlier). Both endpoints
|
||||
answered `/health` and served `/v1/models` under their names.
|
||||
landmines) its boot ledger read `400384 positions, ~27.2 GiB` against a 33.5
|
||||
GiB budget — no clamp — with 12.3 GiB left for everything else (14.7 GiB
|
||||
before the reboot: the same model, a more fragmented host afterwards — the
|
||||
engine reported 240 free contiguous 2 MiB blocks against 475 earlier). That
|
||||
ledger is an era reading of [[frank-dragoon]]; the block-census lesson it
|
||||
teaches is on [[Llama Cpp Serving Sizing]]. Both endpoints answered `/health`
|
||||
and served `/v1/models` under their names.
|
||||
|
||||
|
||||
## Smaller-quant check: Q4_K_S against the recorded Q4_K_M numbers
|
||||
@@ -393,10 +398,10 @@ expert mass) — 580 × 3 MiB slabs:
|
||||
| *the same bytes computed in host RAM at 256 GB/s* | 7.1 ms | the link is **31× slower** than the memory behind it |
|
||||
|
||||
Against this host's measured alternatives — **20.46 tok/s** for llama.cpp with
|
||||
the experts *computed* on the CPU (link carries activations only), 25 for
|
||||
halogen, 26.5 for mainline on the iGPU — an engine that *streams* experts across
|
||||
this link loses by 3–5×, and the port's x8 ceiling would only double it (≈7–9
|
||||
tok/s).
|
||||
the experts *computed* on the CPU (link carries activations only, config B′
|
||||
above), 25 for the halogen shape deployed that day ([[frank-dragoon]]), 26.5 for
|
||||
mainline on the iGPU — an engine that *streams* experts across this link loses
|
||||
by 3–5×, and the port's x8 ceiling would only double it (≈7–9 tok/s).
|
||||
|
||||
For the specific engine that raised the question: FreeToken v0.1.2 keeps expert
|
||||
weights in host RAM with an LRU GPU cache — exactly this streaming shape — and
|
||||
|
||||
Reference in new issue
Block a user