diff --git a/serverAppConfigs-docs/LLM/frank-primary-llamacpp-qwen38-flash-next-q8.md b/serverAppConfigs-docs/LLM/frank-primary-llamacpp-qwen38-flash-next-q8.md index c1849c2..50d757a 100644 --- a/serverAppConfigs-docs/LLM/frank-primary-llamacpp-qwen38-flash-next-q8.md +++ b/serverAppConfigs-docs/LLM/frank-primary-llamacpp-qwen38-flash-next-q8.md @@ -166,7 +166,7 @@ both cards: main 9,408 MiB + indexer 1,536 MiB per card, draft Hot expert blocks stay GPU-resident. `-ot` granularity is the whole per-block expert tensor (512 experts share one tensor), so "critical experts" = blocks co-located on the card that holds their layers: -blocks 0-3 on ROCm0, 45-47 on ROCm1 (measured 2.42/2.49 GiB per +blocks 0-4 on ROCm0, 45-47 on ROCm1 (measured 2.42/2.49 GiB per block). The placement law that makes it work: b11030 wants ONE comma-joined `--override-tensor` (repeats log DEPRECATED), and the loader applies the FIRST matching pattern per tensor — the @@ -174,15 +174,20 @@ block-specific patterns MUST precede the broad `exps=CPU` sweep, or the sweep silently swallows them (verified live: broad-first left every expert on CPU with no warning anywhere in the log). -Measured headroom (checkpoint floor: 3-4 GB per card): card0 5.02 GiB -free at boot, ~4.9 GB under load; card1 3.31 GiB, ~3.4 GB under load -(the vision graph costs ~0.43 GiB on card0 at first image). Blocks are -atomic — a 5th on card0 lands at 2.27 GB free, under the floor — so -4/3 is the deployed fill. Live numbers: content decode 13.05 t/s +Measured headroom, deployed fill **5/3** (blocks 0-4 on ROCm0, 45-47 +on ROCm1 — the operator directed the fill past the 3-4 GB checkpoint +floor the same day, "shift a few more experts"): card0 2.53 GiB free +at boot, 2.23 GiB under load with the vision graph exercised; card1 +3.31 GiB at boot, 3.15 GiB under load. Card1 cannot take another +block — 3.31 − 2.49 = 0.82 GiB, and the vision graph (~0.43 GiB) plus +boot drift alone exceed that — so 5/3 is the physical limit at the +4×262144 / V-q8_0 shape. Earlier 4/3 state for reference: card0 5.02 +GiB boot free. Live numbers (4/3 era): content decode 13.05 t/s (acceptance 0.58, mean len 2.16), reasoning phase 7.24 t/s (acceptance 0.74, mean len 2.48), two concurrent full generations both delivered, prompt cache reusing 42/64 tokens across requests. All -three Prometheus fleet targets up. +three Prometheus fleet targets up; the 5/3 deploy re-verified chat, +vision and the target on the same battery class. ## Rollback