docs: qwen38-fp8 expert-offload tuning — prefetch off hits 37.4-40.6 tok/s (target met)
Measured campaign on the ad hoc qwen-fp8 container (tcclaviger RDNA4 fork, 2x R9700, TP=2): telemetry (with_stack=false) measured the prefetcher at 36% precision — ~half of gather traffic wasted; deleting --expert-prefetch: 26.8 -> 40.57/37.44/37.57 tok/s. Prefetch-on shape survived every other challenger (gather geometry, spec depth, TunableOp [corrupts numerics], MNBT, concurrency, ngram [PLE fork bug]). 256k ctx boots at 4.5 GiB KV (36.65/34.21/34.77, arena -14%). New runbook + index links + log entry.
This commit is contained in:
@@ -101,6 +101,16 @@ frank-small, port 7771]
|
||||
bound, ~23 tok/s ceiling at today's residency). Includes the Affinity
|
||||
V4-Flash-to-V4.1 port assessment — reusable expert tiering, four
|
||||
architecture gaps, one of them a silent-wrongness trap on the Engram key
|
||||
- [[dankserver-qwen38-fp8-expert-offload-tuning]] — VERIFIED 2026-09-29:
|
||||
the flag-level decode-tuning campaign on the ad hoc qwen-fp8 container
|
||||
(tcclaviger RDNA4 fork, orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8,
|
||||
TP=2 across both R9700s). **>35 tok/s MET: 37.44-40.57 tok/s with
|
||||
`--expert-prefetch` DELETED** — telemetry measured the prefetcher at
|
||||
36% precision, wasting ~half the gather traffic (the LRU itself runs at
|
||||
82.45% hit). Prefetch-on incumbent 26.8 survived every other challenger
|
||||
(geometry, spec depth, TunableOp-corrupts, batch knobs, concurrency,
|
||||
ngram-fork-bug); the residual gap to ~60 tok/s is the fork's gather
|
||||
read path (6.5-9.5 GB/s effective vs 22.7 at the same slab size)
|
||||
- [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-26 (replaced by
|
||||
[[frank-primary-lucebox-ds4]], then
|
||||
[[frank-primary-llamacpp-qwen38-flash-next-q8]]): vllm-radiance 0.9.3
|
||||
|
||||
Reference in New Issue
Block a user