docs: what the x4 pair can and cannot do — measured, not argued
The design we sketched (trunk + KV on the 3090, MoE experts on the iGPU) is not expressible in mainline llama.cpp. Per-tensor cross-backend placement aborts in the scheduler -- "pre-allocated tensor (blk.0.ffn_down_exps.weight) in a buffer (Vulkan1) that cannot run the operation (NONE)" -- and both per-layer cross-backend forms fail at load. CUDA + Vulkan cannot serve one model; the CPU is the only fallback side that works, which is exactly why -ot exps=CPU runs. What the same build does instead is the surprise: iGPU-only (Vulkan/RADV, PLE in page cache) gives 32.64 tok/s shallow, 25.77 tok/s at 64k depth and 456.7 tok/s prefill, with the 3090 idle at 0 %. That matches halogen's deployed ~25 tok/s -- but commits 116 of the host's 121 GiB against halogen's 95.4 held with 14.7 spare, so it is not deployable beside the co-tenants. Throughput parity, memory bankruptcy: the fleet's two-endpoint split stands, now on evidence rather than preference. New page LLM/x4-pair-experiment.md carries the tensor inventory, the results table, the two scheduler limits, reproduction commands, and the traps found on the way. Both index pages also stop calling halogen retired -- the ik swap's stale claims were still sitting in them.
This commit is contained in:
1 parent
e2dc2b67ce
commit
65fbef63dc
4 files changed
+224
-8
No files matched your search
@@ -92,6 +92,12 @@ frank-small, port 7771]
|
||||
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO IQ3_S on the RTX 3090, with the model's own
|
||||
Multi-Token Prediction head for speculative decoding and the BF16 projector
|
||||
for the fleet's image path
|
||||
- [[x4-pair-experiment]] — MEASURED 2026-09-16: what mainline llama.cpp can and
|
||||
cannot do with the 3090 + Strix Halo pair, over their only link (the 3090's
|
||||
Gen4 x4). The tensor inventory that decides any split, the two scheduler
|
||||
limits that kill a cross-backend one, and the finding that the iGPU alone
|
||||
matches halogen's deployed decode rate while leaving the 3090 idle — at a
|
||||
memory cost the co-tenants cannot pay
|
||||
- [[frank-large-ik-uncensored]] — RETIRED 2026-09-16 (rollback target): the
|
||||
ik_llama.cpp runbook (root systemd unit, 2026-09-15 → 09-16) serving
|
||||
mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090
|
||||
|
||||
Reference in new issue
Block a user