serverAppConfigs/docs/LLM/qwen3.8-strix-optimizations.md

Qwen3.8-27B on Strix Halo: Speed Without Lowering the Quant

This page answers one question: how do we make Qwen3.8-27B serve faster on Dankclaw (Strix Halo, Radeon 8060S) while keeping UD-Q8_K_XL? It records the model's real architecture, the levers that are available, their evidence, and the test matrix planned for 2026-08-16 morning.

Related pages:

• Speculative-decoding matrix
• Backend and spec benchmark
• Dankclaw Improvements (OS-level items)
• Pipeline Parallelism: Qwen3.8 PP Deployment

TL;DR — ranked levers

#	Lever	Expected gain	Cost	Status
1	Pipeline-parallel to the RTX 3090 (RPC)	~1.5–2× decode	2.5 GbE link, both services stop for test	Kit ready, test 08-16
2	MTP n_max explicit 4–5 + Q4_0 draft head	+5–15% on the spec share	restart, 1.56 GB file	test 08-16
3	Plain Q8_0 A/B vs UD-Q8_K_XL (same 8 bits)	dequant ALUs may differ	restart, 29 GB file	test 08-16
4	Prefill path: ROCm backend + tuned ubatch/threads-batch	2–3× prefill for agentic turns	per-workload backend choice	partial data
5	TTM pages_limit raised to 114 GiB	removes the hard VRAM ceiling; the full 262k f16 KV plus the Q8 AGENT payload, MTP draft, and mmproj fit in one slot	modprobe, set live 2026-08-25	applied (29,883,536 pages)
6	Vulkan ICD pinning + loader-layer disable	robustness; small	service env vars	test 08-16

Everything else (quantization downgrade, NVFP4, larger drafts) is documented below for completeness but is out of scope per the "no lowering" constraint.

What the model actually is

Verified from the GGUF header (/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL.gguf, dumped 2026-08-15) and the official model card.

• Architecture: qwen35 / Qwen3.8-27B, hybrid attention.
• 64 main layers + 1 MTP (nextn) layer = 65 GGUF blocks.
• 48 Gated DeltaNet (linear/recurrent) layers + 16 gated full-attention layers, in a 3:1 pattern (full_attention_interval = 4).
• Hidden 5120, FFN 17408, 24 Q heads / 4 KV heads, head dim 256.
• Native context 262144 (extendable per the card; production runs 262k/slot).
• GDN layers keep a fixed-size recurrent state (constant per layer, not per token). Only the 16 full-attention layers grow with context.
• KV per token (full-attention part only): 16 layers × 2 (K,V) × 4 heads × 256 dim × 1 byte (q8_0) = 32 KiB/token. At 65536 ctx ≈ 2 GiB; at 262144 ctx ≈ 8 GiB.
• Sampling defaults (card, thinking mode on by default): temperature 1.0, top-k 20, top-p 0.95, min-p 0.0. Production already matches.

Consequences of the hybrid design:

1. Decode is cheap in memory. Attention cost scales with only 16 layers; the 48 GDN layers are O(1) per token. Long-context decode degrades far slower than a dense transformer.
2. Prompt cache reuse is unavailable upstream. Recurrent/SSM state cannot be prefix-cached the way attention KV can. Every agentic turn re-prefills the whole conversation. Prefill is the hidden cost of agentic use, and it is the dominant cost we have not yet optimized.
3. MTP is native and strong. The model ships a nextn head; our matrix measured 13.3 t/s vs 7.2 t/s (no spec) at 16k on Vulkan.

Current production state (baseline)

frank-default-large (Dankclaw, Vulkan0, 2026-08-15):

• Model: UD-Q8_K_XL, 31.5 GB, bundled MTP head.
• --spec-default (ngram-mod) with draft-mtp first in the chain.
• 2 parallel slots × 262144 ctx, KV q8_0, batch 8192, ubatch 256, threads 12, -dio off, --fit off.
• Measured: ~25 t/s at 16k ctx; 5.68 t/s at 262k ctx; prefill 302 t/s @64k (Vulkan) vs 410 t/s @64k (ROCm, same build).
• GPU: 28 GiB of 96 GiB GTT used. 74 GiB sits idle.

The levers, in detail

1. Pipeline-parallel to the RTX 3090 (biggest win)

The 3090 on Dankserver is idle (serves a 22 GB MoE small-model service with ~2.4 GB free). llama.cpp RPC splits the model by layers: each device runs its layer range, hidden states cross the 2.5 GbE ethernet once per token. The per-token transfer is ~41 KiB (5120-wide f32 hidden state, both directions); at ~0.5 ms LAN RTT the bandwidth cost is negligible — the split is latency-bound, not bandwidth-bound (see Ethernet Bandwidth Math).

Why this is a real speedup and not just "two GPUs": the 3090 does matmuls roughly 2–3× faster per layer than the 8060S (936 vs 512 GB/s memory, 80 vs 40 CUs). A 50/50 layer split balances per-layer latency, so each token step takes ≈ max(half-on-3090, half-on-Strix) + one RTT instead of all-64-layers-on-Strix.

Design decisions (verified in b10433 source, 2026-08-15):

• Head on Dankclaw, worker on Dankserver. The 31.5 GB model file stays on the head; the worker receives its tensor share over the wire at load (with -c it caches the tensors locally after the first load).
• Head must NOT pass --device. When params.devices is set, llama_prepare_model_devices builds the device list from exactly those devices and the RPC device is silently dropped (--rpc becomes a no-op). Default device selection inserts RPC servers at the front of the list.
• The local iGPU side runs on ROCm, not Vulkan. Default selection walks registered backends in order; HIP registers before Vulkan, and the iGPU dedup rule (PR #23897 workaround) keeps the first backend's device. Verified live 2026-08-15: device list [RPC0, ROCm0], Vulkan0 never listed. Consequence: the Strix half runs on the ROCm backend — better prefill, ~3% slower decode than Vulkan0 (measured on this box). A second head build (/home/dank/llama.cpp/build-vulkan-rpc, HIP off, Vulkan on, RPC on) was built the same night for an A/B; with it the device list becomes [RPC0, Vulkan0].
• Default split is proportional to reported free memory, which is wrong here. The Strix reports up to 96 GiB (GTT ceiling) vs the 3090's 24 GB, so the default lands ~20/80 in favor of the slow device (verified with a 4B smoke test: 4 of 36 layers on the 3090). We pin --tensor-split 1,1 (50/50) explicitly.
• -dio (direct I/O) on the head. Large model + RPC load hangs without it (visorcraft Strix finding; matches our 28 GiB load-stall history).
• Version skew hangs the handshake. Worker built at the identical commit (9b05354ec / b10433) in a worktree /home/dank/llama-cpp-rpc on Dankserver — production b10121 build untouched.

Smoke test (2026-08-15, 4B model, 0.2/0.8 split, production untouched): worker VRAM rose ~819 MiB (real split), 95 t/s prefill, 58.8 t/s decode on the tiny model. RTT on the link: ~0.3–0.5 ms.

Full design, scripts, and morning runbook: Qwen3.8 PP Deployment.

2. Speculative-decoding tuning (same bits, free tokens)

• MTP n_max. Production leaves --spec-draft-n-max unset. Our matrix (Vulkan, 16k) found 13.3 t/s at the tested setting vs 7.2 without spec; the published Strix tuning curve peaks at n_max 3–4 (30.3 t/s at 4 vs 28.9 at 3 vs 29.9 at 5). The PP env sets 4 explicitly; the matrix re-tests 3/4/5 on the real split.
• Q4_0 draft head. The published Strix card measured a standalone Q4_0 MTP head ~6–10% faster than Q8_0 for the same acceptance (the draft head is a 248k-vocab output projection; cheaper dequant, same accuracy). The bundled head inside UD-Q8_K_XL is Q8-class; the standalone ggml-org/mtp-Qwen3.8-27B-Q4_0.gguf (1.56 GB) is being downloaded for an A/B via --spec-draft-model.
• Ngram-mod fallback stays. It is the only spec type that survived parallel load (DSpark crashed ROCm, DFlash corrupts multi-slot).

3. Same-bit quant A/B: UD-Q8_K_XL vs plain Q8_0

Both are 8-bit. UD (unsloth "dynamic") mixes block sizes per tensor; plain Q8_0 uses uniform 32-element blocks. Dequant of a smaller block costs more ALUs per element; on a memory-bound GPU the difference is usually small, but Strix's iGPU has tight ALU headroom. ggml-org/Qwen3.8-27B-GGUF ships a Q8_0 (29.12 GB). One restart to test; if it wins, same quality at zero quality cost.

4. Prefill is the agentic bottleneck — route it right

Every agentic turn re-prefills the whole conversation (no prefix caching for hybrid state, upstream). Levers:

• Backend choice per workload. Measured @64k on this box (b10433): ROCm 410 t/s vs Vulkan 302 t/s prefill; Vulkan 25.1 vs ROCm 24.3 t/s decode. For chat-heavy use: Vulkan. For agentic/batch-heavy use: ROCm. The PP head (ROCm side) gets this for free.
• ubatch/threads-batch. Production runs ubatch 256. Larger ubatches (512/1024) fill the iGPU better during prefill; --threads-batch for the CPU-bound tail. Untested at prefill scale on this box — matrix item.
• KV q4_0 for very long contexts: halves the 16-layer KV (8 GiB → 4 GiB at 262k). Only worth it if long-ctx attention is the bottleneck; q8_0 is the safe default.

5. OS level (from the Improvements doc, re-verified 2026-08-15)

• TTM pages_limit is now 114 GiB (29,883,536 pages) — applied 2026-08-25. Written live to /sys/module/ttm/parameters/pages_limit (no reboot required) and persisted in /etc/modprobe.d/ttm.conf. The full 262k f16 KV cache plus the ~28 GB Q8 AGENT payload, MTP draft, and mmproj now fit in one slot. The load-stall risk from the old 15 GiB cap is resolved. Note: 96 GiB is the driver-reported VRAM carve-out (VGM/SVM pool); the TTM pages_limit is the kernel page map that was the real ceiling.
• iommu=pt — verified present on the kernel cmdline. Done.
• Vulkan ICD pinning — radeon_icd.json exists; no VK_ICD_FILENAMES or loader-layer env is set. Pinning the radeon ICD and disabling loader layers (VK_LOADER_LAYERS_DISABLE=all) is the recommended Strix hardening; testable as service env vars, zero risk.
• Power — GPU DPM is already performance, forced level auto. The power_profile sysfs file assumed in the Improvements doc does not exist on this machine (checked 2026-08-15); that item is N/A here.
• Threads: 12 of 16 cores in production. Community reports 12–16 are equivalent; leave at 12.

6. llama.cpp version watch

b10433 (2026-08-15) is adequate. Through b10451 there are no spec-decoding fixes that help us; DFlash's multi-slot corruption is still open upstream (fix #27117 pending). If that lands, DFlash/DSpark become re-testable candidates (DSpark was the best single-machine draft before the ROCm crash). No rebuild planned until then.

Quantization landscape (reference)

The constraint is to keep UD-Q8_K_XL. This section records what exists so future decisions are informed.

Format	Size	Bits/weight	Notes
UD-Q8_K_XL (unsloth)	31.5 GB	~8.5 (mixed)	Current. Dynamic mixed-block; bundled MTP head; our baseline.
Q8_0 (ggml-org, bartowski)	29.1 GB	8.0	Uniform 32-block. Same-bit A/B candidate (lever 3).
UD-Q6_K_XL (unsloth)	~22 GB	~6.5	Below the 8-bit floor; documented, not recommended under the constraint.
Q6_K	~21.5 GB	6.6	Same tier as above.
ROCmFP4 "STRIX" (ROCmFPX fork)	13.3–13.8 GB	4.25–4.39	Strix-specific NVFP4-style format, ROCm backend only, requires a forked llama.cpp. Published: 30.3 t/s vs 10.7 for Q4_K_M (2.83×) at 8k with MTP n_max≈4; PPL ≈ Q4_K_M. This is the "fast path" — and it is a quant downgrade plus a fork dependency that breaks the Vulkan production path. Kept here as the escape hatch if the 8-bit speed target cannot be met.
NVFP4 (Blackwell)	~14 GB	4.5	Hardware path for RTX 50-series. Irrelevant to 3090/Strix.

If the 8-bit PP result (lever 1) is disappointing, the decision order is: push PP split/latency tuning → ROCmFP4 as a second service (A/B, not a replacement) → revisit the constraint with Frank.

Morning test matrix (2026-08-16)

Policy: stop both production services, run tests, restore. Each config = one head restart (~10 min load incl. weight stream; ~2 min with the worker tensor cache warm).

#	Config	Measures
A	PP split 1,1, ROCm head, MTP n_max=4 + ngram	baseline PP
B	A + SPEC_NONE=1	raw (no spec) PP speed
C	A but head build build-vulkan-rpc (Vulkan side)	backend A/B
D	A + --spec-draft-model Q4_0 MTP file	draft-head A/B
E	A with n_max 3 and 5	n_max curve on the split
F	A, plain Q8_0 model file	same-bit A/B
G	PP default proportional split (no --tensor-split)	confirms the 20/80 handicap
Ref	production frank-default-large (Vulkan, 2 slots)	the bar to beat

Per config: pp-test.sh bench 4 (4k prompt / 256 out) and bench 16 (16k / 256). Report: decode t/s (server print_timing), prefill t/s, MTP acceptance, worker VRAM, head GTT usage.

Success bar: ≥ 30 t/s at 4k and ≥ 20 t/s at 16k decode (vs 25 t/s production @16k) with the same 8-bit model.

Sources

• GGUF header dump of the production model (this machine, 2026-08-15).
• Qwen3.8-27B model card + ggml-org/Qwen3.8-27B-GGUF / unsloth/Qwen3.8-27B-GGUF repo listings (Hugging Face).
• kingjones Qwen3.8-27B-ROCmFP4-GGUF Strix tuning card (MTP n_max curve, Q4_0 vs Q8_0 draft head, ROCmFP4 numbers).
• visorcraft strix-halo-llm-perf (Vulkan tg / ROCm prefill, ICD pinning, -dio for RPC).
• llama.cpp b10433 source: src/llama.cpp (device selection), common/speculative.cpp (MTP/ngram-mod), ggml/src/ggml-rpc (protocol).
• Local benchmarks: frank-default-large-benchmark, qwen3.8-spec-matrix.