docs: record --spec-default + Vulkan0 + PARALLEL=2 final state; 262k-ctx ROCm-vs-Vulkan bench

This commit is contained in:
Frank committed 2026-08-15 17:13:18 -07:00
1 parent 5296cf11d7
commit b6f2ee2945
6 files changed
+77 -44

No files matched your search

+11 -9
View File
@@ -6,20 +6,22 @@ llama.cpp build on that machine (upstream, b10133+).
## Summary
- **Backend**: ROCm (HIP). Vulkan is not viable for the full pipeline.
- **Spec (2026-08-15)**: DSpark sidecar draft (`draft-dspark`,
`magnitudedev/Qwen3.8-27B-DSpark-GGUF` Q8_0, 1.36B, `--spec-draft-n-max
7`). Replaced bundled MTP. **Single-slot (PARALLEL=1)** because DSpark
crashes the ROCm backend under two concurrent 16k prompts (see
- **Backend**: **Vulkan0** (2026-08-15). At 262k ctx Vulkan decodes 48%
faster than ROCm (5.68 vs 3.83 t/s); ROCm only wins prefill 119 vs 106
t/s. Generation is the interactive cost, so Vulkan0.
- **Spec (2026-08-15)**: llama.cpp `--spec-default` preset (ngram-mod,
n_match 24 / n_min 48 / n_max 64, draftless). Replaced DSpark, which
crashed the ROCm backend under two concurrent 16k prompts (see
[[qwen3.8-spec-matrix]]).
- **Context**: 1 slot x 262144.
- **Parallel**: 2 slots — passes the 2x32k parallel stream test.
- **Context**: 2 slots x 262144.
- **KV**: `q8_0` K / `q8_0` V on GPU.
- **Batch**: 8192 / ubatch 256.
> The A/B tables below are retained for history but predate the 2026-08-14
> Qwen3.8-27B `UD-Q8_K_XL` switch to bundled MTP and the 2026-08-15 DSpark
> switch. The current production spec decision and the DSpark parallel crash
> are recorded on [[qwen3.8-spec-matrix]].
> Qwen3.8-27B `UD-Q8_K_XL` switch. The current production spec decision,
> the DSpark parallel crash, and the high-context ROCm-vs-Vulkan bench are
> recorded on [[qwen3.8-spec-matrix]].
## 2026-08-14: Spec A/B (ROCm, production config, default sampling, 1.1k-token prompt)