docs: record --spec-default + Vulkan0 + PARALLEL=2 final state; 262k-ctx ROCm-vs-Vulkan bench
This commit is contained in:
1 parent
5296cf11d7
commit
b6f2ee2945
6 files changed
+77
-44
No files matched your search
@@ -6,20 +6,22 @@ llama.cpp build on that machine (upstream, b10133+).
|
||||
|
||||
## Summary
|
||||
|
||||
- **Backend**: ROCm (HIP). Vulkan is not viable for the full pipeline.
|
||||
- **Spec (2026-08-15)**: DSpark sidecar draft (`draft-dspark`,
|
||||
`magnitudedev/Qwen3.8-27B-DSpark-GGUF` Q8_0, 1.36B, `--spec-draft-n-max
|
||||
7`). Replaced bundled MTP. **Single-slot (PARALLEL=1)** because DSpark
|
||||
crashes the ROCm backend under two concurrent 16k prompts (see
|
||||
- **Backend**: **Vulkan0** (2026-08-15). At 262k ctx Vulkan decodes 48%
|
||||
faster than ROCm (5.68 vs 3.83 t/s); ROCm only wins prefill 119 vs 106
|
||||
t/s. Generation is the interactive cost, so Vulkan0.
|
||||
- **Spec (2026-08-15)**: llama.cpp `--spec-default` preset (ngram-mod,
|
||||
n_match 24 / n_min 48 / n_max 64, draftless). Replaced DSpark, which
|
||||
crashed the ROCm backend under two concurrent 16k prompts (see
|
||||
[[qwen3.8-spec-matrix]]).
|
||||
- **Context**: 1 slot x 262144.
|
||||
- **Parallel**: 2 slots — passes the 2x32k parallel stream test.
|
||||
- **Context**: 2 slots x 262144.
|
||||
- **KV**: `q8_0` K / `q8_0` V on GPU.
|
||||
- **Batch**: 8192 / ubatch 256.
|
||||
|
||||
> The A/B tables below are retained for history but predate the 2026-08-14
|
||||
> Qwen3.8-27B `UD-Q8_K_XL` switch to bundled MTP and the 2026-08-15 DSpark
|
||||
> switch. The current production spec decision and the DSpark parallel crash
|
||||
> are recorded on [[qwen3.8-spec-matrix]].
|
||||
> Qwen3.8-27B `UD-Q8_K_XL` switch. The current production spec decision,
|
||||
> the DSpark parallel crash, and the high-context ROCm-vs-Vulkan bench are
|
||||
> recorded on [[qwen3.8-spec-matrix]].
|
||||
|
||||
## 2026-08-14: Spec A/B (ROCm, production config, default sampling, 1.1k-token prompt)
|
||||
|
||||
|
||||
Reference in new issue
Block a user