docs: record DSpark single-slot switch + ROCm parallel crash (2026-08-15) across KB; refs AGENTS/deployment/benchmark/matrix/index

This commit is contained in:
Frank committed 2026-08-15 14:36:34 -07:00
1 parent d04cc69e09
commit 020e874a5c
6 files changed
+108 -25

No files matched your search

+12 -3
View File
@@ -7,10 +7,19 @@ llama.cpp build on that machine (upstream, b10133+).
## Summary
- **Backend**: ROCm (HIP). Vulkan is not viable for the full pipeline.
- **Spec (2026-08-14)**: MTP sidecar drafter (`draft-mtp`, `a4lg/Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`, `--spec-draft-n-max 3`). Replaced DFlash, which was retired because it releases a generating slot prematurely under parallel load (see below).
- **Context**: 3 parallel slots x 200700 (pool 602100).
- **Spec (2026-08-15)**: DSpark sidecar draft (`draft-dspark`,
`magnitudedev/Qwen3.8-27B-DSpark-GGUF` Q8_0, 1.36B, `--spec-draft-n-max
7`). Replaced bundled MTP. **Single-slot (PARALLEL=1)** because DSpark
crashes the ROCm backend under two concurrent 16k prompts (see
[[qwen3.8-spec-matrix]]).
- **Context**: 1 slot x 262144.
- **KV**: `q8_0` K / `q8_0` V on GPU.
- **Batch**: 8192 / ubatch 8192 (2026-08-14, uses the VRAM headroom; peaks ~65.5 GiB of 96 GiB).
- **Batch**: 8192 / ubatch 256.
> The A/B tables below are retained for history but predate the 2026-08-14
> Qwen3.8-27B `UD-Q8_K_XL` switch to bundled MTP and the 2026-08-15 DSpark
> switch. The current production spec decision and the DSpark parallel crash
> are recorded on [[qwen3.8-spec-matrix]].
## 2026-08-14: Spec A/B (ROCm, production config, default sampling, 1.1k-token prompt)