docs: record DSpark single-slot switch + ROCm parallel crash (2026-08-15) across KB; refs AGENTS/deployment/benchmark/matrix/index
This commit is contained in:
1 parent
d04cc69e09
commit
020e874a5c
6 files changed
+108
-25
No files matched your search
@@ -7,10 +7,19 @@ llama.cpp build on that machine (upstream, b10133+).
|
||||
## Summary
|
||||
|
||||
- **Backend**: ROCm (HIP). Vulkan is not viable for the full pipeline.
|
||||
- **Spec (2026-08-14)**: MTP sidecar drafter (`draft-mtp`, `a4lg/Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`, `--spec-draft-n-max 3`). Replaced DFlash, which was retired because it releases a generating slot prematurely under parallel load (see below).
|
||||
- **Context**: 3 parallel slots x 200700 (pool 602100).
|
||||
- **Spec (2026-08-15)**: DSpark sidecar draft (`draft-dspark`,
|
||||
`magnitudedev/Qwen3.8-27B-DSpark-GGUF` Q8_0, 1.36B, `--spec-draft-n-max
|
||||
7`). Replaced bundled MTP. **Single-slot (PARALLEL=1)** because DSpark
|
||||
crashes the ROCm backend under two concurrent 16k prompts (see
|
||||
[[qwen3.8-spec-matrix]]).
|
||||
- **Context**: 1 slot x 262144.
|
||||
- **KV**: `q8_0` K / `q8_0` V on GPU.
|
||||
- **Batch**: 8192 / ubatch 8192 (2026-08-14, uses the VRAM headroom; peaks ~65.5 GiB of 96 GiB).
|
||||
- **Batch**: 8192 / ubatch 256.
|
||||
|
||||
> The A/B tables below are retained for history but predate the 2026-08-14
|
||||
> Qwen3.8-27B `UD-Q8_K_XL` switch to bundled MTP and the 2026-08-15 DSpark
|
||||
> switch. The current production spec decision and the DSpark parallel crash
|
||||
> are recorded on [[qwen3.8-spec-matrix]].
|
||||
|
||||
## 2026-08-14: Spec A/B (ROCm, production config, default sampling, 1.1k-token prompt)
|
||||
|
||||
|
||||
Reference in new issue
Block a user