docs: record DSpark single-slot switch + ROCm parallel crash (2026-08-15) across KB; refs AGENTS/deployment/benchmark/matrix/index

This commit is contained in:
Frank committed 2026-08-15 14:36:34 -07:00
1 parent d04cc69e09
commit 020e874a5c
6 files changed
+108 -25

No files matched your search

+17 -2
View File
@@ -137,9 +137,24 @@ curl -sf http://127.0.0.1:7770/health
### Dankclaw — frank-default-large
The active service on Dankclaw runs llama.cpp serving the HauhauCS Qwen3.6-27B uncensored `Q8_K_P` model under the alias `frank-default-large` on port 7770. It uses an **MTP sidecar drafter** (`draft-mtp`, `a4lg/Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`, `--spec-draft-n-max 3`) for speculative decoding on the **ROCm (HIP)** backend, with **3 parallel slots at 200700 context each** (pool 602100), KV cache `q8_0` K / `q8_0` V on GPU (pairing required for Flash Attention on this build), full layer offload (`--gpu-layers all`, `--fit off`), and batch 8192 / ubatch 8192 (peak ~65.5 GiB VRAM). The unit file is `scripts/frank-default-large.service`. It launches through `scripts/serve-llama-frank-large.sh`, which reads tunables from `scripts/frank-default-large.env`.
The active service on Dankclaw runs llama.cpp serving the unsloth
Qwen3.8-27B `UD-Q8_K_XL` model under the alias `frank-default-large` on port
7770. It uses a **DSpark sidecar draft model** (`draft-dspark`,
`magnitudedev/Qwen3.8-27B-DSpark-GGUF` Q8_0, 1.36B, `--spec-draft-n-max 7`)
for speculative decoding on the **ROCm (HIP)** backend, with **1 parallel
slot** at 262144 context, KV cache `q8_0` K / `q8_0` V on GPU, full layer
offload (`--gpu-layers all`, `--fit off`), and batch 8192 / ubatch 256.
The unit file is `scripts/frank-default-large.service`. It launches through
`scripts/serve-llama-frank-large.sh`, which reads tunables from
`scripts/frank-default-large.env`.
> Note: MTP replaced DFlash (`draft-dflash`) on 2026-08-14. DFlash retired because it releases a generating slot prematurely under parallel load when another task lands with high prompt-cache reuse. See [[frank-default-large-benchmark]]. Also retired earlier: the vLLM-ROCm CIRU INT4 Ling-3.0-flash runtime and the llama-serve/vLLM units on Dankclaw.
> Note: DSpark replaced **bundled MTP** (`draft-mtp`, in-model, auto-detected
> b10411+) on 2026-08-15 at the user's request. **DSpark is single-slot
> only**: two concurrent 16k prose prompts crash the ROCm backend with
> `ROCm error: an illegal memory access` (status 6/ABRT) during per-sequence
> block sampling, so `PARALLEL=1`. DSpark is the semi-autoregressive
> extension of DFlash; DFlash was previously retired here for a
> shared-draft-context premature-EOS bug. See [[qwen3.8-spec-matrix]].
### Dankserver — frank-default-small