docs: record DSpark single-slot switch + ROCm parallel crash (2026-08-15) across KB; refs AGENTS/deployment/benchmark/matrix/index
This commit is contained in:
1 parent
d04cc69e09
commit
020e874a5c
6 files changed
+108
-25
No files matched your search
+17
-2
@@ -137,9 +137,24 @@ curl -sf http://127.0.0.1:7770/health
|
||||
|
||||
### Dankclaw — frank-default-large
|
||||
|
||||
The active service on Dankclaw runs llama.cpp serving the HauhauCS Qwen3.6-27B uncensored `Q8_K_P` model under the alias `frank-default-large` on port 7770. It uses an **MTP sidecar drafter** (`draft-mtp`, `a4lg/Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`, `--spec-draft-n-max 3`) for speculative decoding on the **ROCm (HIP)** backend, with **3 parallel slots at 200700 context each** (pool 602100), KV cache `q8_0` K / `q8_0` V on GPU (pairing required for Flash Attention on this build), full layer offload (`--gpu-layers all`, `--fit off`), and batch 8192 / ubatch 8192 (peak ~65.5 GiB VRAM). The unit file is `scripts/frank-default-large.service`. It launches through `scripts/serve-llama-frank-large.sh`, which reads tunables from `scripts/frank-default-large.env`.
|
||||
The active service on Dankclaw runs llama.cpp serving the unsloth
|
||||
Qwen3.8-27B `UD-Q8_K_XL` model under the alias `frank-default-large` on port
|
||||
7770. It uses a **DSpark sidecar draft model** (`draft-dspark`,
|
||||
`magnitudedev/Qwen3.8-27B-DSpark-GGUF` Q8_0, 1.36B, `--spec-draft-n-max 7`)
|
||||
for speculative decoding on the **ROCm (HIP)** backend, with **1 parallel
|
||||
slot** at 262144 context, KV cache `q8_0` K / `q8_0` V on GPU, full layer
|
||||
offload (`--gpu-layers all`, `--fit off`), and batch 8192 / ubatch 256.
|
||||
The unit file is `scripts/frank-default-large.service`. It launches through
|
||||
`scripts/serve-llama-frank-large.sh`, which reads tunables from
|
||||
`scripts/frank-default-large.env`.
|
||||
|
||||
> Note: MTP replaced DFlash (`draft-dflash`) on 2026-08-14. DFlash retired because it releases a generating slot prematurely under parallel load when another task lands with high prompt-cache reuse. See [[frank-default-large-benchmark]]. Also retired earlier: the vLLM-ROCm CIRU INT4 Ling-3.0-flash runtime and the llama-serve/vLLM units on Dankclaw.
|
||||
> Note: DSpark replaced **bundled MTP** (`draft-mtp`, in-model, auto-detected
|
||||
> b10411+) on 2026-08-15 at the user's request. **DSpark is single-slot
|
||||
> only**: two concurrent 16k prose prompts crash the ROCm backend with
|
||||
> `ROCm error: an illegal memory access` (status 6/ABRT) during per-sequence
|
||||
> block sampling, so `PARALLEL=1`. DSpark is the semi-autoregressive
|
||||
> extension of DFlash; DFlash was previously retired here for a
|
||||
> shared-draft-context premature-EOS bug. See [[qwen3.8-spec-matrix]].
|
||||
|
||||
### Dankserver — frank-default-small
|
||||
|
||||
|
||||
Reference in new issue
Block a user