docs: reflect Ling-3 vLLM (frank-default-large) on Dankclaw

Update target maps (AGENTS, README, deployment, deploy.sh) from the retired
ROCmFPX llama.cpp stack to the active vLLM-ROCm CIRU INT4 service on port 7770
with MTP speculative decoding. Align Dankserver to the running
vllm-awq-qwen36 (AWQ, port 34401). Refresh KB pages: Dankclaw system specs
(gpu-mem-util 0.95, CIRU venv), quantization choices (INT4 primary, MXFP4
experiment), DFlash/DSpark Dankclaw section, and the KB index topology.
This commit is contained in:
Frank committed 2026-08-10 23:37:27 -07:00
1 parent 52ad985978
commit 903d54c389
9 files changed
+55 -54

No files matched your search

+6 -12
View File
@@ -146,21 +146,15 @@ docker run --rm --name vllm-qwen36-dflash \
## Deployment on Dankclaw (AMD Strix Halo gfx1151)
Dankclaw runs the ROCmFPX llama.cpp runtime as its primary service. The active service uses the Ling-3.0-flash ROCmFP4 MTP model with `--spec-type draft-mtp`. This gives measured results:
Dankclaw runs vLLM-ROCm as its primary runtime. The active service (`frank-default-large`, port 7770) serves the Ling-3.0-flash CIRU INT4 (W4A16) model with native **MTP speculative decoding** (K1, lossless). MTP uses a NextN head preserved in the checkpoint, so no separate draft model is needed.
- 45-50 t/s decode with MTP speculation.
- ~37 t/s plain decode.
- ~95% draft acceptance with `--spec-draft-n-max 2`.
Ling-3.0-flash facts:
ROCmFPX is a llama.cpp fork by charlie12345. It adds ROCmFP4 and ROCmFP2 GGUF weight formats with HIP/ROCm and Vulkan kernels. It is not merged into upstream llama.cpp.
- Base model: inclusionAI/Ling-3.0-flash, 124B total / 5.1B active (hybrid linear attention: KDA + MLA).
- The MTP head is served in-band; vLLM detects it and shares the target embedding and lm_head weights with the draft head.
- The earlier ROCmFPX llama.cpp service (raulvidis ROCmFP4 MTP GGUF) was retired because it was defective. vLLM now handles concurrency and tool calling correctly.
Ling-3.0-flash-related facts:
- Base model: inclusionAI/Ling-3.0-flash, 124B total / 5.1B active.
- The MTP (NextN) head is preserved in the GGUF conversion.
- The raulvidis GGUF repo targets Strix Halo (gfx1151).
If you want DFlash on Dankclaw, use a DFlash GGUF draft. Community conversions exist, for example Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp. The ROCm path for DFlash is newer than the MTP path and less proven on gfx1151.
If you want DFlash or DSpark on Dankclaw, use a vLLM speculative-config draft that pairs with the target model. DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use. MTP is the proven, lossless path on gfx1151.
## Notes