docs: reflect Ling-3 vLLM (frank-default-large) on Dankclaw
Update target maps (AGENTS, README, deployment, deploy.sh) from the retired ROCmFPX llama.cpp stack to the active vLLM-ROCm CIRU INT4 service on port 7770 with MTP speculative decoding. Align Dankserver to the running vllm-awq-qwen36 (AWQ, port 34401). Refresh KB pages: Dankclaw system specs (gpu-mem-util 0.95, CIRU venv), quantization choices (INT4 primary, MXFP4 experiment), DFlash/DSpark Dankclaw section, and the KB index topology.
This commit is contained in:
1 parent
52ad985978
commit
903d54c389
9 files changed
+55
-54
No files matched your search
@@ -146,21 +146,15 @@ docker run --rm --name vllm-qwen36-dflash \
|
||||
|
||||
## Deployment on Dankclaw (AMD Strix Halo gfx1151)
|
||||
|
||||
Dankclaw runs the ROCmFPX llama.cpp runtime as its primary service. The active service uses the Ling-3.0-flash ROCmFP4 MTP model with `--spec-type draft-mtp`. This gives measured results:
|
||||
Dankclaw runs vLLM-ROCm as its primary runtime. The active service (`frank-default-large`, port 7770) serves the Ling-3.0-flash CIRU INT4 (W4A16) model with native **MTP speculative decoding** (K1, lossless). MTP uses a NextN head preserved in the checkpoint, so no separate draft model is needed.
|
||||
|
||||
- 45-50 t/s decode with MTP speculation.
|
||||
- ~37 t/s plain decode.
|
||||
- ~95% draft acceptance with `--spec-draft-n-max 2`.
|
||||
Ling-3.0-flash facts:
|
||||
|
||||
ROCmFPX is a llama.cpp fork by charlie12345. It adds ROCmFP4 and ROCmFP2 GGUF weight formats with HIP/ROCm and Vulkan kernels. It is not merged into upstream llama.cpp.
|
||||
- Base model: inclusionAI/Ling-3.0-flash, 124B total / 5.1B active (hybrid linear attention: KDA + MLA).
|
||||
- The MTP head is served in-band; vLLM detects it and shares the target embedding and lm_head weights with the draft head.
|
||||
- The earlier ROCmFPX llama.cpp service (raulvidis ROCmFP4 MTP GGUF) was retired because it was defective. vLLM now handles concurrency and tool calling correctly.
|
||||
|
||||
Ling-3.0-flash-related facts:
|
||||
|
||||
- Base model: inclusionAI/Ling-3.0-flash, 124B total / 5.1B active.
|
||||
- The MTP (NextN) head is preserved in the GGUF conversion.
|
||||
- The raulvidis GGUF repo targets Strix Halo (gfx1151).
|
||||
|
||||
If you want DFlash on Dankclaw, use a DFlash GGUF draft. Community conversions exist, for example Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp. The ROCm path for DFlash is newer than the MTP path and less proven on gfx1151.
|
||||
If you want DFlash or DSpark on Dankclaw, use a vLLM speculative-config draft that pairs with the target model. DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use. MTP is the proven, lossless path on gfx1151.
|
||||
|
||||
## Notes
|
||||
|
||||
|
||||
Reference in new issue
Block a user