dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

frank-default-large: swap to caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP

Browse Source 
Replace the retired kingjones ROCmFP4 STRIX MTP payload on Dankclaw with
cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF (eemin namespace, public), served by the
charlie12345/ROCmFPX llama.cpp build (ROCmI4 W4A4, c49ebdbd) on the Strix Halo
iGPU. Q4_0_ROCMI4 with an embedded NextN/MTP head (no separate draft file);
strict-Qwen MTP n16/p0.60/backend sampling on ROCm0. Binary
/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server; model dir
/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP. Updated serve script + env,
ubatch bench, large-config bats test, docs, AGENTS.md.
This commit is contained in:
Frank committed 2026-08-25 03:33:29 -07:00
1 parent 7a8500f6fa
commit 7ca9dc248e
10 files changed
+626 -131
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

AGENTS.md
+16 -11
View File
@@ -27,7 +27,7 @@ host.
| Target | Runtime | Service | Model | Port |
|--------|---------|---------|-------|------|
| Dankserver | llama.cpp (upstream b10121) | frank-default-small | HauhauCS Qwen3.6-35B-A3B-Uncensored-Aggressive Q6_K_P + ngram-mod, 36 MoE experts on CPU | 7770 |
| Dankclaw | CIRU ROCmFPX llama.cpp, PromptForge v3 runtime (ROCm 7.15 SDK in the ling3-ciru venv) | frank-default-large | jcbtc Qwen3.8-27B-CIRU-ActiveFPX-PromptForge v3 (Q8-FFNEQ-LateQ6, full-W8 views, Output-K8 MTP proxy, BF16 vision), 262K context / 1 slot | 7770 |
| Dankclaw | charlie12345/ROCmFPX llama.cpp, ROCmI4 W4A4 runtime | frank-default-large | caf-eemin Qwen3.8-27B-ROCmI4-MTP Q4_0_ROCMI4 (embedded NextN/MTP head, gfx1151 IU4 batched verification), 262K context / 1 slot | 7770 |
History: the 2026-08-17 redo retired the pipeline-parallel split and the
FastFlowLM NPU endpoint. The 2026-08-18 swap retired the vLLM service on
@@ -179,17 +179,22 @@ When you rename a service or move a model to a new host, update
## Runtime & Troubleshooting
### v3 CIRU ActiveFPX PromptForge llama.cpp (Dankclaw, ACTIVE)
### charlie12345/ROCmFPX ROCmI4 W4A4 llama.cpp (Dankclaw, ACTIVE)
The v4 runtime serves `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (`eemin` namespace,
public) from `/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server`
(charlie12345/ROCmFPX commit `c49ebdbd` built with `-DGGML_HIP_ROCMI4_W4A4=ON`
against system ROCm 7.2.4 / amdclang 22). The model is `Q4_0_ROCMI4` with an
embedded NextN/MTP head (no separate draft file); the gfx1151 native IU4 WMMA
instruction batch-verifies proposals. Profile: strict Qwen MTP
(`--spec-mtp-strict-qwen`, n_max 16, p_min 0.60, backend sampling, all layers
on ROCm0), `-t 16 -tb 32`, batch 512 / ubatch 256 (ubatch swappable via the
ubatch bench on 8k context), f16 KV, full 262144 context / 1 slot, TEMPERATURE
0.7 / TOP_P 0.95, reasoning on (deepseek, budget -1), `--no-mmap`. Two-file
payload (~14.4 GiB) in `/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/`. Full
runbook:
[docs/LLM/eemin-rocmi4-mtp.md](docs/LLM/eemin-rocmi4-mtp.md).
The v3 runtime serves `jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3`
from `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`
(PromptForge v2.3 + v3 Output-K8 runtime patch). The v3 profile:
`PROMPTFORGE_MODE=m2048_fused_tail1476`, full-W8 companion views, Output-K8
MTP draft proxy, BF16 mmproj vision, TEMPERATURE 0.7 / TOP_P 0.95,
batch 4096 / ubatch 4096, 16 context checkpoints, reasoning on (deepseek,
budget -1), `--no-mmap`. Five-file payload (~36.7 GiB) in
`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. Full runbook:
[docs/LLM/ciru-activefpx-promptforge.md](docs/LLM/ciru-activefpx-promptforge.md).
### Kairic Edge llama.cpp (Dankclaw, RETIRED experiment 2026-08-22)