frank-default-large: swap to caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP

Replace the retired kingjones ROCmFP4 STRIX MTP payload on Dankclaw with
cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF (eemin namespace, public), served by the
charlie12345/ROCmFPX llama.cpp build (ROCmI4 W4A4, c49ebdbd) on the Strix Halo
iGPU. Q4_0_ROCMI4 with an embedded NextN/MTP head (no separate draft file);
strict-Qwen MTP n16/p0.60/backend sampling on ROCm0. Binary
/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server; model dir
/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP. Updated serve script + env,
ubatch bench, large-config bats test, docs, AGENTS.md.
This commit is contained in:
Frank committed 2026-08-25 03:33:29 -07:00
1 parent 7a8500f6fa
commit 7ca9dc248e
10 files changed
+626 -131

No files matched your search

+20 -15
View File
@@ -18,7 +18,7 @@ repository to a running service.
| Target | Host | Runtime | Service | Port |
|--------|------|---------|---------|------|
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q6_K, native MTP draft, selected MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (Qwen3.8-27B-Uncensored-ROCmFP4 Q6 AGENT, standalone MTP Q4_0 head, 262K ctx / 1 slot) | frank-default-large | 7770 |
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (caf-eemin Qwen3.8-27B ROCmI4 Q4_0, embedded MTP, ROCmI4 W4A4, 262K ctx / 1 slot) | frank-default-large | 7770 |
## Workflow
@@ -137,24 +137,29 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
### Dankclaw — frank-default-large
The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` (Q6_0 AGENT, dense)
under the alias `frank-default-large` on port 7770. It runs
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
7.15 SDK in the retired ling3-ciru venv) with a standalone MTP Q4_0 draft
head (`--model-draft` and `--spec-type draft-mtp`), f16 KV, full 262144
context (1 slot. Concurrent requests queue; the server never preempts a
running generation), batch 4096 / ubatch 4096, 16 context checkpoints,
8 GiB prompt cache, and reasoning in deepseek format. The unit file is
The active service on Dankclaw is the charlie12345/ROCmFPX llama.cpp runtime
(with the opt-in `ROCmI4 W4A4` path) serving `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF`
(`eemin` namespace, `Q4_0_ROCMI4`, embedded NextN/MTP head) under the alias
`frank-default-large` on port 7770. It runs
`/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` (built
against system ROCm 7.2.4 / amdclang 22) with a **strict Qwen MTP** draft
(`--spec-type draft-mtp --spec-mtp-strict-qwen`) running on ROCm0 with all
layers (`--spec-draft-ngl all`) — no separate draft file, because the MTP head
is embedded in the main GGUF. K/V are f16, full 262144 context (1 slot,
concurrent requests queue, the server never preempts a running generation),
`-t 16 -tb 32`, batch 512 / ubatch 256 (ubatch is swappable; sweep it on 8k
context with `scripts/bench-ubatch-rocmi4.sh`), 16 context checkpoints, 8 GiB
prompt cache, and reasoning in deepseek format. The unit file is
`scripts/frank-default-large.service`. It launches through
`scripts/serve-frank-large.sh`, which reads tunables from
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
See [[kingjones-strix-mtp]].
See [[eemin-rocmi4-mtp]].
> The kingjones Q6 AGENT model has served the slot since 2026-08-24; the
> v3 PromptForge payload is retired on this slot.
> Restored 2026-08-21. The intermediate DFlash2 Vulkan deployment was
> retired the same day. See [[dflash2-large-vulkan]] for the archived runbook.
> The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
> was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The
> v3 PromptForge payload and the CIRU ROCmFPX runtime are retired on this slot.
> The intermediate DFlash2 Vulkan deployment was retired the same day (2026-08-21).
> See [[dflash2-large-vulkan]] for the archived runbook.
## Validation