frank-default-large: swap to caf-eemin Qwen3.8-27B ROCmI4 embedded-MTP
Replace the retired kingjones ROCmFP4 STRIX MTP payload on Dankclaw with cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF (eemin namespace, public), served by the charlie12345/ROCmFPX llama.cpp build (ROCmI4 W4A4, c49ebdbd) on the Strix Halo iGPU. Q4_0_ROCMI4 with an embedded NextN/MTP head (no separate draft file); strict-Qwen MTP n16/p0.60/backend sampling on ROCm0. Binary /home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server; model dir /home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP. Updated serve script + env, ubatch bench, large-config bats test, docs, AGENTS.md.
This commit is contained in:
1 parent
7a8500f6fa
commit
7ca9dc248e
10 files changed
+626
-131
No files matched your search
+20
-15
@@ -18,7 +18,7 @@ repository to a running service.
|
||||
| Target | Host | Runtime | Service | Port |
|
||||
|--------|------|---------|---------|------|
|
||||
| Dankserver | dank@dankserver | upstream llama.cpp (Ornith-1.5-35B-A3B-CRACK Q6_K, native MTP draft, selected MoE experts offloaded via --override-tensor) | frank-default-small | 7770 |
|
||||
| Dankclaw | dank@dankclaw | CIRU ROCmFPX llama.cpp (Qwen3.8-27B-Uncensored-ROCmFP4 Q6 AGENT, standalone MTP Q4_0 head, 262K ctx / 1 slot) | frank-default-large | 7770 |
|
||||
| Dankclaw | dank@dankclaw | charlie12345/ROCmFPX llama.cpp (caf-eemin Qwen3.8-27B ROCmI4 Q4_0, embedded MTP, ROCmI4 W4A4, 262K ctx / 1 slot) | frank-default-large | 7770 |
|
||||
|
||||
## Workflow
|
||||
|
||||
@@ -137,24 +137,29 @@ from `peculiar-ragdoll/Qwen-Sharp-Chat-Templates`. See [[dealign-ornith-crack]].
|
||||
|
||||
### Dankclaw — frank-default-large
|
||||
|
||||
The active service on Dankclaw is the CIRU ROCmFPX llama.cpp runtime serving
|
||||
`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` (Q6_0 AGENT, dense)
|
||||
under the alias `frank-default-large` on port 7770. It runs
|
||||
`/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` (ROCm
|
||||
7.15 SDK in the retired ling3-ciru venv) with a standalone MTP Q4_0 draft
|
||||
head (`--model-draft` and `--spec-type draft-mtp`), f16 KV, full 262144
|
||||
context (1 slot. Concurrent requests queue; the server never preempts a
|
||||
running generation), batch 4096 / ubatch 4096, 16 context checkpoints,
|
||||
8 GiB prompt cache, and reasoning in deepseek format. The unit file is
|
||||
The active service on Dankclaw is the charlie12345/ROCmFPX llama.cpp runtime
|
||||
(with the opt-in `ROCmI4 W4A4` path) serving `cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF`
|
||||
(`eemin` namespace, `Q4_0_ROCMI4`, embedded NextN/MTP head) under the alias
|
||||
`frank-default-large` on port 7770. It runs
|
||||
`/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` (built
|
||||
against system ROCm 7.2.4 / amdclang 22) with a **strict Qwen MTP** draft
|
||||
(`--spec-type draft-mtp --spec-mtp-strict-qwen`) running on ROCm0 with all
|
||||
layers (`--spec-draft-ngl all`) — no separate draft file, because the MTP head
|
||||
is embedded in the main GGUF. K/V are f16, full 262144 context (1 slot,
|
||||
concurrent requests queue, the server never preempts a running generation),
|
||||
`-t 16 -tb 32`, batch 512 / ubatch 256 (ubatch is swappable; sweep it on 8k
|
||||
context with `scripts/bench-ubatch-rocmi4.sh`), 16 context checkpoints, 8 GiB
|
||||
prompt cache, and reasoning in deepseek format. The unit file is
|
||||
`scripts/frank-default-large.service`. It launches through
|
||||
`scripts/serve-frank-large.sh`, which reads tunables from
|
||||
`scripts/frank-default-large.env`, applying the Qwen-Sharp chat template.
|
||||
See [[kingjones-strix-mtp]].
|
||||
See [[eemin-rocmi4-mtp]].
|
||||
|
||||
> The kingjones Q6 AGENT model has served the slot since 2026-08-24; the
|
||||
> v3 PromptForge payload is retired on this slot.
|
||||
> Restored 2026-08-21. The intermediate DFlash2 Vulkan deployment was
|
||||
> retired the same day. See [[dflash2-large-vulkan]] for the archived runbook.
|
||||
> The kingjones ROCmFP4 STRIX MTP model served the slot from 2026-08-24; it
|
||||
> was replaced by the caf-eemin ROCmI4 embedded-MTP model on 2026-08-25. The
|
||||
> v3 PromptForge payload and the CIRU ROCmFPX runtime are retired on this slot.
|
||||
> The intermediate DFlash2 Vulkan deployment was retired the same day (2026-08-21).
|
||||
> See [[dflash2-large-vulkan]] for the archived runbook.
|
||||
|
||||
## Validation
|
||||
|
||||
|
||||
Reference in new issue
Block a user