Swap LLM fleet to Ornith CRACK (small) and kingjones ROCmFP4 STRIX MTP (large); apply Qwen-Sharp chat template
- frank-default-small (Dankserver): Ornith-1.5-35B-A3B-CRACK Q8_0, bundled native draft-mtp (n_max 4), MoE expert FFN tensors pinned to CPU via an explicit --override-tensor \.ffn_(up|down|gate|gate_up)_(ch|)exps=CPU (replaces --n-cpu-moe), Qwen-Sharp chat template. - frank-default-large (Dankclaw): kingjones Qwen3.8-27B-Uncensored-Q6_0 ROCmFP4 AGENT dense GGUF, standalone mtp-Qwen3.8-27B-Q4_0 draft head (--model-draft, draft-mtp n_max 4), Qwen-Sharp chat template. Drop the retired PromptForge .pfs sidecar lanes. - Track scripts/chat-template.jinja (v22.4.0 Sharp) and apply via --chat-template-file on both services. - Docs: new kingjones-strix-mtp and dealign-ornith-crack runbook pages; update index, deployment, and both system specs; models.json names. Smoke-tested both models on the real hardware on scratch ports before commit: model loads, MTP draft attaches, Sharp template applies, chat completion returns.
This commit is contained in:
1 parent
189934af15
commit
4462aa992c
14 files changed
+827
-156
No files matched your search
@@ -0,0 +1,91 @@
|
||||
---
|
||||
type: reference
|
||||
title: kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
||||
description: kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Dense model with a standalone MTP Q4_0 draft head. frank-default-large.
|
||||
tags:
|
||||
- llama.cpp
|
||||
- rocm
|
||||
- dankclaw
|
||||
- gfx1151
|
||||
- kingjones
|
||||
- rocmfp4
|
||||
- mtp
|
||||
timestamp: 2026-08-24
|
||||
---
|
||||
|
||||
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
||||
|
||||
> **Active 2026-08-24.** `frank-default-large` serves this model. It
|
||||
> replaced the jcbtc v3 CIRU ActiveFPX PromptForge payload on the same
|
||||
> ROCmFPX runtime and the same CIRU llama.cpp binary. See
|
||||
> [[ciru-activefpx-promptforge]] for the retired v3 page.
|
||||
|
||||
`frank-default-large` (port 7770) serves
|
||||
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
|
||||
Face. It is a **dense** Qwen3.8-27B, abliterated (uncensored), quantized to
|
||||
the ROCmFP4 Q6 AGENT format (ftype 114). The MTP speculative head ships as
|
||||
a **separate** Q4_0 draft GGUF, so a bundled-head auto-detect does not apply
|
||||
here; the head is loaded with `--model-draft`.
|
||||
|
||||
## Model Files
|
||||
|
||||
The card ships six text files plus the vision projector. The service uses
|
||||
the Q6 AGENT file (requested for the fleet), the separate MTP draft head
|
||||
and the Q8 vision projector.
|
||||
|
||||
| File | Size |
|
||||
|------|------|
|
||||
| `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` | 23.15 GiB |
|
||||
| `mtp-Qwen3.8-27B-Q4_0.gguf` (draft head) | 1.56 GiB |
|
||||
| `mmproj-Qwen3.8-27B-Q8_0.gguf` (vision) | 0.59 GiB |
|
||||
|
||||
Store the files under `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/`.
|
||||
|
||||
## Speculative Decoding
|
||||
|
||||
Dense model, so the finding measured on the dense Qwen3.8 family holds:
|
||||
**MTP alone at `--spec-draft-n-max 4` is the knee**. ngram under MTP hurts
|
||||
(measured -4 to -5 t/s on the v3; see [[ciru-activefpx-promptforge]]). The
|
||||
draft head is a separate file loaded via:
|
||||
|
||||
```
|
||||
--spec-type draft-mtp
|
||||
--model-draft mtp-Qwen3.8-27B-Q4_0.gguf
|
||||
--spec-draft-n-max 4
|
||||
--spec-draft-n-min 0
|
||||
--spec-draft-p-min 0.0
|
||||
```
|
||||
|
||||
The draft head loads on `ROCm1` with full offload (`--spec-draft-ngl 999`)
|
||||
and f16 KV.
|
||||
|
||||
## Launch Excerpt
|
||||
|
||||
The key flags on the CIRU ROCmFPX runtime:
|
||||
|
||||
```
|
||||
llama-server \
|
||||
-m Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf \
|
||||
--mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
|
||||
--chat-template-file chat-template.jinja \
|
||||
-ngl 999 -dev ROCm0 -c 262144 -np 1 \
|
||||
--spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
|
||||
--spec-draft-ngl 999 --spec-draft-n-max 4 \
|
||||
-fa on --no-mmap --cache-type-k f16 --cache-type-v f16
|
||||
```
|
||||
|
||||
## Known Issues
|
||||
|
||||
1. **GPU only**: Vulkan, CUDA, and CPU cannot load these files.
|
||||
2. `--spec-draft-n-max` defaults to 16; set 4.
|
||||
3. `reasoning_effort: "none"` throws in the Qwen API. Use
|
||||
`enable_thinking: false` instead.
|
||||
4. Small `max_tokens` with thinking on returns empty `content`. The Sharp
|
||||
template drives thinking on by default (effort medium).
|
||||
|
||||
## See Also
|
||||
|
||||
- [[deployment]] — deploy and update process
|
||||
- [[ciru-activefpx-promptforge]] — the retired v3 page
|
||||
- [[Dankclaw System Specs]] — the box
|
||||
- [[frank-redo-2026-08-17]] — fleet topology history
|
||||
Reference in new issue
Block a user