Frank
93d1181fee
- Migrate all 127 non-reserved docs to OKF v0.1 frontmatter (type/description/tags/timestamp); fix 13 YAML issues, 2 duplicate-FM blocks, strip reserved-page FM - Restructure log.md to OKF (## YYYY-MM-DD, newest-first) - Fact-check against scripts/AGENTS.md ground truth: fix 6 false 'Active' banners on retired runbooks, 96->114 GiB GTT, deployment.md vs deploy-plan.py/deploy-remote.sh, Deployment Flow frank-medium=Quadlet, Calibre 9.9.0->9.15.0 + CVE-2025-7404 conflation, Copyparty v1.20.24, RomM 5.3.0 breaking change - Move *arr docs to new FrankBook-Pro/ section (git mv, link-safe) - Write active runbook bodies from ground truth (frank-large, frank-small) - Add 3 research briefs + HTML report: Local TTS and Voice Cloning, AI Job Search and Resume Tools, Most Popular Self-Hosted AI Apps (Jun-Sep 2026) - Add upstream-version rows to ComfyUI/OpenWebUI/Hermes app docs - All 336 tests pass; 136 pages, 0 unresolved wikilinks
101 lines
3.6 KiB
Markdown
101 lines
3.6 KiB
Markdown
---
|
|
type: reference
|
|
title: kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
|
description: kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Dense model with a standalone MTP Q4_0 draft head. frank-large.
|
|
tags:
|
|
- llama.cpp
|
|
- rocm
|
|
- dankclaw
|
|
- gfx1151
|
|
- kingjones
|
|
- rocmfp4
|
|
- mtp
|
|
timestamp: 2026-08-24
|
|
---
|
|
|
|
# kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw
|
|
|
|
> **RETIRED 2026-08-29.** This payload was `frank-large` from 2026-08-25
|
|
> until the Q8_0 AGENT experiment on 2026-08-29 (it was restored again
|
|
> 2026-09-01 and served until 2026-08-30, when [[agentionai-fast-imatrix]]
|
|
> took over). The Q8_0 model was retired because it crashed before
|
|
> serving. Kept as the historical record of the Q6 AGENT model, standalone
|
|
> MTP draft, Q8 projector, and PromptForge runtime.
|
|
|
|
> This page documents a retired model and its runbook.
|
|
|
|
`frank-large` (port 7770) serves
|
|
`Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP` from `kingjones777` on Hugging
|
|
Face. It is a **dense** Qwen3.8-27B, abliterated (uncensored), quantized to
|
|
the ROCmFP4 Q6 AGENT format (ftype 114). The MTP speculative head ships as
|
|
a **separate** Q4_0 draft GGUF, so a bundled-head auto-detect does not apply
|
|
here; the head is loaded with `--model-draft`.
|
|
|
|
## Model Files
|
|
|
|
The card ships six text files plus the vision projector. The service uses
|
|
the Q6 AGENT file (requested for the fleet), the separate MTP draft head
|
|
and the Q8 vision projector (`mmproj-Qwen3.8-27B-Q8_0.gguf`).
|
|
|
|
| File | Size |
|
|
|------|------|
|
|
| `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` | 23.15 GiB |
|
|
| `mtp-Qwen3.8-27B-Q4_0.gguf` (draft head) | 1.56 GiB |
|
|
| `mmproj-Qwen3.8-27B-Q8_0.gguf` (vision) | 0.59 GiB |
|
|
|
|
Store the files under `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/`.
|
|
|
|
## Speculative Decoding
|
|
|
|
Dense model, so the finding measured on the dense Qwen3.8 family holds:
|
|
**MTP alone at `--spec-draft-n-max 4` is the knee**. ngram under MTP hurts
|
|
(measured -4 to -5 t/s on the v3; see [[ciru-activefpx-promptforge]]). The
|
|
draft head is a separate file loaded via:
|
|
|
|
```
|
|
--spec-type draft-mtp
|
|
--model-draft mtp-Qwen3.8-27B-Q4_0.gguf
|
|
--spec-draft-n-max 4
|
|
--spec-draft-n-min 0
|
|
--spec-draft-p-min 0.0
|
|
```
|
|
|
|
The draft head loads on `ROCm1` with full offload (`--spec-draft-ngl 999`)
|
|
and f16 KV.
|
|
|
|
## Launch Excerpt
|
|
|
|
The key flags on the CIRU ROCmFPX runtime:
|
|
|
|
```
|
|
llama-server \
|
|
-m Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf \
|
|
--mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
|
|
--chat-template-file chat-template.jinja \
|
|
--chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
|
|
-ngl 999 -dev ROCm0 -c 196608 -np 1 \
|
|
-b 8192 -ub 8192 -ctxcp 32 \
|
|
--spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
|
|
--spec-draft-ngl 999 --spec-draft-n-max 4 \
|
|
-fa on --no-mmap --cache-type-k f16 --cache-type-v f16
|
|
```
|
|
|
|
## Known Issues
|
|
|
|
1. **GPU only**: Vulkan, CUDA, and CPU cannot load these files.
|
|
2. `--spec-draft-n-max` defaults to 16; set 4.
|
|
3. `reasoning_effort: "none"` throws in the Qwen API. Use
|
|
`enable_thinking: false` instead.
|
|
4. Small `max_tokens` with thinking on returns empty `content`. The Sharp
|
|
template drives thinking on by default (effort medium).
|
|
5. `preserve_thinking:true` (in `chat_template_kwargs`) keeps `<think>`
|
|
blocks in recorded assistant messages. `preserve_thinking:false` strips
|
|
them. See [[deployment]] for the chat_format/reasoning_format pairing
|
|
(`peg-native` + `deepseek`) the Pi agent expects from this endpoint.
|
|
|
|
## See Also
|
|
|
|
- [[deployment]] — deploy and update process
|
|
- [[ciru-activefpx-promptforge]] — the retired v3 page
|
|
- [[Dankclaw System Specs]] — the box
|
|
- [[frank-redo-2026-08-17]] — fleet topology history |