Files

T

Frank 55f87fcdce llm: retire the trio service names — fleet services renamed to frank-archon/frank-dragoon/frank-zealot

The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.

Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.

2026-10-10 17:03:14 -07:00

3.7 KiB

Raw Permalink Blame History

type, title, description, tags, timestamp

type

title

description

kingjones Qwen3.8-27B ROCmFP4 STRIX MTP on Dankclaw

RETIRED 2026-08-29. This payload was frank-dragoon from 2026-08-25 until the Q8_0 AGENT experiment on 2026-08-29 (it was restored again 2026-09-01 and served until 2026-08-30, when agentionai-fast-imatrix took over). The Q8_0 model was retired because it crashed before serving. Kept as the historical record of the Q6 AGENT model, standalone MTP draft, Q8 projector, and PromptForge runtime.

This page documents a retired model and its runbook.

frank-dragoon (port 7770) serves Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP from kingjones777 on Hugging Face. It is a dense Qwen3.8-27B, abliterated (uncensored), quantized to the ROCmFP4 Q6 AGENT format (ftype 114). The MTP speculative head ships as a separate Q4_0 draft GGUF, so a bundled-head auto-detect does not apply here; the head is loaded with --model-draft.

Model Files

The card ships six text files plus the vision projector. The service uses the Q6 AGENT file (requested for the fleet), the separate MTP draft head and the Q8 vision projector (mmproj-Qwen3.8-27B-Q8_0.gguf).

File	Size
`Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`	23.15 GiB
`mtp-Qwen3.8-27B-Q4_0.gguf` (draft head)	1.56 GiB
`mmproj-Qwen3.8-27B-Q8_0.gguf` (vision)	0.59 GiB

Store the files under /home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/.

Speculative Decoding

Dense model, so the finding measured on the dense Qwen3.8 family holds: MTP alone at --spec-draft-n-max 4 is the knee. ngram under MTP hurts (measured -4 to -5 t/s on the v3; see ciru-activefpx-promptforge). The draft head is a separate file loaded via:

--spec-type draft-mtp
--model-draft mtp-Qwen3.8-27B-Q4_0.gguf
--spec-draft-n-max 4
--spec-draft-n-min 0
--spec-draft-p-min 0.0

The draft head loads on ROCm1 with full offload (--spec-draft-ngl 999) and f16 KV.

Launch Excerpt

The key flags on the CIRU ROCmFPX runtime:

llama-server \
  -m Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf \
  --mmproj mmproj-Qwen3.8-27B-Q8_0.gguf \
  --chat-template-file chat-template.jinja \
  --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true}' \
  -ngl 999 -dev ROCm0 -c 196608 -np 1 \
  -b 8192 -ub 8192 -ctxcp 32 \
  --spec-type draft-mtp --model-draft mtp-Qwen3.8-27B-Q4_0.gguf \
  --spec-draft-ngl 999 --spec-draft-n-max 4 \
  -fa on --no-mmap --cache-type-k f16 --cache-type-v f16

Known Issues

GPU only: Vulkan, CUDA, and CPU cannot load these files.
--spec-draft-n-max defaults to 16; set 4.
reasoning_effort: "none" throws in the Qwen API. Use enable_thinking: false instead.
Small max_tokens with thinking on returns empty content. The Sharp template drives thinking on by default (effort medium).
preserve_thinking:true (in chat_template_kwargs) keeps <think> blocks in recorded assistant messages. preserve_thinking:false strips them. See deployment for the chat_format/reasoning_format pairing (peg-native + deepseek) the Pi agent expects from this endpoint.

3.7 KiB Raw Permalink Blame History