serverAppConfigs/serverAppConfigs-docs/LLM/frank-large-rocmfpx-q6agent-uncensored-mtp.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	frank-large on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP	The active frank-large runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Uncensored Q6_0_ROCMFPX_AGENT GGUF (ftype 114, protected output heads), with the repo's own MTP-Q4_0 head drafting four tokens at one native 262,144 window, f16 KV, the BF16 projector for vision, and the measured rejection of the ngram-mod chain.	llm frank-large dankclaw llama-cpp rocmfpx gfx1151 gguf spec-decode mtp	2026-10-03

frank-large — ROCmFPX Uncensored Q6_0_ROCMFPX_AGENT + MTP (gfx1151)

RETIRED 2026-10-03 → 2026-10-05 (rollback target; payload, fork build and build script all on disk). Replaced by frank-large-halogen27b-dflash2 (DFlash2, publisher 31.7 t/s mean vs the 27.6–28.1 measured here). This page remains the contract record for the rollback. Original banner follows. ACTIVE since 2026-10-03. Cut over from frank-large-rocmfpx-q8agent-mtp (ALIGNED Q8_0_ROCMFPX_AGENT, 17.0 t/s) on Frank's request: the uncensored variant of the same protected-heads family, one quant rung down. The swap turned out to be a strict upgrade on the axis that matters here: decode 27.6–28.1 t/s at a 4,813-token prompt against 17.0 at a 2,202-token prompt — faster with a deeper context, not a same-depth comparison, but the sign is not in question.

Identity

Host / device	Dankclaw (192.168.0.143), Radeon 8060S iGPU gfx1151, ~114 GiB GTT aperture
Endpoint	http://192.168.0.143:7770 — alias frank-large, /v1 OpenAI surface
Runtime	kingjones30/ROCmFPX llama.cpp fork @ 36e9acd40e10a87cd3c3ef8ec734668757dc8520, built by scripts/build-rocmfpx.sh — unchanged from the Q8 era; the fork already knew type 102
Unit	root systemd frank-large.service (scripts/frank-large.service), User=dank, Restart=always
Serve script	scripts/serve-frank-large.sh (owns the argv), tunables scripts/frank-large.env
Weights	/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf — 24,856,607,008 B, sha256 e966deb3…
Draft	…/mtp-Qwen3.8-27B-Q4_0.gguf — 1,680,271,648 B, sha256 051a1764… (byte-identical to the aligned repo's head; hardlinked, not re-downloaded)
Projector	…/mmproj-Qwen3.8-27B-BF16.gguf — 931,145,888 B (hardlinked from the aligned payload dir; the same file the Q8 era served)
Fetch	scripts/download-frank-large-kingjones-uncensored.sh — hardlinks the draft/projector from the aligned dir when present, otherwise fetches; -f, byte-count + LFS sha256 verified, --verify-only, token never on the command line
Seam	tests/frank_large_unit.bats (payload gates, deployer/serve parity, spec block, KV dtype)

Why this file

kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF ships the same "protected heads" construction as the aligned repo, in the uncensored tuning. Parsed header of the deployed file:

Field	Value
ftype	114 — the fork's 64-bit-block family, ≈7.44 bpw
Tensor mix	353 F32 + 339 type-102 (GGML_TYPE_Q6_0_ROCMFPX) + 159 type-103 (Q8_0_ROCMFPX)
Protected heads	64 output-head tensors kept at plain precision — the AGENT construction from the aligned repo, at the 6-bit rung
Arch	qwen35, 64 layers, full_attention_interval 4, context_length 262144
Embedded sampling recipe	temp 1.0 / top_p 0.95 / top_k 20

The same lock-in law as the Q8 era applies: types 102 and 103 exist in no loader but this fork; stock llama.cpp and gufo refuse the file at load (unknown tensor type). The loader was not re-litigated — the file picks the runtime, and the pin in scripts/build-rocmfpx.sh (commit by SHA) stays.

Rejected sibling: Q6_0_ROCMFPX_LEAN (6.45 bpw, no protected heads) — the protection is what keeps the output-head logits at full precision and the drafter accurate, so it stays deployed.

Shape and budget

Unchanged from the Q8 era — the swap was payload-only:

Dial	Value	Why
--ctx-size	262,144	the native window, no rope scaling
--parallel	1	one conversation, all memory to it
KV dtype	f16 (both K and V)	~64 KiB/token; free inside a 114 GiB aperture
--fit off	on	the auto-fitter would silently resize the window
--flash-attn on	on	required for the long-window attention path
--direct-io, --ctx-checkpoints 16, --cache-ram 8192, --cache-idle-slots, --jinja, --reasoning on --reasoning-format deepseek --reasoning-budget -1		same as frank-large-rocmfpx-q8agent-mtp

Spec decode: --spec-type draft-mtp, the repo's mtp-Qwen3.8-27B-Q4_0.gguf, --spec-draft-n-max 4, pinned --spec-draft-device ROCm0.

Measured steady state: GTT ~61 GiB under live load (host baseline 16.7), boot to /health ~15 s.

Measured on this payload (2026-10-03, production service, /completion timings, temp 0, cache_prompt: false)

Traffic	Decode t/s	Prefill t/s	Draft acceptance
Prose, 4,813-token prompt, 512 tokens	27.57 / 28.10 (two runs)	~205–217	396/456 = 0.868 → mean acceptance length ≈ 4.4, i.e. drafting at nearly the full n-max 4
Schema echo (structured repetition)	31.44	—	639/639 = 1.000
Block copy (text repeated verbatim)	~28.0	—	0.848

vs the Q8 era's 0.462 acceptance at a 2,202-token prompt: the MTP head drafts materially better on this file. The card's claimed 0.953-class acceptance is no longer science fiction on real prose — but keep reading it as a file property, not a throughput promise.

The ngram-mod chain — measured and rejected

Frank's ask: layer --spec-type draft-mtp,ngram-mod on top of MTP for an extra gain. Measured in three bench windows (production service stopped per the benchmarking law; artifacts /home/dank/fl-ngram-ab/ on Dankclaw), comparing draft-mtp alone vs the chain on the same traffic:

Traffic	MTP alone	MTP + ngram-mod	Delta
Prose	27.57 t/s	23.40 t/s	−15 %
Block copy	28.0 t/s	26.1 t/s	−7 % (with ngram-mod demonstrably firing: 138 proposals/run)

Mechanism, straight from arm B's boot log:

recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support

On the qwen35 hybrid (Gated DeltaNet recurrent layers) adding the ngram layer disables the rollback path MTP's own verification relies on, so acceptance — and throughput — drop on all traffic, even where the ngram matcher fires. Deployed decision: draft-mtp alone. Chaining is not free on recurrent models; the fleet law in AGENTS.md → llama.cpp serving records it.

Bench-window hardening laws (learned building the A/B)

1. Health 200 is not proof of ownership. After boot, verify the pid you started owns the port (ss -ltnp); a zombie server from a prior window answers /health too and silently serves your arm.
2. Kill scratch servers with SIGKILL after a grace period. llama-server's graceful SIGTERM waits on open keep-alive connections — a harness holding the socket leaves the process (and its ~24 GiB of GTT) alive after the script exits.
3. Prove the port is free and GTT dropped between arms, or the second arm measures a collision.

Client contract

Unchanged from the Q8 era and re-verified on this payload 2026-10-03: model id frank-large, native tools, thinking in reasoning_content, reasoning_effort ∈ {xhigh, medium, low} only (anything else HTTP 500), card-recipe sampling defaults server-side, vision through the BF16 projector (red PNG → Red, 74 prompt / 165 completion tokens). The traps list — content: null when reasoning eats the budget, the self-recovering MTP boundary log line, enable_thinking:false not reasoning_effort:"none" — is in frank-large-rocmfpx-q8agent-mtp and still applies.

Sweep key for scratch servers is now the payload name: pkill -f 'Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT' — never pkill -f llama-server (frank-small is also one).

Operations

./scripts/deploy.sh dankclaw --service frank-large
systemctl status frank-large
journalctl -u frank-large -n 200 --no-pager
scripts/download-frank-large-kingjones-uncensored.sh --verify-only

Rollback

Target	How
ALIGNED Q8_0_ROCMFPX_AGENT (previous shape, 17 t/s)	the payload dir /home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/ is untouched on disk; revert the swap commit (unit/env/serve trio) and redeploy, or point FRANK_LARGE_MODEL_FILE + LLM_LARGE_MODEL_FILE at the old paths. Verify with scripts/download-frank-large-kingjones.sh --verify-only
gufo Q4_K_XL era	git tag frank-large-gufo-q4kxl-20261001, but the gufo payloads (UD-Q4_K_XL 16.35 GiB, UD-Q8_K_XL + DFlash2 drafter) were removed from disk 2026-10-03 with Frank's approval — this rollback now requires re-download per frank-large-gufo-qwen38-27b

When it misbehaves

Same decision tree as frank-large-rocmfpx-q8agent-mtp: unknown tensor type ⇒ off the pinned fork; SIGSEGV in ggml_compute_forward_set_rows ⇒ pre-36e9acd build; slow decode with no error ⇒ draft not engaged (draft_n in /completion timings) or an orphaned second copy eating GTT (/sys/class/drm/card1/device/mem_info_gtt_used); 500 naming Jinja ⇒ --jinja lost; 500 naming the template ⇒ unmapped reasoning_effort in models.yml.