frank-large: ROCmFPX llama.cpp on the Q8_0_ROCMFPX_AGENT file, MTP drafting

The repo's highest-quality rung is ftype 115: 311 tensors of
GGML_TYPE_Q8_0_ROCMFPX, a type only the ROCmFPX fork reads, so the runtime
moves from gufo to scripts/build-rocmfpx.sh's pinned fork build (36e9acd) —
the first build on this host measured to load this file without the
2026-08-25 set_rows segfault. One session at the native 262,144 window, f16
KV (64 KiB/token, 16 GiB), the repo's Q4_0 MTP head drafting 4 tokens, plus
the bundled BF16 projector so the endpoint has vision again.

The serve script now honors caller environment overrides over the env file
(a plain source let the file win, which made single-run overrides no-ops) and
defaults apply only when a value is unset, so an explicit LLAMA_BIN='' is the
error it reads as.

scripts/frank-large.env

@@ -1,63 +1,85 @@
-# frank-large — gufo (gufo-org/gufo) serving
-# Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) on the Strix Halo iGPU.
+# frank-large — ROCmFPX llama.cpp serving
+# kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF (Q8_0_ROCMFPX_AGENT, ftype
+# 115) + its Q4_0 MTP draft head, on the Strix Halo iGPU (gfx1151), :7770.
 #
-# These are the tunables read by scripts/serve-frank-large.sh. The flag
-# list is owned by the serve script; this file only sets fleet values.
+# 2026-10-02 (Frank): "the highest quality of that repo, one full context,
+# whatever runtime it requires". The highest-quality rung is Q8_0_ROCMFPX_AGENT
+# (26.28 GiB, 8.39 bpw: 187 tensors at plain Q8_0 + 311 at
+# GGML_TYPE_Q8_0_ROCMFPX); the AGENT routing is also the variant the card
+# tells you to take with MTP, because it keeps the output-head tensors at full
+# Q8_0 (187 vs 2), which is what the draft scores against. The runtime the
+# format REQUIRES is the ROCmFPX fork — type 103 exists nowhere else.
 #
-# DO NOT name a new tunable TEMP, HOST or GPU_MAX_HEAP_SIZE: the HIP
-# runtime reads those bare (the 2026-08-13 TEMP incident). HOST is the one
-# exception because the serve script itself consumes it for --host.
-
-# The target payload (scripts/download-frank-large-q4.sh): the serve script
-# refuses to start without the main GGUF and the DFlash2 drafter.
+# The shape: one session at the model's full native window (262,144), which is
+# the whole point — the Q8_0 head costs quality the fleet wanted back, and
+# the draft head pays for the bandwidth the bigger file spends.
 #
-# 2026-10-01 swap (Frank): unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL, 16.35 GiB,
-# replacing ajgazin's abliterated Swift-1.5 UD-Q8_K_XL (29.3 GiB). Two
-# consequences worth knowing at a glance:
-#   * this is the VANILLA (non-abliterated) Qwen3.8-27B — the endpoint is no
-#     longer uncensored, which was one of the reasons the Swift payload was
-#     chosen on 2026-09-28;
-#   * ~13 GiB of weights became available for KV, which is what makes four
-#     full-sized f16 sessions fit.
-# gufo needs no template for either: its Qwen renderer is compiled in.
-MODEL_DIR=/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL
-MODEL_FILE=/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL/Qwen3.8-27B-UD-Q4_K_XL.gguf
+# Budget (from the GGUF header, not from a boot log): attention KV lives only
+# in the 16 full-attention layers (4 KV heads x 256 head_dim), 64 KiB/token
+# with f16 K and V => 16.0 GiB at 262,144. The 48 Gated DeltaNet layers hold
+# ~144 MiB of fixed state per sequence. Weights 26.28 + draft head 1.57 +
+# mmproj 0.87 + KV 16.0 + state 0.14 + compute buffers ≈ 45-47 GiB against the
+# iGPU's ~114 GiB GTT aperture, so f16 KV (the unquantized choice) is affordable
+# here in a way it is not on frank-small's 24 GiB card.
 
-# The DFlash2 drafter is UNCHANGED by the model swap and still lives beside
-# the previous target, so it carries its own directory. It has a sliding
-# window of 2048, which is what keeps its KV small (~40 MiB).
-DRAFTER_DIR=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL
-DRAFTER_FILE=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf
+# The qualified ROCmFPX fork build (scripts/build-rocmfpx.sh pins the commit).
+LLAMA_BIN=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server
+# One directory per HF repo, created by scripts/download-frank-large-kingjones.sh.
+MODEL_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf
+# The draft head: Q4_0, NOT the Q8_0 head. The head's per-step cost is its
+# projection over a 248K vocabulary, so the cheaper head is faster end to end
+# at equal-or-better acceptance (card: +6.0% at n-max 1, +10.2% at n-max 2).
+DRAFT_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mtp-Qwen3.8-27B-Q4_0.gguf
+# Vision tower, separate from the LLM file. Empty string = text-only.
+MMPROJ_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mmproj-Qwen3.8-27B-BF16.gguf
 
-# The gufo runtime, built from source by scripts/build-gufo.sh.
-RUNTIME_DIR=/home/dank/GitPackages/gufo-v0.1.1
-
-# Fleet endpoint: the same port and alias every frank-large has served, so
-# no client, probe or models.yml entry changes address.
+# Fleet endpoint: same port and alias as every frank-large before it, so the
+# harness models.yml id and the homepage probe do not move.
 ALIAS=frank-large
-HOST=0.0.0.0
-PORT=7770
+# HIP reads a bare HOST, so this is BIND_ADDR — the same reason the file
+# avoids a bare TEMP.
+BIND_ADDR=0.0.0.0
+LISTEN_PORT=7770
 
-# ONE session at the full native window (Frank, 2026-10-01 night: "roll back
-# to qwen3.8 27b, one single max context"). The KV is UNQUANTIZED (f16) —
-# gufo has no KV quantization lever at all (execution_policy.hpp pins
-# kv_cache_storage to kFp16), so "unquantized" is inherent rather than
-# configured.
-#
-# The arithmetic: f16 KV costs 64 KiB/token on this model (16 full-attention
-# layers x 4 KV heads x 256 head_dim x 2 (K,V) x 2 bytes; the other 48 layers
-# are linear-attention with a per-SEQUENCE fp32 state of ~144 MiB, not a
-# per-token cache). 262,144 tokens = 16.0 GiB of KV. Budget: weights 16.35 +
-# KV 16.0 + drafter 1.07 + linear state 0.14 + runtime ~4 = ~37.6 GiB against
-# the 114 GiB GTT — enormous headroom; the session count or context policy
-# can grow from here without pressure.
-CONTEXT_SIZE=262144
-SESSIONS=1
+# One full native window. llama.cpp divides --ctx-size across --parallel
+# slots, so 262144/1 IS the model's trained window; the card validated 65,536
+# and the fleet has been running this family at 262,144 since the gufo era.
+CTX_SIZE=262144
+PARALLEL=1
 
-# DFlash2 spec decode (a block-diffusion drafter). gufo sizes the draft
-# block itself; DRAFT_TOKENS caps the proposals per step (gufo's default is
-# 7, chosen over llama.cpp's fixed depth 4 because gufo's adaptive policy
-# shortens the block when acceptance drops) and DRAFT_POLICY is the block
-# length controller.
-DRAFT_TOKENS=7
-DRAFT_POLICY=adaptive
+# Deck knobs. --ubatch-size has a measured optimum on this stack (512 peak on
+# the sibling qwen35 model; larger ubatch costs GiBs of compute buffer for no
+# speed), and --fit must stay off: --fit's auto placement does not know that
+# 311 tensors of type 103 belong on the iGPU.
+BATCH_SIZE=2048
+UBATCH_SIZE=512
+NGL=999
+FIT=off
+FLASH_ATTN=on
+# Unquantized KV — the quality-first choice, affordable at one slot.
+CACHE_TYPE_K=f16
+CACHE_TYPE_V=f16
+
+# Speculative decode: the model ships a standalone MTP head (mtp-Q4_0.gguf,
+# arch qwen35, nextn_predict_layers=1, a 65-block draft). n-max 4 with n-min 0
+# and p-min 0.0 is the card's measured optimum (+31% end to end, acceptance
+# 0.926; its own n-max 8 measurement only reached 0.757 acceptance and LOST).
+# The draft runs entirely on the iGPU, so cross-device traffic stays off the
+# bandwidth-limited link.
+SPEC_TYPE=draft-mtp
+SPEC_DRAFT_NGL=99
+SPEC_DRAFT_DEVICE=ROCm0
+SPEC_DRAFT_N_MAX=4
+SPEC_DRAFT_N_MIN=0
+SPEC_DRAFT_P_MIN=0.0
+
+# The card's thinking sampling profile, which is also what the harness sends
+# in extraBody, so the endpoint behaves the same to a client that omits it.
+TEMPERATURE=1.0
+TOP_P=0.95
+TOP_K=20
+MIN_P=0.0
+PRESENCE_PENALTY=0.0
+
+NUM_THREADS=16
+THREAD_BATCH=16