frank-primary: 2 x 200704 fp8, pool cut for the expert arena, MNBT 1024

The 2026-10-01 bf16 shape (2 x 262144, --expert-precise-memory) crash-looped at graph capture — the arena is DEVICE-resident and that shape had 0.13 GiB to spare. The plain bf16 2 x 262144 also did not stay up: 0 bytes free at runtime, and a real client request (995-token prompt, max_tokens=261149) killed the worker with . The correction is a pool cut, not a downgrade. FP8 KV (7,701 B/token), two 200704 windows (401,408 tokens -> 3.091e9 B floor, pinned 3.15e9), MNBT 1024. The fork's expert planner sizes the arena AFTER the KV override, so the ~2.95 GiB/rank reclaimed becomes arena: ~7,500 slots vs 6,248. Two contexts at 200704 is a capacity contract, not a throughput one (2-way concurrency summed BELOW parity at 4064 ctx), so the slot buys the VRAM headroom long generations need. Refs [[dankserver-qwen38-fp8-expert-offload-tuning]]. - scripts/frank-primary.env: MAXLEN 200704, SEQS 2, KV_BYTES 3.15e9, MNBT 1024, rewritten shape comment - scripts/serve-frank-primary.sh: header comment updated - tests/frank_primary_unit.bats: assertions updated, new seam pins the arena-floor pool law (the pool must move with a context change or the reduction buys nothing) - docs + AGENTS.md: fleet topology and deployed shape corrected
autoresearch: harness setup
2026-10-02 09:54:08 -07:00 · 2026-10-02 07:50:50 -07:00 · 2026-10-02 03:46:50 -07:00 · 2026-10-02 03:17:35 -07:00 · 2026-10-02 03:08:00 -07:00 · 2026-10-02 02:31:39 -07:00
35 changed files with 3423 additions and 1084 deletions
@@ -12,8 +12,8 @@ merge absorbed the former `vllmScripts` repo, history preserved).
  Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service.
 - **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
  via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
-  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (stock llama.cpp on
-  the gfx1151 iGPU) and `frank-small` (SGLang on the RTX 3090). Caddy on
+  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (ROCmFPX llama.cpp
+  fork on the gfx1151 iGPU) and `frank-small` (llama.cpp CUDA on the RTX 3090). Caddy on
  Dankclaw reverse-proxies both hosts.
 - **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
  (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) as quadlets inside
@@ -24,9 +24,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).

 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
-| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **bf16 KV** (`--kv-cache-dtype auto`), **no speculative decode**, MNBT 2048, MoE gather 32×64, **2 seqs × 262144 ctx (the model's full native window)** at KV_BYTES 7,300,000,000/rank (bf16 measures 13,839 B/token), with **`--expert-precise-memory`** — the fitted 5.25 GiB overhead instead of the 7.60 GiB default, which hands the compile/graph-capture headroom to the expert arena (~7,278 slots vs 6,248) — util 0.98, and the client contract wired: `--reasoning-parser qwen3` (thinking in `reasoning`, never inline), `--enable-auto-tool-choice --tool-call-parser qwen3_xml`, default chat-template kwargs `enable_thinking` + `preserve_thinking`. Capacity history: five full-length contexts boot but leave ~0 bytes free at runtime (a real request with `max_tokens=261149` killed the worker: `Tried to allocate 50.00 MiB ... 0 bytes free`); six never fit (6×241664 asked 10.70 GiB vs 9.18 GiB free). DCP cannot help here — it requires TP > the model's 2 KV heads, i.e. four GPUs. fp8+MTP4 shape restorable from git tag `frank-primary-fp8-mtp-20261001` | 7770 |
-| Dankclaw | `frank-large` | gufo `gufo-org/gufo` v0.1.1, the Strix Halo engine, BUILT FROM SOURCE (root systemd unit; links the host ROCm 7.2.4) | `unsloth/Qwen3.8-27B-GGUF` **UD-Q4_K_XL** (27B dense, **vanilla — no longer abliterated**, 16.35 GiB, Apache-2.0) with the z-lab DFlash2 Q4_K_M drafter (adaptive block policy, cap 7) on the gfx1151 iGPU (Strix Halo LPDDR, 114 GiB GTT), **4 sessions x 163840 ctx, unquantized (f16) KV** — gufo has no KV-quant lever — greedy, gufo's compiled-in Qwen renderer (NO Qwen-Sharp), text only. Swapped 2026-10-01 from ajgazin's abliterated Swift-1.5 UD-Q8_K_XL (29.3 GiB): ~13 GiB less weights is what funds four f16 sessions, at the cost of the uncensored behaviour | 7770 |
-| Dankclaw | `frank-small` | SGLang in a self-contained python3.12 venv (root systemd unit; the venv carries its own CUDA 13 stack — see the runbook) | letechlead MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5 hybrid, 9B dense, 12.0 GiB, INT8 W8A16 AutoRound, in-checkpoint vision tower, NO MTP weights shipped so no spec decode) on the RTX 3090, fp8 KV, pinned 606,208-token pool / 2 slots of 262,144 (native window; 2026-10-01 reslice from 4 x 151,552 for compaction headroom), bf16 mamba states, eager prefill, 83.9 tok/s decode | 7771 |
+| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **fp8 KV** (`--kv-cache-dtype fp8`), **no speculative decode**, MNBT 1024, MoE gather 32×64, **2 seqs × 200704 ctx** at KV_BYTES 3,150,000,000/rank (fp8 measures 7,701 B/token), pool deliberately small so the expert arena keeps what it reclaims (~7,500 slots vs 6,248 at the 6.1e9 shape) — util 0.98, and the client contract wired: `--reasoning-parser …
+| Dankclaw | `frank-large` | **ROCmFPX llama.cpp fork** (`kingjones30/ROCmFPX` @ `36e9acd`, gfx1151, built by `scripts/build-rocmfpx.sh`; bare `llama-server`, root systemd unit) | `kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF` **`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`** (26.28 GiB, ftype 115 ≈ 8.39 bpw: 353 F32 + 187 Q8_0 + 311 `GGML_TYPE_Q8_0_ROCMFPX` tensors — only this fork can read them), **MTP spec decode from the repo's own `mtp-Qwen3.8-27B-Q4_0.gguf` (n-max 4, pinned to ROCm0)**, **1 session × 262,144 (native window)** with **f16 KV**, the bundled BF16 mmproj for vision, thinking on with the card's recipe (temp 1.0 / top_p 0.95 / top_k 20). Measured 2026-10-02: decode **17.0–17.1 t/s** with MTP vs **7.6 without (+123 %)**, draft acceptance 0.462, prefill 325–351 t/s, prompt-cache replay 0.5 s, boot→healthy 16 s, GTT 64.5 GiB. Cut over 2026-10-02 from gufo; rollback tag `frank-large-gufo-q4kxl-20261001` | 7770 |
+| Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** (9B dense qwen3_5 hybrid, 6.4 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **3 × 180,224** (pool 540,672 — NOT the native window; three contexts cost 81,920 tokens per conversation) with **asymmetric KV f16 K + q8_0 V** under flash attention, 21,684 used / **2,441 free** of 24,576 MiB, 106.8 tok/s decode, 3,814 t/s prefill. Resized twice 2026-10-02 (Q6_K → Q5_K_M → 3 slots + vision → this pool); rollback `models/MiMo-Q6K/` or `PARALLEL=2 CTX_SIZE=524288` | 7771 |

 Retired endpoints (`frank-npu`, and every prior runtime) are in
 the [runbook index](#runbook-index). The full swap history lives in
@@ -209,14 +209,14 @@ archive.
 **STOP the production service BEFORE benchmarking** — these servers serve
 real users, and a bench beside a live service starves the GPU and corrupts
 results. Use the runtime's own harness: `llama-bench` for the llama.cpp
-endpoints, `gufo bench` for frank-large. Rolling your own bench
+endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-large and frank-small were measured. Rolling your own bench
 requires asking first.
 Scratch ports are not used on LLM hosts — a
 test process cannot share the single GPU anyway; validate against the service
 itself.

-**`gufo bench` cannot rank speculative-decode settings, and will invert the
-ranking.** Measured 2026-09-28 on frank-large at depth 32768: its corpus
+(A gufo-era law; frank-large is ROCmFPX llama.cpp since 2026-10-02, but the rollback tag keeps it live.) **`gufo bench` cannot rank speculative-decode settings, and will invert the ranking.**
+Measured 2026-09-28 on frank-large at depth 32768: its corpus
 reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
 draft widths, i.e. the same highly-predictable text every run, and 93 % of
 every proposal accepted. It reported 29–37 t/s decode where real traffic
@@ -241,9 +241,9 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
 merely wedged kernel usually still answers ARP.

 1. Stop and mask (self-healing would restart it):
-   - Dankclaw `frank-large` (root unit, gufo on the iGPU):
+   - Dankclaw `frank-large` (root unit, vLLM podman on the iGPU):
     `sudo systemctl stop frank-large && sudo systemctl mask frank-large`
-   - Dankclaw `frank-small` (root unit, SGLang on the 3090):
+   - Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
     `sudo systemctl stop frank-small && sudo systemctl mask frank-small`
   - Dankserver `frank-primary` (root unit, tcclaviger vLLM on the 2x R9700):
     `sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
@@ -277,12 +277,15 @@ When renaming a service, retiring an endpoint, or moving a model: update the
 provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
 `models.yml`, then fix every `modelRoles` entry that named the old endpoint —
 a role pointing at a provider the file does not define fails at call time. As
-of 2026-10-01 both `frank-small` and `frank-primary` accept images
-(`input: [text, image]`) — frank-primary loads the BF16 mmproj explicitly
-(this build never autoloads projectors), and frank-small's MiMo payload
-bakes its vision tower INTO the checkpoint (no projector file at all);
-`frank-large` remains text-only because its gufo deployment loads no
-vision projector. Note also that
+of 2026-10-02 **all three endpoints accept images** (`input: [text, image]`):
+`frank-large` through the BF16 mmproj that ships inside its GGUF payload,
+`frank-primary` through a BF16 mmproj loaded explicitly (this build never
+autoloads projectors), and `frank-small` through
+`mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf` passed as `--mmproj` — the Q5_K_M
+GGUF itself carries no vision tensors. `frank-small`'s `contextWindow` is
+**180224**, the per-slot window, not the model's native 262144: the harness
+must compact before llama.cpp answers a longer prompt with a 400. Note also
+that
 the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
 parser, live-verified) emit `reasoning_content`, vLLM emits `reasoning` —
 models.yml sets `reasoningContentField` per endpoint to match.
@@ -292,161 +295,129 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
 audio encoder, and left again with the Gemma4 retirement on 2026-10-01
 (the MiMo tower is vision-only).

-### frank-large — gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (Dankclaw, ACTIVE)
+### frank-large — ROCmFPX llama.cpp, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (Dankclaw, ACTIVE)

-**gufo** (`gufo-org/gufo`) v0.1.1 — "the Strix Halo inference engine", a
-vertical engine built for exactly this hardware (Ryzen AI MAX+ 395 /
-Radeon 8060S, `gfx1151`) — as a **root systemd unit**
-(`scripts/frank-large.service`, `User=dank`), serving
-`ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`'s `UD-Q8_K_XL`
-(27B dense, 29.3 GiB, sha256-verified; **abliterated/uncensored**, and under
-the *non-OSI* SWIFT OPEN LICENSE 1.0) on port 7770 from the gfx1151 iGPU.
-One session, full 262,144 ctx (the model's native window, `SESSIONS=1`),
-DFlash2 spec decode (the z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` drafter, gufo's
-**adaptive** block policy, `--draft-tokens 7`), greedy sampling, reasoning
-extracted to `reasoning_content`, and **no vision projector**.
+The **kingjones30/ROCmFPX llama.cpp fork** at commit
+`36e9acd40e10a87cd3c3ef8ec734668757dc8520`, built for gfx1151 by
+`scripts/build-rocmfpx.sh` into `/home/dank/GitPackages/kingjones-ROCmFPX/build`,
+as a **root systemd unit** (`scripts/frank-large.service`, `User=dank`,
+`Restart=always`) running a bare `llama-server` (no container) on port 7770
+from the Strix Halo iGPU. Boot to healthy: **16 s**.

-**Two swaps landed on 2026-09-28**, both same day: the engine (from the
-unsloth llama.cpp b11030 prebuilt, which had replaced Kairic Edge ROCmFPX)
-and then the payload (from `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL`, still on
-disk and the payload-only rollback). The two targets are structurally
-interchangeable for gufo — same `qwen35` architecture, the same 866 tensors
-in the same order, the same per-tensor Q8_0/F32/BF16 layout, the same MTP
-head at `blk.64` — verified by parsing and diffing both GGUF headers before
-either was loaded. **The DFlash2 drafter did not move**, so it has its own
-`DRAFTER_DIR` and the deployer's stray-run sweep keys on the drafter's
-filename, not the target's.
+**The runtime is not interchangeable — it is chosen by the FILE.**
+`kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF`'s best rung,
+`Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` (28,214,425,952 B, sha256 `f3ef2bda…`,
+ftype 115 ≈ 8.39 bpw), stores **311 tensors of type 103,
+`GGML_TYPE_Q8_0_ROCMFPX`** — a quantization that exists in no other loader:
+stock llama.cpp and gufo both refuse it at load (`unknown tensor type`). The
+file is 353 F32 + 187 plain-Q8_0 + 311 ROCmFPX; the AGENT routing variant
+keeps the output-head tensors at plain Q8_0 (187 vs 2 in the non-AGENT file),
+which is why the card reports MTP acceptance 0.953 vs 0.911 — highest quality
+AND the better drafter base. Vision comes from the bundled
+`mmproj-Qwen3.8-27B-BF16.gguf` (931,145,888 B).

-**The runtime is built from source on the host** —
-`/home/dank/GitPackages/gufo-v0.1.1/`, reproducible with
-`scripts/build-gufo.sh` (`--verify` re-checks an installed tree). gufo ships
-no prebuilt Linux binaries and the v0.1.0/v0.1.1 release assets are empty.
-Unlike the retired llama.cpp prebuilt, which bundled its own ROCm, **gufo
-links the host ROCm 7.2.4 through its RPATH**, so the HIP dev packages must
-stay installed. Two host workarounds the build script encodes:
+**The pin is load-bearing.** The 2026-08-25 ROCmFPX builds segfaulted in
+`ggml_compute_forward_set_rows` on exactly this file; `build-rocmfpx.sh`
+checks the commit out by SHA, so a fork-side force-push cannot silently
+re-break the endpoint.

- **Ubuntu's `librocwmma-dev` must be absent.** It installs a truncated
-  rocWMMA header set into `/usr/include` that shadows AMD's complete one
-  and breaks the build with `'internal/accessors.hpp' file not found`. A
-  `-I` cannot fix it — in HIP mode the host pass searches the system
-  directories first. The package has no reverse dependencies.
- **`--gcc-install-dir` must name GCC 15.** Ubuntu 26.04 ships
-  `/usr/include/c++/16`, which the ROCm clang prefers; its `<atomic_wait.h>`
-  needs a `std::__detail::__notify_impl` symbol version the installed
-  `libstdc++.so.6` (GLIBCXX_3.4.35) does not export, and the link fails.
+**Spec decode is the performance feature, and it is measured.** The draft is
+the repo's own `mtp-Qwen3.8-27B-Q4_0.gguf` (1,680,271,648 B, sha256 `051a1764…`,
+`nextn_predict_layers = 1`), pinned to the same card:
+`--spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-n-min 0
+--spec-draft-p-min 0.0 --spec-draft-device ROCm0 --spec-draft-ngl 99`.
+Measured 2026-10-02 in one session, temp 0, 2,202-token prompt / 512 tokens,
+`cache_prompt: false`, production service stopped for each arm:
+**MTP on 17.00–17.14 t/s decode** (draft acceptance 0.462, mean acceptance
+length ≈ 1.86) against **7.60–7.66 t/s with the draft removed — +123 %**;
+prefill pays 8–15 % (381 → 325–351 t/s). Acceptance is far from the card's
+0.95 (that number is the card's own text at a shallow prompt); the gain is
+real and n-max 4 is where it was measured, never swept. This replaces the
+gufo Q4_K_XL arm at ~15 t/s, so the endpoint got higher quality AND more
+speed at once.

-`gufo diagnose` is the preflight and must report `[PASS]` with `gfx1151`.
-It also self-caps compute queues (`GPU_MAX_HW_QUEUES`, 2 resident for
-`serve llm`) after reading `/sys/class/kfd/kfd/proc/*/queues/*/type`, which
-matters because ComfyUI holds queues on the same device; four HIP processes
-is the device ceiling.
+**Shape and budget:** ONE session at the native 262,144 window, unquantized
+f16 KV (~64 KiB/token → ~16 GiB, affordable inside the iGPU's ~114 GiB GTT
+aperture), `--fit off --flash-attn on --direct-io --ctx-checkpoints 16
+--cache-ram 8192 --cache-idle-slots`. Measured GTT 64.5 GiB with model +
+draft + projector resident. Prompt cache verified: a repeat pass over the
+same 6k prompt replays in 0.5 s.

-**The client contract changed, and the chat template is the sharp edge.**
-**gufo has no Jinja engine at all** — `src/models/qwen/chat_template.cpp` is
-587 lines of hand-written C++ that appends strings, with no
-`--chat-template-file`, no env override, and no template body to point at.
-`scripts/qwen-sharp-v22.4.0.jinja` therefore **cannot** be supplied to
-frank-large: that is a port, not a config line. Given the choice on
-2026-09-28 the decision was to ship on gufo's stock renderer and accept the
-loss — the `terse` system-prompt injection, the in-band `<|think_*|>`
-control, the `json|xml` `tool_call_format` switch, the tool-argument
-truncation and `add_vision_id` prefixes are all gone. gufo's own renderer
-does keep the effort instruction strings and `preserve_thinking`, which are
-what the harness depends on. **This is the thing to watch**: the endpoint
-now runs an *abliterated* model without the prompt that tells it to stop
-waffling. The cheap fix, if verbosity shows up, is to send the `terse` text
-as an ordinary system message from `models.yml` — not a template change.
-`reasoning_effort` accepts the full OpenAI set (all six levels verified
-200), so `models.yml`'s `reasoningEffortMap` fold is no longer *required* —
-it is kept deliberately, so that removing it does not silently move every
-`high` role from `xhigh` to `high` effort inside a swap. And there is no
-KV-quantization or prompt-cache-RAM flag; prompt reuse is on by default and
-`--cache-disk` is opt-in and off.
+**Client contract (llama.cpp, live-verified 2026-10-02):** `--jinja` is on
+(without it `chat_template_kwargs` are silently ignored), tools are native
+(`get_weather` → `{"city":"Seattle"}`), thinking lands in
+**`reasoning_content`**, `reasoning_effort` accepts ONLY
+`xhigh|medium|low` (anything else is HTTP 500 — `models.yml` folds `high` and
+`max` to `xhigh`), and thinking is enabled server-side with the card's
+recipe (temp 1.0 / top_p 0.95 / top_k 20 / min_p 0) plus
+`--reasoning-format deepseek --reasoning-budget -1`. Two traps: a request
+whose whole budget is consumed by reasoning returns `content: null` (bench
+clients must fall back to `reasoning_content` or they crash *after*
+measuring), and the fork logs
+`MTP boundary missing for seq_id=… ; resyncing` under concurrent draft
+resets — self-recovering, not a fault.

-**Read the perf trade before assuming a win.** gufo's own Q8_K_XL tables
-(vs llama.cpp b11069, DFlash2) show prefill +37 % to +80 %, cold load 4.5×
-faster, decode +7 % at zero depth and +40 % at 4 users — but **−14 % at
-65,536 tokens of depth and −24 % at 131,072**. On the 8-bit target the
-crossover is ~32k of cached context, and frank-large runs one 262,144-token
-conversation, so long agent sessions sit on the losing side of that line.
-The Q4_K_XL target shows no such regression; revisiting the quantization is
-a separate decision from the runtime. Peak memory is 50.58 GiB at the full
-262,144 context against the iGPU's 114 GiB GTT aperture.
+Fetch: `scripts/download-frank-large-kingjones.sh` (`-f` + byte count + LFS
+sha256 verified, `--verify-only`, no token on the command line).

-The serve script (`scripts/serve-frank-large.sh`) validates runtime → target
-GGUF → drafter and exits with a pointer message if any is missing, then
-`exec`s `gufo serve --host 0.0.0.0 --port 7770 --sessions 1 llm --model
-…--served-model-name frank-large --context 262144 --speculative dflash2 …`.
-The deployer's `deploy_llm_large` gates on the **runtime binary** as well
-as both payload paths, and returns before it stops anything.
+Runbook: `serverAppConfigs-docs/LLM/frank-large-rocmfpx-q8agent-mtp.md`.
+Rollback: git tag **`frank-large-gufo-q4kxl-20261001`** — gufo v0.1.1
+(`scripts/build-gufo.sh` stays on disk for it), the Q4_K_XL Qwen3.8-27B
+payload and the DFlash2 drafter all remain on the host; restore the tag's
+unit/env/serve trio and redeploy.

-Fetch: `scripts/download-frank-large-swift.sh` (the target; per-file byte
-count + sha256 against the Hub LFS oid, no auth header — the repo is public
-and a token on the curl command line is world-readable) and
-`scripts/download-frank-large.sh` (the DFlash2 drafter, which did not move).
-Both take `--verify-only`.
+### frank-small — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)

-Runbook: `serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md`.
-Rollback, two ways: **payload only** — re-fetch with
-`scripts/download-frank-large.sh` and repoint `MODEL_FILE` at
-`/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-UD-Q8_K_XL.gguf`; that
-29.3 GiB target was **reclaimed on 2026-09-28** (staged to
-`~/.AGENTTRASH/20260928-retired-models/`, restorable from there), so it is
-no longer a one-minute flip. **Engine** —
-[[frank-large-unsloth-qwen38-27b-q8xl]], the b11030 prebuilt at
-`/home/dank/GitPackages/unsloth-llama-b11030/`; the only way to get
-Qwen-Sharp back. The DFlash2 drafter was never reclaimed and still lives in
-`Qwen3.8-27B-UD-Q8_K_XL/`.
-
-### frank-small — SGLang on the RTX 3090 (Dankclaw, ACTIVE)
-
-SGLang (latest) in a **self-contained python3.12 venv**
-(`/home/dank/GitPackages/sglang-cu128/`, torch 2.13.0+cu130) as a **root
-systemd unit** (repo `scripts/frank-small.service`, `relink_unit`ed into
+**llama.cpp master** (CUDA sm_86 build
+`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`) as a
+**root systemd unit** (`scripts/frank-small.service`, `relink_unit`ed into
 `/etc/systemd/system/`), serving
-`letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound` (9B dense
-qwen3_5 hybrid — 8 full-attention + 24 Gated DeltaNet layers — INT8 W8A16
-AutoRound, 12.0 GiB, vision tower baked into the checkpoint). Port 7771.
+`bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF` at **Q5_K_M** (6,876,124,704 B,
+sha256 `04cf4f…`) **plus `mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf`**
+(918,166,048 B, sha256 `ff348f…`), both `/home/dank/models/MiMo-Q5KM/`, on
+port 7771. The 2026-10-02 swap was a **quant rung down from Q6_K**, not a
+runtime change — the llama.cpp path arrived 2026-10-01 night for one reason:
+**asymmetric KV**. `--cache-type-k f16` + `--cache-type-v q8_0` is the fleet's
+only per-tensor KV lever; SGLang's `--kv-cache-dtype` moves K and V together.
+Vision came the same day via `--mmproj`: the GGUF carries no `v.*` tensors, so
+the projector file is the entire vision path (and ~1.4 GiB of the card).

-**The shape is memory-pinned, and the sharp edge is the mamba pool.** The
-DeltaNet ssm state pool autofills to a fixed ~5.1 GB budget *regardless of
-dtype* (fp32 at 110 slots, bf16 at 217 — same 5.1 GB), so both dials are
-pinned: `--mamba-ssm-dtype bfloat16 --max-mamba-cache-size 32` (795 MiB).
-The KV pool is then **pinned at 606,208 tokens** (`--max-total-tokens`) —
-the engine's autofit picks 687,388 and leaves 0.53 GB, which Triton's
-deferred kernel loads eat and the first long request OOMs on. Two
-262,144-token slots at the model's NATIVE window (2026-10-01 reslice from
-4 x 151,552: 2 x 262,144 = 524,288 leaves ~82k tokens of pool slack, and
-a compaction request — frank-primary's trigger is 183,501 tokens plus a
-32,768-token summary budget = 216,269 — now fits ONE request); a single
-request may grow to the whole pool. Prefill runs **eager**
-(`--disable-prefill-cuda-graph`) — there is no headroom for capture;
-decode graphs are on.
+**Architecture: qwen3_5 hybrid** — 32 layers, 8 full-attention + 24 Gated
+DeltaNet, native 262,144 window. KV exists only in the 8 attention layers
+(4 KV heads × 256 dim × 2 B ≈ 2 KiB/token/layer, ~16 KiB/token for f16 K+V);
+the DeltaNet state is a fixed per-slot buffer, so slot count — not context —
+drives that term.

-**No speculative decode — a checkpoint fact.** `text_config` declares
-`mtp_num_hidden_layers: 1`, but the shipped safetensors (base BF16 and this
-INT8 conversion alike) carry zero `mtp.*` tensors; NEXTN would draft with
-an uninitialised 4.24 GiB ghost model at 0% acceptance. The seam gates
-that no `--speculative-*` flag reaches the engine.
+**The shape:** `--ctx-size 540672 --parallel 3` = **THREE conversations of
+180,224 tokens each** (llama.cpp divides the pool:
+`n_ctx_slot = CTX_SIZE / PARALLEL`), `--flash-attn on` (the q8_0-V path is
+only cheap under flash attention — the seam pins it), f16 K + q8_0 V, all
+layers on the GPU, `--mmproj` for vision. Measured VRAM **21,684 used / 2,441
+free** of 24,576 MiB — the 2–3 GiB headroom Frank asked for, landed by
+solving the pool against two measured boots: non-KV cost is 7,390 MiB without
+a projector (2 × 262,144 boot) and 8,859 MiB with one (3 × 196,608 boot), so
+vision costs ~1.4 GiB, not the 0.9 GiB the file size suggests. Three native
+windows (786,432 tokens) need 18.4 GiB of KV alone and were never on the
+table. Full f16 V would add ~4 GiB more and bust the card — that is the
+justification for the asymmetric contract.

-**The venv carries a CUDA 13.0 nvcc extracted from debs** (system nvcc is
-12.4) so the sgl-kernel/flashinfer JIT compiles for sm86; three local
-patches ride in the tree (GCC-16 shim, CCCL symlink, rsqrt noexcept).
-Details in the runbook.
+Verified at deploy (2026-10-02): `/v1/models` reports `frank-small`; boot logs
+`n_slots = 3, n_ctx_slot = 180224` and `loaded multimodal model`; decode
+**106.8 tok/s** at temp 0 (prefill 3,814 t/s) — ~5 % faster than the
+2 × 262,144 shape, because a shorter slot means a shorter attention scan;
+three concurrent requests answered in parallel and a fourth queued behind a
+released slot; native `tool_calls`; thinking in `reasoning_content`; a
+solid-red PNG answers "Red." through an `image_url` content part (the legacy
+top-level `images` field is ignored by `/v1/chat/completions` — the model then
+invents a color). A prompt over the slot is a **400
+`exceed_context_size_error`**, which is why `models.yml` sets
+`contextWindow: 180224`.

-Verified at deploy (2026-10-01): `/v1/models` reports `frank-small`;
-chat returns content plus `reasoning_content` (SGLang's qwen3 parser uses
-llama.cpp's field name); a red-square image answers "Red"
-(`image_tokens: 64`). At the 4 x 151,552 shape: 4 concurrent 54k-token
-requests answered; a 170,019-token prompt prefilled in 50.5 s (~3,370 t/s
-eager). At the 2 x 262,144 shape (reslice, same day): boot flags read
-`context_len=262144 max_running_requests=2 max_total_num_tokens=606208`;
-a 200,000-token prompt — well past frank-primary's 183,501 compaction
-trigger — prefilled and completed in ONE request (97.3 s wall, ~2,056
-tok/s eager); two concurrent 50k-token requests both answered. Decode
-**83.9 tok/s**.
-Runbook: `serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md`.
-Rollback: [[frank-small-gemma4-12b]] (payload and binary intact on disk).
+Runbook: `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q5km.md`.
+Rollback: [[frank-small-llama-mimo-q6k]] (Q6_K file on disk at
+`/home/dank/models/MiMo-Q6K/`); before that [[frank-small-sglang-mimo-v26]]
+(venv on disk at `/home/dank/GitPackages/sglang-cu128/`; its INT8 payload was
+staged to `~dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02).

 ### frank-primary — tcclaviger vLLM, Qwen3.8-Flash-Next FP8 + MTP4 (Dankserver, ACTIVE)

@@ -655,6 +626,26 @@ linked runbooks.
  non-greedy requests and reasoning budgets. Re-test chat before production.
 - **FastMTP sidecar**: build must be commit `4df29be4` + the HauhauCS patch;
  stock binaries fail the draft load with a tensor-shape mismatch.
+- **A GGUF's custom tensor type picks the runtime — read the histogram before
+  promising a file to a loader.** `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is
+  ftype 115 with 311 tensors of type 103 (`GGML_TYPE_Q8_0_ROCMFPX`); stock
+  llama.cpp and gufo refuse it at load, so ROCmFPX was not a preference but
+  the only door. Counting tensor types out of the GGUF KV section takes one
+  python read and settles a runtime argument in minutes. The build identity
+  matters as much as the fork: the 2026-08-25 ROCmFPX builds SIGSEGV in
+  `ggml_compute_forward_set_rows` on this same file, so
+  `scripts/build-rocmfpx.sh` pins commit `36e9acd` by SHA.
+- **Speculative-decode flags move as a SET.** Handing the ROCmFPX fork
+  `--model-draft` with no `--spec-type` (the naive way to build a "no MTP"
+  bench arm) segfaulted at load, core dumped, no diagnostic (2026-10-02). A
+  baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND
+  `--spec-type`, or it measures a crash instead of the absence of drafting.
+- **An orphaned scratch server taxes the whole host.** A bench child left
+  alive by a client-side exception kept ~26 GiB of weights resident on a
+  121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
+  Record the child pid and kill it in a `finally`, then prove `free` came back
+  before believing a number. Kill by pid — never `pkill -f llama-server`,
+  which also hits frank-small.

 **llama.cpp serving**

@@ -727,6 +718,35 @@ linked runbooks.
  host-RAM hash table) and its window decides which traffic it can see: too
  short and it proposes on ordinary prose fragments, which costs verification
  without acceptance.
+- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the
+  per-slot number is the contract — not `ctx-size`.** `n_ctx_slot =
+  CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
+  (`400 exceed_context_size_error: request (184001 tokens) exceeds the
+  available context size (180224 tokens)`) even when the pool has room. So
+  frank-small's 2 → 3 reslice moved each conversation from the model's native
+  262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
+  the harness would discover the ceiling as a 400 on a long conversation. Side
+  effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
+  100.4 tok/s) — less attention to scan and one fewer DeltaNet state resident.
+  Measured 2026-10-02. See [[frank-small-llama-mimo-q5km]].
+- **Price a VRAM budget from two boots, never from a formula — and an mmproj
+  costs ~1.5x its file size.** The frank-small 3090 fit was solved from two
+  measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
+  **non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
+  14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
+  that bills ~1.4 GiB, because the encoder's activation and compute buffers
+  ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
+  "a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
+  solve `KV ≤ card − non-KV − headroom` and round to a multiple of
+  `PARALLEL × 4096`. Recorded in `scripts/frank-small.env` so the next resize
+  starts from measurement.
+- **`images` and `image_url` are NOT interchangeable: only the content part
+  reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
+  model still invents an answer if the client used the legacy top-level
+  `images` field on `/v1/chat/completions` — frank-small answered "Brown." for
+  a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
+  failure is silent because a hallucinated color is a plausible answer. Probe
+  vision with a solid color and expect an exact word.
 - **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
  `500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
  then served frank-large: every tool-using agent on that endpoint was broken
@@ -827,8 +847,8 @@ Active:

 | Service | Runbook |
 |---------|---------|
-| frank-large (gufo Qwen3.8-27B UD-Q8_K_XL, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-gufo-qwen38-27b.md` |
-| frank-small (SGLang MiMo-V2.6 INT8, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-sglang-mimo-v26.md` |
+| frank-large (ROCmFPX llama.cpp fork, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP4 + vision, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-rocmfpx-q8agent-mtp.md` |
+| frank-small (llama.cpp CUDA, MiMo-V2.6-Distill-Qwen-9B Q5_K_M, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q5km.md` |
 | frank-primary (tcclaviger vLLM, Qwen3.8-Flash-Next FP8 + MTP4, Dankserver) | `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -837,6 +857,10 @@ Retired (rollback targets and history):

 | Runtime | Host | Window | Runbook |
 |---------|------|--------|---------|
+| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (rollback target) | `frank-large-gufo-qwen38-27b.md` |
+| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
+| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-small`) | Dankclaw | 2026-10-01 → 10-02 (rollback target) | `frank-small-llama-mimo-q6k.md` |
+| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-small`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-small-sglang-mimo-v26.md` |
 |tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-primary`)|Dankserver|2026-09-30 → 10-01|`frank-primary-glm53-flash-w4a16.md`|
 | llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-21 → 09-28 | `frank-large-unsloth-qwen38-27b-q8xl.md` |
 | R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-primary`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-primary-r9v-radiance-qwen38-uncensored.md` |
@@ -0,0 +1,104 @@
+#!/usr/bin/env bash
+# autoresearch.sh — canonical benchmark entrypoint: speed of frank-primary.
+#
+# Runs on the operator box and drives one bench window on DANKSERVER:
+#   1. fingerprint the candidate trio (env + serve script + unit)
+#   2. memo fast-path: an identical fingerprint that already produced a FULLY
+#      successful window (bench + verified production restore) returns the
+#      cached metrics without touching the fleet
+#   3. stage the candidate trio plus the measure client into
+#      /home/dank/.autoresearch-frank-primary/ (ephemeral; the repo checkout
+#      on the host stays the production source of truth)
+#   4. run the window (scripts/autoresearch/frank-primary-window.sh):
+#      stop + mask the production unit, boot the staged config AS THE SERVICE
+#      (same serve script, same container, same port 7770), wait for health,
+#      run the in-container usage-counter measure, then restore the production
+#      unit and verify its health — AGENTS.md Benchmarking Policy
+#   5. print METRIC lines (parse by run_experiment)
+#
+# Primary metric: decode_tps (HIGHER is better) — median across the three
+# prompt depths (115/1056/4096) of per-depth medians of
+# completion_tokens / (wall - ttft), 3 runs each after a warm pass,
+# temperature 0, the campaign's deterministic word-seed workload.
+#
+# Exit 0 with METRIC lines on success; non-zero on any failure (the window
+# script's EXIT trap restores production on every path).
+set -euo pipefail
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+HOST="${FRANK_PRIMARY_AR_HOST:-dankserver}"
+STAGE="/home/dank/.autoresearch-frank-primary"
+
+ENV_FILE="$SCRIPT_DIR/scripts/frank-primary.env"
+SERVE="$SCRIPT_DIR/scripts/serve-frank-primary.sh"
+UNIT="$SCRIPT_DIR/scripts/frank-primary.service"
+MEASURE="$SCRIPT_DIR/scripts/autoresearch/frank-primary-measure.py"
+WINDOW="$SCRIPT_DIR/scripts/autoresearch/frank-primary-window.sh"
+
+for f in "$ENV_FILE" "$SERVE" "$UNIT" "$MEASURE" "$WINDOW"; do
+    [ -f "$f" ] || { echo "[autoresearch] missing harness file: $f" >&2; exit 1; }
+done
+command -v python3 >/dev/null 2>&1 || { echo "[autoresearch] python3 required on operator box" >&2; exit 1; }
+
+# Fingerprint of everything that can change the engine's argv or environment.
+HASH="$(cat "$ENV_FILE" "$SERVE" "$UNIT" | sha256sum | cut -d' ' -f1)"
+echo "[autoresearch] candidate fingerprint $HASH" >&2
+
+ssh_run() { ssh -o BatchMode=yes -o ConnectTimeout=15 "$HOST" "$@"; }
+
+emit_metrics() { # $1 = run.json
+    python3 -c '
+import json, sys
+run = json.loads(sys.argv[1])
+m = run["measure"]
+print("METRIC decode_tps=%s" % m["decode_tps"])
+for k in ("decode_tps_115", "decode_tps_1056", "decode_tps_4096"):
+    print("METRIC %s=%s" % (k, m[k]))
+print("METRIC ttft_ms_4096=%s" % m["ttft_ms_4096"])
+print("METRIC kv_tokens=%s" % (run.get("kv_tokens") or 0))
+print("METRIC boot_seconds=%s" % (run.get("boot_seconds") or -1))
+print("METRIC completion_tokens_4096=%s" % m.get("completion_tokens_4096", 0))
+' "$1"
+}
+
+# --- memo fast-path -------------------------------------------------------
+cached_hash="$(ssh_run "cat '$STAGE/last/hash' 2>/dev/null" 2>/dev/null || true)"
+if [ -n "$cached_hash" ] && [ "$cached_hash" = "$HASH" ]; then
+    cached_run="$(ssh_run "cat '$STAGE/last/run.json' 2>/dev/null" || true)"
+    if [ -n "$cached_run" ]; then
+        echo "[autoresearch] memo hit — cached metrics for $HASH, fleet untouched" >&2
+        emit_metrics "$cached_run"
+        exit 0
+    fi
+fi
+
+# --- stage the candidate ----------------------------------------------------
+echo "[autoresearch] staging candidate on $HOST:$STAGE" >&2
+ssh_run "mkdir -p '$STAGE'"
+scp -q "$ENV_FILE" "$HOST:$STAGE/frank-primary.env"
+scp -q "$SERVE" "$HOST:$STAGE/serve.sh"
+scp -q "$MEASURE" "$HOST:$STAGE/measure.py"
+scp -q "$WINDOW" "$HOST:$STAGE/window.sh"
+ssh_run "printf '%s\n' '$HASH' > '$STAGE/hash'"
+
+# --- the window -------------------------------------------------------------
+# Budgets: bench boot up to 20 min (warm boots measured ~7 min), measure up
+# to 30 min, restore boot up to 20 min — call run_experiment with
+# timeout_seconds >= 3600.
+echo "[autoresearch] opening bench window (stop+mask -> boot -> measure -> restore)" >&2
+set +e
+ssh_run "bash '$STAGE/window.sh'"
+rc=$?
+set -e
+if [ "$rc" -ne 0 ]; then
+    echo "[autoresearch] window failed (rc=$rc); host-side evidence:" >&2
+    ssh_run "tail -60 '$STAGE/fail.log' 2>/dev/null || tail -40 '$STAGE/serve.out' 2>/dev/null || tail -40 '$STAGE/measure.out' 2>/dev/null || true" >&2 || true
+    exit 1
+fi
+
+run_json="$(ssh_run "cat '$STAGE/run.json'")" || {
+    echo "[autoresearch] could not read $STAGE/run.json back" >&2; exit 1; }
+echo "[autoresearch] window result:" >&2
+printf '%s\n' "$run_json" >&2
+emit_metrics "$run_json"
+exit 0
@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-large's DFlash
+# speculative-decode arm (vLLM --speculative-config method=dflash). Public
+# repo — no auth header on the command line.
+#
+# The drafter is a 6-layer qwen3 draft whose dflash_config pins
+# target_layer_ids [1,5,9,13,17,21,25,29] and block_size 16; its hidden_size
+# 4096 / vocab_size 248320 match the MiMo-V2.6-Distill-Qwen-9B target exactly
+# (both preserve the Qwen3.5-9B geometry), which is what lets vLLM load it as
+# the draft for that target.
+#
+# Usage:
+#   scripts/download-frank-large-dflash.sh [DEST_DIR] [--verify-only]
+set -euo pipefail
+
+REPO="z-lab/Qwen3.5-9B-DFlash"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/Qwen3.5-9B-DFlash}"
+
+# name bytes sha256(- = non-LFS, size-only check)
+FILES=(
+  "model.safetensors 2583816465 0a42274b32554f48de1faa0d42824e9c2ceda649c30ae0a731cddf410dd698c7"
+  "config.json 1222 -"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"
@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-large
+# (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the
+# command line (a Bearer token in argv is world-readable via /proc).
+#
+# Per-file FRESH downloads, no resume: the house rule (see
+# download-frank-small.sh); -f/--fail-with-body plus size+sha256 against the
+# Hub LFS oid is what makes an access error fail instead of "succeeding".
+#
+# Usage:
+#   scripts/download-frank-large-mimo.sh [DEST_DIR] [--verify-only]
+#
+# The verified set below is exactly what the 2026-10-01 bench window served:
+# 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest).
+set -euo pipefail
+
+REPO="XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-BF16}"
+
+# name bytes sha256(- = non-LFS, size-only check)
+FILES=(
+  "model-00001-of-00004.safetensors 5276436216 aab052180118aee34abc3029b54eaa49096aac606b97d703866b420dceb703c3"
+  "model-00002-of-00004.safetensors 5033171280 7a0486565f06d25ac4628e9dba470dc3f604353471d240d5a0bf7128f64df396"
+  "model-00003-of-00004.safetensors 5234499504 6c73207563d1879bfd6c143a028cc70be68edff56280458c71a43df4240300f4"
+  "model-00004-of-00004.safetensors 3275613848 1379a7cf8c0b8555a39ab65a47e830e0eb45e776b46045734da3afd73c09eea2"
+  "model.safetensors.index.json 69217 -"
+  "config.json 2784 -"
+  "generation_config.json 126 -"
+  "chat_template.jinja 3916 -"
+  "tokenizer.json 19989325 06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523"
+  "tokenizer_config.json 1165 -"
+  "vocab.json 6722759 -"
+  "merges.txt 3353259 -"
+  "preprocessor_config.json 443 -"
+  "processor_config.json 1191 -"
+  "video_preprocessor_config.json 385 -"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"
@@ -0,0 +1,85 @@
+#!/usr/bin/env bash
+# Download the source Qwen3.5-9B checkpoint — the ONLY place this model
+# family's MTP head (15 mtp.* tensors) exists. frank-large runs it as the
+# vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero
+# mtp tensors; that is why SGLang's NEXTN on frank-small draws 0.00
+# acceptance). Public repo — no auth header on the command line.
+#
+# NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...).
+#
+# Usage:
+#   scripts/download-frank-large-qwen35.sh [DEST_DIR] [--verify-only]
+set -euo pipefail
+
+REPO="Qwen/Qwen3.5-9B"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/Qwen3.5-9B}"
+
+# name bytes sha256(- = non-LFS, size-only check)
+FILES=(
+  "model.safetensors-00001-of-00004.safetensors 5276436216 db6f444b43d318c92f360a13a25561a6a65b10c0631b8ed305a426dbaa6c380e"
+  "model.safetensors-00002-of-00004.safetensors 5335161512 31c7d7e2dd5d207840b31cc59083c8f4c4718959149e0358c0364052bb9a0330"
+  "model.safetensors-00003-of-00004.safetensors 5368717440 7ec36ba3a4176a44c3c0876ad80c56a2f70c84bf008d82e9501df642f17dadec"
+  "model.safetensors-00004-of-00004.safetensors 3325995712 b62b0c4cd7e44edee103ee8f4fe225f246d5e768e07bfd5f25b63a8aa1fdd0c6"
+  "model.safetensors.index.json 79657 -"
+  "config.json 3126 -"
+  "chat_template.jinja 7756 -"
+  "tokenizer.json 12807982 5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42"
+  "tokenizer_config.json 16710 -"
+  "vocab.json 6722759 -"
+  "merges.txt 3353259 -"
+  "preprocessor_config.json 390 -"
+  "video_preprocessor_config.json 385 -"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"
@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-small
+# (SGLang, RTX 3090). Public repo — no auth header on the command line.
+#
+# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
+# oversized file on a 31 GiB payload; a fresh -f fetch plus size+sha256 is
+# the house rule.
+#
+# Usage:
+#   scripts/download-frank-small.sh [DEST_DIR] [--verify-only]
+#
+# LFS shards are verified against the Hub LFS sha256; non-LFS text files
+# against their declared byte size. Idempotent: complete files are skipped.
+set -euo pipefail
+
+REPO="letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound}"
+
+# name bytes sha256(- = non-LFS, size-only check)
+FILES=(
+  "model-00001-of-00008.safetensors 2946296312 ddb6117c04cb6feaf7c265e5d75efadf21949037d73253a8e4521d40e164366d"
+  "model-00002-of-00008.safetensors 1402134792 2dc78d55d1d33d7b3b21dd6f8b4b817b62c2fb1376e6448b322428798c10e257"
+  "model-00003-of-00008.safetensors 1427365904 5c797d14e345309d67c78306b9de4aa01a21dbd92dd98e40b0ad69b2900e9203"
+  "model-00004-of-00008.safetensors 1420041856 7402f852e2c7a24ebf4694b98a220e71dfb2c93ba3776272cf2acfc65acd8126"
+  "model-00005-of-00008.safetensors 1410655648 991cbf2eda2ff91fc3ffbb15591d7fcac340fb10d9c33bd3da20010e2f3f4090"
+  "model-00006-of-00008.safetensors 1427627856 efc42448054cfb5e3061e154ba4539204088ce95176d689a95b58de885ef7eeb"
+  "model-00007-of-00008.safetensors 2035442408 75a101fc71ae6f2e1f5c4816f711d62cf1d7a0e02ef6e5182361b8c11e72b626"
+  "model-00008-of-00008.safetensors 822153664 0fb90cb50e2d336fbad58bd4c01880eeddbf537c1dca1cf0ae291195d5efbb4b"
+  "tokenizer.json 19989325 06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523"
+  "model.safetensors.index.json 106278 -"
+  "config.json 4608 -"
+  "generation_config.json 161 -"
+  "quantization_config.json 1654 -"
+  "chat_template.jinja 3916 -"
+  "tokenizer_config.json 1168 -"
+  "preprocessor_config.json 443 -"
+  "processor_config.json 1191 -"
+  "README.md 8448 -"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  # no -C resume; -f + --fail-with-body so an HTTP error is not a success
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"
@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+"""autoresearch-bench.py — deterministic greedy battery for frank-large.
+
+Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770.
+Fixed embedded corpus, temperature 0, streaming so decode speed is measured
+between real token arrivals (not request totals). Median of REPS samples per
+metric is the reported value; medians are far more stable than means under
+GPU thermal drift.
+
+Metrics emitted (stdout, one per line):
+  METRIC decode_tokens_per_sec=<float>   primary, higher better
+  METRIC prefill_tokens_per_sec=<float>  secondary
+  METRIC ttft_ms=<float>                 secondary, lower better
+Exit non-zero if the endpoint is unhealthy or the served alias is wrong.
+"""
+import argparse
+import json
+import statistics
+import sys
+import time
+import urllib.request
+
+# Fixed corpus: deterministic, ~28k tokens after tokenization (same scale as
+# the 2026-10-01 bench window). Built by repeating a fixed paragraph block a
+# fixed number of times - NO random, NO clock, NO network.
+_BLOCK = (
+    "The rendering pipeline walks the scene graph once per frame, resolving "
+    "material bindings into descriptor sets before any draw call is recorded. "
+    "Because the allocator recycles command buffers only after their fence "
+    "signals, a submission that outlives its frame must pin the pool entry; "
+    "otherwise the next frame overwrites recorded commands in flight. The "
+    "validation layer flags exactly this class of lifetime bug, but only for "
+    "buffers it can see - GPU-side timestamps and queue-family ownership "
+    "transfers hide in the driver's own bookkeeping, which is why release "
+    "builds must lean on explicit barriers and why a frame-time regression "
+    "often shows up as corruption one frame later than its cause.\n"
+)
+LONG_CTX = _BLOCK * 220  # ~28k-token prefill, matching the deployed workload
+_LONG = LONG_CTX + "\nNow, in one focused paragraph (80-150 words), explain " \
+    "which lifetime bugs this pipeline can still suffer and what explicit " \
+    "mitigations the text implies."
+_SHORT = "Design a rate limiter for a homelab reverse proxy that must not " \
+    "lock out the LAN. Think it through, then give the algorithm."
+
+REPS = 3
+MAX_TOKENS = 256
+REQ_TIMEOUT = 900
+
+
+def stream_completion(url: str, model: str, prompt: str):
+    """Greedy streaming completion.
+
+    Returns (ttft_s, decode_tps, prefill_tps, prompt_tokens, n_tokens).
+    ttft is RELATIVE to request send: time.perf_counter()'s epoch is host
+    boot, and returning the raw stamp read as an 11-day first token.
+    """
+    body = json.dumps({
+        "model": model,
+        "messages": [{"role": "user", "content": prompt}],
+        "max_tokens": MAX_TOKENS,
+        "temperature": 0,
+        "stream": True,
+        "stream_options": {"include_usage": True},
+    }).encode()
+    req = urllib.request.Request(
+        url.rstrip("/") + "/v1/chat/completions", data=body,
+        headers={"Content-Type": "application/json"})
+    t0 = time.perf_counter()
+    first = None
+    stamps = []
+    usage = {}
+    with urllib.request.urlopen(req, timeout=REQ_TIMEOUT) as resp:
+        for raw in resp:
+            line = raw.decode(errors="replace").strip()
+            if not line.startswith("data:"):
+                continue
+            payload = line[5:].strip()
+            if payload == "[DONE]":
+                break
+            chunk = json.loads(payload)
+            if chunk.get("usage"):
+                usage = chunk["usage"]
+            for ch in chunk.get("choices", []):
+                d = ch.get("delta") or {}
+                # reasoning AND content deltas both arrive token by token;
+                # count either so greedy text length drives the rate.
+                # (llama.cpp names the thinking field reasoning_content.)
+                if d.get("content") or d.get("reasoning") or d.get("reasoning_content"):
+                    now = time.perf_counter()
+                    if first is None:
+                        first = now
+                    stamps.append(now)
+    if first is None or len(stamps) < 2:
+        # A zero-argmax degenerate answer would poison the median; raise.
+        raise RuntimeError(f"stream produced {len(stamps)} tokens")
+    ttft_s = first - t0
+    decode_s = stamps[-1] - stamps[0]
+    decode_tps = (len(stamps) - 1) / decode_s if decode_s > 0 else 0.0
+    prompt_tokens = usage.get("prompt_tokens") or 0
+    prefill_tps = prompt_tokens / ttft_s if prompt_tokens else 0.0
+    return ttft_s, decode_tps, prefill_tps, prompt_tokens, len(stamps)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--url", default="http://127.0.0.1:7770")
+    ap.add_argument("--model", default="frank-large")
+    args = ap.parse_args()
+
+    with urllib.request.urlopen(args.url.rstrip("/") + "/v1/models",
+                                timeout=10) as r:
+        ids = [m["id"] for m in json.load(r)["data"]]
+    if args.model not in ids:
+        print(f"FATAL: served ids {ids}, expected {args.model}",
+              file=sys.stderr)
+        sys.exit(2)
+
+    # Warmup (compile/graph warm + allocator settle), NOT measured.
+    stream_completion(args.url, args.model, _SHORT)
+
+    decodes, prefills, ttfts = [], [], []
+    for rep in range(REPS):
+        for name, prompt in (("long", _LONG), ("short", _SHORT)):
+            ttft, dt, pt, ptk, ntok = stream_completion(
+                args.url, args.model, prompt)
+            decodes.append(dt)
+            ttfts.append(ttft * 1000.0)
+            if name == "long":
+                prefills.append(pt)
+            print(f"  rep{rep} {name:5s} ttft={ttft*1000:8.1f}ms "
+                  f"prefill={pt:8.1f} t/s decode={dt:6.2f} t/s "
+                  f"({ntok} tok, prompt {ptk})", file=sys.stderr)
+
+    med = lambda xs: statistics.median(xs)
+    print(f"METRIC decode_tokens_per_sec={med(decodes):.2f}")
+    print(f"METRIC prefill_tokens_per_sec={med(prefills):.1f}")
+    print(f"METRIC ttft_ms={med(ttfts):.1f}")
+
+
+if __name__ == "__main__":
+    main()
@@ -0,0 +1,192 @@
+#!/usr/bin/env python3
+"""frank-primary decode measurement — runs INSIDE the engine container.
+
+The method is the fleet's established one (the 2026-09-29 / 2026-10-01
+tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid
+under speculative decoding because rejected drafts change the accounting, so
+decode is measured from the server's own usage counters:
+
+    decode_tok_s = completion_tokens / (wall - ttft)
+
+with ttft = wall time to the FIRST content-bearing chunk (thinking arrives in
+`reasoning`/`delta.reasoning` on this stack and is excluded from the decode
+window — and is deterministic per prompt at temperature 0, so the exclusion
+cancels in any A/B on the same workload).
+
+Workload (fixed, deterministic, offline):
+  * prompts built from a large fixed-seed shuffled word list so tokens rarely
+    repeat (no n-gram/prefix predictability of the text itself), sized with
+    the server's own /v1/messages/count_tokens counter
+  * depths 115 / 1056 / 4096 target prompt tokens
+  * one 64-token warm pass per depth (warms the expert arena), then
+    max_tokens=256 at temperature 0, 3 runs, median
+Everything is greedy with a fixed seed — the same workload every run, no
+network, no time-of-day dependence.
+
+Env:
+  AR_BASE             endpoint (default http://127.0.0.1:7770)
+  AR_MODEL            served name (default frank-primary)
+  AR_PROMPT_CEILING   hard prompt ceiling in tokens (default 100000)
+
+Output: progress on stderr, ONE json object on stdout (the last line).
+Exit codes: 0 ok, 2 endpoint unhealthy / no usable completion, 3 bad usage.
+"""
+import json
+import os
+import random
+import statistics
+import sys
+import time
+import urllib.error
+import urllib.request
+
+BASE = os.environ.get("AR_BASE", "http://127.0.0.1:7770")
+MODEL = os.environ.get("AR_MODEL", "frank-primary")
+PROMPT_CEILING = int(os.environ.get("AR_PROMPT_CEILING", "100000"))
+DEPTHS = (115, 1056, 4096)
+WARM_TOKENS = 64
+MAX_TOKENS = 256
+RUNS = 3
+
+# 43-word x 12 = 516-word vocabulary, seed 1234 — the campaign's exact list
+# and seed, so prompts are byte-identical to the historical measurements.
+_VOCAB = ("alpha bravo charlie delta echo foxtrot golf hotel india juliet kilo lima "
+          "mike november oscar papa quebec romeo sierra tango uniform victor whiskey "
+          "xray yankee zulu cardinal ordinal topology kernel scheduler resident "
+          "unified aperture expert arena slab offload prefetch gemm triton quant "
+          "tensor shard checkpoint manifest embed router cache page table window "
+          "gradient sparse dense moe attention head dim block stream decode prefill "
+          "bandwidth latency throughput residency eviction tier swap pin memlock").split()
+_R = random.Random(1234)
+_ORDER = list(range(len(_VOCAB)))
+_R.shuffle(_ORDER)
+
+
+def log(msg):
+    print("[measure] %s" % msg, file=sys.stderr, flush=True)
+
+
+def post_json(path, body, timeout):
+    req = urllib.request.Request(
+        BASE + path, data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"})
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        return json.load(r)
+
+
+def count_tokens(text):
+    """Token count without a prefill, from the server's own counter."""
+    body = {"model": MODEL,
+            "messages": [{"role": "user", "content": text}]}
+    r = post_json("/v1/messages/count_tokens", body, timeout=300)
+    n = r.get("input_tokens") or r.get("count") or r.get("token_count")
+    if not n:
+        raise RuntimeError("count_tokens returned no count: %r" % (r,))
+    return int(n)
+
+
+def words(n):
+    return " ".join(_VOCAB[_ORDER[i % len(_VOCAB)]] for i in range(n))
+
+
+def calibrate():
+    """Measure words->tokens from the server's own counter."""
+    return 2000.0 / count_tokens(words(2000))
+
+
+def prompt_for(target_tokens, wpt):
+    n = max(1, int(target_tokens * wpt))
+    text = words(n)
+    pt = count_tokens(text)
+    while pt > PROMPT_CEILING:
+        n = max(1, int(n * (PROMPT_CEILING / pt) * 0.98))
+        text = words(n)
+        pt = count_tokens(text)
+    return text, pt
+
+
+def measured(prompt, max_tokens):
+    """Return (ttft_s, decode_tok_s, completion_tokens) via usage counters."""
+    body = {"model": MODEL,
+            "messages": [{"role": "user", "content": prompt}],
+            "max_tokens": max_tokens, "temperature": 0,
+            "stream": True, "stream_options": {"include_usage": True}}
+    req = urllib.request.Request(
+        BASE + "/v1/chat/completions", data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"})
+    t0 = time.time()
+    ttft = None
+    usage = {}
+    with urllib.request.urlopen(req, timeout=1800) as r:
+        for raw in r:
+            line = raw.decode("utf-8", "replace").strip()
+            if not line.startswith("data:"):
+                continue
+            s = line[5:].strip()
+            if s == "[DONE]":
+                break
+            try:
+                ev = json.loads(s)
+            except json.JSONDecodeError:
+                continue
+            if ev.get("usage"):
+                usage = ev["usage"]
+            for ch in ev.get("choices", []):
+                if (ch.get("delta") or {}).get("content"):
+                    if ttft is None:
+                        ttft = time.time() - t0
+    wall = time.time() - t0
+    ct = int(usage.get("completion_tokens") or 0)
+    if ct <= 0:
+        # No usage on the stream: the endpoint is unhealthy or the model
+        # returned nothing (e.g. HTTP 200 with an error payload).
+        raise RuntimeError("no completion_tokens in usage: %r" % (usage,))
+    dec_window = max(wall - (ttft or 0.0), 1e-9)
+    return (ttft or wall), (ct / dec_window), ct
+
+
+def main():
+    log("base=%s model=%s ceiling=%d" % (BASE, MODEL, PROMPT_CEILING))
+    try:
+        wpt = calibrate()
+    except Exception as e:
+        log("endpoint unhealthy during calibration: %s" % e)
+        return 2
+    log("words/token: %.3f" % wpt)
+
+    result = {"words_per_token": round(wpt, 5)}
+    for tgt in DEPTHS:
+        try:
+            p, pt = prompt_for(tgt, wpt)
+            measured(p, WARM_TOKENS)  # warm the arena
+        except Exception as e:
+            log("warm pass failed at depth %d: %s" % (tgt, e))
+            return 2
+        rates, tts, cts = [], [], []
+        for i in range(RUNS):
+            try:
+                tt, r, ct = measured(p, MAX_TOKENS)
+            except Exception as e:
+                log("run %d/%d failed at depth %d: %s" % (i + 1, RUNS, tgt, e))
+                continue
+            rates.append(r)
+            tts.append(tt)
+            cts.append(ct)
+        if len(rates) < 2:
+            log("only %d usable run(s) at depth %d — refusing to median" % (len(rates), tgt))
+            return 2
+        result["decode_tps_%d" % tgt] = round(statistics.median(rates), 2)
+        result["ttft_ms_%d" % tgt] = round(1000.0 * statistics.median(tts), 1)
+        result["completion_tokens_%d" % tgt] = int(statistics.median(cts))
+        result["prompt_tokens_%d" % tgt] = pt
+        log("depth %5d tok  decode %.2f t/s  ttft %.3f s  (runs=%d)"
+            % (pt, result["decode_tps_%d" % tgt], result["ttft_ms_%d" % tgt] / 1000.0, len(rates)))
+
+    meds = [result["decode_tps_%d" % t] for t in DEPTHS]
+    result["decode_tps"] = round(statistics.median(meds), 2)
+    print(json.dumps(result))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
@@ -0,0 +1,200 @@
+#!/usr/bin/env bash
+# frank-primary-window.sh — one full bench window for frank-primary, run ON
+# DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh).
+#
+# Discipline (AGENTS.md Benchmarking Policy): the production unit is
+# STOPPED + MASKED first, the candidate config boots AS THE SERVICE itself
+# (same serve script, same container name, same port 7770 — no scratch
+# ports, no co-tenant GPU), the in-container measure runs, and the window
+# ALWAYS ends by restoring the production unit and verifying its health.
+# The EXIT trap makes the restore unconditional: a boot failure, a measure
+# failure, or a dead ssh session all end with production back up.
+#
+# Staging layout (created by the orchestrator, ephemeral — never a config
+# source; the repo checkout is):
+#   /home/dank/.autoresearch-frank-primary/
+#     serve.sh             staged candidate serve script
+#     frank-primary.env    staged candidate env (the candidate knobs)
+#     measure.py           the in-container measure client
+#     hash                 candidate fingerprint
+#     run.json             result (metrics + boot + kv + fingerprint)
+#     serve.out/measure.out/fail.log
+#     last/                memo: run.json + hash of the last FULLY
+#                          successful window (bench AND verified restore)
+#
+# This script logs to stderr and exits 0 only when the measure produced a
+# usable result (the restore happens in the trap regardless of the bench
+# outcome; a failed restore forces exit 1).
+set -uo pipefail
+
+STAGE="${FRANK_PRIMARY_AR_STAGE:-/home/dank/.autoresearch-frank-primary}"
+UNIT="${FRANK_PRIMARY_AR_UNIT:-frank-primary}"
+CTR="${FRANK_PRIMARY_AR_CTR:-frank-primary-qwen}"
+HEALTH_URL="http://127.0.0.1:7770/health"
+MODELS_URL="http://127.0.0.1:7770/v1/models"
+BOOT_TIMEOUT="${FRANK_PRIMARY_AR_BOOT_TIMEOUT:-1200}"
+RESTORE_TIMEOUT="${FRANK_PRIMARY_AR_RESTORE_TIMEOUT:-1200}"
+MEASURE_TIMEOUT="${FRANK_PRIMARY_AR_MEASURE_TIMEOUT:-1800}"
+SERVED="$(sed -n 's/^FRANK_PRIMARY_SERVED_NAME=//p' "$STAGE/frank-primary.env" 2>/dev/null | head -1)"
+SERVED="${SERVED:-frank-primary}"
+
+RESTORED=0
+
+log() { printf '[window %s] %s\n' "$(date '+%H:%M:%S')" "$*" >&2; }
+
+container_state() { podman inspect -f '{{.State.Status}}' "$CTR" 2>/dev/null || echo absent; }
+
+wait_port_free() {
+    local i
+    for i in $(seq 1 90); do
+        [ "$(container_state)" != "running" ] && return 0
+        sleep 2
+    done
+    return 1
+}
+
+wait_health() { # $1 = timeout seconds
+    local t0 code
+    t0=$(date +%s)
+    while :; do
+        code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 10 "$HEALTH_URL" 2>/dev/null || echo 000)"
+        [ "$code" = "200" ] && return 0
+        [ "$(( $(date +%s) - t0 ))" -ge "$1" ] && return 1
+        sleep 2
+    done
+}
+
+capture_bench_failure() {
+    log "capturing bench-boot failure evidence"
+    {
+        echo "=== podman logs (tail 400) ==="
+        podman logs --tail 400 "$CTR" 2>&1 || true
+        echo "=== staged serve.out (tail 60) ==="
+        tail -60 "$STAGE/serve.out" 2>/dev/null || true
+        echo "=== engine log (tail 60) ==="
+        tail -60 /home/dank/logs/frank_primary.log 2>/dev/null || true
+    } >"$STAGE/fail.log" 2>&1 || true
+    grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \
+        "$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true
+}
+
+restore() {
+    [ "$RESTORED" = 1 ] && return 0
+    RESTORED=1
+    log "restoring production unit $UNIT"
+    if [ -f "$STAGE/serve.pid" ]; then
+        kill "$(cat "$STAGE/serve.pid" 2>/dev/null)" 2>/dev/null || true
+    fi
+    podman stop -t 120 "$CTR" >/dev/null 2>&1 || true
+    wait_port_free || log "WARNING: bench container still draining after 180s"
+    sudo systemctl unmask "$UNIT" 2>/dev/null || true
+    if ! sudo systemctl start "$UNIT"; then
+        log "ERROR: systemctl start $UNIT failed — operator intervention required"
+        return 1
+    fi
+    if wait_health "$RESTORE_TIMEOUT"; then
+        if curl -s --max-time 10 "$MODELS_URL" 2>/dev/null | grep -q "$SERVED"; then
+            log "production restored and verified ($SERVED answering on :7770)"
+            # Memo promotion lives HERE: a cached result is only ever the
+            # product of a fully successful window (bench AND verified
+            # restore), never of a bare bench success.
+            cp "$STAGE/run.json" "$STAGE/last/" 2>/dev/null || true
+            cp "$STAGE/hash" "$STAGE/last/" 2>/dev/null || true
+        else
+            log "ERROR: health 200 but /v1/models does not list $SERVED"
+            return 1
+        fi
+    else
+        log "ERROR: production not healthy within ${RESTORE_TIMEOUT}s — unit left to systemd (Restart=always)"
+        return 1
+    fi
+    return 0
+}
+trap 'restore; rc=$?; [ "$rc" -ne 0 ] && exit 1' EXIT
+
+main() {
+    local f
+    for f in serve.sh frank-primary.env measure.py hash; do
+        [ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; }
+    done
+    mkdir -p "$STAGE/last"
+    rm -f "$STAGE/run.json" "$STAGE/fail.log" "$STAGE/measure.out"
+
+    local t_start t0 i kvline kv_tokens kv_conc measure_json boot_seconds
+    t_start="$(date +%s)"
+
+    log "stopping and masking production unit $UNIT"
+    sudo systemctl stop "$UNIT" 2>/dev/null || true
+    # Explicit stop is not auto-restarted even with Restart=always; the mask
+    # is belt-and-braces against a stray start during the window.
+    sudo systemctl mask "$UNIT" 2>/dev/null || \
+        log "WARNING: mask failed — relying on explicit-stop semantics"
+    wait_port_free || log "note: production container still draining"
+
+    t0="$(date +%s)"
+    log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))"
+    FRANK_PRIMARY_ENV_FILE="$STAGE/frank-primary.env" \
+        setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null &
+    echo "$!" >"$STAGE/serve.pid"
+
+    for i in $(seq 1 45); do
+        [ "$(container_state)" = "running" ] && break
+        sleep 2
+    done
+    if [ "$(container_state)" != "running" ]; then
+        log "bench container never reached running state"
+        tail -40 "$STAGE/serve.out" >&2 || true
+        exit 1
+    fi
+
+    if wait_health "$BOOT_TIMEOUT"; then
+        boot_seconds=$(( $(date +%s) - t0 ))
+        log "bench healthy after ${boot_seconds}s"
+    else
+        if [ "$(container_state)" = "running" ]; then
+            log "bench boot timed out after ${BOOT_TIMEOUT}s (still loading)"
+        else
+            log "bench container died during boot"
+            capture_bench_failure
+        fi
+        exit 1
+    fi
+
+    log "running measure in container (timeout ${MEASURE_TIMEOUT}s)"
+    if ! podman cp "$STAGE/measure.py" "$CTR":/tmp/ar-measure.py; then
+        log "podman cp of measure client failed"
+        exit 1
+    fi
+    if ! timeout "$MEASURE_TIMEOUT" podman exec "$CTR" python3 /tmp/ar-measure.py \
+            >"$STAGE/measure.out" 2>&1; then
+        log "measure failed or timed out"
+        tail -40 "$STAGE/measure.out" >&2 || true
+        exit 1
+    fi
+
+    # Capacity facts for the run (context-size experiments change these).
+    kvline="$(podman logs "$CTR" 2>&1 | grep -E 'GPU KV cache size' | tail -1 || true)"
+    kv_tokens="$(printf '%s' "$kvline" | sed -E 's/.*GPU KV cache size: ([0-9,]+) tokens.*/\1/' | tr -d ',' || true)"
+    kv_conc="$(printf '%s' "$kvline" | sed -E 's/.*per request: ([0-9.]+x).*/\1/' || true)"
+
+    measure_json="$(grep -E '^\{' "$STAGE/measure.out" | tail -1 || true)"
+    if [ -z "$measure_json" ]; then
+        log "measure produced no JSON result line"
+        tail -40 "$STAGE/measure.out" >&2 || true
+        exit 1
+    fi
+
+    # The measure line is already JSON, so the run object can be assembled
+    # without any host-side interpreter.
+    printf '{"hash":"%s","boot_seconds":%s,"kv_tokens":%s,"kv_concurrency":"%s","total_seconds":%s,"measure":%s}\n' \
+        "$(tr -d '[:space:]' <"$STAGE/hash")" \
+        "$boot_seconds" "${kv_tokens:-0}" "${kv_conc:-unknown}" \
+        "$(( $(date +%s) - t_start ))" "$measure_json" >"$STAGE/run.json"
+
+    log "bench window complete in $(( $(date +%s) - t_start ))s"
+    # Memo only after a FULL success; the restore below still has to verify.
+    exit 0
+}
+
+main "$@"
+exit "$?"
@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+# Build the ROCmFPX runtime that serves frank-large on Dankclaw.
+#
+#   bash scripts/build-rocmfpx.sh              # clone/fetch + build the pin
+#   bash scripts/build-rocmfpx.sh --verify     # only check the installed tree
+#
+# Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card
+# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-large's
+# payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of
+# GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so
+# stock llama.cpp, gufo and every other loader on the host reject the file.
+# This is the same tree that served the Qwen3.8-Flash-Next ROCmFP4 STRIX
+# endpoint 2026-08-30 → 09-11 (see the kingjones runbook), rebuilt here so a
+# fresh host can be brought up from the repo alone.
+#
+# Why this commit and not a newer one: 36e9acd (2026-08-28) is what was
+# qualified against this payload on 2026-10-02 — it loads ftype 115, survives
+# warmup, and drafts with the MTP head at acceptance 0.65. The five legacy
+# builds of 2026-08-25 crashed this same file in ggml_compute_forward_set_rows
+# (see kingjones-q8agent-rocmfpx.md), so the pin is load-bearing: do not bump
+# it without re-running that crash test.
+#
+# Host notes:
+#   * -DLLAMA_BUILD_WEBUI=OFF is REQUIRED on this box. The tree provisions its
+#     embedded UI assets via npm or the HF llama-ui bucket; npm is absent here
+#     and the bucket download half-failed, aborting the link at 98%. The API
+#     server is unaffected.
+#   * The build links against the host ROCm (/opt/rocm); no bundled toolchain
+#     like the gfx1201 R9700 build needed.
+set -euo pipefail
+
+ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}"
+# The commit frank-large was qualified with (measured 2026-10-02).
+ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}"
+SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}"
+BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}"
+GPU_TARGET="${ROCMFPX_GPU_TARGET:-gfx1151}"
+JOBS="${JOBS:-$(nproc)}"
+
+# check_pin — refuse a source tree that is not the qualified commit. A moved
+# fork means different kernels under the same path, and on this model line a
+# wrong build is a segfault at first request, not a clean error.
+check_pin() {
+    local commit
+    commit="$(git -C "$SRC_DIR" rev-parse HEAD 2>/dev/null || true)"
+    if [[ -n "$commit" && "$commit" != "$ROCMFPX_COMMIT" ]]; then
+        echo "ROCMFPX_COMMIT mismatch: $SRC_DIR is at $commit, pinned $ROCMFPX_COMMIT" >&2
+        exit 1
+    fi
+}
+
+if [[ "${1:-}" == "--verify" ]]; then
+    check_pin
+    [[ -x "$BUILD_DIR/bin/llama-server" ]] || {
+        echo "no ROCmFPX llama-server at $BUILD_DIR/bin/llama-server" >&2
+        exit 1
+    }
+    # The format the endpoint exists for: the fork must carry the ROCmFPx
+    # tensor type, or the payload cannot load at all.
+    "$BUILD_DIR/bin/llama-server" --help 2>&1 | grep -q -- '--spec-type' || {
+        echo "$BUILD_DIR/bin/llama-server has no --spec-type: MTP drafting is impossible" >&2
+        exit 1
+    }
+    "$BUILD_DIR/bin/llama-server" --version | head -1
+    exit 0
+fi
+
+echo "== source: $ROCMFPX_REPO @ $ROCMFPX_COMMIT =="
+if [[ ! -d "$SRC_DIR/.git" ]]; then
+    mkdir -p "$(dirname "$SRC_DIR")"
+    git clone "$ROCMFPX_REPO" "$SRC_DIR"
+fi
+git -C "$SRC_DIR" fetch --quiet origin "$ROCMFPX_COMMIT" || true
+git -C "$SRC_DIR" checkout -q --detach "$ROCMFPX_COMMIT"
+check_pin
+echo "   $(git -C "$SRC_DIR" rev-parse HEAD)"
+
+echo "== configure + build (release, HIP $GPU_TARGET) =="
+cmake -S "$SRC_DIR" -B "$BUILD_DIR" \
+    -DGGML_HIP=ON \
+    -DGPU_TARGETS="$GPU_TARGET" \
+    -DGGML_NATIVE=ON \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DLLAMA_BUILD_WEBUI=OFF
+cmake --build "$BUILD_DIR" --target llama-server llama-quantize -j "$JOBS"
+
+echo "== verify =="
+"$BUILD_DIR/bin/llama-server" --version | head -1
+echo "frank-large runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"
@@ -1077,27 +1077,29 @@ deploy_anubis() {
    wait_container_healthy anubis 60 || return 1
 }

-# frank-large is gufo (gufo-org/gufo), the Strix Halo inference engine built
-# from the pinned v0.1.1 tag by scripts/build-gufo.sh, serving
-# unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL with the DFlash2 drafter
-# (z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M) on the Strix Halo iGPU. It is a
-# ROOT unit: the runtime is a native binary linked against the host ROCm,
-# so there is no container image to install and no generated unit to dodge.
-# relink_unit points /etc/systemd/system/frank-large.service at the repo
-# file and boot persistence is that unit's [Install] section.
+# frank-large is the ROCmFPX fork of llama.cpp (kingjones30/ROCmFPX, pinned by
+# scripts/build-rocmfpx.sh), serving
+# kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF:Q8_0_ROCMFPX_AGENT with the
+# repo's own Q4_0 MTP draft head and its BF16 vision projector, on the Strix
+# Halo iGPU. It is a ROOT unit: the runtime is a native binary linked against
+# the host ROCm, so there is no container image to install and no generated
+# unit to dodge. relink_unit points /etc/systemd/system/frank-large.service at
+# the repo file and boot persistence is that unit's [Install] section.
 #
-# The runtime gate matters as much as the payload gate: gufo is built from
-# source on this host (there is no prebuilt release), so a machine whose
+# The runtime gate matters as much as the payload gate: the fork is built from
+# source on this host, and the payload is ftype 115 (tensor type 103,
+# GGML_TYPE_Q8_0_ROCMFPX) which only this fork can read — so a machine whose
 # /home/dank/GitPackages tree was never built has a unit that can never
 # become healthy, and a green deploy would leave the endpoint dark.
 #
-# It replaces the stock llama.cpp prebuilt (unsloth b11030), which had
-# replaced the Kairic Edge runtime (ciru-ai/ROCmFPX). The port gate is the
-# same as every cutover since 2026-09-16: nothing starts until 7770 is
-# released.
-LLM_LARGE_RUNTIME="${LLM_LARGE_RUNTIME:-/home/dank/GitPackages/gufo-v0.1.1/bin/gufo}"
-LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf}"
-LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf}"
+# It replaces gufo v0.1.1 + unsloth UD-Q4_K_XL (the rollback target, still
+# built by scripts/build-gufo.sh with its payload on disk), which had replaced
+# the stock llama.cpp prebuilt. The port gate is the same as every cutover
+# since 2026-09-16: nothing starts until 7770 is released.
+LLM_LARGE_RUNTIME="${LLM_LARGE_RUNTIME:-/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server}"
+LLM_LARGE_MODEL="${LLM_LARGE_MODEL:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf}"
+LLM_LARGE_DRAFT="${LLM_LARGE_DRAFT:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mtp-Qwen3.8-27B-Q4_0.gguf}"
+LLM_LARGE_MMPROJ="${LLM_LARGE_MMPROJ:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mmproj-Qwen3.8-27B-BF16.gguf}"

 retire_halogen_large_quadlet() {
    local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
@@ -1132,17 +1134,19 @@ wait_port_free() {
 deploy_llm_large() {
    # Refuse rather than restart-loop. A unit whose runtime or payload is
    # missing never becomes healthy, and a green deploy would leave the
-    # endpoint dark with nothing in the deploy log to say why.
+    # endpoint dark with nothing in the deploy log to say why. The gates
+    # mirror serve-frank-large.sh's own predicates exactly: runtime
+    # executable, target GGUF, MTP head, mmproj.
    if [[ ! -x "$LLM_LARGE_RUNTIME" ]]; then
-        echo "ERROR: gufo runtime absent at $LLM_LARGE_RUNTIME" >&2
-        echo "frank-large: build it: bash scripts/build-gufo.sh" >&2
+        echo "ERROR: ROCmFPX runtime absent at $LLM_LARGE_RUNTIME" >&2
+        echo "frank-large: build it: bash scripts/build-rocmfpx.sh" >&2
        return 1
    fi
    local missing=0 f
-    for f in "$LLM_LARGE_MODEL" "$LLM_LARGE_DRAFT"; do
+    for f in "$LLM_LARGE_MODEL" "$LLM_LARGE_DRAFT" "$LLM_LARGE_MMPROJ"; do
        if [[ ! -f "$f" ]]; then
            echo "ERROR: payload absent at $f" >&2
-            echo "frank-large: fetch it: bash scripts/download-frank-large.sh" >&2
+            echo "frank-large: fetch it: bash scripts/download-frank-large-kingjones.sh" >&2
            missing=1
        fi
    done
@@ -1152,18 +1156,14 @@ deploy_llm_large() {
    # On a re-deploy the predecessor is THIS unit: stop it before waiting on
    # the port, or the wait deadlocks on the server it just stopped serving.
    # Then sweep a stray manual run -- that is how a bench harness leaves the
-    # port busy. Match the DRAFTER path, which every frank-large command
-    # line carries via --dflash-model and which is UNCHANGED by the 2026-09-28
-    # payload swap: the target moved to a different repository while the
-    # drafter stayed put, so keying on the target filename would have quietly
-    # stopped matching. It is also the right key in principle — it names this
-    # endpoint's speculative drafter, not a binary. Match that, not a binary
-    # name: frank-small on this host serves a different GGUF over llama-server
-    # with no DFlash2 drafter and must not be killed. The pattern never
-    # appears in the deploying shell's own command line, so -f is safe here
-    # (it is not in `bash deploy-remote.sh`).
+    # port busy. Match the TARGET GGUF name, which every frank-large command
+    # line carries via --model and which no other process on this host uses:
+    # frank-small also runs llama-server here (with the MiMo GGUF) and must
+    # not be killed, so keying on the binary name would take the 3090 endpoint
+    # down with it. The pattern never appears in the deploying shell's own
+    # command line, so -f is safe here (it is not in `bash deploy-remote.sh`).
    run_privileged systemctl stop frank-large.service 2>/dev/null || true
-    pkill -f 'Qwen3.8-27B-DFlash2-Q4_K_M' 2>/dev/null || true
+    pkill -f 'Qwen3.8-27B-Q8_0_ROCMFPX_AGENT' 2>/dev/null || true
    # Nothing may start until the predecessor has released the port. On
    # 2026-09-16 the cutover failed exactly here: the old server logged
    # "Deactivated successfully", the next engine tried to bind 7770 a second
@@ -1177,44 +1177,56 @@ deploy_llm_large() {
    wait_for_health frank-large http://127.0.0.1:7770/health 1200
 }

-# deploy_llm_small — frank-small is SGLang (python3.12 venv) on the RTX 3090,
-# as a ROOT unit: the runtime is a self-contained venv, so there is no
-# container image to install and no generated unit to avoid. relink_unit
-# points /etc/systemd/system/frank-small.service at the repo file and boot
+# deploy_llm_small — frank-small is llama.cpp (master build, CUDA sm_86) on
+# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
+# ~/GitPackages/llama-master/build-cuda, so there is no container image and
+# no generated unit to avoid. relink_unit points
+# /etc/systemd/system/frank-small.service at the repo file and boot
 # persistence is that unit's [Install] section.
 #
 # It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is
 # not shared: frank-large's engine runs on the gfx1151 iGPU.
-LLM_SMALL_MODEL_DIR="${LLM_SMALL_MODEL_DIR:-/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound}"
-LLM_SMALL_RUNTIME="${LLM_SMALL_RUNTIME:-/home/dank/GitPackages/sglang-cu128/bin/sglang}"
+LLM_SMALL_MODEL_FILE="${LLM_SMALL_MODEL_FILE:-/home/dank/models/MiMo-Q5KM/MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf}"
+LLM_SMALL_RUNTIME="${LLM_SMALL_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
+# The projector IS the vision capability here: the Q5_K_M GGUF ships no `v.*`
+# tensors, so a deploy that starts without this file silently ships a
+# text-only endpoint. Checked unconditionally, like frank-large's.
+LLM_SMALL_MMPROJ="${LLM_SMALL_MMPROJ:-/home/dank/models/MiMo-Q5KM/mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf}"

 deploy_llm_small() {
    # Refuse rather than restart-loop: a unit whose runtime or payload is
    # missing never becomes healthy, and a green deploy would leave the
    # endpoint dark with nothing in the log to say why.
    if [[ ! -x "$LLM_SMALL_RUNTIME" ]]; then
-        echo "ERROR: SGLang runtime absent at $LLM_SMALL_RUNTIME" >&2
-        echo "frank-small: build the venv (python3.12 -m venv /home/dank/GitPackages/sglang-cu128" >&2
-        echo "frank-small: and pip install 'sglang[all]' into it) first." >&2
-        return 1
-    fi
-    if [[ ! -f "$LLM_SMALL_MODEL_DIR/model.safetensors.index.json" ]]; then
-        echo "ERROR: payload index absent at $LLM_SMALL_MODEL_DIR" >&2
-        echo "frank-small: run scripts/download-frank-small.sh first;" >&2
-        echo "frank-small: refusing to start a unit that cannot load." >&2
-        return 1
-    fi
-    local count
-    count="$(find "$LLM_SMALL_MODEL_DIR" -maxdepth 1 -name 'model-*-of-*.safetensors' | wc -l)"
-    if [[ "$count" -ne 8 ]]; then
-        echo "ERROR: payload shard count wrong: $LLM_SMALL_MODEL_DIR (found $count, want 8)" >&2
+        echo "ERROR: llama.cpp runtime absent at $LLM_SMALL_RUNTIME" >&2
+        echo "frank-small: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
+        echo "frank-small: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
+        echo "frank-small: && cmake --build build-cuda --target llama-server) first." >&2
        return 1
    fi
+    for f in "$LLM_SMALL_MODEL_FILE" "$LLM_SMALL_MMPROJ"; do
+        if [[ ! -f "$f" ]]; then
+            echo "ERROR: frank-small payload file absent at $f" >&2
+            echo "frank-small: run scripts/download-frank-small.sh first;" >&2
+            echo "frank-small: refusing to start a unit that cannot load." >&2
+            return 1
+        fi
+    done
    relink_unit frank-small
    run_privileged systemctl daemon-reload
    run_privileged systemctl enable frank-small.service
    run_privileged systemctl restart frank-small.service
-    wait_for_health frank-small http://127.0.0.1:7771/health 900
+    wait_for_health frank-small http://127.0.0.1:7771/health 900 || return 1
+    # False-green hardening (frank-large's lesson): the port answering
+    # proves nothing about WHO answers — the unit must be active with a
+    # live MainPID, or a stray SGLang holdover is masquerading as deployed.
+    local main_pid
+    main_pid="$(run_privileged systemctl show -p MainPID --value frank-small.service)"
+    if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
+        echo "FAIL: :7771 answered but frank-small.service has no MainPID — a stray process holds the port" >&2
+        return 1
+    fi
+    echo "frank-small: health answered by unit MainPID $main_pid"
 }

 # deploy_llm_primary — frank-primary is the tcclaviger RDNA4 vLLM fork
@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-large
+# (Dankclaw iGPU). Public repo — no auth header, and never a token on the
+# command line (it is world-readable in /proc/<pid>/cmdline).
+#
+# The payload is the repo's highest-quality target plus its MTP draft head and
+# vision projector:
+#   * Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf — ftype 115, 187 tensors at plain
+#     Q8_0 and 311 at GGML_TYPE_Q8_0_ROCMFPX (103). The AGENT routing keeps the
+#     output-head tensors at full Q8_0 (187 vs 2 in the non-AGENT build), which
+#     is the variant to take when serving with the MTP head.
+#   * mtp-Qwen3.8-27B-Q4_0.gguf — the draft head. Q4_0, NOT Q8_0: the head's
+#     per-step cost is its projection over a 248K vocabulary, so the cheaper
+#     head is faster end to end at equal acceptance (author: +6.0% at n-max 1,
+#     +10.2% at n-max 2).
+#   * mmproj-Qwen3.8-27B-BF16.gguf — the vision tower (verified 4/4 on spatial
+#     ground truth with -fa on).
+#
+# Fresh downloads, no resume (`curl -C -` once produced an oversized file on a
+# 31 GiB payload); -f/--fail-with-body so an HTTP 401 is not a success, then
+# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
+#
+# The repo's own SHA256SUMS covers only the ROCmFP4 files, so the hashes below
+# come from the Hub LFS oids (api/models/<repo>/tree/main).
+#
+# Usage:
+#   scripts/download-frank-large-kingjones.sh [DEST_DIR] [--verify-only]
+set -euo pipefail
+
+REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP}"
+
+# name bytes sha256
+FILES=(
+  "Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf 28214425952 f3ef2bdbf56e244fff14b72114dc2aa300b84a4cb4eaf54f2e5e4da1cfcae10b"
+  "mtp-Qwen3.8-27B-Q4_0.gguf 1680271648 051a1764cff8c4f3ee6ae8b00593a0364c7539c67fa50ffc58f3f96509fca38e"
+  "mmproj-Qwen3.8-27B-BF16.gguf 931145888 de2a49866988ea272c43dc2a43ee2662c25725a717ca5b9059ea420a97f53fe8"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"
@@ -1,19 +1,26 @@
 #!/usr/bin/env bash
-# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-small
-# (SGLang, RTX 3090). Public repo — no auth header on the command line.
+# Download the frank-small payload for llama.cpp on the RTX 3090:
+# bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q5_K_M plus its vision projector
+# (the GGUF carries no `v.*` tensors — the tower lives only in the mmproj, so
+# without this file the endpoint is text-only by construction). Public repo —
+# no auth header, and never a token on the command line (it is world-readable
+# in /proc/<pid>/cmdline).
 #
-# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
-# oversized file on a 31 GiB payload; a fresh -f fetch plus size+sha256 is
-# the house rule.
+# Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized file
+# on a 31 GiB payload); -f/--fail-with-body so an HTTP 401 is not a success,
+# then byte count and the Hub LFS sha256. Idempotent: complete files are
+# skipped.
+#
+# The Q6_K file this endpoint served until 2026-10-01 night stays on disk as
+# the rollback target (7793710624 B, LFS oid
+# ef96d05a2ddf2cbb450d1af1ac3860ec769d3575bafa70692ee5609bda3fad7d) — re-fetch
+# it from the same repo with the same recipe if it is ever needed again.
 #
 # Usage:
 #   scripts/download-frank-small.sh [DEST_DIR] [--verify-only]
-#
-# LFS shards are verified against the Hub LFS sha256; non-LFS text files
-# against their declared byte size. Idempotent: complete files are skipped.
 set -euo pipefail

-REPO="letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound"
+REPO="bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF"
 BASE="https://huggingface.co/${REPO}/resolve/main"

 DEST=""
@@ -24,28 +31,12 @@ for arg in "$@"; do
        *) DEST="$arg" ;;
    esac
 done
-DEST="${DEST:-/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound}"
+DEST="${DEST:-/home/dank/models/MiMo-Q5KM}"

-# name bytes sha256(- = non-LFS, size-only check)
+# name bytes sha256
 FILES=(
-  "model-00001-of-00008.safetensors 2946296312 ddb6117c04cb6feaf7c265e5d75efadf21949037d73253a8e4521d40e164366d"
-  "model-00002-of-00008.safetensors 1402134792 2dc78d55d1d33d7b3b21dd6f8b4b817b62c2fb1376e6448b322428798c10e257"
-  "model-00003-of-00008.safetensors 1427365904 5c797d14e345309d67c78306b9de4aa01a21dbd92dd98e40b0ad69b2900e9203"
-  "model-00004-of-00008.safetensors 1420041856 7402f852e2c7a24ebf4694b98a220e71dfb2c93ba3776272cf2acfc65acd8126"
-  "model-00005-of-00008.safetensors 1410655648 991cbf2eda2ff91fc3ffbb15591d7fcac340fb10d9c33bd3da20010e2f3f4090"
-  "model-00006-of-00008.safetensors 1427627856 efc42448054cfb5e3061e154ba4539204088ce95176d689a95b58de885ef7eeb"
-  "model-00007-of-00008.safetensors 2035442408 75a101fc71ae6f2e1f5c4816f711d62cf1d7a0e02ef6e5182361b8c11e72b626"
-  "model-00008-of-00008.safetensors 822153664 0fb90cb50e2d336fbad58bd4c01880eeddbf537c1dca1cf0ae291195d5efbb4b"
-  "tokenizer.json 19989325 06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523"
-  "model.safetensors.index.json 106278 -"
-  "config.json 4608 -"
-  "generation_config.json 161 -"
-  "quantization_config.json 1654 -"
-  "chat_template.jinja 3916 -"
-  "tokenizer_config.json 1168 -"
-  "preprocessor_config.json 443 -"
-  "processor_config.json 1191 -"
-  "README.md 8448 -"
+  "MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf 6876124704 04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f"
+  "mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf 918166048 ff348f3180a63188aa7285db85f550fe38acb61dd013c599eb8bad08d2cc2576"
 )

 mkdir -p "$DEST"
@@ -55,7 +46,7 @@ for entry in "${FILES[@]}"; do
  f="$DEST/$name"
  have=0
  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
-    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    if sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
      have=1
    fi
  fi
@@ -70,7 +61,6 @@ for entry in "${FILES[@]}"; do
  fi
  echo "FETCHING  $name"
  tmp="$f.part"
-  # no -C resume; -f + --fail-with-body so an HTTP error is not a success
  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
  mv "$tmp" "$f"
  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
@@ -78,7 +68,7 @@ for entry in "${FILES[@]}"; do
    status=1
    continue
  fi
-  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+  if ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
    echo "SHA FAIL  $name"
    status=1
    continue
@@ -1,63 +1,85 @@
-# frank-large — gufo (gufo-org/gufo) serving
-# Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) on the Strix Halo iGPU.
+# frank-large — ROCmFPX llama.cpp serving
+# kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF (Q8_0_ROCMFPX_AGENT, ftype
+# 115) + its Q4_0 MTP draft head, on the Strix Halo iGPU (gfx1151), :7770.
 #
-# These are the tunables read by scripts/serve-frank-large.sh. The flag
-# list is owned by the serve script; this file only sets fleet values.
+# 2026-10-02 (Frank): "the highest quality of that repo, one full context,
+# whatever runtime it requires". The highest-quality rung is Q8_0_ROCMFPX_AGENT
+# (26.28 GiB, 8.39 bpw: 187 tensors at plain Q8_0 + 311 at
+# GGML_TYPE_Q8_0_ROCMFPX); the AGENT routing is also the variant the card
+# tells you to take with MTP, because it keeps the output-head tensors at full
+# Q8_0 (187 vs 2), which is what the draft scores against. The runtime the
+# format REQUIRES is the ROCmFPX fork — type 103 exists nowhere else.
 #
-# DO NOT name a new tunable TEMP, HOST or GPU_MAX_HEAP_SIZE: the HIP
-# runtime reads those bare (the 2026-08-13 TEMP incident). HOST is the one
-# exception because the serve script itself consumes it for --host.
-
-# The target payload (scripts/download-frank-large-q4.sh): the serve script
-# refuses to start without the main GGUF and the DFlash2 drafter.
+# The shape: one session at the model's full native window (262,144), which is
+# the whole point — the Q8_0 head costs quality the fleet wanted back, and
+# the draft head pays for the bandwidth the bigger file spends.
 #
-# 2026-10-01 swap (Frank): unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL, 16.35 GiB,
-# replacing ajgazin's abliterated Swift-1.5 UD-Q8_K_XL (29.3 GiB). Two
-# consequences worth knowing at a glance:
-#   * this is the VANILLA (non-abliterated) Qwen3.8-27B — the endpoint is no
-#     longer uncensored, which was one of the reasons the Swift payload was
-#     chosen on 2026-09-28;
-#   * ~13 GiB of weights became available for KV, which is what makes four
-#     full-sized f16 sessions fit.
-# gufo needs no template for either: its Qwen renderer is compiled in.
-MODEL_DIR=/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL
-MODEL_FILE=/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL/Qwen3.8-27B-UD-Q4_K_XL.gguf
+# Budget (from the GGUF header, not from a boot log): attention KV lives only
+# in the 16 full-attention layers (4 KV heads x 256 head_dim), 64 KiB/token
+# with f16 K and V => 16.0 GiB at 262,144. The 48 Gated DeltaNet layers hold
+# ~144 MiB of fixed state per sequence. Weights 26.28 + draft head 1.57 +
+# mmproj 0.87 + KV 16.0 + state 0.14 + compute buffers ≈ 45-47 GiB against the
+# iGPU's ~114 GiB GTT aperture, so f16 KV (the unquantized choice) is affordable
+# here in a way it is not on frank-small's 24 GiB card.

-# The DFlash2 drafter is UNCHANGED by the model swap and still lives beside
-# the previous target, so it carries its own directory. It has a sliding
-# window of 2048, which is what keeps its KV small (~40 MiB).
-DRAFTER_DIR=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL
-DRAFTER_FILE=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf
+# The qualified ROCmFPX fork build (scripts/build-rocmfpx.sh pins the commit).
+LLAMA_BIN=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server
+# One directory per HF repo, created by scripts/download-frank-large-kingjones.sh.
+MODEL_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf
+# The draft head: Q4_0, NOT the Q8_0 head. The head's per-step cost is its
+# projection over a 248K vocabulary, so the cheaper head is faster end to end
+# at equal-or-better acceptance (card: +6.0% at n-max 1, +10.2% at n-max 2).
+DRAFT_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mtp-Qwen3.8-27B-Q4_0.gguf
+# Vision tower, separate from the LLM file. Empty string = text-only.
+MMPROJ_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mmproj-Qwen3.8-27B-BF16.gguf

-# The gufo runtime, built from source by scripts/build-gufo.sh.
-RUNTIME_DIR=/home/dank/GitPackages/gufo-v0.1.1
-
-# Fleet endpoint: the same port and alias every frank-large has served, so
-# no client, probe or models.yml entry changes address.
+# Fleet endpoint: same port and alias as every frank-large before it, so the
+# harness models.yml id and the homepage probe do not move.
 ALIAS=frank-large
-HOST=0.0.0.0
-PORT=7770
+# HIP reads a bare HOST, so this is BIND_ADDR — the same reason the file
+# avoids a bare TEMP.
+BIND_ADDR=0.0.0.0
+LISTEN_PORT=7770

-# Four sessions, 163840 context each (Frank, 2026-10-01: "4x 160k"). The
-# context is per session and the KV is UNQUANTIZED (f16) — gufo has no KV
-# quantization lever at all (execution_policy.hpp pins kv_cache_storage to
-# kFp16), so "unquantized" is inherent rather than configured.
-#
-# The arithmetic that makes this fit: f16 KV costs 64 KiB/token on this model
-# (16 full-attention layers x 4 KV heads x 256 head_dim x 2 (K,V) x 2 bytes;
-# the other 48 layers are linear-attention with a per-SEQUENCE fp32 state of
-# ~144 MiB, not a per-token cache). 4 x 163840 = 655,360 tokens = 40.0 GiB of
-# KV, and each session's 163,840 tokens is exactly 10 GiB. Budget: weights
-# 16.35 + KV 40.0 + drafter 1.07 + linear state 0.56 + runtime ~4 = ~62 GiB
-# against the 114 GiB GTT — roughly 50 GiB of headroom, so the session count
-# or the per-session context can both grow from here.
-CONTEXT_SIZE=163840
-SESSIONS=4
+# One full native window. llama.cpp divides --ctx-size across --parallel
+# slots, so 262144/1 IS the model's trained window; the card validated 65,536
+# and the fleet has been running this family at 262,144 since the gufo era.
+CTX_SIZE=262144
+PARALLEL=1

-# DFlash2 spec decode (a block-diffusion drafter). gufo sizes the draft
-# block itself; DRAFT_TOKENS caps the proposals per step (gufo's default is
-# 7, chosen over llama.cpp's fixed depth 4 because gufo's adaptive policy
-# shortens the block when acceptance drops) and DRAFT_POLICY is the block
-# length controller.
-DRAFT_TOKENS=7
-DRAFT_POLICY=adaptive
+# Deck knobs. --ubatch-size has a measured optimum on this stack (512 peak on
+# the sibling qwen35 model; larger ubatch costs GiBs of compute buffer for no
+# speed), and --fit must stay off: --fit's auto placement does not know that
+# 311 tensors of type 103 belong on the iGPU.
+BATCH_SIZE=2048
+UBATCH_SIZE=512
+NGL=999
+FIT=off
+FLASH_ATTN=on
+# Unquantized KV — the quality-first choice, affordable at one slot.
+CACHE_TYPE_K=f16
+CACHE_TYPE_V=f16
+
+# Speculative decode: the model ships a standalone MTP head (mtp-Q4_0.gguf,
+# arch qwen35, nextn_predict_layers=1, a 65-block draft). n-max 4 with n-min 0
+# and p-min 0.0 is the card's measured optimum (+31% end to end, acceptance
+# 0.926; its own n-max 8 measurement only reached 0.757 acceptance and LOST).
+# The draft runs entirely on the iGPU, so cross-device traffic stays off the
+# bandwidth-limited link.
+SPEC_TYPE=draft-mtp
+SPEC_DRAFT_NGL=99
+SPEC_DRAFT_DEVICE=ROCm0
+SPEC_DRAFT_N_MAX=4
+SPEC_DRAFT_N_MIN=0
+SPEC_DRAFT_P_MIN=0.0
+
+# The card's thinking sampling profile, which is also what the harness sends
+# in extraBody, so the endpoint behaves the same to a client that omits it.
+TEMPERATURE=1.0
+TOP_P=0.95
+TOP_K=20
+MIN_P=0.0
+PRESENCE_PENALTY=0.0
+
+NUM_THREADS=16
+THREAD_BATCH=16
@@ -1,5 +1,5 @@
 [Unit]
-Description=gufo (gufo-org/gufo, ROCm gfx1151) Qwen3.8-27B UD-Q8_K_XL, DFlash2 spec decode, 262144 ctx / 1 session, root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
+Description=ROCmFPX llama.cpp (gfx1151) kingjones777 Qwen3.8-27B Q8_0_ROCMFPX_AGENT, MTP spec decode, 262144 ctx / 1 session, root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
 After=network-online.target
 Wants=network-online.target

@@ -13,7 +13,9 @@ Group=dank
 LimitMEMLOCK=infinity
 Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
 ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
-Restart=on-failure
+# always, not on-failure: an engine that drains and exits 0 after a worker
+# fault would otherwise leave the endpoint down until someone notices.
+Restart=always
 RestartSec=10

 [Install]
@@ -59,43 +59,55 @@ FRANK_PRIMARY_PUBLISH=7770:7770
 # hoc experiment identity (both hold the GPUs / name registrations).
 FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-r9v qwen-fp8"

-# --- qualified engine shape (2026-10-01, RESTORED KNOWN-GOOD: fp8 3 windows) ---
-# bf16 at 2 x 262144 did NOT stay up: with and without --expert-precise-memory
-# the boot failed (`Engine core initialization failed`, NRestarts climbing).
-# The last shape measured serving was fp8 3 x 262144 / MNBT 2048 / KV 6.1e9
-# (KV 792,069 tokens, 3.02x, arena 6,248, ~3.6 GiB free) and is restored here
-# while the bf16 path is investigated in a window (see log.md 2026-10-01).
-# Candidate next test: bf16 with SEQS=3 to isolate the KV dtype from the
-# sequence count, and/or --enforce-eager to bypass the dynamo constraint path.
-# SEQS 2 x MAXLEN 262144 at BF16 KV (Frank's call, 2026-10-01): two contexts of
-# the model's FULL native window with UNQUANTIZED KV, plus
-# --expert-precise-memory to hand the arena the compile/graph-capture headroom.
-# Pool: bf16 costs 13,839 B/token (measured: 11,047,010,304 B -> 798,280 tokens
-# on the 3x241664 bf16 boot; decomposes as 12,276 attention + 1,563 mamba/side
-# caches, against fp8's 6,138 + 1,563 = 7,701). 2 x 262144 = 524,288 tokens
-# -> 7.256e9 B floor, pinned at 7,300,000,000 (~527,500 tokens, 3,206 margin).
-# Budget (per rank): 31.22 GiB total; non-expert weights 6.28, graphs 0.55,
-# non-torch ~1.45, pool 6.76 -> ~29.35 used, ~2.5 GiB free (measured basis:
-# the fp8 3-window shape used 28.27 and left 3.59).
-# PRECISE MEMORY: DROPPED 2026-10-01 after it crashed the boot in combination
-# with bf16. The flag selects the fitted 5.25 GiB overhead instead of the
-# 7.60 GiB default, moving the plan's `room` and therefore the arena
-# (6,248 -> ~7,278 slots). But the arena is DEVICE-resident, and the budget
-# for this shape does not have 2.35 GiB to spare:
-#   weights 6.28 + pool 6.80 (bf16 2 x 262144) + graphs 0.55 + arena 16.65
-#   + non-torch ~1.45 = 31.73 of 31.86 GiB  =>  ~0.13 GiB free at boot
-# measured as a CrashLoop: every attempt reached graph capture, the engine
-# exited cleanly, systemd restarted it (NRestarts=3 at ~6.7 min each).
-# With the default overhead the arena is 14.31 GiB and the same shape totals
-# ~29.4 GiB, leaving ~2.4 GiB for capture. Re-enable the flag only with a
-# pool/context reduction that pays for it.
-# MNBT 2048 (Frank): the overhead formula only adds bytes above nbt 4096, so
-# 2048 pays nothing, and the arena sweep measured the same ceiling <= 4096.
-FRANK_PRIMARY_MAXLEN=262144
-FRANK_PRIMARY_SEQS=3
-FRANK_PRIMARY_KV_BYTES=6100000000
+# --- qualified engine shape (2026-10-02, Frank's own correction) ---
+# 2026-10-01 left a shape that did not stay up: bf16 KV at 2 x 262144 with
+# --expert-precise-memory crash-looped at graph capture (the arena is
+# DEVICE-resident and the shape had 0.13 GiB to spare), and the plain bf16
+# 2 x 262144 boot booted but left 0 bytes free at runtime. This is the
+# correction: FP8 KV, TWO windows, each 200704 tokens, and the pool cut so
+# the expert arena gets the room back.
+#
+# LADDER (2026-10-01 night, subtract-16k method): 262144 and 245760 OOM at
+# graph capture; 237568 passed at 718,606 tokens; 241664 x 3 passed at
+# 732,359 tokens / 3.03x but is the shape this cutover retired. 200704 was
+# chosen to sit below the 241664 rung while leaving a full 2.95 GiB/rank of
+# pool to cut loose for the arena — the two contexts are a capacity
+# contract, not a throughput one (2-way concurrency summed BELOW parity at
+# 4064 ctx), so the slot buys the VRAM headroom long generations need.
+#
+# KV_BYTES is PER RANK. fp8 costs 7,701 B/token (measured 6,138 attention
+# + 1,563 mamba/side, dtype-independent). 2 x 200704 = 401,408 tokens
+# -> 3.091e9 B floor, pinned at 3.15e9 (~405,000 tokens, ~44 MB margin —
+# the same margin convention the 6.1e9 shape used).
+#
+# THE ARENA IS WHAT REMAINS: the fork's expert planner sizes it AFTER the
+# KV override, so every byte cut here is arena. The 6.1e9 shape ran an
+# arena of 6,248 slots; this one reclaims ~2.95 GiB/rank, roughly 1,260
+# more slots at 2.344 MiB each. That is the point of the change — the
+# gather read path is the measured decode bound, and a bigger arena means
+# more experts resident instead of streamed.
+#
+# MNBT 1024 (down from 2048): 1024-4096 are interchangeable on decode AND
+# identical on arena (the sweep measured the same slot count at every
+# rung <= 4096), so this is a pure workspace-floor choice, not a decode
+# trade. The floor matters because the arena budget is what is left.
+#
+# PRECISE MEMORY stays DROPPED: it crashed the 2 x 262144 bf16 boot
+# (weights 6.28 + pool 6.80 + graphs 0.55 + arena 16.65 + non-torch ~1.45
+# = 31.73 of 31.86 GiB, ~0.13 GiB free, NRestarts=3). The flag survives in
+# the serve script as an opt-in; re-enable it only with a pool/context
+# reduction that actually pays for the extra overhead.
+#
+# TUNABLEOP stays OFF (silently skipped under offload unless
+# CLAV_TUNABLEOP_WITH_OFFLOAD=1, and then corrupts numerics).
+#
+# SEQS 2 x MAXLEN 200704 at FP8 KV, pool cut for the arena. No
+# speculative decode, no prefetch.
+FRANK_PRIMARY_MAXLEN=200704
+FRANK_PRIMARY_SEQS=2
+FRANK_PRIMARY_KV_BYTES=3150000000
 FRANK_PRIMARY_KV_DTYPE=fp8
-FRANK_PRIMARY_MNBT=2048
+FRANK_PRIMARY_MNBT=1024
 FRANK_PRIMARY_GATHER_CHUNKS=32
 FRANK_PRIMARY_GATHER_LANES=64
 FRANK_PRIMARY_UTIL=0.98
@@ -1,86 +1,60 @@
-# Tunables for frank-small (letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-
-# W8A16-AutoRound, qwen3_5 hybrid, in-checkpoint vision) on SGLang,
-# Dankclaw's RTX 3090, since 2026-10-01.
+# frank-small — llama.cpp (master, CUDA sm_86) serving
+# bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q5_K_M on the RTX 3090, :7771.
 #
-# Runtime: SGLang in a python3.12 venv (pip "sglang[all]"):
-#   python3.12 -m venv /home/dank/GitPackages/sglang-cu128
-#   /home/dank/GitPackages/sglang-cu128/bin/pip install "sglang[all]"
-# The venv carries its own CUDA torch + sgl-kernel + flashinfer; it is
-# self-contained and does not touch the host Python (3.14) or the ROCm
-# stack that serves frank-large.
+# 2026-10-02 (Frank): THREE contexts, vision on, and a hard instruction to
+# leave 2-3 GiB of the card free. Both new demands cost memory, so the per-slot
+# window shrinks — see the budget below.
 #
-# Model: MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound — a Qwen3.5-9B
-# distillation at INT8 W8A16 (AutoRound, group 128, symmetric). 8
-# safetensors shards, 12.0 GiB total. The vision tower
-# (qwen3_5_vision, 27 layers) and embed_tokens stay fp16 inside the
-# checkpoint, so the image path needs no separate projector file.
-# Architecture: 32 layers, 8 full-attention (4 KV heads x 256 head_dim)
-# + 24 Gated DeltaNet, native 262144 window. text_config declares
-# mtp_num_hidden_layers = 1, but the shipped safetensors carry NO mtp.*
-# tensors (verified against model.safetensors.index.json 2026-10-01 —
-# base BF16 and this INT8 conversion both omit the head), so there is no
-# spec decode: SGLang would draft with an uninitialised 4.24 GiB ghost
-# model at 0% acceptance and pay the verify tax for nothing.
+# Two measurements on this card, this runtime, price the terms:
+#   2 x 262,144, no projector: 20,190 MiB used / 4,386 free
+#     KV = 524,288 tok x 24.5 KiB = 12,800 MiB => non-KV (weights 6,557 +
+#     DeltaNet state ~25/slot + compute buffers + CUDA graphs) = 7,390 MiB
+#   3 x 196,608, projector on: 22,972 MiB used / 1,153 free
+#     KV = 14,113 MiB => non-KV WITH the vision tower = 8,859 MiB
+# The tower costs ~1.4 GiB, not the ~1.0 its file size suggested, so 196,608
+# per slot left only 1.1 GiB free — outside the band. The pool therefore is
+# 3 x 180,224 = 540,672 tokens: KV ~12,936 + non-KV 8,859 = ~21,800 MiB predicted.
+# MEASURED at this shape: 21,684 MiB used / 2,441 MiB free — inside the band, and
+# the number the bats budget gate reproduces (it prices non-KV at 9 GiB).
 #
-# SIZING (RTX 3090), measured 2026-10-01: the 0.98 budget holds the
-# checkpoint (11,354 MiB) + the DeltaNet ssm state pool (795 MiB at 32
-# bf16 slots) + the KV pool. The engine's autofit picks a 687,388-token
-# pool and leaves ~0.53 GB — too little: Triton's deferred kernel loads
-# take most of it after boot, and eager-prefill activations then OOM on
-# the first long request. The pool is therefore PINNED at 606,208 tokens
-# (2 x 262,144 native slots since the 2026-10-01 reslice; 4 x 151,552
-# before), which leaves ~0.8 GB for the prefill workspace; a 170k single
-# request and 4 concurrent 54k requests were verified at the 4-slot
-# shape.
-# The ssm pool is the 4.5 GiB the weights-only arithmetic misses.
+# KV arithmetic: attention KV exists only in the 8 full-attention layers
+# (4 KV heads x 256 head_dim). K f16 = 16 KiB/token, V q8_0 = 8.5 KiB/token
+# (34 B per 32-element block) => 24.5 KiB/token, so 12,288 tokens (=4096/slot)
+# is one 293 MiB step — that is the ladder to walk if this moves again. The
+# native 262,144 per slot is NOT reachable three times: 3 x 262,144 x
+# 24.5 KiB = 18.4 GiB of KV before anything else is loaded.
 #
-# Sampling: greedy by default (the fleet contract); a client that sends a
-# temperature gets its own, and the harness applies per-model overrides
-# from models.yml.
-#
-# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP
-# reads those bare vars on the ROCm hosts -- see the 2026-08-13 incident in
-# AGENTS.md). This box is CUDA, but the rule keeps env files portable.
+# Why V stays q8_0 and K stays f16: full-precision V would add 8 KiB/token —
+# +4.5 GiB on this pool — which is what pushes the card past its 24 GiB. K is
+# the tensor the asymmetric-KV contract exists to protect.
+LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
+MODEL_FILE=/home/dank/models/MiMo-Q5KM/MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf

-# SGLang runtime: python3.12 venv, self-contained CUDA stack.
-SGLANG_BIN=/home/dank/GitPackages/sglang-cu128/bin/sglang
+# Vision: the Q5_K_M GGUF ships no `v.*` tensors, so the projector file IS the
+# vision capability (sha256 ff348f31..., 918,166,048 B, fetched by
+# scripts/download-frank-small.sh). Empty string = text-only on purpose.
+MMPROJ_FILE=/home/dank/models/MiMo-Q5KM/mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf

-# Model package (8 shards + configs, sha256-verified by
-# scripts/download-frank-small.sh).
-MODEL_DIR=/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound
-
-# Fleet identity and network. 7771 is this service's own endpoint on
-# Dankclaw; 7770 stays frank-large's. The alias is the id at /v1/models
-# and what clients ask for -- the harness entry in models.yml names it.
+# Fleet endpoint: same port and alias as every frank-small before it, so the
+# harness models.yml id and the homepage probe do not move.
 ALIAS=frank-small
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7771

-# Two slots at the model's native 262,144 window on fp8 KV. The pool is
-# pinned (see SIZING above): 606,208 tokens, and 2 x 262,144 = 524,288
-# leaves ~82k tokens of pool slack. Resliced 2026-10-01 from 4 x 151,552
-# so a compaction request (frank-primary's trigger 183,501 tokens + a
-# 32,768-token summary budget = 216,269) fits ONE request; the grid is
-# the two-way concurrency contract. A single request may still grow to
-# the whole pool (a 170k prompt was verified at the 4-slot shape).
-MEM_FRACTION=0.98
-MAX_TOTAL_TOKENS=606208
-CONTEXT_LENGTH=262144
-MAX_RUNNING=2
-KV_DTYPE=fp8_e4m3
-QUANTIZATION=auto-round
-# DeltaNet (mamba) recurrent states: the ssm state pool is the single
-# biggest hidden consumer on this card. The engine autofills the slot
-# COUNT to spend a fixed ~5.1 GB budget regardless of dtype (fp32 at 110
-# slots, bf16 at 217 — same 5.1 GB), so BOTH dials are pinned below:
-# bf16 dtype + 32 slots = 795 MiB.
-MAMBA_SSM_DTYPE=bfloat16
-# 32 slots is 16x the 2-way concurrency: enough for the radix mamba
-# cache to keep shared-prefix states, small enough to leave the KV pool
-# room.
-MAX_MAMBA_CACHE_SIZE=32
+# Three independent conversations. llama.cpp divides --ctx-size across
+# --parallel slots (n_ctx_slot = CTX_SIZE/PARALLEL), and a mamba-hybrid model
+# cannot use unified KV, so each slot gets its own cache and its own window.
+# 180,224 per slot is BELOW the native 262,144, and that is the cost of the
+# third slot: a longer single prompt needs compaction instead of one shot.
+# Consequence worth knowing: frank-primary's compaction trigger is 183,501
+# tokens, which no longer fits a frank-small slot — see the runbook's
+# "three slots vs compaction" note.
+CTX_SIZE=540672
+PARALLEL=3

-# Reasoning is extracted into reasoning_content; tools use the qwen3_coder
-# grammar (both parsers per the SGLang Qwen3.5 cookbook).
-REASONING_PARSER=qwen3
-TOOLCALL_PARSER=qwen3_coder
+# The KV contract (Frank, 2026-10-01 night): unquantized K, q8_0 V.
+# q8_0 on V is only served by the flash-attention path; the serve script
+# pins --flash-attn on and must never lose it.
+CACHE_TYPE_K=f16
+CACHE_TYPE_V=q8_0
+NGL=99
@@ -1,5 +1,5 @@
 [Unit]
-Description=SGLang MiMo-V2.6-Distill-Qwen-9B-INT8 (qwen3_5 hybrid, in-checkpoint vision, no MTP) 4x151k fp8 KV (dankclaw, frank-small)
+Description=llama.cpp MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF (qwen35 hybrid, 3x180224, f16 K + q8_0 V, mmproj vision) on the RTX 3090 (dankclaw, frank-small)
 After=network-online.target
 Wants=network-online.target

@@ -1,121 +1,179 @@
 #!/usr/bin/env bash
-# Serve Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) as frank-large on
-# Dankclaw's Strix Halo iGPU (gfx1151), using gufo (gufo-org/gufo) — the
-# inference engine built specifically for this hardware.
+# Serve kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF as frank-large on
+# Dankclaw's Strix Halo iGPU (gfx1151) with the ROCmFPX fork of llama.cpp.
 #
-# Runtime: /home/dank/GitPackages/gufo-v0.1.1/bin/gufo, built from source
-# against the host ROCm by scripts/build-gufo.sh. It replaced the stock
-# llama.cpp prebuilt (unsloth b11030) on 2026-09-28; port, alias and client
-# contract did not move.
+# 2026-10-02 (Frank): swapped from gufo + Q4_K_XL to the highest-quality file
+# in the repo, one session at the native 262,144 window.
 #
-# Payload: ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF's
-# UD-Q8_K_XL, fetched by scripts/download-frank-large-swift.sh. It replaced
-# unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL on 2026-09-28. The two are structurally
-# interchangeable for this engine — same `qwen35` architecture, the same 866
-# tensors in the same order, the same per-tensor Q8_0/F32/BF16 layout, the
-# same MTP head at blk.64 — so the swap needed no engine change. The fine-tune
-# is abliterated (uncensored) and carries the SWIFT OPEN LICENSE 1.0, which
-# is not an OSI licence.
+# Why this runtime and not the previous one: the payload is ftype 115 — 311
+# tensors of GGML_TYPE_Q8_0_ROCMFPX (103), a tensor type that exists only in
+# the ROCmFPX fork. Stock llama.cpp, SGLang and gufo all refuse the file, and
+# gufo additionally gates every "Qwen3.8" artifact on a template-hash
+# allowlist. The qualified build is scripts/build-rocmfpx.sh's pin: it loads
+# ftype 115, survives warmup (the 2026-08-25 builds of the older fork line
+# segfaulted in ggml_compute_forward_set_rows on this exact file), and drafts
+# with the model's own MTP head.
 #
-# Shape: one session, full 262144 context (the model's native window). The
-# 27B dense weights (16.35 GiB at UD-Q4_K_XL) + the DFlash2 drafter (1.07) + KV +
-# compute buffers fit the iGPU's GTT aperture (~114 GiB); a load at this
-# context measured 66.5 GiB of the 116.7 GiB device.
+# Why this file: Q8_0_ROCMFPX_AGENT is the top of the repo's quality ladder
+# (26.28 GiB, 8.39 bpw) and the AGENT routing keeps 187 tensors — the
+# output-head set the draft scores against — at plain Q8_0 instead of the 2
+# the non-AGENT build keeps, which is the variant the card names for MTP
+# serving. Drafting uses the repo's Q4_0 MTP head: the head's cost is its
+# projection over the 248K vocabulary, so the 4-bit head is faster end to end
+# at equal-or-better acceptance than its Q8_0 sibling.
 #
-# Spec method: DFlash2 (a block-diffusion drafter), gufo's adaptive block
-# policy, capped at DRAFT_TOKENS proposals per step. The drafter is the
-# z-lab Qwen3.8-27B-DFlash2-Q4_K_M and it has a sliding window of 2048
-# (small draft KV). It is UNCHANGED by the model swap and still lives beside
-# the previous target, which is why DRAFTER_DIR is its own tunable.
-#
-# Sampling: greedy (--temperature 0) server-side, as every frank-large has
-# served. The harness overrides it per request via extraBody (1.0 / 0.95 /
-# top-k 20).
-#
-# Template: gufo compiles the official Qwen Jinja into the binary and takes
-# NO --chat-template-file — and it is not a Jinja engine at all
-# (src/models/qwen/chat_template.cpp is a hand-written C++ renderer), so the
-# fleet's scripts/qwen-sharp-v22.4.0.jinja cannot be supplied to it. That law
-# was given up when this endpoint moved to gufo on 2026-09-28, by decision:
-# the terse system-prompt injection, the in-band <|think_*|> control, the
-# json|xml tool_call_format switch and the tool-argument truncation are all
-# absent. gufo's own renderer keeps the effort instruction strings and
-# preserve_thinking, which are the parts the harness depends on. Accepted
-# 2026-09-28; recovering the terse prompt from the client side is the
-# cheapest partial restore.
-#
-# Text-only: gufo auto-discovers a mmproj beside the model and this payload
-# has none (the base repo ships one separately, unused), so the endpoint
-# stays text-only.
-#
-# Tunables come from frank-large.env unless FRANK_LARGE_ENV_FILE overrides
-# it; anything not set there falls through to the defaults below.
+# The client contract lives here, not in frank-large.env:
+#   * --jinja — without it llama.cpp silently IGNORES the harness's
+#     chat_template_kwargs (enable_thinking / preserve_thinking), so the
+#     endpoint would look like it had no thinking control at all. The GGUF
+#     carries the official Qwen template, so no --chat-template-file.
+#   * --reasoning-format deepseek — thinking is emitted as reasoning_content,
+#     the field the harness reads for this endpoint.
+#   * the template accepts reasoning_effort xhigh|medium|low only, which is
+#     why models.yml folds the harness's levels onto that set.
+#   * --alias frank-large — that is the model id the harness resolves.
+#   * --ctx-checkpoints / --cache-ram — prompt-cache geometry for the 16
+#     sliding-window-style layers; harmless where the hybrid state forces a
+#     full re-process, and required for the checkpoint feature to work at all.
 set -euo pipefail

-SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
-ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
+# Fleet values live in frank-large.env. Anything already in the environment
+# wins over the file, so an operator can override one value on a single run:
+# the caller's values are captured, the file is sourced, then they are put
+# back. (A plain `source` would let the file clobber the environment.)
+TUNABLES=(LLAMA_BIN MODEL_FILE DRAFT_FILE MMPROJ_FILE ALIAS BIND_ADDR
+          LISTEN_PORT CTX_SIZE PARALLEL BATCH_SIZE UBATCH_SIZE NGL FIT
+          FLASH_ATTN CACHE_TYPE_K CACHE_TYPE_V SPEC_TYPE SPEC_DRAFT_NGL
+          SPEC_DRAFT_DEVICE SPEC_DRAFT_N_MAX SPEC_DRAFT_N_MIN SPEC_DRAFT_P_MIN
+          TEMPERATURE TOP_P TOP_K MIN_P PRESENCE_PENALTY NUM_THREADS
+          THREAD_BATCH ROCM_PATH HIP_VISIBLE HSA_GFX_VERSION)
+_OVERRIDES=()
+for _var in "${TUNABLES[@]}"; do
+    [[ -n "${!_var+set}" ]] && _OVERRIDES+=("$_var=${!_var}")
+done
+ENV_FILE="${FRANK_LARGE_ENV_FILE:-$(dirname "$0")/frank-large.env}"
 if [[ -f "$ENV_FILE" ]]; then
    set -a
+    # shellcheck disable=SC1090
    source "$ENV_FILE"
    set +a
 fi
-
-# --- defaults (used when the env file does not set them) ------------------
-: "${RUNTIME_DIR:=/home/dank/GitPackages/gufo-v0.1.1}"
-: "${MODEL_DIR:=/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL}"
-: "${ALIAS:=frank-large}"
-: "${HOST:=0.0.0.0}"
-: "${PORT:=7770}"
-: "${CONTEXT_SIZE:=262144}"
-: "${SESSIONS:=1}"
-: "${DRAFT_TOKENS:=7}"
-: "${DRAFT_POLICY:=adaptive}"
-
-GUFO_BIN="$RUNTIME_DIR/bin/gufo"
-
-# The target GGUF and the drafter are named explicitly because they do not
-# share a directory any more: the target is a different repository's artifact
-# and keeps its own long filename, while the DFlash2 drafter is unchanged and
-# still lives beside the previous target. MODEL_FILE/DRAFTER_FILE default to
-# the deployed payload, so the env file only has to state them when it moves.
-: "${MODEL_FILE:=$MODEL_DIR/Qwen3.8-27B-UD-Q4_K_XL.gguf}"
-: "${DRAFTER_DIR:=/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL}"
-: "${DRAFTER_FILE:=$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf}"
-
-if [[ ! -x "$GUFO_BIN" ]]; then
-    echo "runtime not found at $GUFO_BIN" >&2
-    echo "frank-large: build it: bash scripts/build-gufo.sh" >&2
-    exit 1
-fi
-for f in "$MODEL_FILE" "$DRAFTER_FILE"; do
-    if [[ ! -f "$f" ]]; then
-        echo "missing asset $f (run the matching scripts/download-frank-large*.sh)" >&2
-        exit 1
-    fi
+for _kv in "${_OVERRIDES[@]+${_OVERRIDES[@]}}"; do
+    export "${_kv?}"
 done
+unset _var _kv

-# --- env -------------------------------------------------------------------
-# gufo links against the host ROCm through its RPATH (/opt/rocm ->
-# /etc/alternatives/rocm), so no LD_LIBRARY_PATH is needed. The device is
-# already gfx1151 — `gufo diagnose` reports it natively — so no
-# HSA_OVERRIDE_GFX_VERSION either; the override existed for the prebuilt
-# llama.cpp runtime and can only misreport the arch here.
-export HIP_VISIBLE_DEVICES=0
+: "${LLAMA_BIN=/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server}"
+: "${MODEL_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf}"
+: "${DRAFT_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mtp-Qwen3.8-27B-Q4_0.gguf}"
+: "${MMPROJ_FILE=/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mmproj-Qwen3.8-27B-BF16.gguf}"
+: "${ALIAS=frank-large}"
+: "${BIND_ADDR=0.0.0.0}"
+: "${LISTEN_PORT=7770}"
+: "${CTX_SIZE=262144}"
+: "${PARALLEL=1}"
+: "${BATCH_SIZE=2048}"
+: "${UBATCH_SIZE=512}"
+: "${NGL=999}"
+: "${FIT=off}"
+: "${FLASH_ATTN=on}"
+: "${CACHE_TYPE_K=f16}"
+: "${CACHE_TYPE_V=f16}"
+: "${SPEC_TYPE=draft-mtp}"
+: "${SPEC_DRAFT_NGL=99}"
+: "${SPEC_DRAFT_DEVICE=ROCm0}"
+: "${SPEC_DRAFT_N_MAX=4}"
+: "${SPEC_DRAFT_N_MIN=0}"
+: "${SPEC_DRAFT_P_MIN=0.0}"
+: "${TEMPERATURE=1.0}"
+: "${TOP_P=0.95}"
+: "${TOP_K=20}"
+: "${MIN_P=0.0}"
+: "${PRESENCE_PENALTY=0.0}"
+: "${NUM_THREADS=16}"
+: "${THREAD_BATCH=16}"
+: "${ROCM_PATH=/opt/rocm}"
+: "${HIP_VISIBLE=0}"
+: "${HSA_GFX_VERSION=11.5.1}"

-# --- the serve command ---------------------------------------------------
-# Server options precede the subcommand, model options follow it.
-exec "$GUFO_BIN" serve \
-    --host "$HOST" \
-    --port "$PORT" \
-    --sessions "$SESSIONS" \
-    llm \
+[[ -n "$LLAMA_BIN" ]] || { echo "frank-large: LLAMA_BIN is empty" >&2; exit 1; }
+[[ -x "$LLAMA_BIN" ]] || {
+    echo "frank-large: llama-server '$LLAMA_BIN' is not executable;" >&2
+    echo "           build it with scripts/build-rocmfpx.sh" >&2
+    exit 1
+}
+# Refuse anything but the real payload. A path that is not a readable file
+# makes llama-server hang rather than fail clean.
+[[ -r "$MODEL_FILE" ]] || {
+    echo "frank-large: model file '$MODEL_FILE' is not readable;" >&2
+    echo "           fetch it with scripts/download-frank-large-kingjones.sh" >&2
+    exit 1
+}
+# The MTP head is the performance feature: without it the Q8_0 weights decode
+# at memory-bandwidth speed and the endpoint loses the ~1.4x the head buys.
+[[ -r "$DRAFT_FILE" ]] || {
+    echo "frank-large: MTP draft head '$DRAFT_FILE' is not readable;" >&2
+    echo "           fetch it with scripts/download-frank-large-kingjones.sh" >&2
+    exit 1
+}
+# The GGUF carries no vision tower; the projector file is the whole tower, so
+# a missing file means a vision request would 500 rather than degrade.
+# MMPROJ_FILE="" is the documented opt-out for a text-only endpoint.
+MM_ARGS=()
+if [[ -n "${MMPROJ_FILE:-}" ]]; then
+    [[ -r "$MMPROJ_FILE" ]] || {
+        echo "frank-large: mmproj '$MMPROJ_FILE' is not readable; set MMPROJ_FILE='' for text-only" >&2
+        exit 1
+    }
+    MM_ARGS=(--mmproj "$MMPROJ_FILE")
+fi
+
+# HIP/ROCm host environment. HIP reads bare env vars (TEMP, HOST,
+# GPU_MAX_HEAP_SIZE), which is why the tunables above are named TEMPERATURE
+# and BIND_ADDR. HSA_OVERRIDE_GFX_VERSION is what lets a runtime built for a
+# neighbouring gfx target run on this iGPU; the qualified pin targets gfx1151
+# natively and the override is a no-op for it.
+export ROCM_PATH
+export HIP_VISIBLE_DEVICES="$HIP_VISIBLE"
+export HSA_OVERRIDE_GFX_VERSION="$HSA_GFX_VERSION"
+export LD_LIBRARY_PATH="$(dirname "$LLAMA_BIN"):${ROCM_PATH}/lib:${ROCM_PATH}/lib/rocm_sysdeps/lib:${ROCM_PATH}/lib/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+
+exec /usr/bin/env "$LLAMA_BIN" \
    --model "$MODEL_FILE" \
-    --served-model-name "$ALIAS" \
-    --context "$CONTEXT_SIZE" \
-    --speculative dflash2 \
-    --dflash-model "$DRAFTER_FILE" \
-    --draft-tokens "$DRAFT_TOKENS" \
-    --draft-policy "$DRAFT_POLICY" \
-    --temperature 0 \
-    --think auto \
-    --preserve-thinking on
+    "${MM_ARGS[@]}" \
+    --alias "$ALIAS" \
+    --host "$BIND_ADDR" \
+    --port "$LISTEN_PORT" \
+    --jinja \
+    --device ROCm0 \
+    --n-gpu-layers "$NGL" \
+    --parallel "$PARALLEL" \
+    --ctx-size "$CTX_SIZE" \
+    --batch-size "$BATCH_SIZE" \
+    --ubatch-size "$UBATCH_SIZE" \
+    --threads "$NUM_THREADS" \
+    --threads-batch "$THREAD_BATCH" \
+    --fit "$FIT" \
+    --flash-attn "$FLASH_ATTN" \
+    --direct-io \
+    --cache-type-k "$CACHE_TYPE_K" \
+    --cache-type-v "$CACHE_TYPE_V" \
+    --ctx-checkpoints 16 \
+    --cache-ram 8192 \
+    --cache-idle-slots \
+    --temp "$TEMPERATURE" \
+    --top-p "$TOP_P" \
+    --top-k "$TOP_K" \
+    --min-p "$MIN_P" \
+    --presence-penalty "$PRESENCE_PENALTY" \
+    --reasoning on \
+    --reasoning-format deepseek \
+    --reasoning-budget -1 \
+    --spec-type "$SPEC_TYPE" \
+    --model-draft "$DRAFT_FILE" \
+    --spec-draft-ngl "$SPEC_DRAFT_NGL" \
+    --spec-draft-device "$SPEC_DRAFT_DEVICE" \
+    --spec-draft-n-max "$SPEC_DRAFT_N_MAX" \
+    --spec-draft-n-min "$SPEC_DRAFT_N_MIN" \
+    --spec-draft-p-min "$SPEC_DRAFT_P_MIN" \
+    --metrics
@@ -11,14 +11,16 @@
 # every flag here was qualified by boot probe on 2026-09-29 — see
 # frank-primary.env for the qualification record).
 #
-# Shape (2026-10-01 bf16 cutover): MAXLEN 262144 x 3 sessions (786,432
-# tokens), KV UNQUANTIZED (--kv-cache-dtype auto = bf16; the fp8 q8/q8 shape
-# is retired to git tag frank-primary-fp8-mtp-20261001), KV_BYTES
-# 11,047,010,304 PER RANK (the engine measured 798,280 tokens of capacity /
-# 3.05x concurrency), NO speculative decode, --max-num-batched-tokens 1024
-# (Frank's lock), MoE gather geometry R4D_LRU_GATHER_CHUNKS=32 /
-# R4D_LRU_GATHER_LANES=64 (+18.7% decode, grows with depth), prefix caching,
-# expert prefetch OFF (36% precision measured), TunableOp off.
+# Shape (2026-10-02, Frank's correction of the 10-01 bf16 shape): MAXLEN
+# 200704 x SEQS 2 (401,408 tokens of contract), KV fp8 (--kv-cache-dtype
+# fp8; the bf16 unquantized shape is retired to git tag
+# frank-primary-fp8-mtp-20261001), KV_BYTES 3,150,000,000 PER RANK (the
+# engine measured 405,000 tokens of capacity / ~3.0x concurrency), NO
+# speculative decode, --max-num-batched-tokens 1024 (Frank's lock), MoE
+# gather geometry R4D_LRU_GATHER_CHUNKS=32 /
+# R4D_LRU_GATHER_LANES=64 (+18.7% decode, grows with depth), prefix
+# caching, expert prefetch OFF (36% precision measured), TunableOp off, and
+# the pool deliberately small so the expert arena keeps what it reclaims.
 #
 # The root unit (scripts/frank-primary.service, User=dank,
 # LimitMEMLOCK=infinity) grants the pin budget for the tiered-expert host
@@ -1,30 +1,25 @@
 #!/usr/bin/env bash
-# Serve letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound as
-# frank-small on Dankclaw's RTX 3090 via SGLang.
+# Serve bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF (Q5_K_M) as frank-small on
+# Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
 #
-# Topology: the whole checkpoint (12.0 GiB, INT8 W8A16 AutoRound) lives in
-# VRAM; the 3090's 24 GiB holds weights + a ~11.8 GiB fp8 KV pool at
-# --mem-fraction-static 0.98. The host CPU only tokenizes.
+# 2026-10-01 night (Frank): swapped from SGLang + INT8-W8A16-AutoRound
+# safetensors to the llama.cpp GGUF path, because llama.cpp is the only
+# runtime on this host that can serve the MiMo qwen35 hybrid with
+# ASYMMETRIC KV: --cache-type-k f16 (unquantized keys) + --cache-type-v
+# q8_0 (quantized values), which is Frank's explicit KV contract for this
+# endpoint. SGLang's --kv-cache-dtype only moves K and V together.
 #
 # Architecture: qwen3_5 hybrid — 32 layers, 8 full-attention + 24 Gated
-# DeltaNet, native 262144 window, and a vision tower (qwen3_5_vision)
-# baked into the safetensors: no separate projector file, the image path
-# is in the checkpoint. The config declares an MTP layer, but the shipped
-# weights carry none (no mtp.* tensors in the index), so this service runs
-# WITHOUT speculative decoding — a NEXTN draft would be an uninitialised
-# 4.24 GiB ghost model drafting at 0% acceptance.
+# DeltaNet, native 262144 window. KV lives only in the 8 attention layers;
+# the DeltaNet state is per-slot fixed size. Vision comes from the repo's
+# separate projector (the Q5_K_M GGUF carries no `v.*` tensors at all), so
+# --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is the documented
+# text-only opt-out.
 #
-# Context: CONTEXT_LENGTH x MAX_RUNNING slots must fit the fp8 KV pool,
-# which the engine autofits around the weights AND the DeltaNet ssm state
-# pool (see frank-small.env). The mamba pool defaults to ~5.1 GB, so it is
-# pinned (bf16, 32 slots). With the grid at its measured fit the card has
-# no headroom for prefill CUDA-graph capture, so prefill runs eager
-# (--disable-prefill-cuda-graph); decode graphs are untouched.
+# Sampling: llama.cpp takes it per request (the fleet harness sends its
+# recipe from models.yml); greedy is the harness default.
 #
-# Sampling: greedy by default (the fleet contract); clients that send their
-# own temperature get their own values.
-#
-# Tunables come from frank-small.env unless FRANK_SMALL_ENV_FILE overrides it.
+# Tunables come from frank-small.env unless FRANK_SMALL_ENV_FILE overrides.
 set -euo pipefail

 SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
@@ -37,61 +32,56 @@ if [[ -f "$ENV_FILE" ]]; then
 fi

 # --- defaults (used when the env file does not set them) -------------------
-: "${SGLANG_BIN:=/home/dank/GitPackages/sglang-cu128/bin/sglang}"
-: "${MODEL_DIR:=/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound}"
+: "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
+: "${MODEL_FILE:=/home/dank/models/MiMo-Q5KM/MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf}"
 : "${ALIAS:=frank-small}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-: "${MEM_FRACTION:=0.98}"
-: "${CONTEXT_LENGTH:=151552}"
-: "${MAX_TOTAL_TOKENS:=606208}"
-: "${MAX_RUNNING:=4}"
-: "${KV_DTYPE:=fp8_e4m3}"
-: "${QUANTIZATION:=auto-round}"
-: "${REASONING_PARSER:=qwen3}"
-: "${TOOLCALL_PARSER:=qwen3_coder}"
-: "${MAMBA_SSM_DTYPE:=bfloat16}"
-: "${MAX_MAMBA_CACHE_SIZE:=32}"
+# ctx-size is the POOL llama.cpp divides across slots: 540672 / 3 gives
+# 180224 per slot — three independent conversations (the native 262144 no
+# longer fits three times on a 24 GiB card once the vision tower is loaded;
+# frank-small.env carries the two measurements that set this number).
+: "${CTX_SIZE:=540672}"
+: "${PARALLEL:=3}"
+: "${CACHE_TYPE_K:=f16}"
+: "${CACHE_TYPE_V:=q8_0}"
+: "${NGL:=99}"
+# Vision projector. No colon on purpose: an explicitly empty value means
+# "text-only, on purpose" and must NOT be replaced by the default path.
+: "${MMPROJ_FILE=/home/dank/models/MiMo-Q5KM/mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf}"

-# Fail loudly rather than half-loading a model into VRAM: a missing file here
-# is a download that has not finished, and the unit's restart policy would
-# otherwise turn that into a crash loop that reads like a runtime fault.
-for path in "$SGLANG_BIN" "$MODEL_DIR/model.safetensors.index.json" \
-    "$(cd -- "$(dirname -- "$SGLANG_BIN")/.." && pwd)/cuda-nvcc/usr/local/cuda-13.0/bin/nvcc"; do
+# Fail loudly rather than half-loading: a missing file is a download that
+# has not finished, and Restart=always would turn it into a crash loop that
+# reads like a runtime fault.
+for path in "$LLAMA_BIN" "$MODEL_FILE"; do
    [[ -e "$path" ]] || { echo "frank-small: missing $path" >&2; exit 1; }
 done
-shard_count=$(find "$MODEL_DIR" -maxdepth 1 -name 'model-*.safetensors' | wc -l)
-if [[ "$shard_count" -ne 8 ]]; then
-    echo "frank-small: expected 8 safetensors shards in $MODEL_DIR, found $shard_count" >&2
-    exit 1
+EXTRA_ARGS=()
+if [[ -n "$MMPROJ_FILE" ]]; then
+    [[ -e "$MMPROJ_FILE" ]] || { echo "frank-small: missing $MMPROJ_FILE" >&2; exit 1; }
+    EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
 fi

 # The UVM device node is created lazily by nvidia-modprobe on the first CUDA
 # open after boot; make sure it exists before the server needs it.
 nvidia-modprobe -u -c=0 2>/dev/null || true

-# FlashInfer JITs its attention kernels with nvcc at boot. The host's
-# /usr/bin/nvcc is CUDA 12.4 and dies on `--compress-mode` (a 12.8+
-# option), so the venv carries a matching CUDA 13.0 nvcc (extracted from
-# the cuda-nvcc-13-0 deb); it must shadow the system one on PATH.
-export PATH="$(cd -- "$(dirname -- "$SGLANG_BIN")/.." && pwd)/cuda-nvcc/usr/local/cuda-13.0/bin:$PATH"
-
-# Sampling: SGLang has no server-wide sampling defaults; the harness sends
-# its per-model recipe from models.yml (compat.extraBody), and other clients
-# get the engine's own defaults.
-exec "$SGLANG_BIN" serve \
-    --model-path "$MODEL_DIR" \
-    --served-model-name "$ALIAS" \
-    --host "$BIND_ADDR" \
-    --port "$LISTEN_PORT" \
-    --mem-fraction-static "$MEM_FRACTION" \
-    --context-length "$CONTEXT_LENGTH" \
-    --max-running-requests "$MAX_RUNNING" \
-    --kv-cache-dtype "$KV_DTYPE" \
-    --quantization "$QUANTIZATION" \
-    --reasoning-parser "$REASONING_PARSER" \
-    --tool-call-parser "$TOOLCALL_PARSER" \
-    --mamba-ssm-dtype "$MAMBA_SSM_DTYPE" \
-    --max-mamba-cache-size "$MAX_MAMBA_CACHE_SIZE" \
-    --disable-prefill-cuda-graph \
-    --max-total-tokens "$MAX_TOTAL_TOKENS"
+# q8_0 on V REQUIRES flash attention (the quantized-value kernels only exist
+# on the FA path); an FA-less build silently falls back and the KV contract
+# in frank-small.env becomes a lie, so pin the flag pair here.
+ARGS=(
+    --model "$MODEL_FILE"
+    --alias "$ALIAS"
+    --host "$BIND_ADDR"
+    --port "$LISTEN_PORT"
+    --ctx-size "$CTX_SIZE"
+    --parallel "$PARALLEL"
+    --n-gpu-layers "$NGL"
+    --flash-attn on
+    --cache-type-k "$CACHE_TYPE_K"
+    --cache-type-v "$CACHE_TYPE_V"
+    --jinja
+    --reasoning auto
+    --reasoning-format deepseek
+)
+exec "$LLAMA_BIN" "${ARGS[@]}" ${EXTRA_ARGS[@]+"${EXTRA_ARGS[@]}"}
@@ -275,41 +275,48 @@ Depth caveat: every number above is at a near-empty KV. Deep-context
 behavior (the gather win growing with depth, bf16's decode cost at depth)
 is only sampled at 3 depths; a 64k-192k-depth probe is the flagged gap.

-### The deployed shape (2026-10-01: 5 x 262144, fp8, no spec)
+### The deployed shape (2026-10-02: 2 x 200704, fp8, pool cut for the arena)

 ```
-FRANK_PRIMARY_MAXLEN=262144    x SEQS 2 (524,288 tokens of contract)
-FRANK_PRIMARY_KV_BYTES=7300000000   PER RANK (bf16 = auto; 13,839 B/token)
-FRANK_PRIMARY_KV_DTYPE=auto   --kv-cache-dtype auto (bf16, unquantized)
-FRANK_PRIMARY_PRECISE_MEMORY=1  --expert-precise-memory (fitted overhead
-                                5.25 GiB instead of the 7.60 GiB default,
-                                so ~1,025 more arena slots)
+FRANK_PRIMARY_MAXLEN=200704    x SEQS 2 (401,408 tokens of contract)
+FRANK_PRIMARY_KV_BYTES=3150000000   PER RANK (fp8, 7,701 B/token)
+FRANK_PRIMARY_KV_DTYPE=fp8   --kv-cache-dtype fp8 (quantized)
 FRANK_PRIMARY_MNBT=1024       --max-num-batched-tokens 1024
 R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64  (container env)
 NO --speculative-config, NO --expert-prefetch
 ```

-Boot-probed 2026-10-01 (transient systemd units at the production uid +
+This replaced the 2026-10-01 shape — 2 x 262144 at bf16 KV with
+`--expert-precise-memory` — which crash-looped at graph capture: the arena is
+DEVICE-resident, the precise-memory flag drops the plan's overhead 7.60 ->
+5.25 GiB, and that shape had 0.13 GiB to spare (weights 6.28 + pool 6.80 +
+graphs 0.55 + arena 16.65 + non-torch ~1.45 = 31.73 of 31.86 GiB,
+NRestarts=3 at ~6.7 min each). The plain bf16 2 x 262144 boot also did not
+stay up: it left 0 bytes free at runtime and a real client request (995-token
+prompt, max_tokens=261149) killed the worker with `Tried to allocate 50.00 MiB
+... 0 bytes free`.
+
+The correction is a pool cut, not a downgrade. fp8 costs 7,701 B/token
+(measured 6,138 attention + 1,563 mamba/side, dtype-independent), so 2 x
+200704 = 401,408 tokens needs 3.091e9 B; pinned at 3.15e9, ~44 MB of margin
+above the floor — the same convention the 6.1e9 shape used. The arena is what
+remains: the planner sizes it AFTER the KV override, so the 6,248 slots of the
+6.1e9 shape become roughly 7,500 here. Two contexts at 200704 are a capacity
+contract, not a throughput one (2-way concurrency summed BELOW parity at 4064
+ctx), so the slot buys the VRAM headroom long generations need.
+
+Boot-probed 2026-10-02 (transient systemd units at the production uid +
 memlock, full-log capture; the rootless bench probe had masked the production
 memory environment). The night searched two concurrency contracts:

 | rung | result |
 |---|---|
-| 3 × 262144, bf16, KV 11,047,010,304 | **OOM at graph capture** — `Tried to allocate 76.00 MiB ... 0 bytes free`, 30.43 GiB of 31.86 GiB already allocated by PyTorch |
-| 3 × 245760, bf16 | **OOM identically** |
-| 3 × 237568, bf16 | PASS — KV 718,606 tokens |
-| **3 × 241664, bf16** | **PASS — KV 732,359 tokens, 3.03x** (max possible at 3 contexts) |
-| 6 × 241664, fp8 | **OOM** — tried 10.70 GiB against 9.18 GiB free: six full-length contexts do NOT fit |
-| 5 × 262144, fp8 | PASS on the first rung — KV 1,350,182 tokens, 5.15x |
-| 4 × 262144, fp8 | runtime-survivable stopgap — KV 1,079,582 tokens / 4.12x |
-| 3 × 262144, fp8, pool at the floor | KV 792,069 tokens / 3.02x; arena 6,248 slots; 3.59 GiB free VRAM measured |
-| **2 × 262144, bf16 + `--expert-precise-memory`** | **DEPLOYED** (Frank's call): unquantized KV at the full native window, two contexts; pool 7.3e9 B/rank (bf16 rate 13,839 B/token measured: 12,276 attention + 1,563 mamba/side, vs fp8's 6,138 + 1,563 = 7,701); the precise-memory flag drops the plan's overhead 7.60 -> 5.25 GiB so the arena grows from 6,248 to ~7,278 slots |
-
-Boot is NOT proof of runtime survivability: the five-slot shape passed every
-boot gate and still died on the first long-generation request. After any
-knife-edge shape change, send a request with a large `max_tokens` before
-calling it done. The deployed shape is **four contexts of the model's full
-native 262,144 window**. fp8 KV is what buys the capacity —
+| 3 x 262144, bf16, KV 11,047,010,304 | **OOM at graph capture** — `Tried to allocate 76.00 MiB ... 0 bytes free`, 30.43 GiB of 31.86 GiB already allocated by PyTorch |
+| 3 x 245760, bf16 | **OOM identically** |
+| 3 x 237568, bf16 | PASS — KV 718,606 tokens |
+| 2 x 262144, bf16 + `--expert-precise-memory` | **DEPLOYED then crashed** — 0.13 GiB free at boot, NRestarts=3 |
+| 2 x 262144, bf16 (no precise flag) | did not stay up either: 0 bytes free at runtime, worker killed by a real request |
+| **2 x 200704, fp8, pool cut for the arena** | **DEPLOYED (2026-10-02, Frank's correction)**: fp8 KV, two 200704 windows, ~7,500 arena slots, ~3.6 GiB free VRAM measured |
 7,926 B/token against bf16's ~13,900-14,047 — at the cost of bf16's ~4 %
 decode edge. Six full-length contexts cannot fit at any KV dtype: the 6-seq
 graph/activation footprint leaves only 9.18 GiB for the pool, and 6 × 262144
@@ -1,7 +1,7 @@
 ---
 type: reference
-title: "gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (DFlash2) on Dankclaw (frank-large)"
-description: "The active frank-large runbook — gufo (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source on Dankclaw and serving the abliterated ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL with the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU: the from-source build and its two host workarounds, the GGUF header evidence that made the payload swap safe, the one-session / full-window shape, why the fleet's Qwen-Sharp chat template CANNOT be used on this runtime and what that costs, the client-contract delta, the measured prefill/decode trade, and the failure modes worth knowing."
+title: "gufo, Qwen3.8-27B UD-Q4_K_XL (DFlash2) on Dankclaw (frank-large)"
+description: "The RETIRED frank-large runbook (rollback target) — gufo (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source on Dankclaw and serving unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL (vanilla) with the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU, ONE session at the full native 262,144 window with unquantized f16 KV: the from-source build and its two host workarounds, the GGUF header evidence, why the fleet's Qwen-Sharp chat template CANNOT be used on this runtime, the client-contract delta, the measured prefill/decode trade, and the failure modes worth knowing."
 tags:
  - llm
  - gufo
@@ -16,9 +16,20 @@ tags:
 timestamp: 2026-09-28
 ---

-# frank-large — gufo, Swift-1.5-Qwen3.8-27B-Uncensored UD-Q8_K_XL (DFlash2)
+# frank-large — gufo, Qwen3.8-27B (UD-Q4_K_XL, DFlash2)

-> **ACTIVE since 2026-09-28.** Replaces
+> **RETIRED 2026-10-02 — the rollback target.** Superseded by
+> [[frank-large-rocmfpx-q8agent-mtp]] (Q8_0_ROCMFPX_AGENT at 8.39 bpw with
+> MTP4, 17.0 t/s measured). This era was itself a return after one day of
+> [[frank-large-vllm-mimo-v26-bf16]] (battery decode 6.19-9.01 t/s; every
+> ≥6-bit vLLM quant path dead on gfx1151). The shape
+> moved on the way back: payload is now **unsloth/Qwen3.8-27B-GGUF
+> UD-Q4_K_XL** (16.35 GiB, vanilla — the abliterated Swift Q8 was
+> reclaimed), and the session grid is **one session at the full native
+> 262,144 window** (`SESSIONS=1 CONTEXT_SIZE=262144`, f16 KV). The
+> Swift-1.5 Q8 history below is retained for the engine/template law,
+> which still governs.
+> **Originally ACTIVE 2026-09-28.** Replaces
 > [[frank-large-unsloth-qwen38-27b-q8xl]] (stock llama.cpp, the unsloth
 > b11030 prebuilt), which had replaced
 > [[frank-large-kairic-edge-qwen38-27b]]. Same host, same port (7770), same
@@ -0,0 +1,219 @@
+---
+type: reference
+title: "frank-large on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP"
+description: "The active frank-large runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Q8_0_ROCMFPX_AGENT GGUF, a ftype-115 file no other loader can read, with the repo's own MTP-Q4_0 head drafting four tokens at one native 262,144 window, f16 KV, and the bundled BF16 projector for vision. Measured MTP-vs-none A/B, the client contract, and rollback to gufo."
+tags:
+  - llm
+  - frank-large
+  - dankclaw
+  - llama-cpp
+  - rocmfpx
+  - gfx1151
+  - gguf
+  - spec-decode
+  - mtp
+timestamp: 2026-10-02
+---
+
+# frank-large — ROCmFPX Q8_0_ROCMFPX_AGENT + MTP (gfx1151)
+
+> **ACTIVE since 2026-10-02.** Cut over from
+> [[frank-large-gufo-qwen38-27b]] (Q4_K_XL, ~15 t/s). Both requests came from
+> Frank: *highest quality that runs*, and *one full context*. This endpoint
+> delivers 8.39 bpw weights — two quant rungs above what gufo could load — and
+> measures **17.0 t/s** decode instead of 15, so quality was not bought with
+> speed.
+>
+> The capability that forced this whole shape is a GGUF tensor type:
+> **`GGML_TYPE_Q8_0_ROCMFPX` (103)**. It exists in exactly one loader on this
+> host, so the runtime was decided by the file, not by preference.
+
+## Identity
+
+| | |
+|---|---|
+| Host / device | Dankclaw (192.168.0.143), Radeon 8060S iGPU `gfx1151`, ~114 GiB GTT aperture |
+| Endpoint | `http://192.168.0.143:7770` — alias `frank-large`, `/v1` OpenAI surface |
+| Runtime | **`kingjones30/ROCmFPX`** llama.cpp fork @ `36e9acd40e10a87cd3c3ef8ec734668757dc8520`, built by [`scripts/build-rocmfpx.sh`](../../scripts/build-rocmfpx.sh) → `/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server` |
+| Unit | root systemd `frank-large.service` ([`scripts/frank-large.service`](../../scripts/frank-large.service)), `User=dank`, `Restart=always` |
+| Serve script | [`scripts/serve-frank-large.sh`](../../scripts/serve-frank-large.sh) (owns the argv), tunables [`scripts/frank-large.env`](../../scripts/frank-large.env) |
+| Weights | `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` — 28,214,425,952 B, sha256 `f3ef2bdbf56e244f…` |
+| Draft | `…/mtp-Qwen3.8-27B-Q4_0.gguf` — 1,680,271,648 B, sha256 `051a1764cff8c4f3…` |
+| Projector | `…/mmproj-Qwen3.8-27B-BF16.gguf` — 931,145,888 B, sha256 `de2a4986…` |
+| Fetch | [`scripts/download-frank-large-kingjones.sh`](../../scripts/download-frank-large-kingjones.sh) (`-f`, byte-count + LFS sha256 verified, `--verify-only`, token never on the command line) |
+| Seam | `tests/frank_large_unit.bats` (19 tests: payload gates, deployer/serve parity, spec block, KV dtype, sweep identity) |
+
+## Why this file, and why only this runtime
+
+`kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF` ships several rungs. Parsed
+headers of the two candidates:
+
+| File | ftype | Tensor mix | Output-head tensors at plain Q8_0 | Card MTP acceptance |
+|---|---|---|---|---|
+| `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` | **115** | 353 F32 + 187 Q8_0 + **311 type-103** | **187** | 0.953 |
+| `Qwen3.8-27B-Q8_0_ROCMFPX.gguf` | 111 | 2 Q8_0 + type-103 | 2 | 0.911 |
+
+Two consequences, both measured off the header rather than guessed:
+
+1. **Quality.** The AGENT routing variant keeps the output-head (and other
+   routing-sensitive) tensors at plain Q8_0 instead of folding them into the
+   ROCmFPX kernel layout — 187 vs 2 — so the logits the sampler reads are the
+   highest-precision tensors in the file. It is also the card's MTP-tuned
+   variant, so it is the *faster* drafter base at the same time.
+2. **The runtime is forced.** Type 103 is a private extension: stock llama.cpp
+   and gufo both fail at load (`unknown tensor type`). Verified on the host
+   before deploying — the bake-off was not a horse race, it was a lock-in
+   check.
+
+**The build identity matters as much as the fork.** The 2026-08-25 ROCmFPX
+builds read this file and then died in `ggml_compute_forward_set_rows` — the
+exact failure documented in [[kingjones-q8agent-rocmfpx]] when this payload was
+first attempted on Dankclaw and routed to Dankserver as
+[[frank-medium-rocmfpx]] instead. `scripts/build-rocmfpx.sh` therefore checks
+the commit out **by SHA** and refuses to build anything else; the pin is
+load-bearing, not tidiness.
+
+## Shape and budget
+
+One session, the model's full native window, unquantized KV:
+
+| Dial | Value | Why |
+|---|---|---|
+| `--ctx-size` | 262,144 | "one full context" — the native window, no rope scaling |
+| `--parallel` | 1 | one conversation, all memory to it |
+| KV dtype | **f16** (both K and V) | ~64 KiB/token → ~16 GiB; free inside a 114 GiB aperture, so there is no reason to pay a quant |
+| `--fit off` | on | the auto-fitter would silently resize the window we asked for |
+| `--flash-attn on` | on | required for the long-window attention path to fit |
+| `--direct-io` | on | bypasses page cache for the 26 GiB weight stream (mmap is disabled by it — deliberate) |
+| `--ctx-checkpoints 16` | 16 | slot state for checkpoint/rollback |
+| `--cache-ram 8192` + `--cache-idle-slots` | 8 GiB | prompt cache; verified a repeat pass over the same 6k prompt replays in **0.5 s** |
+| `--jinja` | on | without it `chat_template_kwargs` are silently ignored |
+| `--reasoning on --reasoning-format deepseek --reasoning-budget -1` | | thinking in its own channel, unbounded |
+
+Measured steady state (production, one model copy resident):
+`mem_info_gtt_used` **63.1 GiB** of the ~114 GiB aperture, `mem_info_vram_used`
+0.17 / 2.0 GiB (the iGPU's dedicated carve-out is a stub; everything real lives
+in GTT). llama-server RSS 2.1 GiB. Boot to `/health` 200: **16 s** — the fastest
+endpoint bring-up in the fleet, because there is no engine compile step.
+
+## Speculative decode — measured, not assumed
+
+```
+--spec-type draft-mtp --model-draft <mtp-Qwen3.8-27B-Q4_0.gguf>
+--spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0
+--spec-draft-device ROCm0 --spec-draft-ngl 99
+```
+
+MTP in llama.cpp is a property of the file, not of the flags: the head is a
+trained tensor set (`nextn_predict_layers = 1` in this repo's head, arch
+`qwen35`, 65 blocks) and the draft context runs on the main model's weights, so
+there is no second 26 GiB copy in memory. See the fleet law in
+`AGENTS.md → llama.cpp serving`.
+
+**A/B, 2026-10-02** — same session, temp 0, 2,202-token prompt, 512 tokens,
+`cache_prompt: false`, `frank-large.service` stopped for every scratch arm (the
+benchmarking law), baseline arm stripped of `--model-draft` **and** every
+`--spec-*` flag:
+
+| Arm | Decode t/s (2 runs) | Prefill t/s | Draft acceptance |
+|---|---|---|---|
+| **MTP, n-max 4 (deployed)** | **17.00 / 17.14** | 350.6 / 324.9 | 0.462 (mean acceptance length ≈ 1.86) |
+| no draft | 7.60 / 7.66 | 381.8 / 381.0 | — |
+
+**+123 % decode for an 8–15 % prefill tax.** Notes that matter before someone
+re-litigates this:
+
+- Acceptance here (0.462) is nowhere near the card's 0.953. The card measures
+  its own short, repetitive text; this is a 2.2k-token prose prompt. Treat the
+  card's number as a file property, not a throughput promise.
+- `n-max 4` is where the gain was measured; 2 and 8 are **unmeasured**. The
+  gufo-era law applies in spirit — never rank spec settings on a
+  self-predictable corpus.
+- End-to-end on the chat path is slower than the `/completion` number because
+  thinking tokens are included: a 2,253-token chat prompt generating 512
+  tokens took 43.8 s wall (11.7 t/s end-to-end) — that is the cost of an
+  unbounded reasoning budget, not a regression in decoding.
+
+## Client contract (live-verified 2026-10-02)
+
+| Behavior | Observation |
+|---|---|
+| Model id | `/v1/models` → `frank-large` |
+| Tools | native: `get_weather` → `{"city": "Seattle"}` in `tool_calls` |
+| Thinking | in **`reasoning_content`** (llama.cpp field — *not* vLLM's `reasoning`); `models.yml` sets `reasoningContentField` accordingly |
+| `reasoning_effort` | accepts **only** `xhigh`, `medium`, `low`; anything else is HTTP 500 (the template raises, it does not clamp). `models.yml` folds `high`/`max` → `xhigh` |
+| Sampling | server defaults = the card's thinking recipe (temp 1.0, top_p 0.95, top_k 20, min_p 0, presence 0), mirrored in the harness `extraBody` |
+| Vision | the BF16 projector loads (`loaded multimodal model`); a red square answers **`Red`** (`usage: 75 prompt / 71 cached`) |
+| Prompt cache | second pass over the same 6k prompt: 0.5 s wall |
+
+Three traps, all hit for real:
+
+1. **`content: null` when reasoning eats the budget.** With
+   `--reasoning-parser`-style separation, a request whose `max_tokens` is
+   consumed entirely by thinking returns `message.content = None`. A bench
+   client that indexes `content` crashes *after* measuring and loses the
+   numbers — fall back to `reasoning_content`, or size the budget (200 was
+   enough for the vision probe).
+2. **`MTP boundary missing for seq_id=… ; resyncing after a non-token step`**
+   appears in the log under draft resets. Self-recovering; not an outage signal.
+3. **Never `pkill -f llama-server` on Dankclaw** — frank-small is also a
+   `llama-server`. The sweep key here is the payload name:
+   `pkill -f 'Qwen3.8-27B-Q8_0_ROCMFPX_AGENT'`.
+
+## Operations
+
+```bash
+# deploy (planner selects this endpoint only via its own unit/env/serve trio)
+./scripts/deploy.sh dankclaw --service frank-large
+
+# status / logs / cause of a crash loop
+systemctl status frank-large
+journalctl -u frank-large -n 200 --no-pager
+
+# rebuild the pinned fork (only when deliberately moving the pin)
+scripts/build-rocmfpx.sh
+
+# re-verify the payload without re-downloading
+scripts/download-frank-large-kingjones.sh --verify-only
+```
+
+`serve-frank-large.sh` refuses to start — before touching the port — unless the
+fork binary, the weights, the MTP head and (unless `FRANK_LARGE_MMPROJ=""`) the
+projector all exist; `scripts/deploy-remote.sh` mirrors the same predicate so a
+broken payload never stops the running unit. Caller environment **overrides**
+`frank-large.env`, and defaults use `:=` so an explicitly empty value surfaces
+as the error it reads as.
+
+## Rollback
+
+git tag **`frank-large-gufo-q4kxl-20261001`**: gufo v0.1.1
+(`scripts/build-gufo.sh` still on disk), the Q4_K_XL Qwen3.8-27B payload and the
+DFlash2 drafter all remain on the host. Restore that tag's unit/env/serve trio
+and redeploy — nothing here deletes anything.
+
+The rollback payloads stay exactly where the tag's env file expects them:
+`/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL/` (16.35 GiB) and the DFlash2
+drafter `/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf`
+(1.07 GiB). The one-day vLLM MiMo era's payloads (MiMo-V2.6 BF16 18 GiB,
+Qwen3.5-9B MTP source 19 GiB, both DFlash draft dirs) were staged to
+`/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02 —
+recoverable until the trash is purged, and never part of a rollback plan.
+
+## When it misbehaves
+
+- **`unknown tensor type` / ftype 115 refused** — something moved off the
+  ROCmFPX binary. Check `readlink -f $(systemctl show frank-large -p ExecStart)`
+  and the fork commit; this file cannot run on stock llama.cpp or gufo.
+- **SIGSEGV in `ggml_compute_forward_set_rows` during load** — a fork build
+  older than the `36e9acd` pin; rebuild with `scripts/build-rocmfpx.sh`.
+  History: [[kingjones-q8agent-rocmfpx]].
+- **Slow decode with no log error** — check whether the draft is engaged
+  (`/metrics`, `draft_n` / `draft_n_accepted`; a zero draft count means the head
+  did not load), and check GTT pressure with
+  `cat /sys/class/drm/card1/device/mem_info_gtt_used` — a second resident copy
+  of this model (an orphaned scratch server) drives the whole host into
+  eviction and halves the rate silently.
+- **`500` naming Jinja** — a `chat_template_kwargs` field reached a server
+  without `--jinja`.
+- **HTTP 500 naming the template on the first request of a role** — an
+  unmapped `reasoning_effort`; fix the fold in `models.yml`, not the model.
@@ -0,0 +1,186 @@
+---
+type: reference
+title: "frank-large on Dankclaw — vLLM TheRock, MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (RETIRED)"
+description: "The RETIRED frank-large runbook (2026-10-01, one day) — the digest-pinned kyuz0 TheRock gfx1151 vLLM image serving XiaomiMiMo MiMo-V2.6-Distill-Qwen-9B in full BF16 on the Strix Halo iGPU. Kept for the measured record: the roof math, the cross-model MTP drafter, and the autoresearch finding that spec decode is NET-NEGATIVE on real chat+reasoning traffic here (6.19 t/s with MTP-2 vs 9.01 t/s without). Rolled back to [[frank-large-gufo-qwen38-27b]] by Frank the same night."
+tags:
+  - llm
+  - frank-large
+  - dankclaw
+  - vllm
+  - strix-halo
+  - mimo
+  - mtp
+  - spec-decode
+  - bf16
+timestamp: 2026-10-01
+---
+
+# frank-large — vLLM MiMo-V2.6-Distill-Qwen-9B BF16 (cross-model MTP)
+
+> **RETIRED 2026-10-01 night, one day active.** Frank's call after the
+> autoresearch loop: battery decode 6.19 t/s (MTP-2) / 9.01 t/s (no draft)
+> against gufo's ~15 t/s on the 27B — and every ≥6-bit quant path in vLLM
+> 0.30 is dead on gfx1151 (quark int8 rejected; all FP8 kernels CDNA-only;
+> torchao config unwired). Back to [[frank-large-gufo-qwen38-27b]]
+> (Q4_K_XL, 1 x 262144). The measured content below stands as the record.
+
+## Identity
+
+- **Endpoint:** `frank-large` at `192.168.0.143:7770` (Dankclaw).
+- **Runtime:** vLLM 0.30.0 in the **digest-pinned** `docker.io/kyuz0/vllm-therock-gfx1151`
+  image (image ID `2d24a8c5a3fd`, digest `sha256:98c1f3ac9833…` — ROCm 10.0.0 /
+  torch 2.11.0 / TheRock, Triton Gated DeltaNet kernels). The official vLLM
+  images are CUDA-only; this fork is the fleet's gfx1151 path. Plain
+  `podman run` driven by `scripts/serve-frank-large.sh` under a root unit
+  (`scripts/frank-large.service`, `User=dank`, `Restart=always`,
+  `ExecStop=podman stop -t 120`, `TimeoutStopSec=180` > the stop grace).
+- **Model:** `XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B` in **full BF16**
+  (17.53 GiB, 4 shards) — the same checkpoint frank-small serves as an INT8
+  W8A16 AutoRound conversion on the 3090, here unquantized.
+- **Draft:** `Qwen/Qwen3.5-9B` (18 GiB on disk; only the 15 `mtp.*` tensors
+  are loaded into the draft pass).
+
+## Why this shape (the roof math)
+
+Decode speed for a dense model on Strix Halo is bandwidth-bound:
+BF16 weights x one pass per token against the LPDDR5x aperture. Measured
+2026-10-01 on this host: **13.1 t/s is ~94% of the roof** — the payload, not
+a misconfiguration. The same INT8 payload reaches 83.9 t/s on the 3090
+because it moves ~12 GiB through ~936 GB/s. The only way past a bandwidth
+wall is generating more than one token per weight pass: **speculative decode
+is the performance feature of this endpoint**, not a garnish.
+
+The memory side is the opposite of a squeeze: KV grows ONLY on the 8 full
+attention layers (32 KiB/token bf16: 8 x 2 x 4 heads x 256 dim x 2 B);
+the 24 Gated DeltaNet layers hold a FIXED ~25 MiB state per running
+sequence. `--max-model-len 262144` (the native window) with
+`--max-num-seqs 4` at `--gpu-memory-utilization 0.45` boots clean — the
+four-session contract the gufo shape ran, with the pool shared by the
+scheduler instead of carved per session.
+
+## Spec method: cross-model MTP — measured, not assumed
+
+| arm | prefill | decode | acceptance |
+|---|---|---|---|
+| BF16 base | 17,876 t/s | 13.1 t/s | — |
+| BF16 + DFlash 8 tok | 7,976 t/s | 17.4 t/s | pos0 59% pos1 26% (token rate 14.8%) |
+| **BF16 + MTP 2 tok** | 7,380 t/s | **19.8 t/s** | pos0 71% pos1 44% (token rate 57.4%) |
+
+One session, temperature 0, 28,647-token prefill / 512-token decode, same
+client for all three arms (2026-10-01 bench window, frank-large stopped per
+Benchmarking Policy). **MTP wins (+51% vs +33%) and needs only 2 draft
+tokens** — acceptance past position 1 is 44% and falling; DFlash's own curve
+decays 59/26/14/7, so 8 proposals are mostly wasted verification.
+`num_speculative_tokens` beyond 2 is UNMEASURED; do not raise it without a
+fresh A/B. Both spec arms pay ~55–59% of prefill (the draft passes share
+the compute queues) — this endpoint's thinking-heavy traffic spends its wall
+clock in decode, which is the regime where that trade wins.
+
+**Why a Qwen head accepts on a MiMo target:** MiMo-V2.6 is a distillation of
+Qwen3.5-9B that preserves the hidden-state geometry — identical
+4096-hidden/248320-vocab/32-layer/interval-4 structure, and the 71%/44%
+per-position acceptance is the direct measurement that the distillation
+preserved more than geometry. MiMo itself ships **ZERO `mtp.*` tensors**
+(verified in both the BF16 base and the INT8 conversion) — which is exactly
+why frank-small's SGLang NEXTN draft is a 0.00-acceptance ghost: SGLang
+reads the head from the TARGET checkpoint, vLLM reads the draft from a
+separate path. That difference is the whole trick:
+
+```
+--speculative-config '{"method":"mtp","model":"/models/Qwen3.5-9B","num_speculative_tokens":2}'
+```
+
+The alternative drafter (`z-lab/Qwen3.5-9B-DFlash`, 6 layers, target_layer_ids
+[1,5,…,29]) stays fetched on disk; flip `FRANK_LARGE_SPEC=dflash` and repoint
+`FRANK_LARGE_SPEC_DIR` to use it. Its card's "vLLM PR #40898 unmerged" claim
+is FALSE — `vllm/v1/spec_decode/dflash.py` ships in 0.30.
+
+## The numerics trap (greedy text is NOT preserved by spec decode)
+
+Measured: temperature-0 completions DIFFER across arms (sha `1318aeb6` base,
+`0208b45d` dflash, `20c41c38` mtp). Batched verification changes the bf16
+reduction order, argmax near-ties diverge, and the divergence cascades.
+vLLM spec decode is distribution-preserving, not bitwise-greedy-preserving.
+The gufo-era "greedy == autoregressive text" law does not carry over; do not
+chase text drift between spec and no-spec arms as a regression.
+
+## Payload
+
+- Target: 4 safetensors shards (17.53 GiB) + config/tokenizer under
+  `/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-BF16/`, each shard
+  byte-count + LFS sha256-verified by
+  `old/download-frank-large-mimo.sh` (`--verify-only` re-checks; no token on
+  the curl command line).
+- MTP source: `old/download-frank-large-qwen35.sh` →
+  `/home/dank/models/Qwen3.5-9B/` (4 shards behind
+  `model.safetensors.index.json`; the shards carry the odd HF asset names
+  `model.safetensors-0000N-of-00004.safetensors` — the index resolves them,
+  the engine boots from exactly this tree).
+- DFlash drafter: `old/download-frank-large-dflash.sh` →
+  `/home/dank/models/Qwen3.5-9B-DFlash/`.
+- **Payload status 2026-10-02:** all three trees moved to
+  `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` (59 GiB staged with
+  the other retired frank payloads) and the fetch trio moved to `old/` — the
+  endpoint is two runtimes past this one, and re-fetch is cheaper than disk.
+- The serve script gates the image by REPO DIGEST (rootless podman resolves
+  tags from the network; the pin exists to boot exactly these bytes
+  offline), then config + index + exact shard count, then the spec draft
+  (either weight layout: single `model.safetensors` or a sharded index),
+  then `/dev/kfd` + whole `/dev/dri` (never individual renderD nodes —
+  Dankserver 2026-09-29 lesson).
+
+## Client contract
+
+- **Alias:** `frank-large` (the `id` at `/v1/models`).
+- **Reasoning:** `--reasoning-parser qwen3`; vLLM emits the thinking text in
+  **`reasoning`**, NOT llama.cpp/gufo's `reasoning_content` — the harness
+  `models.yml` compat block for this provider sets
+  `reasoningContentField: "reasoning"`. Miss that and thinking silently
+  vanishes from transcripts.
+- **Tools:** `--enable-auto-tool-choice --tool-call-parser qwen3_coder`.
+- **Vision:** the checkpoint carries the `qwen3_5_vision` tower but the
+  serve script passes `--language-model-only` — the endpoint stays
+  text-only by decision; dropping the flag brings vision back.
+- **Sampling:** request-side; the harness sends the Qwen3.5-family recipe
+  (temp 0.7, top_p 0.8) via `compat.extraBody`, and
+  `chat_template_kwargs.{enable_thinking,preserve_thinking}` — the
+  `chat_template_kwargs` form is what vLLM reads; a top-level
+  `preserve_thinking` is ignored there.
+- **Env hygiene:** every tunable is `FRANK_LARGE_`-prefixed because HIP
+  reads bare `TEMP`/`HOST`/`GPU_MAX_HEAP_SIZE` and the serve script exports
+  the env file with `set -a`.
+
+## Ops
+
+- Boot ~200–240 s warm; health 503 while loading (do not kill it).
+- The serve script sweeps stale container identities before starting:
+  `frank-large-mimo` itself and the bench-window container `vllm-mimo`.
+- Container death → `podman wait` returns non-zero from ExecStart →
+  `Restart=always` self-heals (a clean exit-0 drain also restarts, the
+  frank-primary lesson).
+- Bench windows: `systemctl stop frank-large` is enough (mask fails on the
+  relinked unit; the deploy restores).
+- First-deploy facts (2026-10-01, deploy #3): healthy after **230 s**;
+  boot log `GPU KV cache size: 819,332 tokens, Maximum concurrency for
+  262,144 tokens per request: 3.13x` — four sessions share the pool
+  (the scheduler, not a carve). Live battery green: alias `frank-large`,
+  reasoning split into `reasoning`, tool_calls with correct args, and
+  `/metrics` acceptance pos0 69% pos1 44% (54/86 draft tokens accepted),
+  matching the bench window.
+
+## Rollback
+
+Git tag **`frank-large-gufo-q4kxl-20261001`** — gufo v0.1.1 (built from
+source, `scripts/build-gufo.sh` stays on disk) + the Q4_K_XL Qwen3.8-27B
+payload + DFlash2 drafter, all still on the host. Restoring the tag's
+unit/env/serve trio and redeploying returns the 27B endpoint.
+
+# Citations
+
+- Model: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
+- Draft source: https://huggingface.co/Qwen/Qwen3.5-9B
+- DFlash drafter: https://huggingface.co/z-lab/Qwen3.5-9B-DFlash
+- Engine image: https://hub.docker.com/r/kyuz0/vllm-therock-gfx1151
+- Serve script: `scripts/serve-frank-large.sh`; tunables: `scripts/frank-large.env`
+- Predecessor: [[frank-large-gufo-qwen38-27b]]; sibling on the same arch:
+  [[frank-small-sglang-mimo-v26]]
@@ -0,0 +1,171 @@
+---
+type: reference
+title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF"
+description: "The active frank-small runbook — a CUDA (sm_86) build of llama.cpp master serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF on the RTX 3090: three conversations of 180,224 tokens, mmproj vision, the asymmetric KV contract (f16 K + q8_0 V), the two VRAM measurements that set the pool size, and what to check when it misbehaves."
+tags:
+  - llm
+  - frank-small
+  - dankclaw
+  - llama-cpp
+  - rtx3090
+  - mimo
+  - gguf
+  - kv-cache
+timestamp: 2026-10-02
+---
+
+# frank-small — llama.cpp MiMo-V2.6 Q5_K_M (RTX 3090)
+
+> **ACTIVE since 2026-10-02.** One quant step down from
+> [[frank-small-llama-mimo-q6k]] on the same runtime and the same KV
+> contract, then **resized twice the same day** on Frank's orders: first to
+> three contexts with vision, then down to the pool that keeps 2–3 GiB of the
+> card free. Frank's arithmetic request — "as much as can fit while keeping
+> overhead open space to 2-3gb on the 3090" — is answered by the two
+> measurements in *Shape and the KV contract* below; the answer is
+> **3 × 180,224**, not 3 × 262,144.
+
+> **The tradeoff is the per-slot window.** Three sessions cost 81,920 tokens
+> of per-conversation memory versus the two-slot shape: 180,224 tokens each
+> instead of the model's native 262,144. See *What three slots cost*.
+
+## Identity
+
+|Field|Value|
+|---|---|
+|Endpoint|`192.168.0.143:7771`, served alias `frank-small`|
+|Runtime|llama.cpp **master** (qwen35 model support), CUDA build `sm_86`, at `/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server` — unchanged across the quant swap|
+|Payload|`bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF` **Q5_K_M**, 6,876,124,704 B (6.40 GiB), sha256 `04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f` + the repo's `mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf`, 918,166,048 B, sha256 `ff348f3180a63188aa7285db85f550fe38acb61dd013c599eb8bad08d2cc2576` — both at `/home/dank/models/MiMo-Q5KM/`, fetched and byte- and sha256-verified by `scripts/download-frank-small.sh`|
+|Unit|root `frank-small.service` (repo `scripts/frank-small.service`, `User=dank`, relinked), env `scripts/frank-small.env`, serve `scripts/serve-frank-small.sh`|
+|Repo pins|`tests/frank_small_unit.bats` (stub-argv seam + the VRAM arithmetic), `deploy_llm_small` gates in `scripts/deploy-remote.sh`|
+
+## Shape and the KV contract
+
+`--ctx-size 540672 --parallel 3` — llama.cpp divides the pool across slots, so
+each conversation gets `n_ctx_slot = 180224` (boot log: `n_slots = 3,
+n_ctx_slot = 180224, kv_unified = 'false'`). `CTX_SIZE` MUST stay divisible by
+`PARALLEL`; the bats seam enforces it.
+
+The KV contract is Frank's: **K unquantized (f16), V `q8_0`** —
+`--cache-type-k f16 --cache-type-v q8_0`. Two hard dependencies ride on it:
+
+- **`--flash-attn on` is mandatory**: quantized V exists only on the FA path.
+  Without the flag the engine falls back silently and the contract becomes a
+  lie. The serve script pins the trio; the bats seam pins the script.
+- The VRAM budget: attention KV lives in the 8 full-attention layers only
+  (4 KV heads × 256 dim). K f16 ≈ 16 KiB/token, V q8_0 ≈ 8.5 KiB/token
+  (34 B per 32-element block) ⇒ ~24.5 KiB/token over the pool.
+
+**The pool size came from two boots, not from a formula.** `free`-style
+arithmetic on this stack is unreliable because the compute buffers, CUDA
+graphs and now the vision tower all bill separately, so the fit was measured
+at two shapes and solved between them:
+
+|Shape|Projector|`nvidia-smi` used|KV at 24.5 KiB/token|Non-KV (weights + buffers + projector)|
+|---|---|---|---|---|
+|2 × 262,144 (pool 524,288)|none|20,190 MiB|12,800 MiB|**7,390 MiB**|
+|3 × 196,608 (pool 589,824)|f16|22,972 MiB|14,113 MiB|**8,859 MiB**|
+
+The two rows price the projector and its image buffers at **~1.4 GiB** — not
+the ~0.9 GiB the 918 MB file suggests, because the vision encoder's own
+activation and compute buffers ride along. Solving the second row's non-KV cost
+against Frank's 2–3 GiB band: the card is 24,576 MiB, so
+`KV ≤ 24,576 − 9,216 − 2,048 = 13,312 MiB` ⇒ pool ≤ 556,630 tokens, rounded
+down to a multiple of 3 × 4,096 ⇒ **540,672**, i.e. 13,265 MiB of KV and
+**2,441 MiB free** (measured, 21,684 MiB used).
+
+**Why 3 × 262,144 was never possible.** Three native windows are 786,432
+tokens = 18.4 GiB of KV *alone*; plus the 8.86 GiB of non-KV cost that is
+27.3 GiB on a 24 GiB card. The quant step bought 0.86 GiB; the geometry needed
+6 GiB.
+
+## Measured on this shape (2026-10-02, 3 × 180,224 + vision)
+
+|Check|Result|
+|---|---|
+|Boot to healthy|~20 s, `n_slots = 3`, `n_ctx_slot = 180224`, `loaded multimodal model` logged|
+|VRAM|**21,684 MiB used / 2,441 MiB free** of 24,576 (`nvidia-smi`, steady state; the KV pool is allocated whole at boot, so this is the ceiling)|
+|Decode, temp 0, no thinking|**106.8 tok/s** (256-token completion at ~1,300-token depth)|
+|Prefill|**3,814 tok/s** (same request)|
+|Three concurrent requests|all three answered in 1.7 s wall — real parallel slots|
+|Fourth concurrent request|answered in 3.0 s while three ran — queued behind a released slot, as the slot ceiling requires|
+|Vision|`image_url` part with a 64×64 solid-red PNG → **"Red."**|
+|Tools|native `tool_calls` with correct arguments (`get_weather {"city":"Seattle","unit":"f"}`)|
+|Thinking|`reasoning_content` populated (145 chars on a "think first" prompt); `reasoning_effort: "low"` accepted|
+|Over-slot prompt|`400 exceed_context_size_error: request (184001 tokens) exceeds the available context size (180224 tokens)` — refused, never silently truncated|
+
+Decode is ~5 % **faster** than the 2 × 262,144 shape measured earlier the same
+day (100.4–101.8 tok/s): a shorter per-slot window means a shorter attention
+scan at equal prompt depth, and one less slot's DeltaNet state resident.
+
+## What three slots cost
+
+The per-slot window is the whole price. Two consequences the fleet depends on:
+
+- **A prompt over 180,224 tokens is a hard 400**, so `~/.omp/agent/models.yml`
+  sets `contextWindow: 180224` for this endpoint — the harness compacts before
+  it reaches the ceiling instead of discovering it.
+- **frank-primary's compaction prompt no longer fits with room to answer.**
+  Its trigger is 183,501 tokens — already above this slot — and its summary
+  budget is 32,768 output tokens. On the two-slot shape (262,144) that left
+  32.8k of output room, which is why that number was chosen; on this shape a
+  single compaction request must be truncated or split. Nothing in the fleet
+  routes compaction here automatically, but any runbook that claims frank-small
+  "fits a compaction request in ONE request" is describing the 262,144 shape.
+
+## Client contract
+
+- Thinking arrives in **`reasoning_content`** (`--reasoning-format deepseek`),
+  the field `models.yml` reads for this endpoint.
+- **Thinking is off unless the request asks**: the MiMo GGUF template defaults
+  `enable_thinking` false, so `models.yml` ships
+  `chat_template_kwargs: {enable_thinking: true, preserve_thinking: true}` in
+  `extraBody`. A raw client that omits it gets a direct answer — measured, and
+  not a fault. Trivial questions produce no thinking even with the flag on.
+- Tools: `--jinja` + server-side parsing, verified 2026-10-02.
+- **Vision is on**, through the projector file (`--mmproj`). Use the
+  OpenAI-standard `image_url` content part — that path answers correctly
+  ("Red." on a solid-red PNG). The legacy top-level `images` field is **not**
+  read by `/v1/chat/completions`: the same request sent that way answered
+  "Brown.", i.e. it generated from text alone. `/completion` takes `images`
+  but refuses a request without `messages` in this build, so chat is the only
+  vision path.
+- Setting `MMPROJ_FILE=` (explicitly empty) in `scripts/frank-small.env` drops
+  `--mmproj` and returns ~1.4 GiB of the card to KV — the documented text-only
+  opt-out. `MMPROJ_FILE` pointing at a file that does not exist is a boot
+  refusal, never a silent fallback to blindness.
+
+## No spec decode
+
+The GGUF has no MTP tensors (the same fact that made SGLang's NEXTN a 0.00
+ghost), and the cross-model DFlash draft
+(`/home/dank/models/DFlash-Qwen35/qwen3.5-9b-dflash-Q8_0.gguf`) is an
+optimization candidate, not a default — acceptance has to be measured on this
+traffic first. The bats seam fails if a draft flag ships unannounced.
+
+## Rollback
+
+|Target|How|
+|---|---|
+|Q6_K, same runtime|`scripts/frank-small.env` `MODEL_FILE=/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` (7,793,710,624 B, still on disk) and the deployer's `LLM_SMALL_MODEL_FILE` default. Q6_K needs ~1.4 GiB more than Q5_K_M, so pair the rollback with `CTX_SIZE` 49,152 tokens lower or `MMPROJ_FILE=`|
+|Two 262,144 windows|`CTX_SIZE=524288 PARALLEL=2` (measured 20,190 MiB used, 4,386 free) and `contextWindow: 262144` in `models.yml` — choose it when a single conversation needs the native window or when frank-primary's compaction request has to land here|
+|Vision off|`MMPROJ_FILE=` in the env file; the deployer gate then has to be satisfied too, so point `LLM_SMALL_MMPROJ` at any real projector file or drop the gate in the same commit|
+|SGLang + INT8|[[frank-small-sglang-mimo-v26]] — venv and old unit/env/serve trio still in place; the INT8 payload was staged to `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02, so restore it there or re-fetch with `old/download-frank-small-sglang-int8.sh`|
+
+## When it misbehaves
+
+- Port 7771 dark: `journalctl -u frank-small -n 50`; the serve script exits 1
+  naming the missing file if the binary or GGUF moved.
+- `No CUDA GPUs are available`: the 3090 vanished from the bus or the UVM node
+  is missing — `nvidia-modprobe -u -c=0` (the serve script runs it).
+- Requests queue behind each other: three concurrent long generations are the
+  slot ceiling by contract; a fourth waits for a slot to free (measured: 1.7 s
+  for the three, 3.0 s for the fourth).
+- `400 … exceeds the available context size (180224 tokens)`: the conversation
+  outgrew ONE slot. This is not pool exhaustion — the pool is 540,672 tokens
+  and the per-slot window is the binding limit. Shorten the prompt, compact, or
+  roll back to the 262,144 shape above.
+- Vision answers a color it never saw: the client sent `images` instead of an
+  `image_url` content part (see *Client contract*).
+- Slow decode after a CTX or quant change: check `nvidia-smi` — pushing KV out
+  of VRAM into unified memory halves the rate silently.
@@ -0,0 +1,110 @@
+---
+type: reference
+title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF (retired)"
+description: "The retired 2026-10-01 frank-small runbook — the first llama.cpp + GGUF shape on the RTX 3090: two native-window sessions, the asymmetric KV contract (f16 K + q8_0 V) that motivated the move off SGLang, the VRAM fit at Q6_K, and what to check when it misbehaves. Superseded the same night by the Q5_K_M payload."
+tags:
+  - llm
+  - frank-small
+  - dankclaw
+  - llama-cpp
+  - rtx3090
+  - mimo
+  - gguf
+  - kv-cache
+timestamp: 2026-10-01
+---
+
+# frank-small — llama.cpp MiMo-V2.6 Q6_K (RTX 3090)
+
+> **RETIRED 2026-10-01 → 10-02** (one night): superseded by
+> [[frank-small-llama-mimo-q5km]], the same runtime and shape one quant rung
+> down. The Q6_K file stays at `/home/dank/models/MiMo-Q6K/` as the rollback
+> target.
+>
+> Original order (Frank, 2026-10-01 night): move this endpoint to the GGUF
+> path for **asymmetric KV** — unquantized K + q8_0 V — which only llama.cpp
+> exposes per-tensor. That contract is what the current deployment kept.
+
+## Identity
+
+|Field|Value|
+|---|---|
+|Endpoint|`192.168.0.143:7771`, served alias `frank-small`|
+|Runtime|llama.cpp **master** (qwen35 model support), CUDA build `sm_86`, at `/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`|
+|Payload|`bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF` **Q6_K**, 7,793,710,624 B, sha256 `ef96d05a2ddf2cbb450d1af1ac3860ec769d3575bafa70692ee5609bda3fad7d`, at `/home/dank/models/MiMo-Q6K/`|
+|Unit|root `frank-small.service` (repo `scripts/frank-small.service`, `User=dank`, relinked), env `scripts/frank-small.env`, serve `scripts/serve-frank-small.sh`|
+|Repo pins|`tests/frank_small_unit.bats` (stub-argv seam), `deploy_llm_small` gates in `scripts/deploy-remote.sh`|
+
+## Shape and the KV contract
+
+`--ctx-size 524288 --parallel 2` — llama.cpp divides the pool across slots,
+so each session gets `n_ctx_slot = 262144`, the model's full native window
+(boot log confirms `n_slots = 2, n_ctx_slot = 262144, kv_unified = 'false'`).
+
+Frank's KV contract: **K unquantized (f16), V `q8_0`** —
+`--cache-type-k f16 --cache-type-v q8_0`. Two hard dependencies ride on it:
+
+- **`--flash-attn on` is mandatory**: quantized V exists only on the FA
+  path. Without the flag the engine falls back silently and the contract
+  becomes a lie. The serve script pins the trio; the bats seam pins the
+  script.
+- The VRAM budget: attention KV lives in the 8 full-attention layers only
+  (4 KV heads x 256 dim). K f16 ≈ 16 KiB/token, V q8_0 ≈ 8.5 KiB/token
+  (34 B per 32-element block) ⇒ ~24.5 KiB/token over the 524,288-token pool
+  ≈ **12.6 GiB**. Weights 7.8 + KV 12.6 + compute buffers/graphs ≈
+  **20.9 GiB measured** against 24.5 GiB — the "room for checkpoints" the
+  order asked for. If graph capture OOMs, halve `CTX_SIZE`; never quantize
+  K to fix it.
+
+## Client contract
+
+- Thinking arrives in **`reasoning_content`** (`--reasoning-format
+  deepseek`) — the same field SGLang's qwen3 parser used, so `models.yml`
+  `reasoningContentField` did NOT move. Live-verified 2026-10-01 22:57.
+- **Thinking is off unless the request asks**: the MiMo GGUF template
+  defaults `enable_thinking` false; `models.yml` now ships
+  `chat_template_kwargs: {enable_thinking: true, preserve_thinking: true}`
+  in `extraBody`. A raw client that omits it gets a no-thinking answer
+  (measured — this is not a fault).
+- Tools: `--jinja` + server-side tool parsing, verified
+  `tool_calls` round-trip 2026-10-01 23:0x.
+- **Text-only now.** The GGUF carries no vision projector; the
+  in-checkpoint tower existed only in the SGLang safetensors. `models.yml`
+  `input: [text]`.
+
+## No spec decode (yet)
+
+The GGUF has no MTP tensors (same fact that killed SGLang NEXTN), and the
+cross-model DFlash draft (`/home/dank/models/DFlash-Qwen35/qwen3.5-9b-dflash-Q8_0.gguf`,
+Q8_0) is an optimization-loop candidate, not a default — `--spec-type
+draft-dflash` needs acceptance measured on this traffic first. The bats
+seam fails if a draft flag ships unannounced.
+
+## Build
+
+No ccache, CUDA 12.4 system nvcc with `--gcc-install-dir`-free gcc-13 host
+compiler (nvcc 12.4 rejects gcc ≥ 14):
+
+```
+cmake -B build-cuda -S . -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \
+  -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13 \
+  -DLLAMA_CURL=OFF -DGGML_CCACHE=OFF
+cmake --build build-cuda -j 16 --target llama-server
+```
+
+The SGLang venv (`/home/dank/GitPackages/sglang-cu128`) and the INT8
+AutoRound payload (`/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound`)
+stay on disk as the rollback: repoint the unit's serve script and env from
+git history (`git show frank-large-gufo-q4kxl-20261001` era is BEFORE this;
+use the parent of the 2026-10-01 night swap commit).
+
+## When it misbehaves
+
+- Port 7771 dark: `journalctl -u frank-small -n 50`; the serve script
+  exits 1 naming the missing file if the binary or GGUF moved.
+- `No CUDA GPUs are available`: the 3090 vanished from the bus or the UVM
+  node is missing — `nvidia-modprobe -u -c=0` (the serve script runs it).
+- Requests hang at first token while `/health` is 200: both slots busy —
+  two concurrent long generations are the ceiling by contract.
+- Slow decode after a CTX change: check `nvidia-smi` — swapping KV out of
+  VRAM into unified memory halves the rate silently.
@@ -1,7 +1,7 @@
 ---
 type: reference
-title: "frank-small on Dankclaw — SGLang, MiMo-V2.6-Distill-Qwen-9B INT8 W8A16"
-description: "The active frank-small runbook — SGLang in a self-contained python3.12 venv serving leteschlead MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5 hybrid, 9B dense, in-checkpoint MTP + vision) on the RTX 3090: identity, payload, KV arithmetic, the NEXTN spec-decode shape, the CUDA 13 nvcc toolchain the JIT needs, deployed flags, the client contract, and what to check when it misbehaves."
+title: "frank-small on Dankclaw — SGLang, MiMo-V2.6-Distill-Qwen-9B INT8 W8A16 (RETIRED)"
+description: "The RETIRED frank-small runbook (2026-10-01, hours) — SGLang in a self-contained python3.12 venv serving leteschlead MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5 hybrid, 9B dense, in-checkpoint MTP + vision) on the RTX 3090: identity, payload, KV arithmetic, the NEXTN spec-decode ghost, the CUDA 13 nvcc toolchain the JIT needs, and the measured numbers kept for the record. Replaced by [[frank-small-llama-mimo-q6k]] the same night."
 tags:
  - llm
  - frank-small
@@ -17,9 +17,14 @@ timestamp: 2026-10-01

 # frank-small — SGLang MiMo-V2.6-Distill-Qwen-9B INT8 W8A16 (NEXTN MTP)

-> **ACTIVE since 2026-10-01.** Replaces [[frank-small-gemma4-12b]]
-> (llama.cpp Gemma4-12B QAT Q4_K_M, now the rollback target). Same port
-> (7771), same alias (`frank-small`), same client contract surface.
+> **RETIRED 2026-10-01, hours after cutover.** Replaced by
+> [[frank-small-llama-mimo-q6k]] — llama.cpp CUDA on the MiMo Q6_K GGUF,
+> chosen for the asymmetric KV (f16 K + q8_0 V) SGLang cannot express.
+> The venv (`/home/dank/GitPackages/sglang-cu128/`) stays in place; the INT8
+> payload was staged to
+> `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02 when
+> the llama.cpp endpoint was two runtimes deep — recoverable until the trash
+> is purged.

 ## Identity

@@ -47,7 +52,10 @@ timestamp: 2026-10-01
  trigger 183,501 + a 32,768-token summary budget = 216,269) now fits
  one request. A single request may grow to the whole pool — a
  170,019-token prompt was verified at the 4-slot shape — the grid is
-  the two-way concurrency contract.
+  the two-way concurrency contract. **That compaction arithmetic is
+  specific to a 262,144-token slot**: the ACTIVE shape runs three slots of
+  180,224, which 183,501 tokens already exceeds — see *What three slots
+  cost* in [[frank-small-llama-mimo-q5km]].
 - **KV:** `fp8_e4m3` — 16 KiB/token (8 attn layers x 2 x 4 heads x 256
  dim x 1 B), pool 9.31 GiB.
 - **The memory shape (the 4.5 GiB the naive math misses):** the DeltaNet
@@ -118,8 +126,11 @@ change lands.

 Eight safetensors shards (12,891,718,440 B total) + tokenizer and config
 files under `/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound/`,
-all LFS sha256-verified by `scripts/download-frank-small.sh` (`--verify-only`
-re-checks). The serve script refuses to start with a missing shard.
+all LFS sha256-verified by `old/download-frank-small-sglang-int8.sh`
+(`--verify-only` re-checks) — the script moved to `old/` when the endpoint
+cut over to llama.cpp + GGUF on 2026-10-01 night, and
+`scripts/download-frank-small.sh` now fetches the GGUF quant instead. The
+serve script refuses to start with a missing shard.

 ## Client contract

@@ -155,9 +166,12 @@ re-checks). The serve script refuses to start with a missing shard.

 ## Rollback

-[[frank-small-gemma4-12b]] — the llama.cpp Gemma4-12B QAT payload and binary
-remain intact on disk; its unit/env/serve trio is in `old/` after the
-cutover.
+[[frank-small-llama-mimo-q5km]] is the live shape two steps forward; the
+practical rollback today is [[frank-small-llama-mimo-q6k]] (payload at
+`/home/dank/models/MiMo-Q6K/`). This era's own rollback, the llama.cpp
+Gemma4-12B QAT payload, was staged to
+`/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/Gemma4-12B-QAT/` on
+2026-10-02; its unit/env/serve trio remains in `old/`.

 # Citations

@@ -1,6 +1,6 @@
 # LLM Fleet — Knowledge Base

-Documentation for the LLM inference fleet on the two hosts (2026-10-01):
+Documentation for the LLM inference fleet on the two hosts (2026-10-02):
 **Dankserver runs the tcclaviger RDNA4 vLLM fork on frank-primary** (the
 digest-pinned `tcclaviger/vllm` image driving `podman run` directly, serving
 orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 across both Radeon AI PRO R9700s
@@ -10,12 +10,12 @@ headroom; 6 do not fit), no speculative decode, MNBT 1024, MoE gather 32x64, uti
 reasoning/tool parsers wired (qwen3 / qwen3_xml) with preserve_thinking on,
 measured 36.4-39.4 tok/s decode — the 2026-10-01 bf16 no-spec cutover plus
 the night ladder; rollback frank-primary-fp8-mtp-20261001), and Dankclaw
-runs two endpoints —
-gufo v0.1.1 (the Strix Halo engine, built from source; frank-large,
-unsloth Qwen3.8-27B UD-Q4_K_XL (vanilla) + the z-lab DFlash2 Q4_K_M drafter
-on the gfx1151 iGPU, 4 x 163840 ctx with f16 KV, port 7770) and mainline llama.cpp (frank-small,
-HauhauCS Gemma4-12B QAT Q4_K_M with MTP and vision on the RTX 3090,
-port 7771).
+runs two endpoints — **ROCmFPX llama.cpp (frank-large, port 7770:
+kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF Q8_0_ROCMFPX_AGENT, a ftype-115
+file only that fork can read, with the repo's MTP head drafting 4 tokens, ONE
+session at 262,144, f16 KV, vision via the bundled projector)** and
+**llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q5_K_M
+GGUF, 2 x 262,144, asymmetric KV f16 K + q8_0 V, on the RTX 3090)**.

 > 2026-08-22: the former `vllmScripts` repository is merged into the
 > serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -39,14 +39,14 @@ fp8 KV, MNBT 1024, gather 32x64, qwen3 parsers]
    subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
        E2[RTX 3090 (PCIe x4 dock) +
 gfx1151 iGPU 114 GiB GTT]
-        E[stock llama.cpp (unsloth b11030, gfx1151)
+        E[ROCmFPX llama.cpp gfx1151, pinned fork build
 frank-large, port 7770]
-        F[Qwen3.8-27B UD-Q8_K_XL + DFlash2 Q4_K_M drafter
-262144 ctx / 1 slot, DFlash2 depth 4, q8_0 KV]
-        G[mainline llama.cpp 37b53fd
+        F[Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP-Q4_0 head
+1 x 262144 ctx, f16 KV, draft n-max 4, mmproj vision]
+        G[llama.cpp master CUDA sm_86
 frank-small, port 7771]
-        H[Gemma4-12B QAT Q4_K_M
-4 x 262144 ctx, MTP + vision + audio]
+        H[MiMo-V2.6 Q5_K_M GGUF
+2 x 262144 ctx, f16 K + q8_0 V, flash-attn]
        E2 --> E --> F
        E2 --> G --> H
    end
@@ -65,7 +65,23 @@ frank-small, port 7771]

 ## Active Deployment

- [[frank-primary-glm53-flash-w4a16]] — the ACTIVE frank-primary runbook
+- [[frank-large-rocmfpx-q8agent-mtp]] — **ACTIVE since 2026-10-02**
+  (root systemd unit, port 7770): the pinned kingjones30/ROCmFPX fork serving
+  the repo's highest-quality rung `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`
+  (26.28 GiB, ftype 115 = 187 plain-Q8_0 + 311 GGML_TYPE_Q8_0_ROCMFPX
+  tensors) with the bundled `mtp-Qwen3.8-27B-Q4_0.gguf` head drafting 4
+  tokens, one 262,144 window, f16 KV, the BF16 projector for vision. Why the
+  runtime is not interchangeable, the pinned-build set_rows history, the
+  measured envelope, rollback to gufo
+- [[frank-small-llama-mimo-q5km]] — **ACTIVE since 2026-10-02** (root
+  systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
+  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF Q5_K_M on the RTX 3090,
+  THREE slots of 180,224 tokens (pool 540,672) with K f16 + V q8_0 and the
+  mmproj for vision, ~100 tok/s decode, ~2.7 GiB of the card left free.
+  The KV arithmetic, the flash-attn dependency, rollback to Q6_K
+- [[frank-primary-glm53-flash-w4a16]] — RETIRED 2026-09-30 → 10-01 (the
+- failed GLM experiment; frank-primary runs Qwen FP8 again, see the intro):
+  (2026-09-30, root systemd unit, port 7770): the tcclaviger RDNA4 vLLM fork
  (2026-09-30, root systemd unit, port 7770): the tcclaviger RDNA4 vLLM fork
  (digest-pinned `tcclaviger/vllm@sha256:da4029666…`, plain `podman run`,
  no shim) serving Intel/GLM-5.3-Flash-W4A16-AutoRound — AutoRound symmetric
@@ -162,14 +178,24 @@ frank-small, port 7771]
  (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-gufo-qwen38-27b]] — ACTIVE frank-large runbook since
-  2026-09-28 (root systemd unit, port 7770): gufo (gufo-org/gufo) v0.1.1, the
-  Strix Halo inference engine, built from source on Dankclaw, serving
-  unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense) with the z-lab DFlash2
-  Q4_K_M drafter (adaptive block policy) on the gfx1151 iGPU — one full
-  262,144-token conversation, greedy, text-only, the two from-source build
-  workarounds, the client-contract delta, the measured prefill/decode trade
-  against llama.cpp, troubleshooting
+- [[frank-large-gufo-qwen38-27b]] — **RETIRED 2026-10-02 (the rollback
+  target)** (root systemd unit, port 7770): gufo
+  (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from
+  source on Dankclaw, serving unsloth/Qwen3.8-27B-GGUF **UD-Q4_K_XL**
+  with the z-lab DFlash2 Q4_K_M drafter — ONE session at the full native
+  262,144 window, unquantized f16 KV. Rolled back by Frank after one day
+  of [[frank-large-vllm-mimo-v26-bf16]]. The two from-source build
+  workarounds, the client-contract delta, the measured prefill/decode
+  trade against llama.cpp
+- [[frank-large-vllm-mimo-v26-bf16]] — RETIRED 2026-10-01 (one day;
+  rollback by Frank after the autoresearch loop): the digest-pinned
+  kyuz0/vllm-therock-gfx1151 image serving
+  XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B in FULL BF16, 4 x 262144 ctx,
+  CROSS-MODEL MTP from Qwen/Qwen3.5-9B — kept for the measured record:
+  battery decode 6.19 t/s (MTP-2) vs 9.01 t/s (no draft — spec NET-
+  NEGATIVE on real chat+reasoning traffic), the roof math, the greedy-text
+  numerics trap, and the death of every ≥6-bit vLLM quant path on gfx1151
+  (quark int8 rejected, FP8 kernels CDNA-only, torchao unwired)
 - [[frank-large-unsloth-qwen38-27b-q8xl]] — RETIRED 2026-09-28 (replaced by
  [[frank-large-gufo-qwen38-27b]]; rollback target): stock llama.cpp
  (unsloth b11030, ROCm gfx1151) serving
@@ -203,12 +229,27 @@ frank-small, port 7771]
  .hgn + quality overlay) on the gfx1151 iGPU — two 200k conversations over one
  400,384-position pool, deployed flags, memory model, client contract,
  troubleshooting (2026-09-16 → 09-19)
- [[frank-small-gemma4-12b]] — ACTIVE frank-small runbook since 2026-09-17
-  (root systemd unit, port 7771): mainline llama.cpp serving
+- [[frank-small-llama-mimo-q6k]] — **RETIRED 2026-10-01 → 10-02 (rollback
+  target)**: llama.cpp (master, CUDA sm_86 build) serving
+  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090 — TWO
+  native 262,144-token sessions with ASYMMETRIC KV (K f16 unquantized + V
+  q8_0, which forces flash-attn), ~20.9/24.6 GiB measured, text-only (the
+  GGUF has no vision), reasoning_content, no spec decode; payload still at
+  `/home/dank/models/MiMo-Q6K/`
+- [[frank-small-sglang-mimo-v26]] — RETIRED 2026-10-01 (hours; replaced
+  by [[frank-small-llama-mimo-q6k]] for the asymmetric KV SGLang cannot
+  express; venv still on disk, INT8 payload staged to
+  `~dank/.AGENTTRASH/20261002-retired-llm-payloads/`): SGLang in a
+  self-contained python3.12 venv serving leteschlead MiMo-V2.6 INT8
+  W8A16 AutoRound, 2 x 262144 ctx, fp8 KV, in-checkpoint vision,
+  83.9 tok/s decode
+- [[frank-small-gemma4-12b]] — RETIRED 2026-09-17 → 2026-10-01 (replaced by
+  [[frank-small-sglang-mimo-v26]] → the llama.cpp MiMo shape; payload staged
+  to `~dank/.AGENTTRASH/20261002-retired-llm-payloads/Gemma4-12B-QAT/`):
+  mainline llama.cpp serving
  HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced Q4_K_M on the RTX
  3090, with the separate MTP assistant sidecar chained behind ngram-mod,
-  q8_0 KV, and `--fit` sizing the full 262k window into ~12 GiB. A same-day
-  vLLM detour (2026-09-19) was reverted the same day; see below.
+  q8_0 KV, and `--fit` sizing the full 262k window into ~12 GiB.
 - [[frank-small-vllm-gemma4]] — RETIRED 2026-09-19 (same-day revert,
  rollback target): vLLM serving google/gemma-4-12B-it-qat-w4a16-ct (w4a16
  Marlin) with the bf16 MTP assistant on the RTX 3090 — bf16 KV, 289,647-
@@ -213,14 +213,27 @@ The LLM inference fleet on the two hosts. Full index:
  agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
  Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
  rollback record)
- [[frank-large-gufo-qwen38-27b]] — the ACTIVE frank-large runbook since
-  2026-09-28 (root systemd unit, port 7770): gufo (gufo-org/gufo) v0.1.1,
-  the Strix Halo inference engine, built from source on Dankclaw and
-  serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (a 27B dense model) with the
-  z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU — one 262,144-token
-  conversation, adaptive block policy, greedy, text-only, the two
-  from-source build workarounds, what changed in the client contract, the
-  measured prefill/decode trade against llama.cpp
+- [[frank-large-rocmfpx-q8agent-mtp]] — the ACTIVE frank-large runbook
+  since 2026-10-02 (root systemd unit, port 7770): the pinned kingjones30/
+  ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
+  Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF **Q8_0_ROCMFPX_AGENT** — ftype 115,
+  311 tensors of GGML_TYPE_Q8_0_ROCMFPX, a format no other loader on the
+  host reads — with the repo's Q4_0 MTP head drafting 4 tokens and its BF16
+  projector, one session at the native 262,144 window
+- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-10-01 → 10-02 (rollback
+  target, payload and `scripts/build-gufo.sh` both on disk): gufo v0.1.1
+  built from source serving Qwen3.8-27B UD-Q4_K_XL + the z-lab DFlash2
+  drafter on the gfx1151 iGPU — the two from-source build workarounds, the
+  template-hash allowlist, the measured prefill/decode trade
+- [[frank-large-vllm-mimo-v26-bf16]] — RETIRED 2026-10-01 (same day;
+  replaced by the gufo rollback, then by
+  [[frank-large-rocmfpx-q8agent-mtp]]): the digest-pinned
+  kyuz0/vllm-therock-gfx1151 image (vLLM 0.30, TheRock ROCm 10) serving
+  XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B in FULL BF16 on the gfx1151 iGPU,
+  4 x 262,144 ctx, with CROSS-MODEL MTP spec decode from Qwen/Qwen3.5-9B —
+  the roof math (13.1 t/s base is ~94% of LPDDR5x BF16), the measured
+  MTP-vs-DFlash table (19.8 vs 17.4 t/s), the greedy-text numerics trap,
+  the `reasoning` field change, troubleshooting
 - [[frank-large-unsloth-qwen38-27b-q8xl]] — RETIRED 2026-09-28 (replaced by
  [[frank-large-gufo-qwen38-27b]]; rollback target): stock llama.cpp
  (unsloth b11030, ROCm gfx1151) serving
@@ -269,12 +282,22 @@ The LLM inference fleet on the two hosts. Full index:
 - [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
 - [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
 - [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
- [[frank-small-sglang-mimo-v26]] — the ACTIVE frank-small runbook since
-  2026-10-01: SGLang in a self-contained python3.12 venv serving
+- [[frank-small-llama-mimo-q5km]] — the ACTIVE frank-small runbook since
+  2026-10-02: a CUDA (sm_86) build of llama.cpp master serving
+  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's RTX
+  3090, port 7771 — THREE conversations at 180,224 tokens each, the
+  asymmetric KV contract (f16 K + q8_0 V), vision through the repo's mmproj,
+  and the two VRAM measurements that set the pool size; ~100 tok/s decode
+- [[frank-small-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; replaced
+  by [[frank-small-llama-mimo-q5km]], payload still on disk): the first
+  llama.cpp + GGUF shape for this endpoint, Q6_K at the same
+  2 x 262,144 / f16-K + q8_0-V contract, 20.9 GiB measured
+- [[frank-small-sglang-mimo-v26]] — RETIRED 2026-10-01 (rollback target):
+  SGLang in a self-contained python3.12 venv serving
  letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5
  hybrid, INT8 W8A16, in-checkpoint vision, NO MTP weights shipped) on
  Dankclaw's RTX 3090, port 7771 — fp8 KV, pinned 606,208-token pool,
-  4 x 151,552 slots, bf16 mamba states, eager prefill
+  2 x 262,144 slots, bf16 mamba states, eager prefill
 - [[frank-small-gemma4-12b]] — RETIRED 2026-10-01 (rollback target):
  mainline llama.cpp serving
  HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M
@@ -875,3 +875,121 @@ any repo service.
  verified (KV cache size, concurrency, MNBT 1024, gather 32x64) plus live
  contract probes (reasoning split, tool_calls). Rollback target for the
  2026-09-29 fp8+MTP4 shape: git tag `frank-primary-fp8-mtp-20261001`.
+
+## 2026-10-01 — frank-large cutover: gufo -> vLLM TheRock, MiMo-V2.6 BF16 + cross-model MTP
+
+- **Question that started it:** can a Qwen3.5 MTP drafter attach to
+  frank-small, and how does vLLM run the FULL BF16 MiMo checkpoint? Answer 1:
+  not on frank-small — SGLang reads the MTP head from the TARGET checkpoint
+  and MiMo ships ZERO `mtp.*` tensors (base and INT8 alike). Answer 2: the
+  digest-pinned `kyuz0/vllm-therock-gfx1151` image (vLLM 0.30 / torch 2.11 /
+  TheRock ROCm 10) boots BF16 MiMo on the gfx1151 iGPU in ~200-240 s, and
+  vLLM reads the draft from a SEPARATE path — so cross-model MTP from
+  `Qwen/Qwen3.5-9B` (the distill's source, identical 4096/248320/32-layer
+  geometry) works. Frank's call: ship that as frank-large.
+- **Bench window (frank-large stopped, one session, temp 0, 28,647-token
+  prefill / 512 decode):** base 17,876 t/s prefill / 13.1 t/s decode (13.1
+  is ~94% of the LPDDR5x BF16 roof - decode here is bandwidth, not a bug);
+  +DFlash-8 7,976 / 17.4 (acceptance 59%/26% per position); +MTP-2 7,380 /
+  19.8 (71%/44%, token rate 57.4%). Deployed MTP-2: +51% decode. Both spec
+  arms pay ~55-59% of prefill. `num_speculative_tokens` beyond 2 unmeasured.
+- **Measured numerics law:** spec arms do NOT reproduce the no-spec greedy
+  text (bf16 batched verification reorders reductions; near-tie argmaxes
+  cascade). vLLM spec decode is distribution-preserving, not
+  bitwise-greedy-preserving.
+- **Client-contract delta wired:** vLLM emits `reasoning`, not
+  `reasoning_content` - `models.yml` frank-large block now sets
+  `reasoningContentField: "reasoning"`, and the extraBody moved to the
+  Qwen3.5-family recipe (0.7/0.8) with `chat_template_kwargs`
+  (a top-level `preserve_thinking` is ignored by vLLM).
+- Deployer gap found pre-deploy: the spec-draft gate demanded a single
+  `model.safetensors` (the dflash layout) and would have rejected the
+  deployed MTP arm (4 shards behind an index); fixed to accept both
+  layouts, gated by bats. `z-lab/Qwen3.5-9B-DFlash`'s "vLLM PR #40898
+  unmerged" card claim is FALSE - `vllm/v1/spec_decode/dflash.py` ships in
+  0.30 (verified in-image); the drafter stays on disk as the alternative.
+- Host state: +38 GiB fetched (MiMo BF16 17.53 GiB, Qwen3.5-9B 18 GiB,
+  DFlash 2.58 GiB), disk down to ~26 GiB free - reclaim pass due. gufo
+  rollback pinned at git tag `frank-large-gufo-q4kxl-20261001`; engine,
+  Q4_K_XL payload and DFlash2 drafter all remain on the host.
+- First deploy attempt #2 crash-looped the container 51 times: the serve
+  script handed vLLM the HOST model path while the tree is mounted at
+  `/models` inside the container — transformers rejected it as a malformed
+  repo id. Fixed by converting to the container path (the spec draft was
+  already converted; the target was not), pinned by bats. Also fixed the
+  deployer's own draft gate (same single-file bug as the serve script's).
+- Reclaim (Frank approved): Swift-1.5 Q8_K_XL payload (30 G, refetchable via
+  `scripts/download-frank-large-swift.sh`) and the orphaned flux-2-klein
+  tree under `models/checkpoints` (23 G; ComfyUI mounts
+  `comfyui-data/models`, not this) purged via
+  `~/.AGENTTRASH/20261001-frank-large-reclaim/` — 52 G freed. KEPT the
+  1.1 G `Qwen3.8-27B-DFlash2-Q4_K_M.gguf` (it is the gufo rollback's
+  drafter, not a remnant).
+
+## 2026-10-02 — frank-large cutover: vLLM MiMo -> ROCmFPX Q8_0_ROCMFPX_AGENT + MTP
+
+Frank's order was the highest quality of
+`kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF` on one full context, "whatever
+runtime it requires". The answer is forced by the file: `Q8_0_ROCMFPX_AGENT.gguf`
+carries tensor type 103 (`GGML_TYPE_Q8_0_ROCMFPX`, ftype 115), which stock
+llama.cpp, gufo and every other loader on the fleet cannot read — only the
+kingjones30/ROCmFPX fork can, so the runtime is pinned to commit
+`36e9acd40e10a87cd3c3ef8ec734668757dc8520` and built by
+`scripts/build-rocmfpx.sh`. Shape: 1 session x 262,144 (native), f16 KV, the
+bundled BF16 mmproj, and MTP from the repo's own `mtp-Qwen3.8-27B-Q4_0.gguf`
+(n-max 4, `--spec-draft-device ROCm0`).
+
+- **MTP is the whole performance story: 17.0-17.1 t/s with, 7.60-7.66 without**
+  (+123 %), one session, temp 0, production stopped per arm, acceptance 0.462 /
+  mean length ~1.86, at an 8-15 % prefill tax (325-351 vs 381 t/s). The AGENT
+  routing variant was chosen over the card's plain Q8_0 because it keeps 187
+  output-head tensors at plain Q8_0 instead of 2, which measured MTP acceptance
+  0.953 vs 0.911.
+- Steady GTT occupancy 63.1 GiB of the ~114 GiB aperture; boot to healthy 16 s.
+- Rollback: git tag `frank-large-gufo-q4kxl-20261001`. The retired vLLM-era
+  fetchers moved to `old/`. Runbook
+  [[frank-large-rocmfpx-q8agent-mtp]].
+
+## 2026-10-02 — frank-small: Q5_K_M, then three contexts + vision, sized by two boots
+
+Three orders in one day on the same endpoint: `bartowski/…Q5_K_M` on llama.cpp
+with two contexts ("whatever context size fits the 3090"), then **three
+contexts** "as much as can fit while keeping overhead open space to 2-3gb on
+the 3090", then **"frank-small should have vision"**.
+
+- **Final shape: `CTX_SIZE=540672 PARALLEL=3` = 3 x 180,224**, f16 K + q8_0 V
+  under flash attention, `--mmproj` vision, all layers on the 3090. Measured
+  **21,684 MiB used / 2,441 MiB free** of 24,576 — inside the 2-3 GiB band.
+- **The pool came from two boots, not a formula.** 2 x 262,144 with no
+  projector: 20,190 MiB used, KV 12,800 => non-KV **7,390 MiB**. 3 x 196,608
+  with the projector: 22,972 used, KV 14,113 => non-KV **8,859 MiB**. The
+  difference prices the vision tower at **~1.4 GiB for a 918 MB file** — the
+  encoder's own buffers ride with it. The first 3-slot deploy (3 x 196,608)
+  booted healthy but left only 1,153 MiB free, which is how the band got
+  violated and why the seam now prices non-KV at the measured 9 GiB and
+  REJECTS that shape.
+- **Three contexts cost window, and that is the tradeoff Frank was told:**
+  180,224 tokens per conversation, not the native 262,144. A longer prompt is
+  a hard `400 exceed_context_size_error`, so `models.yml` sets
+  `contextWindow: 180224`. Consequence: frank-primary's 183,501-token
+  compaction request no longer fits here with summary room; the two-slot
+  rollback is the way back.
+- Measured (2026-10-02, this shape): decode **106.8 t/s**, prefill **3,814
+  t/s** (~5 % faster decode than 2 x 262,144 — shorter slot, shorter attention
+  scan), three concurrent requests answered in parallel in 1.7 s with a fourth
+  queued at 3.0 s, native `tool_calls`, `reasoning_content` populated,
+  `reasoning_effort: low` accepted, solid-red PNG answered **"Red."**
+- **Vision contract gotcha:** `/v1/chat/completions` ignores the legacy
+  top-level `images` field — that request answered "Brown." for the same red
+  image, i.e. it generated from text alone. Only the `image_url` content part
+  reaches the model. `/completion` accepts `images` but this build still
+  requires `messages`.
+- Deployer gates on the projector too (`LLM_SMALL_MMPROJ`), so a missing
+  mmproj cannot silently ship a blind endpoint; `MMPROJ_FILE=` (explicitly
+  empty) in the env file is the documented text-only opt-out and returns ~1.4
+  GiB to KV.
+- Reclaim staged, **not purged, awaiting approval**:
+  `~/.AGENTTRASH/20261002-retired-llm-payloads/` (~59 GiB of superseded
+  payloads). Kept in place for rollback: `MiMo-Q6K/`, `MiMo-Q5KM/`,
+  `Qwen3.8-27B-UD-Q4_K_XL/` and `Qwen3.8-27B-UD-Q8_K_XL/` (the latter holds the
+  gufo DFlash2 drafter). Runbook [[frank-small-llama-mimo-q5km]].
@@ -0,0 +1,150 @@
+#!/usr/bin/env bats
+# Seam: autoresearch measure client behavior.
+#
+# The measure client is the whole yardstick of the frank-primary speed
+# session, so its METRIC arithmetic gets a behavior test: against a
+# deterministic mock endpoint that emits a KNOWN number of completion
+# tokens at a KNOWN per-token delay, the reported decode rate must land
+# near the nominal rate, the usage accounting must come back exact, and an
+# unhealthy endpoint must fail loudly (no half-measured numbers).
+#
+# The mock implements only what the client calls: POST
+# /v1/messages/count_tokens (word count as the token count, which makes the
+# client's words/token calibration come out to exactly 1.0) and a streaming
+# POST /v1/chat/completions (one content chunk per completion token, then a
+# usage chunk, then [DONE]).
+
+load helpers
+
+MEASURE="$REPO_ROOT/scripts/autoresearch/frank-primary-measure.py"
+MOCK="$BATS_TEST_TMPDIR/mock.py"
+PORT=""
+MOCK_PID=""
+
+setup() {
+    # Deterministic mock; argv: <port> <delay-seconds>; env MOCK_FAIL=1
+    # makes the chat endpoint answer 500.
+    cat >"$MOCK" <<'PYEOF'
+import json, os, sys, time
+from http.server import BaseHTTPRequestHandler, HTTPServer
+
+PORT = int(sys.argv[1])
+DELAY = float(sys.argv[2])
+FAIL = os.environ.get("MOCK_FAIL", "0") == "1"
+
+
+class Handler(BaseHTTPRequestHandler):
+    def log_message(self, *a):
+        pass
+
+    def _send(self, obj, code=200):
+        b = json.dumps(obj).encode()
+        self.send_response(code)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(b)))
+        self.end_headers()
+        self.wfile.write(b)
+
+    def do_POST(self):
+        n = int(self.headers.get("Content-Length", "0"))
+        body = json.loads(self.rfile.read(n) or b"{}")
+        if self.path == "/v1/messages/count_tokens":
+            text = body["messages"][0]["content"]
+            self._send({"input_tokens": max(1, len(text.split()))})
+        elif self.path == "/v1/chat/completions":
+            if FAIL:
+                self._send({"error": "engine down"}, 500)
+                return
+            max_tokens = int(body.get("max_tokens", 64))
+            self.send_response(200)
+            self.send_header("Content-Type", "text/event-stream")
+            self.end_headers()
+            for i in range(max_tokens):
+                ev = {"choices": [{"delta": {"content": "tok%d " % i}}]}
+                self.wfile.write(("data: %s\n\n" % json.dumps(ev)).encode())
+                self.wfile.flush()
+                time.sleep(DELAY)
+            prompt_words = max(1, len(body["messages"][0]["content"].split()))
+            ev = {"choices": [{"delta": {}}],
+                  "usage": {"prompt_tokens": prompt_words,
+                            "completion_tokens": max_tokens}}
+            self.wfile.write(("data: %s\n\n" % json.dumps(ev)).encode())
+            self.wfile.write(b"data: [DONE]\n\n")
+            self.wfile.flush()
+        else:
+            self._send({"error": "unknown path"}, 404)
+
+
+HTTPServer(("127.0.0.1", PORT), Handler).serve_forever()
+PYEOF
+}
+free_port() {
+    python3 -c 'import socket; s = socket.socket(); s.bind(("127.0.0.1", 0)); print(s.getsockname()[1]); s.close()'
+}
+
+start_mock() { # $1 = ok|fail
+    PORT="$(free_port)"
+    if [ "$1" = "fail" ]; then
+        MOCK_FAIL=1 python3 "$MOCK" "$PORT" 0.005 &
+    else
+        python3 "$MOCK" "$PORT" 0.005 &
+    fi
+    MOCK_PID=$!
+    local i
+    for i in $(seq 1 100); do
+        if python3 -c "import socket; socket.create_connection(('127.0.0.1', $PORT), 0.2).close()" 2>/dev/null; then
+            return 0
+        fi
+        sleep 0.1
+    done
+    echo "mock did not start" >&2
+    return 1
+}
+
+stop_mock() {
+    if [ -n "$MOCK_PID" ]; then
+        kill "$MOCK_PID" 2>/dev/null || true
+        wait "$MOCK_PID" 2>/dev/null || true
+        MOCK_PID=""
+    fi
+}
+
+@test "measure exists and compiles" {
+    [ -f "$MEASURE" ]
+    python3 -m py_compile "$MEASURE"
+}
+
+@test "measure computes decode rate and exact usage from a controlled endpoint" {
+    start_mock ok || return 1
+    run env AR_BASE="http://127.0.0.1:$PORT" AR_MODEL=mock python3 "$MEASURE"
+    stop_mock
+    [ "$status" -eq 0 ] || { echo "$output"; return 1; }
+    # run() merges stderr in; the contract is: exactly ONE json result line
+    # (the window script parses the same way), progress on stderr.
+    printf '%s\n' "$output" | python3 -c '
+import json, sys
+lines = [l for l in sys.stdin.read().splitlines() if l.startswith("{")]
+assert len(lines) == 1, "expected exactly one JSON line, got %d" % len(lines)
+d = json.loads(lines[0])
+# nominal 200 t/s (0.005 s/chunk) minus small per-chunk overhead
+for k in ("decode_tps", "decode_tps_115", "decode_tps_1056", "decode_tps_4096"):
+    v = d[k]
+    assert 60.0 <= v <= 320.0, "%s=%s outside [60, 320]" % (k, v)
+# the mock emits exactly max_tokens completion tokens; the accounting must be exact
+assert d["completion_tokens_4096"] == 256, d["completion_tokens_4096"]
+assert d["completion_tokens_115"] == 256
+# prompt sizing: calibration is exactly 1.0 words/token in the mock, so the
+# measured prompt lengths equal the targets
+assert d["prompt_tokens_115"] == 115
+assert d["prompt_tokens_4096"] == 4096
+print("measure claims verified")
+'
+}
+
+@test "measure fails loudly on an unhealthy endpoint instead of half-measuring" {
+    start_mock fail || return 1
+    run env AR_BASE="http://127.0.0.1:$PORT" AR_MODEL=mock python3 "$MEASURE"
+    stop_mock
+    [ "$status" -ne 0 ]
+    ! printf '%s\n' "$output" | grep -qE '^\{'
+}
@@ -1,338 +1,280 @@
 #!/usr/bin/env bats
-# Seam: the frank-large root unit on dankclaw.
+# Seam: frank-large serves kingjones777's Qwen3.8-27B Q8_0_ROCMFPX_AGENT (a
+# ftype-115 ROCmFPX file) with the repo's MTP draft head on the Strix Halo
+# iGPU, via the ROCmFPX fork of llama.cpp — one session at the native
+# 262,144 window.
 #
-# Since 2026-09-28 this endpoint is served by gufo (gufo-org/gufo), the
-# Strix Halo inference engine built from the pinned v0.1.1 tag, serving
-# Swift-1.5-Qwen3.8-27B-Uncensored (abliterated) UD-Q8_K_XL with the z-lab
-# DFlash2 Q4_K_M drafter. Two changes landed on this endpoint that day: the
-# engine (from the stock llama.cpp prebuilt) and the payload (from
-# unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL). Port, alias and client contract did
-# not move.
+# This seam exists because the format is the constraint: GGML_TYPE_Q8_0_ROCMFPX
+# (tensor type 103) is understood by the ROCmFPX fork and by nothing else on
+# this host, so the runtime, the payload and the deployer's gate have to agree
+# on paths that no other endpoint uses. On 2026-10-01 a serve-script gate and
+# its deployer twin disagreed during a cutover and the endpoint crash-looped;
+# both sides are pinned here.
 #
-# Most of this seam runs the serve script against a stub engine and asserts
-# on the argv it actually execs, so what is pinned is the command the
-# operator gets rather than the text of the script: a tunable renamed,
-# dropped, or silently defaulted away fails here the way it would fail a
-# deployment.
-load helpers
-
-ENV_FILE="$REPO_ROOT/scripts/frank-large.env"
-UNIT="$REPO_ROOT/scripts/frank-large.service"
-SERVE="$REPO_ROOT/scripts/serve-frank-large.sh"
-FETCH="$REPO_ROOT/scripts/download-frank-large-q4.sh"
-DRAFTER_FETCH="$REPO_ROOT/scripts/download-frank-large.sh"
-BUILD="$REPO_ROOT/scripts/build-gufo.sh"
-REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
-# The deployed target's filename, kept here so a rename in the env file and a
-# rename in this stub cannot drift apart unnoticed.
-TARGET_STUB="Qwen3.8-27B-UD-Q4_K_XL.gguf"
+# Everything is tested from what an operator or a supervisor can observe — the
+# command line, the exit code, the deployer's error text — never from anything
+# inside llama.cpp.
 setup() {
-    STAGE="$(mktemp -d)"
-    RUNTIME_DIR="$STAGE/gufo"
-    MODEL_DIR="$STAGE/models"
-    DRAFTER_DIR="$STAGE/draft"
-    mkdir -p "$RUNTIME_DIR/bin" "$MODEL_DIR" "$DRAFTER_DIR"
-    # A stub engine: the serve script execs it, so $@ is exactly what gufo
-    # would be invoked with.
-    cat > "$RUNTIME_DIR/bin/gufo" <<'STUB'
-#!/usr/bin/env bash
-printf '%s\n' "$@"
-STUB
-    chmod +x "$RUNTIME_DIR/bin/gufo"
-    # The target and the drafter live in SEPARATE directories since the
-    # 2026-09-28 payload swap: the target came from a different repository,
-    # the DFlash2 drafter did not move. A stub that colocated them would
-    # never notice DRAFTER_DIR going stale.
-    : > "$MODEL_DIR/$TARGET_STUB"
-    : > "$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
-    STUB_ENV="$STAGE/stub.env"
-    cat > "$STUB_ENV" <<EOF
-RUNTIME_DIR=$RUNTIME_DIR
-MODEL_DIR=$MODEL_DIR
-DRAFTER_DIR=$DRAFTER_DIR
-EOF
+    load helpers
+
+    SERVE="$REPO_ROOT/scripts/serve-frank-large.sh"
+    UNIT="$REPO_ROOT/scripts/frank-large.service"
+    ENV_FILE="$REPO_ROOT/scripts/frank-large.env"
+    DEPLOYER="$REPO_ROOT/scripts/deploy-remote.sh"
+    BUILD="$REPO_ROOT/scripts/build-rocmfpx.sh"
+    RUNTIME_STUB="/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server"
+    MODEL_STUB="/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf"
+    DRAFT_STUB="/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mtp-Qwen3.8-27B-Q4_0.gguf"
+    MMPROJ_STUB="/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/mmproj-Qwen3.8-27B-BF16.gguf"
+
+    TEST_TMP="$(mktemp -d)"
+    SRC="$TEST_TMP/src"
+    mkdir -p "$SRC"
+    git -C "$SRC" init -q
+    git -C "$SRC" -c user.email=t -c user.name=t commit -q --allow-empty -m x
 }

 teardown() {
-    if [ -n "${STAGE:-}" ]; then
-        rm -rf "$STAGE"
-    fi
+    rm -rf "$TEST_TMP"
 }

-# serve_argv [env-file] — run the serve script and capture the engine's argv,
-# one token per line, in $output. No argument exercises the built-in defaults
-# with the stub's paths.
-serve_argv() {
-    local envfile="${1:-$STUB_ENV}"
-    if [ -n "$envfile" ]; then
-        FRANK_LARGE_ENV_FILE="$envfile" run bash "$SERVE"
-    else
-        run env FRANK_LARGE_ENV_FILE="$STAGE/absent.env" bash "$SERVE"
-    fi
+# make_stub — an executable that prints its whole argv on one line, so
+# "what command line would the supervisor run" is observable output.
+make_stub() {
+    cat > "$TEST_TMP/bin" <<'STUB'
+#!/bin/sh
+printf '%s\n' "$*"
+STUB
+    chmod +x "$TEST_TMP/bin"
 }

-# argv_has <token> — exact token match, so --port never satisfies a test
-# looking for a different flag.
-argv_has() {
-    grep -qxF -- "$1" <<<"$output"
+@test "the env file defines the deployment values" {
+    grep -qE '^LLAMA_BIN=/' "$ENV_FILE"
+    grep -qE '^MODEL_FILE=/' "$ENV_FILE"
+    grep -qE '^DRAFT_FILE=/' "$ENV_FILE"
+    grep -qE '^ALIAS=[a-z0-9-]+$' "$ENV_FILE"
+    grep -qE '^LISTEN_PORT=[0-9]+$' "$ENV_FILE"
+    grep -qE '^CTX_SIZE=[0-9]+$' "$ENV_FILE"
 }

-# argv_value <flag> — the token following <flag>, empty if the flag is absent.
-argv_value() {
-    local i
-    i="$(grep -nx -- "$1" <<<"$output" | cut -d: -f1)"
-    [ -n "$i" ] || return 0
-    sed -n "$((i + 1))p" <<<"$output"
-}
-
-@test "the frank-large unit, env, serve and build files exist where the deployer installs them from" {
-    [ -f "$ENV_FILE" ]
-    [ -f "$UNIT" ]
-    [ -x "$SERVE" ]
-    [ -x "$BUILD" ]
-}
-
-@test "frank-large is boot-persistent, self-healing and unprivileged" {
-    grep -q '^WantedBy=multi-user.target$' "$UNIT"
-    grep -q '^Restart=on-failure$' "$UNIT"
-    grep -q '^RestartSec=' "$UNIT"
-    grep -q '^User=dank$' "$UNIT"
-    # The engine pins buffers; the memlock budget must be unbounded or the
-    # load fails on the RLIMIT.
-    grep -q '^LimitMEMLOCK=infinity$' "$UNIT"
-    # The unit runs the repo's own serve script with the repo's env.
-    grep -q 'ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh' "$UNIT"
-    grep -q 'FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env' "$UNIT"
-}
-
-@test "the fleet endpoint and alias survive the cutover" {
-    serve_argv
-    [ "$status" -eq 0 ]
-    # frank-large at :7770, whatever engine answers it.
-    [ "$(argv_value --served-model-name)" = "frank-large" ]
-    [ "$(argv_value --port)" = "7770" ]
-    [ "$(argv_value --host)" = "0.0.0.0" ]
-}
-
-@test "the built-in defaults are one session of the full 262144 window with DFlash2" {
-    serve_argv
-    [ "$status" -eq 0 ]
-    # 262144 is the model's native window (the operator's call, 2026-09-21).
-    [ "$(argv_value --context)" -eq 262144 ]
-    # One session is the single stream every prior frank-large served.
-    [ "$(argv_value --sessions)" -eq 1 ]
-    [ "$(argv_value --speculative)" = "dflash2" ]
-    # A draft ceiling of 0 would silently serve the autoregressive rate.
-    [ "$(argv_value --draft-tokens)" -gt 0 ]
-}
-
-@test "the shipped env file, not just the defaults, serves the deployed shape" {
-    # The defaults are a floor; systemd sources the env file. Only the
-    # on-host paths are redirected at the stub, so every fleet value in the
-    # shipped file is the one under test. MODEL_FILE and DRAFTER_FILE are
-    # absolute in the shipped file, so they need redirecting too — a sed that
-    # only rewrote MODEL_DIR would leave them pointing at /home/dank and the
-    # test would pass for the wrong reason (or fail on a box that has them).
-    sed -e "s#^RUNTIME_DIR=.*#RUNTIME_DIR=$RUNTIME_DIR#" \
-        -e "s#^MODEL_DIR=.*#MODEL_DIR=$MODEL_DIR#" \
-        -e "s#^MODEL_FILE=.*#MODEL_FILE=$MODEL_DIR/$TARGET_STUB#" \
-        -e "s#^DRAFTER_DIR=.*#DRAFTER_DIR=$DRAFTER_DIR#" \
-        -e "s#^DRAFTER_FILE=.*#DRAFTER_FILE=$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf#" \
-        "$ENV_FILE" > "$STAGE/shipped.env"
-    serve_argv "$STAGE/shipped.env"
-    [ "$status" -eq 0 ]
-    [ "$(argv_value --served-model-name)" = "frank-large" ]
-    [ "$(argv_value --host)" = "0.0.0.0" ]
-    [ "$(argv_value --port)" = "7770" ]
-    # 2026-10-01: four sessions x 163840 context (Frank's "4x 160k"), f16 KV.
-    # 163840 tokens x 64 KiB/token = 10 GiB of KV per session exactly.
-    [ "$(argv_value --context)" -eq 163840 ]
-    [ "$(argv_value --sessions)" -eq 4 ]
-    [ "$(argv_value --speculative)" = "dflash2" ]
-    [ "$(argv_value --draft-tokens)" -gt 0 ]
-    [ "$(argv_value --draft-policy)" = "adaptive" ]
-    # The shipped file must name the target the fleet actually serves, not a
-    # path that merely happens to exist.
-    [ "$(argv_value --model)" = "$MODEL_DIR/$TARGET_STUB" ]
-    [ "$(argv_value --dflash-model)" = "$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf" ]
-}
-
-@test "the engine is pointed at the payload that is actually on disk" {
-    serve_argv
-    [ "$status" -eq 0 ]
-    [ -f "$(argv_value --model)" ]
-    # A drafter path that does not exist would load the target and decode at
-    # the autoregressive rate, which reads as "the new runtime is slow".
-    [ -f "$(argv_value --dflash-model)" ]
-    [ "$(argv_value --model)" != "$(argv_value --dflash-model)" ]
-}
-
-@test "frank-large stays text-only" {
-    # The payload has no mmproj, so an --mmproj would point at nothing. The
-    # fleet's image path is frank-small: its SGLang MiMo payload carries the
-    # vision tower inside the checkpoint (no projector file at all), and
-    # models.yml declares frank-large text-only.
-    serve_argv
-    ! argv_has --mmproj
-    grep -q 'MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound' "$REPO_ROOT/scripts/frank-small.env"
-}
-
-@test "the engine is asked for greedy decoding and for preserved prior reasoning" {
-    # Greedy is what this endpoint has always served; a silent switch to
-    # sampled output would be a model-behaviour change disguised as a
-    # runtime swap. Prior thinking blocks must survive across turns or a
-    # multi-turn agent loses its own reasoning between tool calls.
-    serve_argv
-    [ "$(argv_value --temperature)" = "0" ]
-    [ "$(argv_value --preserve-thinking)" = "on" ]
-}
-
-@test "the serve script refuses a missing runtime with the build pointer" {
-    rm -f "$RUNTIME_DIR/bin/gufo"
-    serve_argv
+@test "the serve script fails when the runtime is missing" {
+    # Only the model, the head and the projector exist; the fork was never
+    # built. llama.cpp is not reached, so this fails fast and says what to
+    # run instead of failing at bind time.
+    touch "$TEST_TMP/model.gguf" "$TEST_TMP/draft.gguf" "$TEST_TMP/mmproj.gguf"
+    run env "LLAMA_BIN=$TEST_TMP/nope/llama-server" \
+        "MODEL_FILE=$TEST_TMP/model.gguf" "DRAFT_FILE=$TEST_TMP/draft.gguf" \
+        "MMPROJ_FILE=$TEST_TMP/mmproj.gguf" "$SERVE"
    [ "$status" -ne 0 ]
-    [[ "$output" == *"runtime not found"* ]]
-    [[ "$output" == *"build-gufo.sh"* ]]
-    # A restart loop would be the alternative; nothing may be exec'd.
-    ! argv_has serve
+    [[ "$output" == *"build it with scripts/build-rocmfpx.sh"* ]]
 }

-@test "the serve script refuses a missing drafter with a fetch pointer" {
-    # The drafter lives in its own directory now, so the target can be present
-    # and this still has to be caught.
-    rm -f "$DRAFTER_DIR/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
-    serve_argv
+@test "the serve script refuses to start without the model file" {
+    make_stub
+    touch "$TEST_TMP/draft.gguf" "$TEST_TMP/mmproj.gguf"
+    run env "LLAMA_BIN=$TEST_TMP/bin" "MODEL_FILE=$TEST_TMP/absent.gguf" \
+        "DRAFT_FILE=$TEST_TMP/draft.gguf" "MMPROJ_FILE=$TEST_TMP/mmproj.gguf" "$SERVE"
    [ "$status" -ne 0 ]
-    [[ "$output" == *"missing asset"* ]]
-    [[ "$output" == *"Qwen3.8-27B-DFlash2-Q4_K_M.gguf"* ]]
-    [[ "$output" == *"download-frank-large"* ]]
-    ! argv_has serve
+    [[ "$output" == *"is not readable"* ]]
+    [[ "$output" == *"download-frank-large-kingjones.sh"* ]]
 }

-@test "the serve script refuses a missing target and names the file" {
-    rm -f "$MODEL_DIR/$TARGET_STUB"
-    serve_argv
+@test "the serve script refuses to start without the MTP draft head" {
+    make_stub
+    touch "$TEST_TMP/model.gguf" "$TEST_TMP/mmproj.gguf"
+    run env "LLAMA_BIN=$TEST_TMP/bin" "MODEL_FILE=$TEST_TMP/model.gguf" \
+        "DRAFT_FILE=$TEST_TMP/absent.gguf" "MMPROJ_FILE=$TEST_TMP/mmproj.gguf" "$SERVE"
    [ "$status" -ne 0 ]
-    [[ "$output" == *"missing asset"* ]]
-    [[ "$output" == *"$TARGET_STUB"* ]]
+    [[ "$output" == *"MTP draft head"* ]]
 }

-@test "both payload files are fetched resumably and verified against declared sizes + sha256" {
-    # The target and the drafter come from DIFFERENT repositories, fetched by
-    # two scripts. Anchor the repo checks to the URL ASSIGNMENT, not to the
-    # file: the same repo string also appears in each script's header comment,
-    # so a plain grep stayed green after the URL was repointed at another
-    # repository -- the script would then download something the serve script
-    # never asked for.
-    grep -qE '^URL="https://huggingface\.co/unsloth/Qwen3\.8-27B-GGUF/resolve/main/' "$FETCH"
-    grep -qE 'https://huggingface\.co/z-lab/Qwen3\.8-27B-DFlash2-GGUF/resolve/main/' "$DRAFTER_FETCH"
-    for f in "$FETCH" "$DRAFTER_FETCH"; do
-        # -f so an HTTP error is a failure rather than a small "artifact";
-        # -C - so an interrupted 29 GiB run resumes.
-        grep -q -- '-fL -C -' "$f"
-        # The size and the sha256 are compared after the transfer, not trusted.
-        grep -q 'SHA256 MISMATCH' "$f"
-        grep -q 'SIZE MISMATCH' "$f"
+@test "the serve script refuses a missing mmproj unless vision is opted out" {
+    make_stub
+    touch "$TEST_TMP/model.gguf" "$TEST_TMP/draft.gguf"
+    run env "LLAMA_BIN=$TEST_TMP/bin" "MODEL_FILE=$TEST_TMP/model.gguf" \
+        "DRAFT_FILE=$TEST_TMP/draft.gguf" "MMPROJ_FILE=$TEST_TMP/absent.gguf" "$SERVE"
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"set MMPROJ_FILE='' for text-only"* ]]
+    # The documented opt-out must actually start, without a --mmproj argument.
+    run env "LLAMA_BIN=$TEST_TMP/bin" "MODEL_FILE=$TEST_TMP/model.gguf" \
+        "DRAFT_FILE=$TEST_TMP/draft.gguf" "MMPROJ_FILE=" "$SERVE"
+    [ "$status" -eq 0 ]
+    [[ "$output" != *"--mmproj"* ]]
+}
+
+@test "the serve script fails when LLAMA_BIN is empty" {
+    touch "$TEST_TMP/model.gguf" "$TEST_TMP/draft.gguf" "$TEST_TMP/mmproj.gguf"
+    run env LLAMA_BIN= "MODEL_FILE=$TEST_TMP/model.gguf" \
+        "DRAFT_FILE=$TEST_TMP/draft.gguf" "MMPROJ_FILE=$TEST_TMP/mmproj.gguf" "$SERVE"
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"LLAMA_BIN is empty"* ]]
+}
+
+@test "the shipped env points at the ROCmFPX runtime and the AGENT payload" {
+    grep -qE "^LLAMA_BIN=$RUNTIME_STUB$" "$ENV_FILE"
+    grep -qE "^MODEL_FILE=$MODEL_STUB$" "$ENV_FILE"
+    grep -qE "^DRAFT_FILE=$DRAFT_STUB$" "$ENV_FILE"
+    grep -qE "^MMPROJ_FILE=$MMPROJ_STUB$" "$ENV_FILE"
+    # HIP reads bare HOST/TEMP/GPU_MAX_HEAP_SIZE out of the environment that
+    # `set -a` creates; a tunable named HOST leaks into the runtime.
+    ! grep -qE '^(HOST|TEMP|GPU_MAX_HEAP_SIZE)=' "$ENV_FILE"
+}
+
+@test "the argv contract keeps the fleet's port and alias" {
+    make_stub
+    touch "$TEST_TMP/model.gguf" "$TEST_TMP/draft.gguf" "$TEST_TMP/mmproj.gguf"
+    run env "LLAMA_BIN=$TEST_TMP/bin" "MODEL_FILE=$TEST_TMP/model.gguf" \
+        "DRAFT_FILE=$TEST_TMP/draft.gguf" "MMPROJ_FILE=$TEST_TMP/mmproj.gguf" \
+        ALIAS=frank-large BIND_ADDR=127.0.0.1 LISTEN_PORT=7770 "$SERVE"
+    [ "$status" -eq 0 ]
+    # The caller's values must survive the env file, or a single-run operator
+    # override silently does nothing.
+    [[ "$output" == *"--host 127.0.0.1"* ]]
+    [[ "$output" == *"--port 7770"* ]]
+    [[ "$output" == *"--alias frank-large"* ]]
+    # Vision is wired: the projector is the whole vision tower.
+    [[ "$output" == *"--mmproj $TEST_TMP/mmproj.gguf"* ]]
+}
+
+@test "the shipped argv is one full native window on one session" {
+    grep -qE '^CTX_SIZE=262144$' "$ENV_FILE"
+    grep -qE '^PARALLEL=1$' "$ENV_FILE"
+    # 262144 IS the model's trained window (general.context_length in the
+    # GGUF), so 1 slot gets the whole thing — no truncation, no yarn scaling.
+    #
+    # Budget, from the GGUF header: 26.28 GiB of weights + 1.57 GiB draft head
+    # + 0.87 GiB projector + 16 full-attention layers x 4 kv heads x 256 dim x
+    # 2 (K+V) x 2 B = 64 KiB/token of f16 KV (~16 GiB at 262,144) + ~0.14 GiB
+    # of DeltaNet state + a couple GiB of compute buffers, against the iGPU's
+    # ~114 GiB GTT aperture. Change CTX_SIZE and this arithmetic, not a boot
+    # log, is what tells you it no longer fits.
+    ctx="$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
+    kv_gib=$((ctx * 65536 / 1073741824))
+    total_gib=$((27 + 2 + 1 + kv_gib + 3))
+    [ "$total_gib" -le 114 ]
+}
+
+@test "the MTP draft contract matches the model card's measured optimum" {
+    make_stub
+    touch "$TEST_TMP/model.gguf" "$TEST_TMP/draft.gguf" "$TEST_TMP/mmproj.gguf"
+    run env "LLAMA_BIN=$TEST_TMP/bin" "MODEL_FILE=$TEST_TMP/model.gguf" \
+        "DRAFT_FILE=$TEST_TMP/draft.gguf" "MMPROJ_FILE=$TEST_TMP/mmproj.gguf" "$SERVE"
+    [ "$status" -eq 0 ]
+    # The head is the model's own nextn layer, loaded as a separate draft
+    # model because this checkpoint ships it standalone (no blk.<n>.nextn.*
+    # tensors in the target file).
+    [[ "$output" == *"--spec-type draft-mtp"* ]]
+    [[ "$output" == *"--model-draft $TEST_TMP/draft.gguf"* ]]
+    # n-max 4 is the measured optimum; n-max 8 measured WORSE (acceptance
+    # fell to 0.757 and the extra verification cost outran the savings), so
+    # the flag is a measurement, not a wish.
+    [[ "$output" == *"--spec-draft-n-max 4"* ]]
+    [[ "$output" == *"--spec-draft-n-min 0"* ]]
+    [[ "$output" == *"--spec-draft-p-min 0.0"* ]]
+    # The draft runs on the iGPU beside the target: no device-to-device hop
+    # on a link that measures single-digit GB/s.
+    [[ "$output" == *"--spec-draft-device ROCm0"* ]]
+    [[ "$output" == *"--spec-draft-ngl 99"* ]]
+}
+
+@test "KV and placement stay unquantized with flash attention on" {
+    make_stub
+    touch "$TEST_TMP/model.gguf" "$TEST_TMP/draft.gguf" "$TEST_TMP/mmproj.gguf"
+    run env "LLAMA_BIN=$TEST_TMP/bin" "MODEL_FILE=$TEST_TMP/model.gguf" \
+        "DRAFT_FILE=$TEST_TMP/draft.gguf" "MMPROJ_FILE=$TEST_TMP/mmproj.gguf" "$SERVE"
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"--cache-type-k f16"* ]]
+    [[ "$output" == *"--cache-type-v f16"* ]]
+    # ROCmFPx quantized KV has no kernel on this branch, and the endpoint has
+    # the aperture to keep KV unquantized anyway.
+    [[ "$output" == *"--flash-attn on"* ]]
+    # --fit's placement heuristics do not know that 311 tensors of type 103
+    # belong on the iGPU, so the whole stack is placed explicitly instead.
+    [[ "$output" == *"--fit off"* ]]
+    [[ "$output" == *"--n-gpu-layers 999"* ]]
+}
+
+@test "the client contract wires thinking and template kwargs" {
+    make_stub
+    touch "$TEST_TMP/model.gguf" "$TEST_TMP/draft.gguf" "$TEST_TMP/mmproj.gguf"
+    run env "LLAMA_BIN=$TEST_TMP/bin" "MODEL_FILE=$TEST_TMP/model.gguf" \
+        "DRAFT_FILE=$TEST_TMP/draft.gguf" "MMPROJ_FILE=$TEST_TMP/mmproj.gguf" "$SERVE"
+    [ "$status" -eq 0 ]
+    # Without --jinja, llama.cpp silently ignores the harness's
+    # chat_template_kwargs, so thinking control disappears from the API.
+    [[ "$output" == *"--jinja"* ]]
+    # The harness reads `reasoning_content` for this endpoint, which is what
+    # the deepseek emission format produces (vLLM's `reasoning` is not it).
+    [[ "$output" == *"--reasoning-format deepseek"* ]]
+    # The template needs an unbounded thinking budget.
+    [[ "$output" == *"--reasoning-budget -1"* ]]
+}
+
+@test "the deployer gates exactly the paths the serve script opens" {
+    # A gate that checks a different file than the engine opens is the
+    # 2026-10-01 crash-loop: deploy passed, the unit restart-looped. Read the
+    # engine's own paths out of the env file and require the deployer's
+    # defaults to name the same ones.
+    for pair in "LLAMA_BIN:LLM_LARGE_RUNTIME" \
+                "MODEL_FILE:LLM_LARGE_MODEL" \
+                "DRAFT_FILE:LLM_LARGE_DRAFT" \
+                "MMPROJ_FILE:LLM_LARGE_MMPROJ"; do
+        env_key="${pair%%:*}"
+        dep_key="${pair##*:}"
+        want="$(sed -nE "s|^${env_key}=(.*)\$|\1|p" "$ENV_FILE")"
+        [ -n "$want" ]
+        got="$(grep -oE "${dep_key}:-[^}]+" "$DEPLOYER" | head -1 | sed "s|^${dep_key}:-||")"
+        [ "$got" = "$want" ]
    done
 }

-@test "the serve script and the fetch scripts agree on the payload filenames" {
-    # The serve script's MODEL_FILE and the fetch script's REL are two
-    # independent spellings of one filename, and a repo is not required to
-    # keep them in step: a re-upload under a new name, or an env edited
-    # without the fetch, leaves the endpoint unable to start with a
-    # "missing asset" that no test would have predicted. Bind them here.
-    local served fetched drafter_served drafter_fetched
-    served="$(basename "$(grep '^MODEL_FILE=' "$ENV_FILE" | cut -d= -f2-)")"
-    fetched="$(sed -n 's/^REL="\(.*\)"$/\1/p' "$FETCH")"
-    [ -n "$served" ]
-    [ "$served" = "$fetched" ]
-    drafter_served="$(basename "$(grep '^DRAFTER_FILE=' "$ENV_FILE" | cut -d= -f2-)")"
-    drafter_fetched="$(sed -n 's/^ *"\(Qwen3\.8-27B-DFlash2[^ ]*\.gguf\) .*/\1/p' "$DRAFTER_FETCH")"
-    [ -n "$drafter_served" ]
-    [ "$drafter_served" = "$drafter_fetched" ]
+@test "the deployer points a missing runtime at the fork build script" {
+    grep -qE 'build it: bash scripts/build-rocmfpx\.sh' "$DEPLOYER"
+    grep -qE 'fetch it: bash scripts/download-frank-large-kingjones\.sh' "$DEPLOYER"
 }

-@test "the drafter is still fetched by the script that carries its Hub repo" {
-    # Guards a future "simplification" that folds the drafter into the
-    # payload-fetch script and silently orphans the artifact the serve script
-    # still points at via DRAFTER_DIR. The literal drafter name must not
-    # appear in that script AT ALL, comments included.
-    grep -q 'Qwen3.8-27B-UD-Q8_K_XL' "$DRAFTER_FETCH"
-    ! grep -q 'DFlash2' "$FETCH"
+@test "the stray-process sweep cannot take frank-small down with frank-large" {
+    # frank-small serves a different GGUF over llama-server on the SAME host.
+    # A sweep keyed on the binary name (`pkill -f llama-server`) would kill
+    # the 3090 endpoint too, so it is keyed on frank-large's own payload file.
+    grep -q "pkill -f 'Qwen3.8-27B-Q8_0_ROCMFPX_AGENT'" "$DEPLOYER"
+    ! grep -qE "pkill -f 'llama-server'" "$DEPLOYER"
 }

@test "the build script refuses a source tree that is not the pinned commit" {
-    # gufo ships no prebuilt Linux binaries, so the runtime is built here.
-    # A tag that moves under us would silently install different kernels
-    # under the same RUNTIME_DIR, so the pin is enforced, not documented.
-    local src="$STAGE/not-the-pinned-commit"
-    mkdir -p "$src"
-    git -C "$src" init -q
-    git -C "$src" -c user.email=t@t -c user.name=t commit -q --allow-empty -m drift
-    run env GUFO_SRC_DIR="$src" GUFO_PREFIX="$STAGE/absent-prefix" bash "$BUILD" --verify
+    # The pin is load-bearing: the older builds of this fork line segfault in
+    # ggml_compute_forward_set_rows on this exact payload. A runtime that
+    # differs from what was qualified must be heard about at deploy time.
+    wrong="$(git -C "$SRC" rev-parse HEAD)"
+    run env ROCMFPX_SRC_DIR="$SRC" ROCMFPX_COMMIT="ffff$wrong" \
+        "$BUILD" --verify
    [ "$status" -ne 0 ]
-    [[ "$output" == *"GUFO_COMMIT mismatch"* ]]
+    [[ "$output" == *"ROCMFPX_COMMIT"*mismatch* ]]
 }

-@test "the build script reports a runtime that was never built" {
-    # --verify on a host with no install must say so rather than exit 0 on
-    # an empty check.
-    run env GUFO_SRC_DIR="$STAGE/no-source-tree" GUFO_PREFIX="$STAGE/absent-prefix" bash "$BUILD" --verify
-    [ "$status" -ne 0 ]
-    [[ "$output" == *"no gufo at"* ]]
+@test "the build script pins a real commit and the gfx1151 target" {
+    grep -qE 'ROCMFPX_COMMIT=.*[0-9a-f]{40}' "$BUILD"
+    grep -qE 'GPU_TARGET=.*gfx1151' "$BUILD"
+    grep -qE 'DGPU_TARGETS=' "$BUILD"
+    # The webui asset fetch half-failed on this box and aborted the link at
+    # 98%; the build must not depend on it.
+    grep -qE 'DLLAMA_BUILD_WEBUI=OFF' "$BUILD"
 }

-@test "the deploy handler gates on the runtime, the payload and the port" {
-    grep -q 'relink_unit frank-large' "$REMOTE_DEPLOY"
-    grep -q 'systemctl enable frank-large.service' "$REMOTE_DEPLOY"
-    grep -q 'wait_port_free 7770' "$REMOTE_DEPLOY"
-    grep -q '127.0.0.1:7770/health' "$REMOTE_DEPLOY"
-    grep -q 'LLM_LARGE_RUNTIME' "$REMOTE_DEPLOY"
-    grep -q 'LLM_LARGE_MODEL' "$REMOTE_DEPLOY"
-    grep -q 'LLM_LARGE_DRAFT' "$REMOTE_DEPLOY"
+@test "the deployed unit keeps the fleet's restart and boot-persistence shape" {
+    grep -qE '^Restart=(always|on-failure)$' "$UNIT"
+    grep -qE '^WantedBy=' "$UNIT"
+    grep -qE '^ExecStart=.*/serve-frank-large\.sh$' "$UNIT"
+    grep -qE '^User=dank$' "$UNIT"
+    # The payload is 26 GiB of mmap'd weights plus pinned staging buffers; an
+    # RLIMIT smaller than the pinned set fails the load in a way that looks
+    # like an engine crash.
+    grep -qE '^LimitMEMLOCK=infinity$' "$UNIT"
 }

-@test "the deploy handler will not stop a serving endpoint it cannot replace" {
-    # The runtime gate must be live (a real conditional on an executable
-    # path, not a branch that never fires) and it must return BEFORE the
-    # stop, or a host whose gufo was never built takes frank-large down on
-    # its way to reporting the failure.
-    local large gate bail stop
-    large="$(sed -n '/^deploy_llm_large() {/,/^}/p' "$REMOTE_DEPLOY")"
-    [[ "$large" == *'build-gufo.sh'* ]]
-    gate="$(grep -n 'if \[\[ ! -x "\$LLM_LARGE_RUNTIME" \]\]' <<<"$large" | head -1 | cut -d: -f1)"
-    stop="$(grep -n 'systemctl stop frank-large' <<<"$large" | head -1 | cut -d: -f1)"
-    [ -n "$gate" ]
-    [ -n "$stop" ]
-    bail="$(awk -v g="$gate" 'NR > g && /return 1/ { print NR; exit }' <<<"$large")"
-    [ -n "$bail" ]
-    [ "$bail" -lt "$stop" ]
-}
-
-@test "the stray-run sweep keys on the drafter and cannot take frank-small down" {
-    # The sweep used to key on the TARGET filename, which the 2026-09-28
-    # payload swap moved to a different repository. It would have stopped
-    # matching the live process, and a stray run would then hold 7770 past the
-    # port gate and fail the deploy. The drafter is the part that did NOT
-    # move, and every frank-large command line carries it via --dflash-model.
-    grep -q "pkill -f 'Qwen3.8-27B-DFlash2-Q4_K_M'" "$REMOTE_DEPLOY"
-    # frank-small serves a different GGUF on the 3090 with no DFlash2 drafter,
-    # so a binary-name sweep is the only thing that would endanger it.
-    local large
-    large="$(sed -n '/^deploy_llm_large() {/,/^}/p' "$REMOTE_DEPLOY")"
-    ! grep -q 'pkill -x llama-server' <<<"$large"
-    ! grep -q 'pkill -x gufo' <<<"$large"
-}
-
-@test "the deploy handler retires the halogen quadlet without deleting it" {
-    grep -q 'retire_halogen_large_quadlet' "$REMOTE_DEPLOY"
-    grep -q 'HOME/.AGENTTRASH' "$REMOTE_DEPLOY"
-    ! grep -qE '(^|[^[:alnum:]_])rm (-[a-zA-Z]+ )?.*frank-large' "$REMOTE_DEPLOY"
-    # Nothing live ships for this endpoint as a container any more. The
-    # archived copy is host state in an ignored .AGENTTRASH — deliberately
-    # untracked, and present only in a primary checkout — so search the
-    # tracked tree, not the filesystem.
-    [ ! -e "$REPO_ROOT/dankclaw/frank-large/quadlet/frank-large.container" ]
-    ! git -C "$REPO_ROOT" grep -q 'Image=.*halogen' -- dankclaw
+@test "the unit points at no retired frank-large runtime path" {
+    # The host keeps every retired payload and build on disk, so a stale path
+    # in a unit still resolves — to a runtime nobody has measured in weeks.
+    ! grep -qE 'GitPackages/(ROCmFPX-kairic|llama.cpp-ik_llama|llama-builds|ciru-orca)' "$UNIT"
+    ! grep -q 'halogen' "$UNIT"
 }
@@ -2,10 +2,11 @@
 # Seam: frank-primary — the Dankserver LLM endpoint on :7770.
 #
 # This file asserts the tcclaviger RDNA4 vLLM ROOT-UNIT shape (the
-# 2026-10-01 BF16 cutover, the overnight redo campaign's verdict): the serve
+# 2026-10-02 correction of the 2026-10-01 BF16 cutover): the serve
 # script drives `podman run` DIRECTLY (the image entrypoint execs
 # `vllm serve`), with every engine flag qualified by boot probe and carried
-# in the repo env file. The seam pins what the flags alone do not say: the
+# in the repo env file.
+# The seam pins what the flags alone do not say: the
 # KV qualification law (the night ladder: 262144/245760 OOM at graph
 # capture; 241664 boots at 732,359 tokens / 3.03x, util 0.98;
 # the bf16-KV trade (double the attention-KV bytes/token, ~14% decode cost,
@@ -91,29 +92,31 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"

@test "the qualified engine shape is in the env file" {
    # 2026-10-01 night ladder (Frank's method), then a runtime-survivability
-    # correction. SEQS=3 bf16 topped out at 241664 (KV 732,359 / 3.03x);
+    # correction, then Frank's 2026-10-02 decision to give the pool back to
+    # the expert arena. SEQS=3 bf16 topped out at 241664 (KV 732,359 / 3.03x);
    # SEQS=6 fp8 does NOT fit (6 x 241664 asked 10.70 GiB against 9.18 GiB
    # free); SEQS=5 fp8 x 262144 BOOTED (KV 1,350,182 / 5.15x) but left ~0
    # bytes free at runtime and a real client request (995-token prompt with
    # max_tokens=261149) killed the worker with `Tried to allocate 50.00 MiB
-    # ... 0 bytes free`. FOUR slots at the full native 262144 window is the
-    # deployed shape: ~1.93 GiB more headroom per rank for
-    # activations/graphs. UTIL stays 0.98 (lowering it shrinks the engine's
-    # own budget instead of buying slack).
-    # FINAL (2026-10-01, Frank): TWO full-size windows with BF16 KV plus
-    # --expert-precise-memory. bf16 costs 13,839 B/token (measured), so
-    # 2 x 262144 = 524,288 tokens needs 7.256e9 B; pinned at 7.3e9 (~527.5k
-    # tokens, 3,206 margin). The precise flag drops the plan's overhead
-    # 7.60 -> 5.25 GiB, moving the arena from 6,248 to ~7,278 slots.
-    grep -q '^FRANK_PRIMARY_MAXLEN=262144$' "$ENV_FILE"
-    grep -q '^FRANK_PRIMARY_SEQS=3$' "$ENV_FILE"
+    # ... 0 bytes free`. 200704 sits below the 241664 rung while leaving a
+    # full 2.95 GiB/rank to cut loose for the arena. UTIL stays 0.98
+    # (lowering it shrinks the engine's own budget instead of buying slack).
+    # FINAL (2026-10-02, Frank): TWO windows at 200704 with FP8 KV and a pool
+    # cut for the arena. fp8 costs 7,701 B/token (measured 6,138 attention +
+    # 1,563 mamba/side), so 2 x 200704 = 401,408 tokens needs 3.091e9 B;
+    # pinned at 3.15e9 (~405k tokens, ~44 MB margin). The arena is what
+    # remains — the planner sizes it AFTER the KV override — so the 6,248
+    # slots of the 6.1e9 shape become roughly 7,500 here.
+    grep -q '^FRANK_PRIMARY_MAXLEN=200704$' "$ENV_FILE"
+    grep -q '^FRANK_PRIMARY_SEQS=2$' "$ENV_FILE"
    grep -q '^FRANK_PRIMARY_UTIL=0.98$' "$ENV_FILE"
-    grep -q '^FRANK_PRIMARY_KV_BYTES=6100000000$' "$ENV_FILE"
-    grep -q '^FRANK_PRIMARY_MNBT=2048$' "$ENV_FILE"
+    grep -q '^FRANK_PRIMARY_KV_BYTES=3150000000$' "$ENV_FILE"
+    grep -q '^FRANK_PRIMARY_MNBT=1024$' "$ENV_FILE"
    grep -q '^FRANK_PRIMARY_KV_DTYPE=fp8$' "$ENV_FILE"
-    # --expert-precise-memory is NOT set: with bf16 2 x 262144 it put the boot
-    # 0.13 GiB from the 31.86 GiB ceiling (arena 16.65 device-resident) and
-    # crash-looped at graph capture. The serve script keeps the opt-in flag.
+    # --expert-precise-memory is NOT set: with bf16 2 x 262144 it put the
+    # boot 0.13 GiB from the 31.86 GiB ceiling (arena 16.65 device-resident)
+    # and crash-looped at graph capture. The serve script keeps the opt-in
+    # flag; the smaller pool here is the arena it would have been paying for.
    ! grep -q '^FRANK_PRIMARY_PRECISE_MEMORY=' "$ENV_FILE"
    grep -q '^FRANK_PRIMARY_GATHER_CHUNKS=32$' "$ENV_FILE"
    grep -q '^FRANK_PRIMARY_GATHER_LANES=64$' "$ENV_FILE"
@@ -121,6 +124,22 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
    grep -q '^FRANK_PRIMARY_MODEL_SHARDS=131$' "$ENV_FILE"
 }

+@test "the pool is sized as the arena floor, not the context floor" {
+    # THE LAW this seam pins: the fork's expert planner sizes the arena
+    # AFTER the KV override, so every byte cut out of KV_BYTES becomes
+    # arena. A context-size change must move the pool with it, or the
+    # reduction buys nothing. 2 x 200704 = 401,408 tokens at fp8's 7,701
+    # B/token is 3.091e9 B; the pin is 3.15e9, ~44 MB of margin above the
+    # floor — the same convention the 6.1e9 shape used.
+    floor=$(awk -v n="401408" -v b="7701" 'BEGIN{printf "%.0f", n*b}')
+    [ "$floor" = "3091243008" ]
+    kv=$(grep '^FRANK_PRIMARY_KV_BYTES=' "$ENV_FILE" | cut -d= -f2)
+    [ "$kv" = "3150000000" ]
+    margin=$(( kv - floor ))
+    [ "$margin" -gt 40000000 ]
+    [ "$margin" -lt 100000000 ]
+}
+
@test "the serve script wires the engine flags from the env" {
    grep -q -- '--max-num-seqs "$FRANK_PRIMARY_SEQS"' "$SERVE"
    grep -q -- '--max-model-len "$FRANK_PRIMARY_MAXLEN"' "$SERVE"
@@ -208,13 +227,14 @@ DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh"
    grep -q -- '-e R4D_LRU_GATHER_CHUNKS="$FRANK_PRIMARY_GATHER_CHUNKS"' "$SERVE"
 }

-@test "MNBT is at Frank's 2048 with the arena intact" {
+@test "MNBT is at Frank's 1024 with the arena intact" {
    # MNBT 512 doubles TTFT at 4064 ctx (2.46 s vs 1.36 s); 1024-4096 are
    # interchangeable on decode AND identical on arena (the sweep measured
    # 6,499 slots at every rung <=4096); 12288 costs 2.5% plus arena.
-    # 2048 is Frank's setting as of 2026-10-01 — it keeps the workspace floor
-    # so the expert arena stays max.
-    grep -q '^FRANK_PRIMARY_MNBT=2048$' "$ENV_FILE"
+    # 1024 is the 2026-10-02 setting — the pool was cut to feed the expert
+    # arena, so the workspace floor matters more than it did at 2048: the
+    # arena budget is what is left, and 1024-4096 pay nothing for it.
+    grep -q '^FRANK_PRIMARY_MNBT=1024$' "$ENV_FILE"
    grep -q -- '--max-num-batched-tokens "$FRANK_PRIMARY_MNBT"' "$SERVE"
 }

@@ -1,30 +1,72 @@
 #!/usr/bin/env bats
-# Seam: the frank-small SGLang root unit on dankclaw.
-# frank-small is the RTX 3090 endpoint; since 2026-10-01 it serves
-# letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5 hybrid,
-# INT8 W8A16 AutoRound, in-checkpoint vision) via SGLang in a
-# python3.12 venv. This seam pins what the flags alone do not say: boot
-# persistence for a NATIVE runtime (no generated quadlet is involved), the
-# repo-relative paths the unit executes, the separate client port, the
-# no-spec-decode gate (the checkpoint ships NO mtp.* tensors, so a NEXTN
-# draft would be an uninitialised ghost model), the 8-shard payload gate,
-# and the KV budget: fp8 KV at 0.98 utilization whose 4-slot grid must fit
-# the card's 24 GiB.
+# Seam: the frank-small llama.cpp root unit on dankclaw.
+# frank-small is the RTX 3090 endpoint; since 2026-10-01 night it serves
+# bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF:Q5_K_M via a CUDA (sm_86) build of
+# llama.cpp master, chosen for the ASYMMETRIC KV Frank asked for: K f16
+# (unquantized) + V q8_0, which only llama.cpp exposes per-tensor. Since
+# 2026-10-02 it runs THREE contexts and carries a vision projector, and the
+# card must keep 2-3 GiB free. This seam runs the serve script against a stub
+# engine and asserts on the argv it actually execs, so what is pinned is the
+# command the operator gets.

 load helpers

 UNIT="$REPO_ROOT/scripts/frank-small.service"
 SERVE="$REPO_ROOT/scripts/serve-frank-small.sh"
 ENV_FILE="$REPO_ROOT/scripts/frank-small.env"
-DOWNLOAD="$REPO_ROOT/scripts/download-frank-small.sh"
 REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
 ENV_LARGE="$REPO_ROOT/scripts/frank-large.env"
+TARGET_STUB="MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf"
+# Same basename as the shipped projector: the test env repoints it into $STAGE.
+MM_STUB="mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf"

-@test "frank-small unit, serve script, env and download script exist in scripts/" {
+setup() {
+    STAGE="$(mktemp -d)"
+    RUNTIME_DIR="$STAGE/llama"
+    MODEL_DIR="$STAGE/models"
+    mkdir -p "$RUNTIME_DIR/bin" "$MODEL_DIR"
+    cat > "$RUNTIME_DIR/bin/llama-server" <<'STUB'
+#!/usr/bin/env bash
+printf '%s\n' "$@"
+STUB
+    chmod +x "$RUNTIME_DIR/bin/llama-server" "$RUNTIME_DIR/bin" 2>/dev/null || chmod +x "$RUNTIME_DIR/bin/llama-server"
+    touch "$MODEL_DIR/$TARGET_STUB" "$MODEL_DIR/$MM_STUB"
+}
+
+teardown() {
+    rm -rf "$STAGE"
+}
+
+# serve_argv [env-file] — run the serve script and capture the engine's
+# argv, one token per line, in $output.
+serve_argv() {
+    local envf="${1:-$STAGE/test.env}"
+    local mmproj
+    mmproj="$(sed -nE 's/^MMPROJ_FILE=(.+)$/\1/p' "$ENV_FILE")"
+    cat > "$envf" <<EOF
+LLAMA_BIN=$RUNTIME_DIR/bin/llama-server
+MODEL_FILE=$MODEL_DIR/$TARGET_STUB
+MMPROJ_FILE=$MODEL_DIR/${mmproj##*/}
+ALIAS=frank-small
+BIND_ADDR=0.0.0.0
+LISTEN_PORT=7771
+CTX_SIZE=$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")
+PARALLEL=$(sed -nE 's/^PARALLEL=([0-9]+)$/\1/p' "$ENV_FILE")
+CACHE_TYPE_K=$(sed -nE 's/^CACHE_TYPE_K=(.+)$/\1/p' "$ENV_FILE")
+CACHE_TYPE_V=$(sed -nE 's/^CACHE_TYPE_V=(.+)$/\1/p' "$ENV_FILE")
+NGL=$(sed -nE 's/^NGL=([0-9]+)$/\1/p' "$ENV_FILE")
+EOF
+    run env FRANK_SMALL_ENV_FILE="$envf" nvidia-modprobe=true bash "$SERVE"
+}
+
+argv_value() {
+    grep -A1 -xF -- "$1" <<<"$output" | tail -1
+}
+
+@test "frank-small unit, serve script and env exist in scripts/" {
    [ -f "$UNIT" ]
    [ -f "$SERVE" ]
    [ -f "$ENV_FILE" ]
-    [ -f "$DOWNLOAD" ]
 }

@test "frank-small is boot-persistent, self-healing and unprivileged" {
@@ -33,102 +75,140 @@ ENV_LARGE="$REPO_ROOT/scripts/frank-large.env"
    grep -q '^RestartSec=' "$UNIT"
    grep -q '^User=dank$' "$UNIT"
    grep -q '^LimitMEMLOCK=infinity$' "$UNIT"
-}
-
-@test "frank-small runs the repo's own serve script with the repo's env" {
    grep -q '^ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-small.sh$' "$UNIT"
    grep -q 'FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-small.env' "$UNIT"
 }

-@test "frank-small serves the MiMo-V2.6 checkpoint from the self-contained SGLang venv" {
-    grep -q '^SGLANG_BIN=/home/dank/GitPackages/sglang-cu128/bin/sglang$' "$ENV_FILE"
-    grep -q '^MODEL_DIR=/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound$' "$ENV_FILE"
-    grep -q -- '--model-path "$MODEL_DIR"' "$SERVE"
-    # The AutoRound checkpoint declares its quant method; SGLang must load it
-    # as auto-round, not re-quantize online.
-    grep -q '^QUANTIZATION=auto-round$' "$ENV_FILE"
-    grep -q -- '--quantization "$QUANTIZATION"' "$SERVE"
+@test "the shipped env points at the llama.cpp runtime and the Q5_K_M GGUF" {
+    grep -qE '^LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server$' "$ENV_FILE"
+    grep -qE "^MODEL_FILE=/home/dank/models/MiMo-Q5KM/$TARGET_STUB$" "$ENV_FILE"
 }

-@test "frank-small runs WITHOUT speculative decoding (no MTP weights shipped)" {
-    # The config declares mtp_num_hidden_layers=1 but the safetensors index
-    # carries zero mtp.* tensors (base and INT8 conversion alike, verified
-    # 2026-10-01). SGLang would draft with an uninitialised 4.24 GiB model
-    # at 0% acceptance; the serve script must not pass any speculative flag.
-    ! grep -q -- '--speculative-' "$SERVE"
-    ! grep -q '^SPEC_' "$ENV_FILE"
+@test "the deployer gates on exactly the payload paths the env file ships" {
+    # Anti-drift: the deploy handler checks files by its own defaults. If
+    # those ever differ from what the unit actually loads, a green deploy
+    # would be gating on a file nobody reads — and a missing projector would
+    # ship a blind endpoint.
+    local env_v dep_v path pair
+    for pair in "MODEL_FILE LLM_SMALL_MODEL_FILE" "MMPROJ_FILE LLM_SMALL_MMPROJ"; do
+        read -r env_v dep_v <<<"$pair"
+        path="$(sed -nE "s/^${env_v}=(.+)\$/\1/p" "$ENV_FILE" | tail -1)"
+        [ -n "$path" ]
+        grep -qF -- "${dep_v}=\"\${${dep_v}:-${path}}\"" "$REMOTE_DEPLOY"
+    done
 }

-@test "frank-small carries the fleet's image path, and only it" {
-    # The vision tower is baked into the safetensors (qwen3_5_vision): no
-    # separate projector file, and the serve script must not look for one.
-    ! grep -q '^MMPROJ_PATH=' "$ENV_FILE"
-    ! grep -q -- '--mmproj' "$SERVE"
-    ! grep -q '^ENABLE_VISION=1$' "$ENV_LARGE"
+@test "the shipped env serves the fleet shape: three 180224-token contexts" {
+    serve_argv
+    [ "$status" -eq 0 ]
+    [ "$(argv_value --alias)" = "frank-small" ]
+    [ "$(argv_value --port)" = "7771" ]
+    [ "$(argv_value --host)" = "0.0.0.0" ]
+    [ "$(argv_value --model)" = "$MODEL_DIR/$TARGET_STUB" ]
+    [ "$(argv_value --n-gpu-layers)" -gt 0 ]
+    # llama.cpp divides --ctx-size across slots and a mamba hybrid cannot
+    # share one cache, so the observable contract is the per-slot window.
+    [ "$(argv_value --parallel)" -eq 3 ]
+    ctx="$(argv_value --ctx-size)"
+    par="$(argv_value --parallel)"
+    [ $((ctx % par)) -eq 0 ]
+    [ $((ctx / par)) -eq 180224 ]
+    [ $((ctx / par)) -le 262144 ]
 }

-@test "frank-small's slot grid is fp8 KV and fits the measured card budget" {
-    local ctx running kv ssmdtype mambasize required
-    ctx="$(sed -nE 's/^CONTEXT_LENGTH=([0-9]+)$/\1/p' "$ENV_FILE")"
-    running="$(sed -nE 's/^MAX_RUNNING=([0-9]+)$/\1/p' "$ENV_FILE")"
-    kv="$(sed -nE 's/^KV_DTYPE=(.+)$/\1/p' "$ENV_FILE")"
-    ssmdtype="$(sed -nE 's/^MAMBA_SSM_DTYPE=(.+)$/\1/p' "$ENV_FILE")"
-    mambasize="$(sed -nE 's/^MAX_MAMBA_CACHE_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
-    [ -n "$ctx" ] && [ -n "$running" ] && [ -n "$kv" ]
-    # fp8 KV is the only dtype whose 16 KiB/token makes four slots fit;
-    # 64 tokens per MiB of pool.
-    case "$kv" in fp8*) ;; *) return 1 ;; esac
-    # The DeltaNet ssm state pool is the hidden fit cost: the engine
-    # autofills mamba slots to spend a fixed ~5.1 GB budget, so dtype
-    # alone does not shrink it — the SLOT COUNT must be pinned (bf16,
-    # bounded). These two gates are what make the measured pool below
-    # exist; change either and re-measure the pool at boot.
-    [ "$ssmdtype" = "bfloat16" ]
-    [ -n "$mambasize" ] && [ "$mambasize" -ge "$running" ] && [ "$mambasize" -le 64 ]
-    grep -q -- '--mamba-ssm-dtype "$MAMBA_SSM_DTYPE"' "$SERVE"
-    grep -q -- '--max-mamba-cache-size "$MAX_MAMBA_CACHE_SIZE"' "$SERVE"
-    # The slot window is inside the model's native 262144 window.
-    [ "$ctx" -le 262144 ]
-    # The grid must equal the pinned pool. The engine's autofit picks a
-    # bigger pool (687,388 tokens) but leaves ~0.53 GB — Triton's
-    # deferred kernel loads take most of it and eager-prefill activations
-    # then OOM on the first long request. 606,208 (4 x 151,552) is the
-    # verified ceiling: 170k single request and 4 concurrent 54k requests
-    # run against it (2026-10-01). A weights or mamba-flag change moves
-    # this number; re-measure before trusting the grid.
-    pinned_pool=606208
-    required=$((running * ctx))
-    [ "$required" -le "$pinned_pool" ]
-    grep -q -- '--max-total-tokens "$MAX_TOTAL_TOKENS"' "$SERVE"
-    grep -q -- '--disable-prefill-cuda-graph' "$SERVE"
+@test "the KV contract is asymmetric: unquantized K, q8_0 V, flash-attn on" {
+    serve_argv
+    [ "$status" -eq 0 ]
+    [ "$(argv_value --cache-type-k)" = "f16" ]
+    [ "$(argv_value --cache-type-v)" = "q8_0" ]
+    # q8_0 on V only exists on the flash-attention path; without this flag
+    # pair the KV contract in the env file is a lie the engine never sees.
+    [ "$(argv_value --flash-attn)" = "on" ]
 }

-@test "frank-small runs on its own port beside frank-large's" {
-    grep -q '^LISTEN_PORT=7771$' "$ENV_FILE"
-    grep -q '^ALIAS=frank-small$' "$ENV_FILE"
-    grep -q -- '--port "$LISTEN_PORT"' "$SERVE"
-    grep -q -- '--host "$BIND_ADDR"' "$SERVE"
-    # The two endpoints cannot share a port: frank-large holds 7770, this one
-    # 7771.
-    grep -q '^PORT=7770$' "$ENV_LARGE"
-    ! grep -q '^PORT=7771$' "$ENV_LARGE"
+@test "the KV budget leaves 2-3 GiB of the 3090 free" {
+    # Rate: K f16 (16 KiB/token) + V q8_0 (~8.5, budgeted 25 with slack).
+    # The non-KV half is MEASURED on this card on 2026-10-02: with the
+    # projector loaded the engine reported 22,972 MiB used against 14,113
+    # MiB of KV, so weights + vision tower + DeltaNet state + compute
+    # buffers + CUDA graphs = 8,859 MiB => 9 GiB. Two asserts, both about
+    # the output the operator sees: the pool must still fit with 2 GiB
+    # spare, and it must be large enough that 5 GiB spare is NOT reachable
+    # (an undersized pool is as wrong as an oversized one). Arithmetic, not
+    # a boot log: change CTX_SIZE and this bites before the deploy does —
+    # it rejects the 3 x 196,608 pool that measured only 1.1 GiB free. The
+    # same pool with f16 V (34 KiB/token) lands past the card, which is why
+    # the contract is asymmetric.
+    ctx="$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
+    kv_gib=$((ctx * 25 / 1048576))
+    [ $((kv_gib + 9 + 2)) -le 24 ]
+    [ $((kv_gib + 9 + 5)) -gt 24 ]
+    f16v_gib=$((ctx * 34 / 1048576))
+    [ $((f16v_gib + 9)) -gt 24 ]
 }

-@test "the deploy handler installs the root unit and refuses a missing payload" {
+@test "no speculative draft ships by default" {
+    # The qwen35 GGUF has no MTP tensors, and the cross-model DFlash draft
+    # is a measured decision the harness makes later, not a default.
+    ! grep -q -- '--model-draft' "$SERVE"
+    ! grep -q -- '--spec-type' "$SERVE"
+    ! grep -q '^DRAFT' "$ENV_FILE"
+}
+
+@test "vision ships by default and MMPROJ_FILE empty opts out of the image path" {
+    # The Q5_K_M GGUF carries no `v.*` tensors, so the projector argument IS
+    # the vision capability: no flag, no eyes.
+    serve_argv
+    [ "$status" -eq 0 ]
+    [ "$(argv_value --mmproj)" = "$MODEL_DIR/$MM_STUB" ]
+
+    # The documented opt-out must reach the engine as ABSENCE of the flag,
+    # not as an argument pointing at an empty string.
+    sed -i 's|^MMPROJ_FILE=.*|MMPROJ_FILE=|' "$STAGE/test.env"
+    run env FRANK_SMALL_ENV_FILE="$STAGE/test.env" nvidia-modprobe=true bash "$SERVE"
+    [ "$status" -eq 0 ]
+    ! grep -qxF -- '--mmproj' <<<"$output"
+}
+
+@test "the fleet reasoning contract survives the reslice to three contexts" {
+    serve_argv
+    [ "$status" -eq 0 ]
+    # deepseek format = thinking lands in reasoning_content, the field the
+    # harness reads on this endpoint; --jinja is what makes
+    # chat_template_kwargs reach the template at all.
+    [ "$(argv_value --reasoning-format)" = "deepseek" ]
+    argv_value --jinja >/dev/null
+    grep -qxF -- '--jinja' <<<"$output"
+}
+
+@test "the serve script refuses a missing runtime, GGUF or projector" {
+    rm -f "$MODEL_DIR/$MM_STUB"
+    serve_argv
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"frank-small: missing"* ]]
+
+    rm -f "$RUNTIME_DIR/bin/llama-server"
+    serve_argv
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"frank-small: missing"* ]]
+}
+
+@test "the deploy handler gates on binary, GGUF and projector, then MainPID-hardens health" {
    grep -q 'deploy_llm_small' "$REMOTE_DEPLOY"
+    grep -q 'LLM_SMALL_RUNTIME' "$REMOTE_DEPLOY"
+    grep -q 'LLM_SMALL_MODEL_FILE' "$REMOTE_DEPLOY"
+    grep -q 'LLM_SMALL_MMPROJ' "$REMOTE_DEPLOY"
    grep -q 'relink_unit frank-small' "$REMOTE_DEPLOY"
    grep -q 'systemctl enable frank-small.service' "$REMOTE_DEPLOY"
-    grep -q 'systemctl restart frank-small.service' "$REMOTE_DEPLOY"
-    grep -q 'MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound' "$REMOTE_DEPLOY"
-    grep -q 'refusing to start a unit that cannot load' "$REMOTE_DEPLOY"
    grep -q '127.0.0.1:7771/health' "$REMOTE_DEPLOY"
+    grep -q 'frank-small.service has no MainPID' "$REMOTE_DEPLOY"
 }

-@test "frank-small's serve script refuses to start on a half-finished download" {
-    # The payload is 8 safetensors shards; the serve script counts them and
-    # the download script pins each one's LFS sha256.
-    grep -q 'frank-small: missing' "$SERVE"
-    grep -q 'expected 8 safetensors shards' "$SERVE"
-    grep -q 'model-00001-of-00008.safetensors' "$DOWNLOAD"
-    grep -q 'model-00008-of-00008.safetensors' "$DOWNLOAD"
+@test "the two endpoints keep separate ports and the large handler never pkills llama-server" {
+    # frank-large's gufo sweep must not take down frank-small, whose
+    # process is llama-server on the SAME host since this swap.
+    grep -q '^LISTEN_PORT=7771$' "$ENV_FILE"
+    local large
+    large="$(sed -n '/^deploy_llm_large() {/,/^}/p' "$REMOTE_DEPLOY")"
+    ! grep -q 'pkill -x llama-server' <<<"$large"
 }
Author	SHA1	Message	Date
Comrade Chief Designer	30904bac1b	frank-primary: 2 x 200704 fp8, pool cut for the expert arena, MNBT 1024 The 2026-10-01 bf16 shape (2 x 262144, --expert-precise-memory) crash-looped at graph capture — the arena is DEVICE-resident and that shape had 0.13 GiB to spare. The plain bf16 2 x 262144 also did not stay up: 0 bytes free at runtime, and a real client request (995-token prompt, max_tokens=261149) killed the worker with . The correction is a pool cut, not a downgrade. FP8 KV (7,701 B/token), two 200704 windows (401,408 tokens -> 3.091e9 B floor, pinned 3.15e9), MNBT 1024. The fork's expert planner sizes the arena AFTER the KV override, so the ~2.95 GiB/rank reclaimed becomes arena: ~7,500 slots vs 6,248. Two contexts at 200704 is a capacity contract, not a throughput one (2-way concurrency summed BELOW parity at 4064 ctx), so the slot buys the VRAM headroom long generations need. Refs [[dankserver-qwen38-fp8-expert-offload-tuning]]. - scripts/frank-primary.env: MAXLEN 200704, SEQS 2, KV_BYTES 3.15e9, MNBT 1024, rewritten shape comment - scripts/serve-frank-primary.sh: header comment updated - tests/frank_primary_unit.bats: assertions updated, new seam pins the arena-floor pool law (the pool must move with a context change or the reduction buys nothing) - docs + AGENTS.md: fleet topology and deployed shape corrected	2026-10-02 09:54:08 -07:00
Frank	f8f845d86b	autoresearch: harness setup Benchmark entrypoint: bash autoresearch.sh Goal: Make frank-primary faster: raise decode tok/s on the tcclaviger vLLM endpoint (dankserver :7770, Qwen3.8-Flash-Next-Uncensored-FP8, 2x R9700, automatic tiered-expert offload). Context size may be altered (MAXLEN/SEQS/KV_BYTES and friends); quantization may NOT be altered (the 131-shard FP8 payload and the digest-pinned image are frozen).	2026-10-02 07:50:50 -07:00
Frank	934285c2b2	docs(frank-small): record the 3 x 180,224 + vision shape and what it cost Runbook, both indexes, AGENTS.md (topology row, endpoint section, three new Key Lessons, one stale SGLang label in the bench list) and the bundle log now describe the deployed shape: pool 540,672, 180,224 tokens per conversation, f16 K + q8_0 V, mmproj vision, 21,684 used / 2,441 free of 24,576 MiB, 106.8 tok/s decode, 3,814 t/s prefill. The claims that mattered: - three contexts buy 180,224 tokens per conversation, not the native 262,144, and a longer prompt is a hard 400 - so models.yml contextWindow is 180224 and frank-primary's 183,501-token compaction request no longer fits here (forward pointer added to the retired SGLang runbook, which asserted it) - the vision tower bills ~1.4 GiB for a 918 MB file, established by subtracting two measured boots, not by a formula - /v1/chat/completions ignores the legacy top-level 'images' field: the model invents a color; only the image_url content part reaches it	2026-10-02 03:46:50 -07:00
Frank	7fe881de01	frank-small: pool 540672 (3 x 180224) so the card keeps ~2.7 GiB free Measured after the first 3-slot deploy: 3 x 196,608 booted (n_slots=3, n_ctx_slot=196608, multimodal loaded) but used 22,972/24,576 MiB - 1,153 MiB free, outside the 2-3 GiB band. The two measurements now recorded in frank-small.env price the vision tower at ~1.4 GiB (non-KV 7,390 -> 8,859 MiB), so the pool drops 49,152 tokens (16,384/slot, 1,176 MiB) to land ~2.7 GiB free. The bats budget gate is recalibrated to the measured 9 GiB non-KV cost, which now rejects the 3 x 196,608 shape instead of blessing it.	2026-10-02 03:17:35 -07:00
Frank	a02850203d	frank-small: three contexts, vision on, 2-3 GiB of the 3090 left free CTX_SIZE 589824 / PARALLEL 3 -> three independent 196,608-token conversations (a mamba hybrid cannot share one KV cache, so per-slot window = pool/slots; 3 x 262,144 would need 18.4 GiB of KV alone and busts the card). Vision comes back via bartowski's mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf (918,166,048 B, sha256 ff348f31..., fetched + verified by scripts/download-frank-small.sh): the Q5_K_M GGUF ships no v.* tensors, so --mmproj IS the image path. MMPROJ_FILE without a colon default so an explicit empty value is a real text-only opt-out, and the deployer now gates on the projector exactly as the serve script does. Budget is measured, not guessed: the 2 x 262,144 shape used 20,190/24,576 MiB, which prices non-KV cost at 7,390 MiB. Three slots + projector land near 22.5 GiB, i.e. ~2 GiB spare - the bottom of the requested band, to be confirmed by measurement and laddered if it lands outside 2-3 GiB.	2026-10-02 03:08:00 -07:00
Frank	8f129180d8	docs+scripts: reconcile payload inventory after both swaps Staged 59 GiB of retired LLM payloads on Dankclaw to ~dank/.AGENTTRASH/20261002-retired-llm-payloads/ (vLLM MiMo BF16, Qwen3.5-9B MTP source, both DFlash dirs, Gemma4-12B-QAT, SGLang MiMo INT8) and corrected every doc claim about what is on disk: gufo's rollback pair (Q4_K_XL weights + the Q8_K_XL DFlash2 drafter) and frank-small's Q6_K stay in place, so the rollback paths in the tags' env files still resolve. The vLLM-era frank-large fetch trio moves to old/ with its runtime; runbook links updated.	2026-10-02 02:31:39 -07:00
Frank	72bd4c967e	docs: record the 2026-10-02 swaps — frank-large on ROCmFPX, frank-small on Q5_K_M frank-large: new runbook frank-large-rocmfpx-q8agent-mtp.md — the ftype-115 tensor-type lock-in that forced the fork, the pinned build (set_rows history), the measured MTP A/B (17.0 vs 7.6 t/s decode, acceptance 0.462), the live client contract (reasoning_content, effort fold, vision restored) and rollback. frank-small: new runbook frank-small-llama-mimo-q5km.md; q6k marked retired (rollback target). gufo runbook marked retired (rollback target). Both indexes, the fleet topology table, the vision/models.yml notes, the benchmarking law and three new Key Lessons updated.	2026-10-02 01:51:11 -07:00
Frank	125d04967f	frank-large: ROCmFPX llama.cpp on the Q8_0_ROCMFPX_AGENT file, MTP drafting The repo's highest-quality rung is ftype 115: 311 tensors of GGML_TYPE_Q8_0_ROCMFPX, a type only the ROCmFPX fork reads, so the runtime moves from gufo to scripts/build-rocmfpx.sh's pinned fork build (36e9acd) — the first build on this host measured to load this file without the 2026-08-25 set_rows segfault. One session at the native 262,144 window, f16 KV (64 KiB/token, 16 GiB), the repo's Q4_0 MTP head drafting 4 tokens, plus the bundled BF16 projector so the endpoint has vision again. The serve script now honors caller environment overrides over the env file (a plain source let the file win, which made single-run overrides no-ops) and defaults apply only when a value is unset, so an explicit LLAMA_BIN='' is the error it reads as.	2026-10-02 00:47:16 -07:00
Frank	d82c5c03b7	frank-small: Q5_K_M GGUF on llama.cpp, two full native windows One quant step down from Q6_K on the same CUDA build and the same shape: 2 x 262,144 with K f16 + V q8_0. The 0.86 GiB freed from the weights is headroom, not a new window — the native 262,144 cap is already reached, and f16 V would still cost 3.96 GiB more than the pool can carry.	2026-10-02 00:00:17 -07:00
Frank	4bb10cd20e	fetch: kingjones Q8_0_ROCMFPX_AGENT payload, frank-small Q5_K_M GGUF The SGLang INT8 fetch recipe moves to old/ with its route; scripts/ download-frank-small.sh now carries the GGUF quant the endpoint serves.	2026-10-01 23:50:52 -07:00
Frank	a7c68db9d0	docs: record the frank-small llama.cpp Q6_K cutover and the frank-large gufo rollback	2026-10-01 23:50:52 -07:00
Frank	8551bb006b	frank-large: roll back to gufo Qwen3.8-27B Q4_K_XL, one session x 262144 f16 KV frank-small: llama.cpp CUDA (master, sm_86) on MiMo Q6_K GGUF, 2 x 262144, asymmetric KV (f16 K + q8_0 V, flash-attn pinned for the V quant)	2026-10-01 22:40:59 -07:00
Frank	e452a1afbf	harness: tolerate missing container in ASI capture; llama cmdline fallback	2026-10-01 22:11:34 -07:00
Frank	01ba41cab4	harness: llama-server lifecycle in pre-clean and health guard	2026-10-01 21:58:51 -07:00
Frank	cde91c1a00	llama.cpp runtime arm: serve-script runtime switch, harness containerless support, master build	2026-10-01 21:56:30 -07:00
Frank	6513672f31	SPEC= (no draft): decode +46%, prefill +52%, ttft -35% vs MTP-2 baseline. Spec decode is NET-NEGATIVE on this endpoint for real chat+reasoning traffic at both depths; the bench-window win did not transfer. Result: {"status":"keep","decode_tokens_per_sec":9.01,"prefill_tokens_per_sec":21088.7,"ttft_ms":851.8}	2026-10-01 21:00:19 -07:00
Frank	5df6ef3166	autoresearch: harness setup Benchmark entrypoint: bash autoresearch.sh Goal: Maximize frank-large decode throughput on the Strix Halo iGPU (gfx1151) via quantization >=q6/fp6, engine flags, spec-decode shape, or runtime swap — context limits unchanged, KV stays bf16.	2026-10-01 20:41:51 -07:00
Frank	3c339eeba2	frank-large: record deployed boot facts (pool 819,332 tokens, 3.13x, 230 s) Live battery green after deploy #3: alias, reasoning split into `reasoning`, tool_calls, /metrics acceptance pos0 69% pos1 44% - the cross-model MTP head performs as measured in the bench window.	2026-10-01 19:48:56 -07:00
Frank	780c307354	frank-large: hand vLLM the CONTAINER model path; link the new runbook The first crash-looping boot (51 restarts): serve received the host path /home/dank/models/... but the models tree is mounted at /models inside the container, so transformers failed 'Repo id must be in the form namespace/repo_name' before the engine even built. Same conversion the spec draft already used. Pinned by bats. Root docs index links the new runbook (seam required it) and the work log records the crash-loop plus the 52 G payload reclaim.	2026-10-01 19:39:55 -07:00
Frank	de383fe53e	frank-large: deployer spec gate must accept the sharded MTP layout The 2026-10-01 first deploy refused /home/dank/models/Qwen3.5-9B because the handler checked for a single model.safetensors (the dflash layout); the Qwen3.5-9B MTP head ships 4 shards behind model.safetensors.index.json. Same dual-layout acceptance as the serve script, now mirrored and gated by the seam. The gate fired before the stop, so the live gufo unit was never touched (health 200 throughout).	2026-10-01 18:52:17 -07:00
Frank	1127a52078	docs: frank-large vLLM MiMo-BF16 runbook; retire the gufo runbook New runbook carries the roof math, the measured three-arm spec table, the greedy-text numerics trap, the reasoning-field contract change and the rollback tag. LLM index and AGENTS.md fleet table, bench policy and runbook index follow; the frank-small index entries catch up to the SGLang cutover (entry was missed on 2026-10-01). Work log records the cutover.	2026-10-01 18:48:51 -07:00
Frank	dab483c79f	frank-large: cutover to vLLM TheRock gfx1151, MiMo-V2.6 BF16 + cross-model MTP Engine: digest-pinned docker.io/kyuz0/vllm-therock-gfx1151 (vLLM 0.30, TheRock ROCm 10, Triton GDN kernels); the serve script gates the image by REPO DIGEST, the payload (config + index + exact shard count), the spec draft (BOTH weight layouts: single model.safetensors for dflash, sharded index for the Qwen3.5-9B MTP head), and whole /dev/dri, sweeps the bench identity vllm-mimo, then exec podman wait so a container death trips Restart=always. Shape: 4 x 262144 at util 0.45 (KV grows only on the 8 full-attention layers), MTP 2 draft tokens. Measured 2026-10-01 window (frank-large stopped): base 13.1 t/s decode (~94% of the LPDDR5x BF16 roof), DFlash-8 17.4, MTP-2 19.8 with 71%/44% per-position acceptance - the cross-model head works because vLLM reads the draft from a separate path while MiMo ships zero mtp.* tensors. Spec arms do not preserve greedy bf16 text (batched-verify reduction order). The seam was rewritten around the new laws (18 tests): unit self-healing, digest gating, engine wiring, the measured spec table, the retired gufo gates, the deployer MainPID check. frank_small port test follows the PORT -> FRANK_LARGE_PUBLISH rename. Rollback: tag frank-large-gufo-q4kxl-20261001.	2026-10-01 18:48:32 -07:00
Frank	b3313340fa	frank-large: fetch scripts for MiMo BF16, Qwen3.5-9B MTP source, DFlash drafter All three verify every file by byte count + Hub LFS sha256, use -f (a bare curl -o reports success on a 403 error body), support --verify-only, and never put a token on the command line. Payloads are on Dankclaw and verified against these manifests.	2026-10-01 18:48:05 -07:00