docs: frank-large Q6 uncensored cutover + frank-small 2x200704 f16-KV reslice

- new active runbook frank-large-rocmfpx-q6agent-uncensored-mtp.md: the Q6_0_ROCMFPX_AGENT file (ftype 114, 339 type-102 + 159 type-103 tensors), 27.6-28.1 t/s at 86.8% draft acceptance, the measured ngram-mod chain rejection (-15% prose: recurrent-state rollback disabled), the bench-window hardening laws - frank-large-rocmfpx-q8agent-mtp.md retired; rollback section corrected (gufo payloads removed from disk 2026-10-03 - that rollback is now a re-download; the aligned Q8 file is the direct one, still on disk) - frank-small-llama-mimo-q5km.md reshaped to 2 x 200704 with fully unquantized f16 KV: boot-pair table gains the measured row (20,530 used / 4,046 free), 104.84 tok/s, slot-window contract at 200704, rollback to the 3x180224 + q8_0-V shape documented - AGENTS.md: fleet table rows, both endpoint sections, the chain law extended with the recurrent-model measurement, the --parallel law with the reslice, runbook index (active + retired) - LLM/index.md + root index entries updated
2026-10-03 08:01:44 -07:00
parent 47a2c1287b
commit b80886e318
6 changed files with 438 additions and 216 deletions
@@ -0,0 +1,173 @@
+---
+type: reference
+title: "frank-large on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP"
+description: "The active frank-large runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Uncensored Q6_0_ROCMFPX_AGENT GGUF (ftype 114, protected output heads), with the repo's own MTP-Q4_0 head drafting four tokens at one native 262,144 window, f16 KV, the BF16 projector for vision, and the measured rejection of the ngram-mod chain."
+tags:
+  - llm
+  - frank-large
+  - dankclaw
+  - llama-cpp
+  - rocmfpx
+  - gfx1151
+  - gguf
+  - spec-decode
+  - mtp
+timestamp: 2026-10-03
+---
+
+# frank-large — ROCmFPX Uncensored Q6_0_ROCMFPX_AGENT + MTP (gfx1151)
+
+> **ACTIVE since 2026-10-03.** Cut over from
+> [[frank-large-rocmfpx-q8agent-mtp]] (ALIGNED Q8_0_ROCMFPX_AGENT, 17.0 t/s)
+> on Frank's request: the *uncensored* variant of the same protected-heads
+> family, one quant rung down. The swap turned out to be a strict upgrade on
+> the axis that matters here: **decode 27.6–28.1 t/s at a 4,813-token prompt
+> against 17.0 at a 2,202-token prompt** — faster with a deeper context, not a
+> same-depth comparison, but the sign is not in question.
+
+## Identity
+
+| | |
+|---|---|
+| Host / device | Dankclaw (192.168.0.143), Radeon 8060S iGPU `gfx1151`, ~114 GiB GTT aperture |
+| Endpoint | `http://192.168.0.143:7770` — alias `frank-large`, `/v1` OpenAI surface |
+| Runtime | **`kingjones30/ROCmFPX`** llama.cpp fork @ `36e9acd40e10a87cd3c3ef8ec734668757dc8520`, built by [`scripts/build-rocmfpx.sh`](../../scripts/build-rocmfpx.sh) — unchanged from the Q8 era; the fork already knew type 102 |
+| Unit | root systemd `frank-large.service` ([`scripts/frank-large.service`](../../scripts/frank-large.service)), `User=dank`, `Restart=always` |
+| Serve script | [`scripts/serve-frank-large.sh`](../../scripts/serve-frank-large.sh) (owns the argv), tunables [`scripts/frank-large.env`](../../scripts/frank-large.env) |
+| Weights | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` — 24,856,607,008 B, sha256 `e966deb3…` |
+| Draft | `…/mtp-Qwen3.8-27B-Q4_0.gguf` — 1,680,271,648 B, sha256 `051a1764…` (byte-identical to the aligned repo's head; hardlinked, not re-downloaded) |
+| Projector | `…/mmproj-Qwen3.8-27B-BF16.gguf` — 931,145,888 B (hardlinked from the aligned payload dir; the same file the Q8 era served) |
+| Fetch | [`scripts/download-frank-large-kingjones-uncensored.sh`](../../scripts/download-frank-large-kingjones-uncensored.sh) — hardlinks the draft/projector from the aligned dir when present, otherwise fetches; `-f`, byte-count + LFS sha256 verified, `--verify-only`, token never on the command line |
+| Seam | `tests/frank_large_unit.bats` (payload gates, deployer/serve parity, spec block, KV dtype) |
+
+## Why this file
+
+`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF` ships the same
+"protected heads" construction as the aligned repo, in the uncensored tuning.
+Parsed header of the deployed file:
+
+| Field | Value |
+|---|---|
+| ftype | **114** — the fork's 64-bit-block family, ≈7.44 bpw |
+| Tensor mix | 353 F32 + **339 type-102 (`GGML_TYPE_Q6_0_ROCMFPX`)** + **159 type-103 (`Q8_0_ROCMFPX`)** |
+| Protected heads | 64 output-head tensors kept at plain precision — the AGENT construction from the aligned repo, at the 6-bit rung |
+| Arch | qwen35, 64 layers, `full_attention_interval 4`, `context_length 262144` |
+| Embedded sampling recipe | temp 1.0 / top_p 0.95 / top_k 20 |
+
+The same lock-in law as the Q8 era applies: types 102 and 103 exist in no
+loader but this fork; stock llama.cpp and gufo refuse the file at load
+(`unknown tensor type`). The loader was not re-litigated — the file picks the
+runtime, and the pin in `scripts/build-rocmfpx.sh` (commit by SHA) stays.
+
+Rejected sibling: `Q6_0_ROCMFPX_LEAN` (6.45 bpw, no protected heads) — the
+protection is what keeps the output-head logits at full precision and the
+drafter accurate, so it stays deployed.
+
+## Shape and budget
+
+Unchanged from the Q8 era — the swap was payload-only:
+
+| Dial | Value | Why |
+|---|---|---|
+| `--ctx-size` | 262,144 | the native window, no rope scaling |
+| `--parallel` | 1 | one conversation, all memory to it |
+| KV dtype | **f16** (both K and V) | ~64 KiB/token; free inside a 114 GiB aperture |
+| `--fit off` | on | the auto-fitter would silently resize the window |
+| `--flash-attn on` | on | required for the long-window attention path |
+| `--direct-io`, `--ctx-checkpoints 16`, `--cache-ram 8192`, `--cache-idle-slots`, `--jinja`, `--reasoning on --reasoning-format deepseek --reasoning-budget -1` | | same as [[frank-large-rocmfpx-q8agent-mtp]] |
+
+Spec decode: `--spec-type draft-mtp`, the repo's `mtp-Qwen3.8-27B-Q4_0.gguf`,
+`--spec-draft-n-max 4`, pinned `--spec-draft-device ROCm0`.
+
+Measured steady state: GTT **~61 GiB** under live load (host baseline 16.7),
+boot to `/health` **~15 s**.
+
+## Measured on this payload (2026-10-03, production service, `/completion` timings, temp 0, `cache_prompt: false`)
+
+| Traffic | Decode t/s | Prefill t/s | Draft acceptance |
+|---|---|---|---|
+| Prose, 4,813-token prompt, 512 tokens | **27.57 / 28.10** (two runs) | ~205–217 | **396/456 = 0.868** → mean acceptance length ≈ 4.4, i.e. drafting at nearly the full n-max 4 |
+| Schema echo (structured repetition) | **31.44** | — | 639/639 = 1.000 |
+| Block copy (text repeated verbatim) | ~28.0 | — | 0.848 |
+
+vs the Q8 era's 0.462 acceptance at a 2,202-token prompt: the MTP head drafts
+materially better on this file. The card's claimed 0.953-class acceptance is
+no longer science fiction on real prose — but keep reading it as a file
+property, not a throughput promise.
+
+## The ngram-mod chain — measured and rejected
+
+Frank's ask: layer `--spec-type draft-mtp,ngram-mod` on top of MTP for an
+extra gain. Measured in three bench windows (production service stopped per
+the benchmarking law; artifacts `/home/dank/fl-ngram-ab/` on Dankclaw),
+comparing `draft-mtp` alone vs the chain on the same traffic:
+
+| Traffic | MTP alone | MTP + ngram-mod | Delta |
+|---|---|---|---|
+| Prose | 27.57 t/s | 23.40 t/s | **−15 %** |
+| Block copy | 28.0 t/s | 26.1 t/s | **−7 %** (with ngram-mod demonstrably firing: 138 proposals/run) |
+
+Mechanism, straight from arm B's boot log:
+
+```
+recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support
+```
+
+On the qwen35 hybrid (Gated DeltaNet recurrent layers) adding the ngram layer
+disables the rollback path MTP's own verification relies on, so acceptance —
+and throughput — drop on *all* traffic, even where the ngram matcher fires.
+**Deployed decision: `draft-mtp` alone.** Chaining is not free on recurrent
+models; the fleet law in `AGENTS.md → llama.cpp serving` records it.
+
+## Bench-window hardening laws (learned building the A/B)
+
+1. **Health 200 is not proof of ownership.** After boot, verify the pid you
+   started owns the port (`ss -ltnp`); a zombie server from a prior window
+   answers `/health` too and silently serves your arm.
+2. **Kill scratch servers with SIGKILL after a grace period.** llama-server's
+   graceful SIGTERM waits on open keep-alive connections — a harness holding
+   the socket leaves the process (and its ~24 GiB of GTT) alive after the
+   script exits.
+3. **Prove the port is free and GTT dropped between arms**, or the second
+   arm measures a collision.
+
+## Client contract
+
+Unchanged from the Q8 era and re-verified on this payload 2026-10-03: model id
+`frank-large`, native tools, thinking in `reasoning_content`,
+`reasoning_effort` ∈ {xhigh, medium, low} only (anything else HTTP 500),
+card-recipe sampling defaults server-side, vision through the BF16 projector
+(red PNG → **`Red`**, 74 prompt / 165 completion tokens). The traps list —
+`content: null` when reasoning eats the budget, the self-recovering MTP
+boundary log line, `enable_thinking:false` not `reasoning_effort:"none"` — is
+in [[frank-large-rocmfpx-q8agent-mtp]] and still applies.
+
+Sweep key for scratch servers is now the payload name:
+`pkill -f 'Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT'` — never `pkill -f
+llama-server` (frank-small is also one).
+
+## Operations
+
+```bash
+./scripts/deploy.sh dankclaw --service frank-large
+systemctl status frank-large
+journalctl -u frank-large -n 200 --no-pager
+scripts/download-frank-large-kingjones-uncensored.sh --verify-only
+```
+
+## Rollback
+
+| Target | How |
+|---|---|
+| **ALIGNED Q8_0_ROCMFPX_AGENT** (previous shape, 17 t/s) | the payload dir `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` is untouched on disk; revert the swap commit (unit/env/serve trio) and redeploy, or point `FRANK_LARGE_MODEL_FILE` + `LLM_LARGE_MODEL_FILE` at the old paths. Verify with `scripts/download-frank-large-kingjones.sh --verify-only` |
+| gufo Q4_K_XL era | git tag `frank-large-gufo-q4kxl-20261001`, **but the gufo payloads (UD-Q4_K_XL 16.35 GiB, UD-Q8_K_XL + DFlash2 drafter) were removed from disk 2026-10-03 with Frank's approval** — this rollback now requires re-download per [[frank-large-gufo-qwen38-27b]] |
+
+## When it misbehaves
+
+Same decision tree as [[frank-large-rocmfpx-q8agent-mtp]]:
+`unknown tensor type` ⇒ off the pinned fork; SIGSEGV in
+`ggml_compute_forward_set_rows` ⇒ pre-`36e9acd` build; slow decode with no
+error ⇒ draft not engaged (`draft_n` in `/completion` timings) or an orphaned
+second copy eating GTT (`/sys/class/drm/card1/device/mem_info_gtt_used`);
+500 naming Jinja ⇒ `--jinja` lost; 500 naming the template ⇒ unmapped
+`reasoning_effort` in `models.yml`.
@@ -1,7 +1,7 @@
 ---
 type: reference
-title: "frank-large on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP"
-description: "The active frank-large runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Q8_0_ROCMFPX_AGENT GGUF, a ftype-115 file no other loader can read, with the repo's own MTP-Q4_0 head drafting four tokens at one native 262,144 window, f16 KV, and the bundled BF16 projector for vision. Measured MTP-vs-none A/B, the client contract, and rollback to gufo."
+title: "frank-large on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (retired)"
+description: "The retired frank-large runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Q8_0_ROCMFPX_AGENT GGUF, a ftype-115 file no other loader can read, with the repo's own MTP-Q4_0 head, f16 KV, and the BF16 projector. Measured MTP-vs-none A/B, the client contract, and the rollback state as of 2026-10-03."
 tags:
  - llm
  - frank-large
@@ -15,14 +15,20 @@ tags:
 timestamp: 2026-10-02
 ---

-# frank-large — ROCmFPX Q8_0_ROCMFPX_AGENT + MTP (gfx1151)
+# frank-large — ROCmFPX Q8_0_ROCMFPX_AGENT + MTP (gfx1151, retired)

-> **ACTIVE since 2026-10-02.** Cut over from
-> [[frank-large-gufo-qwen38-27b]] (Q4_K_XL, ~15 t/s). Both requests came from
-> Frank: *highest quality that runs*, and *one full context*. This endpoint
-> delivers 8.39 bpw weights — two quant rungs above what gufo could load — and
-> measures **17.0 t/s** decode instead of 15, so quality was not bought with
-> speed.
+> **RETIRED 2026-10-03** — superseded by
+> [[frank-large-rocmfpx-q6agent-uncensored-mtp]]: Frank swapped to the
+> uncensored Q6_0_ROCMFPX_AGENT file (protected heads, one rung down), which
+> measures **27.6–28.1 t/s** at a deeper prompt than this file's 17.0. This
+> runbook stays live as the rollback reference and the trap catalogue; the
+> payload it documents remains on disk at
+> `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`.
+>
+> Cut over 2026-10-02 from [[frank-large-gufo-qwen38-27b]] (Q4_K_XL, ~15 t/s)
+> on two requests from Frank: *highest quality that runs*, and *one full
+> context*. It delivered 8.39 bpw weights — two quant rungs above what gufo
+> could load — at 17.0 t/s, so quality was not bought with speed.
 >
 > The capability that forced this whole shape is a GGUF tensor type:
 > **`GGML_TYPE_Q8_0_ROCMFPX` (103)**. It exists in exactly one loader on this
@@ -184,17 +190,24 @@ broken payload never stops the running unit. Caller environment **overrides**
 `frank-large.env`, and defaults use `:=` so an explicitly empty value surfaces
 as the error it reads as.

-## Rollback
+## Rollback (as of retirement, 2026-10-03)

-git tag **`frank-large-gufo-q4kxl-20261001`**: gufo v0.1.1
-(`scripts/build-gufo.sh` still on disk), the Q4_K_XL Qwen3.8-27B payload and the
-DFlash2 drafter all remain on the host. Restore that tag's unit/env/serve trio
-and redeploy — nothing here deletes anything.
+**Primary rollback: the ALIGNED Q8 file itself is the previous shape's
+payload and remains on disk** at
+`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` — revert the
+2026-10-03 swap commit (unit/env/serve trio) or repoint
+`FRANK_LARGE_MODEL_FILE`/`LLM_LARGE_MODEL_FILE`, verify with
+`scripts/download-frank-large-kingjones.sh --verify-only`, redeploy.

-The rollback payloads stay exactly where the tag's env file expects them:
-`/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL/` (16.35 GiB) and the DFlash2
-drafter `/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-DFlash2-Q4_K_M.gguf`
-(1.07 GiB). The one-day vLLM MiMo era's payloads (MiMo-V2.6 BF16 18 GiB,
+git tag **`frank-large-gufo-q4kxl-20261001`** (gufo v0.1.1, Q4_K_XL +
+DFlash2) is now a **re-download** rollback: the gufo payloads
+`/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL/` (16.35 GiB) and
+`/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/` (incl. the DFlash2 drafter) were
+removed from disk on 2026-10-03 with Frank's explicit approval to free space.
+`scripts/build-gufo.sh` remains on disk; re-fetch per
+[[frank-large-gufo-qwen38-27b]].
+
+The one-day vLLM MiMo era's payloads (MiMo-V2.6 BF16 18 GiB,
 Qwen3.5-9B MTP source 19 GiB, both DFlash draft dirs) were staged to
 `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02 —
 recoverable until the trash is purged, and never part of a rollback plan.
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF"
-description: "The active frank-small runbook — a CUDA (sm_86) build of llama.cpp master serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF on the RTX 3090: three conversations of 180,224 tokens, mmproj vision, the asymmetric KV contract (f16 K + q8_0 V), the two VRAM measurements that set the pool size, and what to check when it misbehaves."
+description: "The active frank-small runbook — a CUDA (sm_86) build of llama.cpp master serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF on the RTX 3090: two conversations of 200,704 tokens, fully unquantized f16 KV, mmproj vision, the boot-pair arithmetic that sizes the pool, and what to check when it misbehaves."
 tags:
  - llm
  - frank-small
@@ -11,23 +11,26 @@ tags:
  - mimo
  - gguf
  - kv-cache
-timestamp: 2026-10-02
+timestamp: 2026-10-03
 ---

 # frank-small — llama.cpp MiMo-V2.6 Q5_K_M (RTX 3090)

-> **ACTIVE since 2026-10-02.** One quant step down from
-> [[frank-small-llama-mimo-q6k]] on the same runtime and the same KV
-> contract, then **resized twice the same day** on Frank's orders: first to
-> three contexts with vision, then down to the pool that keeps 2–3 GiB of the
-> card free. Frank's arithmetic request — "as much as can fit while keeping
-> overhead open space to 2-3gb on the 3090" — is answered by the two
-> measurements in *Shape and the KV contract* below; the answer is
-> **3 × 180,224**, not 3 × 262,144.
+> **ACTIVE since 2026-10-02, resliced 2026-10-03.** One quant step down from
+> [[frank-small-llama-mimo-q6k]] on the same runtime, resized twice on
+> 2026-10-02 (three contexts with vision, then the pool that kept 2–3 GiB of
+> the card free), and on 2026-10-03 Frank ordered the third shape: **two
+> conversations of 200,704 tokens each with the KV VALUES unquantized** —
+> *"2 streams with 200700 tokens context or more… if we have extra space on
+> the 3090, unquantize the kv."* The freed third slot pays for exact V, and
+> the boot lands **20,530 MiB used / 4,046 MiB free** — above the 2–3 GiB
+> floor, with full-precision KV.

-> **The tradeoff is the per-slot window.** Three sessions cost 81,920 tokens
-> of per-conversation memory versus the two-slot shape: 180,224 tokens each
-> instead of the model's native 262,144. See *What three slots cost*.
+> **The tradeoffs of the reslice:** one fewer simultaneous conversation, a
+> per-slot window that grew from 180,224 to 200,704 (still below the native
+> 262,144), and ~2 % slower decode (104.8 vs 106.8 tok/s) from the larger
+> per-token KV traffic. The 3 × 180,224 shape is one env edit away — see
+> *Rollback*.

 ## Identity

@@ -41,77 +44,72 @@ timestamp: 2026-10-02

 ## Shape and the KV contract

-`--ctx-size 540672 --parallel 3` — llama.cpp divides the pool across slots, so
-each conversation gets `n_ctx_slot = 180224` (boot log: `n_slots = 3,
-n_ctx_slot = 180224, kv_unified = 'false'`). `CTX_SIZE` MUST stay divisible by
-`PARALLEL`; the bats seam enforces it.
+`--ctx-size 401408 --parallel 2` — llama.cpp divides the pool across slots,
+so each conversation gets `n_ctx_slot = 200704` (boot log 2026-10-03:
+`n_slots = 2, n_ctx_slot = 200704, kv_unified = 'false'`). `CTX_SIZE` MUST
+stay divisible by `PARALLEL`; the bats seam enforces it.

-The KV contract is Frank's: **K unquantized (f16), V `q8_0`** —
-`--cache-type-k f16 --cache-type-v q8_0`. Two hard dependencies ride on it:
+The KV contract is Frank's, as of 2026-10-03: **both K and V unquantized
+(f16)** — `--cache-type-k f16 --cache-type-v f16`. The asymmetric contract it
+replaced (f16 K + `q8_0` V) had existed to make three slots fit; with two
+slots the exact-value budget opens, and 4 KiB/token × 8 layers × 2 of extra
+V traffic is the measured ~2 % decode tax. The serve script still passes both
+flags explicitly — the asymmetric-era law that a quantized V requires
+`--flash-attn on` stays pinned by the seam.

- **`--flash-attn on` is mandatory**: quantized V exists only on the FA path.
-  Without the flag the engine falls back silently and the contract becomes a
-  lie. The serve script pins the trio; the bats seam pins the script.
- The VRAM budget: attention KV lives in the 8 full-attention layers only
-  (4 KV heads × 256 dim). K f16 ≈ 16 KiB/token, V q8_0 ≈ 8.5 KiB/token
-  (34 B per 32-element block) ⇒ ~24.5 KiB/token over the pool.
+**Pool sizes on this stack come from boots, not formulas.** The compute
+buffers, CUDA graphs and vision tower all bill separately, so every shape
+change is priced from measured rows:

-**The pool size came from two boots, not from a formula.** `free`-style
-arithmetic on this stack is unreliable because the compute buffers, CUDA
-graphs and now the vision tower all bill separately, so the fit was measured
-at two shapes and solved between them:
-
-|Shape|Projector|`nvidia-smi` used|KV at 24.5 KiB/token|Non-KV (weights + buffers + projector)|
+|Shape|KV dtype (K+V)|Projector|`nvidia-smi` used|Non-KV (weights + buffers + projector)|
 |---|---|---|---|---|
-|2 × 262,144 (pool 524,288)|none|20,190 MiB|12,800 MiB|**7,390 MiB**|
-|3 × 196,608 (pool 589,824)|f16|22,972 MiB|14,113 MiB|**8,859 MiB**|
+|2 × 262,144 (pool 524,288)|f16 + q8_0|none|20,190 MiB|**7,390 MiB**|
+|3 × 196,608 (pool 589,824)|f16 + q8_0|f16|22,972 MiB|**8,859 MiB**|
+|3 × 180,224 (pool 540,672)|f16 + q8_0|f16|21,684 MiB|~8,419 MiB|
+|**2 × 200,704 (pool 401,408)**|**f16 + f16**|f16|**20,530 MiB**|**~7,986 MiB**|

-The two rows price the projector and its image buffers at **~1.4 GiB** — not
-the ~0.9 GiB the 918 MB file suggests, because the vision encoder's own
-activation and compute buffers ride along. Solving the second row's non-KV cost
-against Frank's 2–3 GiB band: the card is 24,576 MiB, so
-`KV ≤ 24,576 − 9,216 − 2,048 = 13,312 MiB` ⇒ pool ≤ 556,630 tokens, rounded
-down to a multiple of 3 × 4,096 ⇒ **540,672**, i.e. 13,265 MiB of KV and
-**2,441 MiB free** (measured, 21,684 MiB used).
+The 32 KiB/token full-attention KV price (8 layers × 4 KV heads × 256 dim ×
+2 B × 2 halves) prices the deployed pool at 401,408 × 32 KiB = **12,544 MiB
+of KV**; the card is 24,576 MiB, so the shape leaves **4,046 MiB free**
+(measured steady-state — the KV pool is allocated whole at boot, so this is
+the ceiling). The headroom above the 2–3 GiB floor is deliberate slack, not
+an invitation: another 32,768 tokens of pool (1 GiB) would trade the last
+safety margin for window the harness compacts before reaching anyway.

-**Why 3 × 262,144 was never possible.** Three native windows are 786,432
-tokens = 18.4 GiB of KV *alone*; plus the 8.86 GiB of non-KV cost that is
-27.3 GiB on a 24 GiB card. The quant step bought 0.86 GiB; the geometry needed
-6 GiB.
+**Why 2 × 262,144 with f16 V does not fit either.** The native window pair is
+524,288 tokens = 16,384 MiB of KV plus ~7,986 non-KV = 24.4 GiB: under the
+floor by a hair. 200,704 is the largest 4,096-aligned slice that keeps the
+measured band.

-## Measured on this shape (2026-10-02, 3 × 180,224 + vision)
+## Measured on this shape (2026-10-03, 2 × 200,704 + vision, f16 KV)

 |Check|Result|
 |---|---|
-|Boot to healthy|~20 s, `n_slots = 3`, `n_ctx_slot = 180224`, `loaded multimodal model` logged|
-|VRAM|**21,684 MiB used / 2,441 MiB free** of 24,576 (`nvidia-smi`, steady state; the KV pool is allocated whole at boot, so this is the ceiling)|
-|Decode, temp 0, no thinking|**106.8 tok/s** (256-token completion at ~1,300-token depth)|
-|Prefill|**3,814 tok/s** (same request)|
-|Three concurrent requests|all three answered in 1.7 s wall — real parallel slots|
-|Fourth concurrent request|answered in 3.0 s while three ran — queued behind a released slot, as the slot ceiling requires|
-|Vision|`image_url` part with a 64×64 solid-red PNG → **"Red."**|
-|Tools|native `tool_calls` with correct arguments (`get_weather {"city":"Seattle","unit":"f"}`)|
-|Thinking|`reasoning_content` populated (145 chars on a "think first" prompt); `reasoning_effort: "low"` accepted|
-|Over-slot prompt|`400 exceed_context_size_error: request (184001 tokens) exceeds the available context size (180224 tokens)` — refused, never silently truncated|
+|Boot to healthy|~5 s, `n_slots = 2`, `n_ctx_slot = 200704`, `loaded multimodal model` logged|
+|VRAM|**20,530 MiB used / 4,046 MiB free** of 24,576 (`nvidia-smi`, steady state)|
+|Decode, temp 0, no thinking|**104.84 tok/s** (128-token completion at ~500-token depth; the 3 × 180,224 q8_0-V shape measured 106.8 — ~2 % is the honest price of exact V)|
+|Prefill|1,650 t/s on the same shallow 506-token probe (rates are depth- and prompt-dependent; not comparable across shapes)|
+|Batch geometry|`--batch-size 2048 --ubatch-size 1024` — ubatch 2048 failed the scratch boot; 1024 boots clean|
+|Vision|projector loads at boot (`loaded multimodal model`); the red-PNG probe stands from the 2026-10-02 verification — the vision path is unchanged by this reslice|
+|Tools / thinking|unchanged by the reslice: native `tool_calls`, thinking in `reasoning_content`|

-Decode is ~5 % **faster** than the 2 × 262,144 shape measured earlier the same
-day (100.4–101.8 tok/s): a shorter per-slot window means a shorter attention
-scan at equal prompt depth, and one less slot's DeltaNet state resident.
+## What the slot geometry costs

-## What three slots cost
+The per-slot window is the whole price, in both directions:

-The per-slot window is the whole price. Two consequences the fleet depends on:
-
- **A prompt over 180,224 tokens is a hard 400**, so `~/.omp/agent/models.yml`
-  sets `contextWindow: 180224` for this endpoint — the harness compacts before
+- **A prompt over 200,704 tokens is a hard 400**, so `~/.omp/agent/models.yml`
+  sets `contextWindow: 200704` for this endpoint — the harness compacts before
  it reaches the ceiling instead of discovering it.
- **frank-primary's compaction prompt fits inside one slot again.** At
-  frank-primary's shape-D window (200,704) its compaction trigger is 140,492
-  tokens (70 %), so one compaction request lands here with 39,732 tokens of
-  room. At the previous 262,144 window the trigger was 183,501 — *above* this
-  slot — and the request had to be truncated or split. Nothing in the fleet
-  routes compaction here automatically; the point is that the 2026-10-02 window
-  trim restored the fit.
+- **frank-large's tallest pre-compaction prompt now fits here.** Compaction
+  for frank-large sessions runs on this endpoint (`compactionModel` in
+  `models.yml`); frank-large compacts at 70 % of 262,144 = 183,501 tokens,
+  which did **not** fit the old 180,224 slot and fits 200,704 with 17,163
+  tokens to spare. The 2026-10-02 window trim had restored the fit for
+  frank-primary's smaller 140,492-token trigger; the 2026-10-03 reslice widens
+  the fit to frank-large itself. Nothing routes compaction automatically —
+  the point is the ceiling no longer forbids it.
+- Two simultaneous generations are the slot ceiling; a third request queues
+  behind a released slot (the mechanism measured on the 3-slot shape).

 ## Client contract

@@ -147,8 +145,9 @@ traffic first. The bats seam fails if a draft flag ships unannounced.

 |Target|How|
 |---|---|
+|Previous shape: 3 × 180,224 + q8_0 V (2026-10-02)|`scripts/frank-small.env` `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (measured 21,684 MiB used, 2,441 free) and `contextWindow: 180224` in `models.yml` — choose it when three simultaneous conversations matter more than per-slot window or exact V|
 |Q6_K, same runtime|`scripts/frank-small.env` `MODEL_FILE=/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` (7,793,710,624 B, still on disk) and the deployer's `LLM_SMALL_MODEL_FILE` default. Q6_K needs ~1.4 GiB more than Q5_K_M, so pair the rollback with `CTX_SIZE` 49,152 tokens lower or `MMPROJ_FILE=`|
-|Two 262,144 windows|`CTX_SIZE=524288 PARALLEL=2` (measured 20,190 MiB used, 4,386 free) and `contextWindow: 262144` in `models.yml` — choose it when a single conversation needs the native window or when frank-primary's compaction request has to land here|
+|Two 262,144 windows|`CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=q8_0` (measured 20,190 MiB used, 4,386 free **with q8_0 V** — with f16 V the same shape is 16,384 MiB of KV + ~7,986 non-KV and does not fit) and `contextWindow: 262144` in `models.yml` — choose it when a single conversation needs the native window|
 |Vision off|`MMPROJ_FILE=` in the env file; the deployer gate then has to be satisfied too, so point `LLM_SMALL_MMPROJ` at any real projector file or drop the gate in the same commit|
 |SGLang + INT8|[[frank-small-sglang-mimo-v26]] — venv and old unit/env/serve trio still in place; the INT8 payload was staged to `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02, so restore it there or re-fetch with `old/download-frank-small-sglang-int8.sh`|

@@ -158,13 +157,14 @@ traffic first. The bats seam fails if a draft flag ships unannounced.
  naming the missing file if the binary or GGUF moved.
 - `No CUDA GPUs are available`: the 3090 vanished from the bus or the UVM node
  is missing — `nvidia-modprobe -u -c=0` (the serve script runs it).
- Requests queue behind each other: three concurrent long generations are the
-  slot ceiling by contract; a fourth waits for a slot to free (measured: 1.7 s
-  for the three, 3.0 s for the fourth).
- `400 … exceeds the available context size (180224 tokens)`: the conversation
-  outgrew ONE slot. This is not pool exhaustion — the pool is 540,672 tokens
-  and the per-slot window is the binding limit. Shorten the prompt, compact, or
-  roll back to the 262,144 shape above.
+- Requests queue behind each other: two concurrent long generations are the
+  slot ceiling by contract (2026-10-03 shape); a third waits for a slot to
+  free (the mechanism measured on the 3-slot shape: 1.7 s for three, 3.0 s for
+  the fourth).
+- `400 … exceeds the available context size (200704 tokens)`: the conversation
+  outgrew ONE slot. This is not pool exhaustion — the pool is 401,408 tokens
+  and the per-slot window is the binding limit. Shorten the prompt, compact,
+  or roll back to the 262,144 shape above.
 - Vision answers a color it never saw: the client sent `images` instead of an
  `image_url` content part (see *Client contract*).
 - Slow decode after a CTX or quant change: check `nvidia-smi` — pushing KV out
@@ -1,6 +1,6 @@
 # LLM Fleet — Knowledge Base

-Documentation for the LLM inference fleet on the two hosts (2026-10-02):
+Documentation for the LLM inference fleet on the two hosts (2026-10-03):
 **Dankserver runs the tcclaviger RDNA4 vLLM fork on frank-primary** (the
 digest-pinned `tcclaviger/vllm` image driving `podman run` directly, serving
 orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 across both Radeon AI PRO R9700s
@@ -15,11 +15,12 @@ mounted cache dir** (the image ships no `/root/.cache` at all; unmounted, every
 SEQS 2 cannot compile this model. Rollback frank-primary-fp8-mtp-20261001),
 and Dankclaw
 runs two endpoints — **ROCmFPX llama.cpp (frank-large, port 7770:
-kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF Q8_0_ROCMFPX_AGENT, a ftype-115
-file only that fork can read, with the repo's MTP head drafting 4 tokens, ONE
-session at 262,144, f16 KV, vision via the bundled projector)** and
+kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
+Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT, a ftype-114 file only that fork can
+read, with the repo's MTP head drafting 4 tokens at 27.6-28.1 t/s decode, ONE
+session at 262,144, f16 KV, vision via the BF16 projector)** and
 **llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q5_K_M
-GGUF, 3 x 180,224, asymmetric KV f16 K + q8_0 V, on the RTX 3090)**.
+GGUF, 2 x 200,704, fully unquantized f16 KV, on the RTX 3090)**.

 > 2026-08-22: the former `vllmScripts` repository is merged into the
 > serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -46,12 +47,12 @@ expert-precise-memory arena 7,522 slots, /root/.cache mounted]
 gfx1151 iGPU 114 GiB GTT]
        E[ROCmFPX llama.cpp gfx1151, pinned fork build
 frank-large, port 7770]
-        F[Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP-Q4_0 head
+        F[Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP-Q4_0 head
 1 x 262144 ctx, f16 KV, draft n-max 4, mmproj vision]
        G[llama.cpp master CUDA sm_86
 frank-small, port 7771]
        H[MiMo-V2.6 Q5_K_M GGUF
-3 x 180224 ctx, f16 K + q8_0 V, flash-attn, mmproj vision]
+2 x 200704 ctx, f16 K + f16 V, flash-attn, mmproj vision]
        E2 --> E --> F
        E2 --> G --> H
    end
@@ -70,20 +71,23 @@ frank-small, port 7771]

 ## Active Deployment

- [[frank-large-rocmfpx-q8agent-mtp]] — **ACTIVE since 2026-10-02**
+- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — **ACTIVE since 2026-10-03**
  (root systemd unit, port 7770): the pinned kingjones30/ROCmFPX fork serving
-  the repo's highest-quality rung `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf`
-  (26.28 GiB, ftype 115 = 187 plain-Q8_0 + 311 GGML_TYPE_Q8_0_ROCMFPX
-  tensors) with the bundled `mtp-Qwen3.8-27B-Q4_0.gguf` head drafting 4
-  tokens, one 262,144 window, f16 KV, the BF16 projector for vision. Why the
-  runtime is not interchangeable, the pinned-build set_rows history, the
-  measured envelope, rollback to gufo
- [[frank-small-llama-mimo-q5km]] — **ACTIVE since 2026-10-02** (root
-  systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
-  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF Q5_K_M on the RTX 3090,
-  THREE slots of 180,224 tokens (pool 540,672) with K f16 + V q8_0 and the
-  mmproj for vision, ~100 tok/s decode, ~2.7 GiB of the card left free.
-  The KV arithmetic, the flash-attn dependency, rollback to Q6_K
+  the uncensored protected-heads rung `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`
+  (23.15 GiB, ftype 114 = 159 type-103 + 339 GGML_TYPE_Q6_0_ROCMFPX tensors,
+  64 protected output heads) with the repo's `mtp-Qwen3.8-27B-Q4_0.gguf` head
+  drafting 4 tokens, one 262,144 window, f16 KV, the BF16 projector for
+  vision. Measured 27.6-28.1 t/s prose decode at 86.8 % draft acceptance; the
+  ngram-mod chain onto MTP was measured and REJECTED (-15 %: it disables the
+  hybrid's recurrent state rollback). Rollback: the aligned Q8 file, still on
+  disk
+- [[frank-small-llama-mimo-q5km]] — **ACTIVE since 2026-10-02, resliced
+  2026-10-03** (root systemd unit, port 7771): llama.cpp master (CUDA sm_86)
+  serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF Q5_K_M on the RTX 3090,
+  TWO slots of 200,704 tokens (pool 401,408) with FULLY UNQUANTIZED KV (f16 K
+  + f16 V) and the mmproj for vision, 104.8 tok/s decode, 4,046 MiB of the
+  card left free. The boot-pair pool arithmetic, the slot-window contract,
+  rollback to 3 x 180,224 or Q6_K
 - [[dankserver-qwen38-fp8-expert-offload-tuning]] — **ACTIVE since
  2026-10-02 (shape E)** (root systemd unit, port 7770): the tcclaviger RDNA4
  vLLM fork serving orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 across both
@@ -192,8 +196,13 @@ frank-small, port 7771]
  (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-gufo-qwen38-27b]] — **RETIRED 2026-10-02 (the rollback
-  target)** (root systemd unit, port 7770): gufo
+- [[frank-large-rocmfpx-q8agent-mtp]] — **RETIRED 2026-10-02 → 10-03 (the
+  rollback target; payload still on disk)**: the same pinned ROCmFPX fork
+  serving the ALIGNED `Q8_0_ROCMFPX_AGENT` (ftype 115, 17.0 t/s, MTP
+  acceptance 0.462) — the MTP-vs-none +123 % A/B, the trap catalogue
+- [[frank-large-gufo-qwen38-27b]] — **RETIRED 2026-10-02 (history; the
+  payload dirs were removed from disk 2026-10-03, so this rollback is a
+  re-download)** (root systemd unit, port 7770): gufo
  (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from
  source on Dankclaw, serving unsloth/Qwen3.8-27B-GGUF **UD-Q4_K_XL**
  with the z-lab DFlash2 Q4_K_M drafter — ONE session at the full native
@@ -213,15 +213,21 @@ The LLM inference fleet on the two hosts. Full index:
  agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
  Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
  rollback record)
- [[frank-large-rocmfpx-q8agent-mtp]] — the ACTIVE frank-large runbook
-  since 2026-10-02 (root systemd unit, port 7770): the pinned kingjones30/
-  ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
-  Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF **Q8_0_ROCMFPX_AGENT** — ftype 115,
-  311 tensors of GGML_TYPE_Q8_0_ROCMFPX, a format no other loader on the
-  host reads — with the repo's Q4_0 MTP head drafting 4 tokens and its BF16
-  projector, one session at the native 262,144 window
- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-10-01 → 10-02 (rollback
-  target, payload and `scripts/build-gufo.sh` both on disk): gufo v0.1.1
+- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — the ACTIVE frank-large
+  runbook since 2026-10-03 (root systemd unit, port 7770): the pinned
+  kingjones30/ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
+  Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
+  **Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT** — ftype 114, 339 tensors of
+  GGML_TYPE_Q6_0_ROCMFPX + 159 type-103, a format no other loader on the
+  host reads — with the repo's Q4_0 MTP head drafting 4 tokens (27.6-28.1
+  t/s measured, 86.8 % acceptance; the ngram-mod chain measured and
+  rejected) and its BF16 projector, one session at the native 262,144 window
+- [[frank-large-rocmfpx-q8agent-mtp]] — RETIRED 2026-10-02 → 10-03 (rollback
+  target, aligned Q8 payload still on disk): the same fork serving the
+  ALIGNED Q8_0_ROCMFPX_AGENT at 17.0 t/s — the +123 % MTP A/B, the traps
+- [[frank-large-gufo-qwen38-27b]] — RETIRED 2026-10-01 → 10-02 (history;
+  the payload dirs were removed from disk 2026-10-03, `scripts/build-gufo.sh`
+  remains): gufo v0.1.1
  built from source serving Qwen3.8-27B UD-Q4_K_XL + the z-lab DFlash2
  drafter on the gfx1151 iGPU — the two from-source build workarounds, the
  template-hash allowlist, the measured prefill/decode trade
@@ -283,11 +289,11 @@ The LLM inference fleet on the two hosts. Full index:
 - [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
 - [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
 - [[frank-small-llama-mimo-q5km]] — the ACTIVE frank-small runbook since
-  2026-10-02: a CUDA (sm_86) build of llama.cpp master serving
-  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's RTX
-  3090, port 7771 — THREE conversations at 180,224 tokens each, the
-  asymmetric KV contract (f16 K + q8_0 V), vision through the repo's mmproj,
-  and the two VRAM measurements that set the pool size; ~100 tok/s decode
+  2026-10-02, resliced 2026-10-03: a CUDA (sm_86) build of llama.cpp master
+  serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's
+  RTX 3090, port 7771 — TWO conversations at 200,704 tokens each with fully
+  unquantized f16 KV, vision through the repo's mmproj, and the boot-pair
+  arithmetic that sizes the pool; 104.8 tok/s decode, 4,046 MiB free
 - [[frank-small-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; replaced
  by [[frank-small-llama-mimo-q5km]], payload still on disk): the first
  llama.cpp + GGUF shape for this endpoint, Q6_K at the same