docs: frank-small Q6_K cutover runbook + measured post-boot numbers

- new active runbook frank-small-llama-mimo-q6k-2x200704.md: 21,276 used /
  2,849 true free (inside the 2-3 GiB floor), decode 94.44 t/s vs 104.84 on
  Q5_K_M - the rung costs a measured -10 % (weight-bandwidth-bound card),
  512px red-PNG vision verified, and the new probe-size law (>=256 px: a
  10x10 image undersupplies image tokens and the model answers 'video'),
  nvidia-smi free-field law (driver holds ~450 MiB the subtraction hides)
- frank-small-llama-mimo-q5km.md retired; it keeps the boot-pair table and
  becomes the Q5 rollback reference (3,571 free, 104.84 t/s measured)
- AGENTS.md fleet row, endpoint section, --parallel lesson, runbook index
  (active row + two retired rows); LLM + root index entries
- Q6_K GGUF-vs-GGUF diff proven: template and all 46 KV keys byte-identical
  to Q5_K_M - the vision probe scare was probe size, not payload

serverAppConfigs-docs/LLM/frank-small-llama-mimo-q5km.md

@@ -1,7 +1,7 @@
 ---
 type: reference
-title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF"
-description: "The active frank-small runbook — a CUDA (sm_86) build of llama.cpp master serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF on the RTX 3090: two conversations of 200,704 tokens, fully unquantized f16 KV, mmproj vision, the boot-pair arithmetic that sizes the pool, and what to check when it misbehaves."
+title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF (retired)"
+description: "The retired 2026-10-02/03 frank-small runbook — the Q5_K_M rung at two wide unquantized-KV slots on the RTX 3090, superseded 2026-10-03 by the Q6_K file. Kept as the rollback reference: it carries the measured boot-pair table that prices every pool shape on this card, the slot-window contract, and the misbehaves tree."
 tags:
   - llm
   - frank-small
@@ -14,23 +14,23 @@ tags:
 timestamp: 2026-10-03
 ---
 
-# frank-small — llama.cpp MiMo-V2.6 Q5_K_M (RTX 3090)
+# frank-small — llama.cpp MiMo-V2.6 Q5_K_M (RTX 3090, retired)
 
-> **ACTIVE since 2026-10-02, resliced 2026-10-03.** One quant step down from
-> [[frank-small-llama-mimo-q6k]] on the same runtime, resized twice on
-> 2026-10-02 (three contexts with vision, then the pool that kept 2–3 GiB of
-> the card free), and on 2026-10-03 Frank ordered the third shape: **two
-> conversations of 200,704 tokens each with the KV VALUES unquantized** —
+> **RETIRED 2026-10-03** — superseded by
+> [[frank-small-llama-mimo-q6k-2x200704]]: the ~3.5 GiB this shape left free
+> bought the **Q6_K quant rung** (same runtime, same 2 × 200,704 pool, same
+> f16 KV), which measures 2,849 MiB free and 94.44 tok/s — 10 % slower
+> decode is the measured price of the higher rung on this bandwidth-bound
+> card. This file remains the rollback reference and the owner of the
+> measured boot-pair table.
+
+> Its own history: one quant step down from [[frank-small-llama-mimo-q6k]]
+> (2026-10-01), resized twice on 2026-10-02 (three contexts with vision, then
+> the 2–3 GiB-band pool), and on 2026-10-03 morning reshaped to **two
+> conversations of 200,704 tokens with the KV VALUES unquantized** —
 > *"2 streams with 200700 tokens context or more… if we have extra space on
-> the 3090, unquantize the kv."* The freed third slot pays for exact V, and
-> the boot lands **20,530 MiB used / 4,046 MiB free** — above the 2–3 GiB
-> floor, with full-precision KV.
-
-> **The tradeoffs of the reslice:** one fewer simultaneous conversation, a
-> per-slot window that grew from 180,224 to 200,704 (still below the native
-> 262,144), and ~2 % slower decode (104.8 vs 106.8 tok/s) from the larger
-> per-token KV traffic. The 3 × 180,224 shape is one env edit away — see
-> *Rollback*.
+> the 3090, unquantize the kv."* That shape measured 20,530 used / 4,046
+> apparent free (the true `nvidia-smi` free was 3,571 — see *Shape*).
 
 ## Identity
 
@@ -90,7 +90,7 @@ measured band.
 |Decode, temp 0, no thinking|**104.84 tok/s** (128-token completion at ~500-token depth; the 3 × 180,224 q8_0-V shape measured 106.8 — ~2 % is the honest price of exact V)|
 |Prefill|1,650 t/s on the same shallow 506-token probe (rates are depth- and prompt-dependent; not comparable across shapes)|
 |Batch geometry|`--batch-size 2048 --ubatch-size 1024` — ubatch 2048 failed the scratch boot; 1024 boots clean|
-|Vision|projector loads at boot (`loaded multimodal model`); the red-PNG probe stands from the 2026-10-02 verification — the vision path is unchanged by this reslice|
+|Vision|projector loads at boot (`loaded multimodal model`); red-PNG probe passed on 2026-10-02. Caveat discovered on 2026-10-03: probes must use a >= 256 px image — a 10x10 px PNG undersupplies image tokens and the model hallucinates ("it's a video")|
 |Tools / thinking|unchanged by the reslice: native `tool_calls`, thinking in `reasoning_content`|
 
 ## What the slot geometry costs
@@ -146,7 +146,7 @@ traffic first. The bats seam fails if a draft flag ships unannounced.
 |Target|How|
 |---|---|
 |Previous shape: 3 × 180,224 + q8_0 V (2026-10-02)|`scripts/frank-small.env` `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (measured 21,684 MiB used, 2,441 free) and `contextWindow: 180224` in `models.yml` — choose it when three simultaneous conversations matter more than per-slot window or exact V|
-|Q6_K, same runtime|`scripts/frank-small.env` `MODEL_FILE=/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` (7,793,710,624 B, still on disk) and the deployer's `LLM_SMALL_MODEL_FILE` default. Q6_K needs ~1.4 GiB more than Q5_K_M, so pair the rollback with `CTX_SIZE` 49,152 tokens lower or `MMPROJ_FILE=`|
+|Q6_K, same runtime|`scripts/frank-small.env` `MODEL_FILE=/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` + `MMPROJ_FILE` in that dir — **this is the deployed rung since 2026-10-03** ([[frank-small-llama-mimo-q6k-2x200704]]): at 2 × 200,704 it fits with 2,849 MiB free, no pool cut needed (the older "+1.4 GiB, drop 49,152 tokens" estimate held only for the 3-slot shapes)|
 |Two 262,144 windows|`CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=q8_0` (measured 20,190 MiB used, 4,386 free **with q8_0 V** — with f16 V the same shape is 16,384 MiB of KV + ~7,986 non-KV and does not fit) and `contextWindow: 262144` in `models.yml` — choose it when a single conversation needs the native window|
 |Vision off|`MMPROJ_FILE=` in the env file; the deployer gate then has to be satisfied too, so point `LLM_SMALL_MMPROJ` at any real projector file or drop the gate in the same commit|
 |SGLang + INT8|[[frank-small-sglang-mimo-v26]] — venv and old unit/env/serve trio still in place; the INT8 payload was staged to `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02, so restore it there or re-fetch with `old/download-frank-small-sglang-int8.sh`|
@@ -165,7 +165,9 @@ traffic first. The bats seam fails if a draft flag ships unannounced.
   outgrew ONE slot. This is not pool exhaustion — the pool is 401,408 tokens
   and the per-slot window is the binding limit. Shorten the prompt, compact,
   or roll back to the 262,144 shape above.
-- Vision answers a color it never saw: the client sent `images` instead of an
-  `image_url` content part (see *Client contract*).
+- Vision answers a color it never saw, or claims the image "is a video":
+  either the client sent `images` instead of an `image_url` content part, or
+  the probe image is too small to supply real image tokens (use >= 256 px;
+  see *Client contract*).
 - Slow decode after a CTX or quant change: check `nvidia-smi` — pushing KV out
   of VRAM into unified memory halves the rate silently.

serverAppConfigs-docs/LLM/frank-small-llama-mimo-q6k-2x200704.md

@@ -0,0 +1,106 @@
+---
+type: reference
+title: "frank-small on Dankclaw — llama.cpp, MiMo-V2.6 Q6_K, 2 x 200704, f16 KV"
+description: "The active frank-small runbook — llama.cpp master (CUDA sm_86) serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF on the RTX 3090: two conversations of 200,704 tokens with fully unquantized f16 KV, vision through the bundled projector, and the boot-pair arithmetic that prices every pool change."
+tags:
+  - llm
+  - frank-small
+  - dankclaw
+  - llama-cpp
+  - gguf
+  - rtx3090
+timestamp: 2026-10-03
+---
+
+# frank-small — llama.cpp MiMo Q6_K, 2 × 200,704, f16 KV
+
+> **ACTIVE since 2026-10-03.** The third shape Frank bought in two days on
+> this card: the freed third slot paid for **exact KV values**
+> ([[frank-small-llama-mimo-q5km]] documents that step and remains the
+> rollback), and the ~3.5 GiB still free then paid for the **Q6_K quant rung**
+> — the same file this endpoint served on 2026-10-01, come back as an
+> upgrade rather than a compromise. The pool held at 401,408 (2 × 200,704);
+> only the weights moved up.
+
+## Identity
+
+| | |
+|---|---|
+| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
+| Endpoint | `http://192.168.0.143:7771` — alias `frank-small`, `/v1` OpenAI surface |
+| Runtime | llama.cpp master, CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (`GGML_CUDA_FA_ALL_QUANTS=ON`) |
+| Unit | root systemd `frank-small.service` ([`scripts/frank-small.service`](../../scripts/frank-small.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
+| Serve script | [`scripts/serve-frank-small.sh`](../../scripts/serve-frank-small.sh), tunables [`scripts/frank-small.env`](../../scripts/frank-small.env) |
+| Weights | `/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` — bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K**, 7,793,710,624 B, sha256 `ef96d05a…` (qwen3_5 hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, native 262,144, no MTP tensors) |
+| Projector | `…/MiMo-Q6K/mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf` — 918,166,048 B, sha256 `ff348f…` (hardlink of the Q5KM dir's copy, link-count-2 verified; the dir is self-contained) |
+| Fetch | [`scripts/download-frank-small.sh`](../../scripts/download-frank-small.sh) — `-f`, byte count + LFS sha256 verified, `--verify-only`; both files verified against the pinned sha on the host before this deploy |
+| Seam | `tests/frank_small_unit.bats` (argv assertions against a stub engine: payload path, slot geometry, KV pair, batch pair, projector opt-out) |
+
+## Shape and KV contract
+
+`--ctx-size 401408 --parallel 2` — llama.cpp divides the pool across slots,
+so each conversation gets `n_ctx_slot = 200704`. `CTX_SIZE` MUST stay
+divisible by `PARALLEL`; the seam enforces it. Both K and V are unquantized
+(f16) — the flags stay explicit because `--flash-attn` must not be lost if V
+ever goes back to `q8_0` (the serve script pins it, the seam pins the pin).
+
+**The Q6_K rung is a weights-side spend only.** File delta +917,585,920 B
+(+875 MiB) against Q5_K_M, same KV, same slot geometry, same batch pair.
+Priced from the measured Q5 boot at this shape (non-KV 7,986 MiB): the boot
+landed **21,276 MiB used / 2,849 MiB true free** — non-KV measured 8,732 MiB,
+inside the 2–3 GiB floor, so the pool stayed at 401,408 and the
+12,288-token step-down ladder was never climbed. Quote `nvidia-smi`'s `free`
+field, never `total − used`: the driver holds ~450 MiB of reserve the
+subtraction hides (that error once overstated this card's slack by a
+half-GiB).
+
+**Pool prices on this card come from boots, never formulas** — the measured
+boot-pair table lives in [[frank-small-llama-mimo-q5km]] (*Shape and KV
+contract*) and still prices every rollback shape.
+
+## Measured on this shape (2026-10-03, Q6_K, 2 × 200,704, f16 KV)
+
+|Check|Result|
+|---|---|
+|Boot to healthy|model + projector resident in ~2.3 s (load log `0.02.27`), `n_slots = 2`, `n_ctx_slot = 200704`, `loaded multimodal model`, health 200; deploy e2e gate 33/33|
+|VRAM|**21,276 MiB used / 2,849 MiB free** of 24,576 (`nvidia-smi` `free`, steady state) — inside the 2–3 GiB floor with exact V AND the higher rung|
+|Decode, temp 0, no thinking|**94.44 tok/s** on the identical 506-token probe that measured 104.84 on Q5_K_M — **−10 %**. Decode on this card is weight-bandwidth-bound, and Q6_K streams ~10 % more bytes per token; the quant rung buys quality with decode rate, measurably|
+|Prefill|2,229.9 t/s (same shallow probe; rates are depth-dependent, not comparable across probes)|
+|Vision|512×512 solid-red PNG → **`Red`** (277 prompt tokens — the image really reached the tower). See the probe-size trap in *Client contract*|
+|Tools / thinking|native `tool_calls`, thinking in `reasoning_content`, thinking OFF unless `enable_thinking: true` (MiMo template default; `models.yml` ships it in `extraBody`)|
+
+## Client contract
+
+llama.cpp surface, live-verified on this model family: model id `frank-small`,
+native tools via `--jinja` (`tool_calls` in the response), thinking in
+**`reasoning_content`**, vision via `image_url` content parts. Two vision
+traps, both measured here:
+
+1. A top-level `images` field on `/v1/chat/completions` is silently ignored
+   and the model invents a color.
+2. **The probe image must be big enough to supply real image tokens.** A
+   10×10 px solid-red PNG came back "The image is a video, so I can't see its
+   content" — the engine itself warns at boot that Qwen-VL wants ≥ 1024 image
+   tokens, and a postage-stamp image lands one patch below usable signal.
+   512×512 answers `Red` in 19 tokens. Always probe vision with a ≥ 256 px
+   image; a wrong-color answer from a tiny image is the probe's fault, not
+   the endpoint's.
+
+`~/.omp/agent/models.yml` sets `contextWindow: 200704` — the PER-SLOT window,
+so the harness compacts before llama.cpp answers a longer prompt with a 400.
+
+## Rollback
+
+|Target|How|
+|---|---|
+|Q5_K_M at the same shape|`scripts/frank-small.env` `MODEL_FILE=/home/dank/models/MiMo-Q5KM/MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf` (+ `MMPROJ_FILE` in that dir) and the deployer default — buys back 875 MiB (3,571 free measured) **and the 10 % decode rate** (104.84 tok/s measured). Payload sha-verified on disk|
+|3 × 180,224 + q8_0 V|`CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (measured 21,684 used / 2,441 free **with Q5_K_M** — Q6_K does NOT fit that shape inside the floor; pair them explicitly)|
+|SGLang + INT8|[[frank-small-sglang-mimo-v26]] — payload staged to `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/`|
+
+## When it misbehaves
+
+Same decision tree as [[frank-small-llama-mimo-q5km]] (*When it misbehaves*):
+`No CUDA GPUs are available` ⇒ `nvidia-modprobe -u -c=0`; a 400 naming
+200,704 ⇒ the conversation outgrew one slot, not the pool; slow decode after
+a shape change ⇒ check `nvidia-smi` for KV pushed into unified memory; queue
+behind two long generations ⇒ the slot ceiling is two, by contract.

serverAppConfigs-docs/LLM/index.md

@@ -19,7 +19,7 @@ kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
 Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT, a ftype-114 file only that fork can
 read, with the repo's MTP head drafting 4 tokens at 27.6-28.1 t/s decode, ONE
 session at 262,144, f16 KV, vision via the BF16 projector)** and
-**llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q5_K_M
+**llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q6_K
 GGUF, 2 x 200,704, fully unquantized f16 KV, on the RTX 3090)**.
 
 > 2026-08-22: the former `vllmScripts` repository is merged into the
@@ -81,13 +81,18 @@ frank-small, port 7771]
   ngram-mod chain onto MTP was measured and REJECTED (-15 %: it disables the
   hybrid's recurrent state rollback). Rollback: the aligned Q8 file, still on
   disk
-- [[frank-small-llama-mimo-q5km]] — **ACTIVE since 2026-10-02, resliced
-  2026-10-03** (root systemd unit, port 7771): llama.cpp master (CUDA sm_86)
-  serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF Q5_K_M on the RTX 3090,
-  TWO slots of 200,704 tokens (pool 401,408) with FULLY UNQUANTIZED KV (f16 K
-  + f16 V) and the mmproj for vision, 104.8 tok/s decode, 4,046 MiB of the
-  card left free. The boot-pair pool arithmetic, the slot-window contract,
-  rollback to 3 x 180,224 or Q6_K
+- [[frank-small-llama-mimo-q6k-2x200704]] — **ACTIVE since 2026-10-03**
+  (root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
+  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090, TWO
+  slots of 200,704 tokens (pool 401,408) with FULLY UNQUANTIZED KV (f16 K +
+  f16 V) and the mmproj for vision — 94.4 tok/s decode (the rung costs a
+  measured -10 % vs Q5_K_M: weight-bandwidth-bound), 2,849 MiB of the card
+  left free. Probe-size trap (>=256 px images or the model says "video")
+  documented. Rollback: one MODEL_FILE line back to Q5_K_M
+- [[frank-small-llama-mimo-q5km]] — **RETIRED 2026-10-02 → 10-03 (rollback
+  target; payload on disk; owns the measured boot-pair table)**: the same
+  runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
+  104.8 tok/s, 3,571 MiB true free
 - [[dankserver-qwen38-fp8-expert-offload-tuning]] — **ACTIVE since
   2026-10-02 (shape E)** (root systemd unit, port 7770): the tcclaviger RDNA4
   vLLM fork serving orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 across both
@@ -252,8 +257,10 @@ frank-small, port 7771]
   .hgn + quality overlay) on the gfx1151 iGPU — two 200k conversations over one
   400,384-position pool, deployed flags, memory model, client contract,
   troubleshooting (2026-09-16 → 09-19)
-- [[frank-small-llama-mimo-q6k]] — **RETIRED 2026-10-01 → 10-02 (rollback
-  target)**: llama.cpp (master, CUDA sm_86 build) serving
+- [[frank-small-llama-mimo-q6k]] — **RETIRED 2026-10-01 → 10-02** (the same
+  rung returned ACTIVE 2026-10-03 at the 2 x 200,704 f16-KV shape —
+  [[frank-small-llama-mimo-q6k-2x200704]]): llama.cpp (master, CUDA sm_86
+  build) serving
   bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090 — TWO
   native 262,144-token sessions with ASYMMETRIC KV (K f16 unquantized + V
   q8_0, which forces flash-attn), ~20.9/24.6 GiB measured, text-only (the

serverAppConfigs-docs/index.md

@@ -288,16 +288,20 @@ The LLM inference fleet on the two hosts. Full index:
 - [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
 - [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
 - [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
-- [[frank-small-llama-mimo-q5km]] — the ACTIVE frank-small runbook since
-  2026-10-02, resliced 2026-10-03: a CUDA (sm_86) build of llama.cpp master
-  serving bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q5_K_M** on Dankclaw's
-  RTX 3090, port 7771 — TWO conversations at 200,704 tokens each with fully
-  unquantized f16 KV, vision through the repo's mmproj, and the boot-pair
-  arithmetic that sizes the pool; 104.8 tok/s decode, 4,046 MiB free
-- [[frank-small-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; replaced
-  by [[frank-small-llama-mimo-q5km]], payload still on disk): the first
-  llama.cpp + GGUF shape for this endpoint, Q6_K at the same
-  2 x 262,144 / f16-K + q8_0-V contract, 20.9 GiB measured
+- [[frank-small-llama-mimo-q6k-2x200704]] — the ACTIVE frank-small runbook
+  since 2026-10-03: llama.cpp master (CUDA sm_86) serving
+  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on Dankclaw's RTX 3090,
+  port 7771 — TWO conversations at 200,704 tokens each with fully
+  unquantized f16 KV and mmproj vision; 94.4 tok/s decode (the Q6_K rung
+  costs a measured −10 % vs Q5_K_M — weight-bandwidth), 2,849 MiB free, and
+  the ≥256 px image-probe trap (tiny images answer "video")
+- [[frank-small-llama-mimo-q5km]] — RETIRED 2026-10-02 → 10-03 (rollback
+  target, payload on disk; owns the measured boot-pair table): same runtime
+  and shape one rung lower — 104.8 tok/s, 3,571 MiB true free
+- [[frank-small-llama-mimo-q6k]] — RETIRED 2026-10-01 (same day; that same
+  Q6_K rung returned ACTIVE 2026-10-03 at the new shape, see
+  [[frank-small-llama-mimo-q6k-2x200704]]): the first llama.cpp + GGUF shape
+  for this endpoint, Q6_K at 2 x 262,144 / f16-K + q8_0-V, 20.9 GiB measured
 - [[frank-small-sglang-mimo-v26]] — RETIRED 2026-10-01 (rollback target):
   SGLang in a self-contained python3.12 venv serving
   letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound (qwen3_5