docs(frank-small): record the batch doubling and scope the ubatch law to its backend

The AGENTS.md lesson "`--ubatch-size` has an OPTIMUM on this stack" was load-
bearing and over-general: it was measured on the 2x R9700 layer split, and the
same knob on the CUDA 3090 behaves the opposite way (curve climbs monotonically
to 2048). That difference is now the reason frank-small ships ubatch 2048, so
the lesson names the backend it was measured on and points at the counterpoint.

Adds the 2026-10-05 boot-pair A/B to the runbook (+4.6 % prefill at 8,928
tokens for 616 MiB, decode flat, the sub-ubatch prompt as the control), marks
the superseded 21,276/2,849 VRAM row, and logs the day. The old "ubatch 2048
failed the scratch boot" line stays visible as what it was: a predecessor-shape
observation, not a law.

AGENTS.md

@@ -26,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 |--------|---------|---------|-------|------|
 | Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **fp8 KV** (`--kv-cache-dtype fp8`), **no speculative decode**, MNBT 2048 (arena-free: the registry prices overhead flat up to its nbt 4096 anchor), MoE gather 32×64, **`--expert-precise-memory`** (fitted 5.25 GiB overhead, not the 7.60 GiB default), **`VLLM_CUDAGRAPH_MEM_BASE_MB=0` / `_PER_GRAPH_MB=12`** (the plan's CUDA-graph term priced at the measured capture — 0.31 GiB then a NEGATIVE −0.06 GiB delta, so the 256 MiB analytic base was fiction → graph term 0.07 GiB), **`GPU_MAX_HW_QUEUES=2`**, **mounted cache dir** at `/root/.cache` (the image ships no `/root/.cache` at all; unmounted, every boot re-pays `torch.compile took 181.20 s`), **3 slots × 200704 ctx** at KV_BYTES 3,166,500,000/rank (~400,700 KV tokens at the engine's 7,891.8 B/token here) — the **pool, not the slot count, caps resident conversations** (measured live: `GPU KV cache size: 401,408 tokens, Maximum concurrency for 200,704 tokens per request: 2.00×`; the 3.15e9 pool below it measured 398,658/1.99×), and shorter requests use the spare blocks. **`--max-num-seqs` must stay 3: SEQS=2 cannot boot this model** — torch.fx `ConstraintViolationError` on `query_start_loc`, measured at BOTH MNBT 1024 and 2048. **util stays 0.982 BY LAW**: at `le=1` the worker init refuses `budget == total` with `ValueError: Free memory on device cuda:N (31.29/31.86 GiB)` on both ranks before a single weight loads — measured crash loop 2026-10-03. Arena at shape G: **17.77 GiB / 7,762 slots measured**, KV 401,408 tokens / 2.00× concurrency, 0.89 GiB of real free VRAM per card held through a live 4,096-token generation (the ~3.5 GiB of driver-reported free VRAM at shape E was estimator slack — the registry's 9190 B/token KV reserve vs the real 7,892, and the 5.25 GiB overhead vs far less — NOT allocatable), and the client contract wired: `--reasoning-parser …
 | Dankclaw | `frank-large` | **ROCmFPX llama.cpp fork** (`kingjones30/ROCmFPX` @ `36e9acd`, gfx1151, built by `scripts/build-rocmfpx.sh`; bare `llama-server`, root systemd unit) | `kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF` **`Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`** (23.15 GiB, ftype 114 ≈ 7.44 bpw: 353 F32 + 159 `Q8_0_ROCMFPX` + 339 `GGML_TYPE_Q6_0_ROCMFPX` tensors — only this fork can read them — with 64 protected output-head tensors), **MTP spec decode from the repo's own `mtp-Qwen3.8-27B-Q4_0.gguf` (n-max 4, pinned to ROCm0)**, **1 session × 262,144 (native window)** with **f16 KV**, the BF16 mmproj for vision, thinking on with the card's recipe (temp 1.0 / top_p 0.95 / top_k 20). Measured 2026-10-03: prose decode **27.6–28.1 t/s** at 4,813-token depth (draft acceptance 396/456 = 0.868, mean acceptance length ≈ 4.4), schema-echo 31.4 t/s at 100 %; `ngram-mod` chained onto MTP measured **−15 % on prose** and was rejected (the chain disables the hybrid's recurrent-state rollback). Cutover 2026-10-03 from the aligned Q8 file (17.0 t/s, acceptance 0.462 — rollback payload still on disk) | 7770 |
-| Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** (9B dense qwen3_5 hybrid, 7.26 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **2 × 200,704** (pool 401,408 — NOT the native window) with **fully unquantized KV (f16 K + f16 V)** under flash attention, 21,276 used / **2,849 free** of 24,576 MiB, 94.4 tok/s decode. Resized 2026-10-03 (3 × 180,224 + q8_0 V → 2 × 200,704 + f16 V → + Q6_K rung: the freed slots paid for exact values AND the higher quant; the rung costs a measured −10 % decode — weight-bandwidth-bound); rollback `MODEL_FILE=…/MiMo-Q5KM/…Q5_K_M.gguf` (buys the 10 % back) or `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (Q5 only) | 7771 |
+| Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** (9B dense qwen3_5 hybrid, 7.26 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **2 × 200,704** (pool 401,408 — NOT the native window) with **fully unquantized KV (f16 K + f16 V)** under flash attention, **batch 4096 / ubatch 2048** (doubled 2026-10-05: measured +4.6 % prefill at 8,928 tokens for 616 MiB), 21,846 used / **2,279 free** of 24,576 MiB, 94.4 tok/s decode. Resized 2026-10-03 (3 × 180,224 + q8_0 V → 2 × 200,704 + f16 V → + Q6_K rung: the freed slots paid for exact values AND the higher quant; the rung costs a measured −10 % decode — weight-bandwidth-bound); rollback `MODEL_FILE=…/MiMo-Q5KM/…Q5_K_M.gguf` (buys the 10 % back) or `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (Q5 only) | 7771 |
 
 Retired endpoints (`frank-npu`, and every prior runtime) are in
 the [runbook index](#runbook-index). The full swap history lives in
@@ -413,10 +413,18 @@ drives that term.
 **The shape (2026-10-03):** `--ctx-size 401408 --parallel 2` = **TWO
 conversations of 200,704 tokens each** (llama.cpp divides the pool:
 `n_ctx_slot = CTX_SIZE / PARALLEL`), `--flash-attn on` (the seam pins it),
-**f16 K + f16 V**, all layers on the GPU, `--mmproj` for vision, batch
-2048/ubatch 1024 (ubatch 2048 failed the scratch boot). Measured VRAM **21,276
-used / 2,849 free** of 24,576 MiB — inside the 2–3 GiB floor Frank asked for
-(non-KV 8,732 MiB; f16 K+V is 32 KiB/token across the 8 attention layers;
+**f16 K + f16 V**, all layers on the GPU, `--mmproj` for vision, and since
+2026-10-05 **batch 4096 / ubatch 2048** (doubled; the boot-pair A/B is in the
+runbook). Measured trade at 8,928 prompt tokens: prefill 4,024 → **4,209 t/s,
++4.6 %**, decode flat (91.9–93.0 vs 93.8–94.3 — noise), and a 279-token prompt,
+which sits below either ubatch, did not move at all. The old "ubatch 2048
+failed the scratch boot" note belongs to the Q5_K_M predecessor and does NOT
+generalize: this shape boots 2048 clean. Measured VRAM **21,846 used / 2,279
+free** of 24,576 MiB against **21,276 / 2,849** for the 1024 arm, reproduced
+byte-identically the same hour — the doubling costs **616 MiB** and lands at
+the BOTTOM of the 2–3 GiB floor Frank asked for (`4096/1536` is the middle rung
+if that margin is ever wanted; non-KV 9,302 MiB, up from 8,732 by exactly the
+compute buffers; f16 K+V is 32 KiB/token across the 8 attention layers;
 quote `nvidia-smi`'s `free` field, never `total − used` — the driver holds
 ~450 MiB of reserve the subtraction hides). The 2026-10-02 3 × 180,224 +
 q8_0-V shape stays one env edit away (`CTX_SIZE=540672 PARALLEL=3
@@ -814,7 +822,8 @@ linked runbooks.
   trip moved the balance point FOUR layers (~4.3 GiB of headroom, 40:25 ->
   36:29), which is the practical size of the effect. See
   [[frank-medium-llamacpp-dflash2]].
-- **`--ubatch-size` has an OPTIMUM on this stack, not a plateau — and its
+- **`--ubatch-size` has an OPTIMUM on the 2x R9700 layer split, not a plateau —
+  and the curve is BACKEND-SPECIFIC, so never carry it across (next bullet). Its
   compute buffers land about TWICE as heavy on the card that holds the output
   layers and the drafter.** Measured on frank-medium 2026-09-16 at the deployed
   shape (262144 over two slots, a 64k prefill measured at 64k depth,
@@ -832,6 +841,19 @@ linked runbooks.
   its 2020 MiB pp buffer here under the DSpark drafter, when ROCm1 had 0.08 GiB
   free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
   model — the sweep's or the seam's — can see it.)
+- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
+  frank-small ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
+  live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
+  ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
+  2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
+  279-token prompt, which is below either ubatch, came back identical — that is
+  the mechanical proof the knob was the only variable, since a prompt shorter
+  than the ubatch cannot see it. The logical `--batch-size` was again free. And
+  the predecessor's "ubatch 2048 failed the scratch boot" was a shape
+  observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
+  it by restarting the SERVICE through a transient `FRANK_SMALL_ENV_FILE`
+  drop-in, never a scratch port — one card holds one server. See
+  [[frank-small-llama-mimo-q6k-2x200704]].
 - **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
   size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
   worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one