frank-small: Frank overrules q8_0 V — f16 K+V, ubatch 1024 first, ONE native 262144 slot (the only f16/f16 shape that fits Q8 weights on 24 GiB)

scripts/frank-small.env

@@ -9,21 +9,20 @@
 # 262144, dense (no expert keys), nextn_predict_layers=1 (an MTP head is IN
 # the file — capability only; no spec decode ships, same law as before).
 #
-# THE KV CONTRACT MOVED BACK TO f16 K + q8_0 V, and here is the arithmetic
-# that forces it (2026-10-05, from the header + measured MiMo numbers):
+# THE KV CONTRACT STAYS f16 K + f16 V — NO KV QUANTIZATION (Frank, 2026-10-05,
+# overruling a written q8_0-V proposal). The arithmetic that makes this hard:
 #   KV rate 9 attn layers x 4 heads x 256 x 2 B = 18 KiB/token K, same again
-#   for V at f16 => 36 KiB/token (MiMo was 32: one fewer full-attn layer).
-#   Weights +1,900 MiB (9,334 vs 7,432). Measured free at the Q6_K shape was
-#   2,279 MiB, so Q8_0 at f16/f16 lands ~379 MiB free — a dead boot.
-#   Pool-stepping cannot rescue it: the harness compaction invariant
-#   (0.7 x slot + 57,344 <= slot) needs slot >= 191,148, so two slots need
-#   >= 13.6 GiB of f16/f16 KV even at the minimum legal window. The ONLY
-#   lever that keeps both the 2 x 200,704 window contract and the 2-3 GiB
-#   floor is q8_0 on V (26.25 KiB/token -> ~10.05 GiB KV -> ~2.87 GiB free).
-#   The 2026-10-03 "unquantized both halves" note was bought with the VRAM
-#   the third slot's death freed; the Q8 rung re-spends it. K stays f16.
-# MEASURE after the boot: under ~1.5 GiB free, step UBATCH_SIZE down to 1024
-# (the measured 616 MiB rung), never the window.
+#   for V at f16 => 36 KiB/token (MiMo was ~24.5 measured: one fewer
+#   full-attn layer AND one fewer KV head).
+#   Weights +1,900 MiB (9,334 vs Q6_K's 7,432). Measured free at the Q6_K
+#   4096/2048 shape was 2,279 MiB; Q8_0 at f16/f16 does not fit that shape —
+#   the deficit is made up the way Frank ordered: UBATCH FIRST (2048 -> 1024,
+#   the measured 616 MiB rung), then POOL STEPS, then measure. Guess-and-check
+#   on the live card is the accepted method for llama.cpp sizing here: boot,
+#   read nvidia-smi free, step the pool (12,288-token increments, ~432 MiB
+#   each at the 36 KiB/token rate) until the 2-3 GiB floor holds. Compute
+#   buffers allocate at graph capture during BOOT, so a failure is loud and
+#   fast, not a surprise at the first long prompt.
 #
 # Previous shape history (MiMo era, for the rollback trail): 2026-10-03 the
 # rung went Q5_K_M -> Q6_K with the pool HELD at 401,408; the freed third
@@ -69,11 +68,11 @@
 LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
 MODEL_FILE=/home/dank/models/Ornith-Q8/Ornith-1.5-9B-Q8_0.gguf
 
-# Vision: the Q6_K GGUF ships no `v.*` tensors either, so the projector file
-# IS the vision capability (sha256 ff348f31..., 918,166,048 B, fetched by
-# scripts/download-frank-small.sh; hardlinked into this dir from MiMo-Q5KM/,
-# same bytes — the Q6_K dir is self-contained). Empty string = text-only on
-# purpose.
+# Vision: the Ornith Q8_0 GGUF ships no `v.*` tensors either (dense-9B byte
+# arithmetic: 9.79 GB ≈ 9B params x ~1.06 B/param leaves no room for a tower),
+# so the projector file IS the vision capability (mmproj-Ornith-1.5-9B-BF16
+# .gguf, 921,704,672 B, LFS oid 626f9f90..., fetched by
+# scripts/download-frank-small.sh). Empty string = text-only on purpose.
 MMPROJ_FILE=/home/dank/models/Ornith-Q8/mmproj-Ornith-1.5-9B-BF16.gguf
 
 # Fleet endpoint: same port and alias as every frank-small before it, so the
@@ -82,16 +81,18 @@ ALIAS=frank-small
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7771
 
-# Two independent conversations. llama.cpp divides --ctx-size across
-# --parallel slots (n_ctx_slot = CTX_SIZE/PARALLEL), and a mamba-hybrid model
-# cannot use unified KV, so each slot gets its own cache and its own window.
-# 200,704 per slot is BELOW the native 262,144 — the price of paying for the
-# window in full precision — and ABOVE the harness's compaction trigger, which
-# is the point: a full-length request now lands in one stream instead of a 400
-# exceed_context_size_error. Cost of the third slot going away: a third
-# concurrent request queues behind a released slot instead of running.
-CTX_SIZE=401408
-PARALLEL=2
+# ONE conversation at the NATIVE 262,144 window (Frank, 2026-10-05). The
+# two-slot shape died under Ornith and the NO-QUANTIZE law, by arithmetic:
+# the KV rate is the ARCHITECTURE's, not the quant's — 9 full-attn layers x
+# 4 KV heads x 256 dim x 2 B = 36 KiB/token at f16 K+V (MiMo measured 24.5),
+# so 2 x 200,704 wants ~14.1 GiB of KV on top of +1,900 MiB of Q8 weights
+# against 2,279 MiB measured free; ubatch 1024 claws back 616, and no
+# pool step fits BOTH the 2-3 GiB floor and the harness compaction
+# invariant (slot >= 191,148). One native slot fits with room; the window
+# per conversation RISES 200,704 -> 262,144. The cost: a second concurrent
+# request queues behind the released slot (subagent concurrency 2 -> 1).
+CTX_SIZE=262144
+PARALLEL=1
 
 # Compute-buffer geometry. This endpoint passed neither flag until 2026-10-03,
 # so it ran on llama.cpp's own defaults (2048/512); the pair is now 4096/2048.
@@ -113,19 +114,23 @@ PARALLEL=2
 # graph VRAM on the 35B MoE (7,330 -> 8,574 MiB); this 9B hybrid has far
 # smaller compute tensors, so the delta should be well under that, but the
 # predecessor Q5_K_M shape recorded "ubatch 2048 failed the scratch boot" at
-# ~3,571 MiB free. This shape ships with LESS free (2,815 MiB measured
-# 2026-10-05). Compute buffers are allocated at graph capture during BOOT, so
-# a failure is loud and fast, not a surprise at the first long prompt. MEASURE
-# nvidia-smi free immediately after the deploy: under ~1.5 GiB, or a boot
-# failure, step down to 4096/1536 (or 4096/1024, the last known-good rung).
+# ~3,571 MiB free. The Q8_0 rung ships UBATCH 1024 (Frank, 2026-10-05: lower
+# ubatch FIRST — the measured 616 MiB step is the first VRAM the Q8 weights
+# claw back, before any pool move). Compute buffers are allocated at graph
+# capture during BOOT, so a failure is loud and fast, not a surprise at the
+# first long prompt. MEASURE nvidia-smi free immediately after the deploy:
+# under ~1.5 GiB, or a boot failure, step the POOL down next (12,288-token
+# increments, ~432 MiB each at the Ornith 36 KiB/token f16 KV rate).
 BATCH_SIZE=4096
-UBATCH_SIZE=2048
+UBATCH_SIZE=1024
 
-# The KV contract (2026-10-05, Q8_0 era): f16 K + q8_0 V. See the header
-# arithmetic — the Q8 weights consume the VRAM that funded f16 V on
-# 2026-10-03, and pool-stepping cannot restore the floor. --flash-attn stays
-# pinned on in the serve script: it is the fast path for both f16 and q8_0 V
-# and the only path that serves quantized V at all.
+# The KV contract (2026-10-05, Frank): f16 K + f16 V — NO KV quantization.
+# The q8_0-V arithmetic was written and OVERRULED. The Q8 weights eat the
+# VRAM that funded f16 V on 2026-10-03, so the remedy ladder is UBATCH FIRST
+# (2048 -> 1024, the measured 616 MiB rung), then POOL STEPS — and
+# guess-and-check on the live card is the accepted method: boot, read
+# nvidia-smi, step down until the floor holds. --flash-attn stays pinned on
+# in the serve script: it is the fast path for f16 KV.
 CACHE_TYPE_K=f16
-CACHE_TYPE_V=q8_0
+CACHE_TYPE_V=f16
 NGL=99

tests/frank_small_unit.bats

@@ -100,7 +100,7 @@ argv_value() {
     done
 }
 
-@test "the shipped env serves the fleet shape: two 200704-token contexts" {
+@test "the shipped env serves the fleet shape: one native 262144-token context" {
     serve_argv
     [ "$status" -eq 0 ]
     [ "$(argv_value --alias)" = "frank-small" ]
@@ -110,49 +110,54 @@ argv_value() {
     [ "$(argv_value --n-gpu-layers)" -gt 0 ]
     # llama.cpp divides --ctx-size across slots and a mamba hybrid cannot
     # share one cache, so the observable contract is the per-slot window.
-    [ "$(argv_value --parallel)" -eq 2 ]
+    # 2026-10-05 (Frank): NO KV quantization forces ONE slot at the NATIVE
+    # window — Ornith's 9 full-attn layers price f16 KV at 36 KiB/token, and
+    # two 200,704 slots cannot fit the 2-3 GiB floor on 24 GiB with Q8
+    # weights. The window per conversation ROSE 200,704 -> 262,144.
+    [ "$(argv_value --parallel)" -eq 1 ]
     ctx="$(argv_value --ctx-size)"
     par="$(argv_value --parallel)"
     [ $((ctx % par)) -eq 0 ]
-    [ $((ctx / par)) -eq 200704 ]
-    [ $((ctx / par)) -le 262144 ]
+    [ $((ctx / par)) -eq 262144 ]
 }
 
-@test "the KV contract is f16 K + q8_0 V, flash-attn on" {
+@test "the KV contract is unquantized both halves, flash-attn on" {
     serve_argv
     [ "$status" -eq 0 ]
     [ "$(argv_value --cache-type-k)" = "f16" ]
-    # Frank, 2026-10-05 (1862dc3): the Ornith payload has NINE full-attention
-    # layers (18 KiB/token/half) and +1,900 MiB of weights vs MiMo Q6_K, so
-    # f16/f16 lands ~379 MiB free — a dead boot; q8_0 V keeps 2 x 200,704
-    # AND the 2-3 GiB floor. The flag pair is what makes this observable — a
-    # build without the FA path silently falls back, so both must appear in
-    # the argv the supervisor runs.
-    [ "$(argv_value --cache-type-v)" = "q8_0" ]
+    # Frank, 2026-10-05: NO KV quantization on this endpoint. The Q8_0 rung
+    # funds itself by UBATCH first (2048 -> 1024), then pool steps — a q8_0-V
+    # proposal was written and overruled. The flag pair is what makes this
+    # observable — a build without the FA path silently falls back, so both
+    # must appear in the argv the supervisor runs.
+    [ "$(argv_value --cache-type-v)" = "f16" ]
     [ "$(argv_value --flash-attn)" = "on" ]
 }
 
 @test "the KV budget leaves 2-3 GiB of the 3090 free" {
-    # Rate, from the GGUF geometry: 8 full-attention layers x 4 kv heads x
-    # 256 dim x 2 B = 16 KiB/token for K and the same for V => 32 KiB/token
-    # (budgeted 33 with slack). The non-KV half is MEASURED on this card on
-    # 2026-10-02: with the projector loaded the engine reported 22,972 MiB
-    # used against 14,113 MiB of KV, so weights + vision tower + DeltaNet
-    # state + compute buffers + CUDA graphs = 8,859 MiB => 9 GiB. Two asserts,
-    # both about the output the operator sees: the pool must still fit with
-    # 2 GiB spare, and it must be large enough that 5 GiB spare is NOT
-    # reachable (an undersized pool is as wrong as an oversized one).
+    # Rate, from the Ornith GGUF header (read from the Hub file, 2026-10-05):
+    # NINE full-attention layers (block_count 33, full_attention_interval 4)
+    # x 4 kv heads x 256 dim x 2 B = 18 KiB/token for K and the same for V
+    # => 36 KiB/token (budgeted 37 with slack; MiMo measured 24.5 — one
+    # fewer layer AND one fewer KV head). The non-KV half: Q8_0 weights
+    # 9,334 MiB (9,786,060,384 B) + vision tower ~1.3 GiB (mmproj 879 MiB at
+    # the measured ~1.5x cost law) + DeltaNet state + compute buffers +
+    # CUDA graphs => 11 GiB budgeted. Two asserts, both about the output
+    # the operator sees: the pool must still fit with 2 GiB spare, and it
+    # must be large enough that 5 GiB spare is NOT reachable (an undersized
+    # pool is as wrong as an oversized one).
     # Arithmetic, not a boot log: change CTX_SIZE and this bites before the
-    # deploy does. The same pool with the OLD 24.5 KiB/token rate would leave
-    # ~6 GiB free — i.e. the unquantized V is what puts this shape in the band.
+    # deploy does. At Ornith's rate the two-200704-slot pool (14.1 GiB KV)
+    # FAILS the first assert — which is exactly why the shipped shape is
+    # ONE native slot.
     ctx="$(sed -nE 's/^CTX_SIZE=([0-9]+)$/\1/p' "$ENV_FILE")"
-    kv_gib=$((ctx * 33 / 1048576))
-    [ $((kv_gib + 9 + 2)) -le 24 ]
-    [ $((kv_gib + 9 + 5)) -gt 24 ]
+    kv_gib=$((ctx * 37 / 1048576))
+    [ $((kv_gib + 11 + 2)) -le 24 ]
+    [ $((kv_gib + 11 + 5)) -gt 24 ]
     # And the ceiling that sets the per-slot window: two NATIVE windows cannot
     # be paid for in full precision on this card.
     native=$((2 * 262144))
-    [ $((native * 33 / 1048576 + 9)) -gt 24 ]
+    [ $((native * 37 / 1048576 + 11)) -gt 24 ]
 }
 
 @test "the compute-buffer pair follows the env file and stays self-consistent" {