scripts/run-qwen38-bf16.sh

@@ -48,36 +48,36 @@
 # NO --vram-kv-mib: it was REMOVED at 1.3.0 and naming it is a boot refusal, not
 # a no-op — verified on the pinned image 2026-10-27: "E --vram-kv-mib was
 # removed: the card is split between the weights and the cache from what the
-# model needs." The budget resolver derives that split from --host-pool-mib and
+# model needs." The resolver derives the split from --host-pool-mib,
-# --gpu-headroom-mib, and this rig inherits BOTH from frank-archon.env, the file
+# --gpu-headroom-mib and the activation arena; the rig inherits the first two from
-# that owns them: the cache's funding is `elastic - expert_floor` and the expert
+# frank-archon.env, the file that owns them. On THIS payload the host pool is not
-# floor falls one MiB for every MiB of host pool (frank-archon.md, Laws), so the
+# the cache's supplier: the boot lands in vram_budget.cpp's `no lender` branch,
-# pool buys KV as well as residency — a private pair is not a free knob. What
+# where the experts keep their plan-fixed 25.29 GiB and the cache gets the residue.
-# buying one cost, measured on the first kldref boot the same day with the w4
+# Measured twice — 12,288 and 49,152 MiB of host pool both printed the same
-# era's 12288 / 768: "I kv: the pool backs 6% of the worst case (4 sequences all
+# "kv cache 271.25 MiB ... 7,436"; only --gpu-headroom-mib moved anything
-# at 32768 tokens), 0.2 full-length sessions; a larger --host-pool-mib gives the
+# (768 -> 96 grew vram_weights 29.41 -> 30.06 GiB). The supplier the source names
-# cache more" — a 270 MiB, 7,436-position cache, after which the recorder refused
+# is the arena: vram_budget.cpp:117, "lower --max-num-batched-tokens (the arena
-# every one of the 197 corpus documents ("E kld: document 0 is 10554 tokens and
+# scales with it)" — which is what the step size below now does.
-# the KV pool holds 7436, so it can never be scored") and exited 1. Production's
-# 49,152 per rank is affordable exactly when the rig runs: the rig stops
-# frank-archon first, and that stop frees the same 96 GiB the service was pinning.
 #
-# The 32,768-token window it pairs with is not up for grabs: the corpus's longest
+# The pool, not the window, decides what the recorder accepts: it refuses any
-# prompt is 10,554 tokens by the engine's own count (the header's chars/4.97
+# document longer than the pool, so the 32,768-token window was decorative while the
-# estimate called that record ~7,700; the true ratio for this file is chars/3.6),
+# cache held 7,436 — "E kld: document 196 is 8507 tokens and the KV pool holds
-# so the window clears it by 3x and the cache now funds it.
+# 7436, so it can never be scored" on all 197 records, exit 1. The longest record
+# is 10,554 tokens by the engine's own count (chars/3.6 for this file, not the
+# chars/4.97 this repo measured on the K3 corpus).
 #
 # The step size is ONE value for the recording and every scoring pass:
-# FRANK_QWEN38_MNBT, default 2,048. The gate's whole claim is that the reference,
+# FRANK_QWEN38_MNBT, default 1,024. The gate's whole claim is that the reference,
 # the served iq4r and every arm rode one shape, so the ceiling is set by the
 # TIGHTEST payload the gate boots rather than by the bf16 reference: the quantised
 # expert FILE tiers (the served iq4r, the arms) stream only at MNBT<=2048 and
-# refuse the widened form above it, and 2,048 is what production holds anyway.
+# refuse the widened form above it. 2,048 was still the wrong default on this
-# Recording the reference at 32,768 would therefore measure a shape the payloads
+# payload, because the arena the engine charges for a step comes straight out of
-# cannot reproduce; at 2,048 the 50,432 positions cost more steps, which is still
+# the cache — 1.08 GiB at a 2,048-token step, 504.39 MiB at a 1,024-token one, and
-# one drained window. Calibration keeps the wide 65,536 — grams are not logits,
+# the experts keep their share either way. At 1,024 the cache is 866 MiB: 24,300
-# nothing downstream is compared against a step size, and that is where a big
+# tokens, which scores every one of the 197 documents (longest 11,970).
-# step pays (a disk-tier step costs the same at 1 token as at 64k).
+# Calibration keeps the wide 65,536 — grams are not logits, nothing downstream is
+# compared against a step size, and that is where a big step pays.
 #
 # usage: scripts/run-qwen38-bf16.sh kldref|calib|score RAD OUT [--dry-run] [--image REF]
 #   (RAD and OUT are score's two positionals: RAD is a container under the models
@@ -94,8 +94,9 @@
 #        bf16 container, kldref only), FRANK_QWEN38_CAMPAIGN (holds kld-ref/ and
 #        calib/), FRANK_QWEN38_KLD_CORPUS, FRANK_QWEN38_TPL (the template copy
 #        the container reads through /models), FRANK_QWEN38_LOG_DIR, and
-#        FRANK_QWEN38_MNBT — the step size shared by kldref and score (2,048 is
+#        FRANK_QWEN38_MNBT — the step size shared by kldref and score (default
-#        the file tier's ceiling and production's own value, four slots and all),
+#        1,024: the file tier's ceiling is 2,048 and production runs there, but on
+#        this payload the arena charged for a wider step is taken from the cache),
 #        and FRANK_QWEN38_MAX_MODEL_LEN — the window every mode boots at (32,768;
 #        see the corpus census above, and raise it together with the corpus).
 # kldref refuses (exit 2) when the longest record of the corpus it is about to
@@ -164,9 +165,11 @@ GPU_HEADROOM_MIB="${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB
 
 # ONE step size for both logits modes. The file tier streams only at <=2,048 and
 # the gate scores file-tier payloads (the served iq4r, the arms) against the
-# reference, so the reference is recorded at the ceiling the payloads can meet —
+# reference, so the reference is recorded at a step the payloads can meet — never
-# never two numbers that a KL between them would have to forgive.
+# two numbers that a KL between them would have to forgive. 1,024 rather than the
-MNBT="${FRANK_QWEN38_MNBT:-2048}"
+# 2,048 ceiling: on this payload the arena charged for the step is the cache's only
+# remaining supplier, and 1,024 is the rung that funds every corpus document.
+MNBT="${FRANK_QWEN38_MNBT:-1024}"
 if ! [[ "$MNBT" =~ ^[0-9]+$ ]]; then
     echo "REFUSED: FRANK_QWEN38_MNBT must be a token count (got: $MNBT)" >&2
     exit 2
@@ -223,9 +226,10 @@ MOUNTS=(-v "$MODELS:/models:ro")
 TAPENV=()
 case "$MODE" in
   kldref)
-    # Recording is prefill-only over four slots and the shared step, which is
+    # Recording is prefill-only over four slots at the campaign's single step size
-    # production's own operating point (frank-archon.env: FRANK_ARCHON_MAX_SEQS=4
+    # — 1,024, not production's 2,048 (frank-archon.env: FRANK_ARCHON_MAX_SEQS=4,
-    # with FRANK_ARCHON_MAX_BATCHED_TOKENS=2048). Corpus read-only, record dir rw.
+    # FRANK_ARCHON_MAX_BATCHED_TOKENS=2048): the wider step costs the cache the
+    # room the corpus needs. Corpus read-only, record dir rw.
     ARGS+=(--model "$MODEL_IN"
            --kld-record /data/kld-ref --kld-corpus /data/kld.jsonl
            --max-num-seqs 4 --max-num-batched-tokens "$MNBT")