Merge pull request 'fix(frank-archon): fold the pinned host pool back to 49,152; keep the 2,048 step' (#82) from feat/qwen38-w5nl64 into main

scripts/frank-archon.env

@@ -74,12 +74,14 @@ FRANK_ARCHON_PLACEMENT=expert_tiered
 # residency -- ~3.2 GiB a card of experts now lives in host RAM and is read over the
 # x8 link when the router picks it. The ledger's coverage line is the referee, not
 # this file: if vram_kv comes back under 700,000 tokens, raise this 1024 and redeploy.
-# 2026-10-29: 49,152 -> 66,560 (tp x 65 GiB, inside the 246 GiB the box has). The
+# 2026-10-29 A/B, measured on both payloads and folded back the same day: at 66,560 the
-# campaign rig's bf16 container at the shared step reported "kv cache 269.99 MiB a
+# endpoint printed the identical `elastic 26.73 GiB / experts 20.72 / kv cache 5.64 /
-# card: 7436 tokens", so 682 of the 1,294 calibration documents were refused at that
+# vram_weights 24.30` it had printed at 49,152, opening the bigger pool with `0 B used`;
-# pool; the pool, not the step, is what widens it (scripts/run-qwen38-bf16.sh). The
+# the campaign rig (bf16, same step) moved `vram_kv 269.99 -> 271.25 MiB`, a rounding
-# new cache line belongs to the next boot ledger, not to this comment.
+# step — its window is fixed by its own n-gram table (95.37 GiB) and expert plane, not
-FRANK_ARCHON_HOST_POOL_MIB=66560
+# by this pool, so the refused calibration documents were fixed by cutting them to the
+# window (scripts/rechunk-kld-corpus.py), not by widening the pool.
+FRANK_ARCHON_HOST_POOL_MIB=49152
 # VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
 # floor is built from it.
 FRANK_ARCHON_GPU_HEADROOM_MIB=96
@@ -142,10 +144,10 @@ FRANK_ARCHON_NGRAM_PLACEMENT=ram
 # itself before refusing by name, so recovery is a boot, not a visit.
 # Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT + the
 # auto checkpoint store (MAX_SEQS x 10 slots of 111.98 MiB), per the bats ledger test;
-# at the 66,560 pool with 4 slots that computes to 196,664 MiB (161,848 at the old
+# at the 49,152 pool with 4 slots that computes to 161,848 MiB (196,664 at the
-# 49,152 pool).
+# 66,560 pool the 2026-10-29 A/B rejected).
-FRANK_ARCHON_TTM_MIN_MIB=197000
+FRANK_ARCHON_TTM_MIN_MIB=162000
-FRANK_ARCHON_TTM_TARGET_MIB=224000
+FRANK_ARCHON_TTM_TARGET_MIB=204800
 # Escape hatch for experiments. Unset -> no extra argv; the serve script expands
 # this unquoted on purpose, so keep multi-flag values as one string.
 FRANK_ARCHON_EXTRA_ARGS=

scripts/run-qwen38-bf16.sh

@@ -170,13 +170,14 @@ fi
 
 # ONE step size for every mode, quoted from the file that owns it — the audit
 # law for a radiance rig is to inherit the proven number, not to carry a
-# campaign-private value: production's FRANK_ARCHON_MAX_BATCHED_TOKENS is 2,048,
+# campaign-private value: production's FRANK_ARCHON_MAX_BATCHED_TOKENS. The
-# the widest step the quantised expert FILE tiers can stream (above it the
+# 1,024 default this script carried was PR #80's answer to a starved cache;
-# engine charges a 10.65 GiB arena to the card and the load refuses), and the
+# production moved to its widest legal step (2,048) on 2026-10-29 and the rig
-# arena it charges comes out of the cache — so the corpora are cut to the window
+# follows. The pool raise that shipped with it (66,560 a rank) bought this
-# the boot ledger reports, never the other way round. The 1,024 default this
+# container 1.26 MiB of cache (`vram_kv 269.99 -> 271.25 MiB`, window still
-# script carried was PR #80's answer to a starved cache; the pool raise
+# 7,436 tokens — the 95.37 GiB n-gram table's claim decides the rig's cache,
-# (frank-archon.env: 66,560 MiB a rank) is the fix, not a narrower step.
+# not the pool) and was folded back the same day. The corpora stay cut to the
+# window the boot ledger reports, never the other way round.
 MNBT="${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
 if ! [[ "$MNBT" =~ ^[0-9]+$ ]]; then
     echo "REFUSED: FRANK_ARCHON_MAX_BATCHED_TOKENS must be a token count (got: $MNBT)" >&2

serverAppConfigs-docs/LLM/frank-archon.md

@@ -91,31 +91,36 @@ on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_p
 | Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
 | KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
 | VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
-| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=66560` **per rank** (raised 2026-10-29 from 49152 — see *Eras*) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
+| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=49152` **per rank** (raised to 66560 on 2026-10-29, folded back the same day — the A/B under *Laws* shows the raise bought no cache) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
 | PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
 | Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
 | Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
 | Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
 | Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
-| TTM gate | `TTM_MIN_MIB=197000`, `TTM_TARGET_MIB=224000` |
+| TTM gate | `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
 | Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
 
-Boot ledger for the live shape (2026-10-29, pin 1.8.1, `MAX_SEQS=4`,
+Boot ledger for the live payload. Last boot: 2026-10-29 at `MAX_SEQS=4`,
-`HOST_POOL_MIB=66560`, `MAX_BATCHED_TOKENS=2048`): the resolver prints `elastic 26.73 GiB` →
+`HOST_POOL_MIB=66560`, `MAX_BATCHED_TOKENS=2048`; the live pool is `49152` again, and the
-**experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**, `activation arena 990.36 MiB` (the
+two boots are **identical above `host_pool`**, so one ledger serves both shapes:
-price of the 2,048-token step: `256 tokens use 155.60 MiB, 1024 tokens use 472.86 MiB of
+`elastic 26.73 GiB` → **experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**,
-896.20 MiB`), `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 65.59 GiB`,
+`activation arena 990.36 MiB` (the price of the 2,048-token step: `256 tokens use
-`unclaimed 96.89 MiB`; referee line `kv: the pool backs 93% of the worst case (4 sequences all
+155.60 MiB, 1024 tokens use 472.86 MiB of 896.20 MiB`), `prefill staging 373.65 MiB`,
-at 200000 tokens), 3.7 full-length sessions`; `device: the 2 cards reach each other's memory
+`vram_weights 24.30 GiB`, `host_pool 65.59 GiB` (opened with `0 B used`; `48.59 GiB` at
-directly (peer to peer)`; sampler **12 rows** (= `seqs × (1 + speculative depth)`), 4 hardware
+the 49,152 boot), `unclaimed 96.89 MiB`; referee line `kv: the pool backs 93% of the worst
-queues a card (133.45 MiB), `already held 256.67 MiB`, mover `17519 slab slots, 7624 pinned
+case (4 sequences all at 200000 tokens), 3.7 full-length sessions`; `device: the 2 cards
-pool slots, 3.58 GiB static vram`, `load: rank 0, mover took 24.30 GiB`.
+reach each other's memory directly (peer to peer)`; sampler **12 rows**
+(= `seqs × (1 + speculative depth)`), 4 hardware queues a card (133.45 MiB), `already held
+256.67 MiB`, mover `17519 slab slots, 7624 pinned pool slots, 3.58 GiB static vram`,
+`load: rank 0, mover took 24.30 GiB`.
 
-The 2026-10-20 boot at `HOST_POOL_MIB=49152` printed the **same** `elastic 26.73 GiB /
+The A/B that ended the raise: the 2026-10-20 boot at `HOST_POOL_MIB=49152` printed the
-experts 20.72 GiB / kv cache 5.64 GiB / vram_weights 24.30 GiB`. The pool raise bought the
+**same** `elastic 26.73 / experts 20.72 / kv cache 5.64 / vram_weights 24.30` (Laws: the
-endpoint no cache: the engine opened that pool with `0 B used` and the VRAM split never moved
+lever saturates at the expert floor), and the campaign rig — the other payload that
-(Laws: the lever saturates at the expert floor). The era's own ledger line is superseded by
+inherits this env — moved `vram_kv 269.99 → 271.25 MiB`, a rounding step, because its
-this one; the number the old era quoted is gone with it.
+window is set by its own n-gram table (95.37 GiB) and expert plane rather than by the
+pool. The refused calibration documents are fixed by cutting them to the window, not by
+widening the pool.
 
 The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
 anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
@@ -154,7 +159,7 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
 
 | Window | Payload | Runtime | Shape | Replaced because | Rollback |
 |---|---|---|---|---|---|
-| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | the `qwen38-w5nl64` campaign rig booted this pool at the shared 1,024 step and reported `kv cache 269.99 MiB a card: 7436 tokens`, so 682 of its 1,294 calibration documents were refused; Frank ordered the pool **and** the step raised to production's widest legal step (2,048) as one shape | revert the `feat/frank-archon-pool-step` commit and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `HOST_POOL_MIB=49152`, `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
+| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | Frank ordered the campaign's step (2,048) onto production; the pool half of that order (66,560) booted, came back byte-identical to this row's cache, and was folded back the same day — the calibration documents it was meant to save were cut to the window instead (`scripts/rechunk-kld-corpus.py`) | the live shape is this row's pool with `MAX_BATCHED_TOKENS=2048`: the pair is the rollback unit, not the merge commit (`scripts/frank-archon.env`) |
 | 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
 | 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
 | 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
@@ -248,8 +253,8 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   | 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % |
   | 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % |
   | 2 × 200,000, host pool 44,928 | 400,000 tok | 2.76 GiB | 91 % |
-  | 4 × 200,000, host pool 49,152 | 800,000 tok | 5.64 GiB = 747,492 tok | 93 % |
+  | 4 × 200,000, host pool 49,152 (live) | 800,000 tok | 5.64 GiB = 747,492 tok | 93 % |
-  | 4 × 200,000, host pool 66,560 (live) | 800,000 tok | **5.64 GiB — byte-identical to 49,152** | 93 % |
+  | 4 × 200,000, host pool 66,560 (2026-10-29 A/B, folded back) | 800,000 tok | **5.64 GiB — byte-identical to 49,152** | 93 % |
 
   The 45,568 row is the ceiling case, not a mystery: at 3 slots the cache already held its whole
   `ceiling`, so extra host room could not grow it and shows up as `unclaimed`. **The referee is the

tests/frank_archon_unit.bats

@@ -158,7 +158,7 @@ nosudo_bin() {
     # so the pool's token count is a boot-ledger line — verified live by the
     # capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
     grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
-    grep -q '^FRANK_ARCHON_HOST_POOL_MIB=66560$' "$ENV_FILE"
+    grep -q '^FRANK_ARCHON_HOST_POOL_MIB=49152$' "$ENV_FILE"
     grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
     grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
     # Prefix cache: pinned host RAM only (the disk tier was removed), and the

tests/qwen38_reference.bats

@@ -29,7 +29,7 @@ RUNNER="$REPO_ROOT/scripts/run-qwen38-bf16.sh"
 # The reference shape, spelled exactly as run-qwen38-bf16.sh must pass it: this
 # is the contract the recorded reference is measured at, so a silent edit here
 # is a silent edit to every KLD number in the campaign.
-SHAPE="--tp 2 --placement expert_tiered --weights-disk-tier --host-pool-mib 66560 --gpu-headroom-mib 96 --deterministic --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 32768 --ngram-placement ram --override-chat-template /models/tpl.jinja"
+SHAPE="--tp 2 --placement expert_tiered --weights-disk-tier --host-pool-mib 49152 --gpu-headroom-mib 96 --deterministic --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 32768 --ngram-placement ram --override-chat-template /models/tpl.jinja"
 
 # oneline — both dry runs print one shell-quoted argument a line; joining them
 # lets an option and its value be asserted as the pair they are.