scripts/frank-archon.env

@@ -48,7 +48,7 @@ FRANK_ARCHON_TP=2
 FRANK_ARCHON_TP_WIRE=wht6
 # Decode slots. A slot costs KV and pinned RAM, never decode: the cache target is
 # exactly MAX_SEQS x MAX_MODEL_LEN and the checkpoint store asks (1 + what the
-# retention policy keeps) per slot. Ordered 2026-10-21: FOUR slots over a pool of at
+# retention policy keeps) per slot. Ordered 2026-10-20: FOUR slots over a pool of at
 # least 700,000 fp8 tokens, rounding UP if the resolver cannot land it exactly. The
 # target is therefore 4 x 200,000 = 800,000 tokens and HOST_POOL_MIB decides whether
 # the cache gets it (expert_vram_floor, core/place/planner.cpp:100-104: the card must

scripts/probe-frank-archon-kv.py

@@ -147,10 +147,21 @@ def main():
     deep = corpus(160_000)
     wide = corpus(1_100_000)
 
-    # 3.28 chars/token is the measured ratio of this corpus through the
+    # The corpus's chars/token ratio is MEASURED here, not stored. A stored 3.28
-    # froggeric template (646,000 chars -> 196,919 tokens at the 2026-10-11
+    # (646,000 chars -> 196,919 tokens, 2026-10-11) had drifted to 3.20 by
-    # deploy); 98.5 % of the window leaves the response room under the ceiling.
+    # 2026-10-20 (646,299 chars -> 201,876 tokens) as the repo's docs grew, and a
-    chars = int(0.985 * maxlen * 3.28)
+    # stale constant sends leg B past --max-model-len, where the engine answers a
+    # perfectly legitimate 400 ("the request (201900 tokens) exceeds the available
+    # context size (200000)") and the leg reads as a capacity failure. One small
+    # request prices the ratio; 97 % of the window leaves the response room.
+    sample = prompt(wide, 4000, "CAL", "cal-marker")
+    cal, _, _ = post({"model": MODEL,
+                      "messages": [{"role": "user", "content": sample}],
+                      "max_tokens": 1, "temperature": 0.0,
+                      "chat_template_kwargs": {"enable_thinking": False}})
+    ratio = len(sample) / cal["usage"]["prompt_tokens"]
+    print(f"[cal] corpus ratio {ratio:.2f} chars/token")
+    chars = int(0.97 * maxlen * ratio)
     # Each marker's FIRST hyphen-separated word is unique, because show()
     # scores the hit on that word — a shared prefix would score every slot HIT
     # on any answer that says the word.
@@ -176,7 +187,7 @@ def main():
                 print(f"[B-{s}] FAILED after {time.time() - t0:.1f}s: {e!r}")
     print(f"[B] total {time.time() - t0:.1f}s")
     # D — concurrency decode: seqs streams, 512 tokens each, ~25 % depth.
-    dchars = int(0.25 * maxlen * 3.28)
+    dchars = int(0.25 * maxlen * ratio)
     dbodies = [chat(deep, dchars, f"D{i + 1}-9137", f"flint-marker-{i}", 512)
                for i in range(seqs)]
     print(f"[D] {seqs} concurrent streams x 512 tokens, "

scripts/serve-frank-archon.sh

@@ -15,7 +15,7 @@
 # log states every budget before anything is allocated (weights vs KV
 # split, the expert host pool's coverage of the worst case, the PLE pin,
 # the P2P answer per pair) — read it after every resize.
-# Deployed shape (2026-10-21, radiance 1.8.1): FOUR decode slots x 200,000 tokens over a
+# Deployed shape (2026-10-20, radiance 1.8.1): FOUR decode slots x 200,000 tokens over a
 # >= 700,000-token fp8 KV pool, 48 GiB/rank pinned expert pool, MTP depth 2, prefix cache
 # in pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab and the KV
 # pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair and the budget

serverAppConfigs-docs/LLM/frank-archon.md

@@ -88,25 +88,30 @@ on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_p
 | Knob (env name, `FRANK_ARCHON_` dropped) | Live value |
 |---|---|
 | TP / wire | `TP=2`, `TP_WIRE=wht6` — libr4d one-shot all-reduce, not RCCL; `min-kb` stays the engine default |
-| Slots / window | `MAX_SEQS=2`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**, not a KV allocation |
+| Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
 | KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
 | VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
-| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=44928` **per rank** |
+| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=49152` **per rank** — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
 | PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
 | Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
 | Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
 | Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
 | Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
-| TTM gate | `TTM_MIN_MIB=150000`, `TTM_TARGET_MIB=204800` |
+| TTM gate | `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
 | Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
 
-Boot ledger for the live shape (2026-10-19 11:10, pin 1.8.1, `MAX_SEQS=2`, `HOST_POOL_MIB=44928`):
+Boot ledger for the live shape (2026-10-20, pin 1.8.1, `MAX_SEQS=4`, `HOST_POOL_MIB=49152`):
-the resolver prints `elastic 26.73 GiB` → **experts 23.73 GiB (89 %) / kv cache 2.76 GiB (10 %)**,
+the resolver prints `elastic 26.73 GiB` → **experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**,
-`vram_weights 27.31 GiB`, `host_pool 44.47 GiB`. `kv_attn` (13 layers, 4-token blocks) pages
+`prefill staging 373.65 MiB` out of the cache, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`.
-**26.00 KiB** → 91,301 blocks = **365,204 tokens = 1.8 full-length sessions = 91 % of the
+The referee line: `kv: the cache can grow to 5.64 GiB a card — 747,492 tokens of context, 3.7
-2 × 200,000 worst case** (the resolver never funds 100 %, see Laws); `mover` lends 2,146 slots
+sessions at 200,000 — with every block it lends the experts taken back; 440.04 MiB of that is
-(2.54 GiB) out of the cache to the expert slab; sampler **9 rows**, template 28,234 B, vision live
+per-sequence state, and up to 5.20 GiB is lent to the experts while no request holds it`.
-(≤ 4,194,304 px, 16,384 patches/pass).
+`--checkpoint-slots 40` is auto (`max_num_seqs × 10` slots of 111.98 MiB = 4,479 MiB of pinned
+host), `/health` reports `queue_cap 32` (`max(8, seqs × 8)`), sampler **12 rows**
+(= `seqs × (1 + speculative depth)`), 4 hardware queues a card, template 28,234 B, vision live
+(≤ 4,194,304 px, 16,384 patches/pass), P2P confirmed (`the 2 cards reach each other's memory
+directly`). Experts living in the pinned host pool: **8.99 GiB a rank** (was 5.99) — the price of
+the bigger cache, paid over PCIe when those experts are hit.
 
 The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
 anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
@@ -145,6 +150,7 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
 
 | Window | Payload | Runtime | Shape | Replaced because | Rollback |
 |---|---|---|---|---|---|
+| 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
 | 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
 | 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
 | 2026-10-05 → 10-08 | `qwen4exp-fp8-native.rad`, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation | radiance 1.0.7 line, same digest | — | native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, **1.93 sessions** — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot) | rebuild-only: `scripts/download-frank-archon.sh` (official `Qwen/Qwen3.8-Flash-Next-FP8`, 131 shards, 172.8 GB, ungated) + `scripts/build-frank-archon-rad.sh` (no-recipe convert under the descriptor law); raw **and** `.rad` were `rm`'d 2026-10-15 (Frank's call, to free the volume for the iq4r download) |
@@ -211,27 +217,44 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   never `--kv-cache-size-mib`; with the disk tier engaged the engine also forced ~9.6 GiB of weight
   buffers + arena onto the weights side (~29.6 GiB/card effective). **Neither flag exists at 1.8.1:**
   the same ceiling is enforced by the resolver, and passing the pair is a boot refusal.
-- **At 1.8.1 the split belongs to the resolver, and it funds ~90 % of the worst case, never 100 %.**
+- **At 1.8.1 the split belongs to the resolver, and the source fixes its order.**
-  The engine takes `--gpu-headroom-mib`, the cache bound (`--max-model-len` × `--max-num-seqs`) and
+  `core/mem/vram_budget.cpp` claims the card bottom-up (`capacity → already held → drift → arena →
-  `--host-pool-mib`, prints `claimable → elastic → experts / kv cache` before allocating anything, and
+  staging → queues → prefetch → headroom = claimable`), takes the **static weights off the top as a
-  sizes the expert plane from the model's own weight footprint plus slack — not from a number typed by
+  floor, not a share**, and divides the `elastic` remainder in three lines (146-156):
-  hand. Three boots of evidence:
 
-  | shape | demand | cache given | backing |
+  ```cpp
+  exp = std::min(b.expert_floor, b.elastic);
+  kv  = std::min(kv_cap, std::max(b.elastic - exp, kv_one));
+  exp = std::min(std::max(b.expert_ceiling, b.expert_floor), b.elastic - kv);
+  ```
+
+  `kv_cap` is `KVManager::ceiling()` = `state + Σ_paged max_seqs × ceil(max_ctx / block) × bytes`
+  (`core/mem/kv.cpp:266`), so `--max-num-seqs` and `--max-model-len` set the cache's **demand**, never
+  its funding. The funding is `elastic − expert_floor`, and the floor is
+  `min(plane + spare, plane + spare + 2·unit_max − host_room)` (`core/place/planner.cpp:100-104`, with
+  `host_room = host_pool − file_stage_reserve` at `core/engine_bringup.cpp:1418-1422`). **That is the
+  cache's lever: every MiB added to `--host-pool-mib` drops the expert floor one-for-one and the cache
+  takes it** — minus `stage`, the prefill stager's ring (two layers of the experts left off the card),
+  which is charged back out of the cache whenever the cache lends (`vram_budget.cpp:193-203`). Boots of
+  evidence, all on 1.8.1:
+
+  | shape | demand | cache given | engine's `backs %` |
   |---|---|---|---|
-  | 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % (2.8 sessions) |
+  | 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % |
   | 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % |
-  | 2 × 200,000, host pool 44,928 (live) | 400,000 tok | 2.76 GiB = 365,204 tok | 91 % (1.8 sessions) |
+  | 2 × 200,000, host pool 44,928 | 400,000 tok | 2.76 GiB | 91 % |
+  | 4 × 200,000, host pool 49,152 (live) | 800,000 tok | **5.64 GiB = 747,492 tok** | 93 % |
 
-  So `--max-num-seqs` is a **demand signal** for the resolver and an admission cap for the scheduler —
+  The 45,568 row is the ceiling case, not a mystery: at 3 slots the cache already held its whole
-  never a KV allocation — and no slot count fully funds the worst case. Growing host RAM by less than a
+  `ceiling`, so extra host room could not grow it and shows up as `unclaimed`. **The referee is the
-  slab step (640 MiB) moved nothing: the expert plane's claim decides what is left for KV, so cache is
+  ledger line `kv: the pool backs N % … the cache can grow to … N tokens`, never arithmetic.** To buy
-  grown through slots × window or the dtype, and experts are grown by not leaving elastic pool idle.
+  window: raise `--host-pool-mib` (cost: pinned host RAM, and experts evicted to host cross PCIe —
-  `--host-pool-mib` is still load-bearing the other way — it is where the expert rows the cards cannot
+  `experts-from-host` was 5.99 GiB a rank at 44,928 and is 8.99 at 49,152) or cut the per-token cost
-  hold get pinned, and under-sizing it has always been a boot failure (36,864 refused a boot in the fp8
+  (`fp8` KV). Raising slots raises demand **and** costs 10 × 111.98 MiB of pinned checkpoint-store RAM
-  era). The two-slot boot's own behavioural check: three concurrent conversations all answer `200` —
+  per slot. Under-sizing the pool the other way has always been a boot failure (36,864 refused a boot in
-  the third **queues** (`queue_cap: 16`), it is not refused; the 400 class is a single request over
+  the fp8 era: the host cannot hold what the cards will not). Behavioural check on the live shape:
-  `--max-model-len`, or three concurrent full-height conversations against a 1.8-session cache.
+  `/health` reports `queue_cap 32`; a request beyond the slot count queues rather than being refused,
+  and the 400 class is a single request over `--max-model-len`.
 - **Idle KV VRAM is the expert cache's spillover — the price is paid twice.** `mover: 2275
   slots can be lent out of the KV cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <=
   256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 in the pair era)
@@ -366,17 +389,18 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   `--checkpoint-interval` does **not** buy the deep case: the coarsest ladder scale is the largest
   `interval × 2^k` under the depth, so the deepest gap stays ≈ half the prompt (129,024 tokens at
   interval 2048 against 130,816 at 256) while every cut multiplies the snapshot writes.
-- **Slots cost KV, rows, and window — never speed.** `--max-num-seqs` carries no radiance constraint: it
+- **Slots cost KV, host RAM, rows, and pin — never speed.** `--max-num-seqs` carries no radiance
-  adds ≈110 MiB of recurrent state per slot (the live `kv_full_backing` arithmetic prices them exactly),
+  constraint: it adds ≈110 MiB of recurrent state per slot, 10 × 111.98 MiB of pinned checkpoint-store
-  declares the fused TP all-reduce over `seqs × (1 + spec)` rows, and `wht6` is bit-exact up to 25 rows
+  RAM per slot (`--checkpoint-slots auto`), and declares the fused TP all-reduce over
-  (single-sequence decode and any step of ≤ 25 rows is exact; only busier steps are lossy) — 2 × 3 = 6
+  `seqs × (1 + spec)` rows — `wht6` is bit-exact up to 25 rows, and the live shape is 4 × 3 = **12**, so
-  today. Measured 2026-10-06: seqs 2 and seqs 3 are statistically tied (30.9-33.1 t/s, inside the ±3 %
+  single-sequence decode stays exact and so does any step up to that many rows. Raising it also raises
-  floor). Measured 2026-10-19 at 1.8.1: the third slot's real price is **KV, not rows** — dropping it
+  the cache's *demand* (`ceiling = slots × window`), which the resolver funds from `elastic −
-  moved the resolver's cache from 4.20 GiB to 2.76 GiB and the backing from 2.8 to 1.8 sessions, while
+  expert_floor` — so a slot count without matching host room is paid for by expert residency. Measured
-  the ≈4 GiB of host RAM it freed came back unbudgeted, landing in elastic pool that turned into a bigger
+  2026-10-06: seqs 2 and 3 statistically tied (30.9-33.1 t/s, inside the ±3 % floor). Measured
-  mover lend instead of cache (resolver law above). The old "seqs 2 cannot boot" law belonged to the
+  2026-10-19 → 10-20 at 1.8.1: 2 → 4 slots cost **+4,224 MiB of pinned host pool** and **−3.01 GiB of
-  retired tcclaviger fork's `torch.compile` shape specialization, and radiance booted seqs 2 on its first
+  expert residency a card** and bought 2.76 → 5.64 GiB of cache (365,204 → 747,492 tokens). The old
-  fp8-native start.
+  "seqs 2 cannot boot" law belonged to the retired tcclaviger fork's `torch.compile` shape
+  specialization, and radiance booted seqs 2 on its first fp8-native start.
 - **A dequantize-to-compute path bills host RAM twice.** On the EMULATION backend an offload request
   of **118 GiB** landed as **~238 GiB** of real host RAM — the dequantized copy rides on top of the
   pinned weights, and the doubling is invisible in the weight count. Read the engine log for
@@ -570,13 +594,13 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   — at 64 tokens you get empty `content` and the colour only inside `reasoning_content`.
 - **Capacity behaviour:** a prompt over `--max-model-len` is refused, never truncated; a request
   longer than the KV pool's token count is accepted then refused as too large to fit; a request
-  beyond `--max-num-seqs` is held by the HTTP port (`queue_cap`, default 8 × slots = 16 here) and
+  beyond `--max-num-seqs` is held by the HTTP port (`queue_cap` = `max(8, 8 × slots)` — **32** here)
-  answered 503 only past that, never paged or squeezed. Because the cache funds **1.8** full-height
+  and answered 503 only past that, never paged or squeezed. The cache funds **3.7** full-height
-  sessions, admitting a second near-full-height conversation can **preempt a running sequence from
+  sessions, so four conversations can be resident but a fourth near-full-height one can still
-  the back** to free its blocks (`core/sched/scheduler.cpp:1291-1322`); the victim is recomputed, so
+  **preempt a running sequence from the back** to free its blocks
-  its session halts and then re-prefills. Meter: `vllm:num_preemptions_total` (264 in the current
+  (`core/sched/scheduler.cpp:1291-1322`); the victim is recomputed, so its session halts and then
-  boot, all in prefill-coincident bursts — 108 inside 105 s); `/stats preemptions` is the cumulative
+  re-prefills. Meter: `vllm:num_preemptions_total` (cumulative for the boot, `/stats preemptions`);
-  count and `radiance:num_requests_preempted` only the instant depth.
+  `radiance:num_requests_preempted` is only the instant depth.
 - **Observability:** `/metrics` carries vLLM-compatible names plus `radiance:*`, scraped by the
   Dankclaw Prometheus job at `192.168.0.235:7770` (15 s interval) over plain LAN routing (the
   netavark published-port quirk is same-host only). **Latency lives under `vllm:`, cadence under