serverAppConfigs/serverAppConfigs-docs/LLM/frank-archon.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	frank-archon on Dankserver — radiance (StillDeadcode) 1.8.1, publisher Qwen3.8-Flash-Next fp8-iq4r-moe .rad, 2× R9700 TP=2 (ACTIVE)	The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught.	llm frank-archon dankserver radiance gfx1201 r9700 tp2 expert-offload endpoint	2026-10-19

frank-archon — radiance (StillDeadcode), 2× R9700 TP=2, Dankserver :7770

The radiance engine (C++/HIP, libr4d kernels built for gfx1201, no Python in the serving path) driving podman run straight from the digest-pinned image under a root systemd unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI PRO R9700s. Capacity contract 2 conversations × 200,000 tokens at fp8 KV — at 1.8.1 that pair is the demand signal the budget resolver sizes the cache against, not a KV allocation (Laws). A payload swap is a MODEL_FILE line plus a re-download — see Eras.

Current shape

Authoritative source: scripts/frank-archon.env + scripts/serve-frank-archon.sh (the env header carries the measured arithmetic beside each number).

Payload — /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad, 121,969,901,568 B = 116,322 MiB, sha256 0af5e962… (the Hub LFS oid), from StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe: the engine author's own rad-convert of the official (censored) Qwen/Qwen3.8-Flash-Next bf16 checkpoint. Encodings (recipe qwen4exp-w4nl64-i8-hc8m): routed experts as 4-bit codes in a 16-level non-uniform codebook (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scale per 64 weights); 10 whole bf16-protected experts per layer; int8 W8A8 trunk + lm_head; fp8 HC mixtures; PLE n-gram fp8 at the checkpoint's fixed scale (47.7 GiB) MTP head and the bf16 27-layer vision tower in the container. Card quality vs bf16: mean KL 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %. The abliterated family lives on frank-dragoon. Fetched by scripts/download-frank-archon-iq4r.sh --fetch (hfget resume + declared size + sha256) as the transient unit iq4r-fetch (log /home/dank/logs/iq4r-fetch.log, …rc sentinel); the staging dir /home/dank/models/StillDeadcode/… keeps card, LICENSE and recipe as provenance, and only .rad files are mounted.

Runtime — stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed = pin 1.8.1 (adopted 2026-10-18). The version oracle is the boot's own STARTUP REPORT -- radiance 1.8.1 line, because podman inspect reports only the digest. The image lives in dank's rootless storage and the host needs nothing but the amdgpu driver. Pin the INDEX manifest digest, never the image ID (69d05724… is the config digest and sends podman run to the network). The serve script's gate is that digest ref itself — podman image exists name@sha256:… in one command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike or a newer latest cannot satisfy it. 1.2.0 is a rejected serve pin (Laws). Two CLI breaks ride this pin, and both are boot refusals, never silently-ignored flags: the --vram-weights-mib/--vram-kv-mib pair died at 1.3.0 (the resolver derives the split) and --prefix-cache-dir / --prefix-cache-disk-mib died with the disk tier. A rollback to 1.0.13 (sha256:4c3f4845…) is therefore a trio rewrite, not a digest swap — the Eras row carries it, and tests/frank_archon_unit.bats asserts the retired knobs are ABSENT from env and argv so a stale line cannot reach a boot.

Trio — scripts/frank-archon.env (every number, with its arithmetic in the header) scripts/serve-frank-archon.sh, scripts/frank-archon.service. Gates run in order: image digest present → .rad present and ≥ RAD_MIN_MIB (a truncated convert would otherwise buy a guaranteed long-timeout boot instead of this fast failure) → template present → /dev/kfd + /dev/dri → ttm_ensure_cap (the serve script re-verifies the cap after the unit's hook) → podman rm -f over FRANK_ARCHON_LEGACY_CONTAINERS (frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe) plus its own name, because systemd SIGKILLs conmon on a crash and leaves the name registered, and every one of those identities holds this GPU. Container contract (publisher-vetted, docs/DOCKER.md): --network host, --init (so the engine releases the cards on SIGTERM), --security-opt seccomp=unconfined (the default profile refuses io_uring — the PLE table's reads — and the NUMA calls the ROCm runtime makes for host memory), --ulimit memlock=-1:-1, /models ro + the rad-tune cache at /opt/radiance/share/radiance/tune + the template as a single-file mount (there is no /data any more: it left with the prefix disk tier, and FRANK_ARCHON_STATE_DIR is now only the host-side directory that holds tune/), then exec podman wait: the script's lifetime IS the container's, so a crash is a nonzero ExecStart and the restart policy is real. Unit: After=network-online.target, WantedBy=multi-user.target, User=dank LimitMEMLOCK=infinity, StartLimitIntervalSec=0, OOMScoreAdjust=-500 (the last process the OOM killer should pick), Restart=always not on-failure — a clean API-server exit after a worker fault reads as success to systemd (the 2026-10-01 vLLM lesson) — and TimeoutStopSec=180 > the ExecStop=podman stop -t 120 grace, because systemd's 90 s default cut the stop short and recorded Failed with result timeout (measured 2026-10-01 13:27:17). No drain contract is claimed: the OpenAI front end holds no queue, so an in-flight decode is lost on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute start_period).

Knob (env name, FRANK_ARCHON_ dropped)	Live value
TP / wire	TP=2, TP_WIRE=wht6 — libr4d one-shot all-reduce, not RCCL; min-kb stays the engine default
Slots / window	MAX_SEQS=4, MAX_MODEL_LEN=200000 (native 262,144; longer is refused, never truncated) — this pair is the cache bound: the resolver sizes the KV carve against slots × window, and no slot count funds itself (Laws)
KV dtype	KV_DTYPE=fp8 — the only dtype that funds the slot count (Laws)
VRAM split	derived at boot from GPU_HEADROOM_MIB=96 + the cache bound + HOST_POOL_MIB; naming VRAM_WEIGHTS_MIB/VRAM_KV_MIB is a boot refusal (removed at 1.3.0)
Expert tier	PLACEMENT=expert_tiered, HOST_POOL_MIB=49152 per rank — the pinned host pool is also the KV lever: the expert floor is plane + 2 units − host room, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws)
PLE	NGRAM_PLACEMENT=ram (47.7 GiB pinned; refuses boot on a short memlock)
Spec / batch	SPEC_TOKENS=2 — MTP depth 2, the .rad carries the model's own mtp.* head (mtp.fc_hidden/fc_embedding/draft_head); MAX_BATCHED_TOKENS=2048 (MNBT — 2560 collapses decode, Laws)
Prefix cache	PREFIX_CACHE_MIB=8192, pinned host tier only — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; STATE_DIR=/home/dank/radiance-state now exists only to hold the rad-tune cache
Template	TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja (28,234 B) via --override-chat-template — froggeric's fixed Qwen3.8 template (the repo root's chat_template.jinja, vendored); the checkpoint's own template is baked into the .rad by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it
Step sharing	BUSY_PREFILL_CHUNK=auto — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does not shrink the chunk for decoders alone · PREFILL_INTERLEAVE=1 — the decoder dial, armed 2026-10-20: one decode-only step after each chunk step taken beside decoders; meter engine_steps_held_total, price is the prefilling prompt's TTFT
TTM gate	TTM_MIN_MIB=162000, TTM_TARGET_MIB=204800
Boot	RAD_MIN_MIB=110000 deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s)

Boot ledger for the live shape (2026-10-20, pin 1.8.1, MAX_SEQS=4, HOST_POOL_MIB=49152): the resolver prints elastic 26.73 GiB → experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %), prefill staging 373.65 MiB out of the cache, vram_weights 24.30 GiB, host_pool 48.59 GiB. The referee line: kv: the cache can grow to 5.64 GiB a card — 747,492 tokens of context, 3.7 sessions at 200,000 — with every block it lends the experts taken back; 440.04 MiB of that is per-sequence state, and up to 5.20 GiB is lent to the experts while no request holds it. --checkpoint-slots 40 is auto (max_num_seqs × 10 slots of 111.98 MiB = 4,479 MiB of pinned host), /health reports queue_cap 32 (max(8, seqs × 8)), sampler 12 rows (= seqs × (1 + speculative depth)), 4 hardware queues a card, template 28,234 B, vision live (≤ 4,194,304 px, 16,384 patches/pass), P2P confirmed (the 2 cards reach each other's memory directly). Experts living in the pinned host pool: 8.99 GiB a rank (was 5.99) — the price of the bigger cache, paid over PCIe when those experts are hit.

The boot ledger, not this arithmetic, is the referee: radiance states every budget before anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin, the P2P answer per pair) — read it after every resize, and unlike every predecessor there is no per-boot VRAM planner to second-guess. For comparison, the publisher's own suggested run line on 32 GB cards was --host-pool-mib 12288 --expert-vs-cache-ratio 0.82 with seqs 8 and spec 3 — a 1.0.x-era line, and --expert-vs-cache-ratio was removed at 1.2.0, so today it is a capacity floor quoted from the card, not a run line this pin accepts; the shape above is the fleet's measured one. FRANK_ARCHON_EXTRA_ARGS is the boot-ladder escape hatch (unset → no extra argv; it is expanded unquoted on purpose, so keep multi-flag values as one string).

Harness contract (/home/blank/.omp/agent/models.yml) — provider llamacpp (dankserver) → http://192.168.0.235:7770/v1, api: openai-completions, auth: none, id: frank-archon (= FRANK_ARCHON_SERVED_NAME; the provider label is legacy and renaming it breaks config.yml references). contextWindow: 200000 must equal MAX_MODEL_LEN maxTokens: 16384 (clamp min(maxTokens, window − estimate − 4096); compaction fires at 85 % → 170,000 est × 1.03 + 16,384 ≤ 200,000); input: [text, image] compat.reasoningContentField: reasoning_content; reasoningEffortMap minimal/low→low, medium→medium, high/max→xhigh; extraBody temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 / presence_penalty 0 / repetition_penalty 1.0 + chat_template_kwargs {enable_thinking, preserve_thinking}, every key inside the engine's known-parameter list (unknowns 400). compactionModel routes to llamacpp (dankclaw-3090)/frank-zealot.

Deploy / verify — ./scripts/deploy.sh dankserver --service frank-archon; launch it detached when the swap restarts the endpoint your own session runs on. Then verify behaviourally, not by the green gate: podman inspect frank-archon --format '{{json .Config.Cmd}}' and /v1/models — deploy.sh plans from repo paths and gates on the port, so host-side unit drift is invisible to it.

Eras

One row per retired era on this port. The Rollback cell is the load-bearing one: anything marked "re-fetch" is gone from /home/dank/models/, which today holds only StillDeadcode rad and tpl.jinja.

Window	Payload	Runtime	Shape	Replaced because	Rollback
2026-10-19 → 10-20	same qwen3.8-next-flash-fp8-iq4r-moe.rad (payload unchanged)	radiance 1.8.1, sha256:d590e300… (same digest)	2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, TTM_MIN_MIB=150000, queue_cap 16, sampler 9 rows; --prefill-interleave 1 armed on 10-20	Frank ordered four concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws)	revert 1bb6399 and redeploy scripts/deploy.sh dankserver --service frank-archon — trio back to MAX_SEQS=2 / HOST_POOL_MIB=44928 / TTM_MIN_MIB=150000
2026-10-15 → 10-19	same qwen3.8-next-flash-fp8-iq4r-moe.rad (payload unchanged)	radiance 1.0.13, sha256:4c3f4845…	3 × 200,000 fp8 KV bought with the explicit pair --vram-weights-mib 25600 --vram-kv-mib 5120; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk	decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, --busy-prefill-chunk and --prefill-interleave, exist only after this pin	FRANK_ARCHON_IMAGE back to sha256:4c3f4845… plus a trio rewrite: restore the pair and the host+disk prefix knobs, delete BUSY_PREFILL_CHUNK/PREFILL_INTERLEAVE. A digest swap alone is a boot refusal (Runtime)
2026-10-08 → 10-15	in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %)	radiance 1.0.13, same digest	3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries	the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint	.rad purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700 → qwen4exp-w4nl64-i8-hc8m.rad, 122,071,973,888 B, LFS sha256 09dd9306…, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point MODEL_FILE at it, redeploy. Re-convert is dead: the BF16 source was rm'd 2026-10-15 (kld-ref, grams, corpora and /home/dank/logs/swift15_*.log survive)
2026-10-05 → 10-08	qwen4exp-fp8-native.rad, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation	radiance 1.0.7 line, same digest	—	native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, 1.93 sessions — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot)	rebuild-only: scripts/download-frank-archon.sh (official Qwen/Qwen3.8-Flash-Next-FP8, 131 shards, 172.8 GB, ungated) + scripts/build-frank-archon-rad.sh (no-recipe convert under the descriptor law); raw and .rad were rm'd 2026-10-15 (Frank's call, to free the volume for the iq4r download)
2026-10-05 (same day)	orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 safetensors, 131 shards	FreeToken PR #587 head 77a7201 (rocm714_torch214), image localhost/frank-primary-freetoken:pr587-77a7201	—	TP=2 cannot boot on ROCm: the per-rank pynccl.cu JIT cannot resolve cuda_runtime.h for gfx1201, so the second card idled	git tag frank-archon-freetoken-pr587-20261005 (trio + scripts/docker/frank-primary-freetoken.Dockerfile, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account
2026-09-29 → 10-05 (interrupted 09-30 → 10-01 by GLM)	same orcarouter FP8 safetensors	tcclaviger vLLM fork, docker.io/tcclaviger/vllm@sha256:da402966…	shape G: 3 × 200,704 fp8, KV 3,166,500,000 B/rank → 401,408 tokens, arena 17.77 GiB / 7,762 slots, util 0.982, MNBT 2048, no spec, no prefetch, prefix caching ON	radiance gave real TP=2 (no RCCL) and a 2-3 min boot, and the fork never fixed its long-context class	git tag frank-archon-vllm-shapeG-20261003 (trio + the 362-line env lore); image still in dank's rootless storage; payload needs a re-fetch. The 2026-10-01 fp8+MTP4 rung is tag frank-archon-fp8-mtp-20261001
2026-09-30 → 10-01	Intel/GLM-5.3-Flash-W4A16-AutoRound ~170 GiB (sym int4 g128, auto_gptq)	same tcclaviger digest; R4dW4a16MoEMethod → libr4d grouped W4A16	—	—	payload (170 GiB) + drafter (2.2 GiB) went to ~/.AGENTTRASH/20261001-glm-purge/ and were emptied; the cutover was reversed by d91379c, restoring the 2339fae trio; scripts/patches/glm53-*.patch survive in-tree but none applies byte-exact since the 2026-10-10 rename rewrote their context lines
2026-09-27 (same day)	Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS, 92.39 GiB + PLE 26.82 GiB (34fa36f8…)	same R9V bundle + shim	MAX_MODEL_LEN=196608, offload budgets 146.25 / 78.75 GiB, PLE residency ssd mmap, tool parser qwen3_coder, thinking on the reasoning (vLLM) channel	same-day flip back to the reference quant with the MTP4 head, at Frank's direction	/home/dank/r9v-state/uncensored/ (setup.json, restore-config-20260927.env) is still on disk; the IQ4_XS payload was deleted in the 2026-10-17 cleanup → re-fetch, point R9V_STATE_FILE back, drop the two q8-only keys and the template keys
≤ 2026-09 — 4 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete	—	—	—	—	—

Before 2026-09-26 the port belonged to the frank-medium service, and its era pages are not superseded here: is the locator for that endpoint's whole shape history, including the unresolved end date of the vllm-radiance era (this table opens on 09-26; the era page claims 10-10).

Laws learned here

The radiance engine

• KV dtype law. --kv-cache-dtype takes only bf16|fp8 — ONE dtype for the whole cache, a radiance restriction, not stock vLLM's: upstream advertises auto|fp8|fp8_e5m2|fp8_e4m3 (docs.vllm.ai engine-args) and even there the advertised set is not the usable set — several accepted values crash engine init at the attention layer's own dtype check (vllm-project/vllm#42587). So llama.cpp's exact-K / quantized-V pair (what frank-zealot runs as --cache-type-k/ --cache-type-v) is unservable here: the rung below bf16 is fp8 on both halves. Measured per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot 2026-10-11): kv_attn 13 layers / 4-token blocks pages 26.00 → 52.00 KiB, while kv_qsa_bkey stays 3.25 KiB at both — the QSA block key is an index, not attention state, and does not follow the flag. So 7.3125 KiB/token at fp8 vs 13.8125 at bf16: 1.89×, not 2×, precisely because the bkey plane is dtype-fixed. Secondary saving: a prefix-cache entry tracks the attention plane (110.50 KiB at bf16 → 58.50 at fp8, per the kv tier transfer lines), so the same 8 GiB host tier parks ~5.8 sessions of 200k at fp8 where it parked ~2.9 at bf16.

• Pool sizing law (the explicit-pair era — at 1.8.1 the cache is derived, so the resolver law below owns it). pool_mib ≥ tokens × 7.3125 KiB + 110 × seqs + 448 — the dtype- independent terms are also in the pool: kv_gdn_state 54 MiB × 2 copies per sequence (+conv/ tail/ple ≈ 110 MiB a sequence) and the prefix checkpoint region 8 × 55.99 MiB. Deployed: 3 × 200,000 at fp8 = 4,285 + 330 + 448 = 5,063 MiB, pool 5,120, which predicts (5120 − 330 − 448) × 1024 / 7.3125 = 608,027 tokens against a 600,000 contract (1.3 % slack) — every MiB above 5,063 was handed to the expert slab instead. At bf16 the same three slots cost 8,093 + 778 = 8,871 and the slab falls to 20,249 (−9 %); four slots at bf16 cost 10,791 + 888 = 11,679 and the slab falls to 17,441 (−33 %); fp8 keeps 22,208 of slab at both slot counts. fp8 is not a preference, it is what funds the slot count. Measured backings to sanity- check the formula: bf16 pool 6,912 at seqs 2 backed 462,752 tokens (2026-10-11), the fp8-era pool of 3,543 gave 386,904 (1.93 sessions), and the four-slot fp8 boot of 2026-10-13 19:21 backed 843,276. The ledger's backs N sequences all at 200000 tokens at once line is the referee — a pair this tight is read off the boot, never assumed, and the concurrent probe is its regression test. A fourth full-height conversation needs 6,601 MiB of pool: refusing it is Frank's explicit call, and under that pin serving it was a --vram-kv-mib change, not a code change. At 1.8.1 the same decision is MAX_SEQS — the resolver buys the cache the demand signal implies.

• The hard limit is per request. The engine treats kv group tokens >= max-model-len as a HARD limit — "one request longer than N tokens is accepted by the API and then refused as too large to ever fit — this is a hard limit, not a backing ratio" — so the pool is sized for seqs × max-model-len, not for one sequence. Radiance never truncates, at any size.

• One VRAM budget per card (the arithmetic below is the pair era's; the ceiling survives, the knobs do not): weights + kv ≤ 30,969 MiB, read off the engine's own budget table (boot 2026-10-13 at pair 24000/5120): card 32,624 MiB, budgets claimed 29,120, and "3.42 GiB is neither budgeted nor accounted — the driver's context, the activation arena and --gpu-headroom-mib come out of it; anything past those is throughput on the floor", with the card settling 2.01 GiB free against a 209.19 MiB floor (96 headroom + 113.19 kept for the static region). Reserve above the engine's floor = 2,058 − 209 = 1,849, ceiling 29,120 + 1,849 = 30,969; the deployed pair lands 458 MiB above that floor. The 2026-10-11 bound of 29,120 ("31,856 − 1,710 outside both budgets − keep 1 GiB free") was a hand-picked reserve, not the engine's wall. Under that pin, boot with the explicit --vram-weights-mib/--vram-kv-mib pair, never --kv-cache-size-mib; with the disk tier engaged the engine also forced ~9.6 GiB of weight buffers + arena onto the weights side (~29.6 GiB/card effective). Neither flag exists at 1.8.1: the same ceiling is enforced by the resolver, and passing the pair is a boot refusal.

• At 1.8.1 the split belongs to the resolver, and the source fixes its order. core/mem/vram_budget.cpp claims the card bottom-up (capacity → already held → drift → arena → staging → queues → prefetch → headroom = claimable), takes the static weights off the top as a floor, not a share, and divides the elastic remainder in three lines (146-156):

  exp = std::min(b.expert_floor, b.elastic);
  kv  = std::min(kv_cap, std::max(b.elastic - exp, kv_one));
  exp = std::min(std::max(b.expert_ceiling, b.expert_floor), b.elastic - kv);
  

  kv_cap is KVManager::ceiling() = state + Σ_paged max_seqs × ceil(max_ctx / block) × bytes (core/mem/kv.cpp:266), so --max-num-seqs and --max-model-len set the cache's demand, never its funding. The funding is elastic − expert_floor, and the floor is min(plane + spare, plane + spare + 2·unit_max − host_room) (core/place/planner.cpp:100-104, with host_room = host_pool − file_stage_reserve at core/engine_bringup.cpp:1418-1422). That is the cache's lever: every MiB added to --host-pool-mib drops the expert floor one-for-one and the cache takes it — minus stage, the prefill stager's ring (two layers of the experts left off the card), which is charged back out of the cache whenever the cache lends (vram_budget.cpp:193-203). Boots of evidence, all on 1.8.1:

  shape	demand	cache given	engine's backs %
  3 × 200,000, host pool 44,928	600,000 tok	4.20 GiB	93 %
  3 × 200,000, host pool 45,568	600,000 tok	4.20 GiB — byte-identical	93 %
  2 × 200,000, host pool 44,928	400,000 tok	2.76 GiB	91 %
  4 × 200,000, host pool 49,152 (live)	800,000 tok	5.64 GiB = 747,492 tok	93 %

  The 45,568 row is the ceiling case, not a mystery: at 3 slots the cache already held its whole ceiling, so extra host room could not grow it and shows up as unclaimed. The referee is the ledger line kv: the pool backs N % … the cache can grow to … N tokens, never arithmetic. To buy window: raise --host-pool-mib (cost: pinned host RAM, and experts evicted to host cross PCIe — experts-from-host was 5.99 GiB a rank at 44,928 and is 8.99 at 49,152) or cut the per-token cost (fp8 KV). Raising slots raises demand and costs 10 × 111.98 MiB of pinned checkpoint-store RAM per slot. Under-sizing the pool the other way has always been a boot failure (36,864 refused a boot in the fp8 era: the host cannot hold what the cards will not). Behavioural check on the live shape: /health reports queue_cap 32; a request beyond the slot count queues rather than being refused, and the 400 class is a single request over --max-model-len.

• Idle KV VRAM is the expert cache's spillover — the price is paid twice. mover: 2275 slots can be lent out of the KV cache (2.69 GiB), arena: lends 740 MiB while steps are <= 256 tokens; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 in the pair era) so a bigger KV pool steals weight residency from the slab AND from the lendable cache. Priors on this card/model: stealing 2.14 GiB of weights to buy MAX_MODEL_LEN 262144 cost 2.5 % decode; the other direction (--expert-vs-cache-ratio 0.82 → 0.84, +2.5 GiB of weights) cost 8.9 %.

• A KV-pool cut is a runtime-headroom lever, never an arena lever. The 6.1e9 → 3.15e9 B/rank pool cut (2.75 GiB/rank) bought zero extra expert-cache slots. The arena moves with the reserve, one-for-one (d(reserve) 0.57 GiB = d(arena) 0.57 GiB = +248 slots). The shape that pool cut replaced died with 0 bytes free on a real 995-token request — that is headroom, not slot count. Read expert cache space … / N slots out of the boot ledger before trusting any pool arithmetic: the 2026-10-13 read was 14.87 GiB / 6,496 slots against GPU KV cache size: 398,658 tokens, and it is the POOL that caps resident conversations, not the arena.

• MNBT moves the compile RANGE, not the batch width. The engine sets num_reqs = min(max_num_batched_tokens, max_num_reqs), so the CUDA-graph compile width is SEQS + 1 for any MNBT >= SEQS — 3 at SEQS 2, 4 at SEQS 3, exactly the constant printed in the engine's own message. MNBT only shifts the compiled range ((1,1024) versus (1,2048)), which is why the identical capture failure appears at both rungs; --enforce-eager bypasses the path and loses CUDA graphs with it.

• The prefix cache proves itself at depth, not in the ledger. Two concurrent conversations of 196,919 tokens each answered the marker buried at the head of their own prompt with 393,837 tokens resident — above the fp8 pool's 386,904, so the answer came from a cached prefix and not a re-prefill. Same boot (2026-10-13): decode at 51.5k depth 94.1 cold / 102.3 warm / 117.6 prefix-hit t/s, draft acceptance 0.7284 / 0.7525.

• The host pool moves with the slab. vram_weights and host_pool_mib serve the per-rank resident set together: at the fp8-KV pair (26071/3543) the ledger showed the slab full to 259 KiB of slack, and cutting vram-weights by 3.8 GiB to pay for a bf16 pool spills ~3.8 GiB/rank into host RAM — under-sizing the pool there is a boot failure (36,864 refused the boot in the fp8 era). The ledger's coverage line, not the arithmetic, arbitrates.

• Slab → prefill: the arms disagree, the cliff does not. 2026-10-11 exact-KV A/B (slab 22,208 vs the fp8-era 26,071, −14.8 %): prefill −29 %, TTFT at the full window +25 %, decode inside the noise band on 2 of 3 passes, draft acceptance UP 0.661 → 0.740. 2026-10-13 (probe-frank-archon-kv.py, env header): A-cold 43.5k 22,208 → 3,397, 24,000 → 3,290, 25,600 → 3,584 t/s, and the 3-way 197k arm's aggregate 3,051 → 3,589 (wall 193.7 → 164.5 s), so +1,600 MiB of slab bought +8.9 % single-stream / +17.6 % concurrent — read then as a SLOPE, the 22,208/24,000 pair having landed inside noise of each other and falsely looked like a plateau. Re-measured at 3 slots after the 2026-10-15 swap the 25,600 arm returned 3,290: the 3,584 did not reproduce, so treat 22,208-25,600 as one band until someone re-runs the ladder. What is not disputed is the cliff: 26,071 → 4,379 t/s (the 2026-10-11 control at seqs 3 / pool 3,543), +22 % for +471 MiB — unreachable at 3 slots (26,112 + 5,120 = 31,232 vs 30,969) and free at 2 (pool 3,072 + slab 27,897 = the ceiling exactly). Two points inside noise of each other are not a plateau, and two points are not a curve.

• Decode is nearly window-free; slots are paid in concurrency. 99.0 t/s at 43.5k vs 93.3 at 197k (−6 % for 4.5× depth), while three streams give 113.9 / 43.3 / 10.9 = 18.4 t/s aggregate — a fused-step lottery, not a fair share. Read it from leg D of probe-frank-archon-kv.py with ≥ 128 generated tokens; an 8-token probe measures first-token noise.

• MNBT 2048. 2560 collapses decode to 2.6-7.7 t/s (MTP resonance); the w4 file tier streams in place only at MNBT ≤ 2048 and the activation arena scales with the chunk (32768 → 33.3 GiB, refused).

• Decode pays for the chunk it shares a step with, and --busy-prefill-chunk auto does not protect it. Re-measured on the live 1.8.1 shape (34.8 h boot, Prometheus at 60 s, engine_step_seconds{,_decode}_total / engine_steps{,_decode,_mixed,_prefill}_total / prefill_tokens_total): decode-only steps run at 15.5 ms, chunk steps (a chunk alone, or a chunk with decode rows beside it) at 166 ms — 4.7 % of the steps hold 34 % of the engine's step time, so a session decoding while another prompts advances one token per 166 ms. Bucket the harness's radiance:decode_tokens_per_second by radiance:prefill_tokens_per_second (15 s scrapes; per-stream rate = the gauge ÷ vllm:num_requests_running): median 166 t/s under light prefill, 81 t/s at 1-5k, 13 t/s above 5k — a 12.7× collapse, and at 13 t/s with two sessions running each stream sits at 6.7 t/s = 149 ms a token, one token per shared step. Nothing else moves: decode-only steps stay 14-18 ms in every bucket (14.1 ms in a prefill-free minute, 17.9 ms in the deep one) and the relationship is lag-0 only (r = −0.74 at lag 0, |r| ≤ 0.19 from 30 s on, decode back at baseline by 5 min) — residency is not the mechanism either. The second victim is the arriving request: through a high-prefill window TTFT p90 goes 2.4 → 18.0 s and 15.5 % of requests wait over 10 s for their first token, with http_queue_depth at 2-3, while median TTFT (~1.9 s) and per-token latency barely move. Preemption is real but burst-shaped: 264 vllm:num_preemptions_total in the boot, 108 of them inside 105 s of a 6.5k-tok/s prefill. The chunk is min(remaining, checkpoint interval, MNBT, busy_cap); the first three read 2048 here, and busy_cap is the fourth law.

• auto is a prefill-speed dial, not a decoder-protection dial. Scheduler::busy_cap reserves 1 + spec rows per decoding sequence behind the chunk (3 tokens at MTP 2) plus one eighth of the budget per waiting request or waiting prefill, and the code refuses to do more for decoders: "not for decode rows alone: their reserve is a few tokens, so the chunk ends a quantum short and the next step finishes the interval… Halving would hold a long prompt to half-interval chunks while anything decodes behind it, which costs more" (core/sched/scheduler.cpp:874-878, 1.8.1). Live proof: prefill_chunks_capped 5,018 against 62,386 chunk steps — 8 %, and those are the minutes with a waiter, not the minutes with a decoder. An explicit N is the only value that applies "whenever the chunk shares the step".

• At MNBT 2048 no chunk value keeps both the arena loan and the staged copies. The arena is a ladder of levels (256 and 1024 rows here) and a step's rows are its whole token count — max(n_tok, n_seq × (1 + spec)), core/engine.cpp:1142 — so a 1024-token chunk with two decode rows is a 1030-row step: it falls back to the full plan, which "lends nothing", and it arms the prefill stager. The stager's floor is the largest lending level plus one, 1025 rows here — named on the boot line prefill staging: … a step of more than 1024 tokens copies the pooled experts of 48 routed layers onto the card up to 3 layers ahead — and below it "every routed layer reads its pooled experts synchronously and the prefill slows far more than sharing saves" (GUIDE §4.5). The vendor's own number for naive shrinking, on this architecture: four decoders with a long prompt beside them finished in 29.5 s at a 2048 chunk and 31.9 s at 512.

• A big step takes the arena from the expert slab and keeps it for the whole prefill. The arena lends 987.77 MiB (steps ≤ 256 rows) or 670.26 MiB (≤ 1024) to the slab; a full-plan step reclaims it synchronously before the step — units revert to their host copies (an address change, not a copy) and both read streams drain — and arena_tick regrows the loan only 2 s after the last big step (core/engine.cpp:1129-1202), so every step of the prefill run, decode-only included, runs with a slab ~780 expert units smaller over a link also carrying the staged copies. /diagnostics says the binding constraint in one line: "a prefill step streams every routed layer's pooled experts onto the card, so its speed is bounded by the link; decode reads the ones it routes to". Measured size of the whole effect: the ~10 % above. Real, small, not worth chasing.

• --prefill-interleave is the dial that buys decode its cadence; armed at 1 on 2026-10-20. N decode-only steps follow any chunk step taken beside decoders; held steps admit nothing, so it costs the prefilling prompt its TTFT and an arriving request its first step. Rate arithmetic at the two measured step times and 0.657 acceptance (≈2.3 tokens a step): N=0 → ~14 t/s, N=1 (armed) → ~23, N=2 → ~33, N=4 → ~48, N=8 → ~69, with the prefill slowed by (166 + 15.5N)/166. Arithmetic, not measurement — the arm is chosen by the §9 bucket pair plus steps_held, and auto is not a substitute for it.

• --prefill-span is the shape that keeps the loan, and it is priced. It needs MNBT = N intervals, so --prefill-span 2 --max-num-batched-tokens 4096: a lone prefill carries 2 intervals a step (and on an expert-tiered box only from 8 intervals / 16k tokens, where the span starts to win), while every other step is planned at the shared budget of 2048 — which is a lending level and keeps the stager's ring, the one combination --busy-prefill-chunk cannot express. Costs per GUIDE §4.5: 320 MiB a card of expert residency, ~3 % off the KV pool (§4.3's cliff is sharp), a decode step uploads twice the per-step arrays at 4096, and checkpoints inside a span are not written, so a partially-shared prompt resumes one interval earlier. On the vendor's second box (2×R9700, w4nl64, TP2) it was TTFT-neutral — 2.61 vs 2.56 s at 18k, 10.72 vs 10.55 at 75k — hence their own rule: measure your own prompts at two ranks before turning it on. MNBT 4096 does not boot at 3 slots; this shape is 2.

• Most of this endpoint's prefill work is replay of tokens the prefix cache already had. Over the same window prefill_tokens_total 65.2 M against prefix_hit_cut_tokens_total 24.9 M = 38 % replayed, and it is not an eviction: a hit on a hybrid model reuses only as far as the last linear-state checkpoint (reuse = min(h.n_attn_tokens, h.n_linear_tokens), core/sched/scheduler.cpp:953-978), and the retention default keeps a geometric ladder - about log2(depth/interval) + 2 snapshots - so a turn that diverges d behind the tip can resume as deep as tip − 3d. The live ladder for a 200k session at interval 2048 is 2048 · 131072 · 163840 · … · 198656: diverge at 120k and 21.7k tokens replay, diverge at 50k and 48k. Every prefix_hit_cuts_total carries reason="no_snapshot" (the other two reasons stay dead), so the cause object agrees and hit_cut_tokens is the cost. --checkpoint-policy admits exactly geometric-backoff | tip-only | keep-all and refuses anything else at boot. It governs retention only - the writes happen at every interval whatever the policy says (only --prefill-span skips them) - so keep-all buys a replay bounded by one interval at any depth for pinned host RAM alone: 97 snapshots a 200k session, --checkpoint-slots auto asking 196 × 111.98 MiB = 21.4 GiB against a store budget of an eighth of host RAM (31.5 GiB here, so share admits it) with room - the pinning room left beside the expert pool at boot - as the hard bound the ledger line names. Retention also spares a snapshot a later turn resumed from (core/mem/prefix.cpp:709-723), so a chain that keeps resuming at one deep point stops paying for it. Shortening --checkpoint-interval does not buy the deep case: the coarsest ladder scale is the largest interval × 2^k under the depth, so the deepest gap stays ≈ half the prompt (129,024 tokens at interval 2048 against 130,816 at 256) while every cut multiplies the snapshot writes.

• Slots cost KV, host RAM, rows, and pin — never speed. --max-num-seqs carries no radiance constraint: it adds ≈110 MiB of recurrent state per slot, 10 × 111.98 MiB of pinned checkpoint-store RAM per slot (--checkpoint-slots auto), and declares the fused TP all-reduce over seqs × (1 + spec) rows — wht6 is bit-exact up to 25 rows, and the live shape is 4 × 3 = 12, so single-sequence decode stays exact and so does any step up to that many rows. Raising it also raises the cache's demand (ceiling = slots × window), which the resolver funds from elastic − expert_floor — so a slot count without matching host room is paid for by expert residency. Measured 2026-10-06: seqs 2 and 3 statistically tied (30.9-33.1 t/s, inside the ±3 % floor). Measured 2026-10-19 → 10-20 at 1.8.1: 2 → 4 slots cost +4,224 MiB of pinned host pool and −3.01 GiB of expert residency a card and bought 2.76 → 5.64 GiB of cache (365,204 → 747,492 tokens). The old "seqs 2 cannot boot" law belonged to the retired tcclaviger fork's torch.compile shape specialization, and radiance booted seqs 2 on its first fp8-native start.

• A dequantize-to-compute path bills host RAM twice. On the EMULATION backend an offload request of 118 GiB landed as ~238 GiB of real host RAM — the dequantized copy rides on top of the pinned weights, and the doubling is invisible in the weight count. Read the engine log for Using 'EMULATION' Mxfp4 MoE backend before concluding "doesn't fit"; a page that says a payload "does not fit" without that line has not read the runtime.

• One encoding per expert table (rad_arch_declare): a whole expert kept bf16 is the only thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable.

• Descriptor-selection law. The plugin is picked by the exact (arch_id, quant descriptor) pair with no fallback: a raw official FP8 checkpoint (quant_method: "fp8") is refused by serve and convert, because the shipped qwen4exp_fp8 plugin claims (qwen4exp, ""). Stage a config.json with quantization_config dropped — the descriptor changes, the data stays FP8.

• Three convert laws, each paid in a full verify+declare cycle. Scale-name: scale_name() appends _scale_inv to the tensor name, so stacked …experts.gate_up_proj keeps its scales at …gate_up_proj_scale_inv. File-shadow: the reader opens index-named files sorted and emplaces every tensor each file physically holds, first-wins — a merged PLE shard must sort first (00-ple.merged.safetensors) or a raw shard wins the name. Symlink-through: staging symlinks the raw index, so open(path,"w") writes through it and corrupts the checkpoint — write temp + os.replace.

• Convert is card-less (rad-convert declares on CPU, GPTQ solves on CPU BLAS) and --plan-only goes first: a plan that resolves every op will boot. rad-tune must be declared at the shapes you serve and its cache lives in the image ($RADIANCE_HOME/tune) so it only survives on the $FRANK_ARCHON_STATE_DIR/tune bind-mount the serve script adds.

• Quant gates are scored at the production shape (MNBT 2048, not the recording's 32768) against the BF16 container: KL ≤ 0.03, top-1 ≥ 95 %, and the verdict is chunk-size invariant (arm Δ ≈ the 0.0015 float-order band) and placement-invariant. Protected experts are data-driven — energy share ≥ 3 % AND ≥ 3× routing share, even count per rank half — and the PLE rides the checkpoint's own fixed scale (1.9931793212890625e-04, absmax inside the 2⁻¹³..2⁻¹⁰ plateau).

This host, and the deploy path

• The pinned-RAM ledger is why this is a root unit: 2 × 44 GiB expert pools + the 47.7 GiB PLE n-gram table (51B params, read from pinned host RAM over io_uring — Frank's pin law: "Native FP8. Experts from ram. Pin the PLE to ram.") + the 8 GiB prefix host tier ≈ 147 GiB charged against RLIMIT_MEMLOCK; dank's login session caps at 8 MiB and a user quadlet is capped by the user manager, so only PID1 can grant infinity — and --ngram-placement ram refuses the start when the limit is short.
• TTM is runtime state, and the gate that owns it raises before it refuses. The pinned pages ride /sys/module/ttm/parameters/pages_limit (+ pages_max on kernels that expose one; this kernel exposes none — the cap is the MIN over the knobs present, and an absent knob is skipped, never read as zero, because a zero would refuse a healthy cap). The kernel default is half of RAM, so every raise dies with the reboot: the 2026-10-05 manual raise to 52,428,800 pages (200 GiB) was gone after the 2026-10-12 23:17 power failure, the cap fell to 125,988 MiB, and the preflight refused the start 6,919 times over ~20 h while Restart=always cheerfully re-ran a refusal. A manual sysfs write is therefore not an acceptable repair procedure — scripts/ttm-preflight.sh writes TTM_TARGET_MIB (204,800 MiB = 52,428,800 pages; pages are 4 KiB so pages = MiB × 256; the recorded host ceiling, because writing the 150,000 floor would leave the ledger no slack) into every knob present, re-reads, and only then refuses. Write failures are swallowed deliberately: the caller re-reads, and ttm itself refuses a limit below the pages it has already pinned.
• Privilege of that knob, measured the hard way: it is root-owned 0644 and the service runs User=dank, where -w is false and a plain echo fails. The canonical privileged path is the unit's ExecStartPre=+… (the + runs it as root even under User=); sudo -n is the fallback for a human. The file is sourced by the serve script (which re-verifies the cap before podman run) and executed by the unit. A refusal is invisible outside the journal — no container is created, the unit sits in activating (auto-restart) on one repeating line — and without the gate the load instead dies mid-pin with a bare allocator error.
• The prefix host tier needs real RAM: 8 GiB of it, and with the service loaded the host still has ~104 GiB available (246 GiB total) — that headroom is what the pinned ledger is measured against.
• The deployer's blind spot is host-side unit drift. A leftover frank-archon.service.d/bench.conf overrode both FRANK_ARCHON_ENV_FILE and ExecStart to a staging worktree and every deploy.sh since was green against the old configuration — drop-ins are read after the unit's own Environment= lines, so they win.
• Never systemctl disable a unit on these hosts — it removes the /etc/systemd/system/ entry, which here is the symlink into the checkout. mask is reversible but fails on a relinked unit, so a bench window uses stop and the deploy restores the unit.
• The restore oracle for an experiment window is a conjunction: is-active == active AND is-enabled == enabled AND no /run/systemd/system/<unit> mask symlink AND the container is owned by the unit and not Stopping. The vLLM/GLM-era engines ignored SIGTERM and answered /health 200 for 20+ minutes while Stopping; a runtime mask survives stop and blocks start, and a failed unit needs reset-failed first.
• MemAvailable is an input, not a backdrop: the expert-offload planner reads it, so anything booted beside the live service dies at plan time (ExpertPlanError: host holds 5113 of 12096 swappable experts) — and fleet law forbids GPU experiments beside production anyway. Long remote work runs as a transient unit or setsid+disown: nohup X & inside ssh races ssh teardown and can die before writing a byte.
• The ~40k-token corruption class was the RAM, not the engine. Dropping DDR5 from 4800 to 4600 MT/s ended it, with zero gdn_hip faults either side; the prefix-caching / GDN re-seed theory was confounded on both arms. Consumer non-ECC raises no MCE for a silent flip and a 60 s / 8 GiB pattern check covers 0.5 % of 246 GiB — silent RAM corruption outranks every software theory. That mitigation is a BIOS setting and it is not in effect now: sudo dmidecode -t memory | grep -i configured answers 4800 MT/s on 2026-10-20, and the only event between the fix and today is the 2026-10-12 power failure that also reset TTM. Check the speed before crediting any software revert with a fix — and before believing a clean run of the arm you just tested, because the confounder is still in the slot.
• TP=2 here is legal since the 2026-09-20 ProArt B850 swap (x8/x8 on CPU root ports, Routing+/ReqEn+ end to end) only because radiance's wire is libr4d's own one-shot all-reduce; RCCL needs atomics on the whole chain and fails hipErrorIllegalState without them (). wht6 buys +5.6-6.3 % prefill for ~1.3 % of 8-stream decode, exact is the bit-exact alternative.
• The link is already at the board's ceiling — do not chase the link-below-max warning. Both root ports (00:01.1, 00:01.3) are x8 max at 32 GT/s, so PCIe 5.0 x8 ≈ 31.5 GB/s is the whole budget per card and the cards' own pcie_dpm tables expose only x8 states. /diagnostics still warns that the on-card switch hop (01:00.0, 05:00.0) runs "x8 of its x16 while the card is 100 % busy" and suggests a reboot or a reseat — pointless when the host side of the chain is x8 max, which the same endpoint reports two findings later as link-capped. The width that matters is the minimum of the chain, and it is x8 either way, so the only lever on a link-bound prefill is less traffic over the link (more expert residency, fewer pooled experts), never a wider link. P2P resolves on (auto asks the driver, the two cards reach each other's memory), so the collectives do not hairpin through host RAM, and the engine's copies ride one DMA queue that "give[s] way to the collectives" — the all-reduce wins the link, the staged experts wait.
• Eight cores is the whole host, media stack included. Dankserver is a Ryzen 7 7700X (8 cores / 8 threads, one NUMA node, nodes: -1 for both cards) running Jellyfin, Plex, Calibre and RomM beside the engine, and --http-threads defaults to one thread per core. Measured during a heavy prefill: load average ≈ 1.0, jellyfin 11 % CPU, radiance 7 % — CPU contention is not the current decode culprit, so do not spend an experiment window on it; re-read uptime and ps -eo pcpu,comm --sort=-pcpu | head first if a future collapse looks different from the step-sharing one.
• --prefix-cache-dir creation is not recursive — the serve script mkdir -ps the path or the disk tier silently never opens. The tier does survive a payload swap when arch and tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; cache_n proves a hit).

Dead ends — do not re-try

• radiance 1.2.0 as the serve pin (sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa commit e95d397), benched 2026-10-06 on this exact shape with a 43.6k greedy probe: decode 29.21 t/s and draft acceptance 0.7123 vs the 1.0.13 control's 30.08 / 0.7549, and the control reproduced the recorded 0.755 exactly — greedy acceptance is deterministic, so the drop is a real numerics change (the decode loss tracks the acceptance ratio, 0.943 vs 0.945) with the 1.2.0 MoE weight-rotation change 92b8aaf the named suspect. The vram-pair translation itself is exact (the 1.2.0 ledger took 25.46 GiB weights / 3.46 GiB KV per card, byte-identical to 1.0.13 at EVC 0.82), and 1.2.0 REMOVED --expert-vs-cache-ratio in favour of the pair, which both pins accept. Re-test 1.2.0+ once the rotation change is untangled.
• FreeToken TP=2 on ROCm: the pynccl.cu JIT cannot resolve cuda_runtime.h for gfx1201 — an untested edge of that PR, not a board fault.
• tcclaviger forks: --max-num-seqs 2 (ConstraintViolationError on query_start_loc) --gpu-memory-utilization 1.00 unbootable, 0.982 is the law; expert prefetch is poison (36 % precision — deleting it was +40-51 % decode); TunableOp-with-offload corrupts numerics (repetition loops, early EOS) — autotuning that changes text is broken tuning; ngram-only drafting dies at init (PLE inputs were not prepared); DCP needs TP > num_kv_heads (2 here, so 4 GPUs); 6 × 262,144 fits no KV dtype (6 sequences leave 9.18 GiB of pool where fp8 needs 12.47). fp8 KV on that path also removes the only MLA decode gfx12 has (the aiter op is gated on CDNA > 2, so the draft step demands …mla_decode_fwd, which cannot exist there) and its KV reserve model is dtype-blind (identical 13,600 B/token plan line) — no capacity for the breakage.
• DeepSeek-V4.1-Flash on 32 GiB cards: only --ds4-prefill exact is legal, --ds4-fused-decode is refused, and at ~22 % resident experts each token pulls ~187 × 12 MiB from NVMe — low GPU util with a pegged link is a capacity verdict. Two corrections survive: expert TP does work on gfx1201+gfx1201, and an expert-placement JSON outside the mounted /models tree gives a silent single-tier fallback.
• Which experts you pin does not matter; how many does. An IBS census of the live server (scripts/frank-archon-expert-activity.py) measured per-block read volume flat at 0.93-1.08× of mean across every CPU-resident block — top-k routing fires all of them every token, so at -ot granularity the lever is COUNT.
• Never mount a diagnostic hook on PYTHONPATH without a marker guard: sitecustomize is imported by every interpreter, a child inherits the hook and spawns again — a 19-level python fork bomb inside the container. Kill with podman rm -f (the cgroup owns every generation) never pattern-kill an interpreter name on this host; use TORCH_LOGS for dynamo questions.

Contract (the wire)

• Live surface: /health (== /ping), /v1/models, /v1/chat/completions /v1/completions, /metrics, /stats, /sessions, /dashboard, /diagnostics. /health answers 200 JSON (model id, in_flight, queue_depth) while serving and 503 only while stopping (core/server/admin.cpp:385), and the listener binds after the full load (main.cpp: engine.start() then engine.serve()) — during the load the port refuses connections, so a bare curl -fsS …/health gate is honest and needs no body grep.
• /diagnostics is the instrument for "is the box, not the config, the limit" — one call returns configuration (every flag, so drift from the env file is checkable without the host), environment, derived (the VRAM split as the ledger prints it), hardware (host, per-card link_chain + pcie_dpm + clocks + power cap + DMA engine choice + P2P answer) and findings (level / what / why / fix). On this box it reports experts-from-host (5.99 GiB a rank crosses PCIe when used), link-capped (root ports are x8 max — the board), swap-in-use, tp-wire-lossy, and link-below-max as a warn at 100 % card busy on both cards — which is structural on this board; see the x8-ceiling law under This host, and the deploy path.
• Served id. FRANK_ARCHON_SERVED_NAME=frank-archon (the wire rename of 2026-10-09 moved only the alias; container, unit, scripts and env keep the service name). The env's contract line is that radiance validates the requested model name, so every client — harness, bench scripts, probes — MUST send it — while the era pages also record that a chat response echoes the requested name, which is why /v1/models is the served-id oracle and a 200 chat probe proves nothing about what is actually loaded.
• Reasoning lands in reasoning_content (--reasoning-format defaults to auto; no parser flags exist or are needed) and usage bills it separately (reasoning_tokens). Thinking consumes max_tokens, so the classic false negative is a probe with empty content and finish_reason: length/stop because the whole budget went inside the think block: either set chat_template_kwargs.enable_thinking=false or budget a few hundred tokens and read reasoning_content. Same trap, sharper, on the vision probe below.
• Sampler surface: temperature, top_p, top_k, min_p, typical_p, presence_penalty, frequency_penalty, repetition_penalty, repeat_last_n, chat_template_kwargs, reasoning_effort, enable_thinking, preserve_thinking accepted; unknown parameters are refused 400 (nothing unverified can ride the wire, and an extraBody typo fails loudly); max_tokens is clamped not refused; response_format: json_object honoured; logprobs refused; no seed = the engine picks; n defaults to max(8, max-num-seqs); stop strings ≤ 64 × 4096 B.
• Tools: the qwen3.8 dialect from the chat template → message.tool_calls + finish_reason: tool_calls. reasoning_effort: the froggeric template does not gate the set (xhigh/high/medium/low all 200); the harness map is a safety net, not a requirement.
• Vision is native (the checkpoint's 27-layer Qwen-VL tower through the engine's Qwen-VL processor): an image_url content part, no legacy top-level images field, ≤ 4,194,304 px and 16,384 patches per pass. Probe with a ≥ 256 px solid-colour PNG and a ≥ 256 token budget — at 64 tokens you get empty content and the colour only inside reasoning_content.
• Capacity behaviour: a prompt over --max-model-len is refused, never truncated; a request longer than the KV pool's token count is accepted then refused as too large to fit; a request beyond --max-num-seqs is held by the HTTP port (queue_cap = max(8, 8 × slots) — 32 here) and answered 503 only past that, never paged or squeezed. The cache funds 3.7 full-height sessions, so four conversations can be resident but a fourth near-full-height one can still preempt a running sequence from the back to free its blocks (core/sched/scheduler.cpp:1291-1322); the victim is recomputed, so its session halts and then re-prefills. Meter: vllm:num_preemptions_total (cumulative for the boot, /stats preemptions); radiance:num_requests_preempted is only the instant depth.
• Observability: /metrics carries vLLM-compatible names plus radiance:*, scraped by the Dankclaw Prometheus job at 192.168.0.235:7770 (15 s interval) over plain LAN routing (the netavark published-port quirk is same-host only). Latency lives under vllm:, cadence under radiance: — TTFT / inter-token / per-output-token / e2e / decode-time histograms and the running/waiting gauges are vllm:* with no radiance-native equivalent, so a radiance:-only query misses every latency answer. The step-phase counters are all scraped: engine_steps{,_decode,_mixed,_prefill,_held}_total, engine_step_seconds{,_decode}_total, engine_prefill_chunks_capped_total, prefill_tokens_total, decode_tokens_per_second, prefill_tokens_per_second, prefix_hit_cut_tokens_total, http_queue_depth. Two holes: the engine's own step-detail histograms (prefill_steps_duration, prefill_step_tokens, token_queue_wait) are exported but reach no target, and the arena / expert slab / heat engine / prefill stager have no metric at all — that half of every law above is read from the boot ledger, /stats and /diagnostics, never from a graph. The boot ledger — the referee for every shape number here — is in the container log (podman logs frank-archon); /home/dank/logs/frank_archon.log is where the serve script redirects podman run.

Files

• Shape and law owner: scripts/frank-archon.env (measured arithmetic in its header) scripts/serve-frank-archon.sh (gates image / .rad size / template / GPU nodes, retires stale containers, podman wait), scripts/frank-archon.service, scripts/ttm-preflight.sh.
• Payload doors: scripts/download-frank-archon-iq4r.sh (live), scripts/download-frank-archon.sh
  • scripts/build-frank-archon-rad.sh + scripts/stack-qwen38-official-fp8.py (the fp8-native rebuild), scripts/download-frank-archon-swift15.sh (the BF16 source a w4dn8 re-convert needs), scripts/hfget.sh / scripts/hf-fetch.sh; template scripts/qwen38-froggeric-v22.5.jinja.
• Convert campaign (dormant, kept for a future quant): scripts/convert-swift15-w4dn8.sh scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe, scripts/run-swift15-bf16.sh scripts/kld-gate-swift15.sh, scripts/kld-then-calib.sh, scripts/calib-push.py scripts/make-calib-corpus.py, scripts/derive-protected-experts.py scripts/apply-protected-cast.sh, scripts/verify-swift15-ple.py.
• Probes: scripts/probe-frank-archon-kv.py (reads slots/window/dtype from the deployed env so it cannot drift; leg D = decode), scripts/frank-archon-expert-activity.py scripts/autoresearch-frank-archon.sh + scripts/autoresearch/frank-archon-measure.py.
• Deploy and tests: scripts/deploy.sh / scripts/deploy-remote.sh (deploy_llm_archon) tests/frank_archon_unit.bats (the seams gating the KV-pool arithmetic, the VRAM pair and the ttm gate), tests/changed_tests.bats, tests/deployment_scoping.bats.
• Retired-era leftovers in-tree: scripts/patches/glm53-*.patch scripts/patches/gpu_worker-kv-cap.patch (fork archaeology; none applies byte-exact today).

Related: frank-dragoon (the abliterated family), frank-zealot,