--- type: reference title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.8.1, publisher Qwen3.8-Flash-Next fp8-iq4r-moe .rad, 2× R9700 TP=2 (ACTIVE)" description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught." tags: - llm - frank-archon - dankserver - radiance - gfx1201 - r9700 - tp2 - expert-offload - endpoint timestamp: 2026-10-19 --- # frank-archon — radiance (StillDeadcode), 2× R9700 TP=2, Dankserver :7770 The radiance engine (C++/HIP, `libr4d` kernels built for gfx1201, no Python in the serving path) driving `podman run` straight from the digest-pinned image under a **root** systemd unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI PRO R9700s. Capacity contract **2 conversations × 200,000 tokens at fp8 KV** — at 1.8.1 that pair is the **demand signal** the budget resolver sizes the cache against, not a KV allocation (Laws). A payload swap is a `MODEL_FILE` line plus a re-download — see Eras. ## Current shape Authoritative source: `scripts/frank-archon.env` + `scripts/serve-frank-archon.sh` (the env header carries the measured arithmetic beside each number). **Payload** — `/home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad`, 121,969,901,568 B = **116,322 MiB**, sha256 `0af5e962…` (the Hub LFS oid), from `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`: the engine author's own rad-convert of the **official (censored)** `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint. Encodings (recipe `qwen4exp-w4nl64-i8-hc8m`): routed experts as 4-bit codes in a 16-level non-uniform codebook (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scale per 64 weights); **10 whole bf16-protected experts per layer**; int8 W8A8 trunk + lm_head; fp8 HC mixtures; PLE n-gram fp8 at the checkpoint's fixed scale (47.7 GiB) MTP head and the bf16 27-layer vision tower **in the container**. Card quality vs bf16: mean KL 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %. The abliterated family lives on [[frank-dragoon]]. Fetched by `scripts/download-frank-archon-iq4r.sh --fetch` (hfget resume + declared size + sha256) as the transient unit `iq4r-fetch` (log `/home/dank/logs/iq4r-fetch.log`, `…rc` sentinel); the staging dir `/home/dank/models/StillDeadcode/…` keeps card, LICENSE and recipe as provenance, and only `.rad` files are mounted. **Runtime** — `stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed` = pin **1.8.1** (adopted 2026-10-18). The version oracle is the boot's own `STARTUP REPORT -- radiance 1.8.1` line, because `podman inspect` reports only the digest. The image lives in dank's **rootless** storage and the host needs nothing but the amdgpu driver. Pin the INDEX manifest digest, never the image ID (`69d05724…` is the config digest and sends `podman run` to the network). The serve script's gate is that digest ref itself — `podman image exists name@sha256:…` in one command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike or a newer `latest` cannot satisfy it. 1.2.0 is a rejected serve pin (Laws). **Two CLI breaks ride this pin, and both are boot refusals, never silently-ignored flags:** the `--vram-weights-mib`/`--vram-kv-mib` pair died at 1.3.0 (the resolver derives the split) and `--prefix-cache-dir` / `--prefix-cache-disk-mib` died with the disk tier. A rollback to 1.0.13 (`sha256:4c3f4845…`) is therefore a trio rewrite, not a digest swap — the Eras row carries it, and `tests/frank_archon_unit.bats` asserts the retired knobs are ABSENT from env and argv so a stale line cannot reach a boot. **Trio** — `scripts/frank-archon.env` (every number, with its arithmetic in the header) `scripts/serve-frank-archon.sh`, `scripts/frank-archon.service`. Gates run in order: image digest present → `.rad` present and ≥ `RAD_MIN_MIB` (a truncated convert would otherwise buy a guaranteed long-timeout boot instead of this fast failure) → template present → `/dev/kfd` + `/dev/dri` → `ttm_ensure_cap` (the serve script re-verifies the cap after the unit's hook) → `podman rm -f` over `FRANK_ARCHON_LEGACY_CONTAINERS` (`frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe`) plus its own name, because systemd SIGKILLs conmon on a crash and leaves the name registered, and every one of those identities holds this GPU. Container contract (publisher-vetted, `docs/DOCKER.md`): `--network host`, `--init` (so the engine releases the cards on SIGTERM), `--security-opt seccomp=unconfined` (the default profile refuses io_uring — the PLE table's reads — and the NUMA calls the ROCm runtime makes for host memory), `--ulimit memlock=-1:-1`, `/models` ro + the rad-tune cache at `/opt/radiance/share/radiance/tune` + the template as a single-file mount (there is **no `/data`** any more: it left with the prefix disk tier, and `FRANK_ARCHON_STATE_DIR` is now only the host-side directory that holds `tune/`), then `exec podman wait`: the script's lifetime IS the container's, so a crash is a nonzero `ExecStart` and the restart policy is real. Unit: `After=network-online.target`, `WantedBy=multi-user.target`, `User=dank` `LimitMEMLOCK=infinity`, `StartLimitIntervalSec=0`, `OOMScoreAdjust=-500` (the last process the OOM killer should pick), `Restart=always` **not** `on-failure` — a clean API-server exit after a worker fault reads as success to systemd (the 2026-10-01 vLLM lesson) — and `TimeoutStopSec=180` > the `ExecStop=podman stop -t 120` grace, because systemd's 90 s default cut the stop short and recorded `Failed with result timeout` (measured 2026-10-01 13:27:17). No drain contract is claimed: the OpenAI front end holds no queue, so an in-flight decode is lost on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_period`). | Knob (env name, `FRANK_ARCHON_` dropped) | Live value | |---|---| | TP / wire | `TP=2`, `TP_WIRE=wht6` — libr4d one-shot all-reduce, not RCCL; `min-kb` stays the engine default | | Slots / window | `MAX_SEQS=2`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**, not a KV allocation | | KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) | | VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) | | Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=44928` **per rank** | | PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) | | Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) | | Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache | | Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it | | Step sharing | `BUSY_PREFILL_CHUNK=auto` (a shared chunk reserves a row plus the draft window per waiting decoder) · `PREFILL_INTERLEAVE=0` — held at zero until the decode-vs-prefill buckets ask for it | | TTM gate | `TTM_MIN_MIB=150000`, `TTM_TARGET_MIB=204800` | | Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) | Boot ledger for the live shape (2026-10-19 11:10, pin 1.8.1, `MAX_SEQS=2`, `HOST_POOL_MIB=44928`): the resolver prints `elastic 26.73 GiB` → **experts 23.73 GiB (89 %) / kv cache 2.76 GiB (10 %)**, `vram_weights 27.31 GiB`, `host_pool 44.47 GiB`. `kv_attn` (13 layers, 4-token blocks) pages **26.00 KiB** → 91,301 blocks = **365,204 tokens = 1.8 full-length sessions = 91 % of the 2 × 200,000 worst case** (the resolver never funds 100 %, see Laws); `mover` lends 2,146 slots (2.54 GiB) out of the cache to the expert slab; sampler **9 rows**, template 28,234 B, vision live (≤ 4,194,304 px, 16,384 patches/pass). The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin, the P2P answer per pair) — read it after every resize, and unlike every predecessor there is no per-boot VRAM planner to second-guess. For comparison, the publisher's own suggested run line on 32 GB cards was `--host-pool-mib 12288 --expert-vs-cache-ratio 0.82` with seqs 8 and spec 3 — a 1.0.x-era line, and `--expert-vs-cache-ratio` was removed at 1.2.0, so today it is a capacity floor quoted from the card, not a run line this pin accepts; the shape above is the fleet's measured one. `FRANK_ARCHON_EXTRA_ARGS` is the boot-ladder escape hatch (unset → no extra argv; it is expanded unquoted on purpose, so keep multi-flag values as one string). **Harness contract** (`/home/blank/.omp/agent/models.yml`) — provider `llamacpp (dankserver)` → `http://192.168.0.235:7770/v1`, `api: openai-completions`, `auth: none`, `id: frank-archon` (= `FRANK_ARCHON_SERVED_NAME`; the provider *label* is legacy and renaming it breaks `config.yml` references). `contextWindow: 200000` must equal `MAX_MODEL_LEN` `maxTokens: 16384` (clamp `min(maxTokens, window − estimate − 4096)`; compaction fires at 85 % → 170,000 est × 1.03 + 16,384 ≤ 200,000); `input: [text, image]` `compat.reasoningContentField: reasoning_content`; `reasoningEffortMap` minimal/low→low, medium→medium, high/max→xhigh; `extraBody` temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 / presence_penalty 0 / repetition_penalty 1.0 + `chat_template_kwargs {enable_thinking, preserve_thinking}`, every key inside the engine's known-parameter list (unknowns 400). `compactionModel` routes to `llamacpp (dankclaw-3090)/frank-zealot`. **Deploy / verify** — `./scripts/deploy.sh dankserver --service frank-archon`; launch it detached when the swap restarts the endpoint your own session runs on. Then verify **behaviourally**, not by the green gate: `podman inspect frank-archon --format '{{json .Config.Cmd}}'` and `/v1/models` — `deploy.sh` plans from repo paths and gates on the port, so host-side unit drift is invisible to it. ## Eras One row per retired era on this port. The **Rollback** cell is the load-bearing one: anything marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `StillDeadcode` `rad` and `tpl.jinja`. | Window | Payload | Runtime | Shape | Replaced because | Rollback | |---|---|---|---|---|---| | 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) | | 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) | | 2026-10-05 → 10-08 | `qwen4exp-fp8-native.rad`, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation | radiance 1.0.7 line, same digest | — | native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, **1.93 sessions** — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot) | rebuild-only: `scripts/download-frank-archon.sh` (official `Qwen/Qwen3.8-Flash-Next-FP8`, 131 shards, 172.8 GB, ungated) + `scripts/build-frank-archon-rad.sh` (no-recipe convert under the descriptor law); raw **and** `.rad` were `rm`'d 2026-10-15 (Frank's call, to free the volume for the iq4r download) | | 2026-10-05 (same day) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` safetensors, 131 shards | FreeToken PR #587 head `77a7201` (`rocm714_torch214`), image `localhost/frank-primary-freetoken:pr587-77a7201` | — | TP=2 cannot boot on ROCm: the per-rank `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201, so the second card idled | git tag `frank-archon-freetoken-pr587-20261005` (trio + `scripts/docker/frank-primary-freetoken.Dockerfile`, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account | | 2026-09-29 → 10-05 (interrupted 09-30 → 10-01 by GLM) | same orcarouter FP8 safetensors | tcclaviger vLLM fork, `docker.io/tcclaviger/vllm@sha256:da402966…` | shape G: 3 × 200,704 fp8, KV 3,166,500,000 B/rank → 401,408 tokens, arena 17.77 GiB / 7,762 slots, util 0.982, MNBT 2048, no spec, no prefetch, prefix caching ON | radiance gave real TP=2 (no RCCL) and a 2-3 min boot, and the fork never fixed its long-context class | git tag `frank-archon-vllm-shapeG-20261003` (trio + the 362-line env lore); image still in dank's rootless storage; payload needs a re-fetch. The 2026-10-01 fp8+MTP4 rung is tag `frank-archon-fp8-mtp-20261001` | | 2026-09-30 → 10-01 | `Intel/GLM-5.3-Flash-W4A16-AutoRound` ~170 GiB (sym int4 g128, `auto_gptq`) | same tcclaviger digest; `R4dW4a16MoEMethod` → libr4d grouped W4A16 | — | — | payload (170 GiB) + drafter (2.2 GiB) went to `~/.AGENTTRASH/20261001-glm-purge/` and were emptied; the cutover was reversed by `d91379c`, restoring the `2339fae` trio; `scripts/patches/glm53-*.patch` survive in-tree but none applies byte-exact since the 2026-10-10 rename rewrote their context lines | | 2026-09-27 (same day) | `Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS`, 92.39 GiB + PLE 26.82 GiB (`34fa36f8…`) | same R9V bundle + shim | `MAX_MODEL_LEN=196608`, offload budgets 146.25 / 78.75 GiB, PLE residency `ssd` mmap, tool parser `qwen3_coder`, thinking on the `reasoning` (vLLM) channel | same-day flip back to the reference quant with the MTP4 head, at Frank's direction | `/home/dank/r9v-state/uncensored/` (`setup.json`, `restore-config-20260927.env`) is still on disk; the IQ4_XS payload was deleted in the 2026-10-17 cleanup → re-fetch, point `R9V_STATE_FILE` back, drop the two q8-only keys and the template keys | | ≤ 2026-09 — 4 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete | — | — | — | — | — Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era pages are **not** superseded here: is the locator for that endpoint's whole shape history, including the unresolved end date of the vllm-radiance era (this table opens on 09-26; the era page claims 10-10). ## Laws learned here ### The radiance engine - **KV dtype law.** `--kv-cache-dtype` takes only `bf16|fp8` — ONE dtype for the whole cache, a **radiance** restriction, not stock vLLM's: upstream advertises `auto|fp8|fp8_e5m2|fp8_e4m3` (`docs.vllm.ai` engine-args) and even there the advertised set is not the usable set — several accepted values crash engine init at the attention layer's own dtype check ([vllm-project/vllm#42587](https://github.com/vllm-project/vllm/issues/42587)). So llama.cpp's exact-K / quantized-V pair (what [[frank-zealot]] runs as `--cache-type-k`/ `--cache-type-v`) is unservable here: the rung below bf16 is fp8 on both halves. Measured per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot 2026-10-11): `kv_attn` 13 layers / 4-token blocks pages **26.00 → 52.00 KiB**, while `kv_qsa_bkey` stays **3.25 KiB at both** — the QSA block key is an index, not attention state, and does not follow the flag. So 7.3125 KiB/token at fp8 vs 13.8125 at bf16: **1.89×, not 2×**, precisely because the bkey plane is dtype-fixed. Secondary saving: a prefix-cache entry tracks the attention plane (110.50 KiB at bf16 → 58.50 at fp8, per the `kv tier transfer` lines), so the same 8 GiB host tier parks ~5.8 sessions of 200k at fp8 where it parked ~2.9 at bf16. - **Pool sizing law (the explicit-pair era — at 1.8.1 the cache is derived, so the resolver law below owns it).** `pool_mib ≥ tokens × 7.3125 KiB + 110 × seqs + 448` — the dtype- independent terms are also in the pool: `kv_gdn_state` 54 MiB × 2 copies per sequence (+conv/ tail/ple ≈ 110 MiB a sequence) and the prefix checkpoint region 8 × 55.99 MiB. Deployed: 3 × 200,000 at fp8 = 4,285 + 330 + 448 = **5,063 MiB**, pool 5,120, which predicts (5120 − 330 − 448) × 1024 / 7.3125 = **608,027 tokens** against a 600,000 contract (1.3 % slack) — every MiB above 5,063 was handed to the expert slab instead. At bf16 the same three slots cost 8,093 + 778 = 8,871 and the slab falls to 20,249 (−9 %); four slots at bf16 cost 10,791 + 888 = 11,679 and the slab falls to 17,441 (−33 %); fp8 keeps 22,208 of slab at both slot counts. **fp8 is not a preference, it is what funds the slot count.** Measured backings to sanity- check the formula: bf16 pool 6,912 at seqs 2 backed 462,752 tokens (2026-10-11), the fp8-era pool of 3,543 gave 386,904 (1.93 sessions), and the four-slot fp8 boot of 2026-10-13 19:21 backed 843,276. The ledger's `backs N sequences all at 200000 tokens at once` line is the referee — a pair this tight is read off the boot, never assumed, and the concurrent probe is its regression test. A fourth full-height conversation needs 6,601 MiB of pool: refusing it is Frank's explicit call, and under that pin serving it was a `--vram-kv-mib` change, not a code change. At 1.8.1 the same decision is `MAX_SEQS` — the resolver buys the cache the demand signal implies. - **The hard limit is per request.** The engine treats `kv group tokens >= max-model-len` as a HARD limit — "one request longer than N tokens is accepted by the API and then refused as too large to ever fit — this is a hard limit, not a backing ratio" — so the pool is sized for `seqs × max-model-len`, not for one sequence. Radiance never truncates, at any size. - **One VRAM budget per card** (the arithmetic below is the pair era's; the ceiling survives, the knobs do not): `weights + kv ≤ 30,969 MiB`, read off the engine's own budget table (boot 2026-10-13 at pair 24000/5120): card 32,624 MiB, budgets claimed 29,120, and "*3.42 GiB is neither budgeted nor accounted — the driver's context, the activation arena and `--gpu-headroom-mib` come out of it; anything past those is throughput on the floor*", with the card settling 2.01 GiB free against a 209.19 MiB floor (96 headroom + 113.19 kept for the static region). Reserve above the engine's floor = 2,058 − 209 = 1,849, ceiling 29,120 + 1,849 = 30,969; the deployed pair lands 458 MiB above that floor. The 2026-10-11 bound of 29,120 ("31,856 − 1,710 outside both budgets − keep 1 GiB free") was a hand-picked reserve, not the engine's wall. Under that pin, boot with the explicit `--vram-weights-mib`/`--vram-kv-mib` pair, never `--kv-cache-size-mib`; with the disk tier engaged the engine also forced ~9.6 GiB of weight buffers + arena onto the weights side (~29.6 GiB/card effective). **Neither flag exists at 1.8.1:** the same ceiling is enforced by the resolver, and passing the pair is a boot refusal. - **At 1.8.1 the split belongs to the resolver, and it funds ~90 % of the worst case, never 100 %.** The engine takes `--gpu-headroom-mib`, the cache bound (`--max-model-len` × `--max-num-seqs`) and `--host-pool-mib`, prints `claimable → elastic → experts / kv cache` before allocating anything, and sizes the expert plane from the model's own weight footprint plus slack — not from a number typed by hand. Three boots of evidence: | shape | demand | cache given | backing | |---|---|---|---| | 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % (2.8 sessions) | | 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % | | 2 × 200,000, host pool 44,928 (live) | 400,000 tok | 2.76 GiB = 365,204 tok | 91 % (1.8 sessions) | So `--max-num-seqs` is a **demand signal** for the resolver and an admission cap for the scheduler — never a KV allocation — and no slot count fully funds the worst case. Growing host RAM by less than a slab step (640 MiB) moved nothing: the expert plane's claim decides what is left for KV, so cache is grown through slots × window or the dtype, and experts are grown by not leaving elastic pool idle. `--host-pool-mib` is still load-bearing the other way — it is where the expert rows the cards cannot hold get pinned, and under-sizing it has always been a boot failure (36,864 refused a boot in the fp8 era). The two-slot boot's own behavioural check: three concurrent conversations all answer `200` — the third **queues** (`queue_cap: 16`), it is not refused; the 400 class is a single request over `--max-model-len`, or three concurrent full-height conversations against a 1.8-session cache. - **Idle KV VRAM is the expert cache's spillover — the price is paid twice.** `mover: 2275 slots can be lent out of the KV cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <= 256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 in the pair era) so a bigger KV pool steals weight residency from the slab AND from the lendable cache. Priors on this card/model: stealing 2.14 GiB of weights to buy `MAX_MODEL_LEN 262144` cost 2.5 % decode; the other direction (`--expert-vs-cache-ratio` 0.82 → 0.84, +2.5 GiB of weights) cost 8.9 %. - **A KV-pool cut is a runtime-headroom lever, never an arena lever.** The 6.1e9 → 3.15e9 B/rank pool cut (2.75 GiB/rank) bought **zero** extra expert-cache slots. The arena moves with the reserve, one-for-one (d(reserve) 0.57 GiB = d(arena) 0.57 GiB = **+248 slots**). The shape that pool cut replaced died with `0 bytes free` on a real 995-token request — that is headroom, not slot count. Read `expert cache space … / N slots` out of the boot ledger before trusting any pool arithmetic: the 2026-10-13 read was **14.87 GiB / 6,496 slots** against `GPU KV cache size: 398,658 tokens`, and it is the POOL that caps resident conversations, not the arena. - **MNBT moves the compile RANGE, not the batch width.** The engine sets `num_reqs = min(max_num_batched_tokens, max_num_reqs)`, so the CUDA-graph compile width is **SEQS + 1** for any `MNBT >= SEQS` — 3 at SEQS 2, 4 at SEQS 3, exactly the constant printed in the engine's own message. MNBT only shifts the compiled range ((1,1024) versus (1,2048)), which is why the identical capture failure appears at both rungs; `--enforce-eager` bypasses the path and loses CUDA graphs with it. - **The prefix cache proves itself at depth, not in the ledger.** Two concurrent conversations of 196,919 tokens each answered the marker buried at the head of **their own** prompt with **393,837 tokens resident** — above the fp8 pool's 386,904, so the answer came from a cached prefix and not a re-prefill. Same boot (2026-10-13): decode at 51.5k depth **94.1 cold / 102.3 warm / 117.6 prefix-hit t/s**, draft acceptance **0.7284 / 0.7525**. - **The host pool moves with the slab.** `vram_weights` and `host_pool_mib` serve the per-rank resident set *together*: at the fp8-KV pair (26071/3543) the ledger showed the slab full to 259 KiB of slack, and cutting `vram-weights` by 3.8 GiB to pay for a bf16 pool spills ~3.8 GiB/rank into host RAM — under-sizing the pool there is a **boot failure** (36,864 refused the boot in the fp8 era). The ledger's coverage line, not the arithmetic, arbitrates. - **Slab → prefill: the arms disagree, the cliff does not.** 2026-10-11 exact-KV A/B (slab 22,208 vs the fp8-era 26,071, −14.8 %): prefill −29 %, TTFT at the full window +25 %, decode inside the noise band on 2 of 3 passes, draft acceptance UP 0.661 → 0.740. 2026-10-13 (`probe-frank-archon-kv.py`, env header): A-cold 43.5k 22,208 → 3,397, 24,000 → 3,290, 25,600 → **3,584** t/s, and the 3-way 197k arm's aggregate 3,051 → 3,589 (wall 193.7 → 164.5 s), so +1,600 MiB of slab bought +8.9 % single-stream / +17.6 % concurrent — read then as a SLOPE, the 22,208/24,000 pair having landed inside noise of each other and falsely looked like a plateau. Re-measured at 3 slots after the 2026-10-15 swap the 25,600 arm returned 3,290: the 3,584 did **not** reproduce, so treat 22,208-25,600 as one band until someone re-runs the ladder. What is not disputed is the cliff: 26,071 → 4,379 t/s (the 2026-10-11 control at seqs 3 / pool 3,543), +22 % for +471 MiB — unreachable at 3 slots (26,112 + 5,120 = 31,232 vs 30,969) and free at 2 (pool 3,072 + slab 27,897 = the ceiling exactly). Two points inside noise of each other are not a plateau, and two points are not a curve. - **Decode is nearly window-free; slots are paid in concurrency.** 99.0 t/s at 43.5k vs 93.3 at 197k (−6 % for 4.5× depth), while three streams give 113.9 / 43.3 / 10.9 = 18.4 t/s aggregate — a fused-step lottery, not a fair share. Read it from leg D of `probe-frank-archon-kv.py` with ≥ 128 generated tokens; an 8-token probe measures first-token noise. - **MNBT 2048.** 2560 collapses decode to 2.6-7.7 t/s (MTP resonance); the w4 file tier streams in place only at MNBT ≤ 2048 and the activation arena scales with the chunk (32768 → 33.3 GiB, refused). - **Decode pays for the chunk it shares a step with.** Census of the live engine taken at the 3-slot shape on 1.0.13 (`/stats` + `/metrics`, cumulative over the 2026-10-15 boot): 1,435,585 decode-only steps at **18.7 ms** each against 127,991 prefill+mixed steps at **317 ms** — 60 % of engine wall-clock spent inside 8 % of the steps, because the chunk is `min(remaining, checkpoint interval 2048, MNBT 2048)` = 2048 rows and every decode row in that step waits the whole chunk. Day of Prometheus (120 s step, 721 samples): `decode_tokens_per_second` median **13.3 t/s** in the 56 % of samples where `prefill_tokens_per_second` ≥ 1000, against 106-240 t/s in the light-prefill buckets; bucketed by mean step time it is 128.7 t/s at 0.02-0.05 s steps and 12.0 t/s at 0.15-0.4 s. The stall is **during**, not after — decode in windows 2-14 min after a busy-prefill window sits at the idle baseline (58-64 vs 77-85 t/s, the residual confounded by co-scheduling), so no lingering expert-residency penalty at 2-minute resolution, and `preemptions: 0` rules out preempt-and-re-prefill. Method: [[LLM Performance Diagnosis]] §9. The law is shape-independent; the fix lives at 1.8.1 (next law), and nobody has re-run the census since the migration — so the fix is ordered, **not verified**. - **The knobs for it arrived with 1.8.1, and only one of them is on.** `--busy-prefill-chunk` caps the shared chunk so a step with waiting decoders takes a smaller piece (live: `auto`, the engine picks; the explicit value is 512), and `--prefill-interleave N` inserts N decode-only steps after a chunk step taken beside decoders. `interleave` is deliberately **0** — the env comment holds it there until the decode-vs-prefill buckets ask for it, and those buckets have not been read since the migration. The vendor prices a 1024 chunk at about a fifth off prefill throughput to halve the stall; that is a knob for choosing tomorrow's chunk, not a measurement of this shape. MNBT remains an arm and not an edit: 2560 collapses decode through MTP resonance. Any chunk change re-scores draft acceptance before it is trusted. - **Slots cost KV, rows, and window — never speed.** `--max-num-seqs` carries no radiance constraint: it adds ≈110 MiB of recurrent state per slot (the live `kv_full_backing` arithmetic prices them exactly), declares the fused TP all-reduce over `seqs × (1 + spec)` rows, and `wht6` is bit-exact up to 25 rows (single-sequence decode and any step of ≤ 25 rows is exact; only busier steps are lossy) — 2 × 3 = 6 today. Measured 2026-10-06: seqs 2 and seqs 3 are statistically tied (30.9-33.1 t/s, inside the ±3 % floor). Measured 2026-10-19 at 1.8.1: the third slot's real price is **KV, not rows** — dropping it moved the resolver's cache from 4.20 GiB to 2.76 GiB and the backing from 2.8 to 1.8 sessions, while the ≈4 GiB of host RAM it freed came back unbudgeted, landing in elastic pool that turned into a bigger mover lend instead of cache (resolver law above). The old "seqs 2 cannot boot" law belonged to the retired tcclaviger fork's `torch.compile` shape specialization, and radiance booted seqs 2 on its first fp8-native start. - **A dequantize-to-compute path bills host RAM twice.** On the EMULATION backend an offload request of **118 GiB** landed as **~238 GiB** of real host RAM — the dequantized copy rides on top of the pinned weights, and the doubling is invisible in the weight count. Read the engine log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit"; a page that says a payload "does not fit" without that line has not read the runtime. - **One encoding per expert table** (`rad_arch_declare`): a whole expert kept bf16 is the only thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable. - **Descriptor-selection law.** The plugin is picked by the exact `(arch_id, quant descriptor)` pair with no fallback: a raw official FP8 checkpoint (`quant_method: "fp8"`) is refused by serve *and* convert, because the shipped `qwen4exp_fp8` plugin claims `(qwen4exp, "")`. Stage a `config.json` with `quantization_config` dropped — the descriptor changes, the data stays FP8. - **Three convert laws, each paid in a full verify+declare cycle.** Scale-name: `scale_name()` appends `_scale_inv` to the *tensor* name, so stacked `…experts.gate_up_proj` keeps its scales at `…gate_up_proj_scale_inv`. File-shadow: the reader opens index-named files sorted and emplaces every tensor each file physically holds, first-wins — a merged PLE shard must sort first (`00-ple.merged.safetensors`) or a raw shard wins the name. Symlink-through: staging symlinks the raw index, so `open(path,"w")` writes through it and corrupts the checkpoint — write temp + `os.replace`. - **Convert is card-less** (`rad-convert` declares on CPU, GPTQ solves on CPU BLAS) and `--plan-only` goes first: a plan that resolves every op will boot. `rad-tune` must be declared at the shapes you serve and its cache lives in the *image* (`$RADIANCE_HOME/tune`) so it only survives on the `$FRANK_ARCHON_STATE_DIR/tune` bind-mount the serve script adds. - **Quant gates are scored at the production shape** (MNBT 2048, not the recording's 32768) against the BF16 container: KL ≤ 0.03, top-1 ≥ 95 %, and the verdict is chunk-size invariant (arm Δ ≈ the 0.0015 float-order band) and placement-invariant. Protected experts are data-driven — energy share ≥ 3 % AND ≥ 3× routing share, even count per rank half — and the PLE rides the checkpoint's own fixed scale (1.9931793212890625e-04, absmax inside the 2⁻¹³..2⁻¹⁰ plateau). ### This host, and the deploy path - **The pinned-RAM ledger is why this is a root unit:** 2 × 44 GiB expert pools + the 47.7 GiB PLE n-gram table (51B params, read from pinned host RAM over io_uring — Frank's pin law: "*Native FP8. Experts from ram. Pin the PLE to ram.*") + the 8 GiB prefix host tier ≈ **147 GiB** charged against `RLIMIT_MEMLOCK`; dank's login session caps at 8 MiB and a user quadlet is capped by the user manager, so only PID1 can grant `infinity` — and `--ngram-placement ram` refuses the start when the limit is short. - **TTM is runtime state, and the gate that owns it raises before it refuses.** The pinned pages ride `/sys/module/ttm/parameters/pages_limit` (+ `pages_max` on kernels that expose one; this kernel exposes **none** — the cap is the MIN over the knobs present, and an absent knob is skipped, never read as zero, because a zero would refuse a healthy cap). The kernel default is **half of RAM**, so every raise dies with the reboot: the 2026-10-05 manual raise to 52,428,800 pages (200 GiB) was gone after the **2026-10-12 23:17 power failure**, the cap fell to 125,988 MiB, and the preflight refused the start **6,919 times over ~20 h** while `Restart=always` cheerfully re-ran a refusal. A manual sysfs write is therefore not an acceptable repair procedure — `scripts/ttm-preflight.sh` writes `TTM_TARGET_MIB` (204,800 MiB = 52,428,800 pages; pages are 4 KiB so pages = MiB × 256; the *recorded host ceiling*, because writing the 150,000 floor would leave the ledger no slack) into every knob present, re-reads, and only then refuses. Write failures are swallowed deliberately: the caller re-reads, and ttm itself refuses a limit below the pages it has already pinned. - **Privilege of that knob, measured the hard way:** it is root-owned 0644 and the service runs `User=dank`, where `-w` is false and a plain `echo` fails. The canonical privileged path is the unit's `ExecStartPre=+…` (the `+` runs it as root even under `User=`); `sudo -n` is the fallback for a human. The file is *sourced* by the serve script (which re-verifies the cap before `podman run`) and *executed* by the unit. A refusal is invisible outside the journal — no container is created, the unit sits in `activating (auto-restart)` on one repeating line — and without the gate the load instead dies mid-pin with a bare allocator error. - **The prefix host tier needs real RAM:** 8 GiB of it, and with the service loaded the host still has ~104 GiB available (246 GiB total) — that headroom is what the pinned ledger is measured against. - **The deployer's blind spot is host-side unit drift.** A leftover `frank-archon.service.d/bench.conf` overrode both `FRANK_ARCHON_ENV_FILE` and `ExecStart` to a staging worktree and every `deploy.sh` since was green against the old configuration — drop-ins are read *after* the unit's own `Environment=` lines, so they win. - **Never `systemctl disable` a unit on these hosts** — it removes the `/etc/systemd/system/` entry, which here is the symlink into the checkout. `mask` is reversible but *fails on a relinked unit*, so a bench window uses `stop` and the deploy restores the unit. - **The restore oracle for an experiment window is a conjunction:** `is-active` == active AND `is-enabled` == enabled AND no `/run/systemd/system/` mask symlink AND the container is owned by the unit and not `Stopping`. The vLLM/GLM-era engines ignored SIGTERM and answered `/health` 200 for 20+ minutes while `Stopping`; a runtime mask survives `stop` and blocks `start`, and a failed unit needs `reset-failed` first. - **`MemAvailable` is an input, not a backdrop:** the expert-offload planner reads it, so anything booted beside the live service dies at plan time (`ExpertPlanError: host holds 5113 of 12096 swappable experts`) — and fleet law forbids GPU experiments beside production anyway. Long remote work runs as a transient unit or `setsid`+disown: `nohup X &` inside `ssh` races ssh teardown and can die before writing a byte. - **The ~40k-token corruption class was the RAM, not the engine.** Dropping DDR5 from 4800 to 4600 MT/s ended it, with zero `gdn_hip` faults either side; the prefix-caching / GDN re-seed theory was confounded on both arms. Consumer non-ECC raises no MCE for a silent flip and a 60 s / 8 GiB pattern check covers 0.5 % of 246 GiB — silent RAM corruption outranks every software theory. - **TP=2 here** is legal since the 2026-09-20 ProArt B850 swap (x8/x8 on CPU root ports, `Routing+`/`ReqEn+` end to end) only because radiance's wire is `libr4d`'s own one-shot all-reduce; RCCL needs atomics on the whole chain and fails `hipErrorIllegalState` without them (). `wht6` buys +5.6-6.3 % prefill for ~1.3 % of 8-stream decode, `exact` is the bit-exact alternative. - `--prefix-cache-dir` creation is **not recursive** — the serve script `mkdir -p`s the path or the disk tier silently never opens. The tier does survive a payload swap when arch and tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; `cache_n` proves a hit). ### Dead ends — do not re-try - **radiance 1.2.0 as the serve pin** (`sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa` commit `e95d397`), benched 2026-10-06 on this exact shape with a 43.6k greedy probe: decode 29.21 t/s and draft acceptance 0.7123 vs the 1.0.13 control's 30.08 / 0.7549, and the control reproduced the recorded 0.755 exactly — greedy acceptance is deterministic, so the drop is a real numerics change (the decode loss tracks the acceptance ratio, 0.943 vs 0.945) with the 1.2.0 MoE weight-rotation change `92b8aaf` the named suspect. The vram-pair translation itself is exact (the 1.2.0 ledger took 25.46 GiB weights / 3.46 GiB KV per card, byte-identical to 1.0.13 at EVC 0.82), and 1.2.0 REMOVED `--expert-vs-cache-ratio` in favour of the pair, which both pins accept. Re-test 1.2.0+ once the rotation change is untangled. - **FreeToken TP=2 on ROCm:** the `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201 — an untested edge of that PR, not a board fault. - **tcclaviger forks:** `--max-num-seqs 2` (`ConstraintViolationError` on `query_start_loc`) `--gpu-memory-utilization 1.00` unbootable, 0.982 is the law; expert prefetch is poison (36 % precision — deleting it was +40-51 % decode); TunableOp-with-offload corrupts numerics (repetition loops, early EOS) — autotuning that changes text is broken tuning; ngram-only drafting dies at init (`PLE inputs were not prepared`); DCP needs TP > `num_kv_heads` (2 here, so 4 GPUs); 6 × 262,144 fits no KV dtype (6 sequences leave 9.18 GiB of pool where fp8 needs 12.47). fp8 KV on that path also removes the only MLA decode gfx12 has (the aiter op is gated on CDNA > 2, so the draft step demands `…mla_decode_fwd`, which cannot exist there) and its KV reserve model is dtype-blind (identical 13,600 B/token plan line) — no capacity for the breakage. - **DeepSeek-V4.1-Flash on 32 GiB cards:** only `--ds4-prefill exact` is legal, `--ds4-fused-decode` is refused, and at ~22 % resident experts each token pulls ~187 × 12 MiB from NVMe — low GPU util with a pegged link is a *capacity verdict*. Two corrections survive: expert TP **does** work on gfx1201+gfx1201, and an expert-placement JSON outside the mounted `/models` tree gives a silent single-tier fallback. - **Which experts you pin does not matter; how many does.** An IBS census of the live server (`scripts/frank-archon-expert-activity.py`) measured per-block read volume flat at 0.93-1.08× of mean across every CPU-resident block — top-k routing fires all of them every token, so at `-ot` granularity the lever is COUNT. - **Never mount a diagnostic hook on `PYTHONPATH` without a marker guard:** `sitecustomize` is imported by every interpreter, a child inherits the hook and spawns again — a 19-level python fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation) never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions. ## Contract (the wire) - **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions` `/v1/completions`, `/metrics`, `/stats`, `/sessions`, `/dashboard`. `/health` answers 200 JSON (model id, `in_flight`, `queue_depth`) while serving and **503 only while stopping** (`core/server/admin.cpp:385`), and the listener binds *after* the full load (`main.cpp`: `engine.start()` then `engine.serve()`) — during the load the port refuses connections, so a bare `curl -fsS …/health` gate is honest and needs no body grep. - **Served id.** `FRANK_ARCHON_SERVED_NAME=frank-archon` (the wire rename of 2026-10-09 moved only the alias; container, unit, scripts and env keep the service name). The env's contract line is that **radiance validates the requested model name, so every client — harness, bench scripts, probes — MUST send it** — while the era pages also record that a chat response *echoes* the requested name, which is why `/v1/models` is the served-id oracle and a 200 chat probe proves nothing about what is actually loaded. - **Reasoning** lands in `reasoning_content` (`--reasoning-format` defaults to `auto`; no parser flags exist or are needed) and `usage` bills it separately (`reasoning_tokens`). **Thinking consumes `max_tokens`**, so the classic false negative is a probe with empty `content` and `finish_reason: length`/`stop` because the whole budget went inside the think block: either set `chat_template_kwargs.enable_thinking=false` or budget a few hundred tokens and read `reasoning_content`. Same trap, sharper, on the vision probe below. - **Sampler surface:** `temperature, top_p, top_k, min_p, typical_p, presence_penalty, frequency_penalty, repetition_penalty, repeat_last_n, chat_template_kwargs, reasoning_effort, enable_thinking, preserve_thinking` accepted; **unknown parameters are refused 400** (nothing unverified can ride the wire, and an `extraBody` typo fails loudly); `max_tokens` is clamped not refused; `response_format: json_object` honoured; **`logprobs` refused**; no `seed` = the engine picks; `n` defaults to `max(8, max-num-seqs)`; stop strings ≤ 64 × 4096 B. - **Tools:** the qwen3.8 dialect from the chat template → `message.tool_calls` + `finish_reason: tool_calls`. **`reasoning_effort`:** the froggeric template does not gate the set (xhigh/high/medium/low all 200); the harness map is a safety net, not a requirement. - **Vision is native** (the checkpoint's 27-layer Qwen-VL tower through the engine's Qwen-VL processor): an `image_url` content part, no legacy top-level `images` field, ≤ 4,194,304 px and 16,384 patches per pass. Probe with a ≥ 256 px solid-colour PNG and a ≥ 256 token budget — at 64 tokens you get empty `content` and the colour only inside `reasoning_content`. - **Capacity behaviour:** a prompt over `--max-model-len` is refused, never truncated; a request longer than the KV pool's token count is accepted then refused as too large to fit; under the 3-slot contract a **fourth** full-height conversation is refused, not paged or squeezed. - **Observability:** `/metrics` carries vLLM-compatible names plus `radiance:*`, scraped by the Dankclaw Prometheus job at `192.168.0.235:7770` over plain LAN routing (the netavark published-port quirk is same-host only). The boot ledger — the referee for every shape number above — is in the container log (`podman logs frank-archon`) `/home/dank/logs/frank_archon.log` is where the serve script redirects `podman run`. ## Files - Shape and law owner: `scripts/frank-archon.env` (measured arithmetic in its header) `scripts/serve-frank-archon.sh` (gates image / `.rad` size / template / GPU nodes, retires stale containers, `podman wait`), `scripts/frank-archon.service`, `scripts/ttm-preflight.sh`. - Payload doors: `scripts/download-frank-archon-iq4r.sh` (live), `scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh` + `scripts/stack-qwen38-official-fp8.py` (the fp8-native rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template `scripts/qwen38-froggeric-v22.5.jinja`. - Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh` `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh` `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py` `scripts/make-calib-corpus.py`, `scripts/derive-protected-experts.py` `scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`. - Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py` `scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`. - Deploy and tests: `scripts/deploy.sh` / `scripts/deploy-remote.sh` (`deploy_llm_archon`) `tests/frank_archon_unit.bats` (the seams gating the KV-pool arithmetic, the VRAM pair and the ttm gate), `tests/changed_tests.bats`, `tests/deployment_scoping.bats`. - Retired-era leftovers in-tree: `scripts/patches/glm53-*.patch` `scripts/patches/gpu_worker-kv-cap.patch` (fork archaeology; none applies byte-exact today). Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]], ---