---
type: reference
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.8.1, publisher Qwen3.8-Flash-Next fp8-iq4r-moe .rad, 2× R9700 TP=2 (ACTIVE)"
description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught."
tags:
  - llm
  - frank-archon
  - dankserver
  - radiance
  - gfx1201
  - r9700
  - tp2
  - expert-offload
  - endpoint
timestamp: 2026-10-19
---

# frank-archon — radiance (StillDeadcode), 2× R9700 TP=2, Dankserver :7770

The radiance engine (C++/HIP, `libr4d` kernels built for gfx1201, no Python in the serving
path) driving `podman run` straight from the digest-pinned image under a **root** systemd
unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI
PRO R9700s. Capacity contract **2 conversations × 200,000 tokens at fp8 KV** — at 1.8.1 that pair
is the **demand signal** the budget resolver sizes the cache against, not a KV allocation (Laws).
A payload swap is a `MODEL_FILE` line plus a re-download — see Eras.

## Current shape

Authoritative source: `scripts/frank-archon.env` + `scripts/serve-frank-archon.sh` (the env
header carries the measured arithmetic beside each number).

**Payload** — `/home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad`, 121,969,901,568 B
= **116,322 MiB**, sha256 `0af5e962…` (the Hub LFS oid), from
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`: the engine author's own rad-convert of the
**official (censored)** `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint. Encodings (recipe
`qwen4exp-w4nl64-i8-hc8m`): routed experts as 4-bit codes in a 16-level non-uniform codebook
(128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent
sweeps, E4M3 scale per 64 weights); **10 whole bf16-protected experts per layer**; int8 W8A8
trunk + lm_head; fp8 HC mixtures; PLE n-gram fp8 at the checkpoint's fixed scale (47.7 GiB)
MTP head and the bf16 27-layer vision tower **in the container**. Card quality vs bf16: mean
KL 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %. The abliterated family lives on
[[frank-dragoon]]. Fetched by `scripts/download-frank-archon-iq4r.sh --fetch` (hfget resume +
declared size + sha256) as the transient unit `iq4r-fetch` (log
`/home/dank/logs/iq4r-fetch.log`, `…rc` sentinel); the staging dir
`/home/dank/models/StillDeadcode/…` keeps card, LICENSE and recipe as provenance, and only
`.rad` files are mounted.

**Runtime** — `stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed`
= pin **1.8.1** (adopted 2026-10-18). The version oracle is the boot's own `STARTUP REPORT --
radiance 1.8.1` line, because `podman inspect` reports only the digest. The image lives in dank's
**rootless** storage and the host needs nothing but the amdgpu driver. Pin the INDEX manifest
digest, never the image ID (`69d05724…` is the config digest and sends `podman run` to the
network). The serve script's gate is that digest ref itself — `podman image exists name@sha256:…`
in one command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike or a newer
`latest` cannot satisfy it. 1.2.0 is a rejected serve pin (Laws). **Two CLI breaks ride this pin,
and both are boot refusals, never silently-ignored flags:** the `--vram-weights-mib`/`--vram-kv-mib`
pair died at 1.3.0 (the resolver derives the split) and `--prefix-cache-dir` /
`--prefix-cache-disk-mib` died with the disk tier. A rollback to 1.0.13 (`sha256:4c3f4845…`) is
therefore a trio rewrite, not a digest swap — the Eras row carries it, and
`tests/frank_archon_unit.bats` asserts the retired knobs are ABSENT from env and argv so a stale
line cannot reach a boot.

**Trio** — `scripts/frank-archon.env` (every number, with its arithmetic in the header)
`scripts/serve-frank-archon.sh`, `scripts/frank-archon.service`. Gates run in order: image
digest present → `.rad` present and ≥ `RAD_MIN_MIB` (a truncated convert would otherwise buy a
guaranteed long-timeout boot instead of this fast failure) → template present → `/dev/kfd` +
`/dev/dri` → `ttm_ensure_cap` (the serve script re-verifies the cap after the unit's hook) →
`podman rm -f` over `FRANK_ARCHON_LEGACY_CONTAINERS` (`frank-archon-qwen frank-archon-r9v
qwen-fp8 glm-probe`) plus its own name, because systemd SIGKILLs conmon on a crash and leaves
the name registered, and every one of those identities holds this GPU. Container contract
(publisher-vetted, `docs/DOCKER.md`): `--network host`, `--init` (so the engine releases the
cards on SIGTERM), `--security-opt seccomp=unconfined` (the default profile refuses io_uring —
the PLE table's reads — and the NUMA calls the ROCm runtime makes for host memory), `--ulimit
memlock=-1:-1`, `/models` ro + the rad-tune cache at
`/opt/radiance/share/radiance/tune` + the template as a single-file mount (there is **no `/data`**
any more: it left with the prefix disk tier, and `FRANK_ARCHON_STATE_DIR` is now only the host-side
directory that holds `tune/`), then `exec podman wait`: the script's lifetime IS the
container's, so a crash is a nonzero `ExecStart` and the restart policy is real.
Unit: `After=network-online.target`, `WantedBy=multi-user.target`, `User=dank`
`LimitMEMLOCK=infinity`, `StartLimitIntervalSec=0`, `OOMScoreAdjust=-500` (the last process the
OOM killer should pick), `Restart=always` **not** `on-failure` — a clean API-server exit after a
worker fault reads as success to systemd (the 2026-10-01 vLLM lesson) — and
`TimeoutStopSec=180` > the `ExecStop=podman stop -t 120` grace, because systemd's 90 s default
cut the stop short and recorded `Failed with result timeout` (measured 2026-10-01 13:27:17). No
drain contract is claimed: the OpenAI front end holds no queue, so an in-flight decode is lost
on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_period`).

| Knob (env name, `FRANK_ARCHON_` dropped) | Live value |
|---|---|
| TP / wire | `TP=2`, `TP_WIRE=wht6` — libr4d one-shot all-reduce, not RCCL; `min-kb` stays the engine default |
| Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
| VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=49152` **per rank** — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
| Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
| Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
| Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
| TTM gate | `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |

Boot ledger for the live shape (2026-10-20, pin 1.8.1, `MAX_SEQS=4`, `HOST_POOL_MIB=49152`):
the resolver prints `elastic 26.73 GiB` → **experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**,
`prefill staging 373.65 MiB` out of the cache, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`.
The referee line: `kv: the cache can grow to 5.64 GiB a card — 747,492 tokens of context, 3.7
sessions at 200,000 — with every block it lends the experts taken back; 440.04 MiB of that is
per-sequence state, and up to 5.20 GiB is lent to the experts while no request holds it`.
`--checkpoint-slots 40` is auto (`max_num_seqs × 10` slots of 111.98 MiB = 4,479 MiB of pinned
host), `/health` reports `queue_cap 32` (`max(8, seqs × 8)`), sampler **12 rows**
(= `seqs × (1 + speculative depth)`), 4 hardware queues a card, template 28,234 B, vision live
(≤ 4,194,304 px, 16,384 patches/pass), P2P confirmed (`the 2 cards reach each other's memory
directly`). Experts living in the pinned host pool: **8.99 GiB a rank** (was 5.99) — the price of
the bigger cache, paid over PCIe when those experts are hit.

The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
the P2P answer per pair) — read it after every resize, and unlike every predecessor there is no
per-boot VRAM planner to second-guess. For comparison, the publisher's own suggested run line on 32
GB cards was `--host-pool-mib 12288 --expert-vs-cache-ratio 0.82` with seqs 8 and spec 3 — a
1.0.x-era line, and `--expert-vs-cache-ratio` was removed at 1.2.0, so today it is a capacity floor
quoted from the card, not a run line this pin accepts; the
shape above is the fleet's measured one. `FRANK_ARCHON_EXTRA_ARGS` is the boot-ladder escape
hatch (unset → no extra argv; it is expanded unquoted on purpose, so keep multi-flag values as
one string).

**Harness contract** (`/home/blank/.omp/agent/models.yml`) — provider `llamacpp (dankserver)`
→ `http://192.168.0.235:7770/v1`, `api: openai-completions`, `auth: none`, `id: frank-archon`
(= `FRANK_ARCHON_SERVED_NAME`; the provider *label* is legacy and renaming it breaks
`config.yml` references). `contextWindow: 200000` must equal `MAX_MODEL_LEN`
`maxTokens: 16384` (clamp `min(maxTokens, window − estimate − 4096)`; compaction fires at
85 % → 170,000 est × 1.03 + 16,384 ≤ 200,000); `input: [text, image]`
`compat.reasoningContentField: reasoning_content`; `reasoningEffortMap` minimal/low→low,
medium→medium, high/max→xhigh; `extraBody` temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
presence_penalty 0 / repetition_penalty 1.0 + `chat_template_kwargs {enable_thinking,
preserve_thinking}`, every key inside the engine's known-parameter list (unknowns 400).
`compactionModel` routes to `llamacpp (dankclaw-3090)/frank-zealot`.

**Deploy / verify** — `./scripts/deploy.sh dankserver --service frank-archon`; launch it
detached when the swap restarts the endpoint your own session runs on. Then verify
**behaviourally**, not by the green gate: `podman inspect frank-archon --format '{{json
.Config.Cmd}}'` and `/v1/models` — `deploy.sh` plans from repo paths and gates on the port, so
host-side unit drift is invisible to it.

## Eras

One row per retired era on this port. The **Rollback** cell is the load-bearing one: anything
marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `StillDeadcode`
`rad` and `tpl.jinja`.

| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---|
| 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
| 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
| 2026-10-05 → 10-08 | `qwen4exp-fp8-native.rad`, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation | radiance 1.0.7 line, same digest | — | native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, **1.93 sessions** — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot) | rebuild-only: `scripts/download-frank-archon.sh` (official `Qwen/Qwen3.8-Flash-Next-FP8`, 131 shards, 172.8 GB, ungated) + `scripts/build-frank-archon-rad.sh` (no-recipe convert under the descriptor law); raw **and** `.rad` were `rm`'d 2026-10-15 (Frank's call, to free the volume for the iq4r download) |
| 2026-10-05 (same day) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` safetensors, 131 shards | FreeToken PR #587 head `77a7201` (`rocm714_torch214`), image `localhost/frank-primary-freetoken:pr587-77a7201` | — | TP=2 cannot boot on ROCm: the per-rank `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201, so the second card idled | git tag `frank-archon-freetoken-pr587-20261005` (trio + `scripts/docker/frank-primary-freetoken.Dockerfile`, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account |
| 2026-09-29 → 10-05 (interrupted 09-30 → 10-01 by GLM) | same orcarouter FP8 safetensors | tcclaviger vLLM fork, `docker.io/tcclaviger/vllm@sha256:da402966…` | shape G: 3 × 200,704 fp8, KV 3,166,500,000 B/rank → 401,408 tokens, arena 17.77 GiB / 7,762 slots, util 0.982, MNBT 2048, no spec, no prefetch, prefix caching ON | radiance gave real TP=2 (no RCCL) and a 2-3 min boot, and the fork never fixed its long-context class | git tag `frank-archon-vllm-shapeG-20261003` (trio + the 362-line env lore); image still in dank's rootless storage; payload needs a re-fetch. The 2026-10-01 fp8+MTP4 rung is tag `frank-archon-fp8-mtp-20261001` |
| 2026-09-30 → 10-01 | `Intel/GLM-5.3-Flash-W4A16-AutoRound` ~170 GiB (sym int4 g128, `auto_gptq`) | same tcclaviger digest; `R4dW4a16MoEMethod` → libr4d grouped W4A16 | — | — | payload (170 GiB) + drafter (2.2 GiB) went to `~/.AGENTTRASH/20261001-glm-purge/` and were emptied; the cutover was reversed by `d91379c`, restoring the `2339fae` trio; `scripts/patches/glm53-*.patch` survive in-tree but none applies byte-exact since the 2026-10-10 rename rewrote their context lines |
| 2026-09-27 (same day) | `Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS`, 92.39 GiB + PLE 26.82 GiB (`34fa36f8…`) | same R9V bundle + shim | `MAX_MODEL_LEN=196608`, offload budgets 146.25 / 78.75 GiB, PLE residency `ssd` mmap, tool parser `qwen3_coder`, thinking on the `reasoning` (vLLM) channel | same-day flip back to the reference quant with the MTP4 head, at Frank's direction | `/home/dank/r9v-state/uncensored/` (`setup.json`, `restore-config-20260927.env`) is still on disk; the IQ4_XS payload was deleted in the 2026-10-17 cleanup → re-fetch, point `R9V_STATE_FILE` back, drop the two q8-only keys and the template keys |
| ≤ 2026-09 — 4 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete | — | — | — | — | —

Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era
pages are **not** superseded here:  is the locator for that
endpoint's whole shape history, including the unresolved end date of the
vllm-radiance era (this table opens on 09-26; the era page claims 10-10).

## Laws learned here

### The radiance engine

- **KV dtype law.** `--kv-cache-dtype` takes only `bf16|fp8` — ONE dtype for the whole cache,
  a **radiance** restriction, not stock vLLM's: upstream advertises `auto|fp8|fp8_e5m2|fp8_e4m3`
  (`docs.vllm.ai` engine-args) and even there the advertised set is not the usable set — several
  accepted values crash engine init at the attention layer's own dtype check
  ([vllm-project/vllm#42587](https://github.com/vllm-project/vllm/issues/42587)).
  So llama.cpp's exact-K / quantized-V pair (what [[frank-zealot]] runs as `--cache-type-k`/
  `--cache-type-v`) is unservable here: the rung below bf16 is fp8 on both halves. Measured
  per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot
  2026-10-11): `kv_attn` 13 layers / 4-token blocks pages **26.00 → 52.00 KiB**, while
  `kv_qsa_bkey` stays **3.25 KiB at both** — the QSA block key is an index, not attention state,
  and does not follow the flag. So 7.3125 KiB/token at fp8 vs 13.8125 at bf16: **1.89×, not
  2×**, precisely because the bkey plane is dtype-fixed. Secondary saving: a prefix-cache entry
  tracks the attention plane (110.50 KiB at bf16 → 58.50 at fp8, per the `kv tier transfer`
  lines), so the same 8 GiB host tier parks ~5.8 sessions of 200k at fp8 where it parked ~2.9 at bf16.
- **Pool sizing law (the explicit-pair era — at 1.8.1 the cache is derived, so the resolver law below
  owns it).** `pool_mib ≥ tokens × 7.3125 KiB + 110 × seqs + 448` — the dtype-
  independent terms are also in the pool: `kv_gdn_state` 54 MiB × 2 copies per sequence (+conv/
  tail/ple ≈ 110 MiB a sequence) and the prefix checkpoint region 8 × 55.99 MiB. Deployed: 3 ×
  200,000 at fp8 = 4,285 + 330 + 448 = **5,063 MiB**, pool 5,120, which predicts (5120 − 330 −
  448) × 1024 / 7.3125 = **608,027 tokens** against a 600,000 contract (1.3 % slack) — every
  MiB above 5,063 was handed to the expert slab instead. At bf16 the same three slots cost
  8,093 + 778 = 8,871 and the slab falls to 20,249 (−9 %); four slots at bf16 cost 10,791 + 888
  = 11,679 and the slab falls to 17,441 (−33 %); fp8 keeps 22,208 of slab at both slot counts.
  **fp8 is not a preference, it is what funds the slot count.** Measured backings to sanity-
  check the formula: bf16 pool 6,912 at seqs 2 backed 462,752 tokens (2026-10-11), the fp8-era
  pool of 3,543 gave 386,904 (1.93 sessions), and the four-slot fp8 boot of 2026-10-13 19:21
  backed 843,276. The ledger's `backs N sequences all at 200000 tokens at once` line is the
  referee — a pair this tight is read off the boot, never assumed, and the concurrent probe is
  its regression test. A fourth full-height conversation needs 6,601 MiB of pool: refusing it is
  Frank's explicit call, and under that pin serving it was a `--vram-kv-mib` change, not a code change.
  At 1.8.1 the same decision is `MAX_SEQS` — the resolver buys the cache the demand signal implies.
- **The hard limit is per request.** The engine treats `kv group tokens >= max-model-len` as a
  HARD limit — "one request longer than N tokens is accepted by the API and then refused as too
  large to ever fit — this is a hard limit, not a backing ratio" — so the pool is sized for
  `seqs × max-model-len`, not for one sequence. Radiance never truncates, at any size.
- **One VRAM budget per card** (the arithmetic below is the pair era's; the ceiling survives, the knobs
  do not): `weights + kv ≤ 30,969 MiB`, read off the engine's own budget
  table (boot 2026-10-13 at pair 24000/5120): card 32,624 MiB, budgets claimed 29,120, and
  "*3.42 GiB is neither budgeted nor accounted — the driver's context, the activation arena and
  `--gpu-headroom-mib` come out of it; anything past those is throughput on the floor*", with
  the card settling 2.01 GiB free against a 209.19 MiB floor (96 headroom + 113.19 kept for the
  static region). Reserve above the engine's floor = 2,058 − 209 = 1,849, ceiling 29,120 + 1,849
  = 30,969; the deployed pair lands 458 MiB above that floor. The 2026-10-11 bound of 29,120
  ("31,856 − 1,710 outside both budgets − keep 1 GiB free") was a hand-picked reserve, not the
  engine's wall. Under that pin, boot with the explicit `--vram-weights-mib`/`--vram-kv-mib` pair,
  never `--kv-cache-size-mib`; with the disk tier engaged the engine also forced ~9.6 GiB of weight
  buffers + arena onto the weights side (~29.6 GiB/card effective). **Neither flag exists at 1.8.1:**
  the same ceiling is enforced by the resolver, and passing the pair is a boot refusal.
- **At 1.8.1 the split belongs to the resolver, and the source fixes its order.**
  `core/mem/vram_budget.cpp` claims the card bottom-up (`capacity → already held → drift → arena →
  staging → queues → prefetch → headroom = claimable`), takes the **static weights off the top as a
  floor, not a share**, and divides the `elastic` remainder in three lines (146-156):

  ```cpp
  exp = std::min(b.expert_floor, b.elastic);
  kv  = std::min(kv_cap, std::max(b.elastic - exp, kv_one));
  exp = std::min(std::max(b.expert_ceiling, b.expert_floor), b.elastic - kv);
  ```

  `kv_cap` is `KVManager::ceiling()` = `state + Σ_paged max_seqs × ceil(max_ctx / block) × bytes`
  (`core/mem/kv.cpp:266`), so `--max-num-seqs` and `--max-model-len` set the cache's **demand**, never
  its funding. The funding is `elastic − expert_floor`, and the floor is
  `min(plane + spare, plane + spare + 2·unit_max − host_room)` (`core/place/planner.cpp:100-104`, with
  `host_room = host_pool − file_stage_reserve` at `core/engine_bringup.cpp:1418-1422`). **That is the
  cache's lever: every MiB added to `--host-pool-mib` drops the expert floor one-for-one and the cache
  takes it** — minus `stage`, the prefill stager's ring (two layers of the experts left off the card),
  which is charged back out of the cache whenever the cache lends (`vram_budget.cpp:193-203`). Boots of
  evidence, all on 1.8.1:

  | shape | demand | cache given | engine's `backs %` |
  |---|---|---|---|
  | 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % |
  | 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % |
  | 2 × 200,000, host pool 44,928 | 400,000 tok | 2.76 GiB | 91 % |
  | 4 × 200,000, host pool 49,152 (live) | 800,000 tok | **5.64 GiB = 747,492 tok** | 93 % |

  The 45,568 row is the ceiling case, not a mystery: at 3 slots the cache already held its whole
  `ceiling`, so extra host room could not grow it and shows up as `unclaimed`. **The referee is the
  ledger line `kv: the pool backs N % … the cache can grow to … N tokens`, never arithmetic.** To buy
  window: raise `--host-pool-mib` (cost: pinned host RAM, and experts evicted to host cross PCIe —
  `experts-from-host` was 5.99 GiB a rank at 44,928 and is 8.99 at 49,152) or cut the per-token cost
  (`fp8` KV). Raising slots raises demand **and** costs 10 × 111.98 MiB of pinned checkpoint-store RAM
  per slot. Under-sizing the pool the other way has always been a boot failure (36,864 refused a boot in
  the fp8 era: the host cannot hold what the cards will not). Behavioural check on the live shape:
  `/health` reports `queue_cap 32`; a request beyond the slot count queues rather than being refused,
  and the 400 class is a single request over `--max-model-len`.
- **Idle KV VRAM is the expert cache's spillover — the price is paid twice.** `mover: 2275
  slots can be lent out of the KV cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <=
  256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 in the pair era)
  so a bigger KV pool steals weight residency from the slab AND from the lendable cache. Priors
  on this card/model: stealing 2.14 GiB of weights to buy `MAX_MODEL_LEN 262144` cost 2.5 %
  decode; the other direction (`--expert-vs-cache-ratio` 0.82 → 0.84, +2.5 GiB of weights) cost
  8.9 %.
- **A KV-pool cut is a runtime-headroom lever, never an arena lever.** The 6.1e9 → 3.15e9 B/rank pool cut
  (2.75 GiB/rank) bought **zero** extra expert-cache slots. The arena moves with the reserve, one-for-one
  (d(reserve) 0.57 GiB = d(arena) 0.57 GiB = **+248 slots**). The shape that pool cut replaced died with
  `0 bytes free` on a real 995-token request — that is headroom, not slot count. Read `expert cache space … / N
  slots` out of the boot ledger before trusting any pool arithmetic: the 2026-10-13 read was **14.87 GiB /
  6,496 slots** against `GPU KV cache size: 398,658 tokens`, and it is the POOL that caps resident
  conversations, not the arena.
- **MNBT moves the compile RANGE, not the batch width.** The engine sets `num_reqs = min(max_num_batched_tokens,
  max_num_reqs)`, so the CUDA-graph compile width is **SEQS + 1** for any `MNBT >= SEQS` — 3 at SEQS 2, 4 at
  SEQS 3, exactly the constant printed in the engine's own message. MNBT only shifts the compiled range
  ((1,1024) versus (1,2048)), which is why the identical capture failure appears at both rungs;
  `--enforce-eager` bypasses the path and loses CUDA graphs with it.
- **The prefix cache proves itself at depth, not in the ledger.** Two concurrent conversations of 196,919 tokens
  each answered the marker buried at the head of **their own** prompt with **393,837 tokens resident** — above the
  fp8 pool's 386,904, so the answer came from a cached prefix and not a re-prefill. Same boot (2026-10-13):
  decode at 51.5k depth **94.1 cold / 102.3 warm / 117.6 prefix-hit t/s**, draft acceptance **0.7284 / 0.7525**.
- **The host pool moves with the slab.** `vram_weights` and `host_pool_mib` serve the per-rank
  resident set *together*: at the fp8-KV pair (26071/3543) the ledger showed the slab full to
  259 KiB of slack, and cutting `vram-weights` by 3.8 GiB to pay for a bf16 pool spills ~3.8
  GiB/rank into host RAM — under-sizing the pool there is a **boot failure** (36,864 refused the
  boot in the fp8 era). The ledger's coverage line, not the arithmetic, arbitrates.
- **Slab → prefill: the arms disagree, the cliff does not.** 2026-10-11 exact-KV A/B (slab
  22,208 vs the fp8-era 26,071, −14.8 %): prefill −29 %, TTFT at the full window +25 %, decode
  inside the noise band on 2 of 3 passes, draft acceptance UP 0.661 → 0.740. 2026-10-13
  (`probe-frank-archon-kv.py`, env header): A-cold 43.5k 22,208 → 3,397, 24,000 → 3,290, 25,600
  → **3,584** t/s, and the 3-way 197k arm's aggregate 3,051 → 3,589 (wall 193.7 → 164.5 s), so
  +1,600 MiB of slab bought +8.9 % single-stream / +17.6 % concurrent — read then as a SLOPE,
  the 22,208/24,000 pair having landed inside noise of each other and falsely looked like a
  plateau. Re-measured at 3 slots after the 2026-10-15 swap the 25,600 arm returned 3,290: the
  3,584 did **not** reproduce, so treat 22,208-25,600 as one band until someone re-runs the
  ladder. What is not disputed is the cliff: 26,071 → 4,379 t/s (the 2026-10-11 control at seqs
  3 / pool 3,543), +22 % for +471 MiB — unreachable at 3 slots (26,112 + 5,120 = 31,232 vs
  30,969) and free at 2 (pool 3,072 + slab 27,897 = the ceiling exactly). Two points inside
  noise of each other are not a plateau, and two points are not a curve.
- **Decode is nearly window-free; slots are paid in concurrency.** 99.0 t/s at 43.5k vs 93.3 at
  197k (−6 % for 4.5× depth), while three streams give 113.9 / 43.3 / 10.9 = 18.4 t/s aggregate
  — a fused-step lottery, not a fair share. Read it from leg D of `probe-frank-archon-kv.py`
  with ≥ 128 generated tokens; an 8-token probe measures first-token noise.
- **MNBT 2048.** 2560 collapses decode to 2.6-7.7 t/s (MTP resonance); the w4 file tier streams
  in place only at MNBT ≤ 2048 and the activation arena scales with the chunk (32768 → 33.3
  GiB, refused).
- **Decode pays for the chunk it shares a step with, and `--busy-prefill-chunk auto` does not
  protect it.** Re-measured on the live 1.8.1 shape (34.8 h boot, Prometheus at 60 s,
  `engine_step_seconds{,_decode}_total` / `engine_steps{,_decode,_mixed,_prefill}_total` /
  `prefill_tokens_total`): decode-only steps run at **15.5 ms**, chunk steps (a chunk alone, or a
  chunk with decode rows beside it) at **166 ms** — 4.7 % of the steps hold **34 % of the engine's
  step time**, so a session decoding while another prompts advances one token per 166 ms. Bucket the
  harness's `radiance:decode_tokens_per_second` by `radiance:prefill_tokens_per_second` (15 s
  scrapes; per-stream rate = the gauge ÷ `vllm:num_requests_running`): median **166 t/s** under light
  prefill, 81 t/s at 1-5k, **13 t/s above 5k** — a **12.7×** collapse, and at 13 t/s with two
  sessions running each stream sits at 6.7 t/s = 149 ms a token, one token per shared step. Nothing
  else moves: decode-*only* steps stay 14-18 ms in every bucket (14.1 ms in a prefill-free minute,
  17.9 ms in the deep one) and the relationship is lag-0 only (r = −0.74 at lag 0, |r| ≤ 0.19 from
  30 s on, decode back at baseline by 5 min) — residency is not the mechanism either. The second
  victim is the **arriving** request: through a high-prefill window TTFT p90 goes 2.4 → **18.0 s**
  and 15.5 % of requests wait over 10 s for their first token, with `http_queue_depth` at 2-3, while
  median TTFT (~1.9 s) and per-token latency barely move. Preemption is real but burst-shaped: 264
  `vllm:num_preemptions_total` in the boot, 108 of them inside 105 s of a 6.5k-tok/s prefill. The
  chunk is `min(remaining, checkpoint interval, MNBT, busy_cap)`; the first three read 2048 here,
  and `busy_cap` is the fourth law.
- **`auto` is a prefill-speed dial, not a decoder-protection dial.** `Scheduler::busy_cap` reserves
  `1 + spec` rows per decoding sequence behind the chunk (3 tokens at MTP 2) plus one eighth of the
  budget per *waiting request or waiting prefill*, and the code refuses to do more for decoders:
  "not for decode rows alone: their reserve is a few tokens, so the chunk ends a quantum short and
  the next step finishes the interval… Halving would hold a long prompt to half-interval chunks while
  anything decodes behind it, which costs more" (`core/sched/scheduler.cpp:874-878`, 1.8.1). Live
  proof: `prefill_chunks_capped` 5,018 against 62,386 chunk steps — 8 %, and those are the minutes
  with a waiter, not the minutes with a decoder. An explicit `N` is the only value that applies
  "whenever the chunk shares the step".
- **At MNBT 2048 no chunk value keeps both the arena loan and the staged copies.** The arena is a
  ladder of levels (256 and 1024 rows here) and a step's rows are its **whole** token count —
  `max(n_tok, n_seq × (1 + spec))`, `core/engine.cpp:1142` — so a 1024-token chunk with two decode
  rows is a 1030-row step: it falls back to the full plan, which "lends nothing", and it arms the
  prefill stager. The stager's floor is the largest lending level plus one, 1025 rows here — named on
  the boot line `prefill staging: … a step of more than 1024 tokens copies the pooled experts of 48
  routed layers onto the card up to 3 layers ahead` — and below it "every routed layer reads its
  pooled experts synchronously and the prefill slows far more than sharing saves" (GUIDE §4.5). The
  vendor's own number for naive shrinking, on this architecture: four decoders with a long prompt
  beside them finished in **29.5 s at a 2048 chunk and 31.9 s at 512**.
- **A big step takes the arena from the expert slab and keeps it for the whole prefill.** The arena
  lends 987.77 MiB (steps ≤ 256 rows) or 670.26 MiB (≤ 1024) to the slab; a full-plan step reclaims
  it **synchronously before the step** — units revert to their host copies (an address change, not a
  copy) and both read streams drain — and `arena_tick` regrows the loan only **2 s after the last big
  step** (`core/engine.cpp:1129-1202`), so every step of the prefill run, decode-only included, runs
  with a slab ~780 expert units smaller over a link also carrying the staged copies. `/diagnostics`
  says the binding constraint in one line: "a prefill step streams every routed layer's pooled
  experts onto the card, so its speed is bounded by the link; decode reads the ones it routes to".
  Measured size of the whole effect: the ~10 % above. Real, small, not worth chasing.
- **`--prefill-interleave` is the dial that buys decode its cadence; armed at 1 on 2026-10-20.**
  N decode-only steps follow any chunk step taken beside decoders; held steps admit nothing, so it
  costs the prefilling prompt its TTFT and an arriving request its first step. Rate arithmetic at the
  two measured step times and 0.657 acceptance (≈2.3 tokens a step): N=0 → ~14 t/s, **N=1 (armed) →
  ~23**, N=2 → ~33, N=4 → ~48, N=8 → ~69, with the prefill slowed by (166 + 15.5N)/166. **Arithmetic, not
  measurement** — the arm is chosen by the §9 bucket pair plus `steps_held`, and `auto` is not a
  substitute for it.
- **`--prefill-span` is the shape that keeps the loan, and it is priced.** It needs MNBT = N
  intervals, so `--prefill-span 2 --max-num-batched-tokens 4096`: a *lone* prefill carries 2
  intervals a step (and on an expert-tiered box only from 8 intervals / 16k tokens, where the span
  starts to win), while **every other step is planned at the shared budget of 2048** — which is a
  lending level *and* keeps the stager's ring, the one combination `--busy-prefill-chunk` cannot
  express. Costs per GUIDE §4.5: 320 MiB a card of expert residency, ~3 % off the KV pool (§4.3's
  cliff is sharp), a decode step uploads twice the per-step arrays at 4096, and checkpoints inside a
  span are not written, so a partially-shared prompt resumes one interval earlier. On the vendor's
  second box (2×R9700, w4nl64, TP2) it was TTFT-neutral — 2.61 vs 2.56 s at 18k, 10.72 vs 10.55 at
  75k — hence their own rule: measure your own prompts at two ranks before turning it on. MNBT 4096
  does not boot at 3 slots; this shape is 2.
- **Most of this endpoint's prefill work is replay of tokens the prefix cache already had.** Over
  the same window `prefill_tokens_total` 65.2 M against `prefix_hit_cut_tokens_total` 24.9 M = **38 %
  replayed**, and it is not an eviction: a hit on a hybrid model reuses only as far as the last
  **linear-state** checkpoint (`reuse = min(h.n_attn_tokens, h.n_linear_tokens)`,
  `core/sched/scheduler.cpp:953-978`), and the retention default keeps a **geometric ladder** - about
  `log2(depth/interval) + 2` snapshots - so a turn that diverges `d` behind the tip can resume as
  deep as `tip − 3d`. The live ladder for a 200k session at interval 2048 is 2048 · 131072 · 163840
  · … · 198656: diverge at 120k and 21.7k tokens replay, diverge at 50k and 48k. Every
  `prefix_hit_cuts_total` carries `reason="no_snapshot"` (the other two reasons stay dead), so the
  cause object agrees and `hit_cut_tokens` is the cost. `--checkpoint-policy` admits exactly
  **`geometric-backoff` | `tip-only` | `keep-all`** and refuses anything else at boot. It governs
  retention only - the writes happen at every interval whatever the policy says (only `--prefill-span`
  skips them) - so `keep-all` buys a replay bounded by one interval at any depth for **pinned host RAM
  alone**: 97 snapshots a 200k session, `--checkpoint-slots auto` asking 196 × 111.98 MiB = 21.4 GiB
  against a store budget of an eighth of host RAM (31.5 GiB here, so `share` admits it) with `room` -
  the pinning room left beside the expert pool at boot - as the hard bound the ledger line names.
  Retention also spares a snapshot a later turn resumed from (`core/mem/prefix.cpp:709-723`), so a
  chain that keeps resuming at one deep point stops paying for it. Shortening
  `--checkpoint-interval` does **not** buy the deep case: the coarsest ladder scale is the largest
  `interval × 2^k` under the depth, so the deepest gap stays ≈ half the prompt (129,024 tokens at
  interval 2048 against 130,816 at 256) while every cut multiplies the snapshot writes.
- **Slots cost KV, host RAM, rows, and pin — never speed.** `--max-num-seqs` carries no radiance
  constraint: it adds ≈110 MiB of recurrent state per slot, 10 × 111.98 MiB of pinned checkpoint-store
  RAM per slot (`--checkpoint-slots auto`), and declares the fused TP all-reduce over
  `seqs × (1 + spec)` rows — `wht6` is bit-exact up to 25 rows, and the live shape is 4 × 3 = **12**, so
  single-sequence decode stays exact and so does any step up to that many rows. Raising it also raises
  the cache's *demand* (`ceiling = slots × window`), which the resolver funds from `elastic −
  expert_floor` — so a slot count without matching host room is paid for by expert residency. Measured
  2026-10-06: seqs 2 and 3 statistically tied (30.9-33.1 t/s, inside the ±3 % floor). Measured
  2026-10-19 → 10-20 at 1.8.1: 2 → 4 slots cost **+4,224 MiB of pinned host pool** and **−3.01 GiB of
  expert residency a card** and bought 2.76 → 5.64 GiB of cache (365,204 → 747,492 tokens). The old
  "seqs 2 cannot boot" law belonged to the retired tcclaviger fork's `torch.compile` shape
  specialization, and radiance booted seqs 2 on its first fp8-native start.
- **A dequantize-to-compute path bills host RAM twice.** On the EMULATION backend an offload request
  of **118 GiB** landed as **~238 GiB** of real host RAM — the dequantized copy rides on top of the
  pinned weights, and the doubling is invisible in the weight count. Read the engine log for
  `Using 'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit"; a page that says a payload
  "does not fit" without that line has not read the runtime.
- **One encoding per expert table** (`rad_arch_declare`): a whole expert kept bf16 is the only
  thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable.
- **Descriptor-selection law.** The plugin is picked by the exact `(arch_id, quant descriptor)`
  pair with no fallback: a raw official FP8 checkpoint (`quant_method: "fp8"`) is refused by
  serve *and* convert, because the shipped `qwen4exp_fp8` plugin claims `(qwen4exp, "")`. Stage
  a `config.json` with `quantization_config` dropped — the descriptor changes, the data stays FP8.
- **Three convert laws, each paid in a full verify+declare cycle.** Scale-name: `scale_name()`
  appends `_scale_inv` to the *tensor* name, so stacked `…experts.gate_up_proj` keeps its scales
  at `…gate_up_proj_scale_inv`. File-shadow: the reader opens index-named files sorted and
  emplaces every tensor each file physically holds, first-wins — a merged PLE shard must sort
  first (`00-ple.merged.safetensors`) or a raw shard wins the name. Symlink-through: staging
  symlinks the raw index, so `open(path,"w")` writes through it and corrupts the checkpoint —
  write temp + `os.replace`.
- **Convert is card-less** (`rad-convert` declares on CPU, GPTQ solves on CPU BLAS) and
  `--plan-only` goes first: a plan that resolves every op will boot. `rad-tune` must be
  declared at the shapes you serve and its cache lives in the *image* (`$RADIANCE_HOME/tune`)
  so it only survives on the `$FRANK_ARCHON_STATE_DIR/tune` bind-mount the serve script adds.
- **Quant gates are scored at the production shape** (MNBT 2048, not the recording's 32768)
  against the BF16 container: KL ≤ 0.03, top-1 ≥ 95 %, and the verdict is chunk-size invariant
  (arm Δ ≈ the 0.0015 float-order band) and placement-invariant. Protected experts are
  data-driven — energy share ≥ 3 % AND ≥ 3× routing share, even count per rank half — and the
  PLE rides the checkpoint's own fixed scale (1.9931793212890625e-04, absmax inside the
  2⁻¹³..2⁻¹⁰ plateau).

### This host, and the deploy path

- **The pinned-RAM ledger is why this is a root unit:** 2 × 44 GiB expert pools + the 47.7 GiB
  PLE n-gram table (51B params, read from pinned host RAM over io_uring — Frank's pin law:
  "*Native FP8. Experts from ram. Pin the PLE to ram.*") + the 8 GiB prefix host tier ≈ **147
  GiB** charged against `RLIMIT_MEMLOCK`; dank's login session caps at 8 MiB and a user quadlet
  is capped by the user manager, so only PID1 can grant `infinity` — and
  `--ngram-placement ram` refuses the start when the limit is short.
- **TTM is runtime state, and the gate that owns it raises before it refuses.** The pinned pages
  ride `/sys/module/ttm/parameters/pages_limit` (+ `pages_max` on kernels that expose one; this
  kernel exposes **none** — the cap is the MIN over the knobs present, and an absent knob is
  skipped, never read as zero, because a zero would refuse a healthy cap). The kernel default is
  **half of RAM**, so every raise dies with the reboot: the 2026-10-05 manual raise to
  52,428,800 pages (200 GiB) was gone after the **2026-10-12 23:17 power failure**, the cap fell
  to 125,988 MiB, and the preflight refused the start **6,919 times over ~20 h** while
  `Restart=always` cheerfully re-ran a refusal. A manual sysfs write is therefore not an
  acceptable repair procedure — `scripts/ttm-preflight.sh` writes `TTM_TARGET_MIB` (204,800 MiB
  = 52,428,800 pages; pages are 4 KiB so pages = MiB × 256; the *recorded host ceiling*, because
  writing the 150,000 floor would leave the ledger no slack) into every knob present, re-reads,
  and only then refuses. Write failures are swallowed deliberately: the caller re-reads, and ttm
  itself refuses a limit below the pages it has already pinned.
- **Privilege of that knob, measured the hard way:** it is root-owned 0644 and the service runs
  `User=dank`, where `-w` is false and a plain `echo` fails. The canonical privileged path is the
  unit's `ExecStartPre=+…` (the `+` runs it as root even under `User=`); `sudo -n` is the
  fallback for a human. The file is *sourced* by the serve script (which re-verifies the cap
  before `podman run`) and *executed* by the unit. A refusal is invisible outside the journal —
  no container is created, the unit sits in `activating (auto-restart)` on one repeating line —
  and without the gate the load instead dies mid-pin with a bare allocator error.
- **The prefix host tier needs real RAM:** 8 GiB of it, and with the service loaded the host
  still has ~104 GiB available (246 GiB total) — that headroom is what the pinned ledger is
  measured against.
- **The deployer's blind spot is host-side unit drift.** A leftover
  `frank-archon.service.d/bench.conf` overrode both `FRANK_ARCHON_ENV_FILE` and `ExecStart` to a
  staging worktree and every `deploy.sh` since was green against the old configuration —
  drop-ins are read *after* the unit's own `Environment=` lines, so they win.
- **Never `systemctl disable` a unit on these hosts** — it removes the `/etc/systemd/system/`
  entry, which here is the symlink into the checkout. `mask` is reversible but *fails on a
  relinked unit*, so a bench window uses `stop` and the deploy restores the unit.
- **The restore oracle for an experiment window is a conjunction:** `is-active` == active AND
  `is-enabled` == enabled AND no `/run/systemd/system/<unit>` mask symlink AND the container is
  owned by the unit and not `Stopping`. The vLLM/GLM-era engines ignored SIGTERM and answered
  `/health` 200 for 20+ minutes while `Stopping`; a runtime mask survives `stop` and blocks
  `start`, and a failed unit needs `reset-failed` first.
- **`MemAvailable` is an input, not a backdrop:** the expert-offload planner reads it, so
  anything booted beside the live service dies at plan time (`ExpertPlanError: host holds 5113
  of 12096 swappable experts`) — and fleet law forbids GPU experiments beside production
  anyway. Long remote work runs as a transient unit or `setsid`+disown: `nohup X &` inside `ssh`
  races ssh teardown and can die before writing a byte.
- **The ~40k-token corruption class was the RAM, not the engine.** Dropping DDR5 from 4800 to
  4600 MT/s ended it, with zero `gdn_hip` faults either side; the prefix-caching / GDN re-seed
  theory was confounded on both arms. Consumer non-ECC raises no MCE for a silent flip and a
  60 s / 8 GiB pattern check covers 0.5 % of 246 GiB — silent RAM corruption outranks every
  software theory. **That mitigation is a BIOS setting and it is not in effect now:** `sudo
  dmidecode -t memory | grep -i configured` answers 4800 MT/s on 2026-10-20, and the only event
  between the fix and today is the 2026-10-12 power failure that also reset TTM. Check the speed
  before crediting any software revert with a fix — and before believing a clean run of the arm
  you just tested, because the confounder is still in the slot.
- **TP=2 here** is legal since the 2026-09-20 ProArt B850 swap (x8/x8 on CPU root ports,
  `Routing+`/`ReqEn+` end to end) only because radiance's wire is `libr4d`'s own one-shot
  all-reduce; RCCL needs atomics on the whole chain and fails `hipErrorIllegalState` without
  them (). `wht6` buys +5.6-6.3 % prefill for ~1.3 % of 8-stream
  decode, `exact` is the bit-exact alternative.
- **The link is already at the board's ceiling — do not chase the `link-below-max` warning.**
  Both root ports (`00:01.1`, `00:01.3`) are **x8 max** at 32 GT/s, so PCIe 5.0 x8 ≈ 31.5 GB/s is
  the whole budget per card and the cards' own `pcie_dpm` tables expose only x8 states.
  `/diagnostics` still warns that the on-card switch hop (`01:00.0`, `05:00.0`) runs "x8 of its
  x16 while the card is 100 % busy" and suggests a reboot or a reseat — pointless when the host
  side of the chain is x8 max, which the same endpoint reports two findings later as
  `link-capped`. The width that matters is the minimum of the chain, and it is x8 either way, so
  the only lever on a link-bound prefill is **less traffic over the link** (more expert
  residency, fewer pooled experts), never a wider link. P2P resolves **on** (`auto` asks the
  driver, the two cards reach each other's memory), so the collectives do not hairpin through host
  RAM, and the engine's copies ride one DMA queue that "give[s] way to the collectives" — the
  all-reduce wins the link, the staged experts wait.
- **Eight cores is the whole host, media stack included.** Dankserver is a Ryzen 7 7700X (8
  cores / 8 threads, one NUMA node, `nodes: -1` for both cards) running Jellyfin, Plex, Calibre
  and RomM beside the engine, and `--http-threads` defaults to one thread per core. Measured
  during a heavy prefill: load average ≈ 1.0, `jellyfin` 11 % CPU, `radiance` 7 % — **CPU
  contention is not the current decode culprit**, so do not spend an experiment window on it;
  re-read `uptime` and `ps -eo pcpu,comm --sort=-pcpu | head` first if a future collapse looks
  different from the step-sharing one.
- `--prefix-cache-dir` creation is **not recursive** — the serve script `mkdir -p`s the path or
  the disk tier silently never opens. The tier does survive a payload swap when arch and
  tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; `cache_n` proves a hit).

### Dead ends — do not re-try

- **radiance 1.2.0 as the serve pin** (`sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa`
  commit `e95d397`), benched 2026-10-06 on this exact shape with a 43.6k greedy probe: decode
  29.21 t/s and draft acceptance 0.7123 vs the 1.0.13 control's 30.08 / 0.7549, and the control
  reproduced the recorded 0.755 exactly — greedy acceptance is deterministic, so the drop is a
  real numerics change (the decode loss tracks the acceptance ratio, 0.943 vs 0.945) with the
  1.2.0 MoE weight-rotation change `92b8aaf` the named suspect. The vram-pair translation itself
  is exact (the 1.2.0 ledger took 25.46 GiB weights / 3.46 GiB KV per card, byte-identical to
  1.0.13 at EVC 0.82), and 1.2.0 REMOVED `--expert-vs-cache-ratio` in favour of the pair, which
  both pins accept. Re-test 1.2.0+ once the rotation change is untangled.
- **FreeToken TP=2 on ROCm:** the `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201 —
  an untested edge of that PR, not a board fault.
- **tcclaviger forks:** `--max-num-seqs 2` (`ConstraintViolationError` on `query_start_loc`)
  `--gpu-memory-utilization 1.00` unbootable, 0.982 is the law; expert prefetch is poison (36 %
  precision — deleting it was +40-51 % decode); TunableOp-with-offload corrupts numerics
  (repetition loops, early EOS) — autotuning that changes text is broken tuning; ngram-only
  drafting dies at init (`PLE inputs were not prepared`); DCP needs TP > `num_kv_heads` (2 here,
  so 4 GPUs); 6 × 262,144 fits no KV dtype (6 sequences leave 9.18 GiB of pool where fp8 needs
  12.47). fp8 KV on that path also removes the only MLA decode gfx12 has (the aiter op is gated
  on CDNA > 2, so the draft step demands `…mla_decode_fwd`, which cannot exist there) and its
  KV reserve model is dtype-blind (identical 13,600 B/token plan line) — no capacity for the breakage.
- **DeepSeek-V4.1-Flash on 32 GiB cards:** only `--ds4-prefill exact` is legal,
  `--ds4-fused-decode` is refused, and at ~22 % resident experts each token pulls ~187 × 12 MiB
  from NVMe — low GPU util with a pegged link is a *capacity verdict*. Two corrections survive:
  expert TP **does** work on gfx1201+gfx1201, and an expert-placement JSON outside the mounted
  `/models` tree gives a silent single-tier fallback.
- **Which experts you pin does not matter; how many does.** An IBS census of the live server
  (`scripts/frank-archon-expert-activity.py`) measured per-block read volume flat at 0.93-1.08×
  of mean across every CPU-resident block — top-k routing fires all of them every token, so at
  `-ot` granularity the lever is COUNT.
- **Never mount a diagnostic hook on `PYTHONPATH` without a marker guard:** `sitecustomize` is
  imported by every interpreter, a child inherits the hook and spawns again — a 19-level python
  fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation)
  never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions.

## Contract (the wire)

- **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions`
  `/v1/completions`, `/metrics`, `/stats`, `/sessions`, `/dashboard`, **`/diagnostics`**. `/health`
  answers 200 JSON (model id, `in_flight`, `queue_depth`) while serving and **503 only while
  stopping** (`core/server/admin.cpp:385`), and the listener binds *after* the full load
  (`main.cpp`: `engine.start()` then `engine.serve()`) — during the load the port refuses
  connections, so a bare `curl -fsS …/health` gate is honest and needs no body grep.
- **`/diagnostics` is the instrument for "is the box, not the config, the limit"** — one call
  returns `configuration` (every flag, so drift from the env file is checkable without the host),
  `environment`, `derived` (the VRAM split as the ledger prints it), `hardware` (host, per-card
  `link_chain` + `pcie_dpm` + clocks + power cap + DMA engine choice + P2P answer) and `findings`
  (level / what / why / fix). On this box it reports `experts-from-host` (5.99 GiB a rank crosses
  PCIe when used), `link-capped` (root ports are x8 max — the board), `swap-in-use`,
  `tp-wire-lossy`, and **`link-below-max` as a `warn` at 100 % card busy on both cards** — which is
  structural on this board; see the x8-ceiling law under *This host, and the deploy path*.
- **Served id.** `FRANK_ARCHON_SERVED_NAME=frank-archon` (the wire rename of 2026-10-09 moved
  only the alias; container, unit, scripts and env keep the service name). The env's contract
  line is that **radiance validates the requested model name, so every client — harness, bench
  scripts, probes — MUST send it** — while the era pages also record that a chat response
  *echoes* the requested name, which is why `/v1/models` is the served-id oracle and a 200 chat
  probe proves nothing about what is actually loaded.
- **Reasoning** lands in `reasoning_content` (`--reasoning-format` defaults to `auto`; no
  parser flags exist or are needed) and `usage` bills it separately (`reasoning_tokens`).
  **Thinking consumes `max_tokens`**, so the classic false negative is a probe with empty
  `content` and `finish_reason: length`/`stop` because the whole budget went inside the think
  block: either set `chat_template_kwargs.enable_thinking=false` or budget a few hundred tokens
  and read `reasoning_content`. Same trap, sharper, on the vision probe below.
- **Sampler surface:** `temperature, top_p, top_k, min_p, typical_p, presence_penalty,
  frequency_penalty, repetition_penalty, repeat_last_n, chat_template_kwargs, reasoning_effort,
  enable_thinking, preserve_thinking` accepted; **unknown parameters are refused 400** (nothing
  unverified can ride the wire, and an `extraBody` typo fails loudly); `max_tokens` is clamped
  not refused; `response_format: json_object` honoured; **`logprobs` refused**; no `seed` = the
  engine picks; `n` defaults to `max(8, max-num-seqs)`; stop strings ≤ 64 × 4096 B.
- **Tools:** the qwen3.8 dialect from the chat template → `message.tool_calls` +
  `finish_reason: tool_calls`. **`reasoning_effort`:** the froggeric template does not gate the
  set (xhigh/high/medium/low all 200); the harness map is a safety net, not a requirement.
- **Vision is native** (the checkpoint's 27-layer Qwen-VL tower through the engine's Qwen-VL
  processor): an `image_url` content part, no legacy top-level `images` field, ≤ 4,194,304 px
  and 16,384 patches per pass. Probe with a ≥ 256 px solid-colour PNG and a ≥ 256 token budget
  — at 64 tokens you get empty `content` and the colour only inside `reasoning_content`.
- **Capacity behaviour:** a prompt over `--max-model-len` is refused, never truncated; a request
  longer than the KV pool's token count is accepted then refused as too large to fit; a request
  beyond `--max-num-seqs` is held by the HTTP port (`queue_cap` = `max(8, 8 × slots)` — **32** here)
  and answered 503 only past that, never paged or squeezed. The cache funds **3.7** full-height
  sessions, so four conversations can be resident but a fourth near-full-height one can still
  **preempt a running sequence from the back** to free its blocks
  (`core/sched/scheduler.cpp:1291-1322`); the victim is recomputed, so its session halts and then
  re-prefills. Meter: `vllm:num_preemptions_total` (cumulative for the boot, `/stats preemptions`);
  `radiance:num_requests_preempted` is only the instant depth.
- **Observability:** `/metrics` carries vLLM-compatible names plus `radiance:*`, scraped by the
  Dankclaw Prometheus job at `192.168.0.235:7770` (15 s interval) over plain LAN routing (the
  netavark published-port quirk is same-host only). **Latency lives under `vllm:`, cadence under
  `radiance:`** — TTFT / inter-token / per-output-token / e2e / decode-time histograms and the
  running/waiting gauges are `vllm:*` with no radiance-native equivalent, so a `radiance:`-only
  query misses every latency answer. The step-phase counters are all scraped:
  `engine_steps{,_decode,_mixed,_prefill,_held}_total`, `engine_step_seconds{,_decode}_total`,
  `engine_prefill_chunks_capped_total`, `prefill_tokens_total`, `decode_tokens_per_second`,
  `prefill_tokens_per_second`, `prefix_hit_cut_tokens_total`, `http_queue_depth`. Two holes: the
  engine's own step-detail histograms (`prefill_steps_duration`, `prefill_step_tokens`,
  `token_queue_wait`) are exported but reach no target, and the **arena / expert slab / heat engine /
  prefill stager have no metric at all** — that half of every law above is read from the boot ledger,
  `/stats` and `/diagnostics`, never from a graph. The boot ledger — the referee for every shape
  number here — is in the container log (`podman logs frank-archon`);
  `/home/dank/logs/frank_archon.log` is where the serve script redirects `podman run`.

## Files

- Shape and law owner: `scripts/frank-archon.env` (measured arithmetic in its header)
  `scripts/serve-frank-archon.sh` (gates image / `.rad` size / template / GPU nodes, retires
  stale containers, `podman wait`), `scripts/frank-archon.service`, `scripts/ttm-preflight.sh`.
- Payload doors: `scripts/download-frank-archon-iq4r.sh` (live), `scripts/download-frank-archon.sh`
  + `scripts/build-frank-archon-rad.sh` + `scripts/stack-qwen38-official-fp8.py` (the fp8-native
  rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert
  needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template
  `scripts/qwen38-froggeric-v22.5.jinja`.
- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh`
  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh`
  `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py`
  `scripts/make-calib-corpus.py`, `scripts/derive-protected-experts.py`
  `scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`.
- Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so
  it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`
  `scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`.
- Deploy and tests: `scripts/deploy.sh` / `scripts/deploy-remote.sh` (`deploy_llm_archon`)
  `tests/frank_archon_unit.bats` (the seams gating the KV-pool arithmetic, the VRAM pair and the
  ttm gate), `tests/changed_tests.bats`, `tests/deployment_scoping.bats`.
- Retired-era leftovers in-tree: `scripts/patches/glm53-*.patch`
  `scripts/patches/gpu_worker-kv-cap.patch` (fork archaeology; none applies byte-exact today).

Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
---