serverAppConfigs/serverAppConfigs-docs/LLM/frank-archon.md

---
type: reference
title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught."
tags:
  - llm
  - frank-archon
  - dankserver
  - radiance
  - gfx1201
  - r9700
  - tp2
  - expert-offload
  - endpoint
timestamp: 2026-10-18
---

# frank-archon — radiance (StillDeadcode), 2× R9700 TP=2, Dankserver :7770

The radiance engine (C++/HIP, `libr4d` kernels built for gfx1201, no Python in the serving
path) driving `podman run` straight from the digest-pinned image under a **root** systemd
unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI
PRO R9700s. Capacity contract **3 conversations × 200,000 tokens at fp8 KV**; a payload swap
is a `MODEL_FILE` line plus a re-download — see Eras.

## Current shape

Authoritative source: `scripts/frank-archon.env` + `scripts/serve-frank-archon.sh` (the env
header carries the measured arithmetic beside each number).

**Payload** — `/home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad`, 121,969,901,568 B
= **116,322 MiB**, sha256 `0af5e962…` (the Hub LFS oid), from
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`: the engine author's own rad-convert of the
**official (censored)** `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint. Encodings (recipe
`qwen4exp-w4nl64-i8-hc8m`): routed experts as 4-bit codes in a 16-level non-uniform codebook
(128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent
sweeps, E4M3 scale per 64 weights); **10 whole bf16-protected experts per layer**; int8 W8A8
trunk + lm_head; fp8 HC mixtures; PLE n-gram fp8 at the checkpoint's fixed scale (47.7 GiB)
MTP head and the bf16 27-layer vision tower **in the container**. Card quality vs bf16: mean
KL 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %. The abliterated family lives on
[[frank-dragoon]]. Fetched by `scripts/download-frank-archon-iq4r.sh --fetch` (hfget resume +
declared size + sha256) as the transient unit `iq4r-fetch` (log
`/home/dank/logs/iq4r-fetch.log`, `…rc` sentinel); the staging dir
`/home/dank/models/StillDeadcode/…` keeps card, LICENSE and recipe as provenance, and only
`.rad` files are mounted.

**Runtime** — `stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e`
= pin **1.0.13** (Codeberg `v1.0.13`, commit `53282fa`): a 287 MiB image in dank's
**rootless** storage, host needs nothing but the amdgpu driver. Pin the INDEX manifest digest,
never the image ID (`69d05724…` is the config digest and sends `podman run` to the network).
The serve script's gate is that digest ref itself — `podman image exists name@sha256:…` in one
command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike or a newer
`latest` cannot satisfy it. 1.2.0 is a rejected serve pin (Laws); the
`--vram-weights-mib`/`--vram-kv-mib` pair is accepted by both pins, which is the rollback symmetry.

**Trio** — `scripts/frank-archon.env` (every number, with its arithmetic in the header)
`scripts/serve-frank-archon.sh`, `scripts/frank-archon.service`. Gates run in order: image
digest present → `.rad` present and ≥ `RAD_MIN_MIB` (a truncated convert would otherwise buy a
guaranteed long-timeout boot instead of this fast failure) → template present → `/dev/kfd` +
`/dev/dri` → `ttm_ensure_cap` (the serve script re-verifies the cap after the unit's hook) →
`podman rm -f` over `FRANK_ARCHON_LEGACY_CONTAINERS` (`frank-archon-qwen frank-archon-r9v
qwen-fp8 glm-probe`) plus its own name, because systemd SIGKILLs conmon on a crash and leaves
the name registered, and every one of those identities holds this GPU. Container contract
(publisher-vetted, `docs/DOCKER.md`): `--network host`, `--init` (so the engine releases the
cards on SIGTERM), `--security-opt seccomp=unconfined` (the default profile refuses io_uring —
the PLE table's reads — and the NUMA calls the ROCm runtime makes for host memory), `--ulimit
memlock=-1:-1`, `/models` ro + `/data` rw + the tune cache at `/opt/radiance/share/radiance/tune`
template as a single-file mount, then `exec podman wait`: the script's lifetime IS the
container's, so a crash is a nonzero `ExecStart` and the restart policy is real.
Unit: `After=network-online.target`, `WantedBy=multi-user.target`, `User=dank`
`LimitMEMLOCK=infinity`, `StartLimitIntervalSec=0`, `OOMScoreAdjust=-500` (the last process the
OOM killer should pick), `Restart=always` **not** `on-failure` — a clean API-server exit after a
worker fault reads as success to systemd (the 2026-10-01 vLLM lesson) — and
`TimeoutStopSec=180` > the `ExecStop=podman stop -t 120` grace, because systemd's 90 s default
cut the stop short and recorded `Failed with result timeout` (measured 2026-10-01 13:27:17). No
drain contract is claimed: the OpenAI front end holds no queue, so an in-flight decode is lost
on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_period`).

| Knob (env name, `FRANK_ARCHON_` dropped) | Live value |
|---|---|
| TP / wire | `TP=2`, `TP_WIRE=wht6` — libr4d one-shot all-reduce, not RCCL; `min-kb` stays the engine default |
| Slots / window | `MAX_SEQS=3`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) |
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds 3 slots (Laws) |
| VRAM pair | `VRAM_WEIGHTS_MIB=25600` + `VRAM_KV_MIB=5120` = 30,720 of the 30,969 ceiling; `GPU_HEADROOM_MIB=96` |
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=44928` **per rank** |
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
| Spec / batch | — |
| Prefix cache | `PREFIX_HOST_MIB=8192` + `PREFIX_DISK_MIB=65536` under `STATE_DIR=/home/dank/radiance-state` (mounted rw at `/data`) |
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
| TTM gate | `TTM_MIN_MIB=150000`, `TTM_TARGET_MIB=204800` |
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |

Boot ledger for this shape (2026-10-15, pin 1.0.13): pool 5.00 GiB backs **607,740 tokens**
(`kv_attn` fp8 page 26.00 KiB, `kv_qsa_bkey` 3.25 KiB), `kv_gdn_state` 54 MiB × 6 slots, slab
**17,548 slots** with 4.22 GiB of KV lent to it, sampler **9 rows**, template 28,234 B, vision
live (≤ 4,194,304 px, 16,384 patches/pass).

The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
the P2P answer per pair) — read it after every resize, and unlike every predecessor there is no
per-boot VRAM planner to second-guess. For comparison, the card's own suggested run line on 32
GB cards is `--host-pool-mib 12288 --expert-vs-cache-ratio 0.82` with seqs 8 and spec 3; the
shape above is the fleet's measured one. `FRANK_ARCHON_EXTRA_ARGS` is the boot-ladder escape
hatch (unset → no extra argv; it is expanded unquoted on purpose, so keep multi-flag values as
one string).

**Harness contract** (`/home/blank/.omp/agent/models.yml`) — provider `llamacpp (dankserver)`
→ `http://192.168.0.235:7770/v1`, `api: openai-completions`, `auth: none`, `id: frank-archon`
(= `FRANK_ARCHON_SERVED_NAME`; the provider *label* is legacy and renaming it breaks
`config.yml` references). `contextWindow: 200000` must equal `MAX_MODEL_LEN`
`maxTokens: 16384` (clamp `min(maxTokens, window − estimate − 4096)`; compaction fires at
85 % → 170,000 est × 1.03 + 16,384 ≤ 200,000); `input: [text, image]`
`compat.reasoningContentField: reasoning_content`; `reasoningEffortMap` minimal/low→low,
medium→medium, high/max→xhigh; `extraBody` temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
presence_penalty 0 / repetition_penalty 1.0 + `chat_template_kwargs {enable_thinking,
preserve_thinking}`, every key inside the engine's known-parameter list (unknowns 400).
`compactionModel` routes to `llamacpp (dankclaw-3090)/frank-zealot`.

**Deploy / verify** — `./scripts/deploy.sh dankserver --service frank-archon`; launch it
detached when the swap restarts the endpoint your own session runs on. Then verify
**behaviourally**, not by the green gate: `podman inspect frank-archon --format '{{json
.Config.Cmd}}'` and `/v1/models` — `deploy.sh` plans from repo paths and gates on the port, so
host-side unit drift is invisible to it.

## Eras

One row per retired era on this port. The **Rollback** cell is the load-bearing one: anything
marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `StillDeadcode`
`rad` and `tpl.jinja`.

| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---|
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
| 2026-10-05 → 10-08 | `qwen4exp-fp8-native.rad`, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation | radiance 1.0.7 line, same digest | — | native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, **1.93 sessions** — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot) | rebuild-only: `scripts/download-frank-archon.sh` (official `Qwen/Qwen3.8-Flash-Next-FP8`, 131 shards, 172.8 GB, ungated) + `scripts/build-frank-archon-rad.sh` (no-recipe convert under the descriptor law); raw **and** `.rad` were `rm`'d 2026-10-15 (Frank's call, to free the volume for the iq4r download) |
| 2026-10-05 (same day) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` safetensors, 131 shards | FreeToken PR #587 head `77a7201` (`rocm714_torch214`), image `localhost/frank-primary-freetoken:pr587-77a7201` | — | TP=2 cannot boot on ROCm: the per-rank `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201, so the second card idled | git tag `frank-archon-freetoken-pr587-20261005` (trio + `scripts/docker/frank-primary-freetoken.Dockerfile`, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account |
| 2026-09-29 → 10-05 (interrupted 09-30 → 10-01 by GLM) | same orcarouter FP8 safetensors | tcclaviger vLLM fork, `docker.io/tcclaviger/vllm@sha256:da402966…` | shape G: 3 × 200,704 fp8, KV 3,166,500,000 B/rank → 401,408 tokens, arena 17.77 GiB / 7,762 slots, util 0.982, MNBT 2048, no spec, no prefetch, prefix caching ON | radiance gave real TP=2 (no RCCL) and a 2-3 min boot, and the fork never fixed its long-context class | git tag `frank-archon-vllm-shapeG-20261003` (trio + the 362-line env lore); image still in dank's rootless storage; payload needs a re-fetch. The 2026-10-01 fp8+MTP4 rung is tag `frank-archon-fp8-mtp-20261001` |
| 2026-09-30 → 10-01 | `Intel/GLM-5.3-Flash-W4A16-AutoRound` ~170 GiB (sym int4 g128, `auto_gptq`) | same tcclaviger digest; `R4dW4a16MoEMethod` → libr4d grouped W4A16 | — | — | payload (170 GiB) + drafter (2.2 GiB) went to `~/.AGENTTRASH/20261001-glm-purge/` and were emptied; the cutover was reversed by `d91379c`, restoring the `2339fae` trio; `scripts/patches/glm53-*.patch` survive in-tree but none applies byte-exact since the 2026-10-10 rename rewrote their context lines |
| 2026-09-27 (same day) | `Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS`, 92.39 GiB + PLE 26.82 GiB (`34fa36f8…`) | same R9V bundle + shim | `MAX_MODEL_LEN=196608`, offload budgets 146.25 / 78.75 GiB, PLE residency `ssd` mmap, tool parser `qwen3_coder`, thinking on the `reasoning` (vLLM) channel | same-day flip back to the reference quant with the MTP4 head, at Frank's direction | `/home/dank/r9v-state/uncensored/` (`setup.json`, `restore-config-20260927.env`) is still on disk; the IQ4_XS payload was deleted in the 2026-10-17 cleanup → re-fetch, point `R9V_STATE_FILE` back, drop the two q8-only keys and the template keys |
| ≤ 2026-09 — 4 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete | — | — | — | — | —

Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era
pages are **not** superseded here:  is the locator for that
endpoint's whole shape history, including the unresolved end date of the
vllm-radiance era (this table opens on 09-26; the era page claims 10-10).

## Laws learned here

### The radiance engine

- **KV dtype law.** `--kv-cache-dtype` takes only `bf16|fp8` — ONE dtype for the whole cache,
  a **radiance** restriction, not stock vLLM's: upstream advertises `auto|fp8|fp8_e5m2|fp8_e4m3`
  (`docs.vllm.ai` engine-args) and even there the advertised set is not the usable set — several
  accepted values crash engine init at the attention layer's own dtype check
  ([vllm-project/vllm#42587](https://github.com/vllm-project/vllm/issues/42587)).
  So llama.cpp's exact-K / quantized-V pair (what [[frank-zealot]] runs as `--cache-type-k`/
  `--cache-type-v`) is unservable here: the rung below bf16 is fp8 on both halves. Measured
  per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot
  2026-10-11): `kv_attn` 13 layers / 4-token blocks pages **26.00 → 52.00 KiB**, while
  `kv_qsa_bkey` stays **3.25 KiB at both** — the QSA block key is an index, not attention state,
  and does not follow the flag. So 7.3125 KiB/token at fp8 vs 13.8125 at bf16: **1.89×, not
  2×**, precisely because the bkey plane is dtype-fixed. Secondary saving: a prefix-cache entry
  tracks the attention plane (110.50 KiB at bf16 → 58.50 at fp8, per the `kv tier transfer`
  lines), so the same 8 GiB host tier parks ~5.8 sessions of 200k at fp8 where it parked ~2.9 at bf16.
- **Pool sizing law.** `pool_mib ≥ tokens × 7.3125 KiB + 110 × seqs + 448` — the dtype-
  independent terms are also in the pool: `kv_gdn_state` 54 MiB × 2 copies per sequence (+conv/
  tail/ple ≈ 110 MiB a sequence) and the prefix checkpoint region 8 × 55.99 MiB. Deployed: 3 ×
  200,000 at fp8 = 4,285 + 330 + 448 = **5,063 MiB**, pool 5,120, which predicts (5120 − 330 −
  448) × 1024 / 7.3125 = **608,027 tokens** against a 600,000 contract (1.3 % slack) — every
  MiB above 5,063 was handed to the expert slab instead. At bf16 the same three slots cost
  8,093 + 778 = 8,871 and the slab falls to 20,249 (−9 %); four slots at bf16 cost 10,791 + 888
  = 11,679 and the slab falls to 17,441 (−33 %); fp8 keeps 22,208 of slab at both slot counts.
  **fp8 is not a preference, it is what funds the slot count.** Measured backings to sanity-
  check the formula: bf16 pool 6,912 at seqs 2 backed 462,752 tokens (2026-10-11), the fp8-era
  pool of 3,543 gave 386,904 (1.93 sessions), and the four-slot fp8 boot of 2026-10-13 19:21
  backed 843,276. The ledger's `backs N sequences all at 200000 tokens at once` line is the
  referee — a pair this tight is read off the boot, never assumed, and the concurrent probe is
  its regression test. Fourth full-height conversation needs 6,601 MiB of pool: refusing it is
  Frank's explicit call, and serving it is a `--vram-kv-mib` change, not a code change.
- **The hard limit is per request.** The engine treats `kv group tokens >= max-model-len` as a
  HARD limit — "one request longer than N tokens is accepted by the API and then refused as too
  large to ever fit — this is a hard limit, not a backing ratio" — so the pool is sized for
  `seqs × max-model-len`, not for one sequence. Radiance never truncates, at any size.
- **One VRAM budget per card:** `weights + kv ≤ 30,969 MiB`, read off the engine's own budget
  table (boot 2026-10-13 at pair 24000/5120): card 32,624 MiB, budgets claimed 29,120, and
  "*3.42 GiB is neither budgeted nor accounted — the driver's context, the activation arena and
  `--gpu-headroom-mib` come out of it; anything past those is throughput on the floor*", with
  the card settling 2.01 GiB free against a 209.19 MiB floor (96 headroom + 113.19 kept for the
  static region). Reserve above the engine's floor = 2,058 − 209 = 1,849, ceiling 29,120 + 1,849
  = 30,969; the deployed pair lands 458 MiB above that floor. The 2026-10-11 bound of 29,120
  ("31,856 − 1,710 outside both budgets − keep 1 GiB free") was a hand-picked reserve, not the
  engine's wall. Boot with the explicit `--vram-weights-mib`/`--vram-kv-mib` pair, never
  `--kv-cache-size-mib`; with the disk tier engaged the engine also forces ~9.6 GiB of weight
  buffers + arena onto the weights side (~29.6 GiB/card effective).
- **Idle KV VRAM is the expert cache's spillover — the price is paid twice.** `mover: 2275
  slots can be lent out of the KV cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <=
  256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 today)
  so a bigger KV pool steals weight residency from the slab AND from the lendable cache. Priors
  on this card/model: stealing 2.14 GiB of weights to buy `MAX_MODEL_LEN 262144` cost 2.5 %
  decode; the other direction (`--expert-vs-cache-ratio` 0.82 → 0.84, +2.5 GiB of weights) cost
  8.9 %.
- **A KV-pool cut is a runtime-headroom lever, never an arena lever.** The 6.1e9 → 3.15e9 B/rank pool cut
  (2.75 GiB/rank) bought **zero** extra expert-cache slots. The arena moves with the reserve, one-for-one
  (d(reserve) 0.57 GiB = d(arena) 0.57 GiB = **+248 slots**). The shape that pool cut replaced died with
  `0 bytes free` on a real 995-token request — that is headroom, not slot count. Read `expert cache space … / N
  slots` out of the boot ledger before trusting any pool arithmetic: the 2026-10-13 read was **14.87 GiB /
  6,496 slots** against `GPU KV cache size: 398,658 tokens`, and it is the POOL that caps resident
  conversations, not the arena.
- **MNBT moves the compile RANGE, not the batch width.** The engine sets `num_reqs = min(max_num_batched_tokens,
  max_num_reqs)`, so the CUDA-graph compile width is **SEQS + 1** for any `MNBT >= SEQS` — 3 at SEQS 2, 4 at
  SEQS 3, exactly the constant printed in the engine's own message. MNBT only shifts the compiled range
  ((1,1024) versus (1,2048)), which is why the identical capture failure appears at both rungs;
  `--enforce-eager` bypasses the path and loses CUDA graphs with it.
- **The prefix cache proves itself at depth, not in the ledger.** Two concurrent conversations of 196,919 tokens
  each answered the marker buried at the head of **their own** prompt with **393,837 tokens resident** — above the
  fp8 pool's 386,904, so the answer came from a cached prefix and not a re-prefill. Same boot (2026-10-13):
  decode at 51.5k depth **94.1 cold / 102.3 warm / 117.6 prefix-hit t/s**, draft acceptance **0.7284 / 0.7525**.
- **The host pool moves with the slab.** `vram_weights` and `host_pool_mib` serve the per-rank
  resident set *together*: at the fp8-KV pair (26071/3543) the ledger showed the slab full to
  259 KiB of slack, and cutting `vram-weights` by 3.8 GiB to pay for a bf16 pool spills ~3.8
  GiB/rank into host RAM — under-sizing the pool there is a **boot failure** (36,864 refused the
  boot in the fp8 era). The ledger's coverage line, not the arithmetic, arbitrates.
- **Slab → prefill: the arms disagree, the cliff does not.** 2026-10-11 exact-KV A/B (slab
  22,208 vs the fp8-era 26,071, −14.8 %): prefill −29 %, TTFT at the full window +25 %, decode
  inside the noise band on 2 of 3 passes, draft acceptance UP 0.661 → 0.740. 2026-10-13
  (`probe-frank-archon-kv.py`, env header): A-cold 43.5k 22,208 → 3,397, 24,000 → 3,290, 25,600
  → **3,584** t/s, and the 3-way 197k arm's aggregate 3,051 → 3,589 (wall 193.7 → 164.5 s), so
  +1,600 MiB of slab bought +8.9 % single-stream / +17.6 % concurrent — read then as a SLOPE,
  the 22,208/24,000 pair having landed inside noise of each other and falsely looked like a
  plateau. Re-measured at 3 slots after the 2026-10-15 swap the 25,600 arm returned 3,290: the
  3,584 did **not** reproduce, so treat 22,208-25,600 as one band until someone re-runs the
  ladder. What is not disputed is the cliff: 26,071 → 4,379 t/s (the 2026-10-11 control at seqs
  3 / pool 3,543), +22 % for +471 MiB — unreachable at 3 slots (26,112 + 5,120 = 31,232 vs
  30,969) and free at 2 (pool 3,072 + slab 27,897 = the ceiling exactly). Two points inside
  noise of each other are not a plateau, and two points are not a curve.
- **Decode is nearly window-free; slots are paid in concurrency.** 99.0 t/s at 43.5k vs 93.3 at
  197k (−6 % for 4.5× depth), while three streams give 113.9 / 43.3 / 10.9 = 18.4 t/s aggregate
  — a fused-step lottery, not a fair share. Read it from leg D of `probe-frank-archon-kv.py`
  with ≥ 128 generated tokens; an 8-token probe measures first-token noise.
- **MNBT 2048.** 2560 collapses decode to 2.6-7.7 t/s (MTP resonance); the w4 file tier streams
  in place only at MNBT ≤ 2048 and the activation arena scales with the chunk (32768 → 33.3
  GiB, refused).
- **Slots cost KV and rows, never speed.** `--max-num-seqs` carries no radiance constraint: it
  adds 110 MiB of recurrent state per slot and declares the fused TP all-reduce over `seqs ×
  (1 + spec)` rows, and `wht6` is bit-exact up to 25 rows (single-sequence decode and any step
  of ≤ 25 rows is exact; only busier steps are lossy) — 3 × 3 = 9 today. Measured 2026-10-06:
  seqs 2 and seqs 3 are statistically tied (30.9-33.1 t/s, inside the ±3 % floor). The "seqs 2
  cannot boot" law belonged to the retired tcclaviger fork's `torch.compile` shape
  specialization, and radiance booted seqs 2 on its first fp8-native start.
  **Same era, separate law: a dequantize-to-compute path bills host RAM twice.** On the
  EMULATION backend an offload request of **118 GiB** landed as **~238 GiB** of real host RAM —
  the dequantized copy rides on top of the pinned weights, and the doubling is invisible in the
  weight count. Read the engine log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding
  "doesn't fit"; the container-side procedure and the kernel gate are .
- **One encoding per expert table** (`rad_arch_declare`): a whole expert kept bf16 is the only
  thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable.
- **Descriptor-selection law.** The plugin is picked by the exact `(arch_id, quant descriptor)`
  pair with no fallback: a raw official FP8 checkpoint (`quant_method: "fp8"`) is refused by
  serve *and* convert, because the shipped `qwen4exp_fp8` plugin claims `(qwen4exp, "")`. Stage
  a `config.json` with `quantization_config` dropped — the descriptor changes, the data stays FP8.
- **Three convert laws, each paid in a full verify+declare cycle.** Scale-name: `scale_name()`
  appends `_scale_inv` to the *tensor* name, so stacked `…experts.gate_up_proj` keeps its scales
  at `…gate_up_proj_scale_inv`. File-shadow: the reader opens index-named files sorted and
  emplaces every tensor each file physically holds, first-wins — a merged PLE shard must sort
  first (`00-ple.merged.safetensors`) or a raw shard wins the name. Symlink-through: staging
  symlinks the raw index, so `open(path,"w")` writes through it and corrupts the checkpoint —
  write temp + `os.replace`.
- **Convert is card-less** (`rad-convert` declares on CPU, GPTQ solves on CPU BLAS) and
  `--plan-only` goes first: a plan that resolves every op will boot. `rad-tune` must be
  declared at the shapes you serve and its cache lives in the *image* (`$RADIANCE_HOME/tune`)
  so it only survives on the `$FRANK_ARCHON_STATE_DIR/tune` bind-mount the serve script adds.
- **Quant gates are scored at the production shape** (MNBT 2048, not the recording's 32768)
  against the BF16 container: KL ≤ 0.03, top-1 ≥ 95 %, and the verdict is chunk-size invariant
  (arm Δ ≈ the 0.0015 float-order band) and placement-invariant. Protected experts are
  data-driven — energy share ≥ 3 % AND ≥ 3× routing share, even count per rank half — and the
  PLE rides the checkpoint's own fixed scale (1.9931793212890625e-04, absmax inside the
  2⁻¹³..2⁻¹⁰ plateau).

### This host, and the deploy path

- **The pinned-RAM ledger is why this is a root unit:** 2 × 44 GiB expert pools + the 47.7 GiB
  PLE n-gram table (51B params, read from pinned host RAM over io_uring — Frank's pin law:
  "*Native FP8. Experts from ram. Pin the PLE to ram.*") + the 8 GiB prefix host tier ≈ **147
  GiB** charged against `RLIMIT_MEMLOCK`; dank's login session caps at 8 MiB and a user quadlet
  is capped by the user manager, so only PID1 can grant `infinity` — and
  `--ngram-placement ram` refuses the start when the limit is short.
- **TTM is runtime state, and the gate that owns it raises before it refuses.** The pinned pages
  ride `/sys/module/ttm/parameters/pages_limit` (+ `pages_max` on kernels that expose one; this
  kernel exposes **none** — the cap is the MIN over the knobs present, and an absent knob is
  skipped, never read as zero, because a zero would refuse a healthy cap). The kernel default is
  **half of RAM**, so every raise dies with the reboot: the 2026-10-05 manual raise to
  52,428,800 pages (200 GiB) was gone after the **2026-10-12 23:17 power failure**, the cap fell
  to 125,988 MiB, and the preflight refused the start **6,919 times over ~20 h** while
  `Restart=always` cheerfully re-ran a refusal. A manual sysfs write is therefore not an
  acceptable repair procedure — `scripts/ttm-preflight.sh` writes `TTM_TARGET_MIB` (204,800 MiB
  = 52,428,800 pages; pages are 4 KiB so pages = MiB × 256; the *recorded host ceiling*, because
  writing the 150,000 floor would leave the ledger no slack) into every knob present, re-reads,
  and only then refuses. Write failures are swallowed deliberately: the caller re-reads, and ttm
  itself refuses a limit below the pages it has already pinned.
- **Privilege of that knob, measured the hard way:** it is root-owned 0644 and the service runs
  `User=dank`, where `-w` is false and a plain `echo` fails. The canonical privileged path is the
  unit's `ExecStartPre=+…` (the `+` runs it as root even under `User=`); `sudo -n` is the
  fallback for a human. The file is *sourced* by the serve script (which re-verifies the cap
  before `podman run`) and *executed* by the unit. A refusal is invisible outside the journal —
  no container is created, the unit sits in `activating (auto-restart)` on one repeating line —
  and without the gate the load instead dies mid-pin with a bare allocator error.
- **The prefix host tier needs real RAM:** 8 GiB of it, and with the service loaded the host
  still has ~104 GiB available (246 GiB total) — that headroom is what the pinned ledger is
  measured against.
- **The deployer's blind spot is host-side unit drift.** A leftover
  `frank-archon.service.d/bench.conf` overrode both `FRANK_ARCHON_ENV_FILE` and `ExecStart` to a
  staging worktree and every `deploy.sh` since was green against the old configuration —
  drop-ins are read *after* the unit's own `Environment=` lines, so they win.
- **Never `systemctl disable` a unit on these hosts** — it removes the `/etc/systemd/system/`
  entry, which here is the symlink into the checkout. `mask` is reversible but *fails on a
  relinked unit*, so a bench window uses `stop` and the deploy restores the unit.
- **The restore oracle for an experiment window is a conjunction:** `is-active` == active AND
  `is-enabled` == enabled AND no `/run/systemd/system/<unit>` mask symlink AND the container is
  owned by the unit and not `Stopping`. The vLLM/GLM-era engines ignored SIGTERM and answered
  `/health` 200 for 20+ minutes while `Stopping`; a runtime mask survives `stop` and blocks
  `start`, and a failed unit needs `reset-failed` first.
- **`MemAvailable` is an input, not a backdrop:** the expert-offload planner reads it, so
  anything booted beside the live service dies at plan time (`ExpertPlanError: host holds 5113
  of 12096 swappable experts`) — and fleet law forbids GPU experiments beside production
  anyway. Long remote work runs as a transient unit or `setsid`+disown: `nohup X &` inside `ssh`
  races ssh teardown and can die before writing a byte.
- **The ~40k-token corruption class was the RAM, not the engine.** Dropping DDR5 from 4800 to
  4600 MT/s ended it, with zero `gdn_hip` faults either side; the prefix-caching / GDN re-seed
  theory was confounded on both arms. Consumer non-ECC raises no MCE for a silent flip and a
  60 s / 8 GiB pattern check covers 0.5 % of 246 GiB — silent RAM corruption outranks every
  software theory.
- **TP=2 here** is legal since the 2026-09-20 ProArt B850 swap (x8/x8 on CPU root ports,
  `Routing+`/`ReqEn+` end to end) only because radiance's wire is `libr4d`'s own one-shot
  all-reduce; RCCL needs atomics on the whole chain and fails `hipErrorIllegalState` without
  them (). `wht6` buys +5.6-6.3 % prefill for ~1.3 % of 8-stream
  decode, `exact` is the bit-exact alternative.
- `--prefix-cache-dir` creation is **not recursive** — the serve script `mkdir -p`s the path or
  the disk tier silently never opens. The tier does survive a payload swap when arch and
  tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; `cache_n` proves a hit).

### Dead ends — do not re-try

- **radiance 1.2.0 as the serve pin** (`sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa`
  commit `e95d397`), benched 2026-10-06 on this exact shape with a 43.6k greedy probe: decode
  29.21 t/s and draft acceptance 0.7123 vs the 1.0.13 control's 30.08 / 0.7549, and the control
  reproduced the recorded 0.755 exactly — greedy acceptance is deterministic, so the drop is a
  real numerics change (the decode loss tracks the acceptance ratio, 0.943 vs 0.945) with the
  1.2.0 MoE weight-rotation change `92b8aaf` the named suspect. The vram-pair translation itself
  is exact (the 1.2.0 ledger took 25.46 GiB weights / 3.46 GiB KV per card, byte-identical to
  1.0.13 at EVC 0.82), and 1.2.0 REMOVED `--expert-vs-cache-ratio` in favour of the pair, which
  both pins accept. Re-test 1.2.0+ once the rotation change is untangled.
- **FreeToken TP=2 on ROCm:** the `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201 —
  an untested edge of that PR, not a board fault.
- **tcclaviger forks:** `--max-num-seqs 2` (`ConstraintViolationError` on `query_start_loc`)
  `--gpu-memory-utilization 1.00` unbootable, 0.982 is the law; expert prefetch is poison (36 %
  precision — deleting it was +40-51 % decode); TunableOp-with-offload corrupts numerics
  (repetition loops, early EOS) — autotuning that changes text is broken tuning; ngram-only
  drafting dies at init (`PLE inputs were not prepared`); DCP needs TP > `num_kv_heads` (2 here,
  so 4 GPUs); 6 × 262,144 fits no KV dtype (6 sequences leave 9.18 GiB of pool where fp8 needs
  12.47). fp8 KV on that path also removes the only MLA decode gfx12 has (the aiter op is gated
  on CDNA > 2, so the draft step demands `…mla_decode_fwd`, which cannot exist there) and its
  KV reserve model is dtype-blind (identical 13,600 B/token plan line) — no capacity for the breakage.
- **DeepSeek-V4.1-Flash on 32 GiB cards:** only `--ds4-prefill exact` is legal,
  `--ds4-fused-decode` is refused, and at ~22 % resident experts each token pulls ~187 × 12 MiB
  from NVMe — low GPU util with a pegged link is a *capacity verdict*. Two corrections survive:
  expert TP **does** work on gfx1201+gfx1201, and an expert-placement JSON outside the mounted
  `/models` tree gives a silent single-tier fallback.
- **Which experts you pin does not matter; how many does.** An IBS census of the live server
  (`scripts/frank-archon-expert-activity.py`) measured per-block read volume flat at 0.93-1.08×
  of mean across every CPU-resident block — top-k routing fires all of them every token, so at
  `-ot` granularity the lever is COUNT.
- **Never mount a diagnostic hook on `PYTHONPATH` without a marker guard:** `sitecustomize` is
  imported by every interpreter, a child inherits the hook and spawns again — a 19-level python
  fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation)
  never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions.

## Contract (the wire)

- **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions`
  `/v1/completions`, `/metrics`, `/stats`, `/sessions`, `/dashboard`. `/health` answers 200
  JSON (model id, `in_flight`, `queue_depth`) while serving and **503 only while stopping**
  (`core/server/admin.cpp:385`), and the listener binds *after* the full load (`main.cpp`:
  `engine.start()` then `engine.serve()`) — during the load the port refuses connections, so a
  bare `curl -fsS …/health` gate is honest and needs no body grep.
- **Served id.** `FRANK_ARCHON_SERVED_NAME=frank-archon` (the wire rename of 2026-10-09 moved
  only the alias; container, unit, scripts and env keep the service name). The env's contract
  line is that **radiance validates the requested model name, so every client — harness, bench
  scripts, probes — MUST send it** — while the era pages also record that a chat response
  *echoes* the requested name, which is why `/v1/models` is the served-id oracle and a 200 chat
  probe proves nothing about what is actually loaded.
- **Reasoning** lands in `reasoning_content` (`--reasoning-format` defaults to `auto`; no
  parser flags exist or are needed) and `usage` bills it separately (`reasoning_tokens`).
  **Thinking consumes `max_tokens`**, so the classic false negative is a probe with empty
  `content` and `finish_reason: length`/`stop` because the whole budget went inside the think
  block: either set `chat_template_kwargs.enable_thinking=false` or budget a few hundred tokens
  and read `reasoning_content`. Same trap, sharper, on the vision probe below.
- **Sampler surface:** `temperature, top_p, top_k, min_p, typical_p, presence_penalty,
  frequency_penalty, repetition_penalty, repeat_last_n, chat_template_kwargs, reasoning_effort,
  enable_thinking, preserve_thinking` accepted; **unknown parameters are refused 400** (nothing
  unverified can ride the wire, and an `extraBody` typo fails loudly); `max_tokens` is clamped
  not refused; `response_format: json_object` honoured; **`logprobs` refused**; no `seed` = the
  engine picks; `n` defaults to `max(8, max-num-seqs)`; stop strings ≤ 64 × 4096 B.
- **Tools:** the qwen3.8 dialect from the chat template → `message.tool_calls` +
  `finish_reason: tool_calls`. **`reasoning_effort`:** the froggeric template does not gate the
  set (xhigh/high/medium/low all 200); the harness map is a safety net, not a requirement.
- **Vision is native** (the checkpoint's 27-layer Qwen-VL tower through the engine's Qwen-VL
  processor): an `image_url` content part, no legacy top-level `images` field, ≤ 4,194,304 px
  and 16,384 patches per pass. Probe with a ≥ 256 px solid-colour PNG and a ≥ 256 token budget
  — at 64 tokens you get empty `content` and the colour only inside `reasoning_content`.
- **Capacity behaviour:** a prompt over `--max-model-len` is refused, never truncated; a request
  longer than the KV pool's token count is accepted then refused as too large to fit; under the
  3-slot contract a **fourth** full-height conversation is refused, not paged or squeezed.
- **Observability:** `/metrics` carries vLLM-compatible names plus `radiance:*`, scraped by the
  Dankclaw Prometheus job at `192.168.0.235:7770` over plain LAN routing (the netavark
  published-port quirk is same-host only). The boot ledger — the referee for every shape number
  above — is in the container log (`podman logs frank-archon`)
  `/home/dank/logs/frank_archon.log` is where the serve script redirects `podman run`.

## Files

- Shape and law owner: `scripts/frank-archon.env` (measured arithmetic in its header)
  `scripts/serve-frank-archon.sh` (gates image / `.rad` size / template / GPU nodes, retires
  stale containers, `podman wait`), `scripts/frank-archon.service`, `scripts/ttm-preflight.sh`.
- Payload doors: `scripts/download-frank-archon-iq4r.sh` (live), `scripts/download-frank-archon.sh`
  + `scripts/build-frank-archon-rad.sh` + `scripts/stack-qwen38-official-fp8.py` (the fp8-native
  rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert
  needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template
  `scripts/qwen38-froggeric-v22.5.jinja`.
- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh`
  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh`
  `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py`
  `scripts/make-calib-corpus.py`, `scripts/derive-protected-experts.py`
  `scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`.
- Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so
  it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`
  `scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`.
- Deploy and tests: `scripts/deploy.sh` / `scripts/deploy-remote.sh` (`deploy_llm_archon`)
  `tests/frank_archon_unit.bats` (the seams gating the KV-pool arithmetic, the VRAM pair and the
  ttm gate), `tests/changed_tests.bats`, `tests/deployment_scoping.bats`.
- Retired-era leftovers in-tree: `scripts/patches/glm53-*.patch`
  `scripts/patches/gpu_worker-kv-cap.patch` (fork archaeology; none applies byte-exact today).

Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
---