serverAppConfigs/serverAppConfigs-docs/LLM/frank-archon.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)	The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught.	llm frank-archon dankserver radiance gfx1201 r9700 tp2 expert-offload endpoint	2026-10-18

frank-archon — radiance (StillDeadcode), 2× R9700 TP=2, Dankserver :7770

The radiance engine (C++/HIP, libr4d kernels built for gfx1201, no Python in the serving path) driving podman run straight from the digest-pinned image under a root systemd unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI PRO R9700s. Capacity contract 3 conversations × 200,000 tokens at fp8 KV; a payload swap is a MODEL_FILE line plus a re-download — see Eras.

Current shape

Authoritative source: scripts/frank-archon.env + scripts/serve-frank-archon.sh (the env header carries the measured arithmetic beside each number).

Payload — /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad, 121,969,901,568 B = 116,322 MiB, sha256 0af5e962… (the Hub LFS oid), from StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe: the engine author's own rad-convert of the official (censored) Qwen/Qwen3.8-Flash-Next bf16 checkpoint. Encodings (recipe qwen4exp-w4nl64-i8-hc8m): routed experts as 4-bit codes in a 16-level non-uniform codebook (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scale per 64 weights); 10 whole bf16-protected experts per layer; int8 W8A8 trunk + lm_head; fp8 HC mixtures; PLE n-gram fp8 at the checkpoint's fixed scale (47.7 GiB) MTP head and the bf16 27-layer vision tower in the container. Card quality vs bf16: mean KL 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %. The abliterated family lives on frank-dragoon. Fetched by scripts/download-frank-archon-iq4r.sh --fetch (hfget resume + declared size + sha256) as the transient unit iq4r-fetch (log /home/dank/logs/iq4r-fetch.log, …rc sentinel); the staging dir /home/dank/models/StillDeadcode/… keeps card, LICENSE and recipe as provenance, and only .rad files are mounted.

Runtime — stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e = pin 1.0.13 (Codeberg v1.0.13, commit 53282fa): a 287 MiB image in dank's rootless storage, host needs nothing but the amdgpu driver. Pin the INDEX manifest digest, never the image ID (69d05724… is the config digest and sends podman run to the network). The serve script's gate is that digest ref itself — podman image exists name@sha256:… in one command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike or a newer latest cannot satisfy it. 1.2.0 is a rejected serve pin (Laws); the --vram-weights-mib/--vram-kv-mib pair is accepted by both pins, which is the rollback symmetry.

Trio — scripts/frank-archon.env (every number, with its arithmetic in the header) scripts/serve-frank-archon.sh, scripts/frank-archon.service. Gates run in order: image digest present → .rad present and ≥ RAD_MIN_MIB (a truncated convert would otherwise buy a guaranteed long-timeout boot instead of this fast failure) → template present → /dev/kfd + /dev/dri → ttm_ensure_cap (the serve script re-verifies the cap after the unit's hook) → podman rm -f over FRANK_ARCHON_LEGACY_CONTAINERS (frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe) plus its own name, because systemd SIGKILLs conmon on a crash and leaves the name registered, and every one of those identities holds this GPU. Container contract (publisher-vetted, docs/DOCKER.md): --network host, --init (so the engine releases the cards on SIGTERM), --security-opt seccomp=unconfined (the default profile refuses io_uring — the PLE table's reads — and the NUMA calls the ROCm runtime makes for host memory), --ulimit memlock=-1:-1, /models ro + /data rw + the tune cache at /opt/radiance/share/radiance/tune template as a single-file mount, then exec podman wait: the script's lifetime IS the container's, so a crash is a nonzero ExecStart and the restart policy is real. Unit: After=network-online.target, WantedBy=multi-user.target, User=dank LimitMEMLOCK=infinity, StartLimitIntervalSec=0, OOMScoreAdjust=-500 (the last process the OOM killer should pick), Restart=always not on-failure — a clean API-server exit after a worker fault reads as success to systemd (the 2026-10-01 vLLM lesson) — and TimeoutStopSec=180 > the ExecStop=podman stop -t 120 grace, because systemd's 90 s default cut the stop short and recorded Failed with result timeout (measured 2026-10-01 13:27:17). No drain contract is claimed: the OpenAI front end holds no queue, so an in-flight decode is lost on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute start_period).

Knob (env name, FRANK_ARCHON_ dropped)	Live value
TP / wire	TP=2, TP_WIRE=wht6 — libr4d one-shot all-reduce, not RCCL; min-kb stays the engine default
Slots / window	MAX_SEQS=3, MAX_MODEL_LEN=200000 (native 262,144; longer is refused, never truncated)
KV dtype	KV_DTYPE=fp8 — the only dtype that funds 3 slots (Laws)
VRAM pair	VRAM_WEIGHTS_MIB=25600 + VRAM_KV_MIB=5120 = 30,720 of the 30,969 ceiling; GPU_HEADROOM_MIB=96
Expert tier	PLACEMENT=expert_tiered, HOST_POOL_MIB=44928 per rank
PLE	NGRAM_PLACEMENT=ram (47.7 GiB pinned; refuses boot on a short memlock)
Spec / batch	—
Prefix cache	PREFIX_HOST_MIB=8192 + PREFIX_DISK_MIB=65536 under STATE_DIR=/home/dank/radiance-state (mounted rw at /data)
Template	TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja (28,234 B) via --override-chat-template — froggeric's fixed Qwen3.8 template (the repo root's chat_template.jinja, vendored); the checkpoint's own template is baked into the .rad by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it
TTM gate	TTM_MIN_MIB=150000, TTM_TARGET_MIB=204800
Boot	RAD_MIN_MIB=110000 deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s)

Boot ledger for this shape (2026-10-15, pin 1.0.13): pool 5.00 GiB backs 607,740 tokens (kv_attn fp8 page 26.00 KiB, kv_qsa_bkey 3.25 KiB), kv_gdn_state 54 MiB × 6 slots, slab 17,548 slots with 4.22 GiB of KV lent to it, sampler 9 rows, template 28,234 B, vision live (≤ 4,194,304 px, 16,384 patches/pass).

The boot ledger, not this arithmetic, is the referee: radiance states every budget before anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin, the P2P answer per pair) — read it after every resize, and unlike every predecessor there is no per-boot VRAM planner to second-guess. For comparison, the card's own suggested run line on 32 GB cards is --host-pool-mib 12288 --expert-vs-cache-ratio 0.82 with seqs 8 and spec 3; the shape above is the fleet's measured one. FRANK_ARCHON_EXTRA_ARGS is the boot-ladder escape hatch (unset → no extra argv; it is expanded unquoted on purpose, so keep multi-flag values as one string).

Harness contract (/home/blank/.omp/agent/models.yml) — provider llamacpp (dankserver) → http://192.168.0.235:7770/v1, api: openai-completions, auth: none, id: frank-archon (= FRANK_ARCHON_SERVED_NAME; the provider label is legacy and renaming it breaks config.yml references). contextWindow: 200000 must equal MAX_MODEL_LEN maxTokens: 16384 (clamp min(maxTokens, window − estimate − 4096); compaction fires at 85 % → 170,000 est × 1.03 + 16,384 ≤ 200,000); input: [text, image] compat.reasoningContentField: reasoning_content; reasoningEffortMap minimal/low→low, medium→medium, high/max→xhigh; extraBody temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 / presence_penalty 0 / repetition_penalty 1.0 + chat_template_kwargs {enable_thinking, preserve_thinking}, every key inside the engine's known-parameter list (unknowns 400). compactionModel routes to llamacpp (dankclaw-3090)/frank-zealot.

Deploy / verify — ./scripts/deploy.sh dankserver --service frank-archon; launch it detached when the swap restarts the endpoint your own session runs on. Then verify behaviourally, not by the green gate: podman inspect frank-archon --format '{{json .Config.Cmd}}' and /v1/models — deploy.sh plans from repo paths and gates on the port, so host-side unit drift is invisible to it.

Eras

One row per retired era on this port. The Rollback cell is the load-bearing one: anything marked "re-fetch" is gone from /home/dank/models/, which today holds only StillDeadcode rad and tpl.jinja.

Window	Payload	Runtime	Shape	Replaced because	Rollback
2026-10-08 → 10-15	in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %)	radiance 1.0.13, same digest	3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries	the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint	.rad purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700 → qwen4exp-w4nl64-i8-hc8m.rad, 122,071,973,888 B, LFS sha256 09dd9306…, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point MODEL_FILE at it, redeploy. Re-convert is dead: the BF16 source was rm'd 2026-10-15 (kld-ref, grams, corpora and /home/dank/logs/swift15_*.log survive)
2026-10-05 → 10-08	qwen4exp-fp8-native.rad, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation	radiance 1.0.7 line, same digest	—	native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, 1.93 sessions — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot)	rebuild-only: scripts/download-frank-archon.sh (official Qwen/Qwen3.8-Flash-Next-FP8, 131 shards, 172.8 GB, ungated) + scripts/build-frank-archon-rad.sh (no-recipe convert under the descriptor law); raw and .rad were rm'd 2026-10-15 (Frank's call, to free the volume for the iq4r download)
2026-10-05 (same day)	orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 safetensors, 131 shards	FreeToken PR #587 head 77a7201 (rocm714_torch214), image localhost/frank-primary-freetoken:pr587-77a7201	—	TP=2 cannot boot on ROCm: the per-rank pynccl.cu JIT cannot resolve cuda_runtime.h for gfx1201, so the second card idled	git tag frank-archon-freetoken-pr587-20261005 (trio + scripts/docker/frank-primary-freetoken.Dockerfile, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account
2026-09-29 → 10-05 (interrupted 09-30 → 10-01 by GLM)	same orcarouter FP8 safetensors	tcclaviger vLLM fork, docker.io/tcclaviger/vllm@sha256:da402966…	shape G: 3 × 200,704 fp8, KV 3,166,500,000 B/rank → 401,408 tokens, arena 17.77 GiB / 7,762 slots, util 0.982, MNBT 2048, no spec, no prefetch, prefix caching ON	radiance gave real TP=2 (no RCCL) and a 2-3 min boot, and the fork never fixed its long-context class	git tag frank-archon-vllm-shapeG-20261003 (trio + the 362-line env lore); image still in dank's rootless storage; payload needs a re-fetch. The 2026-10-01 fp8+MTP4 rung is tag frank-archon-fp8-mtp-20261001
2026-09-30 → 10-01	Intel/GLM-5.3-Flash-W4A16-AutoRound ~170 GiB (sym int4 g128, auto_gptq)	same tcclaviger digest; R4dW4a16MoEMethod → libr4d grouped W4A16	—	—	payload (170 GiB) + drafter (2.2 GiB) went to ~/.AGENTTRASH/20261001-glm-purge/ and were emptied; the cutover was reversed by d91379c, restoring the 2339fae trio; scripts/patches/glm53-*.patch survive in-tree but none applies byte-exact since the 2026-10-10 rename rewrote their context lines
2026-09-27 (same day)	Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS, 92.39 GiB + PLE 26.82 GiB (34fa36f8…)	same R9V bundle + shim	MAX_MODEL_LEN=196608, offload budgets 146.25 / 78.75 GiB, PLE residency ssd mmap, tool parser qwen3_coder, thinking on the reasoning (vLLM) channel	same-day flip back to the reference quant with the MTP4 head, at Frank's direction	/home/dank/r9v-state/uncensored/ (setup.json, restore-config-20260927.env) is still on disk; the IQ4_XS payload was deleted in the 2026-10-17 cleanup → re-fetch, point R9V_STATE_FILE back, drop the two q8-only keys and the template keys
≤ 2026-09 — 4 earlier eras: payloads, shapes and every measured number destroyed 2026-10-19 as obsolete	—	—	—	—	—

Before 2026-09-26 the port belonged to the frank-medium service, and its era pages are not superseded here: is the locator for that endpoint's whole shape history, including the unresolved end date of the vllm-radiance era (this table opens on 09-26; the era page claims 10-10).

Laws learned here

The radiance engine

• KV dtype law. --kv-cache-dtype takes only bf16|fp8 — ONE dtype for the whole cache, a radiance restriction, not stock vLLM's: upstream advertises auto|fp8|fp8_e5m2|fp8_e4m3 (docs.vllm.ai engine-args) and even there the advertised set is not the usable set — several accepted values crash engine init at the attention layer's own dtype check (vllm-project/vllm#42587). So llama.cpp's exact-K / quantized-V pair (what frank-zealot runs as --cache-type-k/ --cache-type-v) is unservable here: the rung below bf16 is fp8 on both halves. Measured per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot 2026-10-11): kv_attn 13 layers / 4-token blocks pages 26.00 → 52.00 KiB, while kv_qsa_bkey stays 3.25 KiB at both — the QSA block key is an index, not attention state, and does not follow the flag. So 7.3125 KiB/token at fp8 vs 13.8125 at bf16: 1.89×, not 2×, precisely because the bkey plane is dtype-fixed. Secondary saving: a prefix-cache entry tracks the attention plane (110.50 KiB at bf16 → 58.50 at fp8, per the kv tier transfer lines), so the same 8 GiB host tier parks ~5.8 sessions of 200k at fp8 where it parked ~2.9 at bf16.
• Pool sizing law. pool_mib ≥ tokens × 7.3125 KiB + 110 × seqs + 448 — the dtype- independent terms are also in the pool: kv_gdn_state 54 MiB × 2 copies per sequence (+conv/ tail/ple ≈ 110 MiB a sequence) and the prefix checkpoint region 8 × 55.99 MiB. Deployed: 3 × 200,000 at fp8 = 4,285 + 330 + 448 = 5,063 MiB, pool 5,120, which predicts (5120 − 330 − 448) × 1024 / 7.3125 = 608,027 tokens against a 600,000 contract (1.3 % slack) — every MiB above 5,063 was handed to the expert slab instead. At bf16 the same three slots cost 8,093 + 778 = 8,871 and the slab falls to 20,249 (−9 %); four slots at bf16 cost 10,791 + 888 = 11,679 and the slab falls to 17,441 (−33 %); fp8 keeps 22,208 of slab at both slot counts. fp8 is not a preference, it is what funds the slot count. Measured backings to sanity- check the formula: bf16 pool 6,912 at seqs 2 backed 462,752 tokens (2026-10-11), the fp8-era pool of 3,543 gave 386,904 (1.93 sessions), and the four-slot fp8 boot of 2026-10-13 19:21 backed 843,276. The ledger's backs N sequences all at 200000 tokens at once line is the referee — a pair this tight is read off the boot, never assumed, and the concurrent probe is its regression test. Fourth full-height conversation needs 6,601 MiB of pool: refusing it is Frank's explicit call, and serving it is a --vram-kv-mib change, not a code change.
• The hard limit is per request. The engine treats kv group tokens >= max-model-len as a HARD limit — "one request longer than N tokens is accepted by the API and then refused as too large to ever fit — this is a hard limit, not a backing ratio" — so the pool is sized for seqs × max-model-len, not for one sequence. Radiance never truncates, at any size.
• One VRAM budget per card: weights + kv ≤ 30,969 MiB, read off the engine's own budget table (boot 2026-10-13 at pair 24000/5120): card 32,624 MiB, budgets claimed 29,120, and "3.42 GiB is neither budgeted nor accounted — the driver's context, the activation arena and --gpu-headroom-mib come out of it; anything past those is throughput on the floor", with the card settling 2.01 GiB free against a 209.19 MiB floor (96 headroom + 113.19 kept for the static region). Reserve above the engine's floor = 2,058 − 209 = 1,849, ceiling 29,120 + 1,849 = 30,969; the deployed pair lands 458 MiB above that floor. The 2026-10-11 bound of 29,120 ("31,856 − 1,710 outside both budgets − keep 1 GiB free") was a hand-picked reserve, not the engine's wall. Boot with the explicit --vram-weights-mib/--vram-kv-mib pair, never --kv-cache-size-mib; with the disk tier engaged the engine also forces ~9.6 GiB of weight buffers + arena onto the weights side (~29.6 GiB/card effective).
• Idle KV VRAM is the expert cache's spillover — the price is paid twice. mover: 2275 slots can be lent out of the KV cache (2.69 GiB), arena: lends 740 MiB while steps are <= 256 tokens; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 today) so a bigger KV pool steals weight residency from the slab AND from the lendable cache. Priors on this card/model: stealing 2.14 GiB of weights to buy MAX_MODEL_LEN 262144 cost 2.5 % decode; the other direction (--expert-vs-cache-ratio 0.82 → 0.84, +2.5 GiB of weights) cost 8.9 %.
• A KV-pool cut is a runtime-headroom lever, never an arena lever. The 6.1e9 → 3.15e9 B/rank pool cut (2.75 GiB/rank) bought zero extra expert-cache slots. The arena moves with the reserve, one-for-one (d(reserve) 0.57 GiB = d(arena) 0.57 GiB = +248 slots). The shape that pool cut replaced died with 0 bytes free on a real 995-token request — that is headroom, not slot count. Read expert cache space … / N slots out of the boot ledger before trusting any pool arithmetic: the 2026-10-13 read was 14.87 GiB / 6,496 slots against GPU KV cache size: 398,658 tokens, and it is the POOL that caps resident conversations, not the arena.
• MNBT moves the compile RANGE, not the batch width. The engine sets num_reqs = min(max_num_batched_tokens, max_num_reqs), so the CUDA-graph compile width is SEQS + 1 for any MNBT >= SEQS — 3 at SEQS 2, 4 at SEQS 3, exactly the constant printed in the engine's own message. MNBT only shifts the compiled range ((1,1024) versus (1,2048)), which is why the identical capture failure appears at both rungs; --enforce-eager bypasses the path and loses CUDA graphs with it.
• The prefix cache proves itself at depth, not in the ledger. Two concurrent conversations of 196,919 tokens each answered the marker buried at the head of their own prompt with 393,837 tokens resident — above the fp8 pool's 386,904, so the answer came from a cached prefix and not a re-prefill. Same boot (2026-10-13): decode at 51.5k depth 94.1 cold / 102.3 warm / 117.6 prefix-hit t/s, draft acceptance 0.7284 / 0.7525.
• The host pool moves with the slab. vram_weights and host_pool_mib serve the per-rank resident set together: at the fp8-KV pair (26071/3543) the ledger showed the slab full to 259 KiB of slack, and cutting vram-weights by 3.8 GiB to pay for a bf16 pool spills ~3.8 GiB/rank into host RAM — under-sizing the pool there is a boot failure (36,864 refused the boot in the fp8 era). The ledger's coverage line, not the arithmetic, arbitrates.
• Slab → prefill: the arms disagree, the cliff does not. 2026-10-11 exact-KV A/B (slab 22,208 vs the fp8-era 26,071, −14.8 %): prefill −29 %, TTFT at the full window +25 %, decode inside the noise band on 2 of 3 passes, draft acceptance UP 0.661 → 0.740. 2026-10-13 (probe-frank-archon-kv.py, env header): A-cold 43.5k 22,208 → 3,397, 24,000 → 3,290, 25,600 → 3,584 t/s, and the 3-way 197k arm's aggregate 3,051 → 3,589 (wall 193.7 → 164.5 s), so +1,600 MiB of slab bought +8.9 % single-stream / +17.6 % concurrent — read then as a SLOPE, the 22,208/24,000 pair having landed inside noise of each other and falsely looked like a plateau. Re-measured at 3 slots after the 2026-10-15 swap the 25,600 arm returned 3,290: the 3,584 did not reproduce, so treat 22,208-25,600 as one band until someone re-runs the ladder. What is not disputed is the cliff: 26,071 → 4,379 t/s (the 2026-10-11 control at seqs 3 / pool 3,543), +22 % for +471 MiB — unreachable at 3 slots (26,112 + 5,120 = 31,232 vs 30,969) and free at 2 (pool 3,072 + slab 27,897 = the ceiling exactly). Two points inside noise of each other are not a plateau, and two points are not a curve.
• Decode is nearly window-free; slots are paid in concurrency. 99.0 t/s at 43.5k vs 93.3 at 197k (−6 % for 4.5× depth), while three streams give 113.9 / 43.3 / 10.9 = 18.4 t/s aggregate — a fused-step lottery, not a fair share. Read it from leg D of probe-frank-archon-kv.py with ≥ 128 generated tokens; an 8-token probe measures first-token noise.
• MNBT 2048. 2560 collapses decode to 2.6-7.7 t/s (MTP resonance); the w4 file tier streams in place only at MNBT ≤ 2048 and the activation arena scales with the chunk (32768 → 33.3 GiB, refused).
• Slots cost KV and rows, never speed. --max-num-seqs carries no radiance constraint: it adds 110 MiB of recurrent state per slot and declares the fused TP all-reduce over seqs × (1 + spec) rows, and wht6 is bit-exact up to 25 rows (single-sequence decode and any step of ≤ 25 rows is exact; only busier steps are lossy) — 3 × 3 = 9 today. Measured 2026-10-06: seqs 2 and seqs 3 are statistically tied (30.9-33.1 t/s, inside the ±3 % floor). The "seqs 2 cannot boot" law belonged to the retired tcclaviger fork's torch.compile shape specialization, and radiance booted seqs 2 on its first fp8-native start. Same era, separate law: a dequantize-to-compute path bills host RAM twice. On the EMULATION backend an offload request of 118 GiB landed as ~238 GiB of real host RAM — the dequantized copy rides on top of the pinned weights, and the doubling is invisible in the weight count. Read the engine log for Using 'EMULATION' Mxfp4 MoE backend before concluding "doesn't fit"; the container-side procedure and the kernel gate are .
• One encoding per expert table (rad_arch_declare): a whole expert kept bf16 is the only thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable.
• Descriptor-selection law. The plugin is picked by the exact (arch_id, quant descriptor) pair with no fallback: a raw official FP8 checkpoint (quant_method: "fp8") is refused by serve and convert, because the shipped qwen4exp_fp8 plugin claims (qwen4exp, ""). Stage a config.json with quantization_config dropped — the descriptor changes, the data stays FP8.
• Three convert laws, each paid in a full verify+declare cycle. Scale-name: scale_name() appends _scale_inv to the tensor name, so stacked …experts.gate_up_proj keeps its scales at …gate_up_proj_scale_inv. File-shadow: the reader opens index-named files sorted and emplaces every tensor each file physically holds, first-wins — a merged PLE shard must sort first (00-ple.merged.safetensors) or a raw shard wins the name. Symlink-through: staging symlinks the raw index, so open(path,"w") writes through it and corrupts the checkpoint — write temp + os.replace.
• Convert is card-less (rad-convert declares on CPU, GPTQ solves on CPU BLAS) and --plan-only goes first: a plan that resolves every op will boot. rad-tune must be declared at the shapes you serve and its cache lives in the image ($RADIANCE_HOME/tune) so it only survives on the $FRANK_ARCHON_STATE_DIR/tune bind-mount the serve script adds.
• Quant gates are scored at the production shape (MNBT 2048, not the recording's 32768) against the BF16 container: KL ≤ 0.03, top-1 ≥ 95 %, and the verdict is chunk-size invariant (arm Δ ≈ the 0.0015 float-order band) and placement-invariant. Protected experts are data-driven — energy share ≥ 3 % AND ≥ 3× routing share, even count per rank half — and the PLE rides the checkpoint's own fixed scale (1.9931793212890625e-04, absmax inside the 2⁻¹³..2⁻¹⁰ plateau).

This host, and the deploy path

• The pinned-RAM ledger is why this is a root unit: 2 × 44 GiB expert pools + the 47.7 GiB PLE n-gram table (51B params, read from pinned host RAM over io_uring — Frank's pin law: "Native FP8. Experts from ram. Pin the PLE to ram.") + the 8 GiB prefix host tier ≈ 147 GiB charged against RLIMIT_MEMLOCK; dank's login session caps at 8 MiB and a user quadlet is capped by the user manager, so only PID1 can grant infinity — and --ngram-placement ram refuses the start when the limit is short.
• TTM is runtime state, and the gate that owns it raises before it refuses. The pinned pages ride /sys/module/ttm/parameters/pages_limit (+ pages_max on kernels that expose one; this kernel exposes none — the cap is the MIN over the knobs present, and an absent knob is skipped, never read as zero, because a zero would refuse a healthy cap). The kernel default is half of RAM, so every raise dies with the reboot: the 2026-10-05 manual raise to 52,428,800 pages (200 GiB) was gone after the 2026-10-12 23:17 power failure, the cap fell to 125,988 MiB, and the preflight refused the start 6,919 times over ~20 h while Restart=always cheerfully re-ran a refusal. A manual sysfs write is therefore not an acceptable repair procedure — scripts/ttm-preflight.sh writes TTM_TARGET_MIB (204,800 MiB = 52,428,800 pages; pages are 4 KiB so pages = MiB × 256; the recorded host ceiling, because writing the 150,000 floor would leave the ledger no slack) into every knob present, re-reads, and only then refuses. Write failures are swallowed deliberately: the caller re-reads, and ttm itself refuses a limit below the pages it has already pinned.
• Privilege of that knob, measured the hard way: it is root-owned 0644 and the service runs User=dank, where -w is false and a plain echo fails. The canonical privileged path is the unit's ExecStartPre=+… (the + runs it as root even under User=); sudo -n is the fallback for a human. The file is sourced by the serve script (which re-verifies the cap before podman run) and executed by the unit. A refusal is invisible outside the journal — no container is created, the unit sits in activating (auto-restart) on one repeating line — and without the gate the load instead dies mid-pin with a bare allocator error.
• The prefix host tier needs real RAM: 8 GiB of it, and with the service loaded the host still has ~104 GiB available (246 GiB total) — that headroom is what the pinned ledger is measured against.
• The deployer's blind spot is host-side unit drift. A leftover frank-archon.service.d/bench.conf overrode both FRANK_ARCHON_ENV_FILE and ExecStart to a staging worktree and every deploy.sh since was green against the old configuration — drop-ins are read after the unit's own Environment= lines, so they win.
• Never systemctl disable a unit on these hosts — it removes the /etc/systemd/system/ entry, which here is the symlink into the checkout. mask is reversible but fails on a relinked unit, so a bench window uses stop and the deploy restores the unit.
• The restore oracle for an experiment window is a conjunction: is-active == active AND is-enabled == enabled AND no /run/systemd/system/<unit> mask symlink AND the container is owned by the unit and not Stopping. The vLLM/GLM-era engines ignored SIGTERM and answered /health 200 for 20+ minutes while Stopping; a runtime mask survives stop and blocks start, and a failed unit needs reset-failed first.
• MemAvailable is an input, not a backdrop: the expert-offload planner reads it, so anything booted beside the live service dies at plan time (ExpertPlanError: host holds 5113 of 12096 swappable experts) — and fleet law forbids GPU experiments beside production anyway. Long remote work runs as a transient unit or setsid+disown: nohup X & inside ssh races ssh teardown and can die before writing a byte.
• The ~40k-token corruption class was the RAM, not the engine. Dropping DDR5 from 4800 to 4600 MT/s ended it, with zero gdn_hip faults either side; the prefix-caching / GDN re-seed theory was confounded on both arms. Consumer non-ECC raises no MCE for a silent flip and a 60 s / 8 GiB pattern check covers 0.5 % of 246 GiB — silent RAM corruption outranks every software theory.
• TP=2 here is legal since the 2026-09-20 ProArt B850 swap (x8/x8 on CPU root ports, Routing+/ReqEn+ end to end) only because radiance's wire is libr4d's own one-shot all-reduce; RCCL needs atomics on the whole chain and fails hipErrorIllegalState without them (). wht6 buys +5.6-6.3 % prefill for ~1.3 % of 8-stream decode, exact is the bit-exact alternative.
• --prefix-cache-dir creation is not recursive — the serve script mkdir -ps the path or the disk tier silently never opens. The tier does survive a payload swap when arch and tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; cache_n proves a hit).

Dead ends — do not re-try

• radiance 1.2.0 as the serve pin (sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa commit e95d397), benched 2026-10-06 on this exact shape with a 43.6k greedy probe: decode 29.21 t/s and draft acceptance 0.7123 vs the 1.0.13 control's 30.08 / 0.7549, and the control reproduced the recorded 0.755 exactly — greedy acceptance is deterministic, so the drop is a real numerics change (the decode loss tracks the acceptance ratio, 0.943 vs 0.945) with the 1.2.0 MoE weight-rotation change 92b8aaf the named suspect. The vram-pair translation itself is exact (the 1.2.0 ledger took 25.46 GiB weights / 3.46 GiB KV per card, byte-identical to 1.0.13 at EVC 0.82), and 1.2.0 REMOVED --expert-vs-cache-ratio in favour of the pair, which both pins accept. Re-test 1.2.0+ once the rotation change is untangled.
• FreeToken TP=2 on ROCm: the pynccl.cu JIT cannot resolve cuda_runtime.h for gfx1201 — an untested edge of that PR, not a board fault.
• tcclaviger forks: --max-num-seqs 2 (ConstraintViolationError on query_start_loc) --gpu-memory-utilization 1.00 unbootable, 0.982 is the law; expert prefetch is poison (36 % precision — deleting it was +40-51 % decode); TunableOp-with-offload corrupts numerics (repetition loops, early EOS) — autotuning that changes text is broken tuning; ngram-only drafting dies at init (PLE inputs were not prepared); DCP needs TP > num_kv_heads (2 here, so 4 GPUs); 6 × 262,144 fits no KV dtype (6 sequences leave 9.18 GiB of pool where fp8 needs 12.47). fp8 KV on that path also removes the only MLA decode gfx12 has (the aiter op is gated on CDNA > 2, so the draft step demands …mla_decode_fwd, which cannot exist there) and its KV reserve model is dtype-blind (identical 13,600 B/token plan line) — no capacity for the breakage.
• DeepSeek-V4.1-Flash on 32 GiB cards: only --ds4-prefill exact is legal, --ds4-fused-decode is refused, and at ~22 % resident experts each token pulls ~187 × 12 MiB from NVMe — low GPU util with a pegged link is a capacity verdict. Two corrections survive: expert TP does work on gfx1201+gfx1201, and an expert-placement JSON outside the mounted /models tree gives a silent single-tier fallback.
• Which experts you pin does not matter; how many does. An IBS census of the live server (scripts/frank-archon-expert-activity.py) measured per-block read volume flat at 0.93-1.08× of mean across every CPU-resident block — top-k routing fires all of them every token, so at -ot granularity the lever is COUNT.
• Never mount a diagnostic hook on PYTHONPATH without a marker guard: sitecustomize is imported by every interpreter, a child inherits the hook and spawns again — a 19-level python fork bomb inside the container. Kill with podman rm -f (the cgroup owns every generation) never pattern-kill an interpreter name on this host; use TORCH_LOGS for dynamo questions.

Contract (the wire)

• Live surface: /health (== /ping), /v1/models, /v1/chat/completions /v1/completions, /metrics, /stats, /sessions, /dashboard. /health answers 200 JSON (model id, in_flight, queue_depth) while serving and 503 only while stopping (core/server/admin.cpp:385), and the listener binds after the full load (main.cpp: engine.start() then engine.serve()) — during the load the port refuses connections, so a bare curl -fsS …/health gate is honest and needs no body grep.
• Served id. FRANK_ARCHON_SERVED_NAME=frank-archon (the wire rename of 2026-10-09 moved only the alias; container, unit, scripts and env keep the service name). The env's contract line is that radiance validates the requested model name, so every client — harness, bench scripts, probes — MUST send it — while the era pages also record that a chat response echoes the requested name, which is why /v1/models is the served-id oracle and a 200 chat probe proves nothing about what is actually loaded.
• Reasoning lands in reasoning_content (--reasoning-format defaults to auto; no parser flags exist or are needed) and usage bills it separately (reasoning_tokens). Thinking consumes max_tokens, so the classic false negative is a probe with empty content and finish_reason: length/stop because the whole budget went inside the think block: either set chat_template_kwargs.enable_thinking=false or budget a few hundred tokens and read reasoning_content. Same trap, sharper, on the vision probe below.
• Sampler surface: temperature, top_p, top_k, min_p, typical_p, presence_penalty, frequency_penalty, repetition_penalty, repeat_last_n, chat_template_kwargs, reasoning_effort, enable_thinking, preserve_thinking accepted; unknown parameters are refused 400 (nothing unverified can ride the wire, and an extraBody typo fails loudly); max_tokens is clamped not refused; response_format: json_object honoured; logprobs refused; no seed = the engine picks; n defaults to max(8, max-num-seqs); stop strings ≤ 64 × 4096 B.
• Tools: the qwen3.8 dialect from the chat template → message.tool_calls + finish_reason: tool_calls. reasoning_effort: the froggeric template does not gate the set (xhigh/high/medium/low all 200); the harness map is a safety net, not a requirement.
• Vision is native (the checkpoint's 27-layer Qwen-VL tower through the engine's Qwen-VL processor): an image_url content part, no legacy top-level images field, ≤ 4,194,304 px and 16,384 patches per pass. Probe with a ≥ 256 px solid-colour PNG and a ≥ 256 token budget — at 64 tokens you get empty content and the colour only inside reasoning_content.
• Capacity behaviour: a prompt over --max-model-len is refused, never truncated; a request longer than the KV pool's token count is accepted then refused as too large to fit; under the 3-slot contract a fourth full-height conversation is refused, not paged or squeezed.
• Observability: /metrics carries vLLM-compatible names plus radiance:*, scraped by the Dankclaw Prometheus job at 192.168.0.235:7770 over plain LAN routing (the netavark published-port quirk is same-host only). The boot ledger — the referee for every shape number above — is in the container log (podman logs frank-archon) /home/dank/logs/frank_archon.log is where the serve script redirects podman run.

Files

• Shape and law owner: scripts/frank-archon.env (measured arithmetic in its header) scripts/serve-frank-archon.sh (gates image / .rad size / template / GPU nodes, retires stale containers, podman wait), scripts/frank-archon.service, scripts/ttm-preflight.sh.
• Payload doors: scripts/download-frank-archon-iq4r.sh (live), scripts/download-frank-archon.sh
  • scripts/build-frank-archon-rad.sh + scripts/stack-qwen38-official-fp8.py (the fp8-native rebuild), scripts/download-frank-archon-swift15.sh (the BF16 source a w4dn8 re-convert needs), scripts/hfget.sh / scripts/hf-fetch.sh; template scripts/qwen38-froggeric-v22.5.jinja.
• Convert campaign (dormant, kept for a future quant): scripts/convert-swift15-w4dn8.sh scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe, scripts/run-swift15-bf16.sh scripts/kld-gate-swift15.sh, scripts/kld-then-calib.sh, scripts/calib-push.py scripts/make-calib-corpus.py, scripts/derive-protected-experts.py scripts/apply-protected-cast.sh, scripts/verify-swift15-ple.py.
• Probes: scripts/probe-frank-archon-kv.py (reads slots/window/dtype from the deployed env so it cannot drift; leg D = decode), scripts/frank-archon-expert-activity.py scripts/autoresearch-frank-archon.sh + scripts/autoresearch/frank-archon-measure.py.
• Deploy and tests: scripts/deploy.sh / scripts/deploy-remote.sh (deploy_llm_archon) tests/frank_archon_unit.bats (the seams gating the KV-pool arithmetic, the VRAM pair and the ttm gate), tests/changed_tests.bats, tests/deployment_scoping.bats.
• Retired-era leftovers in-tree: scripts/patches/glm53-*.patch scripts/patches/gpu_worker-kv-cap.patch (fork archaeology; none applies byte-exact today).

Related: frank-dragoon (the abliterated family), frank-zealot,