serverAppConfigs/serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md

frank-archon — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2

Status: RETIRED — superseded 2026-10-15 by the publisher iq4r payload. · Lived 2026-10-08 → 10-15 (main 3ca64e1, e2e 33/33). · Host: Dankserver (192.168.0.235) · Port: 7770 · GPU: 2× Radeon AI PRO R9700 (gfx1201), TP=2

The FP8-native era (qwen4exp-fp8-native.rad, 173.0 GiB, no-recipe convert) is replaced by a GPTQ-calibrated 4-bit container of ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, abliterated finetune) converted with the recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe on engine 1.0.13 (stilldeadcode/radiance@sha256:4c3f484564… — same digest as the FP8-native era; 1.2.0 was used ONLY for the calibration serve). Same port, alias, container name, systemd unit, and every serving argument carried into the successor: frank-archon-radiance-qwen38-iq4r. The .rad itself was purged from disk 2026-10-17; the byte-exact payload lives on the Hub — see Rollback.

Why this swap

• Context economics at production memory budgets: the FP8-native experts (56.4 GiB/rank slice) forced a 40 GiB/rank pinned host pool and left KV ~3.5 GiB. W4Dn8 experts cut the resident set to ~1/4, freeing host and VRAM budget for KV and slots at equal-or-better measured decode (gate: ≥ 30.8 t/s vs the 30.08 t/s control).
• Swift 1.5 base: abliterated finetune of Flash-Next (Frank's fleet-personality call; frank-dragoon serves the same family).
• GPTQ closes the precision gap: RTN alone on BF16 weights lands measurably worse; GPTQ with Hessians from a 10M-token corpus of this model's own distribution recovers most of it, verified end-to-end by a KLD gate against the BF16 container itself (KL ≤ 0.03, top-1 ≥ 95 %).

The payload chain

1. Fetch: scripts/download-frank-archon-swift15.sh → /home/dank/models/Swift1.5-Qwen3.8-Flash-Next (131 shards, 335.3 GiB, per-shard LFS sha256 verified, no token on the command line). 48 layers, 512 experts (top-10 routed), moe_inter 640, hidden 2560, PLE 20M-vocab table 95.37 GiB, MTP head, vision tower.
2. BF16 container: run-swift15-bf16.sh convert → rad-convert no-recipe (pure copy) at --max-ctx 4096 → qwen4exp-swift15-bf16.rad (335.28 GiB). Serves on 1.0.13 as the KLD reference engine (disk-tier placement; boot ~35 min).
3. KLD reference: run-swift15-bf16.sh kld (1.0.13) → /home/dank/corpora/kld-ref — radiance-kld-1 format, rows: 50235, 197 source-labelled docs, top-16 log-probs (f16), corpus fnv1a64 a9d62b33c18b3d9c. ~4 h at 32k prefill steps (--max-num-batched-tokens 32768 — a 65536 step OOMs an arena tensor at --max-model-len 16384). The wrapper's exit line is the receipt: never scp this script while it runs (bash lazy-read; a mid-run swap killed a clean engine exit with rc=127, third fleet occurrence of the script-swap law). kld-then-calib.sh SKIP_KLD=1 resumes from the rows: 50235 receipt.
4. Calibration: run-swift15-bf16.sh calib on 1.2.0 (the only engine with RADIANCE_CALIB_TOKENS; grams are engine-invariant bf16 activations — RADGRAM2 format byte-identical across versions)
   • calib-push.py pushing calib-corpus.jsonl (10,014,932 tokens, 1,294 prompts, assembled from real HF sources, chat-templated, source-labelled). The tap (in the qwen4exp_fp8.so architecture plugin) writes per-expert down-proj Gram matrices gram.r<rank>.blk.L.ffn_down_exps.<e>.bin (32-byte RADGRAM2 header + 640×640 f32; rows = routing count, trace = activation energy) and stops the serve at the token threshold. TP=2: whole experts per rank (256/rank); gate_up input is rank-sliced (plugin warning — the recipe only casts the down projection).
5. Protected-expert derivation: derive-protected-experts.py /home/dank/corpora/calib → the 24 highest-energy experts (the law is data-driven; Swift drew 24, not the fp8-era 10 — its routing is spikier: blk.47 e445 alone carries 32 % of its layer's down energy at 70× routing share) promoted to whole-expert bf16. Protection law: energy share ≥ 3 % AND ≥ 3× routing share; even count per rank half; odd half gains the next most energetic. apply-protected-cast.sh splices the derived names between the # PROTECTED-BEGIN/END markers of the recipe (refuses double-derive).
6. Final convert: convert-swift15-w4dn8.sh (1.0.13, host-backend CPU-only — no GPU visible to rad-convert; GPTQ solve is CPU BLAS) — gates: ≥48 gram files present, bf16 container already deleted (disk), ≥146 GiB free, output ≥108,000 MiB, rad-info shows the gptq receipt. No --reuse. Measured: 5 h 50 m (6.0–6.5 min/layer on 49 GPTQ layers, load-average 8), output 116,416 MiB. The plugin's rad_arch_declare enforces ONE ENCODING PER EXPERT TABLE (one table of experts takes one encoding, and only a whole expert kept plain bf16 is served beside it) — a split tier (down=fp8, gate_up=w4) is unservable: the kernel serves a single w4nl64a8h table; both expert tables ride GPTQ w4 (down calibrated, gate_up uncalibrated), protected experts bf16 beside them.

The recipe tiers (scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe)

Tier	Encoding	Why
MoE experts, both gate_up and down (one w4 table/layer)	GPTQ w4nl64a8h — u4 codes, fwht128 rotation, fp8 scales/64 + f32 scale plane	the campaign tier: GPTQ on down-input Hessians from the 10M-token corpus (the tap rides the down projection; gate_up shares the table uncalibrated — one-encoding-per-table plugin law)
24 derived protected experts (whole)	bf16	energy-share law (step 5)
shared expert + trunk attention/SSM linears	int8, scale per 128 cols, searched	trunk precision law
hyper-cond + MTP head	fp8_e4m3 row-groups	hc+mtp fp8 law
lm_head	int8	i8 lm_head law (E4M3 costs 0.6 top-1 alone)
PLE n-gram table	fp8 at the checkpoint's fixed scale 1.9931793212890625e-04	the fixed-scale law; absmax audit passed the 2⁻¹³..2⁻¹⁰ plateau
embeddings, vision tower, QSA indexer	bf16 passthrough	untouched

The shape (scripts/frank-archon.env)

TP=2 + wht6, --max-model-len 200000, MNBT 2048 (the MTP resonance law — 2560 collapses decode to 2.6–7.7 t/s), --ngram-placement ram, froggeric v22.5 template override. MODEL_FILE points at /home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad (116,416 MiB measured); the serve size floor FRANK_ARCHON_RAD_MIN_MIB moves 177000 → 110000. Concurrency and the KV cache moved on 2026-10-11; that change is documented in its own section below.

2026-10-11 — unquantized KV at 2 × 200,000

The ask: two concurrent 200,000-token conversations on exact (unquantized) attention state. The fp8 pool backed 3,543 MiB → 386,904 tokens = 1.93 sessions of the served window, so two full-height conversations overflow it and the scheduler preempts the overflow to the host/disk prefix tiers (re-prefilling on return). Measured, that overflow is survivable — the fp8 shape passed the same 2 × 197k pair probe — so this is a headroom change, not the repair of a hard failure. The hard limit the engine states is per request, not per pair: a request longer than the pool's token count is "accepted by the API and then refused as too large to ever fit — this is a hard limit, not a backing ratio", and at 200,000 ≤ 386,904 no single request ever hit it.

The dtype law

--kv-cache-dtype takes only bf16 | fp8: one kv_dtype for the whole cache. llama.cpp's exact-K / quantized-V pair (what frank-zealot runs as --cache-type-k f16 --cache-type-v q8_0) has no radiance equivalent — the rung under bf16 is fp8 on both halves. Evidence: the flag table in the image itself (--kv-cache-dtype takes bf16 or fp8, not '%s') and the absence of any cache-type-k/-v symbol in the binary.

bf16 costs 1.89× fp8, not 2×, and the reason is in the group table:

Group	fp8 page	bf16 page	cost per token, per rank
kv_attn — 13 full-attention layers, 4-token blocks	26.00 KiB	52.00 KiB	6.5 → 13.0 KiB
kv_qsa_bkey — 13 layers, the QSA indexer keys	3.25 KiB	3.25 KiB	0.8125 KiB, dtype-fixed
kv_gdn_state 54 MiB × 2 copies per sequence (+ conv/tail/ple)	per slot	per slot	≈ 110 MiB per sequence
prefix checkpoints 8 × 55.99 MiB (inside the pool)	448 MiB	448 MiB	fixed

→ pool_mib ≥ tokens × 13.8125 KiB + 110 × seqs + 448. The formula reproduces both live boots: fp8 (3543 − 330 − 448) → 387,240 vs the ledger's 386,904; bf16 (6912 − 220 − 448) → 462,946 vs the ledger's 462,752. It is encoded as a test seam (the KV pool backs seqs x max-model-len at the declared cache dtype) so a future resize cannot silently under-provision the pool.

Why the weights slab pays — and why that is not pure loss

The two VRAM budgets are one budget, and the ceiling is the engine's own — not a hand-picked reserve. Card 32,624 MiB, and the table that prints at the pair 24,000/5,120 says 3.42 GiB is neither budgeted nor accounted — the driver's context, the activation arena and --gpu-headroom-mib come out of it; anything past those is throughput on the floor, with card 0 settled at 2.01 GiB free against 209.19 MiB (the 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region). Claimable reserve above the ENGINE's floor: 2,058 − 209 = 1,849 MiB ⇒ weights + kv ≤ 30,969 MiB. (The 29,120 bound written on 2026-10-11 was my own 1 GiB "keep free" reserve, measured off a card figure of 31,856 MiB; the engine never asked for that reserve and the seam now derives the number from its table.) Paying for a 6,912 MiB pool therefore cuts the resident slab 26,071 → 22,208 MiB (−14.8 %), and the evicted experts spill into the pinned host pool (40,960 → 44,928 MiB per rank; ledger: pinned pool slots 7,190 → 10,375). The pinned-RAM ledger rises to ≈147 GiB, so the TTM floor moved 140,000 → 150,000 MiB (the kernel cap is 204,800).

Half of that loss is refunded by an engine behaviour worth knowing: idle KV VRAM is the expert cache's spillover. The fp8 pool lent 2.69 GiB to the expert slab; the bf16 pool lends 6.08 GiB (mover: 5146 slots can be lent out of the KV cache). On a llama.cpp endpoint KV is inert storage; on radiance it is a competing cache, and every KV resize is also an expert-residency resize.

seqs 3 → 2

--max-num-seqs 3 was never a radiance requirement: it is the tcclaviger vLLM fork's torch.compile constraint (ConstraintViolationError: Constraints violated (L['query_start_loc'].size()[0])), retired with that runtime — see dankserver-qwen38-fp8-expert-offload-tuning. Radiance booted seqs 2 in the fp8-native era, and the 2026-10-06 decode sweep measured seqs 2 and 3 tied inside the ±3 % noise floor. Two slots × the window is the capacity contract; it also saves 110 MiB of per-slot recurrent state and drops the fused TP all-reduce to 2 × 3 = 6 rows, i.e. deeper inside the 25-row bound at which the wht6 wire is bit-exact.

Boot ledger (deployed 2026-10-11, commit 0bf26b0)

kv cache           6.75 GiB   349% of it -- backs 2 sequences all at 200000
                                tokens at once, which is the worst case
vram_weights      21.69 GiB   stated by --vram-weights-mib
vram_kv            6.75 GiB   stated by --vram-kv-mib
unclaimed          2.03 GiB
kv_attn      full 13 layers block 4 tok page 52.00 KiB 115688 blocks = 5.74 GiB (462752 tokens)
kv_qsa_bkey  full 13 layers block 4 tok page  3.25 KiB 115688 blocks = 367.17 MiB (462752 tokens)
load: rank 0, mover took 21.69 GiB, weights pool 147.00 KiB free of 21.69 GiB
mover: 5146 slots can be lent out of the KV cache (6.08 GiB)
card 0 settled at 1.85 GiB free
serving on http://:::7770

Verification (live service, no scratch container)

Check	Result
/health, /v1/models	200 {"status":"ok","model":"frank-archon"}; served id frank-archon
2 × ~197k concurrent, a unique marker buried at the head of each	both answered their own marker exactly — 196,919 + 196,918 = 393,837 tokens resident, above the fp8 pool's 386,904 and inside bf16's 462,752; walls 138.1 s / 66.2 s, finish_reason: stop, prefill 2,747 / 2,991 t/s
decode at 51.5k depth, 512 tokens, greedy	94.1 cold / 102.3 warm / 117.6 on a prefix-cache hit t/s; draft acceptance 0.7284 / 0.7525 (fp8 control at cutover: 0.7246)
fp8 control arm, same probes (scripts/probe-frank-archon-kv.py)	table in the next section

Probe law re-learned, worth recording: radiance routes the thinking block to reasoning_content, so a chat request whose budget is spent inside <think> comes back with empty content and finish_reason: stop — a probe that forbids thinking (chat_template_kwargs.enable_thinking=false) and reads both fields is the only honest shape. The first pair probe scored 0/2 on exactly that artifact, not on a KV fault.

Measured cost — A/B on the live service

Method (no scratch container, no mask — the unit file is a relink symlink and mask refuses it): write a variant env file on the host (/home/dank/frank-archon-fp8control.env), point a transient drop-in at it (Environment=FRANK_ARCHON_ENV_FILE=…; drop-ins are read AFTER the unit's own Environment= lines, so they win), daemon-reload, restart, probe, delete the drop-in, restart. Control = the deployed 2026-10-08 shape, verified from podman inspect argv (--kv-cache-dtype fp8 --vram-weights-mib 26071 --vram-kv-mib 3543 --max-num-seqs 3) and the ledger page size (26.00 KiB). Identical probes, same salts, loopback, greedy, 512 generated tokens.

Probe	fp8 control (26071/3543, seqs 3)	bf16 deployed (22208/6912, seqs 2)	Δ
prefill 51.5k, cold	4,379 t/s	3,126 t/s	−29 %
prefill 51.5k, warm	4,864 t/s	3,333 t/s	−31 %
decode 51.5k, cold	124.1 t/s	94.1 t/s	−24 %
decode 51.5k, warm	99.2 t/s	102.3 t/s	+3 %
decode on a prefix-cache hit	127.7 t/s	117.6 t/s	−8 %
draft acceptance, mean of passes	0.661	0.740	+12 %
2 × 196,919 concurrent pair	both markers HIT, 110.5 s	both markers HIT, 138.1 s	+25 % wall

Read it as: the price of exact KV on this engine is prefill, not decode. Prefill is expert-bandwidth-bound and the resident slab is 14.8 % smaller, so time-to-first-token at the full window rises ~25 % (the pair: 110 s → 138 s); both prefill passes agree, far outside the ±2 % prefill noise floor. Decode is inside the ±3 % band on two of three passes (warm +3 %, hit −8 %); the cold-pass −24 % is the largest single number, the least reproducible, and it moves the same direction as prefill — the cold arm is paying expert streaming, not attention. Spec-decode acceptance IMPROVED ~8 points (0.661 → 0.740): exact attention state makes the target agree with the MTP drafter more often, which pays on every token of every conversation, and it is the strongest argument for the shape.

Open call (Frank): keep exact KV at the prefill cost, or take the cheap fp8 variant — --vram-kv-mib 3,648 + --max-num-seqs 2 fits 2 × 200,000 at fp8 for a 105 MiB pool bump and ≈ no decode cost. The tool is committed: scripts/probe-frank-archon-kv.py (run it on Dankserver; it rebuilds its prompts from the repo checkout there).

Rollback

Revert scripts/frank-archon.env to the fp8 pair (26071/3543, fp8, seqs 3, host pool 40960, prefix host 4096) and redeploy: no payload change, boot ~2 min. The KLD gate scored at the fp8-KV shape, so a bf16-KV production runs away from the scored configuration — bf16 can only reduce the attention-state error, and a re-score is a drained-window job if the gate agreement is ever needed in the record.

2026-10-13 — the power failure, the ttm reset, and 4 × 200,000 fp8 KV

The outage: a refusal, not a crash

Dankserver lost power on 2026-10-12 ~23:17. The box came back; frank-archon never did. The unit sat in activating (auto-restart), exiting 1 about 50 ms after starting, with one journal line on repeat:

ERROR: ttm pages cap is 125988 MiB, the pinned ledger needs >= 150000
frank-archon: raise /sys/module/ttm/parameters/pages_limit …

6,919 refusals in ~20 hours. No container was created (podman ps -a showed the name with a zero start time), so there was no engine, no port, and no engine log — the signature of a preflight refusal, which every deployer/monitor view outside the journal renders as an unexplained crash loop.

The cause is a class, not an incident. The pinned-RAM ledger (~147 GiB: 2 × 44 GiB expert pools + the 47.7 GiB PLE table + the 8 GiB prefix host tier) rides TTM's shared pages, capped by /sys/module/ttm/parameters/pages_limit. That knob is runtime state, and its default is half of RAM — so the manual raise of 2026-10-05 (52,428,800 pages = 200 GiB) died with the power failure and the cap fell back to 32,253,039 pages = 125,988 MiB, 24 GiB below the floor the serve script enforces. The fp8-native era had planned the persistent half of this ("add ttm.pages_limit=52428800 to the kernel cmdline at the next scheduled reboot", frank-archon-radiance-qwen38-native-fp8); it was never applied, and a power failure is not scheduled.

The repair: the gate raises the knob itself

scripts/ttm-preflight.sh (sourced by serve-frank-archon.sh) reads the cap as the MIN over the knobs the running kernel exposes, and when it is below FRANK_ARCHON_TTM_MIN_MIB it writes FRANK_ARCHON_TTM_TARGET_MIB (204,800 MiB = 52,428,800 pages) into every writable knob, re-reads, and only then refuses. The unit is root and the knob is a 0644 root-owned module parameter, so the write is legal; a kernel that refuses the move produces the same named refusal as before, and a reboot is now a boot rather than a visit to the machine.

Knob	Power-failure default	After the gate
/sys/module/ttm/parameters/pages_limit	32,253,039 pages = 125,988 MiB	52,428,800 pages = 204,800 MiB

Seam-tested, not inspection-tested: the ttm gate raises a reset pages cap and refuses when the write fails and the ttm cap counts only the knobs the kernel exposes drive the gate against a temp knob directory — a below-floor writable knob ends at the target and the start passes; a read-only knob leaves the cap untouched and the start refuses by name; an absent pages_max is skipped, never read as zero (a zero reading would refuse a healthy host).

The shape: 4 slots × 200,000, fp8 KV (same day, superseded by the re-cuts below)

Answers the open call of 2026-10-11 from the other side: Frank asked for four concurrent 200,000-token conversations with quantized KV.

fp8 is not a preference at four slots, it is the only dtype that affords them. The pool must back seqs × max-model-len (the engine's hard limit), so the fourth slot is bought in KV, out of the one per-card budget (weights + kv ≤ 29,120 MiB, the bound as it was understood then):

dtype	pool for 4 × 200,000	weights slab left
bf16	10,791 + 440 + 448 = 11,679 MiB	17,441 MiB (−33 % vs the fp8-era 26,071)
fp8 (deployed)	5,713 + 440 + 448 = 6,601 MiB	22,208 MiB — unchanged

So --vram-weights-mib 22208 / --vram-kv-mib 6912 stay exactly where the 2026-10-11 change put them (pair sum = the measured 29,120 ceiling), the 44,928 MiB/rank pinned host pool is untouched, and the RAM/TTM ledger does not move. What the fp8 pool buys: (6912 − 110×4 − 448) × 1024 / 7.3125 = 843,566 predicted tokens = four full-height conversations with 5.4 % slack (the bf16 pool backed 462,752 = two; the 2026-10-08 fp8 pool backed 386,904 = 1.93).

Terms that move with the dtype and the slot count:

• Per-slot recurrent state is dtype-independent and lives inside the pool: +110 MiB per slot, 440 MiB for four (already in the arithmetic above).
• The fused TP all-reduce is declared over seqs × (1 + spec) rows: 4 × 3 = 12 rows, still inside the 25-row bound at which the wht6 wire is bit-exact — so four slots cost nothing in wire exactness.
• Prefix-cache entries shrink with the attention plane (110.50 KiB at bf16 → 58.50 KiB at fp8), so the 8 GiB host tier parks ~5.8 sessions of 200k where at bf16 it parked ~2.9 — four live slots plus a parked set fit with no re-size.
• models.yml is untouched: contextWindow quotes the per-request window (200,000), which did not change; only the concurrency did.

Expectation carried from the 2026-10-11 A/B, whose arms bracket this shape (same slab 22,208): prefill is set by the slab, not the dtype, so ~3.1k t/s cold / ~3.3k warm; decode sits in the 94–118 t/s band with draft acceptance ~0.73–0.75. The 4,379 t/s fp8 prefill of the 2026-10-08 control belongs to the 26,071 MiB slab and does not come back by quantizing KV.

Boot ledger (2026-10-13 19:21, four slots, fp8 — superseded same evening)

The sizing law's prediction, checked against the engine's own arithmetic: predicted 843,566 tokens, ledger 843,276 (0.03 % block rounding).

Group	Page / state	Size	Tokens
kv_attn	13 layers, 4-token blocks, 26.00 KiB (fp8 page)	5.23 GiB	843,276
kv_qsa_bkey	13 layers, 4-token blocks, 3.25 KiB (dtype-fixed)	669.10 MiB	843,276
kv_gdn_state	36 layers, 54.00 MiB, 8 slots (4 × 2 copies)	432.00 MiB	per-slot
kv_gdn_conv / kv_ple_*	36 + 1 + 1 layers, 4 slots	7.91 MiB	per-slot

kv cache 6.75 GiB … backs 4 sequences all at 200000 tokens at once, which is the worst case — the capacity contract, in the engine's words. The rest of the boot is the fp8-era picture: vram_weights 21.69 GiB filled to 147 KiB (rank 0) / 355 KiB (rank 1) of slack, unclaimed 2.03 GiB, cards settled at 1.99 / 2.00 GiB free, mover: 4967 slab slots (64 shadow), 10375 pinned pool slots, 4.24 GiB static vram, and the half-refund in plain sight: 4967 slots can be lent out of the KV cache (5.87 GiB) (the fp8-era 3,543 MiB pool lent 2.69 GiB).

Two ledger lines that confirm the derived consequences: the sampler prints 12 rows (4 sequences × (1 + 2 speculative) — inside the 25-row bound where wht6 is bit-exact), and the prefix/tier arithmetic runs on the fp8 entry size (58.50 KiB, half of the bf16 entry).

The same-day re-cuts: 4 → 3 slots, then the slab takes the card's slack

Frank's call, twice in one evening. Both are one-env-pair changes, and both were read off the boot ledger rather than assumed.

4 → 3 slots (22208/6912 → 24000/5120). Three slots need 3 × 200,000 at fp8 = 4,285 + 330 + 448 = 5,063 MiB of pool; the 6,912 pool was backing 858,969 tokens — 43 % more window than the contract asks for. Cutting it to 5,120 (predicts 608,027; ledger 607,740, 0.05 %) hands 1,792 MiB to the slab and drops the all-reduce to 9 rows.

24,000 → 25,600 (the card's settling slack). The 24,000 boot printed card 0 settled at 2.01 GiB free against 209.19 MiB (the 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region) — 1,849 MiB above the ENGINE's own floor, with the budget table calling the surplus throughput on the floor. Claiming 1,600 MiB of it puts the pair at 30,720 and the ledger at unclaimed 477.30 MiB (cards ~336 MiB free, floor 209.19).

pair	slab slots (rank 0)	pinned pool slots	A-cold 43.5k	concurrent 197k
22208/6912, seqs 4	14,753	10,375	3,397 t/s	3,307 t/s (4-way)
24000/5120, seqs 3	16,235	8,893	3,290 t/s	3,051 t/s (3-way)
25600/5120, seqs 3	17,548	7,580	3,584 t/s	3,589 t/s (3-way)
26071/3543, seqs 3 (2026-10-11 control)	—	7,190	4,379 t/s	—

The claim shipped with the 25,600 arm — that the slab is a THRESHOLD below ~26 GiB so the extra 1.6 GiB buys ~0 — is falsified by its own measurement: +8.9 % single-stream, +17.6 % concurrent, both outside the ±3 % noise floor. Two slab points that land inside noise of each other (22,208 vs 24,000) are not a plateau; the pinned host pool is the slow path, so every MiB moved from RAM onto the card is throughput (the ledger shows the transfer directly: slab +1,313 slots, pinned pool −1,313).

The cliff is real, it just sits above 25,600: 26,071 measured 4,379 t/s, +22 % for +471 MiB. At three slots that is out of reach (26,112 + 5,120 = 31,232 > the 30,969 ceiling). At two slots the pool falls to 3,072 and the slab reaches 27,897 — the ceiling exactly, slab above the cliff. That arm is untested and it is a capacity decision, not a tuning one.

Capacity contract, live-verified on the 3-slot shape: three concurrent prompts of 199,984 tokens each (599,952 tokens) were accepted and completed on /v1/completions; a fourth full-height request is refused as "too large to ever fit" — radiance never truncates.

The decode ladder (2026-10-13): slots are paid in concurrency, not prefill

Frank's challenge — "why are you not tracking decode?" — was right, and the answer is a number now. 512 generated tokens (the bench law), greedy, cold prompts, this shape (3 x 200,000, fp8, slab 25,600):

1-stream  43.5k: decode  99.0 t/s  (prefill 4,042, accept 0.952)
1-stream   197k: decode  93.3 t/s  (prefill 4,826, accept 0.938)
3-stream  43.5k: decode 113.9 / 43.3 / 10.9 t/s   wall 83.4s
3-stream   197k: decode  88.9 / 16.3 / 64.6 t/s   wall 122.4s

Two laws came out of it. Decode is nearly window-free — 99.0 at 43.5k, 93.3 at 197k, −6 % for 4.5x the depth — so a long conversation costs prefill, not tokens-per-second. Concurrency is where the slot count is paid: three simultaneous streams give 18.4 t/s aggregate at 43.5k and 12.5 at 197k against 99 single-stream, and the split is a lottery (113.9/43.3/10.9), not a fair share — the fused-step schedule hands the step to whoever it reached first, so one stream finishes at 113 t/s while its sibling crawls at 10.9. That is what "3 slots" owes per token, and no prefill number shows it.

Corpus caveat, stated at the claim: this probe's filler is a repetitive ladder, so MTP acceptance is 0.93–0.97 against the 0.740 of the 2026-10-11 table below; these rows flatter decode relative to that table and are the baseline for this corpus, not a replacement for those rows. probe-frank-archon-kv.py now carries this leg as D (leg B generates 24 tokens, which is why a day of shape work priced only prefill).

Recovery evidence

Oct 13 19:21:08 dankserver ttm-preflight.sh[1623778]: frank-archon: ttm pages cap
  is 125988 MiB, below the pinned-ledger floor 150000 — raising the knob to
  204800 MiB (52428800 pages)
Oct 13 19:21:08 dankserver ttm-preflight.sh[1623778]: frank-archon: ttm pages cap
  204800 MiB clears the pinned-ledger floor 150000
Oct 13 19:21:08 dankserver systemd[1]: Started frank-archon.service

cat /sys/module/ttm/parameters/pages_limit → 52,428,800 (204,800 MiB) after the boot, i.e. the root ExecStartPre hook did the write the service process (User=dank) cannot.

Capacity probe (2026-10-13, live service)

scripts/probe-frank-archon-kv.py now reads the slot count, window and KV dtype from the deployed env on the host, so the probe cannot drift from the service: it printed shape: 4 x 200,000 tokens, kv fp8 and fired four concurrent chats of 196,960 prompt tokens each (646,299 chars, 98.5 % of the window), each carrying a marker whose first word is unique.

Stream	prompt	wall	prefill	retrieval
B1	196,961	238.3 s	1,423 t/s	exact, HIT
B2	196,960	98.0 s	2,043 t/s	exact, HIT
B3	196,961	201.1 s	993 t/s	exact, HIT
B4	196,960	149.9 s	1,327 t/s	exact, HIT
total	787,842	238.4 s	3,305 t/s aggregate	4/4

All four answered their own marker; nothing was preempted, refused, or truncated (finish=stop, cache_n=0, so no prefix cache padded the result). Aggregate prefill equals the single-stream rate measured in the same probe (3,397 t/s at 43.5k), i.e. the four streams share one prefill pipeline at the slab-limited ceiling and the engine never ran short of KV. Prefix cache intact: the same 43.5k prompt repeated answered in 1.0 s with cache_n=43008.

The per-stream decode column is NOT a decode rate here — 8 generated tokens make it first-token dominated (the bench law: generate ≥ 128 tokens to read a rate). This probe answers one question only: does the endpoint owe four full-height conversations at once, and answer each one exactly. It does.

Gates (measured at cutover)

• KLD gate: PASS (2026-10-08 04:12) — kld-gate-swift15.sh, candidate on 1.0.13 vs kld-ref: mean KL 0.0146 (median 0.00126, p90 0.0315, p99 0.194, max 4.12), top-1 97.84 %, ppl ratio 1.0126 (candidate 1.6913 vs reference 1.6702), 50,235 positions / 197 docs. Report: /home/dank/radiance-state/kld-gate.json (+ .rows per position). Gate shape: disk-tier, pool 40960, MNBT 3072 (the w4 activation arena scales hard with MNBT: 32768 → 33.3 GiB arena, refused), KV 2048, headroom 512, no speculation. The engine ran rank 0 fully on card 0 (25.88 GiB) with rank 1 host-side in this shape — scoring is placement-invariant; scored at 28 positions/s.
• KLD gate re-PASS (2026-10-08 04:49) at the production-legal shape: MNBT 2048 (the w4 file-tier law: experts stream in place only at MNBT ≤ 2048 — the recording's 32768 demands a widened form the disk tier refuses), pool 40960, vram-kv 3072 fp8, tp 2, headroom 768. mean KL 0.0131, top-1 97.92 %. The Δ vs the first PASS (0.0015) is the float-order noise band of the chunk-size change — the verdict is chunk-size invariant, ~2.3× inside the 0.03 bar on both shapes.
• Throughput gate: PASS (2026-10-08 05:40) — decode at 44.5k depth (streamed chat probe, temp 0, 512 tokens, salted): arm A (main serve shape, EVC 0.82) 146–173 t/s, engine timings 121.9 t/s, draft accept 0.7246 (350/483); arm C (deployed vram-pair shape 26071/3543) median 136.0 t/s, warm TTFT 9.45 s. Control (fp8-native 1.0.13): 30.08 / accept 0.7549. Prefill ~4.7k t/s warm. The w4 expert table is resident — the fp8-native era's NVMe expert streaming (the 3.5 GB/s wall) is gone; the MTP ×3 head rides a 45–60 t/s base decode. Conversation battery (short context): geomean 172 t/s (min 100 / max 208), mean ttft 0.09 s. The EVC ladder is moot: the w4 resident set fits fully inside the 40 GiB pool at 0.82 (ledger: mover full fill). Depth-output audit: the 44.5k-depth summary is a coherent ordered index of the concatenated corpus (no repetition, no NaN-token garbage) — the depth-corruption class of 2026-10-04 is clean.
• Fleet probes: PASS, re-run on the deployed prod — vision (solid-red 512 px → "Red"), tool_calls ({"expr":"23 * 47"}), reasoning_effort xhigh/medium/low/high all 200 (the Swift1.5 froggeric template accepts the full set), reasoning_content present, response_format json_object honoured. Vision probe law: budget ≥ 256 tokens and read reasoning_content as fallback — at 64 tokens thinking exhausts the budget and content comes back empty (the answer lands in the reasoning channel).
• Deployed verification (2026-10-08): deploy.sh dankserver --service frank-archon green incl. Playwright e2e 33/33; unit active, /health 200, /v1/models id frank-archon (the harness alias is the served-model-name, independent of the file — models.yml untouched). Deployed shape: fp8 KV (KLD-gate shape), spec depth 2, seqs 3, vram pair 26071/3543. Boot ledger: KV pool 3.46 GiB, expert pools 57.84 GiB, cards settled 1.41 GiB free, serving on :7770. Live decode probe (512 tokens, streamed, temp 0): ~190 t/s.

Wire rename + the bench drop-in trap (2026-10-09)

FRANK_ARCHON_SERVED_NAME moved the served id frank-primary → frank-archon (commit 7c77326, 2026-10-09); the service, unit, scripts and env file kept frank-primary until the 2026-10-10 rename (55f87fc) brought them to frank-archon too. Radiance is the only fleet engine whose /v1/models listing reflects the flag, and it echoes the requested model name in chat responses — so a chat probe that returns 200 proves nothing about the served id. Always read /v1/models.

The first deploy after the edit came up green and /v1/models still said frank-primary. Cause: a campaign leftover drop-in /etc/systemd/system/frank-archon.service.d/bench.conf (2026-10-08 04:09) overrode BOTH Environment=FRANK_ARCHON_ENV_FILE and ExecStart to the staging worktree /home/dank/serverAppConfigs-swift15. Every deploy.sh dankserver --service frank-archon since then restarted the staging tree: green health gate, old configuration, silently. Verified before removing: the staging serve-frank-archon.sh and frank-archon.service are byte-identical to the checkout, its frank-archon.env differed only by the alias line, and its HEAD (8375567) is an ancestor of main. The drop-in was staged in /home/dank/.AGENTTRASH/20261009-frank-archon-bench-dropin/ (that dated dir is gone — the trash dir is empty on Dankserver as of 2026-10-18). After daemon-reload + restart, /v1/models reports frank-archon (boot healthy in 102 s, shape unchanged: fp8 KV, spec depth 2, seqs 3, vram pair 26071/3543).

Deployer blind spot: deploy.sh plans from repo paths and gates on the port. It cannot see host-side drop-ins, so unit drift is invisible to a green run. After an LLM deploy, verify the served configuration behaviourally (here: podman inspect <name> --format {{json .Config.Cmd}} and /v1/models).

Deploy + verify

sudo systemctl stop frank-archon          # drained window
bash convert-swift15-w4dn8.sh              # if not already run
./scripts/deploy.sh dankserver --service frank-archon

Post-cutover checklist: unit active/enabled; /health 200; boot ledger (expert pool coverage now w4-sized, KV pool should be larger than the FP8-native era's 399,824 tokens); MTP draft counts in timings; froggeric probe; ≥256 px image_url vision probe answers the color; tool-call probe; reasoning_effort fold; Prometheus target green; tests/run.sh

• Playwright e2e green.

Rollback

Layer	Path
Publisher iq4r (successor)	frank-archon-radiance-qwen38-iq4r — the LIVE payload since 2026-10-15
FP8-native radiance	rebuild-only — the .rad (173.0 GiB) was rm'd 2026-10-15: scripts/download-frank-archon.sh + scripts/build-frank-archon-rad.sh
FreeToken PR #587	git tag frank-archon-freetoken-pr587-20261005
tcclaviger vLLM shape G	git tag frank-archon-vllm-shapeG-20261003

This era's own payload (qwen4exp-swift15-w4dn8-i8-hc8m.rad, 116,416 MiB) was purged in the 2026-10-17 disk cleanup (trash-staged, then emptied with Frank's approval). The byte-exact container is published as frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700 (qwen4exp-w4nl64-i8-hc8m.rad, 122,071,973,888 B, Hub LFS sha256 09dd93064365009c508b59012adadab6a36791a50fa319f76b31bf359c01e9ff) — re-download restores the exact payload; a full re-convert is no longer possible without re-fetching a BF16 source (see next paragraph).

Campaign artifacts: the raw BF16 checkpoint was rm'd 2026-10-15 (Frank's one-time authorization), so re-convert needs a fresh BF16 source; kld-ref (24 GiB), grams (/home/dank/corpora/calib), corpora JSONL, and the logs /home/dank/logs/swift15_*.log remain. The BF16 .rad container was deleted before the final convert (disk gate).

Links

• frank-archon-radiance-qwen38-native-fp8 — the replaced FP8-native era
• LLM/index — fleet index
• swift15-campaign-worklog — campaign record
• Dual-R9700-Cooperation — the board swap that made TP=2 legal here