serverAppConfigs/serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md

frank-archon — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2

Status: LIVE — deployed 2026-10-08 (main 3ca64e1, e2e 33/33). · Host: Dankserver (192.168.0.235) · Port: 7770 · GPU: 2× Radeon AI PRO R9700 (gfx1201), TP=2

The FP8-native era (qwen4exp-fp8-native.rad, 173.0 GiB, no-recipe convert) is replaced by a GPTQ-calibrated 4-bit container of ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, abliterated finetune) converted with the recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe on engine 1.0.13 (stilldeadcode/radiance@sha256:4c3f484564… — same digest as the FP8-native era; 1.2.0 was used ONLY for the calibration serve). Same port, alias, container name, systemd unit. Rollback: revert the MODEL_FILE line in scripts/frank-archon.env + redeploy — /home/dank/models/rad/qwen4exp-fp8-native.rad stays on disk (trash policy; 173 GiB, purge when the window closes).

Why this swap

• Context economics at production memory budgets: the FP8-native experts (56.4 GiB/rank slice) forced a 40 GiB/rank pinned host pool and left KV ~3.5 GiB. W4Dn8 experts cut the resident set to ~1/4, freeing host and VRAM budget for KV and slots at equal-or-better measured decode (gate: ≥ 30.8 t/s vs the 30.08 t/s control).
• Swift 1.5 base: abliterated finetune of Flash-Next (Frank's fleet-personality call; frank-dragoon serves the same family).
• GPTQ closes the precision gap: RTN alone on BF16 weights lands measurably worse; GPTQ with Hessians from a 10M-token corpus of this model's own distribution recovers most of it, verified end-to-end by a KLD gate against the BF16 container itself (KL ≤ 0.03, top-1 ≥ 95 %).

The payload chain

1. Fetch: scripts/download-frank-archon-swift15.sh → /home/dank/models/Swift1.5-Qwen3.8-Flash-Next (131 shards, 335.3 GiB, per-shard LFS sha256 verified, no token on the command line). 48 layers, 512 experts (top-10 routed), moe_inter 640, hidden 2560, PLE 20M-vocab table 95.37 GiB, MTP head, vision tower.
2. BF16 container: run-swift15-bf16.sh convert → rad-convert no-recipe (pure copy) at --max-ctx 4096 → qwen4exp-swift15-bf16.rad (335.28 GiB). Serves on 1.0.13 as the KLD reference engine (disk-tier placement; boot ~35 min).
3. KLD reference: run-swift15-bf16.sh kld (1.0.13) → /home/dank/corpora/kld-ref — radiance-kld-1 format, rows: 50235, 197 source-labelled docs, top-16 log-probs (f16), corpus fnv1a64 a9d62b33c18b3d9c. ~4 h at 32k prefill steps (--max-num-batched-tokens 32768 — a 65536 step OOMs an arena tensor at --max-model-len 16384). The wrapper's exit line is the receipt: never scp this script while it runs (bash lazy-read; a mid-run swap killed a clean engine exit with rc=127, third fleet occurrence of the script-swap law). kld-then-calib.sh SKIP_KLD=1 resumes from the rows: 50235 receipt.
4. Calibration: run-swift15-bf16.sh calib on 1.2.0 (the only engine with RADIANCE_CALIB_TOKENS; grams are engine-invariant bf16 activations — RADGRAM2 format byte-identical across versions)
   • calib-push.py pushing calib-corpus.jsonl (10,014,932 tokens, 1,294 prompts, assembled from real HF sources, chat-templated, source-labelled). The tap (in the qwen4exp_fp8.so architecture plugin) writes per-expert down-proj Gram matrices gram.r<rank>.blk.L.ffn_down_exps.<e>.bin (32-byte RADGRAM2 header + 640×640 f32; rows = routing count, trace = activation energy) and stops the serve at the token threshold. TP=2: whole experts per rank (256/rank); gate_up input is rank-sliced (plugin warning — the recipe only casts the down projection).
5. Protected-expert derivation: derive-protected-experts.py /home/dank/corpora/calib → the 24 highest-energy experts (the law is data-driven; Swift drew 24, not the fp8-era 10 — its routing is spikier: blk.47 e445 alone carries 32 % of its layer's down energy at 70× routing share) promoted to whole-expert bf16. Protection law: energy share ≥ 3 % AND ≥ 3× routing share; even count per rank half; odd half gains the next most energetic. apply-protected-cast.sh splices the derived names between the # PROTECTED-BEGIN/END markers of the recipe (refuses double-derive).
6. Final convert: convert-swift15-w4dn8.sh (1.0.13, host-backend CPU-only — no GPU visible to rad-convert; GPTQ solve is CPU BLAS) — gates: ≥48 gram files present, bf16 container already deleted (disk), ≥146 GiB free, output ≥108,000 MiB, rad-info shows the gptq receipt. No --reuse. Measured: 5 h 50 m (6.0–6.5 min/layer on 49 GPTQ layers, load-average 8), output 116,416 MiB. The plugin's rad_arch_declare enforces ONE ENCODING PER EXPERT TABLE (one table of experts takes one encoding, and only a whole expert kept plain bf16 is served beside it) — a split tier (down=fp8, gate_up=w4) is unservable: the kernel serves a single w4nl64a8h table; both expert tables ride GPTQ w4 (down calibrated, gate_up uncalibrated), protected experts bf16 beside them.

The recipe tiers (scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe)

Tier	Encoding	Why
MoE experts, both gate_up and down (one w4 table/layer)	GPTQ w4nl64a8h — u4 codes, fwht128 rotation, fp8 scales/64 + f32 scale plane	the campaign tier: GPTQ on down-input Hessians from the 10M-token corpus (the tap rides the down projection; gate_up shares the table uncalibrated — one-encoding-per-table plugin law)
24 derived protected experts (whole)	bf16	energy-share law (step 5)
shared expert + trunk attention/SSM linears	int8, scale per 128 cols, searched	trunk precision law
hyper-cond + MTP head	fp8_e4m3 row-groups	hc+mtp fp8 law
lm_head	int8	i8 lm_head law (E4M3 costs 0.6 top-1 alone)
PLE n-gram table	fp8 at the checkpoint's fixed scale 1.9931793212890625e-04	the fixed-scale law; absmax audit passed the 2⁻¹³..2⁻¹⁰ plateau
embeddings, vision tower, QSA indexer	bf16 passthrough	untouched

The shape (scripts/frank-archon.env)

TP=2 + wht6, --max-model-len 200000, MNBT 2048 (the MTP resonance law — 2560 collapses decode to 2.6–7.7 t/s), --ngram-placement ram, froggeric v22.5 template override. MODEL_FILE points at /home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad (116,416 MiB measured); the serve size floor FRANK_ARCHON_RAD_MIN_MIB moves 177000 → 110000. Concurrency and the KV cache moved on 2026-10-11; that change is documented in its own section below.

2026-10-11 — unquantized KV at 2 × 200,000

The ask: two concurrent 200,000-token conversations on exact (unquantized) attention state. The fp8 pool backed 3,543 MiB → 386,904 tokens = 1.93 sessions of the served window, so two full-height conversations overflow it and the scheduler preempts the overflow to the host/disk prefix tiers (re-prefilling on return). Measured, that overflow is survivable — the fp8 shape passed the same 2 × 197k pair probe — so this is a headroom change, not the repair of a hard failure. The hard limit the engine states is per request, not per pair: a request longer than the pool's token count is "accepted by the API and then refused as too large to ever fit — this is a hard limit, not a backing ratio", and at 200,000 ≤ 386,904 no single request ever hit it.

The dtype law

--kv-cache-dtype takes only bf16 | fp8: one kv_dtype for the whole cache. llama.cpp's exact-K / quantized-V pair (what frank-zealot runs as --cache-type-k f16 --cache-type-v q8_0) has no radiance equivalent — the rung under bf16 is fp8 on both halves. Evidence: the flag table in the image itself (--kv-cache-dtype takes bf16 or fp8, not '%s') and the absence of any cache-type-k/-v symbol in the binary.

bf16 costs 1.89× fp8, not 2×, and the reason is in the group table:

Group	fp8 page	bf16 page	cost per token, per rank
kv_attn — 13 full-attention layers, 4-token blocks	26.00 KiB	52.00 KiB	6.5 → 13.0 KiB
kv_qsa_bkey — 13 layers, the QSA indexer keys	3.25 KiB	3.25 KiB	0.8125 KiB, dtype-fixed
kv_gdn_state 54 MiB × 2 copies per sequence (+ conv/tail/ple)	per slot	per slot	≈ 110 MiB per sequence
prefix checkpoints 8 × 55.99 MiB (inside the pool)	448 MiB	448 MiB	fixed

→ pool_mib ≥ tokens × 13.8125 KiB + 110 × seqs + 448. The formula reproduces both live boots: fp8 (3543 − 330 − 448) → 387,240 vs the ledger's 386,904; bf16 (6912 − 220 − 448) → 462,946 vs the ledger's 462,752. It is encoded as a test seam (the KV pool backs seqs x max-model-len at the declared cache dtype) so a future resize cannot silently under-provision the pool.

Why the weights slab pays — and why that is not pure loss

The two VRAM budgets are one budget: card 31,856 MiB, minus the 1,710 MiB a boot consumes outside both (driver context, the 990 MiB activation arena, hardware queues, headroom, step staging), minus 1,024 MiB kept free ⇒ weights + kv ≤ 29,120 MiB. Paying for a 6,912 MiB pool therefore cuts the resident slab 26,071 → 22,208 MiB (−14.8 %), and the evicted experts spill into the pinned host pool (40,960 → 44,928 MiB per rank; ledger: pinned pool slots 7,190 → 10,375). The pinned-RAM ledger rises to ≈147 GiB, so the TTM floor moved 140,000 → 150,000 MiB (the kernel cap is 204,800).

Half of that loss is refunded by an engine behaviour worth knowing: idle KV VRAM is the expert cache's spillover. The fp8 pool lent 2.69 GiB to the expert slab; the bf16 pool lends 6.08 GiB (mover: 5146 slots can be lent out of the KV cache). On a llama.cpp endpoint KV is inert storage; on radiance it is a competing cache, and every KV resize is also an expert-residency resize.

seqs 3 → 2

--max-num-seqs 3 was never a radiance requirement: it is the tcclaviger vLLM fork's torch.compile constraint (ConstraintViolationError: Constraints violated (L['query_start_loc'].size()[0])), retired with that runtime — see dankserver-qwen38-fp8-expert-offload-tuning. Radiance booted seqs 2 in the fp8-native era, and the 2026-10-06 decode sweep measured seqs 2 and 3 tied inside the ±3 % noise floor. Two slots × the window is the capacity contract; it also saves 110 MiB of per-slot recurrent state and drops the fused TP all-reduce to 2 × 3 = 6 rows, i.e. deeper inside the 25-row bound at which the wht6 wire is bit-exact.

Boot ledger (deployed 2026-10-11, commit 0bf26b0)

kv cache           6.75 GiB   349% of it -- backs 2 sequences all at 200000
                                tokens at once, which is the worst case
vram_weights      21.69 GiB   stated by --vram-weights-mib
vram_kv            6.75 GiB   stated by --vram-kv-mib
unclaimed          2.03 GiB
kv_attn      full 13 layers block 4 tok page 52.00 KiB 115688 blocks = 5.74 GiB (462752 tokens)
kv_qsa_bkey  full 13 layers block 4 tok page  3.25 KiB 115688 blocks = 367.17 MiB (462752 tokens)
load: rank 0, mover took 21.69 GiB, weights pool 147.00 KiB free of 21.69 GiB
mover: 5146 slots can be lent out of the KV cache (6.08 GiB)
card 0 settled at 1.85 GiB free
serving on http://:::7770

Verification (live service, no scratch container)

Check	Result
/health, /v1/models	200 {"status":"ok","model":"frank-archon"}; served id frank-archon
2 × ~197k concurrent, a unique marker buried at the head of each	both answered their own marker exactly — 196,919 + 196,918 = 393,837 tokens resident, above the fp8 pool's 386,904 and inside bf16's 462,752; walls 138.1 s / 66.2 s, finish_reason: stop, prefill 2,747 / 2,991 t/s
decode at 51.5k depth, 512 tokens, greedy	94.1 cold / 102.3 warm / 117.6 on a prefix-cache hit t/s; draft acceptance 0.7284 / 0.7525 (fp8 control at cutover: 0.7246)
fp8 control arm, same probes (scripts/probe-frank-archon-kv.py)	table in the next section

Probe law re-learned, worth recording: radiance routes the thinking block to reasoning_content, so a chat request whose budget is spent inside <think> comes back with empty content and finish_reason: stop — a probe that forbids thinking (chat_template_kwargs.enable_thinking=false) and reads both fields is the only honest shape. The first pair probe scored 0/2 on exactly that artifact, not on a KV fault.

Measured cost — A/B on the live service

Method (no scratch container, no mask — the unit file is a relink symlink and mask refuses it): write a variant env file on the host (/home/dank/frank-archon-fp8control.env), point a transient drop-in at it (Environment=FRANK_ARCHON_ENV_FILE=…; drop-ins are read AFTER the unit's own Environment= lines, so they win), daemon-reload, restart, probe, delete the drop-in, restart. Control = the deployed 2026-10-08 shape, verified from podman inspect argv (--kv-cache-dtype fp8 --vram-weights-mib 26071 --vram-kv-mib 3543 --max-num-seqs 3) and the ledger page size (26.00 KiB). Identical probes, same salts, loopback, greedy, 512 generated tokens.

Probe	fp8 control (26071/3543, seqs 3)	bf16 deployed (22208/6912, seqs 2)	Δ
prefill 51.5k, cold	4,379 t/s	3,126 t/s	−29 %
prefill 51.5k, warm	4,864 t/s	3,333 t/s	−31 %
decode 51.5k, cold	124.1 t/s	94.1 t/s	−24 %
decode 51.5k, warm	99.2 t/s	102.3 t/s	+3 %
decode on a prefix-cache hit	127.7 t/s	117.6 t/s	−8 %
draft acceptance, mean of passes	0.661	0.740	+12 %
2 × 196,919 concurrent pair	both markers HIT, 110.5 s	both markers HIT, 138.1 s	+25 % wall

Read it as: the price of exact KV on this engine is prefill, not decode. Prefill is expert-bandwidth-bound and the resident slab is 14.8 % smaller, so time-to-first-token at the full window rises ~25 % (the pair: 110 s → 138 s); both prefill passes agree, far outside the ±2 % prefill noise floor. Decode is inside the ±3 % band on two of three passes (warm +3 %, hit −8 %); the cold-pass −24 % is the largest single number, the least reproducible, and it moves the same direction as prefill — the cold arm is paying expert streaming, not attention. Spec-decode acceptance IMPROVED ~8 points (0.661 → 0.740): exact attention state makes the target agree with the MTP drafter more often, which pays on every token of every conversation, and it is the strongest argument for the shape.

Open call (Frank): keep exact KV at the prefill cost, or take the cheap fp8 variant — --vram-kv-mib 3,648 + --max-num-seqs 2 fits 2 × 200,000 at fp8 for a 105 MiB pool bump and ≈ no decode cost. The tool is committed: scripts/probe-frank-archon-kv.py (run it on Dankserver; it rebuilds its prompts from the repo checkout there).

Rollback

Revert scripts/frank-archon.env to the fp8 pair (26071/3543, fp8, seqs 3, host pool 40960, prefix host 4096) and redeploy: no payload change, boot ~2 min. The KLD gate scored at the fp8-KV shape, so a bf16-KV production runs away from the scored configuration — bf16 can only reduce the attention-state error, and a re-score is a drained-window job if the gate agreement is ever needed in the record.

Gates (measured at cutover)

• KLD gate: PASS (2026-10-08 04:12) — kld-gate-swift15.sh, candidate on 1.0.13 vs kld-ref: mean KL 0.0146 (median 0.00126, p90 0.0315, p99 0.194, max 4.12), top-1 97.84 %, ppl ratio 1.0126 (candidate 1.6913 vs reference 1.6702), 50,235 positions / 197 docs. Report: /home/dank/radiance-state/kld-gate.json (+ .rows per position). Gate shape: disk-tier, pool 40960, MNBT 3072 (the w4 activation arena scales hard with MNBT: 32768 → 33.3 GiB arena, refused), KV 2048, headroom 512, no speculation. The engine ran rank 0 fully on card 0 (25.88 GiB) with rank 1 host-side in this shape — scoring is placement-invariant; scored at 28 positions/s.
• KLD gate re-PASS (2026-10-08 04:49) at the production-legal shape: MNBT 2048 (the w4 file-tier law: experts stream in place only at MNBT ≤ 2048 — the recording's 32768 demands a widened form the disk tier refuses), pool 40960, vram-kv 3072 fp8, tp 2, headroom 768. mean KL 0.0131, top-1 97.92 %. The Δ vs the first PASS (0.0015) is the float-order noise band of the chunk-size change — the verdict is chunk-size invariant, ~2.3× inside the 0.03 bar on both shapes.
• Throughput gate: PASS (2026-10-08 05:40) — decode at 44.5k depth (streamed chat probe, temp 0, 512 tokens, salted): arm A (main serve shape, EVC 0.82) 146–173 t/s, engine timings 121.9 t/s, draft accept 0.7246 (350/483); arm C (deployed vram-pair shape 26071/3543) median 136.0 t/s, warm TTFT 9.45 s. Control (fp8-native 1.0.13): 30.08 / accept 0.7549. Prefill ~4.7k t/s warm. The w4 expert table is resident — the fp8-native era's NVMe expert streaming (the 3.5 GB/s wall) is gone; the MTP ×3 head rides a 45–60 t/s base decode. Conversation battery (short context): geomean 172 t/s (min 100 / max 208), mean ttft 0.09 s. The EVC ladder is moot: the w4 resident set fits fully inside the 40 GiB pool at 0.82 (ledger: mover full fill). Depth-output audit: the 44.5k-depth summary is a coherent ordered index of the concatenated corpus (no repetition, no NaN-token garbage) — the depth-corruption class of 2026-10-04 is clean.
• Fleet probes: PASS, re-run on the deployed prod — vision (solid-red 512 px → "Red"), tool_calls ({"expr":"23 * 47"}), reasoning_effort xhigh/medium/low/high all 200 (the Swift1.5 froggeric template accepts the full set), reasoning_content present, response_format json_object honoured. Vision probe law: budget ≥ 256 tokens and read reasoning_content as fallback — at 64 tokens thinking exhausts the budget and content comes back empty (the answer lands in the reasoning channel).
• Deployed verification (2026-10-08): deploy.sh dankserver --service frank-archon green incl. Playwright e2e 33/33; unit active, /health 200, /v1/models id frank-archon (the harness alias is the served-model-name, independent of the file — models.yml untouched). Deployed shape: fp8 KV (KLD-gate shape), spec depth 2, seqs 3, vram pair 26071/3543. Boot ledger: KV pool 3.46 GiB, expert pools 57.84 GiB, cards settled 1.41 GiB free, serving on :7770. Live decode probe (512 tokens, streamed, temp 0): ~190 t/s.

Wire rename + the bench drop-in trap (2026-10-09)

FRANK_ARCHON_SERVED_NAME moved frank-archon → frank-archon; the service, unit, scripts and env file keep the old name. Radiance is the only fleet engine whose /v1/models listing reflects the flag, and it echoes the requested model name in chat responses — so a chat probe that returns 200 proves nothing about the served id. Always read /v1/models.

The first deploy after the edit came up green and /v1/models still said frank-archon. Cause: a campaign leftover drop-in /etc/systemd/system/frank-archon.service.d/bench.conf (2026-10-08 04:09) overrode BOTH Environment=FRANK_ARCHON_ENV_FILE and ExecStart to the staging worktree /home/dank/serverAppConfigs-swift15. Every deploy.sh dankserver --service frank-archon since then restarted the staging tree: green health gate, old configuration, silently. Verified before removing: the staging serve-frank-archon.sh and frank-archon.service are byte-identical to the checkout, its frank-archon.env differed only by the alias line, and its HEAD (8375567) is an ancestor of main. The drop-in is staged in /home/dank/.AGENTTRASH/20261009-frank-archon-bench-dropin/. After daemon-reload + restart, /v1/models reports frank-archon (boot healthy in 102 s, shape unchanged: fp8 KV, spec depth 2, seqs 3, vram pair 26071/3543).

Deployer blind spot: deploy.sh plans from repo paths and gates on the port. It cannot see host-side drop-ins, so unit drift is invisible to a green run. After an LLM deploy, verify the served configuration behaviourally (here: podman inspect <name> --format {{json .Config.Cmd}} and /v1/models).

Deploy + verify

sudo systemctl stop frank-archon          # drained window
bash convert-swift15-w4dn8.sh              # if not already run
./scripts/deploy.sh dankserver --service frank-archon

Post-cutover checklist: unit active/enabled; /health 200; boot ledger (expert pool coverage now w4-sized, KV pool should be larger than the FP8-native era's 399,824 tokens); MTP draft counts in timings; froggeric probe; ≥256 px image_url vision probe answers the color; tool-call probe; reasoning_effort fold; Prometheus target green; tests/run.sh

• Playwright e2e green.

Rollback

Layer	Path
FP8-native radiance (immediate)	revert MODEL_FILE in scripts/frank-archon.env + redeploy; qwen4exp-fp8-native.rad (173.0 GiB) on disk
FreeToken PR #587	git tag frank-archon-freetoken-pr587-20261005
tcclaviger vLLM shape G	git tag frank-archon-vllm-shapeG-20261003

Campaign artifacts on disk: BF16 checkpoint (335 GiB), kld-ref (24 GiB), grams (/home/dank/corpora/calib), corpora JSONL, logs /home/dank/logs/swift15_*.log. The BF16 .rad container is deleted before the final convert (disk gate) — the raw checkpoint is the re-convert source if the window ever reopens.

Links

• frank-archon-radiance-qwen38-native-fp8 — the replaced FP8-native era
• LLM/index — fleet index
• swift15-campaign-worklog — campaign record
• Dual-R9700-Cooperation — the board swap that made TP=2 legal here