frank-primary — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2

Status: STAGED — activates at the cutover below. · Host: Dankserver (192.168.0.235) · Port: 7770 · GPU: 2× Radeon AI PRO R9700 (gfx1201), TP=2

The FP8-native era (qwen4exp-fp8-native.rad, 173.0 GiB, no-recipe convert) is replaced by a GPTQ-calibrated 4-bit container of ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, abliterated finetune) converted with the recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe on engine 1.0.13 (stilldeadcode/radiance@sha256:4c3f484564… — same digest as the FP8-native era; 1.2.0 was used ONLY for the calibration serve). Same port, alias, container name, systemd unit. Rollback: revert the MODEL_FILE line in scripts/frank-primary.env + redeploy — /home/dank/models/rad/qwen4exp-fp8-native.rad stays on disk (trash policy; 173 GiB, purge when the window closes).

Why this swap

Context economics at production memory budgets: the FP8-native experts (56.4 GiB/rank slice) forced a 40 GiB/rank pinned host pool and left KV ~3.5 GiB. W4Dn8 experts cut the resident set to ~1/4, freeing host and VRAM budget for KV and slots at equal-or-better measured decode (gate: ≥ 30.8 t/s vs the 30.08 t/s control).
Swift 1.5 base: abliterated finetune of Flash-Next (Frank's fleet-personality call; frank-large serves the same family).
GPTQ closes the precision gap: RTN alone on BF16 weights lands measurably worse; GPTQ with Hessians from a 10M-token corpus of this model's own distribution recovers most of it, verified end-to-end by a KLD gate against the BF16 container itself (KL ≤ 0.03, top-1 ≥ 95 %).

The payload chain

Fetch: scripts/download-frank-primary-swift15.sh → /home/dank/models/Swift1.5-Qwen3.8-Flash-Next (131 shards, 335.3 GiB, per-shard LFS sha256 verified, no token on the command line). 48 layers, 512 experts (top-10 routed), moe_inter 640, hidden 2560, PLE 20M-vocab table 95.37 GiB, MTP head, vision tower.
BF16 container: run-swift15-bf16.sh convert → rad-convert no-recipe (pure copy) at --max-ctx 4096 → qwen4exp-swift15-bf16.rad (335.28 GiB). Serves on 1.0.13 as the KLD reference engine (disk-tier placement; boot ~35 min).
KLD reference: run-swift15-bf16.sh kld (1.0.13) → /home/dank/corpora/kld-ref — radiance-kld-1 format, rows: 50235, 197 source-labelled docs, top-16 log-probs (f16), corpus fnv1a64 a9d62b33c18b3d9c. ~4 h at 32k prefill steps (--max-num-batched-tokens 32768 — a 65536 step OOMs an arena tensor at --max-model-len 16384). The wrapper's exit line is the receipt: never scp this script while it runs (bash lazy-read; a mid-run swap killed a clean engine exit with rc=127, third fleet occurrence of the script-swap law). kld-then-calib.sh SKIP_KLD=1 resumes from the rows: 50235 receipt.
Calibration: run-swift15-bf16.sh calib on 1.2.0 (the only engine with RADIANCE_CALIB_TOKENS; grams are engine-invariant bf16 activations — RADGRAM2 format byte-identical across versions)
- calib-push.py pushing calib-corpus.jsonl (10,014,932 tokens, 1,294 prompts, assembled from real HF sources, chat-templated, source-labelled). The tap (in the qwen4exp_fp8.so architecture plugin) writes per-expert down-proj Gram matrices gram.r<rank>.blk.L.ffn_down_exps.<e>.bin (32-byte RADGRAM2 header + 640×640 f32; rows = routing count, trace = activation energy) and stops the serve at the token threshold. TP=2: whole experts per rank (256/rank); gate_up input is rank-sliced (plugin warning — the recipe only casts the down projection).
Protected-expert derivation: derive-protected-experts.py /home/dank/corpora/calib → the 24 highest-energy experts (the law is data-driven; Swift drew 24, not the fp8-era 10 — its routing is spikier: blk.47 e445 alone carries 32 % of its layer's down energy at 70× routing share) promoted to whole-expert bf16. Protection law: energy share ≥ 3 % AND ≥ 3× routing share; even count per rank half; odd half gains the next most energetic. apply-protected-cast.sh splices the derived names between the # PROTECTED-BEGIN/END markers of the recipe (refuses double-derive).
Final convert: convert-swift15-w4dn8.sh (1.0.13, host-backend CPU-only — no GPU visible to rad-convert; GPTQ solve is CPU BLAS) — gates: ≥48 gram files present, bf16 container already deleted (disk), ≥146 GiB free, output ≥108,000 MiB, rad-info shows the gptq receipt. No --reuse. Measured: 5 h 50 m (6.0–6.5 min/layer on 49 GPTQ layers, load-average 8), output 116,416 MiB. The plugin's rad_arch_declare enforces ONE ENCODING PER EXPERT TABLE (one table of experts takes one encoding, and only a whole expert kept plain bf16 is served beside it) — a split tier (down=fp8, gate_up=w4) is unservable: the kernel serves a single w4nl64a8h table; both expert tables ride GPTQ w4 (down calibrated, gate_up uncalibrated), protected experts bf16 beside them.

The recipe tiers (scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe)

Tier	Encoding	Why
MoE experts, both `gate_up` and `down` (one w4 table/layer)	GPTQ w4nl64a8h — u4 codes, fwht128 rotation, fp8 scales/64 + f32 scale plane	the campaign tier: GPTQ on down-input Hessians from the 10M-token corpus (the tap rides the down projection; gate_up shares the table uncalibrated — one-encoding-per-table plugin law)
24 derived protected experts (whole)	bf16	energy-share law (step 5)
shared expert + trunk attention/SSM linears	int8, scale per 128 cols, searched	trunk precision law
hyper-cond + MTP head	fp8_e4m3 row-groups	hc+mtp fp8 law
lm_head	int8	i8 lm_head law (E4M3 costs 0.6 top-1 alone)
PLE n-gram table	fp8 at the checkpoint's fixed scale 1.9931793212890625e-04	the fixed-scale law; absmax audit passed the 2⁻¹³..2⁻¹⁰ plateau
embeddings, vision tower, QSA indexer	bf16 passthrough	untouched

The shape (`scripts/frank-primary.env`)

Unchanged from the FP8-native era: TP=2 + wht6, --max-num-seqs 3, --max-model-len 200000, MNBT 2048 (the MTP resonance law — 2560 collapses decode to 2.6–7.7 t/s), fp8 KV, expert_tiered host pool 40960 (the load-bearing floor), --ngram-placement ram, froggeric v22.5 template override. MODEL_FILE points at /home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad (116,416 MiB measured); the serve size floor FRANK_PRIMARY_RAD_MIN_MIB moves 177000 → 110000. /home/dank/models/rad/qwen4exp-swift15-w4dn8.rad; the serve size floor FRANK_PRIMARY_RAD_MIN_MIB moves 177000 → (real output size, set at cutover from rad-info, MEASURED@cutover).

Gates (measured at cutover)

KLD gate: PASS (2026-10-08 04:12) — kld-gate-swift15.sh, candidate on 1.0.13 vs kld-ref: mean KL 0.0146 (median 0.00126, p90 0.0315, p99 0.194, max 4.12), top-1 97.84 %, ppl ratio 1.0126 (candidate 1.6913 vs reference 1.6702), 50,235 positions / 197 docs. Report: /home/dank/radiance-state/kld-gate.json (+ .rows per position). Gate shape: disk-tier, pool 40960, MNBT 3072 (the w4 activation arena scales hard with MNBT: 32768 → 33.3 GiB arena, refused), KV 2048, headroom 512, no speculation. The engine ran rank 0 fully on card 0 (25.88 GiB) with rank 1 host-side in this shape — scoring is placement-invariant; scored at 28 positions/s.
Throughput gate: decode ≥ 30.8 t/s at the 43.6k greedy probe (control: 1.0.13 + fp8-native drew 30.08 / accept 0.7549). Fail → fall back to the w4 down tier (Q35).
MEASURED@cutover: decode t/s, acceptance, prefill, boot ledger, KV pool, KLD numbers.

Deploy + verify

sudo systemctl stop frank-primary          # drained window
bash convert-swift15-w4dn8.sh              # if not already run
./scripts/deploy.sh dankserver --service frank-primary

Post-cutover checklist: unit active/enabled; /health 200; boot ledger (expert pool coverage now w4-sized, KV pool should be larger than the FP8-native era's 399,824 tokens); MTP draft counts in timings; froggeric probe; ≥256 px image_url vision probe answers the color; tool-call probe; reasoning_effort fold; Prometheus target green; tests/run.sh

Playwright e2e green.

Rollback

Layer	Path
FP8-native radiance (immediate)	revert `MODEL_FILE` in `scripts/frank-primary.env` + redeploy; `qwen4exp-fp8-native.rad` (173.0 GiB) on disk
FreeToken PR #587	git tag `frank-primary-freetoken-pr587-20261005`
tcclaviger vLLM shape G	git tag `frank-primary-vllm-shapeG-20261003`

Campaign artifacts on disk: BF16 checkpoint (335 GiB), kld-ref (24 GiB), grams (/home/dank/corpora/calib), corpora JSONL, logs /home/dank/logs/swift15_*.log. The BF16 .rad container is deleted before the final convert (disk gate) — the raw checkpoint is the re-convert source if the window ever reopens.

9.2 KiB Raw Blame History Unescape Escape