0 Fork 0

Files

T

Frank f747cb0dab docs(runbook): staged Swift15 w4dn8 runbook skeleton + index link

Contract facts frozen (payload chain, recipe tiers, shape, gates);
MEASURED@cutover slots for bench/KLD numbers.

2026-10-07 04:40:31 -07:00

7.5 KiB

Raw Blame History

frank-primary — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2

Status: STAGED — activates at the cutover below. · Host: Dankserver (192.168.0.235) · Port: 7770 · GPU: 2× Radeon AI PRO R9700 (gfx1201), TP=2

The FP8-native era (qwen4exp-fp8-native.rad, 173.0 GiB, no-recipe convert) is replaced by a GPTQ-calibrated 4-bit container of ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, abliterated finetune) converted with the recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe on engine 1.0.13 (stilldeadcode/radiance@sha256:4c3f484564… — same digest as the FP8-native era; 1.2.0 was used ONLY for the calibration serve). Same port, alias, container name, systemd unit. Rollback: revert the MODEL_FILE line in scripts/frank-primary.env + redeploy — /home/dank/models/rad/qwen4exp-fp8-native.rad stays on disk (trash policy; 173 GiB, purge when the window closes).

Why this swap

Context economics at production memory budgets: the FP8-native experts (56.4 GiB/rank slice) forced a 40 GiB/rank pinned host pool and left KV ~3.5 GiB. W4Dn8 experts cut the resident set to ~1/4, freeing host and VRAM budget for KV and slots at equal-or-better measured decode (gate: ≥ 30.8 t/s vs the 30.08 t/s control).
Swift 1.5 base: abliterated finetune of Flash-Next (Frank's fleet-personality call; frank-large serves the same family).
GPTQ closes the precision gap: RTN alone on BF16 weights lands measurably worse; GPTQ with Hessians from a 10M-token corpus of this model's own distribution recovers most of it, verified end-to-end by a KLD gate against the BF16 container itself (KL ≤ 0.03, top-1 ≥ 95 %).

The payload chain

Fetch: scripts/download-frank-primary-swift15.sh → /home/dank/models/Swift1.5-Qwen3.8-Flash-Next (131 shards, 335.3 GiB, per-shard LFS sha256 verified, no token on the command line). 48 layers, 512 experts (top-10 routed), moe_inter 640, hidden 2560, PLE 20M-vocab table 95.37 GiB, MTP head, vision tower.
BF16 container: run-swift15-bf16.sh convert → rad-convert no-recipe (pure copy) at --max-ctx 4096 → qwen4exp-swift15-bf16.rad (335.28 GiB). Serves on 1.0.13 as the KLD reference engine (disk-tier placement; boot ~35 min).
KLD reference: run-swift15-bf16.sh kld (1.0.13) → /home/dank/corpora/kld-ref — radiance-kld-1 format, rows: 50235, 197 source-labelled docs, top-16 log-probs (f16), corpus fnv1a64 a9d62b33c18b3d9c. ~4 h at 32k prefill steps (--max-num-batched-tokens 32768 — a 65536 step OOMs an arena tensor at --max-model-len 16384). The wrapper's exit line is the receipt: never scp this script while it runs (bash lazy-read; a mid-run swap killed a clean engine exit with rc=127, third fleet occurrence of the script-swap law). kld-then-calib.sh SKIP_KLD=1 resumes from the rows: 50235 receipt.
Calibration: run-swift15-bf16.sh calib on 1.2.0 (the only engine with RADIANCE_CALIB_TOKENS; grams are engine-invariant bf16 activations — RADGRAM2 format byte-identical across versions)
- calib-push.py pushing calib-corpus.jsonl (10,014,932 tokens, 1,294 prompts, assembled from real HF sources, chat-templated, source-labelled). The tap (in the qwen4exp_fp8.so architecture plugin) writes per-expert down-proj Gram matrices gram.r<rank>.blk.L.ffn_down_exps.<e>.bin (32-byte RADGRAM2 header + 640×640 f32; rows = routing count, trace = activation energy) and stops the serve at the token threshold. TP=2: whole experts per rank (256/rank); gate_up input is rank-sliced (plugin warning — the recipe only casts the down projection).
Protected-expert derivation: derive-protected-experts.py /home/dank/corpora/calib → the 10 highest-energy experts (5 per rank half) promoted to bf16. Protection law: energy share ≥ 3 % AND ≥ 3× routing share; even count per rank half; odd half gains the next most energetic. apply-protected-cast.sh splices the derived names between the # PROTECTED-BEGIN/END markers of the recipe (refuses double-derive).
Final convert: convert-swift15-w4dn8.sh (1.0.13, ~2 h, CPU-only) — gates: ≥48 gram files present, bf16 container already deleted (disk), ≥146 GiB free, output ≥126,000 MiB, rad-info shows the gptq receipt. No --reuse.

The recipe tiers (scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe)

Tier	Encoding	Why
MoE experts `ffn_down`	gptq w4n8 (W4Dn8)	the campaign tier: GPTQ on Hessians from the 10M-token corpus
MoE `gate_up`, attention, shared expert	w4n8	matched quantization; KLD-gated
10 derived experts	bf16 protected	energy-share law (step 5)
trunk (embeddings, norms, conv)	int8 (i8)	trunk precision law
hyper-cond + MTP head	fp8	hc+mtp fp8 law
lm_head	int8 (i8)	i8 lm_head law
PLE n-gram table	fp8 at the checkpoint's fixed scale 1.9931793212890625e-04	the fixed-scale law; absmax audit passed the 2⁻¹³..2⁻¹⁰ plateau

The shape (`scripts/frank-primary.env`)

Unchanged from the FP8-native era: TP=2 + wht6, --max-num-seqs 3, --max-model-len 200000, MNBT 2048 (the MTP resonance law — 2560 collapses decode to 2.6–7.7 t/s), fp8 KV, expert_tiered host pool 40960 (the load-bearing floor), --ngram-placement ram, froggeric v22.5 template override. MODEL_FILE points at /home/dank/models/rad/qwen4exp-swift15-w4dn8.rad; the serve size floor FRANK_PRIMARY_RAD_MIN_MIB moves 177000 → (real output size, set at cutover from rad-info, MEASURED@cutover).

Gates (measured at cutover)

KLD gate (kld-gate-swift15.sh, candidate on 1.0.13 vs kld-ref): mean KL ≤ 0.03, top-1 ≥ 95 %. Fail → escalate gate_up w4→w5nl (pre-authorized), re-convert, re-gate.
Throughput gate: decode ≥ 30.8 t/s at the 43.6k greedy probe (control: 1.0.13 + fp8-native drew 30.08 / accept 0.7549). Fail → fall back to the w4 down tier (Q35).
MEASURED@cutover: decode t/s, acceptance, prefill, boot ledger, KV pool, KLD numbers.

Deploy + verify

sudo systemctl stop frank-primary          # drained window
bash convert-swift15-w4dn8.sh              # if not already run
./scripts/deploy.sh dankserver --service frank-primary

Post-cutover checklist: unit active/enabled; /health 200; boot ledger (expert pool coverage now w4-sized, KV pool should be larger than the FP8-native era's 399,824 tokens); MTP draft counts in timings; froggeric probe; ≥256 px image_url vision probe answers the color; tool-call probe; reasoning_effort fold; Prometheus target green; tests/run.sh

Playwright e2e green.

Rollback

Layer	Path
FP8-native radiance (immediate)	revert `MODEL_FILE` in `scripts/frank-primary.env` + redeploy; `qwen4exp-fp8-native.rad` (173.0 GiB) on disk
FreeToken PR #587	git tag `frank-primary-freetoken-pr587-20261005`
tcclaviger vLLM shape G	git tag `frank-primary-vllm-shapeG-20261003`

Campaign artifacts on disk: BF16 checkpoint (335 GiB), kld-ref (24 GiB), grams (/home/dank/corpora/calib), corpora JSONL, logs /home/dank/logs/swift15_*.log. The BF16 .rad container is deleted before the final convert (disk gate) — the raw checkpoint is the re-convert source if the window ever reopens.

7.5 KiB Raw Blame History Unescape Escape