From f747cb0dab6d0f22c2d8a9c1f576a9a92d2f83fc Mon Sep 17 00:00:00 2001 From: Frank Date: Wed, 7 Oct 2026 04:40:31 -0700 Subject: [PATCH] docs(runbook): staged Swift15 w4dn8 runbook skeleton + index link Contract facts frozen (payload chain, recipe tiers, shape, gates); MEASURED@cutover slots for bench/KLD numbers. --- .../frank-primary-radiance-swift15-w4dn8.md | 144 ++++++++++++++++++ serverAppConfigs-docs/LLM/index.md | 4 + 2 files changed, 148 insertions(+) create mode 100644 serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md diff --git a/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md b/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md new file mode 100644 index 0000000..499df03 --- /dev/null +++ b/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md @@ -0,0 +1,144 @@ +# frank-primary — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2 + +**Status: STAGED — activates at the cutover below.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2 + +The FP8-native era (`qwen4exp-fp8-native.rad`, 173.0 GiB, no-recipe convert) +is replaced by a **GPTQ-calibrated 4-bit container** of +`ukisai/Swift1.5-Qwen3.8-Flash-Next` (BF16, 335.3 GiB, abliterated +finetune) converted with the recipe +`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` on engine +**1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…` — same digest as +the FP8-native era; 1.2.0 was used ONLY for the calibration serve). +Same port, alias, container name, systemd unit. Rollback: revert the +`MODEL_FILE` line in `scripts/frank-primary.env` + redeploy — +`/home/dank/models/rad/qwen4exp-fp8-native.rad` stays on disk (trash +policy; 173 GiB, purge when the window closes). + +## Why this swap + +- **Context economics at production memory budgets**: the FP8-native + experts (56.4 GiB/rank slice) forced a 40 GiB/rank pinned host pool and + left KV ~3.5 GiB. W4Dn8 experts cut the resident set to ~1/4, freeing + host and VRAM budget for KV and slots at equal-or-better measured + decode (gate: ≥ 30.8 t/s vs the 30.08 t/s control). +- **Swift 1.5 base**: abliterated finetune of Flash-Next (Frank's + fleet-personality call; frank-large serves the same family). +- **GPTQ closes the precision gap**: RTN alone on BF16 weights lands + measurably worse; GPTQ with Hessians from a 10M-token corpus of this + model's own distribution recovers most of it, verified end-to-end by a + KLD gate against the BF16 container itself (KL ≤ 0.03, top-1 ≥ 95 %). + +## The payload chain + +1. **Fetch**: `scripts/download-frank-primary-swift15.sh` → + `/home/dank/models/Swift1.5-Qwen3.8-Flash-Next` (131 shards, 335.3 + GiB, per-shard LFS sha256 verified, no token on the command line). + 48 layers, 512 experts (top-10 routed), moe_inter 640, hidden 2560, + PLE 20M-vocab table 95.37 GiB, MTP head, vision tower. +2. **BF16 container**: `run-swift15-bf16.sh convert` → + `rad-convert` no-recipe (pure copy) at `--max-ctx 4096` → + `qwen4exp-swift15-bf16.rad` (335.28 GiB). Serves on 1.0.13 as the + KLD reference engine (disk-tier placement; boot ~35 min). +3. **KLD reference**: `run-swift15-bf16.sh kld` (1.0.13) → + `/home/dank/corpora/kld-ref` — `radiance-kld-1` format, `rows: + 50235`, 197 source-labelled docs, top-16 log-probs (f16), corpus + fnv1a64 `a9d62b33c18b3d9c`. ~4 h at 32k prefill steps + (`--max-num-batched-tokens 32768` — a 65536 step OOMs an arena + tensor at `--max-model-len 16384`). **The wrapper's exit line is the + receipt: never scp this script while it runs** (bash lazy-read; + a mid-run swap killed a clean engine exit with rc=127, third fleet + occurrence of the script-swap law). `kld-then-calib.sh SKIP_KLD=1` + resumes from the `rows: 50235` receipt. +4. **Calibration**: `run-swift15-bf16.sh calib` on **1.2.0** (the only + engine with `RADIANCE_CALIB_TOKENS`; grams are engine-invariant + bf16 activations — RADGRAM2 format byte-identical across versions) + + `calib-push.py` pushing `calib-corpus.jsonl` (10,014,932 tokens, + 1,294 prompts, assembled from real HF sources, chat-templated, + source-labelled). The tap (in the `qwen4exp_fp8.so` architecture + plugin) writes per-expert down-proj Gram matrices + `gram.r.blk.L.ffn_down_exps..bin` (32-byte RADGRAM2 header + + 640×640 f32; `rows` = routing count, `trace` = activation energy) + and stops the serve at the token threshold. TP=2: whole experts per + rank (256/rank); gate_up input is rank-sliced (plugin warning — the + recipe only casts the down projection). +5. **Protected-expert derivation**: + `derive-protected-experts.py /home/dank/corpora/calib` → the 10 + highest-energy experts (5 per rank half) promoted to bf16. Protection + law: energy share ≥ 3 % AND ≥ 3× routing share; even count per rank + half; odd half gains the next most energetic. + `apply-protected-cast.sh` splices the derived names between the + `# PROTECTED-BEGIN/END` markers of the recipe (refuses double-derive). +6. **Final convert**: `convert-swift15-w4dn8.sh` (1.0.13, ~2 h, CPU-only) + — gates: ≥48 gram files present, bf16 container already deleted + (disk), ≥146 GiB free, output ≥126,000 MiB, `rad-info` shows the + gptq receipt. No `--reuse`. + +## The recipe tiers (scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe) + +| Tier | Encoding | Why | +|---|---|---| +| MoE experts `ffn_down` | **gptq w4n8 (W4Dn8)** | the campaign tier: GPTQ on Hessians from the 10M-token corpus | +| MoE `gate_up`, attention, shared expert | w4n8 | matched quantization; KLD-gated | +| 10 derived experts | **bf16 protected** | energy-share law (step 5) | +| trunk (embeddings, norms, conv) | **int8 (i8)** | trunk precision law | +| hyper-cond + MTP head | **fp8** | hc+mtp fp8 law | +| lm_head | **int8 (i8)** | i8 lm_head law | +| PLE n-gram table | fp8 at the checkpoint's fixed scale 1.9931793212890625e-04 | the fixed-scale law; absmax audit passed the 2⁻¹³..2⁻¹⁰ plateau | + +## The shape (`scripts/frank-primary.env`) + +Unchanged from the FP8-native era: TP=2 + `wht6`, `--max-num-seqs 3`, +`--max-model-len 200000`, MNBT 2048 (the MTP resonance law — 2560 +collapses decode to 2.6–7.7 t/s), fp8 KV, `expert_tiered` host pool +40960 (the load-bearing floor), `--ngram-placement ram`, froggeric +v22.5 template override. `MODEL_FILE` points at +`/home/dank/models/rad/qwen4exp-swift15-w4dn8.rad`; the serve size floor +`FRANK_PRIMARY_RAD_MIN_MIB` moves 177000 → (real output size, set at +cutover from `rad-info`, MEASURED@cutover). + +## Gates (measured at cutover) + +- **KLD gate** (`kld-gate-swift15.sh`, candidate on 1.0.13 vs + `kld-ref`): mean KL ≤ 0.03, top-1 ≥ 95 %. Fail → escalate gate_up + w4→w5nl (pre-authorized), re-convert, re-gate. +- **Throughput gate**: decode ≥ 30.8 t/s at the 43.6k greedy probe + (control: 1.0.13 + fp8-native drew 30.08 / accept 0.7549). Fail → + fall back to the w4 down tier (Q35). +- MEASURED@cutover: decode t/s, acceptance, prefill, boot ledger, KV pool, + KLD numbers. + +## Deploy + verify + +```bash +sudo systemctl stop frank-primary # drained window +bash convert-swift15-w4dn8.sh # if not already run +./scripts/deploy.sh dankserver --service frank-primary +``` + +Post-cutover checklist: unit active/enabled; `/health` 200; boot ledger +(expert pool coverage now w4-sized, KV pool should be larger than the +FP8-native era's 399,824 tokens); MTP draft counts in `timings`; froggeric +probe; ≥256 px `image_url` vision probe answers the color; tool-call +probe; `reasoning_effort` fold; Prometheus target green; `tests/run.sh` ++ Playwright e2e green. + +## Rollback + +| Layer | Path | +|---|---| +| FP8-native radiance (immediate) | revert `MODEL_FILE` in `scripts/frank-primary.env` + redeploy; `qwen4exp-fp8-native.rad` (173.0 GiB) on disk | +| FreeToken PR #587 | git tag `frank-primary-freetoken-pr587-20261005` | +| tcclaviger vLLM shape G | git tag `frank-primary-vllm-shapeG-20261003` | + +Campaign artifacts on disk: BF16 checkpoint (335 GiB), `kld-ref` (24 +GiB), grams (`/home/dank/corpora/calib`), corpora JSONL, logs +`/home/dank/logs/swift15_*.log`. The BF16 `.rad` container is deleted +before the final convert (disk gate) — the raw checkpoint is the +re-convert source if the window ever reopens. + +## Links + +- [[frank-primary-radiance-qwen38-native-fp8]] — the replaced FP8-native era +- [[LLM/index]] — fleet index +- [[swift15-campaign-worklog]] — campaign record +- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here diff --git a/serverAppConfigs-docs/LLM/index.md b/serverAppConfigs-docs/LLM/index.md index 2f6a513..8777124 100644 --- a/serverAppConfigs-docs/LLM/index.md +++ b/serverAppConfigs-docs/LLM/index.md @@ -137,6 +137,10 @@ frank-secondary, port 7771] target; payload on disk; owns the measured boot-pair table)**: the same runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower — 104.8 tok/s, 3,571 MiB true free +- [[frank-primary-radiance-swift15-w4dn8]] — **STAGED (campaign running)**: + Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance + 1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container); + replaces the FP8-native payload when the gates pass - [[frank-primary-radiance-qwen38-native-fp8]] — **ACTIVE since 2026-10-05** (root systemd unit, port 7770): the StillDeadcode **radiance** engine (C++/HIP, libr4d gfx1201 kernels, digest-pinned 287 MiB image, engine as