docs(frank-archon): iq4r era runbook + index/AGENTS re-key

New runbook frank-archon-radiance-qwen38-iq4r.md (publisher container, payload chain, shape-verbatim cutover record with the boot-ledger and measurement slots left open for the first boot, rollback). LLM index, root vault index, and AGENTS.md (fleet table row, section, runbook index: iq4r ACTIVE, w4dn8 RETIRED/on-disk, fp8-native deleted). Full suite green.
2026-10-15 04:16:28 -07:00
parent 6dcf4a5913
commit f44c39812f
4 changed files with 185 additions and 36 deletions
@@ -0,0 +1,120 @@
+# frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, 2× R9700 TP=2
+
+**Status: LIVE — deployed 2026-10-15.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
+
+The in-house w4dn8 conversion of the Swift1.5 finetune
+([[frank-archon-radiance-swift15-w4dn8]]) is replaced by the **engine
+author's published container**:
+[`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`](https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe)
+— the official `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint rad-converted by
+StillDeadcode. Same port, alias, container name, systemd unit, digest-pinned
+engine pin **1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…`), and
+**every serving argument carried verbatim** from the 2026-10-13 shape.
+Rollback: revert the swap commit — the w4dn8 `.rad` stays on disk.
+
+## The payload
+
+`qwen3.8-next-flash-fp8-iq4r-moe.rad` — **121,969,901,568 B = 116,322 MiB**,
+sha256 `0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20`
+(Hub LFS oid), fetched byte-exact by `scripts/download-frank-archon-iq4r.sh`
+(hfget resume + size + sha256; staging dir
+`/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe/` keeps the
+card, LICENSE, and recipe `qwen4exp-w4nl64-i8-hc8m` as provenance).
+
+| Plane | Encoding (publisher recipe `qwen4exp-w4nl64-i8-hc8m`) |
+| --- | --- |
+| Routed experts | 4-bit codes into a 16-level **non-uniform** codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes by GPTQ (10M-token calibration, per-expert down-proj Hessians) + 3 coordinate-descent sweeps |
+| Protected experts | **ten** per layer, whole-expert bf16 (down-projection energy) |
+| Trunk + lm_head | int8 W8A8, scale per 128 columns searched; HC mixtures E4M3 |
+| PLE n-gram | fp8 fixed-scale, in-container (served `--ngram-placement ram`, same 47.7 GiB pin law) |
+| Speculator | the model's own MTP head (card default depth 3; served at 2) |
+| Vision | bf16 vision tower, in-container |
+
+**Base model: the OFFICIAL (censored) Qwen3.8-Flash-Next** — the fleet's
+only frontier model at base alignment; the abliterated family lives on
+frank-dragoon. Card quality: full-vocab teacher-forced KL vs bf16 over
+~75K positions — mean 0.0789 all-tokens / 0.0239 assistant turns, top-1
+agreement 92.85 %.
+
+## Why this swap
+
+- **Publisher-graded precision**: the w4nl codebook + coordinate-descent
+  refinement is the engine author's own quality ceiling for this
+  checkpoint; the card's KL ladder (p99 1.65, p99.9 5.33) is published and
+  reproducible, where our w4dn8 gate measured against our own reference.
+- **Formal parity**: the served file is the one the engine author supports
+  — decode/prefill behavior on 1.0.13 is their tested path (the card's run
+  line speaks the 1.0.13-era flags; the `--vram-weights-mib`/`--vram-kv-mib`
+  pair we ship is accepted by pin 1.0.13 per the 2026-10-06 A/B, so the
+  shape needed no translation).
+- **Cost**: the Swift1.5 finetune's refusal-bending behavior leaves this
+  endpoint (it returns to the official base alignment); the 4-slot/200k
+  economics and all shape laws carry over unchanged.
+
+## Shape (unchanged from 2026-10-13)
+
+All of it, verbatim — the payload moved 116,416 → 116,322 MiB, inside the
+same budget:
+
+- `--tp 2 --tp-wire wht6`, `--max-num-seqs 3`, `--max-model-len 200000`
+  (capacity contract 3 × 200,000), `--kv-cache-dtype fp8`
+- `--placement expert_tiered --host-pool-mib 44928` + explicit
+  `--vram-weights-mib 25600 --vram-kv-mib 5120` (30,720 of the engine's
+  30,969 MiB ceiling; pool = contract + 1.3 %)
+- `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`
+  (MNBT 2048 law), prefix cache 8 GiB host + 64 GiB disk,
+  `--ngram-placement ram`, froggeric v22.5 `--override-chat-template`
+- TTM preflight floor 150,000 / target 204,800 MiB; pinned ledger
+  2 × 44 GiB pool + 47.7 GiB PLE + 8 GiB prefix ≈ 147 GiB (see
+  [[frank-archon-radiance-swift15-w4dn8]] for the ledger arithmetic and
+  the KV dtype law — both are engine properties, payload-independent)
+
+The publisher's card run line (`--host-pool-mib 12288`,
+`--expert-vs-cache-ratio 0.82`, seqs 8, spec 3, ~32 GB host RAM) is their
+shape on stock 32 GB cards; the fleet ships its own measured shape against
+the same file.
+
+## Cutover record — 2026-10-15
+
+1. `rm` (Frank's one-time authorization): the raw
+   `ukisai/Swift1.5-Qwen3.8-Flash-Next` BF16 checkpoint (336 GiB, spent
+   conversion source) and `qwen4exp-fp8-native.rad` (173 GiB, two eras
+   back) left `/home/dank/models` — 150 GB free → 658 GB free. The fp8-native
+   payload is **no longer on disk**: rebuild =
+   `scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh`.
+2. Fetch: `scripts/download-frank-archon-iq4r.sh --fetch` as transient unit
+   `iq4r-fetch` (detached-job protocol; log
+   `/home/dank/logs/iq4r-fetch.log`, rc sentinel `…rc`).
+3. Deploy: `scripts/deploy.sh dankserver --service frank-archon` — launched
+   detached (the swap restarts the very endpoint the operator session runs
+   on; the harness timeout catches the session on return).
+4. **Boot ledger (fill after first boot)**: `weights/kv` per-rank budgets,
+   `kv_attn` page size, slab slot count, `backs N sequences` line, sampler
+   rows, unclaimed MiB. The load of this file on pin 1.0.13 is the gate: a
+   refusal naming the w4nl codebook means the pin predates the encoding —
+   the documented repair is the pin bump to the card's era, which the
+   2026-10-06 1.2.0 A/B (decode 29.21 vs 30.08 t/s, acceptance 0.7123 vs
+   0.7549) makes a paid decision.
+5. **Post-boot measurements (pending)**: decode ladder (99.0 t/s single
+   @ 43.5k, 93.3 @ 197k, 3-way 113.9/43.3/10.9 = 18.4 t/s aggregate —
+   these are the **w4dn8** numbers, the record to beat, not this file's),
+   slab-limited prefill (22208/3,397 → 25600/3,584 t/s A-cold 43.5k), draft
+   acceptance, and `scripts/probe-frank-archon-kv.py` leg D.
+
+## Rollback
+
+Revert the swap commit (`scripts/frank-archon.env` MODEL_FILE line) +
+redeploy: `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad`
+(116,416 MiB) stays on disk, KLD-gated, same shape. The fp8-native era is
+rebuild-only (see the cutover record). The w4dn8 runbook keeps every shape
+law, the KV dtype table, the slab ladder, and the tuning levers:
+[[frank-archon-radiance-swift15-w4dn8]].
+
+## Related
+
+- [[frank-archon-radiance-swift15-w4dn8]] — the replaced payload and the
+  full shape-law library (slab slope, KV dtype arithmetic, TTM law)
+- [[frank-archon-radiance-qwen38-native-fp8]] — the fp8-native era
+  (rebuild-only now; owns the descriptor-selection law)
+- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
+- [[deployment]] — the service-scoped deploy this cutover rides
@@ -3,14 +3,17 @@
 **Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
 (digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
 driving `podman run` directly, root systemd unit, :7770):
-`ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4**
-(`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s
+the **publisher's** 4-bit-codebook container of the official
+`Qwen/Qwen3.8-Flash-Next` (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
+sha256-verified at fetch — w4nl codebook experts + int8 trunk, PLE n-gram,
+MTP head and vision tower in-container), on BOTH R9700s
 TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
 (explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
 30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
 table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
-froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005
-and frank-archon-vllm-shapeG-20261003),
+froggeric v22.5 template. Rollback: the in-house Swift1.5 w4dn8 `.rad` (on
+disk); tags frank-archon-freetoken-pr587-20261005 and
+frank-archon-vllm-shapeG-20261003),
 and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
 (frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
 digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
@@ -37,10 +40,10 @@ graph LR
 gfx1201, 32 GiB each]
        B2[radiance StillDeadcode, digest-pinned image
 podman run, frank-archon, port 7770]
-        C[Swift1.5-Qwen3.8-Flash-Next GPTQ w4 .rad
+        C[Publisher qwen3.8-next-flash-fp8-iq4r-moe .rad
 2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM,
 PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
-4 x 200000, froggeric v22.5]
+3 x 200000, froggeric v22.5]
    end

    subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
@@ -153,7 +156,22 @@ frank-zealot, port 7771]
  target; payload on disk; owns the measured boot-pair table)**: the same
  runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
  104.8 tok/s, 3,571 MiB true free
- [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**:
+- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE since 2026-10-15**:
+  the engine author's published container of the OFFICIAL (base-alignment)
+  Qwen3.8-Flash-Next — `qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
+  Hub-sha256-verified at fetch: 4-bit codes into a 16-level non-uniform
+  codebook after a Walsh–Hadamard rotation (GPTQ + 3 coordinate-descent
+  sweeps, 10M-token calibration), ten bf16-protected experts per layer,
+  int8 trunk/lm_head, fp8 HC + PLE, MTP head, bf16 vision tower; card KL
+  0.0789 all-tokens / top-1 92.85 % vs bf16. **MODEL_FILE is the only line
+  that moved** — the 2026-10-13 shape (25600/5120 pair, 44928 pool, fp8 KV
+  3 × 200,000, MTP 2, wht6, MNBT 2048, froggeric v22.5) carries verbatim.
+  Rollback: revert the swap commit (w4dn8 `.rad` stays on disk). The
+  fp8-native `.rad` and the raw Swift1.5 BF16 source were DELETED from
+  dankserver 2026-10-15 (Frank's one-time rm, volume for this payload;
+  rebuild via the fetch + build scripts named in the fp8-native runbook)
+- [[frank-archon-radiance-swift15-w4dn8]] — **RETIRED 2026-10-08 → 10-15
+  (superseded by the publisher iq4r swap; payload on disk as the rollback)**:
  Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
  1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
  the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
@@ -182,12 +182,20 @@ The LLM inference fleet on the two hosts. Full index:
 ### Deployment and Topology

 - [[deployment]] — unified deploy and update process for both servers
- [[frank-archon-radiance-swift15-w4dn8]] — STAGED frank-archon runbook:
+- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE frank-archon runbook
+  (2026-10-15)**: the PUBLISHER's container of the official
+  Qwen3.8-Flash-Next (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
+  w4nl 4-bit-codebook experts + int8 trunk, Hub-sha256-verified at fetch)
+  swapped in as a MODEL_FILE-only change — the 2026-10-13 shape (slab
+  25600 / fp8-KV 5120, 3 × 200,000, host pool 44,928, MTP depth 2) carries
+  verbatim; rollback = revert the swap commit (the w4dn8 `.rad` stays on
+  disk; the fp8-native `.rad` was deleted 2026-10-15, rebuildable)
+- [[frank-archon-radiance-swift15-w4dn8]] — RETIRED 2026-10-08 → 10-15 frank-archon runbook:
  Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
  1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
  activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
  = revert that line (the FP8-native `.rad` stays on disk)
- [[frank-archon-radiance-qwen38-native-fp8]] — the ACTIVE frank-archon
+- [[frank-archon-radiance-qwen38-native-fp8]] — the RETIRED 2026-10-05 → 10-08 frank-archon
  runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
  287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
  `Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert