docs(frank-archon): iq4r era runbook + index/AGENTS re-key

New runbook frank-archon-radiance-qwen38-iq4r.md (publisher container,
payload chain, shape-verbatim cutover record with the boot-ledger and
measurement slots left open for the first boot, rollback). LLM index,
root vault index, and AGENTS.md (fleet table row, section, runbook
index: iq4r ACTIVE, w4dn8 RETIRED/on-disk, fp8-native deleted). Full
suite green.
This commit is contained in:
Frank committed 2026-10-15 04:16:28 -07:00
1 parent 6dcf4a5913
commit f44c39812f
4 files changed
+185 -36

No files matched your search

+30 -27
View File
@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Model | Port | | Target | Service | Runtime | Model | Port |
|--------|---------|---------|-------|------| |--------|---------|---------|-------|------|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **44 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the **30,969 MiB** the engine's own budget table makes claimable: card 32,624 MiB, settle floor 209 MiB), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (pool 5,120 MiB = the 3-slot contract + 1.3 % — the only dtype that funds 3 × 200,000 without gutting the slab, since bf16 bills 13.8 KiB/token), **MTP spec decode depth 2**, `--max-num-seqs 3` (the capacity contract is 3 × 200,000; 3 × (1+2) = 9 all-reduce rows, inside the 25-row wht6 exactness bound), `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8); prefill is **slab-limited and pays for it** — 22,208 → 3,397, 24,000 → 3,290, 25,600 → **3,584** t/s at 43.5k single-stream, 3-way 197k aggregate 3,051 → **3,589** t/s, with a cliff at ~26 GiB (26,071 → 4,379). Cutover 2026-10-08 from fp8-native (payload on disk = rollback); 3-slot fp8-KV shape 2026-10-13, re-cut twice in the slab's favour (22208/6912 at 4 slots → 24000/5120 → 25600/5120); the 4-slot shape (22208/6912, `--max-num-seqs 4`) is the documented rollback. | 7770 | | Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: revert the swap commit (w4dn8 `.rad` on disk); the fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770| | Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 | | Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
@@ -498,32 +498,33 @@ Rollback: revert the swap commit — the Gemma4 payload
(`/home/dank/models/Ornith-Q8/`) all remain on disk. (`/home/dank/models/Ornith-Q8/`) all remain on disk.
### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE) ### frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, TP=2 (Dankserver, ACTIVE)
The **radiance engine** (StillDeadcode, Apache-2.0; image The **radiance engine** (StillDeadcode, Apache-2.0; image
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13** `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root for serve; `libr4d` gfx1201 kernels, plain `podman run`, root
systemd unit, container name = unit name) serving systemd unit, container name = unit name) serving the
**ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards, **publisher's container of the official Qwen/Qwen3.8-Flash-Next**:
397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh` `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` →
with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to `qwen3.8-next-flash-fp8-iq4r-moe.rad` (116,322 MiB, Hub-LFS sha256
`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ verified at fetch by `scripts/download-frank-archon-iq4r.sh`; recipe
w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine `qwen4exp-w4nl64-i8-hc8m`): **routed experts as 4-bit codes into a
**1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin 16-level non-uniform codebook** after a 128-point Walsh–Hadamard rotation
with the token knob; the RADGRAM2 grams are format-identical across pins and (E4M3 scale per 64 weights; codes by GPTQ on a 10M-token calibration with
cross the boundary by design), **24 whole experts per layer cast bf16** (the per-expert down-proj Hessians, refined by 3 coordinate-descent sweeps),
PROTECTED set, derived from the calibration grams' energy share), **10 whole bf16-protected experts/layer**, **int8 W8A8 trunk/lm_head**
**int8 trunk/lm_head**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE n-gram (scale per 128 columns searched), **fp8 HC mixtures**, PLE n-gram table at
table at fp8 fixed scale (47.7 GiB in the container, from 95.4 GiB bf16). fp8 fixed scale (47.7 GiB), **MTP head and bf16 vision tower in the
The plugin's **one-table-one-encoding law** (measured at rad-convert container**. Card quality (full-vocab teacher-forced KL vs bf16, ~75K
declare): one expert table takes ONE encoding; only whole bf16 experts are positions): mean 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %.
legal beside it — no w4/fp8 split of the down/gate_up pair (24-value **MODEL_FILE is the only serving argument that moved** (2026-10-15); every
`rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16 number of the 2026-10-13 shape below carries verbatim. The plugin's
reference (recorded on the same 1.0.13 pin) is the precision referee; **one-table-one-encoding law** (fleet-measured at rad-convert declare):
the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar one expert table takes ONE encoding; only whole bf16 experts are legal
(122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook. beside it. Rollback: revert the swap commit — the in-house w4dn8 `.rad`
Rollback: revert the swap commit — the fp8-native `.rad` (KLD-gated Swift1.5 conversion) stays on disk; the fp8-native `.rad` was
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk. deleted 2026-10-15 (rebuild: `scripts/download-frank-archon.sh` +
`scripts/build-frank-archon-rad.sh`).
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`, **Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
`--max-model-len 200000` (the harness contextWindow must quote this), `--max-model-len 200000` (the harness contextWindow must quote this),
@@ -630,8 +631,9 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
`--mm-max-patches`); `/metrics` carries vLLM-compatible names + `--mm-max-patches`); `/metrics` carries vLLM-compatible names +
`radiance:*` families — the Dankclaw Prometheus scrape is re-added. `radiance:*` families — the Dankclaw Prometheus scrape is re-added.
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`. Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md`.
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`. Retired Swift1.5 w4dn8 era (payload on disk = rollback): `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
Retired fp8-native era (payload deleted 2026-10-15, rebuildable): `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`. Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`. Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`. Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
@@ -1091,7 +1093,7 @@ Active:
|---------|---------| |---------|---------|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` | | frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` | | frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` | | frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` | | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` | | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -1099,6 +1101,7 @@ Retired (rollback targets and history):
| Runtime | Host | Window | Runbook | | Runtime | Host | Window | Runbook |
|---------|------|--------|---------| |---------|------|--------|---------|
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` on disk; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` | | llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` | | ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` | | tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
@@ -0,0 +1,120 @@
# frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, 2× R9700 TP=2
**Status: LIVE — deployed 2026-10-15.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
The in-house w4dn8 conversion of the Swift1.5 finetune
([[frank-archon-radiance-swift15-w4dn8]]) is replaced by the **engine
author's published container**:
[`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`](https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe)
— the official `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint rad-converted by
StillDeadcode. Same port, alias, container name, systemd unit, digest-pinned
engine pin **1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…`), and
**every serving argument carried verbatim** from the 2026-10-13 shape.
Rollback: revert the swap commit — the w4dn8 `.rad` stays on disk.
## The payload
`qwen3.8-next-flash-fp8-iq4r-moe.rad` — **121,969,901,568 B = 116,322 MiB**,
sha256 `0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20`
(Hub LFS oid), fetched byte-exact by `scripts/download-frank-archon-iq4r.sh`
(hfget resume + size + sha256; staging dir
`/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe/` keeps the
card, LICENSE, and recipe `qwen4exp-w4nl64-i8-hc8m` as provenance).
| Plane | Encoding (publisher recipe `qwen4exp-w4nl64-i8-hc8m`) |
| --- | --- |
| Routed experts | 4-bit codes into a 16-level **non-uniform** codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes by GPTQ (10M-token calibration, per-expert down-proj Hessians) + 3 coordinate-descent sweeps |
| Protected experts | **ten** per layer, whole-expert bf16 (down-projection energy) |
| Trunk + lm_head | int8 W8A8, scale per 128 columns searched; HC mixtures E4M3 |
| PLE n-gram | fp8 fixed-scale, in-container (served `--ngram-placement ram`, same 47.7 GiB pin law) |
| Speculator | the model's own MTP head (card default depth 3; served at 2) |
| Vision | bf16 vision tower, in-container |
**Base model: the OFFICIAL (censored) Qwen3.8-Flash-Next** — the fleet's
only frontier model at base alignment; the abliterated family lives on
frank-dragoon. Card quality: full-vocab teacher-forced KL vs bf16 over
~75K positions — mean 0.0789 all-tokens / 0.0239 assistant turns, top-1
agreement 92.85 %.
## Why this swap
- **Publisher-graded precision**: the w4nl codebook + coordinate-descent
refinement is the engine author's own quality ceiling for this
checkpoint; the card's KL ladder (p99 1.65, p99.9 5.33) is published and
reproducible, where our w4dn8 gate measured against our own reference.
- **Formal parity**: the served file is the one the engine author supports
— decode/prefill behavior on 1.0.13 is their tested path (the card's run
line speaks the 1.0.13-era flags; the `--vram-weights-mib`/`--vram-kv-mib`
pair we ship is accepted by pin 1.0.13 per the 2026-10-06 A/B, so the
shape needed no translation).
- **Cost**: the Swift1.5 finetune's refusal-bending behavior leaves this
endpoint (it returns to the official base alignment); the 4-slot/200k
economics and all shape laws carry over unchanged.
## Shape (unchanged from 2026-10-13)
All of it, verbatim — the payload moved 116,416 → 116,322 MiB, inside the
same budget:
- `--tp 2 --tp-wire wht6`, `--max-num-seqs 3`, `--max-model-len 200000`
(capacity contract 3 × 200,000), `--kv-cache-dtype fp8`
- `--placement expert_tiered --host-pool-mib 44928` + explicit
`--vram-weights-mib 25600 --vram-kv-mib 5120` (30,720 of the engine's
30,969 MiB ceiling; pool = contract + 1.3 %)
- `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`
(MNBT 2048 law), prefix cache 8 GiB host + 64 GiB disk,
`--ngram-placement ram`, froggeric v22.5 `--override-chat-template`
- TTM preflight floor 150,000 / target 204,800 MiB; pinned ledger
2 × 44 GiB pool + 47.7 GiB PLE + 8 GiB prefix ≈ 147 GiB (see
[[frank-archon-radiance-swift15-w4dn8]] for the ledger arithmetic and
the KV dtype law — both are engine properties, payload-independent)
The publisher's card run line (`--host-pool-mib 12288`,
`--expert-vs-cache-ratio 0.82`, seqs 8, spec 3, ~32 GB host RAM) is their
shape on stock 32 GB cards; the fleet ships its own measured shape against
the same file.
## Cutover record — 2026-10-15
1. `rm` (Frank's one-time authorization): the raw
`ukisai/Swift1.5-Qwen3.8-Flash-Next` BF16 checkpoint (336 GiB, spent
conversion source) and `qwen4exp-fp8-native.rad` (173 GiB, two eras
back) left `/home/dank/models` — 150 GB free → 658 GB free. The fp8-native
payload is **no longer on disk**: rebuild =
`scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh`.
2. Fetch: `scripts/download-frank-archon-iq4r.sh --fetch` as transient unit
`iq4r-fetch` (detached-job protocol; log
`/home/dank/logs/iq4r-fetch.log`, rc sentinel `…rc`).
3. Deploy: `scripts/deploy.sh dankserver --service frank-archon` — launched
detached (the swap restarts the very endpoint the operator session runs
on; the harness timeout catches the session on return).
4. **Boot ledger (fill after first boot)**: `weights/kv` per-rank budgets,
`kv_attn` page size, slab slot count, `backs N sequences` line, sampler
rows, unclaimed MiB. The load of this file on pin 1.0.13 is the gate: a
refusal naming the w4nl codebook means the pin predates the encoding —
the documented repair is the pin bump to the card's era, which the
2026-10-06 1.2.0 A/B (decode 29.21 vs 30.08 t/s, acceptance 0.7123 vs
0.7549) makes a paid decision.
5. **Post-boot measurements (pending)**: decode ladder (99.0 t/s single
@ 43.5k, 93.3 @ 197k, 3-way 113.9/43.3/10.9 = 18.4 t/s aggregate —
these are the **w4dn8** numbers, the record to beat, not this file's),
slab-limited prefill (22208/3,397 → 25600/3,584 t/s A-cold 43.5k), draft
acceptance, and `scripts/probe-frank-archon-kv.py` leg D.
## Rollback
Revert the swap commit (`scripts/frank-archon.env` MODEL_FILE line) +
redeploy: `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad`
(116,416 MiB) stays on disk, KLD-gated, same shape. The fp8-native era is
rebuild-only (see the cutover record). The w4dn8 runbook keeps every shape
law, the KV dtype table, the slab ladder, and the tuning levers:
[[frank-archon-radiance-swift15-w4dn8]].
## Related
- [[frank-archon-radiance-swift15-w4dn8]] — the replaced payload and the
full shape-law library (slab slope, KV dtype arithmetic, TTM law)
- [[frank-archon-radiance-qwen38-native-fp8]] — the fp8-native era
(rebuild-only now; owns the descriptor-selection law)
- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
- [[deployment]] — the service-scoped deploy this cutover rides
+25 -7
View File
@@ -3,14 +3,17 @@
**Dankserver runs the radiance engine (StillDeadcode) on frank-archon** **Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint, (digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
driving `podman run` directly, root systemd unit, :7770): driving `podman run` directly, root systemd unit, :7770):
`ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4** the **publisher's** 4-bit-codebook container of the official
(`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s `Qwen/Qwen3.8-Flash-Next` (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
sha256-verified at fetch — w4nl codebook experts + int8 trunk, PLE n-gram,
MTP head and vision tower in-container), on BOTH R9700s
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the (explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE 30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2, table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005 froggeric v22.5 template. Rollback: the in-house Swift1.5 w4dn8 `.rad` (on
and frank-archon-vllm-shapeG-20261003), disk); tags frank-archon-freetoken-pr587-20261005 and
frank-archon-vllm-shapeG-20261003),
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2 and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned (frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
@@ -37,10 +40,10 @@ graph LR
gfx1201, 32 GiB each] gfx1201, 32 GiB each]
B2[radiance StillDeadcode, digest-pinned image B2[radiance StillDeadcode, digest-pinned image
podman run, frank-archon, port 7770] podman run, frank-archon, port 7770]
C[Swift1.5-Qwen3.8-Flash-Next GPTQ w4 .rad C[Publisher qwen3.8-next-flash-fp8-iq4r-moe .rad
2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM, 2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM,
PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2, PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
4 x 200000, froggeric v22.5] 3 x 200000, froggeric v22.5]
end end
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04] subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
@@ -153,7 +156,22 @@ frank-zealot, port 7771]
target; payload on disk; owns the measured boot-pair table)**: the same target; payload on disk; owns the measured boot-pair table)**: the same
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower — runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
104.8 tok/s, 3,571 MiB true free 104.8 tok/s, 3,571 MiB true free
- [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**: - [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE since 2026-10-15**:
the engine author's published container of the OFFICIAL (base-alignment)
Qwen3.8-Flash-Next — `qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
Hub-sha256-verified at fetch: 4-bit codes into a 16-level non-uniform
codebook after a Walsh–Hadamard rotation (GPTQ + 3 coordinate-descent
sweeps, 10M-token calibration), ten bf16-protected experts per layer,
int8 trunk/lm_head, fp8 HC + PLE, MTP head, bf16 vision tower; card KL
0.0789 all-tokens / top-1 92.85 % vs bf16. **MODEL_FILE is the only line
that moved** — the 2026-10-13 shape (25600/5120 pair, 44928 pool, fp8 KV
3 × 200,000, MTP 2, wht6, MNBT 2048, froggeric v22.5) carries verbatim.
Rollback: revert the swap commit (w4dn8 `.rad` stays on disk). The
fp8-native `.rad` and the raw Swift1.5 BF16 source were DELETED from
dankserver 2026-10-15 (Frank's one-time rm, volume for this payload;
rebuild via the fetch + build scripts named in the fp8-native runbook)
- [[frank-archon-radiance-swift15-w4dn8]] — **RETIRED 2026-10-08 → 10-15
(superseded by the publisher iq4r swap; payload on disk as the rollback)**:
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container); 1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
+10 -2
View File
@@ -182,12 +182,20 @@ The LLM inference fleet on the two hosts. Full index:
### Deployment and Topology ### Deployment and Topology
- [[deployment]] — unified deploy and update process for both servers - [[deployment]] — unified deploy and update process for both servers
- [[frank-archon-radiance-swift15-w4dn8]] — STAGED frank-archon runbook: - [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE frank-archon runbook
(2026-10-15)**: the PUBLISHER's container of the official
Qwen3.8-Flash-Next (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
w4nl 4-bit-codebook experts + int8 trunk, Hub-sha256-verified at fetch)
swapped in as a MODEL_FILE-only change — the 2026-10-13 shape (slab
25600 / fp8-KV 5120, 3 × 200,000, host pool 44,928, MTP depth 2) carries
verbatim; rollback = revert the swap commit (the w4dn8 `.rad` stays on
disk; the fp8-native `.rad` was deleted 2026-10-15, rebuildable)
- [[frank-archon-radiance-swift15-w4dn8]] — RETIRED 2026-10-08 → 10-15 frank-archon runbook:
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s; 1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
= revert that line (the FP8-native `.rad` stays on disk) = revert that line (the FP8-native `.rad` stays on disk)
- [[frank-archon-radiance-qwen38-native-fp8]] — the ACTIVE frank-archon - [[frank-archon-radiance-qwen38-native-fp8]] — the RETIRED 2026-10-05 → 10-08 frank-archon
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL 287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert `Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert