Frank
2def6282b5
The Swift1.5 w4dn8 .rad, the GLM-4.7 GGUF, and every derived model cache were purged from dankserver (trash-staged, emptied with approval; 763 GiB free). Rollback of the iq4r swap is now an HF re-download of the byte-exact frank8465 mirror (LFS sha256 09dd9306...). Corrects the w4dn8 runbook header (still said LIVE) and rebuilds both Rollback sections; log.md carries the purge record.
155 lines
9.0 KiB
Markdown
155 lines
9.0 KiB
Markdown
# frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, 2× R9700 TP=2
|
||
|
||
**Status: LIVE — deployed 2026-10-15.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
|
||
|
||
The in-house w4dn8 conversion of the Swift1.5 finetune
|
||
([[frank-archon-radiance-swift15-w4dn8]]) is replaced by the **engine
|
||
author's published container**:
|
||
[`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`](https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe)
|
||
— the official `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint rad-converted by
|
||
StillDeadcode. Same port, alias, container name, systemd unit, digest-pinned
|
||
engine pin **1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…`), and
|
||
**every serving argument carried verbatim** from the 2026-10-13 shape.
|
||
Rollback: re-download the w4dn8 `.rad` from the HF mirror + point
|
||
`MODEL_FILE` at it (the local copy was purged 2026-10-17, see Rollback).
|
||
|
||
## The payload
|
||
|
||
`qwen3.8-next-flash-fp8-iq4r-moe.rad` — **121,969,901,568 B = 116,322 MiB**,
|
||
sha256 `0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20`
|
||
(Hub LFS oid), fetched byte-exact by `scripts/download-frank-archon-iq4r.sh`
|
||
(hfget resume + size + sha256; staging dir
|
||
`/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe/` keeps the
|
||
card, LICENSE, and recipe `qwen4exp-w4nl64-i8-hc8m` as provenance).
|
||
|
||
| Plane | Encoding (publisher recipe `qwen4exp-w4nl64-i8-hc8m`) |
|
||
| --- | --- |
|
||
| Routed experts | 4-bit codes into a 16-level **non-uniform** codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes by GPTQ (10M-token calibration, per-expert down-proj Hessians) + 3 coordinate-descent sweeps |
|
||
| Protected experts | **ten** per layer, whole-expert bf16 (down-projection energy) |
|
||
| Trunk + lm_head | int8 W8A8, scale per 128 columns searched; HC mixtures E4M3 |
|
||
| PLE n-gram | fp8 fixed-scale, in-container (served `--ngram-placement ram`, same 47.7 GiB pin law) |
|
||
| Speculator | the model's own MTP head (card default depth 3; served at 2) |
|
||
| Vision | bf16 vision tower, in-container |
|
||
|
||
**Base model: the OFFICIAL (censored) Qwen3.8-Flash-Next** — the fleet's
|
||
only frontier model at base alignment; the abliterated family lives on
|
||
frank-dragoon. Card quality: full-vocab teacher-forced KL vs bf16 over
|
||
~75K positions — mean 0.0789 all-tokens / 0.0239 assistant turns, top-1
|
||
agreement 92.85 %.
|
||
|
||
## Why this swap
|
||
|
||
- **Publisher-graded precision**: the w4nl codebook + coordinate-descent
|
||
refinement is the engine author's own quality ceiling for this
|
||
checkpoint; the card's KL ladder (p99 1.65, p99.9 5.33) is published and
|
||
reproducible, where our w4dn8 gate measured against our own reference.
|
||
- **Formal parity**: the served file is the one the engine author supports
|
||
— decode/prefill behavior on 1.0.13 is their tested path (the card's run
|
||
line speaks the 1.0.13-era flags; the `--vram-weights-mib`/`--vram-kv-mib`
|
||
pair we ship is accepted by pin 1.0.13 per the 2026-10-06 A/B, so the
|
||
shape needed no translation).
|
||
- **Cost**: the Swift1.5 finetune's refusal-bending behavior leaves this
|
||
endpoint (it returns to the official base alignment); the 4-slot/200k
|
||
economics and all shape laws carry over unchanged.
|
||
|
||
## Shape (unchanged from 2026-10-13)
|
||
|
||
All of it, verbatim — the payload moved 116,416 → 116,322 MiB, inside the
|
||
same budget:
|
||
|
||
- `--tp 2 --tp-wire wht6`, `--max-num-seqs 3`, `--max-model-len 200000`
|
||
(capacity contract 3 × 200,000), `--kv-cache-dtype fp8`
|
||
- `--placement expert_tiered --host-pool-mib 44928` + explicit
|
||
`--vram-weights-mib 25600 --vram-kv-mib 5120` (30,720 of the engine's
|
||
30,969 MiB ceiling; pool = contract + 1.3 %)
|
||
- `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`
|
||
(MNBT 2048 law), prefix cache 8 GiB host + 64 GiB disk,
|
||
`--ngram-placement ram`, froggeric v22.5 `--override-chat-template`
|
||
- TTM preflight floor 150,000 / target 204,800 MiB; pinned ledger
|
||
2 × 44 GiB pool + 47.7 GiB PLE + 8 GiB prefix ≈ 147 GiB (see
|
||
[[frank-archon-radiance-swift15-w4dn8]] for the ledger arithmetic and
|
||
the KV dtype law — both are engine properties, payload-independent)
|
||
|
||
The publisher's card run line (`--host-pool-mib 12288`,
|
||
`--expert-vs-cache-ratio 0.82`, seqs 8, spec 3, ~32 GB host RAM) is their
|
||
shape on stock 32 GB cards; the fleet ships its own measured shape against
|
||
the same file.
|
||
|
||
## Cutover record — 2026-10-15
|
||
|
||
1. `rm` (Frank's one-time authorization): the raw
|
||
`ukisai/Swift1.5-Qwen3.8-Flash-Next` BF16 checkpoint (336 GiB, spent
|
||
conversion source) and `qwen4exp-fp8-native.rad` (173 GiB, two eras
|
||
back) left `/home/dank/models` — 150 GB free → 658 GB free. The fp8-native
|
||
payload is **no longer on disk**: rebuild =
|
||
`scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh`.
|
||
2. Fetch: `scripts/download-frank-archon-iq4r.sh --fetch` as transient unit
|
||
`iq4r-fetch` (detached-job protocol; log
|
||
`/home/dank/logs/iq4r-fetch.log`, rc sentinel `…rc`).
|
||
3. Deploy: `scripts/deploy.sh dankserver --service frank-archon` — launched
|
||
detached (the swap restarts the very endpoint the operator session runs
|
||
on; the harness timeout catches the session on return).
|
||
4. **Boot ledger (2026-10-15, first boot, pin 1.0.13 — the w4nl codebook
|
||
loaded, no pin bump)**: `kv_attn` fp8 page **26.00 KiB** at **607,740
|
||
tokens** (151,935 blocks = 3.77 GiB) — byte-identical to the w4dn8 boot,
|
||
the sizing law carries; `kv_qsa_bkey` 3.25 KiB at the same token count;
|
||
`kv_gdn_state` 54 MiB × 6 slots, `kv_gdn_conv` 1.76 MiB × 3,
|
||
`kv_qsa_tail` 19.5 KiB × 6; pool **5.00 GiB**, prefix checkpoints 8 ×
|
||
56 MiB; the sizing line reads `kv cache 5.00 GiB … backs 3 sequences all
|
||
at 200000 tokens at once, which is the worst case`; samplers print **9
|
||
rows** per rank (3 × (1+2), inside the wht6 25-row bound);
|
||
`speculation: 'mtp' drafts 2 token(s) a step in 2 serial pass(es)` +
|
||
prompt-lookup fallback; slab **17,610 (rank0) / 17,658 (rank1) slots**
|
||
(64 shadow) — above the w4dn8 record of 17,548 (ten bf16-protected
|
||
experts per layer replace twenty-four, so more of the 4-bit table
|
||
resides), pinned pool 7,532/7,484 slots; KV lending **4.22 GiB**
|
||
(3,571/3,568 mover slots); arenas 896.02/988.17 MiB charged before the
|
||
pair split; prefix cache RECOVERED 900,176 blocks + 126 checkpoint slots
|
||
from the 64 GiB disk tier (same arch + tokenizer as w4dn8 — the tier
|
||
survived the payload swap); froggeric template 28,234 B; vision encoder
|
||
live (≤ 4,194,304 px, 16,384 patches/pass). Health gate passed in ~100 s
|
||
(radiance binds the port after the full load).
|
||
5. **Vision probe (2026-10-15)**: 300 px solid-red PNG via `image_url` —
|
||
reasoning stream: "a solid fill of a deep, saturated red hue". Content
|
||
came back EMPTY at `max_tokens 64`: the thinking budget consumed all 64
|
||
(`reasoning_tokens: 64`, finish `length`) — fleet law, probe thinking
|
||
models with ≥ a few hundred tokens.
|
||
6. **e2e gate**: deploy exited DEPLOY INCOMPLETE — 30/33 Playwright green,
|
||
3 red (`12-crawl`, `13-sso` search SSO, `16-marreta`) on DANKCLAW
|
||
surfaces (vhosts intermittently returned `ERR_CONNECTION_RESET` from the
|
||
workstation); frank-archon's own health gate passed. The swap touched no
|
||
dankclaw runtime file (only homepage services.json text). Census of the
|
||
Caddy/auth stack pending; see `log.md`.
|
||
7. **Post-boot measurements (pending)**: decode ladder (99.0 t/s single
|
||
@ 43.5k, 93.3 @ 197k, 3-way 113.9/43.3/10.9 = 18.4 t/s aggregate —
|
||
these are the **w4dn8** numbers, the record to beat, not this file's),
|
||
slab-limited prefill (22208/3,397 → 25600/3,584 t/s A-cold 43.5k), draft
|
||
acceptance, and `scripts/probe-frank-archon-kv.py` leg D.
|
||
|
||
## Rollback
|
||
|
||
Revert the swap commit (`scripts/frank-archon.env` MODEL_FILE line) +
|
||
redeploy: `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad`
|
||
(116,416 MiB), KLD-gated, same shape — **no longer on disk**. The 2026-10-17
|
||
disk purge (Frank's approval, trash-staged then emptied: 763 GiB free after)
|
||
removed it; the byte-exact copy lives on the Hub as
|
||
`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`
|
||
(`qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`)
|
||
and re-fetches through `scripts/download-frank-archon-iq4r.sh`'s
|
||
hfget/verify pattern retargeted at that repo. The same purge deleted the
|
||
`radiance-state` kvcache tiers, `~/.radiance-cache-w4a8-*`,
|
||
`~/.cache/frank-primary`, and `/opt/vllm/moremodels` (GLM-4.7 GGUF) — all
|
||
derived or retired. The fp8-native era is rebuild-only (see the cutover
|
||
record). The w4dn8 runbook keeps every shape law, the KV dtype table, the
|
||
slab ladder, and the tuning levers:
|
||
[[frank-archon-radiance-swift15-w4dn8]].
|
||
|
||
## Related
|
||
|
||
- [[frank-archon-radiance-swift15-w4dn8]] — the replaced payload and the
|
||
full shape-law library (slab slope, KV dtype arithmetic, TTM law)
|
||
- [[frank-archon-radiance-qwen38-native-fp8]] — the fp8-native era
|
||
(rebuild-only now; owns the descriptor-selection law)
|
||
- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
|
||
- [[deployment]] — the service-scoped deploy this cutover rides
|