docs(frank-archon): iq4r era runbook + index/AGENTS re-key
New runbook frank-archon-radiance-qwen38-iq4r.md (publisher container, payload chain, shape-verbatim cutover record with the boot-ledger and measurement slots left open for the first boot, rollback). LLM index, root vault index, and AGENTS.md (fleet table row, section, runbook index: iq4r ACTIVE, w4dn8 RETIRED/on-disk, fp8-native deleted). Full suite green.
This commit is contained in:
1 parent
6dcf4a5913
commit
f44c39812f
4 files changed
+185
-36
No files matched your search
@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
|||||||
|
|
||||||
| Target | Service | Runtime | Model | Port |
|
| Target | Service | Runtime | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **44 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the **30,969 MiB** the engine's own budget table makes claimable: card 32,624 MiB, settle floor 209 MiB), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (pool 5,120 MiB = the 3-slot contract + 1.3 % — the only dtype that funds 3 × 200,000 without gutting the slab, since bf16 bills 13.8 KiB/token), **MTP spec decode depth 2**, `--max-num-seqs 3` (the capacity contract is 3 × 200,000; 3 × (1+2) = 9 all-reduce rows, inside the 25-row wht6 exactness bound), `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8); prefill is **slab-limited and pays for it** — 22,208 → 3,397, 24,000 → 3,290, 25,600 → **3,584** t/s at 43.5k single-stream, 3-way 197k aggregate 3,051 → **3,589** t/s, with a cliff at ~26 GiB (26,071 → 4,379). Cutover 2026-10-08 from fp8-native (payload on disk = rollback); 3-slot fp8-KV shape 2026-10-13, re-cut twice in the slab's favour (22208/6912 at 4 slots → 24000/5120 → 25600/5120); the 4-slot shape (22208/6912, `--max-num-seqs 4`) is the documented rollback. | 7770 |
|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: revert the swap commit (w4dn8 `.rad` on disk); the fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
||||||
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
||||||
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
||||||
|
|
||||||
@@ -498,32 +498,33 @@ Rollback: revert the swap commit — the Gemma4 payload
|
|||||||
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
||||||
|
|
||||||
|
|
||||||
### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
|
### frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, TP=2 (Dankserver, ACTIVE)
|
||||||
|
|
||||||
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
||||||
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
||||||
for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root
|
for serve; `libr4d` gfx1201 kernels, plain `podman run`, root
|
||||||
systemd unit, container name = unit name) serving
|
systemd unit, container name = unit name) serving the
|
||||||
**ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards,
|
**publisher's container of the official Qwen/Qwen3.8-Flash-Next**:
|
||||||
397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh`
|
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` →
|
||||||
with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to
|
`qwen3.8-next-flash-fp8-iq4r-moe.rad` (116,322 MiB, Hub-LFS sha256
|
||||||
`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ
|
verified at fetch by `scripts/download-frank-archon-iq4r.sh`; recipe
|
||||||
w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine
|
`qwen4exp-w4nl64-i8-hc8m`): **routed experts as 4-bit codes into a
|
||||||
**1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin
|
16-level non-uniform codebook** after a 128-point Walsh–Hadamard rotation
|
||||||
with the token knob; the RADGRAM2 grams are format-identical across pins and
|
(E4M3 scale per 64 weights; codes by GPTQ on a 10M-token calibration with
|
||||||
cross the boundary by design), **24 whole experts per layer cast bf16** (the
|
per-expert down-proj Hessians, refined by 3 coordinate-descent sweeps),
|
||||||
PROTECTED set, derived from the calibration grams' energy share),
|
**10 whole bf16-protected experts/layer**, **int8 W8A8 trunk/lm_head**
|
||||||
**int8 trunk/lm_head**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE n-gram
|
(scale per 128 columns searched), **fp8 HC mixtures**, PLE n-gram table at
|
||||||
table at fp8 fixed scale (47.7 GiB in the container, from 95.4 GiB bf16).
|
fp8 fixed scale (47.7 GiB), **MTP head and bf16 vision tower in the
|
||||||
The plugin's **one-table-one-encoding law** (measured at rad-convert
|
container**. Card quality (full-vocab teacher-forced KL vs bf16, ~75K
|
||||||
declare): one expert table takes ONE encoding; only whole bf16 experts are
|
positions): mean 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %.
|
||||||
legal beside it — no w4/fp8 split of the down/gate_up pair (24-value
|
**MODEL_FILE is the only serving argument that moved** (2026-10-15); every
|
||||||
`rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16
|
number of the 2026-10-13 shape below carries verbatim. The plugin's
|
||||||
reference (recorded on the same 1.0.13 pin) is the precision referee;
|
**one-table-one-encoding law** (fleet-measured at rad-convert declare):
|
||||||
the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar
|
one expert table takes ONE encoding; only whole bf16 experts are legal
|
||||||
(122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook.
|
beside it. Rollback: revert the swap commit — the in-house w4dn8 `.rad`
|
||||||
Rollback: revert the swap commit — the fp8-native `.rad`
|
(KLD-gated Swift1.5 conversion) stays on disk; the fp8-native `.rad` was
|
||||||
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
|
deleted 2026-10-15 (rebuild: `scripts/download-frank-archon.sh` +
|
||||||
|
`scripts/build-frank-archon-rad.sh`).
|
||||||
|
|
||||||
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
|
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
|
||||||
`--max-model-len 200000` (the harness contextWindow must quote this),
|
`--max-model-len 200000` (the harness contextWindow must quote this),
|
||||||
@@ -630,8 +631,9 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
|
|||||||
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
||||||
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
||||||
|
|
||||||
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
|
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md`.
|
||||||
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
|
Retired Swift1.5 w4dn8 era (payload on disk = rollback): `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
|
||||||
|
Retired fp8-native era (payload deleted 2026-10-15, rebuildable): `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
|
||||||
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
|
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
|
||||||
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
||||||
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
|
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
|
||||||
@@ -1091,7 +1093,7 @@ Active:
|
|||||||
|---------|---------|
|
|---------|---------|
|
||||||
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
||||||
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||||
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
|
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
|
||||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||||
|
|
||||||
@@ -1099,6 +1101,7 @@ Retired (rollback targets and history):
|
|||||||
|
|
||||||
| Runtime | Host | Window | Runbook |
|
| Runtime | Host | Window | Runbook |
|
||||||
|---------|------|--------|---------|
|
|---------|------|--------|---------|
|
||||||
|
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` on disk; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
|
||||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||||
|
|||||||
@@ -0,0 +1,120 @@
|
|||||||
|
# frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, 2× R9700 TP=2
|
||||||
|
|
||||||
|
**Status: LIVE — deployed 2026-10-15.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
|
||||||
|
|
||||||
|
The in-house w4dn8 conversion of the Swift1.5 finetune
|
||||||
|
([[frank-archon-radiance-swift15-w4dn8]]) is replaced by the **engine
|
||||||
|
author's published container**:
|
||||||
|
[`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`](https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe)
|
||||||
|
— the official `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint rad-converted by
|
||||||
|
StillDeadcode. Same port, alias, container name, systemd unit, digest-pinned
|
||||||
|
engine pin **1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…`), and
|
||||||
|
**every serving argument carried verbatim** from the 2026-10-13 shape.
|
||||||
|
Rollback: revert the swap commit — the w4dn8 `.rad` stays on disk.
|
||||||
|
|
||||||
|
## The payload
|
||||||
|
|
||||||
|
`qwen3.8-next-flash-fp8-iq4r-moe.rad` — **121,969,901,568 B = 116,322 MiB**,
|
||||||
|
sha256 `0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20`
|
||||||
|
(Hub LFS oid), fetched byte-exact by `scripts/download-frank-archon-iq4r.sh`
|
||||||
|
(hfget resume + size + sha256; staging dir
|
||||||
|
`/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe/` keeps the
|
||||||
|
card, LICENSE, and recipe `qwen4exp-w4nl64-i8-hc8m` as provenance).
|
||||||
|
|
||||||
|
| Plane | Encoding (publisher recipe `qwen4exp-w4nl64-i8-hc8m`) |
|
||||||
|
| --- | --- |
|
||||||
|
| Routed experts | 4-bit codes into a 16-level **non-uniform** codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes by GPTQ (10M-token calibration, per-expert down-proj Hessians) + 3 coordinate-descent sweeps |
|
||||||
|
| Protected experts | **ten** per layer, whole-expert bf16 (down-projection energy) |
|
||||||
|
| Trunk + lm_head | int8 W8A8, scale per 128 columns searched; HC mixtures E4M3 |
|
||||||
|
| PLE n-gram | fp8 fixed-scale, in-container (served `--ngram-placement ram`, same 47.7 GiB pin law) |
|
||||||
|
| Speculator | the model's own MTP head (card default depth 3; served at 2) |
|
||||||
|
| Vision | bf16 vision tower, in-container |
|
||||||
|
|
||||||
|
**Base model: the OFFICIAL (censored) Qwen3.8-Flash-Next** — the fleet's
|
||||||
|
only frontier model at base alignment; the abliterated family lives on
|
||||||
|
frank-dragoon. Card quality: full-vocab teacher-forced KL vs bf16 over
|
||||||
|
~75K positions — mean 0.0789 all-tokens / 0.0239 assistant turns, top-1
|
||||||
|
agreement 92.85 %.
|
||||||
|
|
||||||
|
## Why this swap
|
||||||
|
|
||||||
|
- **Publisher-graded precision**: the w4nl codebook + coordinate-descent
|
||||||
|
refinement is the engine author's own quality ceiling for this
|
||||||
|
checkpoint; the card's KL ladder (p99 1.65, p99.9 5.33) is published and
|
||||||
|
reproducible, where our w4dn8 gate measured against our own reference.
|
||||||
|
- **Formal parity**: the served file is the one the engine author supports
|
||||||
|
— decode/prefill behavior on 1.0.13 is their tested path (the card's run
|
||||||
|
line speaks the 1.0.13-era flags; the `--vram-weights-mib`/`--vram-kv-mib`
|
||||||
|
pair we ship is accepted by pin 1.0.13 per the 2026-10-06 A/B, so the
|
||||||
|
shape needed no translation).
|
||||||
|
- **Cost**: the Swift1.5 finetune's refusal-bending behavior leaves this
|
||||||
|
endpoint (it returns to the official base alignment); the 4-slot/200k
|
||||||
|
economics and all shape laws carry over unchanged.
|
||||||
|
|
||||||
|
## Shape (unchanged from 2026-10-13)
|
||||||
|
|
||||||
|
All of it, verbatim — the payload moved 116,416 → 116,322 MiB, inside the
|
||||||
|
same budget:
|
||||||
|
|
||||||
|
- `--tp 2 --tp-wire wht6`, `--max-num-seqs 3`, `--max-model-len 200000`
|
||||||
|
(capacity contract 3 × 200,000), `--kv-cache-dtype fp8`
|
||||||
|
- `--placement expert_tiered --host-pool-mib 44928` + explicit
|
||||||
|
`--vram-weights-mib 25600 --vram-kv-mib 5120` (30,720 of the engine's
|
||||||
|
30,969 MiB ceiling; pool = contract + 1.3 %)
|
||||||
|
- `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`
|
||||||
|
(MNBT 2048 law), prefix cache 8 GiB host + 64 GiB disk,
|
||||||
|
`--ngram-placement ram`, froggeric v22.5 `--override-chat-template`
|
||||||
|
- TTM preflight floor 150,000 / target 204,800 MiB; pinned ledger
|
||||||
|
2 × 44 GiB pool + 47.7 GiB PLE + 8 GiB prefix ≈ 147 GiB (see
|
||||||
|
[[frank-archon-radiance-swift15-w4dn8]] for the ledger arithmetic and
|
||||||
|
the KV dtype law — both are engine properties, payload-independent)
|
||||||
|
|
||||||
|
The publisher's card run line (`--host-pool-mib 12288`,
|
||||||
|
`--expert-vs-cache-ratio 0.82`, seqs 8, spec 3, ~32 GB host RAM) is their
|
||||||
|
shape on stock 32 GB cards; the fleet ships its own measured shape against
|
||||||
|
the same file.
|
||||||
|
|
||||||
|
## Cutover record — 2026-10-15
|
||||||
|
|
||||||
|
1. `rm` (Frank's one-time authorization): the raw
|
||||||
|
`ukisai/Swift1.5-Qwen3.8-Flash-Next` BF16 checkpoint (336 GiB, spent
|
||||||
|
conversion source) and `qwen4exp-fp8-native.rad` (173 GiB, two eras
|
||||||
|
back) left `/home/dank/models` — 150 GB free → 658 GB free. The fp8-native
|
||||||
|
payload is **no longer on disk**: rebuild =
|
||||||
|
`scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh`.
|
||||||
|
2. Fetch: `scripts/download-frank-archon-iq4r.sh --fetch` as transient unit
|
||||||
|
`iq4r-fetch` (detached-job protocol; log
|
||||||
|
`/home/dank/logs/iq4r-fetch.log`, rc sentinel `…rc`).
|
||||||
|
3. Deploy: `scripts/deploy.sh dankserver --service frank-archon` — launched
|
||||||
|
detached (the swap restarts the very endpoint the operator session runs
|
||||||
|
on; the harness timeout catches the session on return).
|
||||||
|
4. **Boot ledger (fill after first boot)**: `weights/kv` per-rank budgets,
|
||||||
|
`kv_attn` page size, slab slot count, `backs N sequences` line, sampler
|
||||||
|
rows, unclaimed MiB. The load of this file on pin 1.0.13 is the gate: a
|
||||||
|
refusal naming the w4nl codebook means the pin predates the encoding —
|
||||||
|
the documented repair is the pin bump to the card's era, which the
|
||||||
|
2026-10-06 1.2.0 A/B (decode 29.21 vs 30.08 t/s, acceptance 0.7123 vs
|
||||||
|
0.7549) makes a paid decision.
|
||||||
|
5. **Post-boot measurements (pending)**: decode ladder (99.0 t/s single
|
||||||
|
@ 43.5k, 93.3 @ 197k, 3-way 113.9/43.3/10.9 = 18.4 t/s aggregate —
|
||||||
|
these are the **w4dn8** numbers, the record to beat, not this file's),
|
||||||
|
slab-limited prefill (22208/3,397 → 25600/3,584 t/s A-cold 43.5k), draft
|
||||||
|
acceptance, and `scripts/probe-frank-archon-kv.py` leg D.
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
Revert the swap commit (`scripts/frank-archon.env` MODEL_FILE line) +
|
||||||
|
redeploy: `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad`
|
||||||
|
(116,416 MiB) stays on disk, KLD-gated, same shape. The fp8-native era is
|
||||||
|
rebuild-only (see the cutover record). The w4dn8 runbook keeps every shape
|
||||||
|
law, the KV dtype table, the slab ladder, and the tuning levers:
|
||||||
|
[[frank-archon-radiance-swift15-w4dn8]].
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — the replaced payload and the
|
||||||
|
full shape-law library (slab slope, KV dtype arithmetic, TTM law)
|
||||||
|
- [[frank-archon-radiance-qwen38-native-fp8]] — the fp8-native era
|
||||||
|
(rebuild-only now; owns the descriptor-selection law)
|
||||||
|
- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
|
||||||
|
- [[deployment]] — the service-scoped deploy this cutover rides
|
||||||
@@ -3,14 +3,17 @@
|
|||||||
**Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
|
**Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
|
||||||
(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
|
(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
|
||||||
driving `podman run` directly, root systemd unit, :7770):
|
driving `podman run` directly, root systemd unit, :7770):
|
||||||
`ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4**
|
the **publisher's** 4-bit-codebook container of the official
|
||||||
(`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s
|
`Qwen/Qwen3.8-Flash-Next` (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
|
sha256-verified at fetch — w4nl codebook experts + int8 trunk, PLE n-gram,
|
||||||
|
MTP head and vision tower in-container), on BOTH R9700s
|
||||||
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
|
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
|
||||||
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
|
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
|
||||||
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
|
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
|
||||||
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
|
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
|
||||||
froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005
|
froggeric v22.5 template. Rollback: the in-house Swift1.5 w4dn8 `.rad` (on
|
||||||
and frank-archon-vllm-shapeG-20261003),
|
disk); tags frank-archon-freetoken-pr587-20261005 and
|
||||||
|
frank-archon-vllm-shapeG-20261003),
|
||||||
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
||||||
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
|
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
|
||||||
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
||||||
@@ -37,10 +40,10 @@ graph LR
|
|||||||
gfx1201, 32 GiB each]
|
gfx1201, 32 GiB each]
|
||||||
B2[radiance StillDeadcode, digest-pinned image
|
B2[radiance StillDeadcode, digest-pinned image
|
||||||
podman run, frank-archon, port 7770]
|
podman run, frank-archon, port 7770]
|
||||||
C[Swift1.5-Qwen3.8-Flash-Next GPTQ w4 .rad
|
C[Publisher qwen3.8-next-flash-fp8-iq4r-moe .rad
|
||||||
2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM,
|
2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM,
|
||||||
PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
|
PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
|
||||||
4 x 200000, froggeric v22.5]
|
3 x 200000, froggeric v22.5]
|
||||||
end
|
end
|
||||||
|
|
||||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||||
@@ -153,7 +156,22 @@ frank-zealot, port 7771]
|
|||||||
target; payload on disk; owns the measured boot-pair table)**: the same
|
target; payload on disk; owns the measured boot-pair table)**: the same
|
||||||
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
|
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
|
||||||
104.8 tok/s, 3,571 MiB true free
|
104.8 tok/s, 3,571 MiB true free
|
||||||
- [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**:
|
- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE since 2026-10-15**:
|
||||||
|
the engine author's published container of the OFFICIAL (base-alignment)
|
||||||
|
Qwen3.8-Flash-Next — `qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
|
Hub-sha256-verified at fetch: 4-bit codes into a 16-level non-uniform
|
||||||
|
codebook after a Walsh–Hadamard rotation (GPTQ + 3 coordinate-descent
|
||||||
|
sweeps, 10M-token calibration), ten bf16-protected experts per layer,
|
||||||
|
int8 trunk/lm_head, fp8 HC + PLE, MTP head, bf16 vision tower; card KL
|
||||||
|
0.0789 all-tokens / top-1 92.85 % vs bf16. **MODEL_FILE is the only line
|
||||||
|
that moved** — the 2026-10-13 shape (25600/5120 pair, 44928 pool, fp8 KV
|
||||||
|
3 × 200,000, MTP 2, wht6, MNBT 2048, froggeric v22.5) carries verbatim.
|
||||||
|
Rollback: revert the swap commit (w4dn8 `.rad` stays on disk). The
|
||||||
|
fp8-native `.rad` and the raw Swift1.5 BF16 source were DELETED from
|
||||||
|
dankserver 2026-10-15 (Frank's one-time rm, volume for this payload;
|
||||||
|
rebuild via the fetch + build scripts named in the fp8-native runbook)
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — **RETIRED 2026-10-08 → 10-15
|
||||||
|
(superseded by the publisher iq4r swap; payload on disk as the rollback)**:
|
||||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
|
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
|
||||||
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
|
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
|
||||||
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
|
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
|
||||||
|
|||||||
@@ -182,12 +182,20 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
### Deployment and Topology
|
### Deployment and Topology
|
||||||
|
|
||||||
- [[deployment]] — unified deploy and update process for both servers
|
- [[deployment]] — unified deploy and update process for both servers
|
||||||
- [[frank-archon-radiance-swift15-w4dn8]] — STAGED frank-archon runbook:
|
- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE frank-archon runbook
|
||||||
|
(2026-10-15)**: the PUBLISHER's container of the official
|
||||||
|
Qwen3.8-Flash-Next (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
|
w4nl 4-bit-codebook experts + int8 trunk, Hub-sha256-verified at fetch)
|
||||||
|
swapped in as a MODEL_FILE-only change — the 2026-10-13 shape (slab
|
||||||
|
25600 / fp8-KV 5120, 3 × 200,000, host pool 44,928, MTP depth 2) carries
|
||||||
|
verbatim; rollback = revert the swap commit (the w4dn8 `.rad` stays on
|
||||||
|
disk; the fp8-native `.rad` was deleted 2026-10-15, rebuildable)
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — RETIRED 2026-10-08 → 10-15 frank-archon runbook:
|
||||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
|
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
|
||||||
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
|
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
|
||||||
activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
|
activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
|
||||||
= revert that line (the FP8-native `.rad` stays on disk)
|
= revert that line (the FP8-native `.rad` stays on disk)
|
||||||
- [[frank-archon-radiance-qwen38-native-fp8]] — the ACTIVE frank-archon
|
- [[frank-archon-radiance-qwen38-native-fp8]] — the RETIRED 2026-10-05 → 10-08 frank-archon
|
||||||
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
|
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
|
||||||
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
|
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
|
||||||
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
|
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
|
||||||
|
|||||||
Reference in new issue
Block a user