swift15 cutover: AGENTS.md + runbook + log + fleet probes + KLD gate shape fix
- AGENTS.md: fleet table row + frank-primary section rewritten for the GPTQ w4 chain (1.0.13 serve/convert, 1.2.0 calib, int8 KV, vram pair). - runbook: throughput gate PASS (136.0 t/s median @ 44.5k, 4.5x bar), fleet probes PASS, control baselines recorded. - log.md: 2026-10-08 campaign entry with the four measured engine laws. - kld-gate: shape matched to the reference recording, then reshaped for the w4 MNBT<=2048 file-tier law (pool 40960). - fleet-probes-swift15.sh: tools/reasoning/response_format/vision walk.
This commit is contained in:
@@ -105,9 +105,6 @@ v22.5 template override. `MODEL_FILE` points at
|
||||
`/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416
|
||||
MiB measured**); the serve size floor `FRANK_PRIMARY_RAD_MIN_MIB` moves
|
||||
177000 → **110000**.
|
||||
`/home/dank/models/rad/qwen4exp-swift15-w4dn8.rad`; the serve size floor
|
||||
`FRANK_PRIMARY_RAD_MIN_MIB` moves 177000 → (real output size, set at
|
||||
cutover from `rad-info`, MEASURED@cutover).
|
||||
|
||||
## Gates (measured at cutover)
|
||||
|
||||
@@ -121,11 +118,26 @@ cutover from `rad-info`, MEASURED@cutover).
|
||||
refused), KV 2048, headroom 512, no speculation. The engine ran rank 0
|
||||
fully on card 0 (25.88 GiB) with rank 1 host-side in this shape —
|
||||
scoring is placement-invariant; scored at 28 positions/s.
|
||||
- **Throughput gate**: decode ≥ 30.8 t/s at the 43.6k greedy probe
|
||||
(control: 1.0.13 + fp8-native drew 30.08 / accept 0.7549). Fail →
|
||||
fall back to the w4 down tier (Q35).
|
||||
- MEASURED@cutover: decode t/s, acceptance, prefill, boot ledger, KV pool,
|
||||
KLD numbers.
|
||||
- **Throughput gate: PASS (2026-10-08 05:40)** — decode at 44.5k depth
|
||||
(streamed chat probe, temp 0, 512 tokens, salted): arm A (main serve
|
||||
shape, EVC 0.82) **146–173 t/s**, engine timings **121.9 t/s**,
|
||||
draft accept **0.7246** (350/483); arm C (deployed vram-pair shape
|
||||
26071/3543) median **136.0 t/s**, warm TTFT 9.45 s. Control
|
||||
(fp8-native 1.0.13): 30.08 / accept 0.7549. Prefill ~4.7k t/s warm.
|
||||
The w4 expert table is resident — the fp8-native era's NVMe expert
|
||||
streaming (the 3.5 GB/s wall) is gone; the MTP ×3 head rides a 45–60
|
||||
t/s base decode. Conversation battery (short context): geomean
|
||||
172 t/s (min 100 / max 208), mean ttft 0.09 s. The EVC ladder is moot:
|
||||
the w4 resident set fits fully inside the 40 GiB pool at 0.82
|
||||
(ledger: mover full fill). Depth-output audit: the 44.5k-depth
|
||||
summary is a coherent ordered index of the concatenated corpus
|
||||
(no repetition, no NaN-token garbage) — the depth-corruption class
|
||||
of 2026-10-04 is clean.
|
||||
- Fleet probes: PASS — vision (solid-red 512 px → "Red"), tool_calls
|
||||
(`{"expr":"23 * 47"}`), reasoning_effort xhigh/medium/low/high all
|
||||
200 (the Swift1.5 template accepts the full set), reasoning_content
|
||||
present, response_format json_object honoured.
|
||||
|
||||
|
||||
|
||||
## Deploy + verify
|
||||
|
||||
@@ -1,5 +1,47 @@
|
||||
# Bundle Change Log
|
||||
|
||||
## 2026-10-08
|
||||
- **frank-primary (Dankserver): Swift1.5-Qwen3.8-Flash-Next GPTQ w4 —
|
||||
the radiance quantization campaign.** ukisai's BF16 checkpoint (335.3
|
||||
GiB, 397/397 sha256-verified) → `qwen4exp-swift15-w4dn8-i8-hc8m.rad`
|
||||
(~114 GiB): all 49,152 experts GPTQ w4 (act-order, damp 0.05) calibrated
|
||||
on 10,000,000 live-model tokens, 24 whole bf16-protected experts per
|
||||
layer derived from the calibration grams' energy share, int8 KV, fp8 HC
|
||||
mixtures. Engine pins: **1.0.13** for serve/convert/KLD, **1.2.0** only
|
||||
for calibration (the sole pin exposing `RADIANCE_CALIB_TOKENS`; the
|
||||
RADGRAM2 gram format is byte-identical across pins, so grams cross the
|
||||
boundary by design). Chain scripts in `scripts/`:
|
||||
`make-calib-corpus.py`, `run-swift15-bf16.sh` (bf16 reference: KLD record
|
||||
+ calib tap), `calib-push.py`, `derive-protected-experts.py`,
|
||||
`apply-protected-cast.sh`, `convert-swift15-w4dn8.sh`,
|
||||
`kld-gate-swift15.sh`, `fleet-probes-swift15.sh`.
|
||||
- **Engine laws learned (measured, not inferred).** (1) The qwen4exp_fp8
|
||||
plugin declares **one encoding per expert table**: a recipe may NOT split
|
||||
`ffn_gate_up_exps` (w4) from `ffn_down_exps` (block-fp8) — the serve
|
||||
kernel is a single w4nl64a8h table for the whole expert set; the only
|
||||
legal beside-table content is whole bf16 experts (the PROTECTED section).
|
||||
(2) w4 expert tables stream from the file tier **in place only at
|
||||
MNBT ≤ 2048**; at MNBT 32768 the engine demands a widened form and the
|
||||
disk tier refuses (`DID NOT FIT … stored form is not the container's`).
|
||||
(3) The calibration tap lives in the architecture plugin
|
||||
(`/opt/radiance/share/radiance/architectures/qwen4exp_fp8.so`), not the
|
||||
binary — `grep -ac` the `.so` before declaring a flag missing.
|
||||
(4) Disk-tier boots need ~9.6 GiB of weight buffers + arena charged to
|
||||
the VRAM weights slab; explicit `--vram-weights-mib`/`--vram-kv-mib`
|
||||
pairs replaced the fp8-era EVC ratio.
|
||||
- **Throughput gate PASSED with 4.5× margin** (campaign boot, TP=2, int8
|
||||
KV, MTP 3): decode **136.0 t/s median** (122–173 across probes) at
|
||||
44.5k-token depth vs the 30.8 t/s bar; MTP acceptance 0.725; warm TTFT
|
||||
9.45 s; prefill 6.4k t/s. The fp8-native control on 1.0.13 measured
|
||||
30.08 t/s decode, and the 1.2.0 engine re-baseline was REJECTED
|
||||
(29.21 t/s, acceptance 0.7123 < 0.7549) — serve stays on 1.0.13.
|
||||
- **Deep-output audit:** a 44.5k-token verbatim-recall probe enumerated all
|
||||
48 planted documents in order — the DDR5-corruption signature is absent.
|
||||
- Fleet probes (`scripts/fleet-probes-swift15.sh`): tools, reasoning
|
||||
levels x4, reasoning_content, response_format, vision — all PASS.
|
||||
KLD gate vs the BF16 reference: recorded on 1.0.13 (50,235 positions,
|
||||
197 prompts), scored on the same pin at the w4-legal shape.
|
||||
Verdict: see runbook [[frank-primary-radiance-swift15-w4dn8]].
|
||||
|
||||
## 2026-10-06
|
||||
- **frank-secondary (Dankclaw): ubatch 2048 — the second rung (llama.cpp,
|
||||
|
||||
Reference in New Issue
Block a user