diff --git a/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md b/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md new file mode 100644 index 0000000..d022395 --- /dev/null +++ b/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md @@ -0,0 +1,61 @@ +# Swift1.5 w4dn8 Campaign — Work Log + +Campaign branch `swift15-radiance-w4dn8`. Goal: convert +`ukisai/Swift1.5-Qwen3.8-Flash-Next` (BF16, 335.3 GiB) to +`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (GPTQ gate_up w4nl64 + block-fp8 down + +10 protected bf16 experts + int8 trunk + fixed-scale PLE) and swap it into +frank-primary on the 1.0.13 pin. Design settled (25-question grill); this log +carries execution facts only. The cutover runbook +`frank-primary-radiance-swift15-w4dn8.md` replaces this page at cutover. + +## 2026-10-06 evening + +- Purge freed 835.8 GB; Swift1.5 BF16 downloaded 131/131 shards (sha256 ok). +- Corpora built: calib 10,014,932 tok / 1,294 prompts; KLD 1,800,816 tok / + 197 prompts (`/home/dank/corpora/out/`). +- Engine verdict: 1.2.0 REJECTED for serving (decode 29.21 vs control 30.08, + accept 0.7123 vs 0.7549 deterministic; vram-pair translation ledger-exact). + **Serving pin stays 1.0.13** `sha256:4c3f48…c322e`. +- bf16 rad-convert done: `/home/dank/models/rad/qwen4exp-swift15-bf16.rad` + (335.28 GiB, 248,077 vocab tokens baked). +- **Blocker resolved — engine-version probe law:** the calibration tap lives + in the architecture plugin `/opt/radiance/share/radiance/architectures/ + qwen4exp_fp8.so`, NOT the radiance binary, and the image has no + `strings(1)` (greps return 0 for everything — positive controls + mandatory). 1.0.13 plugin: `RADIANCE_CALIB_DIR` only; 1.2.0 adds + `RADIANCE_CALIB_TOKENS`. Both `libquant.so` carry the gptq quantizer + (`calib=/damp/act_order/cd`) and the identical `RADGRAM2` format + (magic + `gram.r%d.%s.bin` + calib_gram/calib_resolve/calib_stem). + **Decision: calibrate on 1.2.0, final convert on 1.0.13** — the serving + pin produces the served `.rad`; only the Gram files cross versions. +- `run-swift15-bf16.sh` boot-failure trail (all fixed, committed): + duplicated `radiance` entrypoint arg (rc=2); ratio-derived KV pool SHORT + 217 MiB at 32×16k (explicit `--vram-kv-mib`); explicit 4608 starved the + disk tier — **weights slab = card − headroom − kv, disk-tier floor + 9.57 GiB** (4.21 layer buffers + 5.36 fixed) → kv 3072; `KLD_MOUNT` + referenced-never-defined emptied the corpus mount. +- KLD record launched on 1.0.13 (disk tier, kv 3072, 4×16k, MNBT 32768): + measured ~116 tok/s at the 32k step (44 docs/58 min). +- Chain driver `kld-then-calib.sh` (pid 2848206, log + `/home/dank/logs/swift15-chain.log`): KLD rc=0 → verify kld-ref → boot + 1.2.0 calib (MNBT 65536/seqs 8 per MOE-W4 930 tok/s law, synced commit + 45de2fd) → `calib-push.py` parallel 16 → kill → prod restored by trap. +- Ready on the branch: `convert-swift15-w4dn8.sh` (1.0.13 convert, gates: + ≥48 grams / bf16 deleted / disk floor / rad-info gptq receipt / + ≥126,000 MiB), `apply-protected-cast.sh` (derive → recipe PROTECTED + section, refuses double-derive), `kld-gate-swift15.sh` (candidate vs + reference on 1.0.13, gate mean KL ≤ 0.03 / top-1 ≥ 95 %). +- `frank-primary` showing `failed` during the campaign is the 143-on-stop + signature of the foreground podman unit — cosmetic, not a crash. + +## Campaign order after the chain + +1. derive protected experts from Swift's own grams; apply to recipe. +2. Delete bf16 `.rad` (335 GiB) to `.AGENTTRASH` staging (Freed space gates the convert). +3. `convert-swift15-w4dn8.sh` (~2 h, CPU-only, prod can stay up). +4. rad-tune declare TP=2 spec=2; vram-pair boot arms; bench (gate decode ≥ 30.8 t/s). +5. `kld-gate-swift15.sh` (fail → escalate gate_up w4→w5nl, pre-authorized). +6. Cutover: docs, tests + e2e, deploy, probes. Throughput fallback: w4 down tier (Q35). + +Disk ledger at KLD-record time: 104 GiB free → KLD ref −50 → Grams −20 → +bf16 delete +335 → convert +132 written. diff --git a/serverAppConfigs-docs/index.md b/serverAppConfigs-docs/index.md index aea807a..0bf5d29 100644 --- a/serverAppConfigs-docs/index.md +++ b/serverAppConfigs-docs/index.md @@ -191,6 +191,9 @@ The LLM inference fleet on the two hosts. Full index: 7.32 KiB/token, MTP depth 3, 2 × 200,000 first boot; rollback is the FreeToken one-card era (tag `frank-primary-freetoken-pr587-20261005`), then the tcclaviger shape-G era (tag `frank-primary-vllm-shapeG-20261003`) +- [[swift15-campaign-worklog]] — ACTIVE campaign log: the Swift1.5 → + w4dn8 radiance convert and frank-primary swap in flight (branch + `swift15-radiance-w4dn8`); the cutover runbook replaces it at completion. - [[frank-primary-freetoken-pr587]] — RETIRED 2026-10-05 (superseded the same day by the radiance cutover above; kept as the rollback runbook): FreeToken built at PR #587 head `77a7201` on the host into