diff --git a/AGENTS.md b/AGENTS.md index 835db6b..95e71c8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved). | Target | Service | Runtime | Model | Port | |--------|---------|---------|-------|------| -| Dankserver | `frank-primary` | **radiance** (StillDeadcode v1.0.7 line; image `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned INDEX digest, 287 MiB, `libr4d` gfx1201 kernels, plain `podman run`, root systemd unit) | `Qwen/Qwen3.8-Flash-Next-FP8` official (172.8 GB, 131 shards, sha256-verified) converted **no-recipe** to `qwen4exp-fp8-native.rad` (177,109 MiB measured = 173.0 GiB) — **native modelopt block-FP8, no requantisation** — on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, `--expert-vs-cache-ratio 0.82` = the 99 %-hit point), **PLE 47.7 GiB pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV 7.32 KiB/token**, **MTP spec decode depth 3**, *… +| Dankserver | `frank-primary` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer**, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **PLE 95.4 GiB bf16 as mmap page cache** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **int8 KV 4.2 KiB/token**, **MTP spec decode depth 3**, `--max-num-seqs 4`, `--max-model-len 200000`, MNBT 2048, fp8 HC mixtures, Froggeric v22.5 template. Decode 122–173 t/s at 44.5k depth (KLD-gated). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 | | Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770| | Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 | @@ -489,68 +489,62 @@ Rollback: revert the swap commit — the Gemma4 payload (`/home/dank/models/Ornith-Q8/`) all remain on disk. -### frank-primary — Radiance (StillDeadcode), Qwen3.8-Flash-Next FP8 native, TP=2 (Dankserver, ACTIVE) +### frank-primary — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE) The **radiance engine** (StillDeadcode, Apache-2.0; image -`stilldeadcode/radiance@sha256:4c3f484564bb2872784e4…` digest-pinned, -287 MiB, `libr4d@0.5.0` kernels prebuilt for gfx1201, engine as the image -entrypoint) serving the **official `Qwen/Qwen3.8-Flash-Next-FP8` -checkpoint at NATIVE precision** — modelopt block-FP8 throughout, no -requantisation, no calibration. Plain `podman run` through -`scripts/serve-frank-primary.sh` under the root systemd unit -(`frank-primary.service`, `User=dank`, `LimitMEMLOCK=infinity`); the -container name is the unit name (`frank-primary`). Cutover LIVE 2026-10-06 -00:55 (chain built 2026-10-05/06); -rollback tag `frank-primary-freetoken-pr587-20261005` (the FreeToken trio -+ Dockerfile are in the tag; the image and the orcarouter payload stay on -disk until the window closes), older rollback `frank-primary-vllm-shapeG-20261003`. +`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13** +for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root +systemd unit, container name = unit name) serving +**ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards, +397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh` +with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to +`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ +w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine +**1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin +with the token knob; the RADGRAM2 grams are format-identical across pins and +cross the boundary by design), **24 whole experts per layer cast bf16** (the +PROTECTED set, derived from the calibration grams' energy share), +**int8 KV**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE bf16 as shipped. +The plugin's **one-table-one-encoding law** (measured at rad-convert +declare): one expert table takes ONE encoding; only whole bf16 experts are +legal beside it — no w4/fp8 split of the down/gate_up pair (24-value +`rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16 +reference (recorded on the same 1.0.13 pin) is the precision referee; +the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar +(122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook. +Rollback: revert the swap commit — the fp8-native `.rad` +(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk. -**Payload chain**: official checkpoint → staging dir whose `config.json` -drops `quantization_config` → `stack-qwen38-official-fp8.py` (two -value-preserving reshapes: experts byte-stacked into the plugin's stacked -`gate_up_proj`/`down_proj` layout with `_scale_inv` at the same index — 49 -stacks × 512 experts, no requantisation; PLE's 128 scaleless fp8 shards -decoded to one bf16 shard with the checkpoint's own bf16 scale -1.9931793212890625e-04) → `rad-convert` with ONE `--quant` rule — pure -copy for every weight, and the rtn rule re-encodes the bf16 PLE at that -same fixed scale, an exact round-trip that restores the original codes and -lands the plugin's one-element scale plane → -`/home/dank/models/rad/qwen4exp-fp8-native.rad` (177,109 MiB measured, -floor `FRANK_PRIMARY_RAD_MIN_MIB=177000`). The staging drop is the plugin -SELECTION law (spec §2.4): the descriptor is `(arch_id, quant)` matched -exactly, no shipped plugin claims `(qwen4exp, "fp8")`, the shipped -`qwen4exp_fp8` plugin claims `(qwen4exp, "")` — and the descriptor is only -the selection key, per-weight encodings are data, so nothing else is -requantised. -Raw serve AND unmodified convert both refuse; that is the design. The -checkpoint itself is sha256-verified against the Hub (131/131 shards, -2026-10-05). Built by `scripts/build-frank-primary-rad.sh` (`--plan-only` -first — its declare phase is the engine's own, so a resolving plan boots). +**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 4`, +`--max-model-len 200000` (the harness contextWindow must quote this), +`--placement expert_tiered --host-pool-mib 40960` + **explicit VRAM +partition `--vram-weights-mib 26071 --vram-kv-mib 3543`** (int8 KV halves +the fp8-era cost; the vram-pair replaced `--expert-vs-cache-ratio`), +`--kv-cache-dtype int8`, `--num-speculative-tokens 3`, +`--max-num-batched-tokens 2048`, prefix cache 4 GiB pinned host + 64 GiB +disk at `/home/dank/radiance-state`, **`--ngram-placement ram`**, Froggeric +v22.5 chat template vendored (`scripts/qwen38-froggeric-v22.5.jinja`, +`--override-chat-template`). The 47.7 GiB PLE table rides the ~95 GiB +bf16 tensor pool in host RAM. -**Shape (both R9700s, TP=2)**: `--tp-wire wht6` (fused lossy all-reduce, -+5.6–6.3 % prefill / ~−1.3 % 8-stream decode at TP2), `--max-num-seqs 2` -(Frank's first boot; raise from the ledger + slot occupancy), `--max-model-len -200000` (native 262,144; the harness contextWindow must quote this — a longer -prompt is refused, never truncated), `--placement expert_tiered ---host-pool-mib 40960` (native FP8 experts are 112.8 GiB total, 56.4 GiB -per rank slice — the pinned pool is the price of native precision), -`--expert-vs-cache-ratio 0.82` (the measured 99 %-hit point; 0.87 collapses -to 0 %), `--kv-cache-dtype fp8` (**7.32 KiB/token**; a 200K sequence is -1.43 GiB), `--num-speculative-tokens 3` (the checkpoint's own MTP head — -3,101 `mtp.*` tensors, probe depth 3), `--max-num-batched-tokens 2048`, -prefix cache 4 GiB pinned host + 64 GiB disk at -`/home/dank/radiance-state`, and **`--ngram-placement ram`**: the 47.7 GiB -PLE n-gram table (51.2 GB = 29 % of all parameters) serves from pinned -host RAM. Froggeric v22.5 chat template vendored -(`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`). +Shape notes carried from the fp8-native era and unchanged: wht6 gains, +MNDT 2048 law, MTP depth 3 (the checkpoint carries the 3,101 `mtp.*` +tensors), the 64 GiB disk prefix cache, and the rule that a prompt over +`--max-model-len` is refused, never truncated. What changed: int8 KV +replaces fp8 KV (4.2 KiB/token, half the fp8 rate at 64k depth), and the +EVC ratio is replaced by the explicit `--vram-weights-mib`/`--vram-kv-mib` +pair — sizing law for the disk-tier shape: the engine forces ~9.6 GiB of +disk-tier weight buffers + arena onto the weights side, so `weights + kv` +is capped near 29.6 GiB/card; boot with the pair, never `--kv-cache-size-mib`. -**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB pools + -47.7 GiB PLE + 4 GiB prefix host ≈ **134 GiB** charged against -`RLIMIT_MEMLOCK`. `ttm.pages_limit`/`pages_max` cap GPU-mappable pinned -pages (pre-cutover cap 123 GiB — BELOW the plan); the serve script's -TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to start until -both knobs are raised in the stop window (sysfs write first, kernel -cmdline + reboot only if read-only). +**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB expert +pools + 4 GiB prefix host ≈ **84 GiB** charged against `RLIMIT_MEMLOCK`; +the 95.4 GiB bf16 PLE pool rides as **mmap page cache** (never mlocked — +the `-ot`/no-mmap law), so it is invisible to the limit. +`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve +script's TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to +start until both knobs are raised in the stop window (sysfs write first, +kernel cmdline + reboot only if read-only). **Contract facts** (source-verified): the HTTP listener binds AFTER the full load, so a bare `curl /health` gate is honest (`/health` == `/ping`, @@ -562,7 +556,8 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower, `--mm-max-patches`); `/metrics` carries vLLM-compatible names + `radiance:*` families — the Dankclaw Prometheus scrape is re-added. -Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`. +Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`. +Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`. Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`. Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`. Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`. diff --git a/scripts/apply-protected-cast.sh b/scripts/apply-protected-cast.sh new file mode 100755 index 0000000..a1553c5 --- /dev/null +++ b/scripts/apply-protected-cast.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# apply-protected-cast.sh — RUNS ON DANKSERVER. Rewrite the recipe's +# PROTECTED-BEGIN/END section from Swift's OWN calibration grams: run +# derive-protected-experts.py over the calib dir, splice its cast lines into +# the marked section of the recipe in place, keep the pre-edit recipe in the +# campaign trash. Refuses to run twice (marker text says derived). +set -euo pipefail +CALIB=${1:-/home/dank/corpora/calib} +RECIPE=${2:-/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe} +PY=/home/dank/venvs/radcorpus/bin/python +DERIVE=/home/dank/serverAppConfigs-swift15/scripts/derive-protected-experts.py +TS=$(date +%Y%m%d-%H%M%S) + +grep -q "derive-protected-experts.py rewrote" "$RECIPE" && { + echo "recipe already carries derived cast lines; refusing to re-derive" >&2; exit 1; } +grep -q "PROTECTED-BEGIN" "$RECIPE" || { echo "no PROTECTED markers in $RECIPE" >&2; exit 1; } + +"$PY" "$DERIVE" "$CALIB" > "/tmp/protected-cast-$TS.txt" +N=$(grep -c 'cast dtype=bf16' "/tmp/protected-cast-$TS.txt") +[ "$N" -ge 2 ] || { echo "derive produced $N cast lines; refusing" >&2; exit 1; } +echo "derived $N protected experts -> /tmp/protected-cast-$TS.txt" + +cp -f "$RECIPE" "/home/dank/.AGENTTRASH/$TS-qwen4exp-swift15-w4dn8-i8-hc8m.recipe" +"$PY" - "$RECIPE" "/tmp/protected-cast-$TS.txt" <<'PYEOF' +import sys +recipe, castfile = sys.argv[1], sys.argv[2] +lines = open(recipe).read().splitlines() +cast = [l for l in open(castfile).read().splitlines() if 'cast dtype=bf16' in l] +out, insec = [], False +for l in lines: + if 'PROTECTED-BEGIN' in l: + insec = True + out.append(l) + out.extend(cast) + elif 'PROTECTED-END' in l: + insec = False + out.append(l) + out.append('# (derive-protected-experts.py rewrote this section from Swift\'s own grams)') + elif not insec: + out.append(l) +open(recipe, 'w').write('\n'.join(out) + '\n') +print(f'spliced {len(cast)} cast lines into {recipe}') +PYEOF diff --git a/scripts/autoresearch-frank-primary.sh b/scripts/autoresearch-frank-primary.sh index 44dcc2a..9d9f857 100755 --- a/scripts/autoresearch-frank-primary.sh +++ b/scripts/autoresearch-frank-primary.sh @@ -87,7 +87,8 @@ podman run -d --name "$SCRATCH" \ --placement "$FRANK_PRIMARY_PLACEMENT" \ --host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \ --gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \ - --expert-vs-cache-ratio "$FRANK_PRIMARY_EXPERT_VS_CACHE" \ + --vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \ + --vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \ --kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \ --prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \ --prefix-cache-dir "/data/kvcache/$SCRATCH" \ diff --git a/scripts/bench-probe.py b/scripts/bench-probe.py new file mode 100755 index 0000000..234ed59 --- /dev/null +++ b/scripts/bench-probe.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python3 +"""bench-probe.py — RUNS ON DANKSERVER. The campaign's decode probe: one +/completion request, a prompt assembled from calib.jsonl to ~TARGET tokens, +temperature 0, max_tokens 512. Reports the engine's own timings: decode rate, +prefill rate, draft accept. Run 3x; the median is the arm's number.""" +import json, sys, urllib.request + +TARGET = 43600 # tokens +BYTES_PER_TOK = 3.8 +URL = "http://localhost:7770/v1/completions" + +def build_prompt(): + want = int(TARGET * BYTES_PER_TOK) + parts = [] + n = 0 + with open("/home/dank/corpora/out/calib.jsonl") as f: + for line in f: + d = json.loads(line) + p = d.get("prompt") or d.get("messages", [{}])[0].get("content", "") + parts.append(p) + n += len(p.encode()) + if n >= want: + break + return "".join(parts) + +def probe(salt): + prompt = f"[probe {salt}]\n" + build_prompt() + body = json.dumps({ + "model": "frank-primary", + "prompt": prompt, "temperature": 0.0, "max_tokens": 512, + }).encode() + req = urllib.request.Request(URL, data=body, + headers={"Content-Type": "application/json"}) + with urllib.request.urlopen(req, timeout=1800) as r: + d = json.load(r) + t = d.get("timings", {}) + return { + "prompt_n": t.get("prompt_n"), + "prefill_tps": round(t.get("prompt_per_second", 0), 1), + "pred_n": t.get("predicted_n"), + "decode_tps": round(t.get("predicted_per_second", 0), 2), + "draft_n": t.get("draft_n"), + "accept": (round(t["draft_n_accepted"] / t["draft_n"], 4) + if t.get("draft_n") else None), + } + +if __name__ == "__main__": + print(json.dumps(probe(sys.argv[1] if len(sys.argv) > 1 else "a"))) diff --git a/scripts/calib-push.py b/scripts/calib-push.py new file mode 100755 index 0000000..ac0b069 --- /dev/null +++ b/scripts/calib-push.py @@ -0,0 +1,110 @@ +#!/usr/bin/env python3 +"""calib-push.py — drive a calibration corpus through a radiance scratch +server's prefill so its tap writes layer Hessians (docs/MOE-W4.md). + +Runs ON DANKSERVER next to the scratch server (the one autoresearch- / +serve-style scratch container booted with RADIANCE_CALIB_DIR set). +Reads the corpus JSONL produced by make-calib-corpus.py +({"prompt": "..."} a line), POSTs each prompt to /v1/completions with +max_tokens=1 (prefill-only cost, one token out), at N-way concurrency +matching the scratch server's --max-num-seqs budget. + +Corpus rounds repeat until the engine prints the tap's completion line +("calibration wrote layer Hessians") in `podman logs ` — that +is the ground truth that RADIANCE_CALIB_TOKENS was reached. After that +line, further prompts are pure waste, so the driver exits. + +Mechanics notes: +- requests carry no sampling params that matter (max_tokens=1, greedy). +- a per-request timeout of 30 min: at ~1k tok/s prefill a 12k-token + prompt is a few seconds; the timeout only catches a wedged engine. +- failures print and continue: the tap pools over tokens, a few lost + prompts do not bias the Hessians (the corpus is drawn round-robin over + source classes, so loss is class-neutral as long as failures are rare; + a systematic 4xx is loud in the log and must be fixed, not ignored). + +Usage: calib-push.py [--port 7770] [--parallel 64] [--container arbench] + [--corpus /home/dank/corpora/out/calib.jsonl] [--max-rounds 4] +""" +import argparse +import concurrent.futures +import json +import subprocess +import sys +import time +import urllib.request + +DONE_MARK = "calibration wrote" + + +def tap_done(container): + """True if the engine logged the tap's completion line.""" + try: + out = subprocess.run( + ["podman", "logs", "--tail", "50", container], + capture_output=True, text=True, timeout=30, + ) + except Exception: + return False + return DONE_MARK in (out.stdout + out.stderr) + + +def push_one(url, prompt): + body = json.dumps({ + "model": "frank-primary", + "prompt": prompt, + "max_tokens": 1, + "temperature": 0, + "stream": False, + }).encode() + req = urllib.request.Request( + url, data=body, headers={"Content-Type": "application/json"}) + try: + with urllib.request.urlopen(req, timeout=1800) as r: + r.read() + return True + except Exception as e: + print(f"FAIL {str(e)[:120]}", flush=True) + return False + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("--port", type=int, default=7770) + ap.add_argument("--parallel", type=int, default=64) + ap.add_argument("--container", default="arbench") + ap.add_argument("--corpus", default="/home/dank/corpora/out/calib.jsonl") + ap.add_argument("--max-rounds", type=int, default=4) + a = ap.parse_args() + url = f"http://127.0.0.1:{a.port}/v1/completions" + + prompts = [] + with open(a.corpus) as f: + for line in f: + line = line.strip() + if line: + prompts.append(json.loads(line)["prompt"]) + print(f"corpus: {len(prompts)} prompts", flush=True) + + for rnd in range(1, a.max_rounds + 1): + if tap_done(a.container): + break + t0 = time.time() + ok = 0 + with concurrent.futures.ThreadPoolExecutor(a.parallel) as ex: + for good in ex.map(lambda p: push_one(url, p), prompts): + ok += 1 if good else 0 + dt = time.time() - t0 + print(f"round {rnd}: {ok}/{len(prompts)} ok in {dt:.0f}s " + f"({sum(len(p) for p in prompts)/4/dt:.0f} chars/s est)", + flush=True) + if tap_done(a.container): + break + else: + print("max rounds exhausted, tap not done", flush=True) + sys.exit(1) + print("TAP_DONE: calibration wrote layer Hessians", flush=True) + + +if __name__ == "__main__": + main() diff --git a/scripts/convert-swift15-w4dn8.sh b/scripts/convert-swift15-w4dn8.sh new file mode 100755 index 0000000..6a19690 --- /dev/null +++ b/scripts/convert-swift15-w4dn8.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash +# convert-swift15-w4dn8.sh — RUNS ON DANKSERVER. The final recipe convert of +# the Swift 1.5 checkpoint: gate_up experts through GPTQ against the 10M-token +# calibration Grams, down projections block-fp8, protected experts bf16, +# trunk int8, PLE fixed-scale fp8 (scripts/recipes/ +# qwen4exp-swift15-w4dn8-i8-hc8m.recipe). +# +# The convert engine is 1.0.13 — the SAME pin that will serve the result, so +# the .rad crosses no version boundary. Only the Gram files cross versions +# (1.2.0 wrote them; the RADGRAM2 magic, gram.r%d.%s.bin naming and the +# calib_gram/calib_resolve/calib_stem symbols are byte-identical between the +# two libquant.so). rad-convert runs on the host backend (no GPU visible in +# the container) — production can stay up through this script. +# +# Prereqs (campaign order): calibration complete (calib-push printed TAP_DONE), +# the protected-expert cast lines rewritten into the recipe by +# derive-protected-experts.py, and the 335 GiB bf16 container deleted — +# the peak-disk arithmetic needs that space (checkpoint 335.3 + output ~132 +# GiB on the volume; --reuse is NOT used, see the recipe header). +set -euo pipefail + +PIN_1013=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e +IMAGE="stilldeadcode/radiance@$PIN_1013" +CKPT=/home/dank/models/ukisai/Swift1.5-Qwen3.8-Flash-Next +RECIPE=/home/dank/serverAppConfigs-swift15/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe +CALIB=/home/dank/corpora/calib +OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad +LOG=/home/dank/logs/swift15_convert.log +# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table; +# the plugin's one-encoding law dropped the block-fp8 down tier -- see the recipe header). +MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000} + +# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a +# per-tensor stderr note, which is silently worse than failing here. +[ -d "$CKPT" ] || { echo "no checkpoint at $CKPT" >&2; exit 1; } +[ -f "$RECIPE" ] || { echo "no recipe at $RECIPE" >&2; exit 1; } +GRAMS=$(find "$CALIB" -maxdepth 1 -name 'gram.r*.bin' 2>/dev/null | wc -l) +[ "$GRAMS" -ge 48 ] || { echo "calib dir $CALIB holds $GRAMS gram files (<48)" >&2; exit 1; } +if [ -e "$OUT" ]; then echo "$OUT already exists; move it to trash first" >&2; exit 1; fi +if [ -e /home/dank/models/rad/qwen4exp-swift15-bf16.rad ]; then + echo "the bf16 container is still on disk; delete it first (peak disk)" >&2; exit 1 +fi + +# Gate 2: disk. Output ~132 GiB plus convert staging, 20 GiB of slack. +FREE_MIB=$(df --output=avail -m /home/dank/models | tail -1 | tr -d ' ') +[ "$FREE_MIB" -ge $((MIN_MIB + 20480)) ] || { + echo "only ${FREE_MIB} MiB free on /home/dank/models" >&2; exit 1; } + +# The recipe names $CALIB; rad-convert expands it from the environment +# (MOE-W4.md:275). The recipe's cast section must name THIS checkpoint's +# protected experts — derive-protected-experts.py rewrote it. +echo "convert: $CKPT -> $OUT (gptq calib=$CALIB, $(date -Is))" | tee "$LOG" +podman run --rm --name swift15-convert \ + -v /home/dank/models:/models \ + -v /home/dank/serverAppConfigs-swift15:/campaign:ro \ + -v "$CALIB:/data/calib:ro" \ + -e CALIB=/data/calib \ + --entrypoint /opt/radiance/bin/rad-convert \ + "$IMAGE" "/models/${CKPT#/home/dank/models/}" \ + --recipe /campaign/"${RECIPE#/home/dank/serverAppConfigs-swift15/}" \ + -o /models/rad/"$(basename "$OUT")" 2>&1 | tee -a "$LOG" +rc=${PIPESTATUS[0]} +[ "$rc" -eq 0 ] || { echo "rad-convert rc=$rc; log: $LOG" >&2; exit "$rc"; } + +# Gate 3: the artifact. rad-info's recipe echo is the receipt that GPTQ ran. +SIZE_MIB=$(( $(stat -c %s "$OUT") / 1048576 )) +[ "$SIZE_MIB" -ge "$MIN_MIB" ] || { + echo "$OUT is ${SIZE_MIB} MiB < floor ${MIN_MIB}" >&2; exit 1; } +podman run --rm --entrypoint /opt/radiance/bin/rad-info \ + -v /home/dank/models:/models:ro "$IMAGE" /models/rad/"$(basename "$OUT")" \ + 2>&1 | tee -a "$LOG" | grep -c 'gptq' >/dev/null \ + || echo "WARNING: rad-info shows no gptq rule in the recorded recipe" >&2 +echo "CONVERT_DONE ${SIZE_MIB} MiB -> $OUT" diff --git a/scripts/deep-probe.py b/scripts/deep-probe.py new file mode 100755 index 0000000..2653e8a --- /dev/null +++ b/scripts/deep-probe.py @@ -0,0 +1,55 @@ +#!/usr/bin/env python3 +"""deep-probe.py — one chat completion at ~43.6k prompt tokens, temp 0, 512 +generated, streamed; decode = completion_tokens / (total - ttft).""" +import json, time, urllib.request + +TARGET_BYTES = int(43600 * 3.8) + +def build(): + parts, n = [], 0 + with open("/home/dank/corpora/out/calib.jsonl") as f: + for line in f: + p = json.loads(line)["prompt"] + parts.append(p) + n += len(p.encode()) + if n >= TARGET_BYTES: + break + return "".join(parts) + +def main(salt): + body = json.dumps({ + "model": "frank-primary", + "messages": [{"role": "user", "content": f"[probe {salt}] Continue and summarize this material in English, 400 words.\n\n" + build()}], + "temperature": 0.0, "max_tokens": 512, + "stream": True, "stream_options": {"include_usage": True}, + }).encode() + req = urllib.request.Request("http://localhost:7770/v1/chat/completions", + data=body, headers={"Content-Type": "application/json"}) + t0 = time.time(); ttft = None; ctok = 0; usage = {} + with urllib.request.urlopen(req, timeout=2400) as r: + for raw in r: + line = raw.decode().strip() + if not line.startswith("data:"): + continue + d = line[5:].strip() + if d == "[DONE]": + break + j = json.loads(d) + if j.get("usage"): + usage = j["usage"] + for ch in j.get("choices", []): + delta = (ch.get("delta") or {}) + piece = delta.get("content") or delta.get("reasoning_content") or "" + if piece and ttft is None: + ttft = time.time() - t0 + ctok += 1 + total = time.time() - t0 + gen = usage.get("completion_tokens") or ctok + dec = gen / (total - (ttft or 0)) if total > (ttft or 0) else 0 + print(json.dumps({"salt": salt, "prompt_tokens": usage.get("prompt_tokens"), + "completion_tokens": gen, "ttft_s": round(ttft or 0, 2), + "total_s": round(total, 1), "decode_tps": round(dec, 2)})) + +if __name__ == "__main__": + import sys + main(sys.argv[1] if len(sys.argv) > 1 else "x") diff --git a/scripts/derive-protected-experts.py b/scripts/derive-protected-experts.py new file mode 100755 index 0000000..4a3503b --- /dev/null +++ b/scripts/derive-protected-experts.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python3 +"""derive-protected-experts.py — rewrite the recipe's protected-expert section +from a calibration recording. RUNS ON DANKSERVER (venv: numpy). + +The law (docs/MOE-W4.md "Protected experts", restated in the recipe): +an expert whose share of its layer's down-input energy is at least 3% AND +at least three times its share of the routing is kept bf16; the count must +be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest +as two equal tables by parity) — an odd half gains its next most energetic +non-qualifying expert. + +Data: the calibration dir's per-expert down Grams, +gram.r.blk.L.ffn_down_exps..bin — 32-byte RADGRAM2 header +(magic, k, rank, rows, domain) then k*k f32, named with the checkpoint's +GLOBAL expert numbering (arch/common/rad_block_moe_fp8.h: ge = +expert_base + e). rows (header) is the expert's routing count; the trace +of the Gram is its down-input energy. Rank halves: 0-255, 256-511 +(contiguous expert_base split at TP=2). + +Output: the cast-rule lines (recipe format) on stdout, a report on stderr. + +Usage: derive-protected-experts.py [--min-share 0.03] + [--min-route-mult 3.0] +""" +import argparse +import glob +import os +import re +import struct +import sys + +import numpy as np + +NAME_RE = re.compile(r"^gram\.r(\d)\.blk\.(\d+)\.ffn_down_exps\.(\d+)\.bin$") + + +def read_gram(path): + """(k, rank, rows, trace) from a RADGRAM2 file (full matrix on disk).""" + with open(path, "rb") as f: + hdr = f.read(32) + magic, k, rank, rows, domain = struct.unpack("<8sIIQI", hdr[:28]) + if magic != b"RADGRAM2": + raise SystemExit(f"not a RADGRAM2 file: {path}") + n = k * k + a = np.frombuffer(f.read(n * 4), dtype=np.float32) + if a.size != n: + raise SystemExit(f"short gram file: {path}") + tr = float(a.reshape(k, k).diagonal().sum()) + return k, rank, int(rows), tr + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("calib_dir") + ap.add_argument("--min-share", type=float, default=0.03) + ap.add_argument("--min-route-mult", type=float, default=3.0) + a = ap.parse_args() + + # layer -> global expert -> (rank, rows, energy) + data = {} + for path in sorted(glob.glob(os.path.join( + a.calib_dir, "gram.r*.ffn_down_exps.*.bin"))): + m = NAME_RE.match(os.path.basename(path)) + if not m: + continue + rank, layer, expert = int(m.group(1)), int(m.group(2)), int(m.group(3)) + k, _, rows, tr = read_gram(path) + data.setdefault(layer, {})[expert] = (rank, rows, tr) + + if not data: + raise SystemExit("no per-expert down grams found") + + lines, report = [], [] + total = 0 + for layer in sorted(data): + experts = data[layer] + tot_e = sum(v[2] for v in experts.values()) + tot_r = sum(v[1] for v in experts.values()) + if tot_e <= 0 or tot_r <= 0: + report.append(f"blk.{layer}: empty recording, skipped") + continue + qual, pool = [], [] + for e, (rank, rows, tr) in experts.items(): + es, rs = tr / tot_e, rows / max(tot_r, 1) + rec = (e, rank, es, rs) + if es >= a.min_share and es >= a.min_route_mult * rs: + qual.append(rec) + elif es > 0: + pool.append(rec) + chosen = {r[0]: (r[2], r[3]) for r in qual} + for half in (0, 1): # rank halves by the contiguous expert_base split + n_half = sum(1 for e in chosen if e // 256 == half) + if n_half % 2: + pool.sort(key=lambda r: -r[2]) + for cand in pool: + if cand[1] == half and cand[0] not in chosen: + chosen[cand[0]] = (cand[2], cand[3]) + report.append( + f"blk.{layer}: half {half} odd -> added e{cand[0]} " + f"(energy {cand[2]*100:.2f}%)") + break + for e in sorted(chosen): + es, rs = chosen[e] + lines.append(f"blk.{layer}.ffn_*_exps.{e}.weight" + f" cast dtype=bf16") + report.append(f"blk.{layer} e{e:3d} (rank {e // 256}): " + f"energy {es*100:5.2f}%, routing {rs*100:5.3f}%, " + f"ratio {es/max(rs,1e-12):6.1f}") + total += len(chosen) + + print("\n".join(lines)) + print("\n".join(report), file=sys.stderr) + print(f"total protected: {total}", file=sys.stderr) + + +if __name__ == "__main__": + main() diff --git a/scripts/download-frank-primary-swift15.sh b/scripts/download-frank-primary-swift15.sh new file mode 100755 index 0000000..acb5c14 --- /dev/null +++ b/scripts/download-frank-primary-swift15.sh @@ -0,0 +1,113 @@ +#!/usr/bin/env bash +# download-frank-primary-swift15.sh — payload fetch/verify for the planned +# frank-primary swap to the Swift 1.5 finetune: ukisai/Swift1.5-Qwen3.8-Flash-Next +# (BF16 source, 149 files, 360,037,355,038 bytes = 335.3 GiB: 131 safetensors +# shards + weight-map index + tokenizer + vision preprocessing configs + chat +# template + licenses). Key set verified identical to the official +# Qwen/Qwen3.8-Flash-Next base (0 added / 0 missing), so the qwen4exp recipes +# apply verbatim; the checkpoint ships BF16 with NO quantization_config. +# Repo is UNGATED — no token needed (never put one on the command line). +# +# bash scripts/download-frank-primary-swift15.sh --fetch resume/complete +# bash scripts/download-frank-primary-swift15.sh verify only +# +# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0 +# listing/resume downloader, curl+jq only): per-file size checks against +# the Hub listing, `-C -` resume, pickle-format duplicates skipped. +# Verify (default) is the AGENTS.md law in three parts: every listed file +# present with the listed size, the shard count matches the weight-map +# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs +# only the public API. +set -euo pipefail + +REPO="ukisai/Swift1.5-Qwen3.8-Flash-Next" +DEST="/home/dank/models/ukisai/Swift1.5-Qwen3.8-Flash-Next" +EXPECTED_SHARDS=131 + +mode="verify" +[[ "${1:-}" == "--fetch" ]] && mode="fetch" +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + +if [[ "$mode" == "fetch" ]]; then + # HFGET_JOBS=4 is hfget's default (a few streams saturate the link). + bash "$SCRIPT_DIR/hfget.sh" "$REPO" "$DEST" main +fi + +echo "verifying local payload: $DEST" +fail=0 + +INDEX="$DEST/model.safetensors.index.json" +if [[ ! -f "$INDEX" ]]; then + echo " BAD weight-map index missing: $INDEX" >&2 + exit 1 +fi + +python3 - "$REPO" "$DEST" <<'PY' || fail=1 +import hashlib, json, os, sys, urllib.request + +repo, dest = sys.argv[1], sys.argv[2] + +def listing(): + """The tree API, following the Link cursor exactly like hfget.sh.""" + out, api, pages = [], ( + f"https://huggingface.co/api/models/{repo}/tree/main" + "?recursive=true&expand=true"), 0 + while api: + with urllib.request.urlopen(api) as r: + out += json.load(r) + api = None + link = r.headers.get("Link", "") + for part in link.split(","): + if 'rel="next"' in part: + api = part[part.find("<") + 1:part.find(">")] + pages += 1 + if pages > 200: + raise SystemExit("listing did not terminate") + return out + +files = [e for e in listing() if e.get("type") == "file"] +sizes = {e["path"]: e.get("size", 0) for e in files} +lfs = {e["path"]: e["lfs"]["oid"] for e in files if e.get("lfs")} + +bad = 0 +for path, want in sorted(sizes.items()): + p = os.path.join(dest, path) + if not os.path.isfile(p): + print(f" BAD {path}: missing") + bad += 1 + elif os.path.getsize(p) != want: + print(f" BAD {path}: {os.path.getsize(p)} bytes, listing says {want}") + bad += 1 +print(f" {'OK' if bad == 0 else 'FAIL'}: {len(sizes) - bad}/{len(sizes)} files present at listed size") +if bad: + raise SystemExit(1) + +# Shard set against the weight-map index. +wm = json.load(open(os.path.join(dest, "model.safetensors.index.json")))["weight_map"] +shards = sorted(set(wm.values())) +print(f" index maps {len(wm)} tensors across {len(shards)} shards") +if len(shards) != 131: + print(f" BAD shard count {len(shards)}, want 131") + raise SystemExit(1) + +# Content: every LFS-tracked file hashed against the Hub oid. The safetensors +# shards dominate the bytes; tokenizer/config files ride the same check when +# the Hub tracks them in LFS (they are plain blobs here, already size-checked). +todo = [p for p in shards if p in lfs] +for i, path in enumerate(todo): + h = hashlib.sha256() + with open(os.path.join(dest, path), "rb") as f: + for chunk in iter(lambda: f.read(1 << 24), b""): + h.update(chunk) + if h.hexdigest() != lfs[path]: + print(f" BAD {path}: sha256 mismatch (want {lfs[path]})") + bad += 1 + if (i + 1) % 16 == 0: + print(f" ...hashed {i + 1}/{len(todo)}") +print(f" {'OK' if bad == 0 else 'FAIL'}: {len(todo) - bad}/{len(todo)} shards sha256-match the Hub") +if bad: + raise SystemExit(1) +PY + +[[ "$fail" -eq 0 ]] || exit 1 +echo "payload verified: 131 shards + index + tokenizer + configs, sizes and sha256 checked" diff --git a/scripts/fleet-probes-swift15.sh b/scripts/fleet-probes-swift15.sh new file mode 100755 index 0000000..bf15ed7 --- /dev/null +++ b/scripts/fleet-probes-swift15.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# fleet-probes-swift15.sh — RUNS ON DANKSERVER. The cutover contract probes. +set -uo pipefail +B=http://localhost:7770/v1 + +echo "== vision (solid red 512px, expect 'Red')" +python3 - <<'PY' +import base64, json, urllib.request, struct, zlib +def png(color): + w = h = 512 + raw = b"".join(b"\x00" + bytes(color) * w for _ in range(h)) + def chunk(t, d): + c = t + d + return struct.pack(">I", len(d)) + c + struct.pack(">I", zlib.crc32(c)) + return (b"\x89PNG\r\n\x1a\n" + + chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, 2, 0, 0, 0)) + + chunk(b"IDAT", zlib.compress(raw)) + chunk(b"IEND", b"")) +b64 = base64.b64encode(png((255, 0, 0))).decode() +body = json.dumps({"model": "frank-primary", "max_tokens": 64, "temperature": 0, + "messages": [{"role": "user", "content": [ + {"type": "image_url", "image_url": {"url": "data:image/png;base64," + b64}}, + {"type": "text", "text": "What color is this image? Answer with one word."}]}]}).encode() +r = urllib.request.Request(B + "/chat/completions", data=body, + headers={"Content-Type": "application/json"}) +d = json.load(urllib.request.urlopen(r, timeout=300)) +m = d["choices"][0]["message"] +print("VISION ANSWER:", ((m.get("content") or "")[:80]).replace("\n", " ")) +PY + +echo "== tools (expect tool_calls)" +curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' -d '{ + "model":"frank-primary","temperature":0,"max_tokens":128, + "messages":[{"role":"user","content":"What is 23 times 47? Use the calculator tool."}], + "tools":[{"type":"function","function":{"name":"calculator","description":"evaluate arithmetic", + "parameters":{"type":"object","properties":{"expr":{"type":"string"}},"required":["expr"]}}}]}' \ + | python3 -c 'import json,sys; d=json.load(sys.stdin); m=d["choices"][0]["message"]; print("TOOL_CALLS:", m.get("tool_calls"))' + +echo "== reasoning_effort fold (xhigh/medium/low ok, high -> 500)" +for lvl in xhigh medium low high; do + code=$(curl -s -o /dev/null -w "%{http_code}" -m 120 $B/chat/completions \ + -H 'Content-Type: application/json' \ + -d "{\"model\":\"frank-primary\",\"max_tokens\":8,\"reasoning_effort\":\"$lvl\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}") + echo " reasoning_effort=$lvl -> HTTP $code" +done + +echo "== reasoning_content channel" +curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' \ + -d '{"model":"frank-primary","max_tokens":64,"messages":[{"role":"user","content":"Is 97 prime? Think briefly."}]}' \ + | python3 -c 'import json,sys; m=json.load(sys.stdin)["choices"][0]["message"]; print("has reasoning_content:", bool(m.get("reasoning_content")), "| content:", (m.get("content") or "")[:60].replace(chr(10)," "))' + +echo "== response_format json_object" +curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' \ + -d '{"model":"frank-primary","max_tokens":64,"response_format":{"type":"json_object"},"messages":[{"role":"user","content":"Return JSON {\"ok\": true}"}]}' \ + | python3 -c 'import json,sys; d=json.load(sys.stdin); print("JSON MODE:", (d["choices"][0]["message"].get("content") or "")[:60])' +echo "PROBES_DONE" diff --git a/scripts/frank-primary.env b/scripts/frank-primary.env index 298e625..8088eef 100644 --- a/scripts/frank-primary.env +++ b/scripts/frank-primary.env @@ -50,14 +50,30 @@ # publisher's throughput peak for this model. Stated explicitly with # --num-speculative-tokens 3. # - The fused TP all-reduce is declared over max-num-seqs x -# (1 + --num-speculative-tokens) rows; 2 x 4 = 8 rows fits either wire. +# (1 + --num-speculative-tokens) rows; 3 x 3 = 9 rows fits either wire. +# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13, +# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb +# 25558e07fc60d20328f1346f96f3c6e00845eb763f01fa, tag 1.2.0, commit +# e95d397) was benched 2026-10-06 on this exact shape (fp8-native rad, +# vram pair, spec 2, seqs 3, 43.6k greedy probe): decode 29.21 t/s and +# draft acceptance 0.7123 vs the 1.0.13 control's 30.08 and 0.7549. +# Acceptance on the greedy harness is deterministic (the control +# reproduced the recorded 0.755 exactly), so the drop is a real numerics +# change — the 1.2.0 MoE weight-rotation change (92b8aaf) is the named +# suspect; decode loss tracks the acceptance ratio (0.943 vs 0.945). +# The vram-pair translation itself is exact (1.2.0 boot ledger took +# vram_weights 25.46 GiB / vram_kv 3.46 GiB per card byte-identical to +# the 1.0.13 ledger at EVC 0.82). Re-test 1.2.0+ when the rotation change +# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of +# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this +# file already uses the pair, which BOTH pins accept (rollback symmetry). FRANK_PRIMARY_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e # The .rad container FILE (mounted under /models) and the state dir for the # prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY # .rad files; the raw HF download and its staging dir live beside it and # are never mounted. FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad -FRANK_PRIMARY_MODEL_FILE=qwen4exp-fp8-native.rad +FRANK_PRIMARY_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad FRANK_PRIMARY_STATE_DIR=/home/dank/radiance-state # The state dir's tune/ subdirectory holds rad-tune's kernel-selection # cache (the engine's selector reads $RADIANCE_HOME/tune/.tune @@ -106,9 +122,15 @@ FRANK_PRIMARY_PLACEMENT=expert_tiered FRANK_PRIMARY_HOST_POOL_MIB=40960 # VRAM left unclaimed per card (publisher default 96 MiB). FRANK_PRIMARY_GPU_HEADROOM_MIB=96 -# Weights share of claimed VRAM (publisher production value; higher = more -# prefix-cache hit rate, less KV headroom). -FRANK_PRIMARY_EXPERT_VS_CACHE=0.82 +# VRAM split of claimed memory per card, MiB (0 = derive from the card). +# Engine 1.2.0 REMOVED --expert-vs-cache-ratio; these two knobs replace +# it. The values transcribe the live 1.0.13 boot ledger at ratio 0.82: +# vram_weights 25.46 GiB, vram_kv 3.46 GiB per card -> 26071 / 3543. +# Bracket arms scale the KV term linearly in (1-r) from the 0.82 anchor +# with weights fixed: ratio 0.90 -> KV 1968, ratio 0.95 -> KV 984. The +# boot ledger arbitrates; re-derive after any payload change. +FRANK_PRIMARY_VRAM_WEIGHTS_MIB=26071 +FRANK_PRIMARY_VRAM_KV_MIB=3543 # KV cache in FP8 (7.32 KiB/token at this model's 12 full-attention layers, # 2 KV heads, 256 head_dim, K+V). FRANK_PRIMARY_KV_DTYPE=fp8 @@ -139,4 +161,4 @@ FRANK_PRIMARY_EXTRA_ARGS= # container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the # plan-only estimate 172.76 GiB covers the data planes but not the metadata # tail) - floor sits just under the payload. -FRANK_PRIMARY_RAD_MIN_MIB=177000 +FRANK_PRIMARY_RAD_MIN_MIB=110000 diff --git a/scripts/kld-gate-swift15.sh b/scripts/kld-gate-swift15.sh new file mode 100755 index 0000000..5e9b0fd --- /dev/null +++ b/scripts/kld-gate-swift15.sh @@ -0,0 +1,91 @@ +#!/usr/bin/env bash +# kld-gate-swift15.sh — RUNS ON DANKSERVER. Score the quantized candidate +# against the BF16 reference and apply the pre-approved gate: +# mean KL <= 0.03 AND top-1 agreement >= 95%. +# Same engine (1.0.13) and same boot shape as the reference RECORD +# (run-swift15-bf16.sh kld): disk-tier, no speculation, deterministic, fp8 KV. +# --kld-ref reads its tokens from the reference (no --kld-corpus here — the +# (run-swift15-bf16.sh kld): 1.0.13, tp 2, disk-tier, fp8 KV, no speculation, +# deterministic, max-model-len 16384, vram-kv 3072, 4 slots. Two deviations +# are forced by the w4 file: MNBT 2048 (the w4 kernels stream from the file +# tier ONLY at MNBT<=2048 — at the recording's 32768 the engine demands a +# widened form: "DID NOT FIT ... stored form is not the container's"), and +# pool 40960 (the recording's 12 GiB pool predates the w4 tables; at 12288 the +# gate_up slab cannot be cached). Chunk size is softmax-invariant, so the +# MNBT delta is float-order noise riding on the KL floor. +set -uo pipefail +PIN=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e +C=swift15gate +LOG=/home/dank/logs/swift15_kldgate.log +KLD_REF_DIR=/home/dank/corpora/kld-ref +KLD_OUT=/home/dank/radiance-state/kld-gate.json + +[ -d "$KLD_REF_DIR" ] || { echo "missing reference: $KLD_REF_DIR"; exit 2; } +: > "$KLD_OUT" +cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \ + /home/dank/models/tpl.jinja 2>/dev/null || true +sudo systemctl stop frank-primary >/dev/null 2>&1 || true +podman rm -f "$C" >/dev/null 2>&1 || true + +podman run --rm --name "$C" \ + --network host --init \ + --security-opt seccomp=unconfined \ + --ulimit memlock=-1:-1 \ + --device /dev/kfd --device /dev/dri \ + -v /home/dank/models:/models:ro \ + -v /home/dank/radiance-state:/data \ + -v "$KLD_REF_DIR:/data/kld-ref:ro" \ + "stilldeadcode/radiance@$PIN" \ + --model /models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad \ + --tp 2 --placement expert_tiered --weights-disk-tier \ + --host-pool-mib 40960 --deterministic --gpu-headroom-mib 768 \ + --num-speculative-tokens 0 \ + --kv-cache-dtype fp8 --max-model-len 16384 \ + --ngram-placement ram \ + --override-chat-template /models/tpl.jinja \ + --kld-ref /data/kld-ref --kld-out /data/kld-gate.json \ + --vram-kv-mib 3072 \ + --max-num-seqs 4 --max-num-batched-tokens 2048 \ + 2>&1 | tee "$LOG" +rc=${PIPESTATUS[0]} +[ "$rc" -eq 0 ] || { echo "KLD GATE BOOT FAILED rc=$rc; log: $LOG"; exit "$rc"; } + +python3 - "$KLD_OUT" "$LOG" <<'PY' +import json, sys, re +out, log = sys.argv[1], sys.argv[2] +def num(pat): + m = re.findall(pat, open(log, errors="replace").read(), re.I) + return float(m[-1]) if m else None +kl = top1 = None +try: + d = json.load(open(out)) + def find(obj, keys): + if isinstance(obj, dict): + for k, v in obj.items(): + if any(s in k.lower() for s in keys) and isinstance(v, (int, float)): + return float(v) + for v in obj.values(): + r = find(v, keys) + if r is not None: return r + return None + kl = find(d, ("mean_kl", "mean kl", "kl_mean", "mean")) + top1 = find(d, ("top1", "top_1", "top-1", "agreement")) + if top1 is not None and top1 <= 1.0: + top1 *= 100.0 +except Exception as e: + print("report JSON unusable (%s); falling back to log" % e) +if kl is None: + kl = num(r"mean[_ ]kl[^0-9]*([0-9]*\.?[0-9]+(?:e[-+][0-9]+)?)") +if top1 is None: + top1 = num(r"top[-_ ]?1[^0-9]*([0-9]*\.?[0-9]+)") +if kl is None or top1 is None: + print("KLD GATE PARSE FAILED: mean_kl=%s top1=%s — inspect %s / %s" % (kl, top1, out, log)) + sys.exit(2) +print("KLD GATE: mean KL = %.4f (max 0.03) | top-1 = %.2f%% (min 95.0)" % (kl, top1)) +ok = (kl <= 0.03) and (top1 >= 95.0) +print("KLD GATE " + ("PASS" if ok else "FAIL")) +sys.exit(0 if ok else 1) +PY +gate=$? +echo "kld-gate complete rc=$gate; log: $LOG report: $KLD_OUT" +exit "$gate" diff --git a/scripts/kld-then-calib.sh b/scripts/kld-then-calib.sh new file mode 100755 index 0000000..8a42bb8 --- /dev/null +++ b/scripts/kld-then-calib.sh @@ -0,0 +1,40 @@ +#!/bin/bash +# kld-then-calib.sh — RUNS ON DANKSERVER. Campaign chain: wait for the KLD +# record to finish, verify it, then boot the 1.2.0 calibration server and +# push the 10M-token corpus. One GPU, so the phases serialize; prod is +# restored by each run script's trap. +set -u +KLOG=/home/dank/logs/swift15-kld-driver.log +CLOG=/home/dank/logs/swift15-calib-driver.log +S=/home/dank/serverAppConfigs-swift15/scripts +# Phase 1: KLD record (SKIP_KLD=1 when the record is already complete — +# kld.json's rows count is the receipt; a mid-run script swap can kill the +# wrapper with rc=127 after the engine exited clean). +if [ "${SKIP_KLD:-0}" != 1 ]; then + while ! grep -q "exited rc=" "$KLOG" 2>/dev/null; do sleep 60; done + RC=$(grep -o "exited rc=[0-9]*" "$KLOG" | tail -1) + if [ "$RC" != "exited rc=0" ]; then echo "KLD FAILED ($RC); calib NOT started"; exit 1; fi +fi +N=$(find /home/dank/corpora/kld-ref -type f | wc -l) +SZ=$(du -sm /home/dank/corpora/kld-ref | cut -f1) +echo "KLD OK files=$N size_mib=$SZ" +[ "$N" -ge 5 ] && [ "$SZ" -ge 1000 ] || { echo "kld-ref looks thin; calib NOT started"; exit 1; } +grep -q '"rows": 50235' /home/dank/corpora/kld-ref/kld.json || { + echo "kld.json rows != 50235; calib NOT started"; exit 1; } + +# Phase 2: calibration serve (1.2.0) + corpus push. +nohup bash "$S/run-swift15-bf16.sh" calib > "$CLOG" 2>&1 & +CP=$! +for _ in $(seq 1 160); do + sleep 15 + curl -sf localhost:7770/health >/dev/null 2>&1 && break +done +if ! curl -sf localhost:7770/health >/dev/null 2>&1; then + echo "CALIB BOOT FAILED; see /home/dank/logs/swift15_bf16.log"; kill "$CP" 2>/dev/null; exit 1 +fi +echo "CALIB UP, pushing corpus" +python3 "$S/calib-push.py" --port 7770 --parallel 16 --container swift15ref --max-rounds 6 +PUSH=$? +kill "$CP" 2>/dev/null +sleep 10 +echo "CALIB_CHAIN_DONE push=$PUSH grams=$(find /home/dank/corpora/calib -name 'gram.r*.bin' 2>/dev/null | wc -l)" diff --git a/scripts/make-calib-corpus.py b/scripts/make-calib-corpus.py new file mode 100755 index 0000000..e6cf6eb --- /dev/null +++ b/scripts/make-calib-corpus.py @@ -0,0 +1,317 @@ +#!/usr/bin/env python3 +"""Build the two Swift1.5 campaign corpora (calibration + KLD). + +calib.jsonl — 10M tokens of agent-shaped RAW text (45% code, 30% prose, +25% tool-call JSON), one {"prompt": text} per line, pushed to +/v1/completions with max_tokens=1 during the calibration run (the Hessian +tap is of the prompt tokens; docs/MOE-W4.md step 2). Code and prose mix +the repo checkout with HF dataset parquets (Frank's call 2026-10-06: +local material is only ~1.6M unique tokens; MOE-W4 wants 10M): +nampdn-ai/tiny-codes (code) and Salesforce/wikitext wikitext-103-raw +(prose). Tool calls are synthesized from commands harvested from the +repo's own scripts plus a fleet-ops set, with argument-shaped variants. + +kld.jsonl — 1.8M tokens, one {"prompt", "score_from", "source"} per +line; each prompt is a multi-turn conversation rendered through the +Froggeric v22.5 template with a >= --kld-min-prompt (8192) token target, +and the last 256 tokens (the tail of the assistant answer) are the +scored span: score_from = n - 256. At 8k-token prompts the 1.8M corpus +is ~220 prompts = ~56k scored positions ~= 25 GB of record, matching +the published Flash-Next KLD scale (83k positions, MOE-W4.md). The KLD +reference run and every candidate run consume the SAME file (TOOLS.md +--kld-record / --kld-ref). The KLD corpus is LOCAL material only, so the +file-level disjointness rule (fnv1a64(path) % 5 == 0 -> KLD pool) is +untouched by the HF additions. + +Deps: tokenizers, jinja2, pyarrow (venv /home/dank/venvs/radcorpus). +Usage: make-calib-corpus.py --repo ROOT --tokenizer-json PATH + --template PATH --out-dir DIR [--hf-code-parquet F,F] + [--hf-prose-parquet F,F] [--calib-tokens 10000000] + [--kld-tokens 1800000] +""" +import argparse +import json +import os +import random +import re + +from jinja2 import Environment, FileSystemLoader +from tokenizers import Tokenizer + +CODE_EXT = (".sh", ".py", ".jinja", ".recipe", ".env", ".example", + ".yaml", ".yml", ".bats", ".service", ".container", ".toml") +JSONL_EXTS = (".jsonl", ".json") +DOC_DIR = "serverAppConfigs-docs" + +FLEET_CMDS = [ + "systemctl status frank-primary --no-pager", + "journalctl -u frank-primary -n 100 --no-pager", + "podman ps --format '{{.Names}} {{.Status}}'", + "podman logs --tail 50 frank-primary", + "curl -s http://192.168.0.235:7770/health", + "curl -s http://192.168.0.235:7770/metrics | grep radiance_", + "df -h /home/dank | tail -1", + "free -g", + "sudo systemctl restart frank-primary", + "cd /home/dank/serverAppConfigs && git pull --ff-only", + "./scripts/deploy.sh dankserver --service frank-primary", + "./tests/run.sh 2>&1 | tail -5", + "jq . /home/dank/models/rad/index.json | head -40", + "python3 -c \"import json; print(json.load(open('config.json'))['num_hidden_layers'])\"", + "ip neigh show 192.168.0.235", + "amdgpu_top --device-info 2>/dev/null | head -20", + "grep -E 'MemAvailable' /proc/meminfo", + "sha256sum model-00001-of-00131.safetensors", + "git log --oneline -10", + "awk '{s+=$1} END {print s}' shard_sizes.txt", +] +TOOL_NAMES = ["bash", "read", "grep", "web_search", "edit", "write"] +WORKDIRS = ["/home/dank", "/home/dank/serverAppConfigs", "/home/dank/models", + "/etc/systemd/system", "/var/log"] +DESCS = ["check service state", "tail the engine ledger", "list containers", + "query health", "read metrics", "check disk", "check memory", + "restart the endpoint", "sync the checkout", "run deployer", + "run the test seams", "inspect the index", "probe the peer"] + + +def fnv(s: str) -> int: + h = 0xcbf29ce484222325 + for b in s.encode(): + h = ((h ^ b) * 0x100000001b3) & 0xFFFFFFFFFFFFFFFF + return h + + +def harvest_cmds(repo: str, rng: random.Random) -> list[str]: + cmds = list(FLEET_CMDS) + pat = re.compile(r"^\s*(?:sudo\s+)?([a-z][\w./-]*(?:\s+\S+){1,6})\s*$") + for root, dirs, files in os.walk(repo): + dirs[:] = [d for d in dirs if not d.startswith(".")] + for f in files: + if not f.endswith(".sh"): + continue + try: + for line in open(os.path.join(root, f), encoding="utf-8", + errors="replace"): + m = pat.match(line.rstrip()) + if m and len(m.group(1)) > 12: + cmds.append("ssh dank@192.168.0.235 '" + + m.group(1)[:90] + "'") + except OSError: + pass + rng.shuffle(cmds) + return cmds[:4000] + + +def tool_call(rng: random.Random, cmds: list[str]) -> str: + """One agent turn: 2-4 tool calls with argument-shaped JSON.""" + parts = [] + for _ in range(rng.randint(2, 4)): + args: dict = {"command": rng.choice(cmds)} + if rng.random() < 0.4: + args["cwd"] = rng.choice(WORKDIRS) + if rng.random() < 0.3: + args["timeout"] = rng.choice([10, 30, 60, 300, 600]) + if rng.random() < 0.5: + args["description"] = rng.choice(DESCS) + parts.append('{"name": "%s", "arguments": %s}<|im_end|>' + % (rng.choice(TOOL_NAMES), + json.dumps(args, ensure_ascii=False))) + return "\n".join(parts) + + +def load_parquet_texts(path: str, kind: str, rng: random.Random, + cap: int) -> list[str]: + import pyarrow.parquet as pq + d = pq.read_table(path).to_pydict() + n = len(next(iter(d.values()))) + idx = range(n) if n <= cap else rng.sample(range(n), cap) + out = [] + for i in idx: + if kind == "code": + if "content" in d: + frag = d["content"][i] + else: + frag = ("// %s\n### Instruction:\n%s\n### Response:\n%s" + % (d.get("file_name", [""])[i], d["prompt"][i], + d["code"][i] or d["response"][i])) + else: + frag = d["text"][i] + if frag and frag.strip(): + out.append(frag[:8000]) + rng.shuffle(out) + return out[:cap] + + +def main() -> None: + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--repo", required=True) + ap.add_argument("--tokenizer-json", required=True) + ap.add_argument("--template", required=True) + ap.add_argument("--out-dir", required=True) + ap.add_argument("--calib-tokens", type=int, default=10_000_000) + ap.add_argument("--kld-tokens", type=int, default=1_800_000) + ap.add_argument("--chunk-tokens", type=int, default=8192) + ap.add_argument("--score-tail", type=int, default=256) + ap.add_argument("--kld-min-prompt", type=int, default=8192) + ap.add_argument("--hf-code-parquet", default="") + ap.add_argument("--hf-prose-parquet", default="") + ap.add_argument("--seed", type=int, default=20261006) + a = ap.parse_args() + + rng = random.Random(a.seed) + tokenizer = Tokenizer.from_file(a.tokenizer_json) + env = Environment(loader=FileSystemLoader(os.path.dirname(a.template)), + trim_blocks=True, lstrip_blocks=True) + env.filters["tojson"] = lambda v: json.dumps(v, ensure_ascii=False) + env.filters["raise_exception"] = lambda m: (_ for _ in ()).throw( + ValueError(m)) + tname = os.path.basename(a.template) + + code, prose, jsonl = [], [], [] + for root, dirs, files in os.walk(a.repo): + dirs[:] = [d for d in dirs if not d.startswith(".")] + for f in sorted(files): + p = os.path.relpath(os.path.join(root, f), a.repo) + if DOC_DIR in p and f.endswith(".md"): + prose.append(p) + elif p.startswith(DOC_DIR): + pass + elif f.endswith(CODE_EXT): + code.append(p) + elif f.endswith(JSONL_EXTS) and f != "package-lock.json": + jsonl.append(p) + # file-level split: bucket 0 -> KLD pool, 1..4 -> calibration pool + pools: dict[str, tuple[list, list]] = {} + for cls, lst in (("code", code), ("prose", prose), ("tools", jsonl)): + pools[cls] = ([p for p in lst if fnv(p) % 5 == 0], + [p for p in lst if fnv(p) % 5 != 0]) + + def text_of(p: str) -> str: + path = os.path.join(a.repo, p) + raw = open(path, encoding="utf-8", errors="replace").read() + if p.endswith(JSONL_EXTS): + lines = [json.loads(l)["content"] for l in raw.splitlines() + if l.strip() and "content" in l] + raw = "\n".join(lines) + toks = tokenizer.encode(raw, add_special_tokens=False).ids + return tokenizer.decode(toks[:2400], skip_special_tokens=True) + + cmds = harvest_cmds(a.repo, rng) + # fragment pools per class, corpus-split first + frags: dict[str, dict[str, list[str]]] = {} + for cls in pools: + kld_files, cal_files = pools[cls] + fr = {} + src = {"kld": kld_files, "cal": cal_files} + if cls == "tools": + fr["kld"] = [tool_call(rng, cmds) for _ in range(600)] + fr["cal"] = [tool_call(rng, cmds) for _ in range(40000)] + else: + fr["kld"] = [text_of(p) for p in src["kld"]] + fr["cal"] = [text_of(p) for p in src["cal"]] + frags[cls] = fr + # per-class fragment caps: prose rows are short (~66 tok) so need many; + # codeparrot files are long (~2.6k tok) so a small cap suffices + hfcap = {"code": 40000, "prose": 60000, "tools": 0} + for cls, spec in (("code", a.hf_code_parquet), + ("prose", a.hf_prose_parquet)): + for path in [p for p in spec.split(",") if p]: + frags[cls]["cal"].extend( + load_parquet_texts(path, cls, rng, cap=hfcap[cls])) + for cls in frags: + rng.shuffle(frags[cls]["cal"]) + + def toks(s: str) -> int: + return len(tokenizer.encode(s, add_special_tokens=False).ids) + + def emit_calib() -> tuple[int, list[dict]]: + targets = {"code": .45, "prose": .30, "tools": .25} + rows: list[dict] = [] + for cls, frac in targets.items(): + want = int(a.calib_tokens * frac) + pool = frags[cls]["cal"] + if not pool: + continue + got, i, buf, nbuf = 0, 0, [], 0 + while got < want: + frag = pool[i % len(pool)] + i += 1 + n = toks(frag) + if nbuf + n > a.chunk_tokens and buf: + rows.append({"prompt": "\n".join(buf), "cls": cls}) + got += nbuf + buf, nbuf = [], 0 + if got >= want: + break + buf.append(frag) + nbuf += n + if i > len(pool) * 40: + break # pool exhausted with repeats; stop + if buf and got < want: + rows.append({"prompt": "\n".join(buf), "cls": cls}) + got += nbuf + print(f"calib {cls}: {got} tokens, {i} fragments") + rng.shuffle(rows) + return len(rows), [{k: v for k, v in r.items() if k == "prompt"} + for r in rows] + + def emit_kld() -> tuple[int, list[dict]]: + rows, total = [], 0 + classes = ["code", "prose", "tools"] + i = {c: 0 for c in classes} + guard = 0 + while total < a.kld_tokens and guard < 5000: + guard += 1 + msgs, srcs = [], [] + for t in range(rng.choice([2, 3, 4])): + cls = classes[(len(rows) + t) % 3] + pool = frags[cls]["kld"] + if not pool: + continue + frag = pool[i[cls] % len(pool)] + i[cls] += 2 + msgs.append({"role": "user", + "content": "Summarize and explain this " + f"engineering material:\n{frag}"}) + msgs.append({"role": "assistant", + "content": frag[:4000]}) + srcs.append(cls) + rendered = env.get_template(tname).render( + messages=msgs, add_generation_prompt=False) + n = toks(rendered) + if n < a.kld_min_prompt: + continue + rows.append({"prompt": rendered, + "score_from": n - a.score_tail, + "source": "+".join(sorted(set(srcs)))}) + total += n + return len(rows), rows + + os.makedirs(a.out_dir, exist_ok=True) + nc, rows = emit_calib() + with open(os.path.join(a.out_dir, "calib.jsonl"), "w") as f: + for r in rows: + f.write(json.dumps(r) + "\n") + nk, krows = emit_kld() + with open(os.path.join(a.out_dir, "kld.jsonl"), "w") as f: + for r in krows: + f.write(json.dumps(r) + "\n") + scored = sum(len(tokenizer.encode(r["prompt"], add_special_tokens=False).ids) + - r["score_from"] for r in krows) + manifest = { + "seed": a.seed, "template": tname, + "tokenizer": os.path.basename(a.tokenizer_json), + "calib_prompts": nc, "kld_prompts": nk, + "kld_scored_positions": scored, + "kld_score_tail": a.score_tail, + "disjoint_file_pools": "local KLD material only; HF code feeds calib", + "calib_mix": {"code": .45, "prose": .30, "tools": .25}, + "hf_code": a.hf_code_parquet, "hf_prose": a.hf_prose_parquet, + } + with open(os.path.join(a.out_dir, "manifest.json"), "w") as f: + json.dump(manifest, f, indent=1) + print(json.dumps(manifest, indent=1)) + + +if __name__ == "__main__": + main() diff --git a/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe b/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe new file mode 100644 index 0000000..b368f44 --- /dev/null +++ b/scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe @@ -0,0 +1,105 @@ +# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next +# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official +# base) as a radiance container for frank-primary: EVERY routed expert -- gate_up AND down -- +# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law +# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served +# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 + +# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it. +# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers +# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head. +# +# Why down is the calibrated half: the calibration tap rides the down projection, so its GPTQ +# rounding propagates against real down-input Hessians; gate_up shares the same w4 table +# uncalibrated. The published w5dn64 comparison (mean KL 0.0789 -> 0.0717, top-1 92.85% -> +# 93.34%, decode +4%, prefill -19%) is the 5-bit upper bound; this tier trades part of that +# quality back for prefill. +# +# CALIB=DIR rad-convert \ +# --recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe -o qwen4exp-swift15-w4dn8-i8-hc8m.rad +# +# DIR is the calibration directory the bf16 container wrote (RADIANCE_CALIB_DIR, 10M tokens of +# traffic-shaped corpus). Everything not named here is served as the checkpoint holds it (bf16: +# embeddings, the vision tower, the QSA indexer); the first rule that matches a weight decides it. +# The bf16 container is NOT passed with --reuse: the peak-disk arithmetic does not fit (checkpoint + +# container + container + output > purge-freed space), and --reuse only saves a re-read. + +# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE +# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only. +# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites +# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's +# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three +# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM +# takes the rest as two equal tables by parity) by adding that half's next most energetic expert. +# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py) +blk.1.ffn_*_exps.271.weight cast dtype=bf16 +blk.1.ffn_*_exps.355.weight cast dtype=bf16 +blk.2.ffn_*_exps.279.weight cast dtype=bf16 +blk.2.ffn_*_exps.413.weight cast dtype=bf16 +blk.34.ffn_*_exps.146.weight cast dtype=bf16 +blk.34.ffn_*_exps.153.weight cast dtype=bf16 +blk.34.ffn_*_exps.407.weight cast dtype=bf16 +blk.34.ffn_*_exps.453.weight cast dtype=bf16 +blk.37.ffn_*_exps.288.weight cast dtype=bf16 +blk.37.ffn_*_exps.377.weight cast dtype=bf16 +blk.38.ffn_*_exps.406.weight cast dtype=bf16 +blk.38.ffn_*_exps.420.weight cast dtype=bf16 +blk.42.ffn_*_exps.37.weight cast dtype=bf16 +blk.42.ffn_*_exps.90.weight cast dtype=bf16 +blk.42.ffn_*_exps.302.weight cast dtype=bf16 +blk.42.ffn_*_exps.388.weight cast dtype=bf16 +blk.44.ffn_*_exps.315.weight cast dtype=bf16 +blk.44.ffn_*_exps.350.weight cast dtype=bf16 +blk.46.ffn_*_exps.290.weight cast dtype=bf16 +blk.46.ffn_*_exps.419.weight cast dtype=bf16 +blk.47.ffn_*_exps.119.weight cast dtype=bf16 +blk.47.ffn_*_exps.143.weight cast dtype=bf16 +blk.47.ffn_*_exps.399.weight cast dtype=bf16 +blk.47.ffn_*_exps.445.weight cast dtype=bf16 +# PROTECTED-END +# (derive-protected-experts.py rewrote this section from Swift's own grams) + +# Every other routed expert, trunk and MTP layer alike: u4 codes into the w4nl table, rotated by +# the 128-wide Hadamard, an E4M3 scale a group of 64 under the fixed 2^-13 searched for least +# error as stored, chosen by GPTQ against $CALIB and swept three times by coordinate descent. +# The calibration Grams are down-input Hessians, so down is the calibrated half of the table; +# gate_up has no recording of its own and quantises uncalibrated (said on stderr, as the base +# recipe says). The MTP layer has no recording either. Expert mass: gate_up 150.00 GiB -> ~37.6, +# down 75.00 GiB -> ~18.8: ~56 GiB of experts, ~28 GiB a rank at TP=2 -- well inside the pool. +blk.*.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB + +# Every connection's two mixing matrices -- the 96 trunk connections, the final mixer and the MTP +# head's three -- as E4M3 rows with an f32 scale a group of each row. A 128x128 block cannot cover +# them: lowrank is 320, 2.5 blocks. Down is [320, 10240] and takes a scale every 128 columns; up is +# [10240, 320] and takes four scales a row, 80 columns each. +*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32 +*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32 + +# The MTP head's fc_hidden and fc_embedding, the same rows. They only ever change the drafts. +mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32 +mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32 + +# The trunk's block linears as int8, a scale per 128 columns of a row searched for least error. +# The QSA indexer's projection is not one of them: its top-k is a hard choice between blocks, and +# keeping it as the checkpoint holds it costs 20 MB. +blk.*.attn_qg.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search +blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search +blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search +blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search +blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search +blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search +blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search +blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search + +# The lm_head, int8 with a bf16 scale a row per 128 columns (logits_gemm_i8): at E4M3 it costs +# ~0.6 points of top-1 on its own. +output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search + +# The n-gram table at one fixed E4M3 scale; qwen4exp-w4.recipe says where the value comes from. +# The value sits on a three-octave plateau (2^-13..2^-10, rel_l2 0.02662) with 1.7x headroom over +# the base table's absmax; the prep phase verifies Swift's PLE absmax lands inside the plateau +# before this rule is trusted. +blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4 + +# The MTP head's 2-bit draft head. INERT for this checkpoint: the HF weight map has no +# mtp.draft_head.weight (same as the base), kept for shape parity with the family. +mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16 diff --git a/scripts/run-swift15-bf16.sh b/scripts/run-swift15-bf16.sh new file mode 100755 index 0000000..6aa3aed --- /dev/null +++ b/scripts/run-swift15-bf16.sh @@ -0,0 +1,116 @@ +#!/usr/bin/env bash +# run-swift15-bf16.sh — RUNS ON DANKSERVER. Boot the no-recipe bf16 Swift1.5 +# container for the two offline campaign phases. The engine owns the GPUs, so +# the production unit must be stopped first (this script does it and restores +# on exit, autoresearch pattern: stop, never mask — relinked unit). +# +# kld --kld-record the reference logits from the bf16 model +# (runs the corpus and EXITS; engine writes DIR and leaves) +# calib serve with the calibration tap; a separate driver +# (scripts/calib-push.py) pushes the corpus; the engine writes the +# Hessians at RADIANCE_CALIB_TOKENS and the tap switches itself off. +# Stop this script after calib-push prints TAP_DONE. +# +# Shape = the vendor's bf16 calibration lines (GUIDE.md §4.3:528 + MOE-W4.md +# "calibrate the checkpoint as it ships"): expert_tiered + weights-disk-tier, +# no speculation, deterministic. Large prefill steps because a disk-tier step +# costs the same at 1 token as at 32K. The container declared --max-ctx 4096; +# --max-model-len 16384 exceeds it on purpose (the KLD prompts run to 12,007 +# tokens, calib to 8,242) — the boot ledger arbitrates the QSA buffers; if +# declare refuses, re-convert with a wider --max-ctx. +set -uo pipefail + +MODE="${1:?usage: run-swift15-bf16.sh kld|calib}" +PIN_1013=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e +PIN_120=sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa +RAD=/home/dank/models/rad/qwen4exp-swift15-bf16.rad +C=swift15ref +LOG=/home/dank/logs/swift15_bf16.log +KLD_REF_DIR=/home/dank/corpora/kld-ref +CALIB_DIR=/home/dank/corpora/calib +KLD_CORPUS=/home/dank/corpora/out/kld.jsonl + +restore() { sudo systemctl start frank-primary >/dev/null 2>&1 || true; } +trap restore EXIT +sudo systemctl stop frank-primary >/dev/null 2>&1 || true +podman rm -f "$C" >/dev/null 2>&1 || true + +ARGS=(--model /models/rad/qwen4exp-swift15-bf16.rad + --tp 2 --placement expert_tiered --weights-disk-tier + --host-pool-mib 12288 --deterministic --gpu-headroom-mib 768 + --num-speculative-tokens 0 + --kv-cache-dtype fp8 --max-model-len 16384 + --ngram-placement ram + --override-chat-template /models/tpl.jinja) + +case "$MODE" in + kld) + # 1.0.13: the engine that will serve the candidate — the reference is + # recorded on the same engine that scores it. The weights slab DERIVES as + # card - headroom - kv, and the disk tier refuses under 9.57 GiB (4.21 of + # layer buffers + 5.36 fixed), so kv is explicit SMALL: recording is + # prefill-only, self-batched at MNBT 32768 = two 16k prompts at a time; + # 4 slots x 16k fp8 (29 KiB/pos, 1.7 GiB) + headroom fits 3072. + IMAGE="stilldeadcode/radiance@$PIN_1013" + mkdir -p "$KLD_REF_DIR" + ARGS+=(--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl + --vram-kv-mib 3072 + --max-num-seqs 4 --max-num-batched-tokens 32768) + ;; + calib) + # 1.2.0: the calibration tap (RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS) + # lives in the 1.2.0 qwen4exp plugin; 1.0.13 has only the DIR half with an + # unparameterisable threshold. Grams are bf16 activations — engine-invariant. + # Same KV ceiling as kld: the disk-tier weights floor (9.57 GiB) plus + # ~18 GiB of fixed per-rank claims cap the pool near 3.5 GiB on a 32 GiB + # card — pool minus the 2.17 GiB fixed holds ~237k fp8 tokens a rank, so + # the vendor's big-step calibration shape is affordable: MOE-W4.md:244 + # measures 640 tok/s at a 32k step against 930 at 55k, and the doc's own + # run drives MNBT 65536. 8 slots x 16k = 131k resident tokens fits the + # pool; the step batches 65k. KLD's observed rate confirmed step size, + # not slot count, is the throughput lever. + IMAGE="stilldeadcode/radiance@$PIN_120" + mkdir -p "$CALIB_DIR" + ARGS+=(--port 7770 --vram-kv-mib 3072 + --max-num-seqs 8 --max-num-batched-tokens 65536 + --prefix-cache-dir /data/calib-kvcache) + ;; + probe) + IMAGE="stilldeadcode/radiance@$PIN_1013" + ARGS+=(--port 7770 --max-num-seqs 8 --max-num-batched-tokens 4096) + ;; + *) echo "unknown mode: $MODE" >&2; exit 2 ;; +esac + +# The prod template copy, readable from the container: mount the models tree +# ro plus the state dir rw (KLD record + prefix tier land there). +cp -f /home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja \ + /home/dank/models/tpl.jinja 2>/dev/null || true + +KLD_MOUNT=() +if [ "$MODE" = kld ]; then + # Nested under the /data mount, so these must come after it (line order + # in podman run below): the corpus read-only, the record dir writable. + KLD_MOUNT=(-v "$KLD_CORPUS:/data/kld.jsonl:ro" + -v "$KLD_REF_DIR:/data/kld-ref") +fi +EXTRA_ENV=() +if [ "$MODE" = calib ]; then + EXTRA_ENV=(-v "$CALIB_DIR:/data/calib" + -e RADIANCE_CALIB_DIR=/data/calib + -e RADIANCE_CALIB_TOKENS=10000000) +fi + +podman run --rm --name "$C" \ + --network host --init \ + --security-opt seccomp=unconfined \ + --ulimit memlock=-1:-1 \ + --device /dev/kfd --device /dev/dri \ + -v /home/dank/models:/models:ro \ + -v /home/dank/radiance-state:/data \ + "${KLD_MOUNT[@]}" \ + "${EXTRA_ENV[@]}" \ + "$IMAGE" "${ARGS[@]}" 2>&1 | tee "$LOG" +rc=${PIPESTATUS[0]} +echo "swift15-bf16 $MODE exited rc=$rc; log: $LOG" +exit "$rc" diff --git a/scripts/serve-frank-primary.sh b/scripts/serve-frank-primary.sh index a0bf8a2..d30a1b2 100755 --- a/scripts/serve-frank-primary.sh +++ b/scripts/serve-frank-primary.sh @@ -50,7 +50,8 @@ set +a : "${FRANK_PRIMARY_PLACEMENT:?FRANK_PRIMARY_PLACEMENT must be set}" : "${FRANK_PRIMARY_HOST_POOL_MIB:?FRANK_PRIMARY_HOST_POOL_MIB must be set}" : "${FRANK_PRIMARY_GPU_HEADROOM_MIB:?FRANK_PRIMARY_GPU_HEADROOM_MIB must be set}" -: "${FRANK_PRIMARY_EXPERT_VS_CACHE:?FRANK_PRIMARY_EXPERT_VS_CACHE must be set}" +: "${FRANK_PRIMARY_VRAM_WEIGHTS_MIB:?FRANK_PRIMARY_VRAM_WEIGHTS_MIB must be set}" +: "${FRANK_PRIMARY_VRAM_KV_MIB:?FRANK_PRIMARY_VRAM_KV_MIB must be set}" : "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set}" : "${FRANK_PRIMARY_SPEC_TOKENS:?FRANK_PRIMARY_SPEC_TOKENS must be set}" : "${FRANK_PRIMARY_MAX_BATCHED_TOKENS:?FRANK_PRIMARY_MAX_BATCHED_TOKENS must be set}" @@ -186,7 +187,8 @@ podman run -d --name "$FRANK_PRIMARY_CONTAINER" \ --placement "$FRANK_PRIMARY_PLACEMENT" \ --host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB" \ --gpu-headroom-mib "$FRANK_PRIMARY_GPU_HEADROOM_MIB" \ - --expert-vs-cache-ratio "$FRANK_PRIMARY_EXPERT_VS_CACHE" \ + --vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB" \ + --vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB" \ --kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \ --prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB" \ --prefix-cache-dir /data/kvcache/frank-primary \ diff --git a/scripts/verify-swift15-ple.py b/scripts/verify-swift15-ple.py new file mode 100755 index 0000000..71df63f --- /dev/null +++ b/scripts/verify-swift15-ple.py @@ -0,0 +1,87 @@ +#!/usr/bin/env python3 +"""Streaming absmax audit of the Swift1.5 PLE n-gram table (BF16). + +The PLE fp8 encoding uses ONE fixed scale (1.9931793212890625e-04, the +official checkpoint's plateau value). fp8_e4m3 saturates at 448, so the +fixed scale is legal iff absmax <= 448 * SCALE. This walks every +ngram_embedding.shard_*.weight tensor in the checkpoint (mmap, no copy), +computes the absmax from the BF16 bit pattern (clear the sign bit, max +the remaining uint16 — BF16 magnitudes are monotone in their bits), and +reports the headroom. Exit 0 only when headroom >= 1.5x. + +Run from any cwd EXCEPT /tmp on dankserver (/tmp/inspect.py shadows the +stdlib). Usage: verify-swift15-ple.py +""" +import json +import mmap +import os +import struct +import sys + +SCALE = 1.9931793212890625e-04 # bf16 0x3951, the official plateau scale +FP8_E4M3_MAX = 448.0 + +import numpy as np +gmax = 0.0 +gfile = "" +over_cap = 0 +total = 0 + +ckpt = sys.argv[1] +index = json.load(open(os.path.join(ckpt, "model.safetensors.index.json"))) +shards: dict[str, list[str]] = {} +for k, f in index["weight_map"].items(): + if "ngram_embedding.shard_" in k and k.endswith(".weight"): + shards.setdefault(f, []).append(k) + +files = len(shards) +tensors = sum(len(v) for v in shards.values()) +print(f"n-gram shards: {files} files, {tensors} tensors") + +import math + +cap = FP8_E4M3_MAX * SCALE +# largest bf16 <= cap: truncate the float32 bits (cap > 0); bf16 +# magnitudes are monotone in their bits, so |w| > cap <=> bits > cap_bits +cap_bits = int((np.float32(cap).view(np.uint32) >> 16) & 0xFFFF) +for fname, keys in sorted(shards.items()): + path = os.path.join(ckpt, fname) + with open(path, "rb") as fh: + hlen = struct.unpack(" np.uint16(cap_bits)).sum()) + total += v.size + del a + if m > gmax: + gmax = m + gfile = k + mm.close() + +# rebuild the BF16 value: the 16 bits are the HIGH half of the float32 +absmax = np.frombuffer(b"\x00\x00" + np.uint16(gmax).tobytes(), + dtype=np.float32)[0] + +print(f"absmax = {absmax!r} (tensor: {gfile})") +print(f"2^log2 = 2^{math.log2(absmax):.3f}") +print(f"fp8 cap = 448 * {SCALE:.8g} = {cap:.8g}") +print(f"headroom = {cap / absmax:.3f}x") +ok = cap / absmax >= 1.5 +print("VERDICT:", "PASS (>= 1.5x headroom over the fixed scale)" if ok + else "FAIL — fixed scale does not cover the Swift PLE") +print(f"values > cap = {over_cap} of {total} ({over_cap / total:.3e})") +nxt_bits = int((np.float32(absmax / FP8_E4M3_MAX).view(np.uint32) + >> 16) & 0xFFFF) + 1 +nxt_scale = np.frombuffer(b"\x00\x00" + np.uint16(nxt_bits).tobytes(), + dtype=np.float32)[0] +print(f"next-bf16 scale covering absmax: {nxt_scale:.8g} " + f"(0x{nxt_bits:04x}, headroom {FP8_E4M3_MAX * nxt_scale / absmax:.4f}x)") +sys.exit(0 if ok else 1) diff --git a/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md b/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md new file mode 100644 index 0000000..e990347 --- /dev/null +++ b/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md @@ -0,0 +1,177 @@ +# frank-primary — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2 + +**Status: STAGED — activates at the cutover below.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2 + +The FP8-native era (`qwen4exp-fp8-native.rad`, 173.0 GiB, no-recipe convert) +is replaced by a **GPTQ-calibrated 4-bit container** of +`ukisai/Swift1.5-Qwen3.8-Flash-Next` (BF16, 335.3 GiB, abliterated +finetune) converted with the recipe +`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` on engine +**1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…` — same digest as +the FP8-native era; 1.2.0 was used ONLY for the calibration serve). +Same port, alias, container name, systemd unit. Rollback: revert the +`MODEL_FILE` line in `scripts/frank-primary.env` + redeploy — +`/home/dank/models/rad/qwen4exp-fp8-native.rad` stays on disk (trash +policy; 173 GiB, purge when the window closes). + +## Why this swap + +- **Context economics at production memory budgets**: the FP8-native + experts (56.4 GiB/rank slice) forced a 40 GiB/rank pinned host pool and + left KV ~3.5 GiB. W4Dn8 experts cut the resident set to ~1/4, freeing + host and VRAM budget for KV and slots at equal-or-better measured + decode (gate: ≥ 30.8 t/s vs the 30.08 t/s control). +- **Swift 1.5 base**: abliterated finetune of Flash-Next (Frank's + fleet-personality call; frank-large serves the same family). +- **GPTQ closes the precision gap**: RTN alone on BF16 weights lands + measurably worse; GPTQ with Hessians from a 10M-token corpus of this + model's own distribution recovers most of it, verified end-to-end by a + KLD gate against the BF16 container itself (KL ≤ 0.03, top-1 ≥ 95 %). + +## The payload chain + +1. **Fetch**: `scripts/download-frank-primary-swift15.sh` → + `/home/dank/models/Swift1.5-Qwen3.8-Flash-Next` (131 shards, 335.3 + GiB, per-shard LFS sha256 verified, no token on the command line). + 48 layers, 512 experts (top-10 routed), moe_inter 640, hidden 2560, + PLE 20M-vocab table 95.37 GiB, MTP head, vision tower. +2. **BF16 container**: `run-swift15-bf16.sh convert` → + `rad-convert` no-recipe (pure copy) at `--max-ctx 4096` → + `qwen4exp-swift15-bf16.rad` (335.28 GiB). Serves on 1.0.13 as the + KLD reference engine (disk-tier placement; boot ~35 min). +3. **KLD reference**: `run-swift15-bf16.sh kld` (1.0.13) → + `/home/dank/corpora/kld-ref` — `radiance-kld-1` format, `rows: + 50235`, 197 source-labelled docs, top-16 log-probs (f16), corpus + fnv1a64 `a9d62b33c18b3d9c`. ~4 h at 32k prefill steps + (`--max-num-batched-tokens 32768` — a 65536 step OOMs an arena + tensor at `--max-model-len 16384`). **The wrapper's exit line is the + receipt: never scp this script while it runs** (bash lazy-read; + a mid-run swap killed a clean engine exit with rc=127, third fleet + occurrence of the script-swap law). `kld-then-calib.sh SKIP_KLD=1` + resumes from the `rows: 50235` receipt. +4. **Calibration**: `run-swift15-bf16.sh calib` on **1.2.0** (the only + engine with `RADIANCE_CALIB_TOKENS`; grams are engine-invariant + bf16 activations — RADGRAM2 format byte-identical across versions) + + `calib-push.py` pushing `calib-corpus.jsonl` (10,014,932 tokens, + 1,294 prompts, assembled from real HF sources, chat-templated, + source-labelled). The tap (in the `qwen4exp_fp8.so` architecture + plugin) writes per-expert down-proj Gram matrices + `gram.r.blk.L.ffn_down_exps..bin` (32-byte RADGRAM2 header + + 640×640 f32; `rows` = routing count, `trace` = activation energy) + and stops the serve at the token threshold. TP=2: whole experts per + rank (256/rank); gate_up input is rank-sliced (plugin warning — the + recipe only casts the down projection). +5. **Protected-expert derivation**: `derive-protected-experts.py + /home/dank/corpora/calib` → the 24 highest-energy experts (the law + is data-driven; Swift drew 24, not the fp8-era 10 — its routing is + spikier: `blk.47 e445` alone carries 32 % of its layer's down energy + at 70× routing share) promoted to whole-expert bf16. + Protection law: energy share ≥ 3 % AND ≥ 3× routing share; even count + per rank half; odd half gains the next most energetic. + `apply-protected-cast.sh` splices the derived names between the + `# PROTECTED-BEGIN/END` markers of the recipe (refuses double-derive). +6. **Final convert**: `convert-swift15-w4dn8.sh` (1.0.13, host-backend + CPU-only — no GPU visible to `rad-convert`; GPTQ solve is CPU BLAS) — + gates: ≥48 gram files present, bf16 container already deleted + (disk), ≥146 GiB free, output ≥108,000 MiB, `rad-info` shows the + gptq receipt. No `--reuse`. Measured: 5 h 50 m (6.0–6.5 min/layer on + 49 GPTQ layers, load-average 8), output **116,416 MiB**. + The plugin's `rad_arch_declare` enforces ONE ENCODING PER EXPERT + TABLE (`one table of experts takes one encoding, and only a whole + expert kept plain bf16 is served beside it`) — a split tier + (down=fp8, gate_up=w4) is unservable: the kernel serves a single + w4nl64a8h table; both expert tables ride GPTQ w4 (down calibrated, + gate_up uncalibrated), protected experts bf16 beside them. + +## The recipe tiers (scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe) + +| Tier | Encoding | Why | +|---|---|---| +| MoE experts, both `gate_up` and `down` (one w4 table/layer) | **GPTQ w4nl64a8h** — u4 codes, fwht128 rotation, fp8 scales/64 + f32 scale plane | the campaign tier: GPTQ on down-input Hessians from the 10M-token corpus (the tap rides the down projection; gate_up shares the table uncalibrated — one-encoding-per-table plugin law) | +| 24 derived protected experts (whole) | **bf16** | energy-share law (step 5) | +| shared expert + trunk attention/SSM linears | **int8**, scale per 128 cols, searched | trunk precision law | +| hyper-cond + MTP head | **fp8_e4m3 row-groups** | hc+mtp fp8 law | +| lm_head | **int8** | i8 lm_head law (E4M3 costs 0.6 top-1 alone) | +| PLE n-gram table | fp8 at the checkpoint's fixed scale 1.9931793212890625e-04 | the fixed-scale law; absmax audit passed the 2⁻¹³..2⁻¹⁰ plateau | +| embeddings, vision tower, QSA indexer | **bf16 passthrough** | untouched | + +## The shape (`scripts/frank-primary.env`) + +Unchanged from the FP8-native era: TP=2 + `wht6`, `--max-num-seqs 3`, +`--max-model-len 200000`, MNBT 2048 (the MTP resonance law — 2560 +collapses decode to 2.6–7.7 t/s), fp8 KV, `expert_tiered` host pool +40960 (the load-bearing floor), `--ngram-placement ram`, froggeric +v22.5 template override. `MODEL_FILE` points at +`/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 +MiB measured**); the serve size floor `FRANK_PRIMARY_RAD_MIN_MIB` moves +177000 → **110000**. + +## Gates (measured at cutover) + +- **KLD gate: PASS (2026-10-08 04:12)** — `kld-gate-swift15.sh`, + candidate on 1.0.13 vs `kld-ref`: **mean KL 0.0146** (median 0.00126, + p90 0.0315, p99 0.194, max 4.12), **top-1 97.84 %**, + **ppl ratio 1.0126** (candidate 1.6913 vs reference 1.6702), 50,235 + positions / 197 docs. Report: `/home/dank/radiance-state/kld-gate.json` + (+ `.rows` per position). Gate shape: disk-tier, pool 40960, MNBT 3072 + (the w4 activation arena scales hard with MNBT: 32768 → 33.3 GiB arena, + refused), KV 2048, headroom 512, no speculation. The engine ran rank 0 + fully on card 0 (25.88 GiB) with rank 1 host-side in this shape — + scoring is placement-invariant; scored at 28 positions/s. +- **Throughput gate: PASS (2026-10-08 05:40)** — decode at 44.5k depth + (streamed chat probe, temp 0, 512 tokens, salted): arm A (main serve + shape, EVC 0.82) **146–173 t/s**, engine timings **121.9 t/s**, + draft accept **0.7246** (350/483); arm C (deployed vram-pair shape + 26071/3543) median **136.0 t/s**, warm TTFT 9.45 s. Control + (fp8-native 1.0.13): 30.08 / accept 0.7549. Prefill ~4.7k t/s warm. + The w4 expert table is resident — the fp8-native era's NVMe expert + streaming (the 3.5 GB/s wall) is gone; the MTP ×3 head rides a 45–60 + t/s base decode. Conversation battery (short context): geomean + 172 t/s (min 100 / max 208), mean ttft 0.09 s. The EVC ladder is moot: + the w4 resident set fits fully inside the 40 GiB pool at 0.82 + (ledger: mover full fill). Depth-output audit: the 44.5k-depth + summary is a coherent ordered index of the concatenated corpus + (no repetition, no NaN-token garbage) — the depth-corruption class + of 2026-10-04 is clean. +- Fleet probes: PASS — vision (solid-red 512 px → "Red"), tool_calls + (`{"expr":"23 * 47"}`), reasoning_effort xhigh/medium/low/high all + 200 (the Swift1.5 template accepts the full set), reasoning_content + present, response_format json_object honoured. + + + +## Deploy + verify + +```bash +sudo systemctl stop frank-primary # drained window +bash convert-swift15-w4dn8.sh # if not already run +./scripts/deploy.sh dankserver --service frank-primary +``` + +Post-cutover checklist: unit active/enabled; `/health` 200; boot ledger +(expert pool coverage now w4-sized, KV pool should be larger than the +FP8-native era's 399,824 tokens); MTP draft counts in `timings`; froggeric +probe; ≥256 px `image_url` vision probe answers the color; tool-call +probe; `reasoning_effort` fold; Prometheus target green; `tests/run.sh` ++ Playwright e2e green. + +## Rollback + +| Layer | Path | +|---|---| +| FP8-native radiance (immediate) | revert `MODEL_FILE` in `scripts/frank-primary.env` + redeploy; `qwen4exp-fp8-native.rad` (173.0 GiB) on disk | +| FreeToken PR #587 | git tag `frank-primary-freetoken-pr587-20261005` | +| tcclaviger vLLM shape G | git tag `frank-primary-vllm-shapeG-20261003` | + +Campaign artifacts on disk: BF16 checkpoint (335 GiB), `kld-ref` (24 +GiB), grams (`/home/dank/corpora/calib`), corpora JSONL, logs +`/home/dank/logs/swift15_*.log`. The BF16 `.rad` container is deleted +before the final convert (disk gate) — the raw checkpoint is the +re-convert source if the window ever reopens. + +## Links + +- [[frank-primary-radiance-qwen38-native-fp8]] — the replaced FP8-native era +- [[LLM/index]] — fleet index +- [[swift15-campaign-worklog]] — campaign record +- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here diff --git a/serverAppConfigs-docs/LLM/index.md b/serverAppConfigs-docs/LLM/index.md index 88a1ba6..348c40a 100644 --- a/serverAppConfigs-docs/LLM/index.md +++ b/serverAppConfigs-docs/LLM/index.md @@ -152,6 +152,10 @@ frank-secondary, port 7771] target; payload on disk; owns the measured boot-pair table)**: the same runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower — 104.8 tok/s, 3,571 MiB true free +- [[frank-primary-radiance-swift15-w4dn8]] — **STAGED (campaign running)**: + Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance + 1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container); + replaces the FP8-native payload when the gates pass - [[frank-primary-radiance-qwen38-native-fp8]] — **ACTIVE since 2026-10-05** (root systemd unit, port 7770): the StillDeadcode **radiance** engine (C++/HIP, libr4d gfx1201 kernels, digest-pinned 287 MiB image, engine as diff --git a/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md b/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md new file mode 100644 index 0000000..bddcc2f --- /dev/null +++ b/serverAppConfigs-docs/LLM/swift15-campaign-worklog.md @@ -0,0 +1,72 @@ +# Swift1.5 w4dn8 Campaign — Work Log + +Campaign branch `swift15-radiance-w4dn8`. Goal: convert +`ukisai/Swift1.5-Qwen3.8-Flash-Next` (BF16, 335.3 GiB) to +`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (GPTQ gate_up w4nl64 + block-fp8 down + +10 protected bf16 experts + int8 trunk + fixed-scale PLE) and swap it into +frank-primary on the 1.0.13 pin. Design settled (25-question grill); this log +carries execution facts only. The cutover runbook +`frank-primary-radiance-swift15-w4dn8.md` replaces this page at cutover. + +## 2026-10-06 evening + +- Purge freed 835.8 GB; Swift1.5 BF16 downloaded 131/131 shards (sha256 ok). +- Corpora built: calib 10,014,932 tok / 1,294 prompts; KLD 1,800,816 tok / + 197 prompts (`/home/dank/corpora/out/`). +- Engine verdict: 1.2.0 REJECTED for serving (decode 29.21 vs control 30.08, + accept 0.7123 vs 0.7549 deterministic; vram-pair translation ledger-exact). + **Serving pin stays 1.0.13** `sha256:4c3f48…c322e`. +- bf16 rad-convert done: `/home/dank/models/rad/qwen4exp-swift15-bf16.rad` + (335.28 GiB, 248,077 vocab tokens baked). +- **Blocker resolved — engine-version probe law:** the calibration tap lives + in the architecture plugin `/opt/radiance/share/radiance/architectures/ + qwen4exp_fp8.so`, NOT the radiance binary, and the image has no + `strings(1)` (greps return 0 for everything — positive controls + mandatory). 1.0.13 plugin: `RADIANCE_CALIB_DIR` only; 1.2.0 adds + `RADIANCE_CALIB_TOKENS`. Both `libquant.so` carry the gptq quantizer + (`calib=/damp/act_order/cd`) and the identical `RADGRAM2` format + (magic + `gram.r%d.%s.bin` + calib_gram/calib_resolve/calib_stem). + **Decision: calibrate on 1.2.0, final convert on 1.0.13** — the serving + pin produces the served `.rad`; only the Gram files cross versions. +- `run-swift15-bf16.sh` boot-failure trail (all fixed, committed): + duplicated `radiance` entrypoint arg (rc=2); ratio-derived KV pool SHORT + 217 MiB at 32×16k (explicit `--vram-kv-mib`); explicit 4608 starved the + disk tier — **weights slab = card − headroom − kv, disk-tier floor + 9.57 GiB** (4.21 layer buffers + 5.36 fixed) → kv 3072; `KLD_MOUNT` + referenced-never-defined emptied the corpus mount. +- KLD record launched on 1.0.13 (disk tier, kv 3072, 4×16k, MNBT 32768): + measured ~116 tok/s at the 32k step (44 docs/58 min). +- Chain driver `kld-then-calib.sh` (pid 2848206, log + `/home/dank/logs/swift15-chain.log`): KLD rc=0 → verify kld-ref → boot + 1.2.0 calib (MNBT 65536/seqs 8 per MOE-W4 930 tok/s law, synced commit + 45de2fd) → `calib-push.py` parallel 16 → kill → prod restored by trap. +- Ready on the branch: `convert-swift15-w4dn8.sh` (1.0.13 convert, gates: + ≥126,000 MiB), `apply-protected-cast.sh` (derive → recipe PROTECTED + section, refuses double-derive), `kld-gate-swift15.sh` (candidate vs + reference on 1.0.13, gate mean KL ≤ 0.03 / top-1 ≥ 95 %). +- `frank-primary` showing `failed` during the campaign is the 143-on-stop + signature of the foreground podman unit — cosmetic, not a crash. + +## 2026-10-07 morning + +- **KLD reference COMPLETE** on 1.0.13: `/home/dank/corpora/kld-ref` 24 GiB, + `radiance-kld-1`, `rows: 50235`, 197 docs, top-16 log-probs (f16), + corpus fnv1a64 `a9d62b33c18b3d9c`. Record took ~4 h at the 32k step. + The wrapper died rc=127 AFTER the engine exited clean: the mid-run scp of + commit 45de2fd shifted bash's lazy read offset — script-swap law, third + fleet occurrence. Resume path added: `SKIP_KLD=1` + `rows: 50235` receipt + gate (cf6c62d). **Never scp a script a running process owns.** +- Calibration chain3 live on 1.2.0 (big-step shape MNBT 65536/seqs 8): push + started, requests landing at 7–8k tokens. + +## Campaign order after the chain + +1. derive protected experts from Swift's own grams; apply to recipe. +2. Delete bf16 `.rad` (335 GiB) to `.AGENTTRASH` staging (Freed space gates the convert). +3. `convert-swift15-w4dn8.sh` (~2 h, CPU-only, prod can stay up). +4. rad-tune declare TP=2 spec=2; vram-pair boot arms; bench (gate decode ≥ 30.8 t/s). +5. `kld-gate-swift15.sh` (fail → escalate gate_up w4→w5nl, pre-authorized). +6. Cutover: docs, tests + e2e, deploy, probes. Throughput fallback: w4 down tier (Q35). + +Disk ledger at KLD-record time: 104 GiB free → KLD ref −50 → Grams −20 → +bf16 delete +335 → convert +132 written. diff --git a/serverAppConfigs-docs/index.md b/serverAppConfigs-docs/index.md index 8076442..aa1369f 100644 --- a/serverAppConfigs-docs/index.md +++ b/serverAppConfigs-docs/index.md @@ -182,6 +182,11 @@ The LLM inference fleet on the two hosts. Full index: ### Deployment and Topology - [[deployment]] — unified deploy and update process for both servers +- [[frank-primary-radiance-swift15-w4dn8]] — STAGED frank-primary runbook: + Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance + 1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s; + activates by flipping `MODEL_FILE` in `scripts/frank-primary.env`, rollback + = revert that line (the FP8-native `.rad` stays on disk) - [[frank-primary-radiance-qwen38-native-fp8]] — the ACTIVE frank-primary runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned 287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL @@ -191,6 +196,9 @@ The LLM inference fleet on the two hosts. Full index: 7.32 KiB/token, MTP depth 3, 2 × 200,000 first boot; rollback is the FreeToken one-card era (tag `frank-primary-freetoken-pr587-20261005`), then the tcclaviger shape-G era (tag `frank-primary-vllm-shapeG-20261003`) +- [[swift15-campaign-worklog]] — ACTIVE campaign log: the Swift1.5 → + w4dn8 radiance convert and frank-primary swap in flight (branch + `swift15-radiance-w4dn8`); the cutover runbook replaces it at completion. - [[frank-primary-freetoken-pr587]] — RETIRED 2026-10-05 (superseded the same day by the radiance cutover above; kept as the rollback runbook): FreeToken built at PR #587 head `77a7201` on the host into diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index 6ed349e..f06208b 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -1,5 +1,47 @@ # Bundle Change Log +## 2026-10-08 +- **frank-primary (Dankserver): Swift1.5-Qwen3.8-Flash-Next GPTQ w4 — + the radiance quantization campaign.** ukisai's BF16 checkpoint (335.3 + GiB, 397/397 sha256-verified) → `qwen4exp-swift15-w4dn8-i8-hc8m.rad` + (~114 GiB): all 49,152 experts GPTQ w4 (act-order, damp 0.05) calibrated + on 10,000,000 live-model tokens, 24 whole bf16-protected experts per + layer derived from the calibration grams' energy share, int8 KV, fp8 HC + mixtures. Engine pins: **1.0.13** for serve/convert/KLD, **1.2.0** only + for calibration (the sole pin exposing `RADIANCE_CALIB_TOKENS`; the + RADGRAM2 gram format is byte-identical across pins, so grams cross the + boundary by design). Chain scripts in `scripts/`: + `make-calib-corpus.py`, `run-swift15-bf16.sh` (bf16 reference: KLD record + + calib tap), `calib-push.py`, `derive-protected-experts.py`, + `apply-protected-cast.sh`, `convert-swift15-w4dn8.sh`, + `kld-gate-swift15.sh`, `fleet-probes-swift15.sh`. +- **Engine laws learned (measured, not inferred).** (1) The qwen4exp_fp8 + plugin declares **one encoding per expert table**: a recipe may NOT split + `ffn_gate_up_exps` (w4) from `ffn_down_exps` (block-fp8) — the serve + kernel is a single w4nl64a8h table for the whole expert set; the only + legal beside-table content is whole bf16 experts (the PROTECTED section). + (2) w4 expert tables stream from the file tier **in place only at + MNBT ≤ 2048**; at MNBT 32768 the engine demands a widened form and the + disk tier refuses (`DID NOT FIT … stored form is not the container's`). + (3) The calibration tap lives in the architecture plugin + (`/opt/radiance/share/radiance/architectures/qwen4exp_fp8.so`), not the + binary — `grep -ac` the `.so` before declaring a flag missing. + (4) Disk-tier boots need ~9.6 GiB of weight buffers + arena charged to + the VRAM weights slab; explicit `--vram-weights-mib`/`--vram-kv-mib` + pairs replaced the fp8-era EVC ratio. +- **Throughput gate PASSED with 4.5× margin** (campaign boot, TP=2, int8 + KV, MTP 3): decode **136.0 t/s median** (122–173 across probes) at + 44.5k-token depth vs the 30.8 t/s bar; MTP acceptance 0.725; warm TTFT + 9.45 s; prefill 6.4k t/s. The fp8-native control on 1.0.13 measured + 30.08 t/s decode, and the 1.2.0 engine re-baseline was REJECTED + (29.21 t/s, acceptance 0.7123 < 0.7549) — serve stays on 1.0.13. +- **Deep-output audit:** a 44.5k-token verbatim-recall probe enumerated all + 48 planted documents in order — the DDR5-corruption signature is absent. +- Fleet probes (`scripts/fleet-probes-swift15.sh`): tools, reasoning + levels x4, reasoning_content, response_format, vision — all PASS. + KLD gate vs the BF16 reference: recorded on 1.0.13 (50,235 positions, + 197 prompts), scored on the same pin at the w4-legal shape. + Verdict: see runbook [[frank-primary-radiance-swift15-w4dn8]]. ## 2026-10-06 - **frank-secondary (Dankclaw): ubatch 2048 — the second rung (llama.cpp, diff --git a/tests/frank_primary_unit.bats b/tests/frank_primary_unit.bats index 61a09ed..d3d255e 100755 --- a/tests/frank_primary_unit.bats +++ b/tests/frank_primary_unit.bats @@ -24,8 +24,8 @@ UNIT="$REPO_ROOT/scripts/frank-primary.service" SERVE="$REPO_ROOT/scripts/serve-frank-primary.sh" ENV_FILE="$REPO_ROOT/scripts/frank-primary.env" REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh" -DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh" -BUILD="$REPO_ROOT/scripts/build-frank-primary-rad.sh" +DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary-swift15.sh" +BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh" HFGET="$REPO_ROOT/scripts/hfget.sh" TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja" @@ -118,19 +118,21 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja" grep -q '^FRANK_PRIMARY_MAX_MODEL_LEN=200000$' "$ENV_FILE" ml=$(awk -F= '/^FRANK_PRIMARY_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE") [ "$ml" -ge 177356 ] - # First-boot conservative shape (Frank's call): 2 decode slots. The - # fused TP all-reduce is declared over seqs x (1 + spec) rows = 8. - grep -q '^FRANK_PRIMARY_MAX_SEQS=2$' "$ENV_FILE" + # Decode-sweep shape (2026-10-06): 3 slots, spec depth 2 — the measured + # optimum (33.06 t/s). The fused TP all-reduce is declared over + # seqs x (1 + spec) rows = 9. + grep -q '^FRANK_PRIMARY_MAX_SEQS=3$' "$ENV_FILE" grep -q '^FRANK_PRIMARY_TP=2$' "$ENV_FILE" grep -q '^FRANK_PRIMARY_TP_WIRE=wht6$' "$ENV_FILE" - grep -q '^FRANK_PRIMARY_SPEC_TOKENS=3$' "$ENV_FILE" + grep -q '^FRANK_PRIMARY_SPEC_TOKENS=2$' "$ENV_FILE" # Native-precision economics: the expert pool is per rank (56.4 GiB # slice per rank at TP2), the PLE ngram table is pinned in host RAM per # Frank's law, KV is fp8, the ratio is the publisher's production # value. grep -q '^FRANK_PRIMARY_PLACEMENT=expert_tiered$' "$ENV_FILE" grep -q '^FRANK_PRIMARY_HOST_POOL_MIB=40960$' "$ENV_FILE" - grep -q '^FRANK_PRIMARY_EXPERT_VS_CACHE=0.82$' "$ENV_FILE" + grep -q '^FRANK_PRIMARY_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE" + grep -q '^FRANK_PRIMARY_VRAM_KV_MIB=[1-9]' "$ENV_FILE" grep -q '^FRANK_PRIMARY_KV_DTYPE=fp8$' "$ENV_FILE" grep -q '^FRANK_PRIMARY_NGRAM_PLACEMENT=ram$' "$ENV_FILE" # Prefix cache tiers (publisher pattern; disk tier sized for 2 slots). @@ -162,7 +164,8 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja" grep -q -- '--max-model-len "$FRANK_PRIMARY_MAX_MODEL_LEN"' "$SERVE" grep -q -- '--placement "$FRANK_PRIMARY_PLACEMENT"' "$SERVE" grep -q -- '--host-pool-mib "$FRANK_PRIMARY_HOST_POOL_MIB"' "$SERVE" - grep -q -- '--expert-vs-cache-ratio "$FRANK_PRIMARY_EXPERT_VS_CACHE"' "$SERVE" + grep -q -- '--vram-weights-mib "$FRANK_PRIMARY_VRAM_WEIGHTS_MIB"' "$SERVE" + grep -q -- '--vram-kv-mib "$FRANK_PRIMARY_VRAM_KV_MIB"' "$SERVE" grep -q -- '--kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE"' "$SERVE" grep -q -- '--prefix-cache-host-mib "$FRANK_PRIMARY_PREFIX_HOST_MIB"' "$SERVE" grep -q -- '--prefix-cache-disk-mib "$FRANK_PRIMARY_PREFIX_DISK_MIB"' "$SERVE" @@ -236,18 +239,17 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja" } @test "the model id matches the payload's build and fetch scripts" { - # The payload dir/file are the build script's output layout; the fetch - # script must name the repo the build stages from. + # The payload dir/file are the convert script's output layout; the fetch + # script must name the repo the convert stages from. (FP8-native era, + # the rollback payload, keeps its own trio: build-frank-primary-rad.sh + + # download-frank-primary.sh.) grep -q '^FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE" - grep -q '^FRANK_PRIMARY_MODEL_FILE=qwen4exp-fp8-native.rad$' "$ENV_FILE" - grep -q 'qwen4exp-fp8-native.rad' "$BUILD" - grep -q 'Qwen/Qwen3.8-Flash-Next-FP8' "$DOWNLOAD" + grep -q '^FRANK_PRIMARY_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad$' "$ENV_FILE" + grep -q 'qwen4exp-swift15-w4dn8-i8-hc8m.rad' "$BUILD" + grep -q 'ukisai/Swift1.5-Qwen3.8-Flash-Next' "$DOWNLOAD" grep -q 'hfget.sh' "$DOWNLOAD" # The repo is UNGATED — no token may be required on the fetch path. ! grep -q 'HF_TOKEN' "$DOWNLOAD" - # The staging step drops quantization_config (the descriptor fix) — - # document it where it happens. - grep -q 'quantization_config' "$BUILD" } @test "no bare HIP-reserved env var is defined" {