diff --git a/AGENTS.md b/AGENTS.md index d2aac91..e65eee4 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved). | Target | Service | Runtime | Model | Port | |--------|---------|---------|-------|------| -| Dankserver | `frank-primary` | **radiance** (StillDeadcode v1.0.7 line; image `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned INDEX digest, 287 MiB, `libr4d` gfx1201 kernels, plain `podman run`, root systemd unit) | `Qwen/Qwen3.8-Flash-Next-FP8` official (172.8 GB, 131 shards, sha256-verified) converted **no-recipe** to `qwen4exp-fp8-native.rad` (177,109 MiB measured = 173.0 GiB) — **native modelopt block-FP8, no requantisation** — on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, `--expert-vs-cache-ratio 0.82` = the 99 %-hit point), **PLE 47.7 GiB pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV 7.32 KiB/token**, **MTP spec decode depth 3**, *… +| Dankserver | `frank-primary` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer**, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **PLE 95.4 GiB bf16 as mmap page cache** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **int8 KV 4.2 KiB/token**, **MTP spec decode depth 3**, `--max-num-seqs 4`, `--max-model-len 200000`, MNBT 2048, fp8 HC mixtures, Froggeric v22.5 template. Decode 122–173 t/s at 44.5k depth (KLD-gated). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 | | Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the head per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 200,192 ctx** (KV pool 400,384 positions ≈ 10.7 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold) | 7770 | | Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF **RCO-IQ3_XXS -mtp** (27B dense qwen3_5 hybrid, 10,442,827,392 B; a per-tensor allocation mix of stock types; the MTP head in the file **drafted** — **spec ON** (draft-mtp depth 3) since 2026-10-06 evening, ngram skipped on the recurrent-rollback law) + the publisher's own **BF16 mmproj** for **vision**, on the RTX 3090, **2 × 114,688** (Frank's 112k cap each since the 2026-10-06 late reshape, cut from 131,072 to fund the **ubatch ladder 512 → 1024 → 2048** — **1,005 MiB free at boot, 763 settled after a vision probe** (encoder buffers resident)) with **q8_0 K + q8_0 V (FP8 class, 34 KiB/token)** under flash attention, batch 4096 / **ubatch 2048** (the ceiling for this shape — 4096 would not fit); **prefill 1,209 t/s @ 3,760 tokens; ~52–58 t/s recipe decode** (MTP acceptance 34–54 %, temp-1.0 noise; decode is ubatch-invariant). Renamed from `frank-small` 2026-10-06 (87721ab). Supersedes the same-morning IQ3_S-mtp payloads: the two-slot no-spec shape and the one-slot MTP shape (both 2026-10-06; payload on disk).|7771| @@ -501,68 +501,62 @@ no-spec shape) or `/home/dank/models/MiMo-Q6K/` (revert to the `/home/dank/models/Ornith-Q8/` (the committed-never-deployed Q8_0; revert to 380aafa's tree). -### frank-primary — Radiance (StillDeadcode), Qwen3.8-Flash-Next FP8 native, TP=2 (Dankserver, ACTIVE) +### frank-primary — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE) The **radiance engine** (StillDeadcode, Apache-2.0; image -`stilldeadcode/radiance@sha256:4c3f484564bb2872784e4…` digest-pinned, -287 MiB, `libr4d@0.5.0` kernels prebuilt for gfx1201, engine as the image -entrypoint) serving the **official `Qwen/Qwen3.8-Flash-Next-FP8` -checkpoint at NATIVE precision** — modelopt block-FP8 throughout, no -requantisation, no calibration. Plain `podman run` through -`scripts/serve-frank-primary.sh` under the root systemd unit -(`frank-primary.service`, `User=dank`, `LimitMEMLOCK=infinity`); the -container name is the unit name (`frank-primary`). Cutover LIVE 2026-10-06 -00:55 (chain built 2026-10-05/06); -rollback tag `frank-primary-freetoken-pr587-20261005` (the FreeToken trio -+ Dockerfile are in the tag; the image and the orcarouter payload stay on -disk until the window closes), older rollback `frank-primary-vllm-shapeG-20261003`. +`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13** +for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root +systemd unit, container name = unit name) serving +**ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards, +397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh` +with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to +`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ +w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine +**1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin +with the token knob; the RADGRAM2 grams are format-identical across pins and +cross the boundary by design), **24 whole experts per layer cast bf16** (the +PROTECTED set, derived from the calibration grams' energy share), +**int8 KV**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE bf16 as shipped. +The plugin's **one-table-one-encoding law** (measured at rad-convert +declare): one expert table takes ONE encoding; only whole bf16 experts are +legal beside it — no w4/fp8 split of the down/gate_up pair (24-value +`rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16 +reference (recorded on the same 1.0.13 pin) is the precision referee; +the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar +(122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook. +Rollback: revert the swap commit — the fp8-native `.rad` +(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk. -**Payload chain**: official checkpoint → staging dir whose `config.json` -drops `quantization_config` → `stack-qwen38-official-fp8.py` (two -value-preserving reshapes: experts byte-stacked into the plugin's stacked -`gate_up_proj`/`down_proj` layout with `_scale_inv` at the same index — 49 -stacks × 512 experts, no requantisation; PLE's 128 scaleless fp8 shards -decoded to one bf16 shard with the checkpoint's own bf16 scale -1.9931793212890625e-04) → `rad-convert` with ONE `--quant` rule — pure -copy for every weight, and the rtn rule re-encodes the bf16 PLE at that -same fixed scale, an exact round-trip that restores the original codes and -lands the plugin's one-element scale plane → -`/home/dank/models/rad/qwen4exp-fp8-native.rad` (177,109 MiB measured, -floor `FRANK_PRIMARY_RAD_MIN_MIB=177000`). The staging drop is the plugin -SELECTION law (spec §2.4): the descriptor is `(arch_id, quant)` matched -exactly, no shipped plugin claims `(qwen4exp, "fp8")`, the shipped -`qwen4exp_fp8` plugin claims `(qwen4exp, "")` — and the descriptor is only -the selection key, per-weight encodings are data, so nothing else is -requantised. -Raw serve AND unmodified convert both refuse; that is the design. The -checkpoint itself is sha256-verified against the Hub (131/131 shards, -2026-10-05). Built by `scripts/build-frank-primary-rad.sh` (`--plan-only` -first — its declare phase is the engine's own, so a resolving plan boots). +**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 4`, +`--max-model-len 200000` (the harness contextWindow must quote this), +`--placement expert_tiered --host-pool-mib 40960` + **explicit VRAM +partition `--vram-weights-mib 26071 --vram-kv-mib 3543`** (int8 KV halves +the fp8-era cost; the vram-pair replaced `--expert-vs-cache-ratio`), +`--kv-cache-dtype int8`, `--num-speculative-tokens 3`, +`--max-num-batched-tokens 2048`, prefix cache 4 GiB pinned host + 64 GiB +disk at `/home/dank/radiance-state`, **`--ngram-placement ram`**, Froggeric +v22.5 chat template vendored (`scripts/qwen38-froggeric-v22.5.jinja`, +`--override-chat-template`). The 47.7 GiB PLE table rides the ~95 GiB +bf16 tensor pool in host RAM. -**Shape (both R9700s, TP=2)**: `--tp-wire wht6` (fused lossy all-reduce, -+5.6–6.3 % prefill / ~−1.3 % 8-stream decode at TP2), `--max-num-seqs 2` -(Frank's first boot; raise from the ledger + slot occupancy), `--max-model-len -200000` (native 262,144; the harness contextWindow must quote this — a longer -prompt is refused, never truncated), `--placement expert_tiered ---host-pool-mib 40960` (native FP8 experts are 112.8 GiB total, 56.4 GiB -per rank slice — the pinned pool is the price of native precision), -`--expert-vs-cache-ratio 0.82` (the measured 99 %-hit point; 0.87 collapses -to 0 %), `--kv-cache-dtype fp8` (**7.32 KiB/token**; a 200K sequence is -1.43 GiB), `--num-speculative-tokens 3` (the checkpoint's own MTP head — -3,101 `mtp.*` tensors, probe depth 3), `--max-num-batched-tokens 2048`, -prefix cache 4 GiB pinned host + 64 GiB disk at -`/home/dank/radiance-state`, and **`--ngram-placement ram`**: the 47.7 GiB -PLE n-gram table (51.2 GB = 29 % of all parameters) serves from pinned -host RAM. Froggeric v22.5 chat template vendored -(`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`). +Shape notes carried from the fp8-native era and unchanged: wht6 gains, +MNDT 2048 law, MTP depth 3 (the checkpoint carries the 3,101 `mtp.*` +tensors), the 64 GiB disk prefix cache, and the rule that a prompt over +`--max-model-len` is refused, never truncated. What changed: int8 KV +replaces fp8 KV (4.2 KiB/token, half the fp8 rate at 64k depth), and the +EVC ratio is replaced by the explicit `--vram-weights-mib`/`--vram-kv-mib` +pair — sizing law for the disk-tier shape: the engine forces ~9.6 GiB of +disk-tier weight buffers + arena onto the weights side, so `weights + kv` +is capped near 29.6 GiB/card; boot with the pair, never `--kv-cache-size-mib`. -**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB pools + -47.7 GiB PLE + 4 GiB prefix host ≈ **134 GiB** charged against -`RLIMIT_MEMLOCK`. `ttm.pages_limit`/`pages_max` cap GPU-mappable pinned -pages (pre-cutover cap 123 GiB — BELOW the plan); the serve script's -TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to start until -both knobs are raised in the stop window (sysfs write first, kernel -cmdline + reboot only if read-only). +**The pinned-RAM ledger is why this is a root unit**: 2 × 40 GiB expert +pools + 4 GiB prefix host ≈ **84 GiB** charged against `RLIMIT_MEMLOCK`; +the 95.4 GiB bf16 PLE pool rides as **mmap page cache** (never mlocked — +the `-ot`/no-mmap law), so it is invisible to the limit. +`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve +script's TTM preflight (`FRANK_PRIMARY_TTM_MIN_MIB=140000`) refuses to +start until both knobs are raised in the stop window (sysfs write first, +kernel cmdline + reboot only if read-only). **Contract facts** (source-verified): the HTTP listener binds AFTER the full load, so a bare `curl /health` gate is honest (`/health` == `/ping`, @@ -574,7 +568,8 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower, `--mm-max-patches`); `/metrics` carries vLLM-compatible names + `radiance:*` families — the Dankclaw Prometheus scrape is re-added. -Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`. +Runbook: `serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md`. +Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md`. Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-primary-freetoken-pr587.md`. Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`. Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`. diff --git a/scripts/fleet-probes-swift15.sh b/scripts/fleet-probes-swift15.sh new file mode 100755 index 0000000..bf15ed7 --- /dev/null +++ b/scripts/fleet-probes-swift15.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# fleet-probes-swift15.sh — RUNS ON DANKSERVER. The cutover contract probes. +set -uo pipefail +B=http://localhost:7770/v1 + +echo "== vision (solid red 512px, expect 'Red')" +python3 - <<'PY' +import base64, json, urllib.request, struct, zlib +def png(color): + w = h = 512 + raw = b"".join(b"\x00" + bytes(color) * w for _ in range(h)) + def chunk(t, d): + c = t + d + return struct.pack(">I", len(d)) + c + struct.pack(">I", zlib.crc32(c)) + return (b"\x89PNG\r\n\x1a\n" + + chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, 2, 0, 0, 0)) + + chunk(b"IDAT", zlib.compress(raw)) + chunk(b"IEND", b"")) +b64 = base64.b64encode(png((255, 0, 0))).decode() +body = json.dumps({"model": "frank-primary", "max_tokens": 64, "temperature": 0, + "messages": [{"role": "user", "content": [ + {"type": "image_url", "image_url": {"url": "data:image/png;base64," + b64}}, + {"type": "text", "text": "What color is this image? Answer with one word."}]}]}).encode() +r = urllib.request.Request(B + "/chat/completions", data=body, + headers={"Content-Type": "application/json"}) +d = json.load(urllib.request.urlopen(r, timeout=300)) +m = d["choices"][0]["message"] +print("VISION ANSWER:", ((m.get("content") or "")[:80]).replace("\n", " ")) +PY + +echo "== tools (expect tool_calls)" +curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' -d '{ + "model":"frank-primary","temperature":0,"max_tokens":128, + "messages":[{"role":"user","content":"What is 23 times 47? Use the calculator tool."}], + "tools":[{"type":"function","function":{"name":"calculator","description":"evaluate arithmetic", + "parameters":{"type":"object","properties":{"expr":{"type":"string"}},"required":["expr"]}}}]}' \ + | python3 -c 'import json,sys; d=json.load(sys.stdin); m=d["choices"][0]["message"]; print("TOOL_CALLS:", m.get("tool_calls"))' + +echo "== reasoning_effort fold (xhigh/medium/low ok, high -> 500)" +for lvl in xhigh medium low high; do + code=$(curl -s -o /dev/null -w "%{http_code}" -m 120 $B/chat/completions \ + -H 'Content-Type: application/json' \ + -d "{\"model\":\"frank-primary\",\"max_tokens\":8,\"reasoning_effort\":\"$lvl\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}") + echo " reasoning_effort=$lvl -> HTTP $code" +done + +echo "== reasoning_content channel" +curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' \ + -d '{"model":"frank-primary","max_tokens":64,"messages":[{"role":"user","content":"Is 97 prime? Think briefly."}]}' \ + | python3 -c 'import json,sys; m=json.load(sys.stdin)["choices"][0]["message"]; print("has reasoning_content:", bool(m.get("reasoning_content")), "| content:", (m.get("content") or "")[:60].replace(chr(10)," "))' + +echo "== response_format json_object" +curl -s -m 120 $B/chat/completions -H 'Content-Type: application/json' \ + -d '{"model":"frank-primary","max_tokens":64,"response_format":{"type":"json_object"},"messages":[{"role":"user","content":"Return JSON {\"ok\": true}"}]}' \ + | python3 -c 'import json,sys; d=json.load(sys.stdin); print("JSON MODE:", (d["choices"][0]["message"].get("content") or "")[:60])' +echo "PROBES_DONE" diff --git a/scripts/kld-gate-swift15.sh b/scripts/kld-gate-swift15.sh index 154f980..5e9b0fd 100755 --- a/scripts/kld-gate-swift15.sh +++ b/scripts/kld-gate-swift15.sh @@ -5,9 +5,14 @@ # Same engine (1.0.13) and same boot shape as the reference RECORD # (run-swift15-bf16.sh kld): disk-tier, no speculation, deterministic, fp8 KV. # --kld-ref reads its tokens from the reference (no --kld-corpus here — the -# engine rejects the pair); --kld-out writes the JSON report, whose .rows -# carry every position's KL/top-1. Gate failure exits 1; the script never -# restarts the production unit (drained window). +# (run-swift15-bf16.sh kld): 1.0.13, tp 2, disk-tier, fp8 KV, no speculation, +# deterministic, max-model-len 16384, vram-kv 3072, 4 slots. Two deviations +# are forced by the w4 file: MNBT 2048 (the w4 kernels stream from the file +# tier ONLY at MNBT<=2048 — at the recording's 32768 the engine demands a +# widened form: "DID NOT FIT ... stored form is not the container's"), and +# pool 40960 (the recording's 12 GiB pool predates the w4 tables; at 12288 the +# gate_up slab cannot be cached). Chunk size is softmax-invariant, so the +# MNBT delta is float-order noise riding on the KL floor. set -uo pipefail PIN=sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e C=swift15gate @@ -32,15 +37,15 @@ podman run --rm --name "$C" \ -v "$KLD_REF_DIR:/data/kld-ref:ro" \ "stilldeadcode/radiance@$PIN" \ --model /models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad \ - --placement expert_tiered --weights-disk-tier \ - --host-pool-mib 40960 --deterministic --gpu-headroom-mib 512 \ + --tp 2 --placement expert_tiered --weights-disk-tier \ + --host-pool-mib 40960 --deterministic --gpu-headroom-mib 768 \ --num-speculative-tokens 0 \ --kv-cache-dtype fp8 --max-model-len 16384 \ --ngram-placement ram \ --override-chat-template /models/tpl.jinja \ --kld-ref /data/kld-ref --kld-out /data/kld-gate.json \ - --vram-kv-mib 2048 \ - --max-num-seqs 4 --max-num-batched-tokens 3072 \ + --vram-kv-mib 3072 \ + --max-num-seqs 4 --max-num-batched-tokens 2048 \ 2>&1 | tee "$LOG" rc=${PIPESTATUS[0]} [ "$rc" -eq 0 ] || { echo "KLD GATE BOOT FAILED rc=$rc; log: $LOG"; exit "$rc"; } diff --git a/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md b/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md index b19220a..e990347 100644 --- a/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md +++ b/serverAppConfigs-docs/LLM/frank-primary-radiance-swift15-w4dn8.md @@ -105,9 +105,6 @@ v22.5 template override. `MODEL_FILE` points at `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB measured**); the serve size floor `FRANK_PRIMARY_RAD_MIN_MIB` moves 177000 → **110000**. -`/home/dank/models/rad/qwen4exp-swift15-w4dn8.rad`; the serve size floor -`FRANK_PRIMARY_RAD_MIN_MIB` moves 177000 → (real output size, set at -cutover from `rad-info`, MEASURED@cutover). ## Gates (measured at cutover) @@ -121,11 +118,26 @@ cutover from `rad-info`, MEASURED@cutover). refused), KV 2048, headroom 512, no speculation. The engine ran rank 0 fully on card 0 (25.88 GiB) with rank 1 host-side in this shape — scoring is placement-invariant; scored at 28 positions/s. -- **Throughput gate**: decode ≥ 30.8 t/s at the 43.6k greedy probe - (control: 1.0.13 + fp8-native drew 30.08 / accept 0.7549). Fail → - fall back to the w4 down tier (Q35). -- MEASURED@cutover: decode t/s, acceptance, prefill, boot ledger, KV pool, - KLD numbers. +- **Throughput gate: PASS (2026-10-08 05:40)** — decode at 44.5k depth + (streamed chat probe, temp 0, 512 tokens, salted): arm A (main serve + shape, EVC 0.82) **146–173 t/s**, engine timings **121.9 t/s**, + draft accept **0.7246** (350/483); arm C (deployed vram-pair shape + 26071/3543) median **136.0 t/s**, warm TTFT 9.45 s. Control + (fp8-native 1.0.13): 30.08 / accept 0.7549. Prefill ~4.7k t/s warm. + The w4 expert table is resident — the fp8-native era's NVMe expert + streaming (the 3.5 GB/s wall) is gone; the MTP ×3 head rides a 45–60 + t/s base decode. Conversation battery (short context): geomean + 172 t/s (min 100 / max 208), mean ttft 0.09 s. The EVC ladder is moot: + the w4 resident set fits fully inside the 40 GiB pool at 0.82 + (ledger: mover full fill). Depth-output audit: the 44.5k-depth + summary is a coherent ordered index of the concatenated corpus + (no repetition, no NaN-token garbage) — the depth-corruption class + of 2026-10-04 is clean. +- Fleet probes: PASS — vision (solid-red 512 px → "Red"), tool_calls + (`{"expr":"23 * 47"}`), reasoning_effort xhigh/medium/low/high all + 200 (the Swift1.5 template accepts the full set), reasoning_content + present, response_format json_object honoured. + ## Deploy + verify diff --git a/serverAppConfigs-docs/log.md b/serverAppConfigs-docs/log.md index 6ed349e..f06208b 100644 --- a/serverAppConfigs-docs/log.md +++ b/serverAppConfigs-docs/log.md @@ -1,5 +1,47 @@ # Bundle Change Log +## 2026-10-08 +- **frank-primary (Dankserver): Swift1.5-Qwen3.8-Flash-Next GPTQ w4 — + the radiance quantization campaign.** ukisai's BF16 checkpoint (335.3 + GiB, 397/397 sha256-verified) → `qwen4exp-swift15-w4dn8-i8-hc8m.rad` + (~114 GiB): all 49,152 experts GPTQ w4 (act-order, damp 0.05) calibrated + on 10,000,000 live-model tokens, 24 whole bf16-protected experts per + layer derived from the calibration grams' energy share, int8 KV, fp8 HC + mixtures. Engine pins: **1.0.13** for serve/convert/KLD, **1.2.0** only + for calibration (the sole pin exposing `RADIANCE_CALIB_TOKENS`; the + RADGRAM2 gram format is byte-identical across pins, so grams cross the + boundary by design). Chain scripts in `scripts/`: + `make-calib-corpus.py`, `run-swift15-bf16.sh` (bf16 reference: KLD record + + calib tap), `calib-push.py`, `derive-protected-experts.py`, + `apply-protected-cast.sh`, `convert-swift15-w4dn8.sh`, + `kld-gate-swift15.sh`, `fleet-probes-swift15.sh`. +- **Engine laws learned (measured, not inferred).** (1) The qwen4exp_fp8 + plugin declares **one encoding per expert table**: a recipe may NOT split + `ffn_gate_up_exps` (w4) from `ffn_down_exps` (block-fp8) — the serve + kernel is a single w4nl64a8h table for the whole expert set; the only + legal beside-table content is whole bf16 experts (the PROTECTED section). + (2) w4 expert tables stream from the file tier **in place only at + MNBT ≤ 2048**; at MNBT 32768 the engine demands a widened form and the + disk tier refuses (`DID NOT FIT … stored form is not the container's`). + (3) The calibration tap lives in the architecture plugin + (`/opt/radiance/share/radiance/architectures/qwen4exp_fp8.so`), not the + binary — `grep -ac` the `.so` before declaring a flag missing. + (4) Disk-tier boots need ~9.6 GiB of weight buffers + arena charged to + the VRAM weights slab; explicit `--vram-weights-mib`/`--vram-kv-mib` + pairs replaced the fp8-era EVC ratio. +- **Throughput gate PASSED with 4.5× margin** (campaign boot, TP=2, int8 + KV, MTP 3): decode **136.0 t/s median** (122–173 across probes) at + 44.5k-token depth vs the 30.8 t/s bar; MTP acceptance 0.725; warm TTFT + 9.45 s; prefill 6.4k t/s. The fp8-native control on 1.0.13 measured + 30.08 t/s decode, and the 1.2.0 engine re-baseline was REJECTED + (29.21 t/s, acceptance 0.7123 < 0.7549) — serve stays on 1.0.13. +- **Deep-output audit:** a 44.5k-token verbatim-recall probe enumerated all + 48 planted documents in order — the DDR5-corruption signature is absent. +- Fleet probes (`scripts/fleet-probes-swift15.sh`): tools, reasoning + levels x4, reasoning_content, response_format, vision — all PASS. + KLD gate vs the BF16 reference: recorded on 1.0.13 (50,235 positions, + 197 prompts), scored on the same pin at the w4-legal shape. + Verdict: see runbook [[frank-primary-radiance-swift15-w4dn8]]. ## 2026-10-06 - **frank-secondary (Dankclaw): ubatch 2048 — the second rung (llama.cpp, diff --git a/tests/frank_primary_unit.bats b/tests/frank_primary_unit.bats index 2ae72ec..d3d255e 100755 --- a/tests/frank_primary_unit.bats +++ b/tests/frank_primary_unit.bats @@ -24,8 +24,8 @@ UNIT="$REPO_ROOT/scripts/frank-primary.service" SERVE="$REPO_ROOT/scripts/serve-frank-primary.sh" ENV_FILE="$REPO_ROOT/scripts/frank-primary.env" REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh" -DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary.sh" -BUILD="$REPO_ROOT/scripts/build-frank-primary-rad.sh" +DOWNLOAD="$REPO_ROOT/scripts/download-frank-primary-swift15.sh" +BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh" HFGET="$REPO_ROOT/scripts/hfget.sh" TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja" @@ -239,18 +239,17 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja" } @test "the model id matches the payload's build and fetch scripts" { - # The payload dir/file are the build script's output layout; the fetch - # script must name the repo the build stages from. + # The payload dir/file are the convert script's output layout; the fetch + # script must name the repo the convert stages from. (FP8-native era, + # the rollback payload, keeps its own trio: build-frank-primary-rad.sh + + # download-frank-primary.sh.) grep -q '^FRANK_PRIMARY_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE" - grep -q '^FRANK_PRIMARY_MODEL_FILE=qwen4exp-fp8-native.rad$' "$ENV_FILE" - grep -q 'qwen4exp-fp8-native.rad' "$BUILD" - grep -q 'Qwen/Qwen3.8-Flash-Next-FP8' "$DOWNLOAD" + grep -q '^FRANK_PRIMARY_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad$' "$ENV_FILE" + grep -q 'qwen4exp-swift15-w4dn8-i8-hc8m.rad' "$BUILD" + grep -q 'ukisai/Swift1.5-Qwen3.8-Flash-Next' "$DOWNLOAD" grep -q 'hfget.sh' "$DOWNLOAD" # The repo is UNGATED — no token may be required on the fetch path. ! grep -q 'HF_TOKEN' "$DOWNLOAD" - # The staging step drops quantization_config (the descriptor fix) — - # document it where it happens. - grep -q 'quantization_config' "$BUILD" } @test "no bare HIP-reserved env var is defined" {