From bae78e7dae6623ae3e67a315a33d959264eb18f2 Mon Sep 17 00:00:00 2001 From: Frank Date: Tue, 13 Oct 2026 19:28:25 -0700 Subject: [PATCH] docs(frank-archon): the ttm reset outage, and the 4 x 200,000 fp8 shape Runbook gains the 2026-10-13 section (symptom signature, the runtime-knob law, the shape arithmetic, the boot ledger that confirms it: 843,276 tokens = 4 x 200,000, 12 all-reduce rows, 5.87 GiB of KV lent to the slab). AGENTS.md gains the fleet-row update, the KV dtype law at four slots, the sysfs-raise lesson under Key Lessons, and the pinned-RAM paragraph now points at scripts/ttm-preflight.sh as the repair. LLM index entry + topology node match. --- AGENTS.md | 95 +++++++++---- .../frank-archon-radiance-swift15-w4dn8.md | 133 ++++++++++++++++++ serverAppConfigs-docs/LLM/index.md | 38 ++--- 3 files changed, 218 insertions(+), 48 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 4d08d4c..22c151a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved). | Target | Service | Runtime | Model | Port | |--------|---------|---------|-------|------| -| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 | +| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **44 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 22208 --vram-kv-mib 6912` = the 29,120 MiB/card ceiling), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (pool 6,912 MiB ≈ 843k tokens — the only dtype that funds four slots, since bf16 at 4 × 200,000 costs 11,679 MiB of pool and would cut the slab to 17,441), **MTP spec decode depth 2**, `--max-num-seqs 4` (the capacity contract is 4 × 200,000; 4 × (1+2) = 12 all-reduce rows, inside the 25-row wht6 exactness bound), `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback); 4-slot fp8-KV shape 2026-10-13, the 2-slot bf16 shape (22208/6912, `--max-num-seqs 2`) is the documented rollback. | 7770 | | Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770| | Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 | @@ -525,21 +525,26 @@ the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar Rollback: revert the swap commit — the fp8-native `.rad` (`qwen4exp-fp8-native.rad`) and every prior payload remain on disk. -**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 2`, +**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 4`, `--max-model-len 200000` (the harness contextWindow must quote this), `--placement expert_tiered --host-pool-mib 44928` + **explicit VRAM partition `--vram-weights-mib 22208 --vram-kv-mib 6912`** (the vram-pair replaced `--expert-vs-cache-ratio`; the two are ONE budget — `weights + kv ≤ 29,120 MiB`, from the measured card 31,856 minus 1,710 consumed outside both budgets minus 1,024 to keep free), `--kv-cache-dtype` -**`bf16` — unquantized KV since 2026-10-11**, `--num-speculative-tokens 2`, +**`fp8` — quantized KV, restored 2026-10-13 because it is the only dtype that +funds four slots at this window**, `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`, prefix cache 8 GiB pinned host + 64 GiB disk at `/home/dank/radiance-state`, **`--ngram-placement ram`**, Froggeric v22.5 chat template vendored (`scripts/qwen38-froggeric-v22.5.jinja`, -`--override-chat-template`). Boot ledger: KV pool 6.75 GiB = **462,752 bf16 -tokens, "backs 2 sequences all at 200000 tokens at once"** (the fp8 pool's -386,904 backed 1.93), weights slab filled to 147 KiB of slack, host pool -10,375 slots, cards settled 1.85 GiB free. +`--override-chat-template`). Boot ledger (2026-10-13): `kv cache 6.75 GiB … +backs 4 sequences all at 200000 tokens at once, which is the worst case` — +`kv_attn` page 26.00 KiB (fp8) at 843,276 tokens (the sizing law predicted +843,566), `kv_qsa_bkey` 3.25 KiB at the same token count, `kv_gdn_state` 54 MiB +× 8 slots (4 sequences × 2 copies) = 432 MiB; the sampler prints **12 rows** +(4 × (1+2), inside the 25-row `wht6` exactness bound); weights filled to +147 KiB of slack, cards settled ~2 GiB free, and the KV pool lends 5.87 GiB to +the expert slab (the fp8-era 3,543 MiB pool lent 2.69 GiB). Shape notes carried from the fp8-native era and unchanged: wht6 gains, MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk @@ -551,35 +556,52 @@ on the fp8 shape (512-token streamed probe). The campaign also measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth, accept 0.7246). -**The bf16 KV law (2026-10-11)**: `--kv-cache-dtype` takes ONLY `bf16|fp8` — -one dtype for the whole cache, so llama.cpp's exact-K/quantized-V pair (what -frank-zealot runs) has NO radiance equivalent; the rung under bf16 is fp8 on -both halves. Measured per rank from the two boot ledgers: `kv_attn` (13 -layers, 4-token blocks) pages 26.00 → 52.00 KiB, while `kv_qsa_bkey` (13 -layers) stays 3.25 KiB at BOTH dtypes (the QSA block key is an index, not -attention state) — so a token costs 7.3125 KiB at fp8 and 13.8125 KiB at -bf16: **1.89×, not 2×**. Pool sizing must also pay the dtype-independent terms -that live in the same pool (~110 MiB a sequence of recurrent state + 448 MiB -of prefix checkpoints): `(vram_kv − 110·seqs − 448) MiB / 13.8125 KiB` = -462,946 where the engine reported 462,752 (0.04 % page rounding). The cost is -paid twice on this engine, and half-refunded: idle KV VRAM is also the expert -cache's spillover, and the bigger pool *lends* 6.08 GiB to the slab where the -fp8 pool lent 2.69 GiB. `--max-num-seqs 3 → 2` came with the change (two slots -× the window is the capacity contract; the "seqs 2 cannot boot" law belongs to -the retired tcclaviger fork's torch.compile, not radiance) and it also drops -the fused all-reduce to 2 × 3 = 6 rows, under the 25-row bound where wht6 is -bit-exact. Disk-tier sizing law: the engine forces ~9.6 GiB of disk-tier weight -buffers + arena onto the weights side, so `weights + kv` is capped near -29.6 GiB/card; boot with the pair, never `--kv-cache-size-mib`. +**The KV dtype law (measured 2026-10-11, applied at 4 slots 2026-10-13)**: +`--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the whole cache, so +llama.cpp's exact-K/quantized-V pair (what frank-zealot runs) has NO radiance +equivalent; the rung under bf16 is fp8 on both halves. Measured per rank from +the two boot ledgers: `kv_attn` (13 layers, 4-token blocks) pages 26.00 → 52.00 +KiB, while `kv_qsa_bkey` (13 layers) stays 3.25 KiB at BOTH dtypes (the QSA +block key is an index, not attention state) — so a token costs 7.3125 KiB at fp8 +and 13.8125 KiB at bf16: **1.89×, not 2×**. Pool sizing must also pay the +dtype-independent terms that live in the same pool (~110 MiB a sequence of +recurrent state + 448 MiB of prefix checkpoints): +`(vram_kv − 110·seqs − 448) MiB / rate` — bf16 at 6,912 gives 462,946 where the +engine reported 462,752 (0.04 % page rounding); fp8 at 6,912 with four slots +gives 843,566, i.e. the deployed capacity contract 4 × 200,000. **That is why +the dtype is fp8: the pool must back `seqs × max-model-len` as a hard limit, and +bf16 at four slots costs 11,679 MiB — the slab would fall to 17,441 MiB (−33 %), +where fp8 leaves it at 22,208.** The cost is paid twice on this engine, and +half-refunded: idle KV VRAM is also the expert cache's spillover, and the bigger +pool *lends* 6.08 GiB to the slab where the 3,543 MiB fp8 pool lent 2.69 GiB. +`--max-num-seqs` rides 4 × (1 + spec) = 12 fused-all-reduce rows, under the +25-row bound where wht6 is bit-exact (the "seqs 2 cannot boot" law belongs to +the retired tcclaviger fork's torch.compile, not radiance). Disk-tier sizing +law: the engine forces ~9.6 GiB of disk-tier weight buffers + arena onto the +weights side, so `weights + kv` is capped near 29.6 GiB/card; boot with the +pair, never `--kv-cache-size-mib`. + +Shape notes carried from the fp8-native era and unchanged: wht6 gains, +MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk +prefix cache, and the rule that a prompt over `--max-model-len` is +refused, never truncated. What changed: the fp8-native era's NVMe expert +streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the +host pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s +on the fp8 shape (512-token streamed probe). The campaign also +measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth, +accept 0.7246). **The pinned-RAM ledger is why this is a root unit**: 2 × 44 GiB expert pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM the card reads) + 8 GiB prefix host ≈ **147 GiB** charged against `RLIMIT_MEMLOCK` — the TTM floor is set just above it. -`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve -script's TTM preflight (`FRANK_ARCHON_TTM_MIN_MIB=150000`, cap on this host -204,800) refuses to start until both knobs are raised in the stop window -(sysfs write first, kernel cmdline + reboot only if read-only). +`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages, and **the knob is +runtime state whose default is half of RAM**: `scripts/ttm-preflight.sh` (sourced +by the serve script) therefore RAISES it to `FRANK_ARCHON_TTM_TARGET_MIB=204800` +before refusing, because the 2026-10-12 power failure reset it to 125,988 MiB +and the preflight re-refused 6,919 times. A manual sysfs write in the stop +window is a convenience, not the repair; kernel cmdline persistence +(`ttm.pages_limit=52428800`) is the belt to it. **Contract facts** (source-verified): the HTTP listener binds AFTER the full load, so a bare `curl /health` gate is honest (`/health` == `/ping`, @@ -761,6 +783,17 @@ linked runbooks. the KV pool is worse than the OOM it prevents. - Host swap state is invisible to `deploy.sh` (no repo file) — check `swapon --show` before reasoning about headroom. +- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults + to half of RAM, so the manual raise to 200 GiB (2026-10-05) vanished with the + 2026-10-12 23:17 POWER FAILURE on dankserver: the cap fell to 125,988 MiB, + frank-archon's pinned-ledger preflight (floor 150,000 MiB) refused the start, + and `Restart=always` re-ran that refusal **6,919 times in ~20 h** — a unit in + `activating (auto-restart)` that exits 1 in 50 ms with no container and no + engine log is a PREFLIGHT refusal, not an engine fault. Repo law now: + `scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) raises the knob + to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only + after a write the kernel rejected. Never prescribe a manual sysfs write as the + recovery procedure for a service that can boot before a human reaches it. - **Silent RAM corruption outranks every software theory.** frank-archon's ~40k-token output corruption (2026-10-04/05, repetition/garbage in long agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed diff --git a/serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md b/serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md index 9797ad0..48c830f 100644 --- a/serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md +++ b/serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md @@ -253,6 +253,139 @@ from the scored configuration — bf16 can only reduce the attention-state error, and a re-score is a drained-window job if the gate agreement is ever needed in the record. +## 2026-10-13 — the power failure, the ttm reset, and 4 × 200,000 fp8 KV + +### The outage: a refusal, not a crash + +Dankserver lost power on 2026-10-12 ~23:17. The box came back; frank-archon +never did. The unit sat in `activating (auto-restart)`, exiting 1 about 50 ms +after starting, with one journal line on repeat: + +```text +ERROR: ttm pages cap is 125988 MiB, the pinned ledger needs >= 150000 +frank-archon: raise /sys/module/ttm/parameters/pages_limit … +``` + +**6,919 refusals in ~20 hours.** No container was created (`podman ps -a` showed +the name with a zero start time), so there was no engine, no port, and no engine +log — the signature of a preflight refusal, which every deployer/monitor view +outside the journal renders as an unexplained crash loop. + +The cause is a class, not an incident. The pinned-RAM ledger (~147 GiB: 2 × 44 GiB +expert pools + the 47.7 GiB PLE table + the 8 GiB prefix host tier) rides TTM's +shared pages, capped by `/sys/module/ttm/parameters/pages_limit`. **That knob is +runtime state, and its default is half of RAM** — so the manual raise of +2026-10-05 (52,428,800 pages = 200 GiB) died with the power failure and the cap +fell back to 32,253,039 pages = 125,988 MiB, 24 GiB below the floor the serve +script enforces. The fp8-native era had planned the persistent half of this +("add `ttm.pages_limit=52428800` to the kernel cmdline at the next scheduled +reboot", [[frank-archon-radiance-qwen38-native-fp8]]); it was never applied, and +a power failure is not scheduled. + +### The repair: the gate raises the knob itself + +`scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) reads the cap as +the MIN over the knobs the running kernel exposes, and when it is below +`FRANK_ARCHON_TTM_MIN_MIB` it writes `FRANK_ARCHON_TTM_TARGET_MIB` (204,800 MiB = +52,428,800 pages) into every writable knob, **re-reads**, and only then refuses. +The unit is root and the knob is a 0644 root-owned module parameter, so the write +is legal; a kernel that refuses the move produces the same named refusal as +before, and a reboot is now a boot rather than a visit to the machine. + +| Knob | Power-failure default | After the gate | +|---|---|---| +| `/sys/module/ttm/parameters/pages_limit` | 32,253,039 pages = 125,988 MiB | 52,428,800 pages = 204,800 MiB | + +Seam-tested, not inspection-tested: *the ttm gate raises a reset pages cap and +refuses when the write fails* and *the ttm cap counts only the knobs the kernel +exposes* drive the gate against a temp knob directory — a below-floor writable +knob ends at the target and the start passes; a read-only knob leaves the cap +untouched and the start refuses by name; an absent `pages_max` is skipped, never +read as zero (a zero reading would refuse a healthy host). + +### The shape: 4 slots × 200,000, fp8 KV + +Answers the open call of 2026-10-11 from the other side: Frank asked for **four** +concurrent 200,000-token conversations with quantized KV. + +fp8 is not a preference at four slots, it is the only dtype that affords them. +The pool must back `seqs × max-model-len` (the engine's hard limit), so the +fourth slot is bought in KV, out of the one per-card budget +(`weights + kv ≤ 29,120 MiB`): + +| dtype | pool for 4 × 200,000 | weights slab left | +|---|---|---| +| bf16 | 10,791 + 440 + 448 = 11,679 MiB | 17,441 MiB (−33 % vs the fp8-era 26,071) | +| **fp8 (deployed)** | 5,713 + 440 + 448 = 6,601 MiB | **22,208 MiB — unchanged** | + +So `--vram-weights-mib 22208` / `--vram-kv-mib 6912` stay exactly where the +2026-10-11 change put them (pair sum = the measured 29,120 ceiling), the +44,928 MiB/rank pinned host pool is untouched, and the RAM/TTM ledger does not +move. What the fp8 pool buys: `(6912 − 110×4 − 448) × 1024 / 7.3125` = +**843,566 predicted tokens** = four full-height conversations with 5.4 % slack +(the bf16 pool backed 462,752 = two; the 2026-10-08 fp8 pool backed 386,904 = +1.93). + +Terms that move with the dtype and the slot count: + +- **Per-slot recurrent state** is dtype-independent and lives inside the pool: + +110 MiB per slot, 440 MiB for four (already in the arithmetic above). +- **The fused TP all-reduce** is declared over `seqs × (1 + spec)` rows: 4 × 3 = + **12 rows**, still inside the 25-row bound at which the `wht6` wire is + bit-exact — so four slots cost nothing in wire exactness. +- **Prefix-cache entries shrink** with the attention plane (110.50 KiB at bf16 → + 58.50 KiB at fp8), so the 8 GiB host tier parks ~5.8 sessions of 200k where at + bf16 it parked ~2.9 — four live slots plus a parked set fit with no re-size. +- **`models.yml` is untouched**: `contextWindow` quotes the per-request window + (200,000), which did not change; only the concurrency did. + +Expectation carried from the 2026-10-11 A/B, whose arms bracket this shape +(same slab 22,208): prefill is set by the slab, not the dtype, so ~3.1k t/s +cold / ~3.3k warm; decode sits in the 94–118 t/s band with draft acceptance +~0.73–0.75. The 4,379 t/s fp8 prefill of the 2026-10-08 control belongs to the +26,071 MiB slab and does not come back by quantizing KV. + +### Boot ledger (2026-10-13 19:21, four slots, fp8) + +The sizing law's prediction, checked against the engine's own arithmetic: +predicted 843,566 tokens, ledger **843,276** (0.03 % block rounding). + +| Group | Page / state | Size | Tokens | +|---|---|---|---| +| `kv_attn` | 13 layers, 4-token blocks, **26.00 KiB** (fp8 page) | 5.23 GiB | **843,276** | +| `kv_qsa_bkey` | 13 layers, 4-token blocks, **3.25 KiB** (dtype-fixed) | 669.10 MiB | **843,276** | +| `kv_gdn_state` | 36 layers, 54.00 MiB, **8 slots** (4 × 2 copies) | 432.00 MiB | per-slot | +| `kv_gdn_conv` / `kv_ple_*` | 36 + 1 + 1 layers, 4 slots | 7.91 MiB | per-slot | + +`kv cache 6.75 GiB … backs 4 sequences all at 200000 tokens at once, which is +the worst case` — the capacity contract, in the engine's words. The rest of the +boot is the fp8-era picture: `vram_weights` 21.69 GiB filled to 147 KiB (rank 0) +/ 355 KiB (rank 1) of slack, `unclaimed` 2.03 GiB, cards settled at 1.99 / +2.00 GiB free, `mover: 4967 slab slots (64 shadow), 10375 pinned pool slots, +4.24 GiB static vram`, and the half-refund in plain sight: `4967 slots can be +lent out of the KV cache (5.87 GiB)` (the fp8-era 3,543 MiB pool lent 2.69 GiB). + +Two ledger lines that confirm the derived consequences: the sampler prints +**12 rows** (4 sequences × (1 + 2 speculative) — inside the 25-row bound where +`wht6` is bit-exact), and the prefix/tier arithmetic runs on the fp8 entry size +(58.50 KiB, half of the bf16 entry). + +### Recovery evidence + +```text +Oct 13 19:21:08 dankserver ttm-preflight.sh[1623778]: frank-archon: ttm pages cap + is 125988 MiB, below the pinned-ledger floor 150000 — raising the knob to + 204800 MiB (52428800 pages) +Oct 13 19:21:08 dankserver ttm-preflight.sh[1623778]: frank-archon: ttm pages cap + 204800 MiB clears the pinned-ledger floor 150000 +Oct 13 19:21:08 dankserver systemd[1]: Started frank-archon.service +``` + +`cat /sys/module/ttm/parameters/pages_limit` → 52,428,800 (204,800 MiB) after +the boot, i.e. the root `ExecStartPre` hook did the write the service process +(User=dank) cannot. + + ## Gates (measured at cutover) - **KLD gate: PASS (2026-10-08 04:12)** — `kld-gate-swift15.sh`, diff --git a/serverAppConfigs-docs/LLM/index.md b/serverAppConfigs-docs/LLM/index.md index bc1cccc..524c508 100644 --- a/serverAppConfigs-docs/LLM/index.md +++ b/serverAppConfigs-docs/LLM/index.md @@ -2,13 +2,13 @@ **Dankserver runs the radiance engine (StillDeadcode) on frank-archon** (digest-pinned `stilldeadcode/radiance` image, engine as entrypoint, -driving `podman run` directly, root systemd unit, :7770): the OFFICIAL -`Qwen/Qwen3.8-Flash-Next-FP8` checkpoint at NATIVE block-FP8 precision -(converted no-recipe to `qwen4exp-fp8-native.rad` — see the runbook for -the descriptor-staging law), on BOTH R9700s TP=2 (`--tp-wire wht6`), -experts in a 40 GiB/rank pinned host pool + VRAM cache (ratio 0.82), the -47.7 GiB PLE table pinned to host RAM, fp8 KV, MTP spec decode depth 3, -2 × 200,000 first boot. Rollback tags frank-archon-freetoken-pr587-20261005 +driving `podman run` directly, root systemd unit, :7770): +`ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4** +(`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s +TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache +(explicit `--vram-weights-mib 22208 --vram-kv-mib 6912`), the 47.7 GiB PLE table +pinned to host RAM, **fp8 KV at 4 × 200,000**, MTP spec decode depth 2, +froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005 and frank-archon-vllm-shapeG-20261003), and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2 (frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned @@ -36,10 +36,10 @@ graph LR gfx1201, 32 GiB each] B2[radiance StillDeadcode, digest-pinned image podman run, frank-archon, port 7770] - C[Qwen3.8-Flash-Next FP8 OFFICIAL, native block-FP8 -2x R9700 TP=2 wht6, experts 40 GiB/rank pinned pool + VRAM, -PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 3, -2 x 200000, froggeric v22.5] + C[Swift1.5-Qwen3.8-Flash-Next GPTQ w4 .rad +2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM, +PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2, +4 x 200000, froggeric v22.5] end subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04] @@ -155,12 +155,16 @@ frank-zealot, port 7771] - [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**: Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance 1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container); - the FP8-native payload remains the rollback. **2026-10-11: the KV cache is - UNQUANTIZED (bf16) at 2 x 200,000** — pool 3543 → 6912 MiB (386,904 → - 462,752 tokens), weights slab 26071 → 22208, host pool 44928, - `--max-num-seqs 2`. bf16 costs 1.89x fp8 per token (the QSA block key is - dtype-fixed, so it is not a clean 2x); measured price is prefill (-29%), - NOT decode, and spec-decode acceptance rose 0.661 → 0.740. + the FP8-native payload remains the rollback. **2026-10-13: 4 x 200,000 with + fp8 KV** — `--max-num-seqs 4` + `--kv-cache-dtype fp8` in the same 6912 MiB + pool (843,566 tokens), slab 22208 and host pool 44928 untouched. fp8 is what + funds the fourth slot: the pool must back seqs x window, and at bf16 four + slots cost 11,679 MiB, which would cut the resident slab to 17,441 (-33 %). + The bf16 arm (2 x 200,000, 462,752 tokens, prefill -29 %, acceptance + 0.661 → 0.740) is the documented rollback. **2026-10-13 also: the pinned-ledger + preflight now raises `ttm.pages_limit` itself** — the 2026-10-12 power failure + reset the knob to its default (125,988 MiB) and the endpoint refused to boot + 6,919 times in ~20 h; `scripts/ttm-preflight.sh` is the repair. - [[frank-archon-radiance-qwen38-native-fp8]] — **RETIRED 2026-10-08 (the w4dn8 swap replaced it; its `.rad` stays on disk as the rollback)** (root systemd unit, port 7770): the StillDeadcode **radiance** engine