Compare commits
14
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f96898d4c5 | ||
|
|
3e140cd077 | ||
|
|
c5ac0725f7 | ||
|
|
f44c39812f | ||
|
|
6dcf4a5913 | ||
|
|
a9d45e7012 | ||
|
|
2581b8d4b6 | ||
|
|
09a0251d2d | ||
|
|
066813ea95 | ||
|
|
7daa886dd6 | ||
|
|
bae78e7dae | ||
|
|
a6f55f4cc6 | ||
|
|
a9da6589a6 | ||
|
|
1a6a78ef73 |
No files matched your search
@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
|||||||
|
|
||||||
| Target | Service | Runtime | Model | Port |
|
| Target | Service | Runtime | Model | Port |
|
||||||
|--------|---------|---------|-------|------|
|
|--------|---------|---------|-------|------|
|
||||||
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 26071 --vram-kv-mib 3543`), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (inherited; pool 3.46 GiB), **MTP spec decode depth 2**, `--max-num-seqs 3`, `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8). Cutover 2026-10-08 from fp8-native (payload on disk = rollback). | 7770 |
|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: revert the swap commit (w4dn8 `.rad` on disk); the fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
|
||||||
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
||||||
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
||||||
|
|
||||||
@@ -498,48 +498,71 @@ Rollback: revert the swap commit — the Gemma4 payload
|
|||||||
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
(`/home/dank/models/Ornith-Q8/`) all remain on disk.
|
||||||
|
|
||||||
|
|
||||||
### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
|
### frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, TP=2 (Dankserver, ACTIVE)
|
||||||
|
|
||||||
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
The **radiance engine** (StillDeadcode, Apache-2.0; image
|
||||||
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
|
||||||
for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root
|
for serve; `libr4d` gfx1201 kernels, plain `podman run`, root
|
||||||
systemd unit, container name = unit name) serving
|
systemd unit, container name = unit name) serving the
|
||||||
**ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards,
|
**publisher's container of the official Qwen/Qwen3.8-Flash-Next**:
|
||||||
397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh`
|
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` →
|
||||||
with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to
|
`qwen3.8-next-flash-fp8-iq4r-moe.rad` (116,322 MiB, Hub-LFS sha256
|
||||||
`qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ
|
verified at fetch by `scripts/download-frank-archon-iq4r.sh`; recipe
|
||||||
w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine
|
`qwen4exp-w4nl64-i8-hc8m`): **routed experts as 4-bit codes into a
|
||||||
**1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin
|
16-level non-uniform codebook** after a 128-point Walsh–Hadamard rotation
|
||||||
with the token knob; the RADGRAM2 grams are format-identical across pins and
|
(E4M3 scale per 64 weights; codes by GPTQ on a 10M-token calibration with
|
||||||
cross the boundary by design), **24 whole experts per layer cast bf16** (the
|
per-expert down-proj Hessians, refined by 3 coordinate-descent sweeps),
|
||||||
PROTECTED set, derived from the calibration grams' energy share),
|
**10 whole bf16-protected experts/layer**, **int8 W8A8 trunk/lm_head**
|
||||||
**int8 trunk/lm_head**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE n-gram
|
(scale per 128 columns searched), **fp8 HC mixtures**, PLE n-gram table at
|
||||||
table at fp8 fixed scale (47.7 GiB in the container, from 95.4 GiB bf16).
|
fp8 fixed scale (47.7 GiB), **MTP head and bf16 vision tower in the
|
||||||
The plugin's **one-table-one-encoding law** (measured at rad-convert
|
container**. Card quality (full-vocab teacher-forced KL vs bf16, ~75K
|
||||||
declare): one expert table takes ONE encoding; only whole bf16 experts are
|
positions): mean 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %.
|
||||||
legal beside it — no w4/fp8 split of the down/gate_up pair (24-value
|
**MODEL_FILE is the only serving argument that moved** (2026-10-15); every
|
||||||
`rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16
|
number of the 2026-10-13 shape below carries verbatim. The plugin's
|
||||||
reference (recorded on the same 1.0.13 pin) is the precision referee;
|
**one-table-one-encoding law** (fleet-measured at rad-convert declare):
|
||||||
the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar
|
one expert table takes ONE encoding; only whole bf16 experts are legal
|
||||||
(122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook.
|
beside it. Rollback: revert the swap commit — the in-house w4dn8 `.rad`
|
||||||
Rollback: revert the swap commit — the fp8-native `.rad`
|
(KLD-gated Swift1.5 conversion) stays on disk; the fp8-native `.rad` was
|
||||||
(`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
|
deleted 2026-10-15 (rebuild: `scripts/download-frank-archon.sh` +
|
||||||
|
`scripts/build-frank-archon-rad.sh`).
|
||||||
|
|
||||||
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 2`,
|
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
|
||||||
`--max-model-len 200000` (the harness contextWindow must quote this),
|
`--max-model-len 200000` (the harness contextWindow must quote this),
|
||||||
`--placement expert_tiered --host-pool-mib 44928` + **explicit VRAM
|
`--placement expert_tiered --host-pool-mib 44928` + **explicit VRAM
|
||||||
partition `--vram-weights-mib 22208 --vram-kv-mib 6912`** (the vram-pair
|
partition `--vram-weights-mib 25600 --vram-kv-mib 5120`** (the vram-pair
|
||||||
replaced `--expert-vs-cache-ratio`; the two are ONE budget —
|
replaced `--expert-vs-cache-ratio`; the two are ONE budget —
|
||||||
`weights + kv ≤ 29,120 MiB`, from the measured card 31,856 minus 1,710
|
`weights + kv ≤ 30,969 MiB`, derived from the engine's own budget table: card
|
||||||
consumed outside both budgets minus 1,024 to keep free), `--kv-cache-dtype`
|
32,624 MiB, the pair at 29,120 settling 2,058 MiB free against the engine's
|
||||||
**`bf16` — unquantized KV since 2026-10-11**, `--num-speculative-tokens 2`,
|
209.19 MiB floor (`--gpu-headroom-mib 96` + 113.19 static), so 1,849 MiB is
|
||||||
`--max-num-batched-tokens 2048`, prefix cache 8 GiB pinned host + 64 GiB
|
claimable and the old 29,120 bound was my hand-picked 1 GiB reserve, not a
|
||||||
disk at `/home/dank/radiance-state`, **`--ngram-placement ram`**, Froggeric
|
hardware wall), `--kv-cache-dtype` **`fp8` — quantized KV, restored 2026-10-13
|
||||||
v22.5 chat template vendored (`scripts/qwen38-froggeric-v22.5.jinja`,
|
because it is the only dtype that funds the slot count at this window**,
|
||||||
`--override-chat-template`). Boot ledger: KV pool 6.75 GiB = **462,752 bf16
|
`--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`, prefix cache
|
||||||
tokens, "backs 2 sequences all at 200000 tokens at once"** (the fp8 pool's
|
8 GiB pinned host + 64 GiB disk at `/home/dank/radiance-state`,
|
||||||
386,904 backed 1.93), weights slab filled to 147 KiB of slack, host pool
|
**`--ngram-placement ram`**, Froggeric v22.5 chat template vendored
|
||||||
10,375 slots, cards settled 1.85 GiB free.
|
(`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`). Boot
|
||||||
|
ledger (2026-10-13, pair 25600/5120): `kv cache 5.00 GiB … backs 3 sequences
|
||||||
|
all at 200000 tokens at once, which is the worst case` — `kv_attn` page
|
||||||
|
26.00 KiB (fp8) at 607,740 tokens (the sizing law predicted 608,027),
|
||||||
|
`kv_qsa_bkey` 3.25 KiB at the same token count, `kv_gdn_state` 54 MiB × 6 slots
|
||||||
|
(3 sequences × 2 copies) = 324 MiB; the sampler prints **9 rows**
|
||||||
|
(3 × (1+2), inside the 25-row `wht6` exactness bound); the slab filled to
|
||||||
|
563 KiB of slack with **17,548 slots** (16,235 at 24,000, 14,753 at 22,208 —
|
||||||
|
every 1,792 MiB buys ~1,300 resident experts, and the pinned host pool hands
|
||||||
|
the same experts back: 8,893 → 7,580 slots); `unclaimed 477.30 MiB`, cards
|
||||||
|
settled ~336 MiB free against the engine's 209.19 MiB floor, KV lending
|
||||||
|
4.22 GiB to the expert cache. Prefill tracks this slab only up to a
|
||||||
|
**threshold**: 22,208 → 3,397 and 24,000 → 3,290 t/s at 43.5k are one band
|
||||||
|
(±3 %), and the 25,600 arm measured **3,290** — the third step bought nothing
|
||||||
|
(the 3,584 arm did not reproduce at 3 slots); the cliff is at ~26 GiB
|
||||||
|
(26,071 → 4,379 on 2026-10-11's arm), so below that the slab is a threshold,
|
||||||
|
not a slope.
|
||||||
|
|
||||||
|
**Decode ladder (2026-10-13, 512 tokens, greedy, this shape)**: 99.0 t/s
|
||||||
|
single-stream at 43.5k, 93.3 at 197k — decode is nearly window-free (−6 % for
|
||||||
|
4.5× depth); **3 concurrent streams give 113.9/43.3/10.9 t/s = 18.4 t/s
|
||||||
|
aggregate**, so the slot count is paid in decode concurrency (a fused-step
|
||||||
|
lottery), not in prefill; leg D of `probe-frank-archon-kv.py` measures it.
|
||||||
|
|
||||||
Shape notes carried from the fp8-native era and unchanged: wht6 gains,
|
Shape notes carried from the fp8-native era and unchanged: wht6 gains,
|
||||||
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
|
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
|
||||||
@@ -551,35 +574,52 @@ on the fp8 shape (512-token streamed probe). The campaign also
|
|||||||
measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
|
measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
|
||||||
accept 0.7246).
|
accept 0.7246).
|
||||||
|
|
||||||
**The bf16 KV law (2026-10-11)**: `--kv-cache-dtype` takes ONLY `bf16|fp8` —
|
**The KV dtype law (measured 2026-10-11, applied at 4 slots 2026-10-13)**:
|
||||||
one dtype for the whole cache, so llama.cpp's exact-K/quantized-V pair (what
|
`--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the whole cache, so
|
||||||
frank-zealot runs) has NO radiance equivalent; the rung under bf16 is fp8 on
|
llama.cpp's exact-K/quantized-V pair (what frank-zealot runs) has NO radiance
|
||||||
both halves. Measured per rank from the two boot ledgers: `kv_attn` (13
|
equivalent; the rung under bf16 is fp8 on both halves. Measured per rank from
|
||||||
layers, 4-token blocks) pages 26.00 → 52.00 KiB, while `kv_qsa_bkey` (13
|
the two boot ledgers: `kv_attn` (13 layers, 4-token blocks) pages 26.00 → 52.00
|
||||||
layers) stays 3.25 KiB at BOTH dtypes (the QSA block key is an index, not
|
KiB, while `kv_qsa_bkey` (13 layers) stays 3.25 KiB at BOTH dtypes (the QSA
|
||||||
attention state) — so a token costs 7.3125 KiB at fp8 and 13.8125 KiB at
|
block key is an index, not attention state) — so a token costs 7.3125 KiB at fp8
|
||||||
bf16: **1.89×, not 2×**. Pool sizing must also pay the dtype-independent terms
|
and 13.8125 KiB at bf16: **1.89×, not 2×**. Pool sizing must also pay the
|
||||||
that live in the same pool (~110 MiB a sequence of recurrent state + 448 MiB
|
dtype-independent terms that live in the same pool (~110 MiB a sequence of
|
||||||
of prefix checkpoints): `(vram_kv − 110·seqs − 448) MiB / 13.8125 KiB` =
|
recurrent state + 448 MiB of prefix checkpoints):
|
||||||
462,946 where the engine reported 462,752 (0.04 % page rounding). The cost is
|
`(vram_kv − 110·seqs − 448) MiB / rate` — bf16 at 6,912 gives 462,946 where the
|
||||||
paid twice on this engine, and half-refunded: idle KV VRAM is also the expert
|
engine reported 462,752 (0.04 % page rounding); fp8 at 6,912 with four slots
|
||||||
cache's spillover, and the bigger pool *lends* 6.08 GiB to the slab where the
|
gives 843,566, i.e. the deployed capacity contract 4 × 200,000. **That is why
|
||||||
fp8 pool lent 2.69 GiB. `--max-num-seqs 3 → 2` came with the change (two slots
|
the dtype is fp8: the pool must back `seqs × max-model-len` as a hard limit, and
|
||||||
× the window is the capacity contract; the "seqs 2 cannot boot" law belongs to
|
bf16 at four slots costs 11,679 MiB — the slab would fall to 17,441 MiB (−33 %),
|
||||||
the retired tcclaviger fork's torch.compile, not radiance) and it also drops
|
where fp8 leaves it at 22,208.** The cost is paid twice on this engine, and
|
||||||
the fused all-reduce to 2 × 3 = 6 rows, under the 25-row bound where wht6 is
|
half-refunded: idle KV VRAM is also the expert cache's spillover, and the bigger
|
||||||
bit-exact. Disk-tier sizing law: the engine forces ~9.6 GiB of disk-tier weight
|
pool *lends* 6.08 GiB to the slab where the 3,543 MiB fp8 pool lent 2.69 GiB.
|
||||||
buffers + arena onto the weights side, so `weights + kv` is capped near
|
`--max-num-seqs` rides 4 × (1 + spec) = 12 fused-all-reduce rows, under the
|
||||||
29.6 GiB/card; boot with the pair, never `--kv-cache-size-mib`.
|
25-row bound where wht6 is bit-exact (the "seqs 2 cannot boot" law belongs to
|
||||||
|
the retired tcclaviger fork's torch.compile, not radiance). Disk-tier sizing
|
||||||
|
law: the engine forces ~9.6 GiB of disk-tier weight buffers + arena onto the
|
||||||
|
weights side, so `weights + kv` is capped near 29.6 GiB/card; boot with the
|
||||||
|
pair, never `--kv-cache-size-mib`.
|
||||||
|
|
||||||
|
Shape notes carried from the fp8-native era and unchanged: wht6 gains,
|
||||||
|
MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
|
||||||
|
prefix cache, and the rule that a prompt over `--max-model-len` is
|
||||||
|
refused, never truncated. What changed: the fp8-native era's NVMe expert
|
||||||
|
streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the
|
||||||
|
host pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s
|
||||||
|
on the fp8 shape (512-token streamed probe). The campaign also
|
||||||
|
measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
|
||||||
|
accept 0.7246).
|
||||||
|
|
||||||
**The pinned-RAM ledger is why this is a root unit**: 2 × 44 GiB expert
|
**The pinned-RAM ledger is why this is a root unit**: 2 × 44 GiB expert
|
||||||
pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
|
pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
|
||||||
the card reads) + 8 GiB prefix host ≈ **147 GiB** charged against
|
the card reads) + 8 GiB prefix host ≈ **147 GiB** charged against
|
||||||
`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
|
`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
|
||||||
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages; the serve
|
`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages, and **the knob is
|
||||||
script's TTM preflight (`FRANK_ARCHON_TTM_MIN_MIB=150000`, cap on this host
|
runtime state whose default is half of RAM**: `scripts/ttm-preflight.sh` (sourced
|
||||||
204,800) refuses to start until both knobs are raised in the stop window
|
by the serve script) therefore RAISES it to `FRANK_ARCHON_TTM_TARGET_MIB=204800`
|
||||||
(sysfs write first, kernel cmdline + reboot only if read-only).
|
before refusing, because the 2026-10-12 power failure reset it to 125,988 MiB
|
||||||
|
and the preflight re-refused 6,919 times. A manual sysfs write in the stop
|
||||||
|
window is a convenience, not the repair; kernel cmdline persistence
|
||||||
|
(`ttm.pages_limit=52428800`) is the belt to it.
|
||||||
|
|
||||||
**Contract facts** (source-verified): the HTTP listener binds AFTER the
|
**Contract facts** (source-verified): the HTTP listener binds AFTER the
|
||||||
full load, so a bare `curl /health` gate is honest (`/health` == `/ping`,
|
full load, so a bare `curl /health` gate is honest (`/health` == `/ping`,
|
||||||
@@ -591,8 +631,9 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
|
|||||||
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
|
||||||
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
|
||||||
|
|
||||||
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
|
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md`.
|
||||||
Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
|
Retired Swift1.5 w4dn8 era (payload on disk = rollback): `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
|
||||||
|
Retired fp8-native era (payload deleted 2026-10-15, rebuildable): `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
|
||||||
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
|
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
|
||||||
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
|
||||||
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
|
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
|
||||||
@@ -761,6 +802,17 @@ linked runbooks.
|
|||||||
the KV pool is worse than the OOM it prevents.
|
the KV pool is worse than the OOM it prevents.
|
||||||
- Host swap state is invisible to `deploy.sh` (no repo file) — check
|
- Host swap state is invisible to `deploy.sh` (no repo file) — check
|
||||||
`swapon --show` before reasoning about headroom.
|
`swapon --show` before reasoning about headroom.
|
||||||
|
- **A sysfs raise is a countdown to the next reboot.** `ttm.pages_limit` defaults
|
||||||
|
to half of RAM, so the manual raise to 200 GiB (2026-10-05) vanished with the
|
||||||
|
2026-10-12 23:17 POWER FAILURE on dankserver: the cap fell to 125,988 MiB,
|
||||||
|
frank-archon's pinned-ledger preflight (floor 150,000 MiB) refused the start,
|
||||||
|
and `Restart=always` re-ran that refusal **6,919 times in ~20 h** — a unit in
|
||||||
|
`activating (auto-restart)` that exits 1 in 50 ms with no container and no
|
||||||
|
engine log is a PREFLIGHT refusal, not an engine fault. Repo law now:
|
||||||
|
`scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) raises the knob
|
||||||
|
to `FRANK_ARCHON_TTM_TARGET_MIB` itself, re-reads, and refuses by name only
|
||||||
|
after a write the kernel rejected. Never prescribe a manual sysfs write as the
|
||||||
|
recovery procedure for a service that can boot before a human reaches it.
|
||||||
- **Silent RAM corruption outranks every software theory.** frank-archon's
|
- **Silent RAM corruption outranks every software theory.** frank-archon's
|
||||||
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
|
~40k-token output corruption (2026-10-04/05, repetition/garbage in long
|
||||||
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
|
agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
|
||||||
@@ -1041,7 +1093,7 @@ Active:
|
|||||||
|---------|---------|
|
|---------|---------|
|
||||||
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
|
||||||
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
|
||||||
| frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
|
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
|
||||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||||
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
|
||||||
|
|
||||||
@@ -1049,6 +1101,7 @@ Retired (rollback targets and history):
|
|||||||
|
|
||||||
| Runtime | Host | Window | Runbook |
|
| Runtime | Host | Window | Runbook |
|
||||||
|---------|------|--------|---------|
|
|---------|------|--------|---------|
|
||||||
|
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` on disk; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
|
||||||
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
|
||||||
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
|
||||||
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
|
||||||
|
|||||||
@@ -1277,11 +1277,14 @@ deploy_llm_zealot() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
|
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
|
||||||
# digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
|
# digest-pinned stilldeadcode/radiance image) serving the PUBLISHER's
|
||||||
# Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
|
# 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next
|
||||||
# rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
|
# (qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by
|
||||||
|
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2) as a
|
||||||
# ROOT unit whose serve script drives `podman run` directly (the engine is
|
# ROOT unit whose serve script drives `podman run` directly (the engine is
|
||||||
# the image entrypoint; the image lives in dank's rootless podman storage).
|
# the image entrypoint; the image lives in dank's rootless podman storage).
|
||||||
|
# 2026-10-15 payload swap from the in-house w4dn8 (payload on disk =
|
||||||
|
# rollback).
|
||||||
# 2026-10-05 cutover from FreeToken (pre-cutover tag
|
# 2026-10-05 cutover from FreeToken (pre-cutover tag
|
||||||
# frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
|
# frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
|
||||||
# fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
|
# fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
|
||||||
@@ -1290,7 +1293,7 @@ deploy_llm_zealot() {
|
|||||||
# Gates: the image must exist under its digest ref in dank's rootless
|
# Gates: the image must exist under its digest ref in dank's rootless
|
||||||
# storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
|
# storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
|
||||||
# env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
|
# env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
|
||||||
# a truncated convert is a guaranteed long-timeout boot), and the unit env
|
# a truncated fetch is a guaranteed long-timeout boot), and the unit env
|
||||||
# must be complete. Per-file sha256 verification is the fetch script's job
|
# must be complete. Per-file sha256 verification is the fetch script's job
|
||||||
# at download time and rad-convert/rad-info's job at build time, never
|
# at download time and rad-convert/rad-info's job at build time, never
|
||||||
# re-hashed here.
|
# re-hashed here.
|
||||||
@@ -1325,13 +1328,13 @@ deploy_llm_archon() {
|
|||||||
fi
|
fi
|
||||||
if [[ ! -f "$model_dir/$model_file" ]]; then
|
if [[ ! -f "$model_dir/$model_file" ]]; then
|
||||||
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
|
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
|
||||||
echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
|
echo "frank-archon: run scripts/download-frank-archon-iq4r.sh --fetch" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
|
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
|
||||||
if [[ "$mib" -lt "$rad_min" ]]; then
|
if [[ "$mib" -lt "$rad_min" ]]; then
|
||||||
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
|
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
|
||||||
echo "frank-archon: rebuild it (scripts/build-frank-archon-rad.sh --force)" >&2
|
echo "frank-archon: re-fetch it (scripts/download-frank-archon-iq4r.sh --fetch; delete the staged file first if its sha256 failed)" >&2
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
|||||||
Executable
+83
@@ -0,0 +1,83 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# download-frank-archon-iq4r.sh — payload fetch/verify for the frank-archon
|
||||||
|
# swap to the PUBLISHER's 4-bit-codebook container:
|
||||||
|
# StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe — the official
|
||||||
|
# Qwen/Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's own
|
||||||
|
# author into ONE .rad: routed experts as 4-bit codes into a 16-level
|
||||||
|
# non-uniform codebook (Walsh–Hadamard rotation, GPTQ + coordinate-descent,
|
||||||
|
# 10M-token calibration), ten bf16-protected experts per layer, int8 trunk +
|
||||||
|
# lm_head, fp8 HC mixtures, the PLE n-gram table at fp8 fixed scale, the MTP
|
||||||
|
# head, and the bf16 vision tower. File:
|
||||||
|
# qwen3.8-next-flash-fp8-iq4r-moe.rad — 121,969,901,568 B (113.6 GiB,
|
||||||
|
# 116,322 MiB), card KL 0.0789 all-tokens / top-1 92.85 % vs bf16.
|
||||||
|
# Repo is UNGATED — no token needed (never put one on the command line).
|
||||||
|
#
|
||||||
|
# bash scripts/download-frank-archon-iq4r.sh --fetch resume/complete
|
||||||
|
# bash scripts/download-frank-archon-iq4r.sh verify only
|
||||||
|
#
|
||||||
|
# Fetch uses the vendored scripts/hfget.sh (curl+jq only): per-file size
|
||||||
|
# checks against the Hub listing, `-C -` resume, disk-fit gate. The side
|
||||||
|
# files (LICENSE, README, the w4nl recipe) stay in the staging dir as
|
||||||
|
# provenance; the serve gate mounts ONLY /home/dank/models/rad, so the
|
||||||
|
# verified .rad alone moves into it.
|
||||||
|
#
|
||||||
|
# Verify (default) is the AGENTS.md law in two parts: the file lands at the
|
||||||
|
# final path with the Hub's EXACT byte count, and its sha256 equals the
|
||||||
|
# Hub's LFS oid. A truncated transfer is a guaranteed long-timeout boot
|
||||||
|
# without this gate.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
REPO="StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe"
|
||||||
|
RAD_NAME="qwen3.8-next-flash-fp8-iq4r-moe.rad"
|
||||||
|
STAGE="/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe"
|
||||||
|
DEST="/home/dank/models/rad/$RAD_NAME"
|
||||||
|
SIZE=121969901568
|
||||||
|
SHA256="0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20"
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
|
||||||
|
mode="verify"
|
||||||
|
[[ "${1:-}" == "--fetch" ]] && mode="fetch"
|
||||||
|
|
||||||
|
if [[ "$mode" == "fetch" ]]; then
|
||||||
|
# HFGET_JOBS=4 is hfget's default (a few streams saturate the link).
|
||||||
|
bash "$SCRIPT_DIR/hfget.sh" "$REPO" "$STAGE" main
|
||||||
|
|
||||||
|
staged="$STAGE/$RAD_NAME"
|
||||||
|
if [[ ! -f "$staged" ]]; then
|
||||||
|
echo "FETCH FAILED: $staged absent after hfget" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
got=$(stat -c %s "$staged")
|
||||||
|
if [[ "$got" != "$SIZE" ]]; then
|
||||||
|
echo "FETCH FAILED: staged .rad is $got B, want $SIZE B" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "staged .rad byte-exact ($SIZE B); hashing (one full read, minutes)"
|
||||||
|
got_sha=$(sha256sum "$staged" | cut -d' ' -f1)
|
||||||
|
if [[ "$got_sha" != "$SHA256" ]]; then
|
||||||
|
echo "FETCH FAILED: sha256 $got_sha, want $SHA256 — DELETE the staged" \
|
||||||
|
"file before re-fetching (curl -C - cannot repair a corrupt body)" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
mkdir -p "$(dirname "$DEST")"
|
||||||
|
mv "$staged" "$DEST"
|
||||||
|
echo "payload placed: $DEST"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "verifying deployed payload: $DEST"
|
||||||
|
if [[ ! -f "$DEST" ]]; then
|
||||||
|
echo " BAD payload absent: $DEST" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
got=$(stat -c %s "$DEST")
|
||||||
|
if [[ "$got" != "$SIZE" ]]; then
|
||||||
|
echo " BAD payload is $got B, want $SIZE B" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "byte-exact ($SIZE B); hashing (one full read, minutes)"
|
||||||
|
got_sha=$(sha256sum "$DEST" | cut -d' ' -f1)
|
||||||
|
if [[ "$got_sha" != "$SHA256" ]]; then
|
||||||
|
echo " BAD sha256 $got_sha, want $SHA256" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "payload verified: $RAD_NAME, size and sha256 match the Hub LFS oid"
|
||||||
+155
-77
@@ -1,12 +1,31 @@
|
|||||||
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving
|
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving the
|
||||||
# ukisai/Swift1.5-Qwen3.8-Flash-Next converted to GPTQ w4 (MODEL_FILE below,
|
# PUBLISHER's 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next
|
||||||
# built by scripts/convert-swift15-w4dn8.sh) on BOTH R9700s (TP=2),
|
# (MODEL_FILE below, fetched by scripts/download-frank-archon-iq4r.sh) on
|
||||||
# dankserver :7770.
|
# BOTH R9700s (TP=2), dankserver :7770.
|
||||||
|
# 2026-10-15 PAYLOAD SWAP: the model moved from the in-house w4dn8 conversion
|
||||||
|
# of the Swift1.5 finetune to the engine author's published container
|
||||||
|
# qwen3.8-next-flash-fp8-iq4r-moe.rad (116,322 MiB, sha256-verified at fetch —
|
||||||
|
# the w4dn8 measured 116,416 MiB, so this shape carries over with 94 MiB to
|
||||||
|
# spare). NO argument changed: every number below is the 2026-10-13 shape,
|
||||||
|
# kept verbatim. The swap commit is the rollback (the w4dn8 payload stays on
|
||||||
|
# disk).
|
||||||
|
|
||||||
#
|
#
|
||||||
# SHAPE (2026-10-11): 2 decode slots x 200,000 tokens with an UNQUANTIZED
|
# SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV,
|
||||||
# (bf16) KV cache — the pool is sized for the pair, and the weights slab pays
|
# with the per-card budget spent on resident experts: 25,600 MiB of slab +
|
||||||
# for it (see the KV block). The fp8-native payload this file was written for
|
# 5,120 MiB of pool = 30,720 MiB, inside the 30,969 MiB ceiling the engine's own
|
||||||
# is the rollback target and keeps its own trio under the fp8-native runbook.
|
# budget table allows (card 32,624 MiB, settle floor 209 MiB). The pool is sized
|
||||||
|
# to the contract and nothing more (3 x 200,000 needs 5,063 MiB at fp8; 5,120
|
||||||
|
# predicts 608,027 tokens, 1.3 % slack), because every MiB taken from the slab
|
||||||
|
# is prefill and TTFT (2026-10-11 A/B: slab -14.8 % cost prefill -29 %, TTFT
|
||||||
|
# +25 % at the full window) while KV beyond the contract buys nothing.
|
||||||
|
# Consequence, deliberate: a FOURTH full-height conversation does not fit this
|
||||||
|
# pair, and the engine refuses it — it never truncates. See the KV block for the
|
||||||
|
# arithmetic and the measured dtype law. The w4dn8 payload is the swap
|
||||||
|
# rollback and keeps its trio under the w4dn8 runbook; the fp8-native payload
|
||||||
|
# that came before it was DELETED from disk 2026-10-15 (Frank's call, to free
|
||||||
|
# the volume for this download — rebuildable via the fetch + build scripts
|
||||||
|
# named in the fp8-native runbook).
|
||||||
#
|
#
|
||||||
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
|
||||||
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
|
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
|
||||||
@@ -21,20 +40,30 @@
|
|||||||
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
|
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
|
||||||
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
|
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
|
||||||
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
|
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
|
||||||
# The 4-bit-codebook publisher shape (qwen3.8-next-flash-fp8-iq4r-moe.rad,
|
# THE 4-bit-codebook publisher shape is the DEPLOYED payload since
|
||||||
# --host-pool-mib 12288) remains the documented fallback if the native
|
# 2026-10-15. The card's own run line (--host-pool-mib 12288,
|
||||||
# container proves unbootable.
|
# --expert-vs-cache-ratio 0.82, seqs 8, spec 3) is the publisher's
|
||||||
|
# suggestion on 32 GB cards; this file's shape is the fleet's measured one.
|
||||||
#
|
#
|
||||||
# PAYLOAD: /home/dank/models/rad/qwen4exp-fp8-native.rad — a rad-convert
|
# PAYLOAD: /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad — the
|
||||||
# NO-RECIPE pure copy of the official 131-shard checkpoint (every weight
|
# official Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's
|
||||||
# keeps its block-FP8 encoding; ~161 GiB). The convert runs against a
|
# own author (recipe qwen4exp-w4nl64-i8-hc8m: experts as 4-bit codes into a
|
||||||
# staging dir whose config.json drops `quantization_config`: the descriptor
|
# 16-level non-uniform codebook after a 128-point Walsh–Hadamard rotation,
|
||||||
# string is only the plugin SELECTION key (spec §2.4 — exact match, no
|
# GPTQ + 3 coordinate-descent sweeps on a 10M-token calibration, ten
|
||||||
# fallback), and no shipped plugin claims (qwen4exp, "fp8"); the shipped
|
# bf16-protected experts per layer, int8 W8A8 trunk + lm_head, fp8 HC
|
||||||
# qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
|
# mixtures, PLE n-gram fp8 fixed-scale, MTP head, bf16 vision tower; card KL
|
||||||
# containers carry. Per-weight encodings are data, not the descriptor —
|
# 0.0789 all-tokens / top-1 92.85 % vs bf16). Fetched byte-exact + sha256 by
|
||||||
# removing the key changes nothing about what the weights hold. Built by
|
# scripts/download-frank-archon-iq4r.sh; the recipe/LICENSE stay in the
|
||||||
# scripts/build-frank-archon-rad.sh (staging + convert + rad-info).
|
# staging dir beside it as provenance. ROLLBACK: revert the swap commit — the
|
||||||
|
# in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad (scripts/convert-swift15-w4dn8.sh,
|
||||||
|
# KLD-gated) stays on disk. Behind it, the fp8-native
|
||||||
|
# qwen4exp-fp8-native.rad was DELETED from disk 2026-10-15 (Frank's call,
|
||||||
|
# one-time rm, to free the volume for this payload); rebuild =
|
||||||
|
# scripts/download-frank-archon.sh + scripts/build-frank-archon-rad.sh — a
|
||||||
|
# NO-RECIPE rad-convert copy of the official 131-shard checkpoint under the
|
||||||
|
# descriptor law (no shipped plugin claims (qwen4exp, "fp8"), so the staging
|
||||||
|
# config.json drops quantization_config; per-weight encodings are data, not
|
||||||
|
# the descriptor).
|
||||||
#
|
#
|
||||||
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
|
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
|
||||||
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
|
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
|
||||||
@@ -50,12 +79,12 @@
|
|||||||
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
|
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
|
||||||
# models.yml's extraBody set is fully inside it. Unknown params are
|
# models.yml's extraBody set is fully inside it. Unknown params are
|
||||||
# refused (400), which is why nothing unverified rides the wire.
|
# refused (400), which is why nothing unverified rides the wire.
|
||||||
# - MTP: the official checkpoint carries the mtp.* head (3,101 tensors,
|
# - MTP: the .rad carries the model's mtp.* head (the publisher's recipe
|
||||||
# mtp_num_hidden_layers=1) and the plugin probes draft depth 3 — the
|
# names the mtp.fc_hidden/fc_embedding/draft_head rules; card default
|
||||||
# publisher's throughput peak for this model. Stated explicitly with
|
# depth 3). Served at the depth named by FRANK_ARCHON_SPEC_TOKENS below —
|
||||||
# --num-speculative-tokens 3.
|
# the depth this shape's decode ladder was measured at.
|
||||||
# - The fused TP all-reduce is declared over max-num-seqs x
|
# - The fused TP all-reduce is declared over max-num-seqs x
|
||||||
# (1 + --num-speculative-tokens) rows; 2 x 3 = 6 rows, under the 25-row
|
# (1 + --num-speculative-tokens) rows; 3 x (1+2) = 9 rows, under the 25-row
|
||||||
# exactness bound of the wht6 wire.
|
# exactness bound of the wht6 wire.
|
||||||
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
|
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
|
||||||
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
|
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
|
||||||
@@ -79,7 +108,7 @@ FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd7
|
|||||||
# .rad files; the raw HF download and its staging dir live beside it and
|
# .rad files; the raw HF download and its staging dir live beside it and
|
||||||
# are never mounted.
|
# are never mounted.
|
||||||
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
|
||||||
FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
|
||||||
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
|
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
|
||||||
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
|
||||||
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
|
||||||
@@ -113,18 +142,29 @@ FRANK_ARCHON_TP=2
|
|||||||
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
|
# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
|
||||||
# engine default.
|
# engine default.
|
||||||
FRANK_ARCHON_TP_WIRE=wht6
|
FRANK_ARCHON_TP_WIRE=wht6
|
||||||
# Decode slots: TWO, so the served window is a CAPACITY CONTRACT — two
|
# Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three
|
||||||
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
# concurrent conversations of --max-model-len fit the KV pool (see the KV
|
||||||
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
|
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
|
||||||
# tcclaviger vLLM fork's torch.compile shape specialization on
|
# tcclaviger vLLM fork's torch.compile shape specialization on
|
||||||
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
|
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
|
||||||
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
|
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
|
||||||
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
|
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
|
||||||
# (30.9-33.1 t/s, inside the +-3% noise floor). Side benefit: the fused TP
|
# (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys concurrency,
|
||||||
# all-reduce is declared over seqs x (1 + spec) = 6 rows, under the 25-row
|
# not speed, so a slot costs KV, never decode.
|
||||||
# bound at which the wht6 wire stays EXACT (single-sequence decode and any
|
# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
|
||||||
# step of <= 25 rows is bit-exact; only busier steps are lossy).
|
# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
|
||||||
FRANK_ARCHON_MAX_SEQS=2
|
# 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT
|
||||||
|
# (single-sequence decode and any step of <= 25 rows is bit-exact; only busier
|
||||||
|
# steps are lossy).
|
||||||
|
# The pool is sized to this contract and nothing more (see the KV block):
|
||||||
|
# 3 x 200,000 at fp8 = 5,063 MiB, pool 5,120, and the freed 1,792 MiB went into
|
||||||
|
# the resident expert slab. The trade is explicit and it is Frank's call:
|
||||||
|
# a FOURTH full-height conversation no longer fits (5,120 MiB of pool backs
|
||||||
|
# ~608k tokens; 4 x 200,000 needs 6,601), so a fourth 200k request is refused
|
||||||
|
# as too large to fit — it is NOT squeezed, paged, or truncated. Third-party
|
||||||
|
# traffic that wants four full-height sessions is a --vram-kv-mib change, not
|
||||||
|
# a code change.
|
||||||
|
FRANK_ARCHON_MAX_SEQS=3
|
||||||
# Served window. Native is 262,144; 200,000 is the publisher's production
|
# Served window. Native is 262,144; 200,000 is the publisher's production
|
||||||
# value (compose) and what models.yml's contextWindow must quote — a longer
|
# value (compose) and what models.yml's contextWindow must quote — a longer
|
||||||
# prompt is refused by the engine, never truncated. The engine treats
|
# prompt is refused by the engine, never truncated. The engine treats
|
||||||
@@ -145,25 +185,52 @@ FRANK_ARCHON_HOST_POOL_MIB=44928
|
|||||||
# VRAM left unclaimed per card (publisher default 96 MiB).
|
# VRAM left unclaimed per card (publisher default 96 MiB).
|
||||||
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
||||||
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
|
# VRAM split of claimed memory per card, MiB (0 = derive from the card).
|
||||||
# THE CEILING, measured on this host 2026-10-11: card 31,856 MiB, a boot
|
# THE CEILING, read off this engine's own VRAM-budget table (boot 2026-10-13,
|
||||||
# consumes 1,710 MiB OUTSIDE both budgets (driver context + activation arena
|
# pair 24000/5120): card 32,624 MiB; budgets claimed 29,120; `3.42 GiB is
|
||||||
# 990 + hardware queues 100 + headroom 96 + step staging), and the fp8 pair
|
# neither budgeted nor accounted — the driver's context, the activation arena
|
||||||
# settled at 1,379 MiB free — so `vram_weights + vram_kv <= 29,120 MiB` keeps
|
# and --gpu-headroom-mib come out of it; anything past those is throughput on
|
||||||
# >= 1 GiB free. The pair below is exactly that sum.
|
# the floor`; and `card 0 settled at 2.01 GiB free against 209.19 MiB (the
|
||||||
|
# 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)`.
|
||||||
|
# So the reserve above the ENGINE's floor is 2,058 - 209 = 1,849 MiB and the
|
||||||
|
# ceiling is 29,120 + 1,849 = 30,969 MiB. The 2026-10-11 bound of 29,120
|
||||||
|
# ("31,856 - 1,710 outside both budgets - keep 1 GiB free") was MY reserve, not
|
||||||
|
# the engine's; the pair below spends the difference and lands 458 MiB above
|
||||||
|
# the engine's floor. `weights + kv <= 30,969` is the gate.
|
||||||
# THE PRICE, and it is paid twice: on this engine idle KV VRAM is ALSO the
|
# THE PRICE, and it is paid twice: on this engine idle KV VRAM is ALSO the
|
||||||
# expert cache's spillover (`mover: 2275 slots can be lent out of the KV
|
# expert cache's spillover (`mover: 2275 slots can be lent out of the KV
|
||||||
# cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <= 256 tokens`),
|
# cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <= 256 tokens`),
|
||||||
# so a bigger KV pool steals weight residency from the slab AND from the
|
# so a bigger KV pool steals weight residency from the slab AND from the
|
||||||
# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
|
# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
|
||||||
# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
|
# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
|
||||||
# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. Expect -3..-8%
|
# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. The 2026-10-11
|
||||||
# vs the fp8 control (136 t/s median at 44.5k depth, acceptance 0.7246,
|
# exact-KV A/B measured the slab's own price (22208 vs the fp8-era 26071):
|
||||||
# ~190 t/s on a live 512-token probe); re-tune from a boot ledger + probe,
|
# prefill -29 %, TTFT at the full window +25 %, decode inside the noise band on
|
||||||
# never from that estimate.
|
# 2 of 3 passes, draft acceptance UP 0.661 -> 0.740.
|
||||||
FRANK_ARCHON_VRAM_WEIGHTS_MIB=22208
|
# 2026-10-13 (late): THREE slots, so the pool is cut to the contract (5,120)
|
||||||
FRANK_ARCHON_VRAM_KV_MIB=6912
|
# and every spare MiB goes to the slab: 22208 -> 24000 (the third slot's
|
||||||
# KV cache UNQUANTIZED (bf16). This engine's flag takes ONLY bf16|fp8 — there
|
# surplus KV), then 24000 -> 25600 (the card's settling slack). The direction
|
||||||
# is ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
|
# the A/B data points: prefill is slab-limited, KV beyond the contract is not.
|
||||||
|
# The margin is thin by design (608,027 predicted tokens against the 600,000
|
||||||
|
# contract), so the boot ledger's `backs N sequences all at 200000 tokens at
|
||||||
|
# once` line is the referee for THIS pair and the concurrent probe is its
|
||||||
|
# regression test.
|
||||||
|
# VERDICT on the 25,600 arm (measured 2026-10-13, scripts/probe-frank-archon-kv.py):
|
||||||
|
# the threshold claim is FALSIFIED — the slab is a SLOPE at this end. A-cold
|
||||||
|
# 43.5k single-stream: 22,208 -> 3,397, 24,000 -> 3,290, 25,600 -> 3,584 t/s;
|
||||||
|
# the 3-way 197k arm's aggregate prefill 3,051 -> 3,589 t/s (wall
|
||||||
|
# 193.7 -> 164.5 s). So +1,600 MiB of slab = +8.9 % single-stream and
|
||||||
|
# +17.6 % concurrent, both outside the +-3 % noise floor. The 22,208/24,000
|
||||||
|
# pair landed inside noise of EACH OTHER, which is what made it read as a
|
||||||
|
# plateau; two points are not a curve.
|
||||||
|
# The cliff that remains is between 25,600 (3,584) and 26,071 (4,379, the
|
||||||
|
# 2026-10-11 fp8 control at seqs 3 / pool 3,543) — +22 % for +471 MiB. At THREE
|
||||||
|
# slots that is only reachable by breaking the ceiling (26,112 + 5,120 = 31,232
|
||||||
|
# vs 30,969, ~214 MiB above the engine's floor); at TWO slots it is free:
|
||||||
|
# pool 3,072 + slab 27,897 = exactly the ceiling, slab above the cliff.
|
||||||
|
FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
|
||||||
|
FRANK_ARCHON_VRAM_KV_MIB=5120
|
||||||
|
# KV cache QUANTIZED (fp8). This engine's flag takes ONLY bf16|fp8 — there is
|
||||||
|
# ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
|
||||||
# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
|
# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
|
||||||
# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
|
# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
|
||||||
# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
|
# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
|
||||||
@@ -177,30 +244,36 @@ FRANK_ARCHON_VRAM_KV_MIB=6912
|
|||||||
# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
|
# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
|
||||||
# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
|
# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
|
||||||
# checkpoint region 8 x 55.99 = 448 MiB. So:
|
# checkpoint region 8 x 55.99 = 448 MiB. So:
|
||||||
# pool_mib >= tokens x 13.8125 KiB + 110 x seqs + 448
|
# pool_mib >= tokens x 7.3125 KiB + 110 x seqs + 448
|
||||||
# 6912 -> (6912 - 220 - 448) x 1024 / 13.8125 = 462,946 tokens; the engine
|
# 3 x 200,000 at fp8 needs 4,285 + 330 + 448 = 5,063 MiB; the pool is 5,120,
|
||||||
# reported 462,752 (0.04% page rounding) — 2 x 200,000 with 62k of slack,
|
# and it predicts (5120 - 330 - 448) x 1024 / 7.3125 = 608,027 tokens: the
|
||||||
# and the ledger says it outright: "backs 2 sequences all at 200000 tokens
|
# three-slot contract with 1.3 % slack, every MiB above 5,063 having been
|
||||||
# at once, which is the worst case".
|
# handed to the expert slab instead. The ledger line is the referee ("backs N
|
||||||
# (the fp8 pool of 3543 gave 386,904 — it never held 2 x 200k, it held 1.93
|
# sequences all at 200000 tokens at once") — a pair this tight is read off the
|
||||||
# sessions, and a third long conversation preempted to the host/disk tiers).
|
# boot, never assumed.
|
||||||
# Secondary cost: the prefix-cache entry grows with the attention plane
|
# At bf16 the same three slots cost 8,093 + 778 = 8,871 MiB, which would cut
|
||||||
# (one entry 58.50 KiB at fp8 -> 110.50 KiB at bf16, measured on the two boot
|
# the slab to 20,249 MiB (-9 %); at FOUR slots bf16 costs 10,791 + 888 =
|
||||||
# ledgers' `kv tier transfer` lines), so the host tier below doubles to keep
|
# 11,679 MiB and the slab falls to 17,441 (-33 %). fp8 keeps 22,208 at both
|
||||||
# the same number of parked conversations, out of RAM the host has
|
# slot counts. The 2026-10-11 bf16 shape (6,912 / seqs 2) backed 462,752
|
||||||
# (104 GiB available with the service loaded).
|
# tokens = 2 x 200,000; the historical fp8 pool of 3,543 gave 386,904 (1.93
|
||||||
FRANK_ARCHON_KV_DTYPE=bf16
|
# sessions), and the four-slot fp8 boot of 2026-10-13 19:21 backed 843,276.
|
||||||
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3
|
# Secondary saving: the prefix-cache entry shrinks with the attention plane
|
||||||
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
|
# (one entry 110.50 KiB at bf16 -> 58.50 KiB at fp8, measured on the two boot
|
||||||
# flag states it so the ledger records the intent).
|
# ledgers' `kv tier transfer` lines), so the host tier below parks twice as
|
||||||
|
# many conversations as it did at bf16 — no re-size needed to park three slots.
|
||||||
|
FRANK_ARCHON_KV_DTYPE=fp8
|
||||||
|
# Spec decode: the checkpoint's own MTP head, served at depth 2 — the
|
||||||
|
# publisher's card ships depth 3 as its peak; this fleet's ladder was
|
||||||
|
# measured at 2 and the swap keeps every argument, so 2 stays (rows
|
||||||
|
# 3 x (1+2) = 9, inside the wht6 25-row exactness bound).
|
||||||
FRANK_ARCHON_SPEC_TOKENS=2
|
FRANK_ARCHON_SPEC_TOKENS=2
|
||||||
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
|
||||||
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
|
||||||
# The host tier is sized in ENTRIES, and one entry doubles with the cache
|
# and one entry tracks the cache dtype (58.50 KiB at fp8, 117 KiB at bf16, per
|
||||||
# dtype (58.50 KiB at fp8 -> 117 KiB at bf16, per the ledger's `kv tier
|
# the ledger's `kv tier transfer` line): at fp8, 8192 MiB parks ~5.8 sessions
|
||||||
# transfer` line): 4096 MiB at bf16 parks ~1.4 sessions of 200k where it
|
# of 200k (at bf16 the same tier parked ~2.9), so four live slots plus a full
|
||||||
# parked ~2.9 at fp8. Doubled to keep the capacity, at RAM the host has
|
# parked set fits with the tier as it stands — at RAM the host has (104 GiB
|
||||||
# (104 GiB available with the service loaded).
|
# available with the service loaded).
|
||||||
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
FRANK_ARCHON_PREFIX_HOST_MIB=8192
|
||||||
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
FRANK_ARCHON_PREFIX_DISK_MIB=65536
|
||||||
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
|
||||||
@@ -211,18 +284,23 @@ FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
|||||||
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
|
||||||
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
||||||
# takes the min over the knobs that exist). Below the plan the load dies
|
# takes the min over the knobs that exist). Below the plan the load dies
|
||||||
# mid-pin with an allocator error instead of this named refusal — raise the
|
# mid-pin with an allocator error instead of this named refusal.
|
||||||
# knob first (runtime sysfs write; kernel cmdline fallback + one reboot only
|
# THE CAP IS A RUNTIME KNOB — the kernel default is half of RAM, so every
|
||||||
# if the file is read-only), see the runbook. Cap raised 2026-10-05 to
|
# raise dies with a reboot. Measured the hard way: the 2026-10-05 manual raise
|
||||||
# 52,428,800 pages = 200 GiB, which still clears this floor.
|
# to 52,428,800 pages (200 GiB) was gone after the 2026-10-12 23:17 POWER
|
||||||
|
# FAILURE, the cap fell back to 125,988 MiB, and the preflight refused the
|
||||||
|
# start 6,919 times in ~20 h while Restart=always re-ran it. The gate in
|
||||||
|
# scripts/ttm-preflight.sh now RAISES the knob itself before refusing (the
|
||||||
|
# root unit is privileged; the knob is a 0644 root module parameter), so the
|
||||||
|
# recovery is a boot, not a visit to the machine.
|
||||||
FRANK_ARCHON_TTM_MIN_MIB=150000
|
FRANK_ARCHON_TTM_MIN_MIB=150000
|
||||||
|
FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||||
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
|
# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
|
||||||
# multi-flag values as one string; the serve script expands it unquoted on
|
# multi-flag values as one string; the serve script expands it unquoted on
|
||||||
# purpose.
|
# purpose.
|
||||||
FRANK_ARCHON_EXTRA_ARGS=
|
FRANK_ARCHON_EXTRA_ARGS=
|
||||||
# The .rad's minimum size in MiB (deployer gate; a truncated convert is a
|
# The .rad's minimum size in MiB (deployer gate; a truncated fetch is a
|
||||||
# guaranteed long-timeout boot instead of a fast failure). The converted
|
# guaranteed long-timeout boot instead of a fast failure). The deployed
|
||||||
# container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
|
# publisher container measures 116,322 MiB (121,969,901,568 B); the
|
||||||
# plan-only estimate 172.76 GiB covers the data planes but not the metadata
|
# in-house w4dn8 rollback measured 116,416 MiB. Floor sits just under both.
|
||||||
# tail) - floor sits just under the payload.
|
|
||||||
FRANK_ARCHON_RAD_MIN_MIB=110000
|
FRANK_ARCHON_RAD_MIN_MIB=110000
|
||||||
@@ -1,9 +1,9 @@
|
|||||||
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
|
||||||
# (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
|
# (the publisher's 4-bit-codebook container of the official
|
||||||
# rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
|
# Qwen/Qwen3.8-Flash-Next, qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by
|
||||||
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
|
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2, wht6 wire),
|
||||||
# frank-archon-freetoken-pr587-20261005 is the pre-cutover state; the
|
# dankserver :7770. 2026-10-15 payload swap from the in-house w4dn8 (the
|
||||||
# retired unit/env/serve trio and Dockerfile are in that tag).
|
# swap commit is the revert; its .rad stays on disk).
|
||||||
#
|
#
|
||||||
# Same port, same served alias, same container identity as the unit name;
|
# Same port, same served alias, same container identity as the unit name;
|
||||||
# the harness models.yml id is unchanged. The serve script drives
|
# the harness models.yml id is unchanged. The serve script drives
|
||||||
@@ -19,7 +19,7 @@
|
|||||||
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
|
||||||
# engine still runs out of dank's rootless podman storage.
|
# engine still runs out of dank's rootless podman storage.
|
||||||
[Unit]
|
[Unit]
|
||||||
Description=frank-archon — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
|
Description=frank-archon — radiance Qwen3.8-Flash-Next (publisher iq4r) on 2x R9700 TP=2
|
||||||
After=network-online.target
|
After=network-online.target
|
||||||
Wants=network-online.target
|
Wants=network-online.target
|
||||||
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
# Never let the start-rate limiter give up after a crash loop (a failed LLM
|
||||||
@@ -35,6 +35,13 @@ Group=dank
|
|||||||
LimitMEMLOCK=infinity
|
LimitMEMLOCK=infinity
|
||||||
Environment="FRANK_ARCHON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-archon.env"
|
Environment="FRANK_ARCHON_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-archon.env"
|
||||||
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-archon.sh
|
ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-archon.sh
|
||||||
|
# The `+` prefix runs this as root even though the service is User=dank — the
|
||||||
|
# ttm pages cap is a root-owned knob and the pinned-RAM ledger (~147 GiB) is
|
||||||
|
# unusable below it. After a reboot the knob is back at its default (half of
|
||||||
|
# RAM), so the raise must be part of the start, not a note in a runbook: the
|
||||||
|
# 2026-10-12 power failure reset it and the endpoint refused to boot 6,919
|
||||||
|
# times. The serve script re-verifies the cap after this hook.
|
||||||
|
ExecStartPre=+/home/dank/serverAppConfigs/scripts/ttm-preflight.sh --raise
|
||||||
# systemd stop: wait alone cannot stop the container child; stop it
|
# systemd stop: wait alone cannot stop the container child; stop it
|
||||||
# explicitly. No graceful drain contract is claimed (the OpenAI front end
|
# explicitly. No graceful drain contract is claimed (the OpenAI front end
|
||||||
# holds no queue to drain — in-flight decodes are lost, which a stop
|
# holds no queue to drain — in-flight decodes are lost, which a stop
|
||||||
|
|||||||
@@ -1,22 +1,30 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""probe-frank-archon-kv.py — verify the 2 x 200,000 KV contract on a LIVE
|
"""probe-frank-archon-kv.py — verify the seqs x max-model-len KV contract on a
|
||||||
frank-archon (radiance) service. Run it ON dankserver: it talks to
|
frank-archon (radiance) service. Run it ON dankserver: it talks to
|
||||||
127.0.0.1:7770 and builds its prompts from the repo checkout there.
|
127.0.0.1:7770 and builds its prompts from the repo checkout there.
|
||||||
No scratch container, no mask: the service under test is the service.
|
No scratch container, no mask: the service under test is the service.
|
||||||
|
|
||||||
ssh dankserver 'python3 /home/dank/serverAppConfigs/scripts/probe-frank-archon-kv.py'
|
ssh dankserver 'python3 /home/dank/serverAppConfigs/scripts/probe-frank-archon-kv.py'
|
||||||
|
|
||||||
Three probes, in increasing cost:
|
Four probes, in increasing cost:
|
||||||
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
|
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
|
||||||
Reports the engine's own `timings` (prefill, decode, draft acceptance).
|
Reports the engine's own `timings` (prefill, decode, draft acceptance).
|
||||||
The bench law applies: >=128 generated tokens, greedy, real text.
|
The bench law applies: >=128 generated tokens, greedy, real text.
|
||||||
B the capacity contract — TWO concurrent chats of ~197k prompt tokens
|
B the capacity contract — as many CONCURRENT chats of ~197k prompt tokens as
|
||||||
(~394k tokens resident) each carrying a unique marker at its head. Both
|
the deployed `FRANK_ARCHON_MAX_SEQS` (read from the repo env file on this
|
||||||
must answer their own marker. This is what a 2 x 200,000 endpoint owes:
|
host), each carrying a unique marker at its head. All must answer their own
|
||||||
two full-height conversations resident at once, exact retrieval in both.
|
marker. This is what the endpoint owes: N full-height conversations
|
||||||
|
resident at once, exact retrieval in every one of them.
|
||||||
C prefix-cache restore — probe A's cold prompt again, so the disk tier has
|
C prefix-cache restore — probe A's cold prompt again, so the disk tier has
|
||||||
to give the blocks back (cache_n > 0, TTFT collapses).
|
to give the blocks back (cache_n > 0, TTFT collapses).
|
||||||
|
D concurrency decode — `seqs` CONCURRENT 512-token generations at ~25 % of
|
||||||
|
the window. This is the cost a slot count imposes on DECODE, which no
|
||||||
|
prefill number shows: the fused-step schedule is unfair, so every stream is
|
||||||
|
printed plus the aggregate. Measured 2026-10-13 at 3 x 200,000 fp8:
|
||||||
|
single-stream 99.0 t/s (43.5k) and 93.3 (197k) — decode is nearly
|
||||||
|
window-free — against 3-stream 113.9/43.3/10.9 t/s = 18.4 t/s aggregate.
|
||||||
|
Concurrency, not the slab, is what decode pays for slots. The 2026-10-13
|
||||||
|
shape hunt missed this for a whole day because leg B generates 24 tokens.
|
||||||
PROBE LAW (learned the hard way, twice): radiance routes the thinking block to
|
PROBE LAW (learned the hard way, twice): radiance routes the thinking block to
|
||||||
`reasoning_content`, so a chat request whose budget is spent inside
|
`reasoning_content`, so a chat request whose budget is spent inside
|
||||||
`<think>` returns EMPTY `content` with `finish_reason: stop`. A probe that
|
`<think>` returns EMPTY `content` with `finish_reason: stop`. A probe that
|
||||||
@@ -114,23 +122,51 @@ def show(tag, j, wall, expect=None):
|
|||||||
print(line)
|
print(line)
|
||||||
|
|
||||||
|
|
||||||
|
def deployed_shape():
|
||||||
|
"""The contract is the deployed env, not a constant: slot count and window
|
||||||
|
come from the checkout on this host, so the probe cannot drift from the
|
||||||
|
service it is testing (the 2026-10-13 shape is 4 x 200,000; before that
|
||||||
|
2 x 200,000, and the fp8 era 3 x 200,000)."""
|
||||||
|
shape = {}
|
||||||
|
with open(f"{REPO}/scripts/frank-archon.env") as fh:
|
||||||
|
for line in fh:
|
||||||
|
line = line.strip()
|
||||||
|
if line.startswith("FRANK_ARCHON_") and "=" in line:
|
||||||
|
key, _, val = line.partition("=")
|
||||||
|
shape[key] = val.strip().strip('"')
|
||||||
|
return (int(shape["FRANK_ARCHON_MAX_SEQS"]),
|
||||||
|
int(shape["FRANK_ARCHON_MAX_MODEL_LEN"]),
|
||||||
|
shape["FRANK_ARCHON_KV_DTYPE"])
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
|
seqs, maxlen, dtype = deployed_shape()
|
||||||
|
print(f"shape: {seqs} x {maxlen:,} tokens, kv {dtype}")
|
||||||
print("health:", json.dumps(get("/health")))
|
print("health:", json.dumps(get("/health")))
|
||||||
print("models:", [m.get("id") for m in get("/v1/models").get("data", [])])
|
print("models:", [m.get("id") for m in get("/v1/models").get("data", [])])
|
||||||
deep = corpus(160_000)
|
deep = corpus(160_000)
|
||||||
wide = corpus(1_100_000)
|
wide = corpus(1_100_000)
|
||||||
|
|
||||||
facts = {"B1": "cobalt-harbor-13", "B2": "quartz-meadow-42"}
|
# 3.28 chars/token is the measured ratio of this corpus through the
|
||||||
bodies = {s: chat(wide, 646_000, s, f, 24) for s, f in facts.items()}
|
# froggeric template (646,000 chars -> 196,919 tokens at the 2026-10-11
|
||||||
|
# deploy); 98.5 % of the window leaves the response room under the ceiling.
|
||||||
|
chars = int(0.985 * maxlen * 3.28)
|
||||||
|
# Each marker's FIRST hyphen-separated word is unique, because show()
|
||||||
|
# scores the hit on that word — a shared prefix would score every slot HIT
|
||||||
|
# on any answer that says the word.
|
||||||
|
words = ["cobalt", "quartz", "basalt", "ember", "willow", "cobweb", "flint"]
|
||||||
|
facts = {f"B{i+1}": f"{words[i % len(words)]}-flag-{(i + 1) * 137}"
|
||||||
|
for i in range(seqs)}
|
||||||
|
bodies = {s: chat(wide, chars, s, f, 24) for s, f in facts.items()}
|
||||||
print(f"[B] {len(bodies['B1']['messages'][0]['content']):,} chars per "
|
print(f"[B] {len(bodies['B1']['messages'][0]['content']):,} chars per "
|
||||||
f"request, x2 concurrent")
|
f"request, x{seqs} concurrent")
|
||||||
|
|
||||||
# A + C first (cheap), B last (the expensive pair).
|
# A + C first (cheap), B last (the expensive set).
|
||||||
for tag, salt in (("A-cold", "A1-9137"), ):
|
for tag, salt in (("A-cold", "A1-9137"), ):
|
||||||
j, wall, _ = post(chat(deep, 140_000, salt, "zephyr-lantern-7", 512))
|
j, wall, _ = post(chat(deep, 140_000, salt, "zephyr-lantern-7", 512))
|
||||||
show(tag, j, wall, "zephyr-lantern-7")
|
show(tag, j, wall, "zephyr-lantern-7")
|
||||||
t0 = time.time()
|
t0 = time.time()
|
||||||
with concurrent.futures.ThreadPoolExecutor(max_workers=2) as ex:
|
with concurrent.futures.ThreadPoolExecutor(max_workers=seqs) as ex:
|
||||||
futs = {s: ex.submit(post, b) for s, b in bodies.items()}
|
futs = {s: ex.submit(post, b) for s, b in bodies.items()}
|
||||||
for s, fut in futs.items():
|
for s, fut in futs.items():
|
||||||
try:
|
try:
|
||||||
@@ -139,6 +175,24 @@ def main():
|
|||||||
except Exception as e: # a preemption/timeout lands here
|
except Exception as e: # a preemption/timeout lands here
|
||||||
print(f"[B-{s}] FAILED after {time.time() - t0:.1f}s: {e!r}")
|
print(f"[B-{s}] FAILED after {time.time() - t0:.1f}s: {e!r}")
|
||||||
print(f"[B] total {time.time() - t0:.1f}s")
|
print(f"[B] total {time.time() - t0:.1f}s")
|
||||||
|
# D — concurrency decode: seqs streams, 512 tokens each, ~25 % depth.
|
||||||
|
dchars = int(0.25 * maxlen * 3.28)
|
||||||
|
dbodies = [chat(deep, dchars, f"D{i + 1}-9137", f"flint-marker-{i}", 512)
|
||||||
|
for i in range(seqs)]
|
||||||
|
print(f"[D] {seqs} concurrent streams x 512 tokens, "
|
||||||
|
f"{dchars:,} chars each")
|
||||||
|
t0 = time.time()
|
||||||
|
with concurrent.futures.ThreadPoolExecutor(max_workers=seqs) as ex:
|
||||||
|
futs = [ex.submit(post, b) for b in dbodies]
|
||||||
|
for i, fut in enumerate(futs, 1):
|
||||||
|
try:
|
||||||
|
j, wall, _ = fut.result()
|
||||||
|
show(f"D-s{i}", j, wall)
|
||||||
|
except Exception as e: # a preemption/timeout lands here
|
||||||
|
print(f"[D-s{i}] FAILED after {time.time() - t0:.1f}s: {e!r}")
|
||||||
|
tot = time.time() - t0
|
||||||
|
print(f"[D] {seqs} x 512 tokens in {tot:.1f}s = "
|
||||||
|
f"{seqs * 512 / tot:.1f} t/s aggregate decode")
|
||||||
j, wall, _ = post(chat(deep, 140_000, "A1-9137", "zephyr-lantern-7", 512))
|
j, wall, _ = post(chat(deep, 140_000, "A1-9137", "zephyr-lantern-7", 512))
|
||||||
show("C-prefix-hit", j, wall, "zephyr-lantern-7")
|
show("C-prefix-hit", j, wall, "zephyr-lantern-7")
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,13 @@
|
|||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
# serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
|
# serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
|
||||||
# Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
|
# Apache-2.0) serving the PUBLISHER's 4-bit-codebook container of the
|
||||||
# NATIVE block-FP8 precision from a rad-convert container
|
# official Qwen/Qwen3.8-Flash-Next (the engine author's own rad-convert of
|
||||||
# (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
|
# the bf16 checkpoint: qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by
|
||||||
# wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
|
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2, wht6 wire),
|
||||||
# one card; git tag frank-archon-freetoken-pr587-20261005 is the
|
# dankserver :7770. 2026-10-15 payload swap from the in-house w4dn8 of the
|
||||||
# pre-cutover state).
|
# Swift1.5 finetune (payload on disk = rollback; the swap commit is the
|
||||||
|
# revert); 2026-10-05 radiance cutover from FreeToken (git tag
|
||||||
|
# frank-archon-freetoken-pr587-20261005 is the pre-radiance state).
|
||||||
#
|
#
|
||||||
# The run line below IS the engine argv: the image's entrypoint is the
|
# The run line below IS the engine argv: the image's entrypoint is the
|
||||||
# engine and its tools sit beside it on PATH. Unlike every predecessor
|
# engine and its tools sit beside it on PATH. Unlike every predecessor
|
||||||
@@ -13,9 +15,11 @@
|
|||||||
# log states every budget before anything is allocated (weights vs KV
|
# log states every budget before anything is allocated (weights vs KV
|
||||||
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
# split, the expert host pool's coverage of the worst case, the PLE pin,
|
||||||
# the P2P answer per pair) — read it after every resize.
|
# the P2P answer per pair) — read it after every resize.
|
||||||
# Deployed shape (2026-10-11): 2 decode slots x 200,000 tokens, bf16
|
# Deployed shape (2026-10-13): THREE decode slots x 200,000 tokens, fp8
|
||||||
# (unquantized) KV in a 6,912 MiB pool, the weights slab cut to 22,208 MiB to
|
# (quantized) KV in a 5,120 MiB pool (contract 3 x 200,000 + 1.3 %) and a
|
||||||
# pay for it, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
|
# 25,600 MiB resident expert slab — 30,720 MiB of the 30,969 MiB the engine's
|
||||||
|
# own budget table makes claimable, spent on the slab because prefill is
|
||||||
|
# slab-limited. 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
|
||||||
# 8 GiB + disk 64 GiB, PLE pinned in RAM.
|
# 8 GiB + disk 64 GiB, PLE pinned in RAM.
|
||||||
#
|
#
|
||||||
# The root unit (scripts/frank-archon.service, User=dank,
|
# The root unit (scripts/frank-archon.service, User=dank,
|
||||||
@@ -61,6 +65,7 @@ set +a
|
|||||||
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
|
: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
|
||||||
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
|
: "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
|
||||||
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
|
: "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
|
||||||
|
: "${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
|
||||||
: "${FRANK_ARCHON_RAD_MIN_MIB:?FRANK_ARCHON_RAD_MIN_MIB must be set}"
|
: "${FRANK_ARCHON_RAD_MIN_MIB:?FRANK_ARCHON_RAD_MIN_MIB must be set}"
|
||||||
|
|
||||||
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
# The image gate is the digest-pinned ref itself: a name@sha256 ref exists
|
||||||
@@ -117,34 +122,19 @@ if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# TTM preflight: the pinned ledger (~2 x 40 GiB pools + 47.7 GiB PLE + 4 GiB
|
# TTM preflight: the pinned ledger (2 x FRANK_ARCHON_HOST_POOL_MIB + the
|
||||||
# prefix host = ~132 GiB) rides TTM's shared pages, capped by ttm's
|
# 47.7 GiB PLE table + the prefix host tier, ~147 GiB) rides TTM's shared
|
||||||
# pages_limit (the current kernel exposes NO pages_max; the cap function
|
# pages, capped by ttm's pages_limit (the current kernel exposes NO
|
||||||
# takes the min over the knobs that exist). Below FRANK_ARCHON_TTM_MIN_MIB
|
# pages_max; the cap function takes the min over the knobs that exist).
|
||||||
# the load dies mid-pin with an allocator error instead of this named
|
# Below FRANK_ARCHON_TTM_MIN_MIB the load dies mid-pin with an allocator
|
||||||
# refusal — raise the knob first (runtime sysfs write; kernel cmdline
|
# error, so the gate refuses first — and RAISES the knob before refusing:
|
||||||
# fallback + one reboot only if the file is read-only), see the runbook.
|
# the cap is a runtime knob whose default is half of RAM, so a power failure
|
||||||
# RAISED 2026-10-05 pre-cutover to 52,428,800 pages = 200 GiB.
|
# resets it (measured 2026-10-12 23:17 on this host: 200 GiB -> 125,988 MiB,
|
||||||
ttm_cap_mib() {
|
# and Restart=always re-ran the refusal 6,919 times over ~20 h).
|
||||||
# min over the knobs THIS kernel exposes — the current kernel has
|
# FRANK_ARCHON_TTM_TARGET_MIB is the recorded host ceiling, not the floor.
|
||||||
# pages_limit only (no pages_max); a knob that is absent is skipped,
|
# shellcheck source=/dev/null
|
||||||
# not a zero (a zero would refuse a healthy cap).
|
source "$SCRIPT_DIR/ttm-preflight.sh"
|
||||||
local v cur=""
|
ttm_ensure_cap || exit 1
|
||||||
for p in /sys/module/ttm/parameters/pages_limit /sys/module/ttm/parameters/pages_max; do
|
|
||||||
[ -r "$p" ] || continue
|
|
||||||
v="$(cat "$p" 2>/dev/null)" || continue
|
|
||||||
[ -n "$v" ] || continue
|
|
||||||
v=$(( v * 4 / 1024 ))
|
|
||||||
if [ "$v" -lt "${cur:-999999999}" ]; then cur=$v; fi
|
|
||||||
done
|
|
||||||
echo "${cur:-0}"
|
|
||||||
}
|
|
||||||
TTM_CAP="$(ttm_cap_mib)"
|
|
||||||
if [ "$TTM_CAP" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
|
|
||||||
echo "ERROR: ttm pages cap is ${TTM_CAP} MiB, the pinned ledger needs >= $FRANK_ARCHON_TTM_MIN_MIB" >&2
|
|
||||||
echo "frank-archon: raise /sys/module/ttm/parameters/pages_limit (and pages_max if the kernel exposes one) to >= $FRANK_ARCHON_TTM_MIN_MIB in 4 KiB pages" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
# A crashed/killed unit can leave the container name registered (systemd
|
# A crashed/killed unit can leave the container name registered (systemd
|
||||||
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
|
# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
|
||||||
|
|||||||
Executable
+104
@@ -0,0 +1,104 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# ttm-preflight.sh — the TTM pages-cap gate for engines that PIN host RAM.
|
||||||
|
#
|
||||||
|
# Sourced, never executed. An engine that pins tens of GiB of host RAM (the
|
||||||
|
# radiance expert pools + the PLE n-gram table) has its pinned pages charged
|
||||||
|
# against the kernel's ttm page cap: /sys/module/ttm/parameters/pages_limit
|
||||||
|
# (and pages_max on kernels that expose one). The cap function below takes the
|
||||||
|
# MIN over the knobs the running kernel actually exposes — an absent knob is
|
||||||
|
# skipped, never read as zero, because a zero would refuse a healthy cap.
|
||||||
|
#
|
||||||
|
# THE LAW THIS FILE EXISTS FOR: the cap is a RUNTIME knob. The kernel default is
|
||||||
|
# half of RAM, so every raise is lost on reboot. On 2026-10-12 23:17 a power
|
||||||
|
# failure on dankserver reset it 200 GiB -> 125,988 MiB, and frank-archon's
|
||||||
|
# preflight refused the start — 6,919 crash-restarts over ~20 hours, with
|
||||||
|
# `Restart=always` cheerfully re-running a refusal. A manual sysfs write is
|
||||||
|
# therefore not an acceptable repair procedure: the gate raises the knob itself
|
||||||
|
# (the units that need it run as root, and the knob is a 0644 root-owned module
|
||||||
|
# parameter), re-reads, and only then refuses.
|
||||||
|
#
|
||||||
|
# Contract (set by the sourcing script's env file):
|
||||||
|
# FRANK_ARCHON_TTM_MIN_MIB floor the pinned ledger needs; refuse below it
|
||||||
|
# FRANK_ARCHON_TTM_TARGET_MIB value to raise the cap to when it is below the
|
||||||
|
# floor (the recorded host ceiling — writing the
|
||||||
|
# floor itself would leave the ledger no slack)
|
||||||
|
# FRANK_ARCHON_TTM_SYS_DIR knob directory; defaults to the real sysfs
|
||||||
|
# path, a test points it at a temp dir.
|
||||||
|
# Pages are 4 KiB: pages = MiB x 256.
|
||||||
|
|
||||||
|
: "${FRANK_ARCHON_TTM_SYS_DIR:=/sys/module/ttm/parameters}"
|
||||||
|
|
||||||
|
# Prints the effective cap in MiB, 0 when no knob is readable.
|
||||||
|
ttm_cap_mib() {
|
||||||
|
local p v cur=""
|
||||||
|
for p in "$FRANK_ARCHON_TTM_SYS_DIR/pages_limit" \
|
||||||
|
"$FRANK_ARCHON_TTM_SYS_DIR/pages_max"; do
|
||||||
|
[ -r "$p" ] || continue
|
||||||
|
v="$(cat "$p" 2>/dev/null)" || continue
|
||||||
|
[ -n "$v" ] || continue
|
||||||
|
v=$(( v * 4 / 1024 ))
|
||||||
|
if [ "$v" -lt "${cur:-999999999}" ]; then cur=$v; fi
|
||||||
|
done
|
||||||
|
echo "${cur:-0}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Writes $1 MiB (as 4 KiB pages) into every knob present. Failures are
|
||||||
|
# swallowed: the caller re-reads the cap, and a write the kernel refused shows
|
||||||
|
# up there (ttm refuses a limit below the pages it has already pinned).
|
||||||
|
#
|
||||||
|
# PRIVILEGE, measured the hard way: the knob is root-owned 0644 and the service
|
||||||
|
# process runs as User=dank, so `-w` is FALSE there and a plain echo fails. The
|
||||||
|
# canonical privileged path is the unit's ExecStartPre with the `+` prefix
|
||||||
|
# (systemd runs it as root even under User=); `sudo -n` is the fallback for a
|
||||||
|
# human running the gate from a shell (passwordless sudo on both hosts).
|
||||||
|
ttm_raise_cap() {
|
||||||
|
local pages=$(( $1 * 256 )) p
|
||||||
|
for p in "$FRANK_ARCHON_TTM_SYS_DIR/pages_limit" \
|
||||||
|
"$FRANK_ARCHON_TTM_SYS_DIR/pages_max"; do
|
||||||
|
[ -e "$p" ] || continue
|
||||||
|
if [ -w "$p" ]; then
|
||||||
|
echo "$pages" > "$p" 2>/dev/null || true
|
||||||
|
elif command -v sudo >/dev/null 2>&1; then
|
||||||
|
sudo -n sh -c "echo $pages > '$p'" >/dev/null 2>&1 || true
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# Returns 0 when the cap clears FRANK_ARCHON_TTM_MIN_MIB, raising it first if
|
||||||
|
# needed; returns 1 with a named refusal when it cannot be made to.
|
||||||
|
ttm_ensure_cap() {
|
||||||
|
local cap
|
||||||
|
cap="$(ttm_cap_mib)"
|
||||||
|
if [ "$cap" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
|
||||||
|
echo "frank-archon: ttm pages cap is ${cap} MiB, below the pinned-ledger" \
|
||||||
|
"floor $FRANK_ARCHON_TTM_MIN_MIB — raising the knob to" \
|
||||||
|
"$FRANK_ARCHON_TTM_TARGET_MIB MiB ($(( FRANK_ARCHON_TTM_TARGET_MIB * 256 )) pages)" >&2
|
||||||
|
ttm_raise_cap "$FRANK_ARCHON_TTM_TARGET_MIB"
|
||||||
|
cap="$(ttm_cap_mib)"
|
||||||
|
fi
|
||||||
|
if [ "$cap" -lt "$FRANK_ARCHON_TTM_MIN_MIB" ]; then
|
||||||
|
echo "ERROR: ttm pages cap is ${cap} MiB, the pinned ledger needs >= $FRANK_ARCHON_TTM_MIN_MIB" >&2
|
||||||
|
echo "frank-archon: raise $FRANK_ARCHON_TTM_SYS_DIR/pages_limit (and pages_max if the kernel exposes one) to >= $(( FRANK_ARCHON_TTM_MIN_MIB * 256 )) in 4 KiB pages; a read-only knob needs the module option (options ttm pages_limit=N) and a reboot" >&2
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
echo "frank-archon: ttm pages cap ${cap} MiB clears the pinned-ledger floor $FRANK_ARCHON_TTM_MIN_MIB" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
# Executed (not sourced) it is the privileged pre-start hook: systemd runs this
|
||||||
|
# as root from the unit's `ExecStartPre=+…`, where the knob write is legal and
|
||||||
|
# the `Restart=always` loop can never outrun it.
|
||||||
|
if [ "${BASH_SOURCE[0]}" = "${0}" ]; then
|
||||||
|
set -euo pipefail
|
||||||
|
case "${1:-}" in
|
||||||
|
--raise)
|
||||||
|
TTM_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
# shellcheck source=/dev/null
|
||||||
|
source "${FRANK_ARCHON_ENV_FILE:-$TTM_DIR/frank-archon.env}"
|
||||||
|
ttm_ensure_cap
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "usage: ${0##*/} --raise (raise the ttm pages cap, then verify)" >&2
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
@@ -452,7 +452,8 @@ access is decided entirely by GROUP MEMBERSHIP consumed downstream —
|
|||||||
- `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
|
- `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
|
||||||
every human account).
|
every human account).
|
||||||
- **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
|
- **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
|
||||||
debo, freya — kirby/ry/nic/debo/freya (pk 17–21) created 2026-10-09, kassie
|
debo, freya, mitch, eric — mitch (pk 22) and eric (pk 23) created
|
||||||
|
2026-10-15, kirby/ry/nic/debo/freya (pk 17–21) created 2026-10-09, kassie
|
||||||
and alex created 2026-10-08, artem (pk 16) created 2026-10-07): `username` +
|
and alex created 2026-10-08, artem (pk 16) created 2026-10-07): `username` +
|
||||||
`<user>@franksdank.cloud` + display name +
|
`<user>@franksdank.cloud` + display name +
|
||||||
`is_active=True` + the password set via `set_password`, membership in
|
`is_active=True` + the password set via `set_password`, membership in
|
||||||
|
|||||||
@@ -0,0 +1,144 @@
|
|||||||
|
# frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, 2× R9700 TP=2
|
||||||
|
|
||||||
|
**Status: LIVE — deployed 2026-10-15.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
|
||||||
|
|
||||||
|
The in-house w4dn8 conversion of the Swift1.5 finetune
|
||||||
|
([[frank-archon-radiance-swift15-w4dn8]]) is replaced by the **engine
|
||||||
|
author's published container**:
|
||||||
|
[`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`](https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe)
|
||||||
|
— the official `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint rad-converted by
|
||||||
|
StillDeadcode. Same port, alias, container name, systemd unit, digest-pinned
|
||||||
|
engine pin **1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…`), and
|
||||||
|
**every serving argument carried verbatim** from the 2026-10-13 shape.
|
||||||
|
Rollback: revert the swap commit — the w4dn8 `.rad` stays on disk.
|
||||||
|
|
||||||
|
## The payload
|
||||||
|
|
||||||
|
`qwen3.8-next-flash-fp8-iq4r-moe.rad` — **121,969,901,568 B = 116,322 MiB**,
|
||||||
|
sha256 `0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20`
|
||||||
|
(Hub LFS oid), fetched byte-exact by `scripts/download-frank-archon-iq4r.sh`
|
||||||
|
(hfget resume + size + sha256; staging dir
|
||||||
|
`/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe/` keeps the
|
||||||
|
card, LICENSE, and recipe `qwen4exp-w4nl64-i8-hc8m` as provenance).
|
||||||
|
|
||||||
|
| Plane | Encoding (publisher recipe `qwen4exp-w4nl64-i8-hc8m`) |
|
||||||
|
| --- | --- |
|
||||||
|
| Routed experts | 4-bit codes into a 16-level **non-uniform** codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes by GPTQ (10M-token calibration, per-expert down-proj Hessians) + 3 coordinate-descent sweeps |
|
||||||
|
| Protected experts | **ten** per layer, whole-expert bf16 (down-projection energy) |
|
||||||
|
| Trunk + lm_head | int8 W8A8, scale per 128 columns searched; HC mixtures E4M3 |
|
||||||
|
| PLE n-gram | fp8 fixed-scale, in-container (served `--ngram-placement ram`, same 47.7 GiB pin law) |
|
||||||
|
| Speculator | the model's own MTP head (card default depth 3; served at 2) |
|
||||||
|
| Vision | bf16 vision tower, in-container |
|
||||||
|
|
||||||
|
**Base model: the OFFICIAL (censored) Qwen3.8-Flash-Next** — the fleet's
|
||||||
|
only frontier model at base alignment; the abliterated family lives on
|
||||||
|
frank-dragoon. Card quality: full-vocab teacher-forced KL vs bf16 over
|
||||||
|
~75K positions — mean 0.0789 all-tokens / 0.0239 assistant turns, top-1
|
||||||
|
agreement 92.85 %.
|
||||||
|
|
||||||
|
## Why this swap
|
||||||
|
|
||||||
|
- **Publisher-graded precision**: the w4nl codebook + coordinate-descent
|
||||||
|
refinement is the engine author's own quality ceiling for this
|
||||||
|
checkpoint; the card's KL ladder (p99 1.65, p99.9 5.33) is published and
|
||||||
|
reproducible, where our w4dn8 gate measured against our own reference.
|
||||||
|
- **Formal parity**: the served file is the one the engine author supports
|
||||||
|
— decode/prefill behavior on 1.0.13 is their tested path (the card's run
|
||||||
|
line speaks the 1.0.13-era flags; the `--vram-weights-mib`/`--vram-kv-mib`
|
||||||
|
pair we ship is accepted by pin 1.0.13 per the 2026-10-06 A/B, so the
|
||||||
|
shape needed no translation).
|
||||||
|
- **Cost**: the Swift1.5 finetune's refusal-bending behavior leaves this
|
||||||
|
endpoint (it returns to the official base alignment); the 4-slot/200k
|
||||||
|
economics and all shape laws carry over unchanged.
|
||||||
|
|
||||||
|
## Shape (unchanged from 2026-10-13)
|
||||||
|
|
||||||
|
All of it, verbatim — the payload moved 116,416 → 116,322 MiB, inside the
|
||||||
|
same budget:
|
||||||
|
|
||||||
|
- `--tp 2 --tp-wire wht6`, `--max-num-seqs 3`, `--max-model-len 200000`
|
||||||
|
(capacity contract 3 × 200,000), `--kv-cache-dtype fp8`
|
||||||
|
- `--placement expert_tiered --host-pool-mib 44928` + explicit
|
||||||
|
`--vram-weights-mib 25600 --vram-kv-mib 5120` (30,720 of the engine's
|
||||||
|
30,969 MiB ceiling; pool = contract + 1.3 %)
|
||||||
|
- `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`
|
||||||
|
(MNBT 2048 law), prefix cache 8 GiB host + 64 GiB disk,
|
||||||
|
`--ngram-placement ram`, froggeric v22.5 `--override-chat-template`
|
||||||
|
- TTM preflight floor 150,000 / target 204,800 MiB; pinned ledger
|
||||||
|
2 × 44 GiB pool + 47.7 GiB PLE + 8 GiB prefix ≈ 147 GiB (see
|
||||||
|
[[frank-archon-radiance-swift15-w4dn8]] for the ledger arithmetic and
|
||||||
|
the KV dtype law — both are engine properties, payload-independent)
|
||||||
|
|
||||||
|
The publisher's card run line (`--host-pool-mib 12288`,
|
||||||
|
`--expert-vs-cache-ratio 0.82`, seqs 8, spec 3, ~32 GB host RAM) is their
|
||||||
|
shape on stock 32 GB cards; the fleet ships its own measured shape against
|
||||||
|
the same file.
|
||||||
|
|
||||||
|
## Cutover record — 2026-10-15
|
||||||
|
|
||||||
|
1. `rm` (Frank's one-time authorization): the raw
|
||||||
|
`ukisai/Swift1.5-Qwen3.8-Flash-Next` BF16 checkpoint (336 GiB, spent
|
||||||
|
conversion source) and `qwen4exp-fp8-native.rad` (173 GiB, two eras
|
||||||
|
back) left `/home/dank/models` — 150 GB free → 658 GB free. The fp8-native
|
||||||
|
payload is **no longer on disk**: rebuild =
|
||||||
|
`scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh`.
|
||||||
|
2. Fetch: `scripts/download-frank-archon-iq4r.sh --fetch` as transient unit
|
||||||
|
`iq4r-fetch` (detached-job protocol; log
|
||||||
|
`/home/dank/logs/iq4r-fetch.log`, rc sentinel `…rc`).
|
||||||
|
3. Deploy: `scripts/deploy.sh dankserver --service frank-archon` — launched
|
||||||
|
detached (the swap restarts the very endpoint the operator session runs
|
||||||
|
on; the harness timeout catches the session on return).
|
||||||
|
4. **Boot ledger (2026-10-15, first boot, pin 1.0.13 — the w4nl codebook
|
||||||
|
loaded, no pin bump)**: `kv_attn` fp8 page **26.00 KiB** at **607,740
|
||||||
|
tokens** (151,935 blocks = 3.77 GiB) — byte-identical to the w4dn8 boot,
|
||||||
|
the sizing law carries; `kv_qsa_bkey` 3.25 KiB at the same token count;
|
||||||
|
`kv_gdn_state` 54 MiB × 6 slots, `kv_gdn_conv` 1.76 MiB × 3,
|
||||||
|
`kv_qsa_tail` 19.5 KiB × 6; pool **5.00 GiB**, prefix checkpoints 8 ×
|
||||||
|
56 MiB; the sizing line reads `kv cache 5.00 GiB … backs 3 sequences all
|
||||||
|
at 200000 tokens at once, which is the worst case`; samplers print **9
|
||||||
|
rows** per rank (3 × (1+2), inside the wht6 25-row bound);
|
||||||
|
`speculation: 'mtp' drafts 2 token(s) a step in 2 serial pass(es)` +
|
||||||
|
prompt-lookup fallback; slab **17,610 (rank0) / 17,658 (rank1) slots**
|
||||||
|
(64 shadow) — above the w4dn8 record of 17,548 (ten bf16-protected
|
||||||
|
experts per layer replace twenty-four, so more of the 4-bit table
|
||||||
|
resides), pinned pool 7,532/7,484 slots; KV lending **4.22 GiB**
|
||||||
|
(3,571/3,568 mover slots); arenas 896.02/988.17 MiB charged before the
|
||||||
|
pair split; prefix cache RECOVERED 900,176 blocks + 126 checkpoint slots
|
||||||
|
from the 64 GiB disk tier (same arch + tokenizer as w4dn8 — the tier
|
||||||
|
survived the payload swap); froggeric template 28,234 B; vision encoder
|
||||||
|
live (≤ 4,194,304 px, 16,384 patches/pass). Health gate passed in ~100 s
|
||||||
|
(radiance binds the port after the full load).
|
||||||
|
5. **Vision probe (2026-10-15)**: 300 px solid-red PNG via `image_url` —
|
||||||
|
reasoning stream: "a solid fill of a deep, saturated red hue". Content
|
||||||
|
came back EMPTY at `max_tokens 64`: the thinking budget consumed all 64
|
||||||
|
(`reasoning_tokens: 64`, finish `length`) — fleet law, probe thinking
|
||||||
|
models with ≥ a few hundred tokens.
|
||||||
|
6. **e2e gate**: deploy exited DEPLOY INCOMPLETE — 30/33 Playwright green,
|
||||||
|
3 red (`12-crawl`, `13-sso` search SSO, `16-marreta`) on DANKCLAW
|
||||||
|
surfaces (vhosts intermittently returned `ERR_CONNECTION_RESET` from the
|
||||||
|
workstation); frank-archon's own health gate passed. The swap touched no
|
||||||
|
dankclaw runtime file (only homepage services.json text). Census of the
|
||||||
|
Caddy/auth stack pending; see `log.md`.
|
||||||
|
7. **Post-boot measurements (pending)**: decode ladder (99.0 t/s single
|
||||||
|
@ 43.5k, 93.3 @ 197k, 3-way 113.9/43.3/10.9 = 18.4 t/s aggregate —
|
||||||
|
these are the **w4dn8** numbers, the record to beat, not this file's),
|
||||||
|
slab-limited prefill (22208/3,397 → 25600/3,584 t/s A-cold 43.5k), draft
|
||||||
|
acceptance, and `scripts/probe-frank-archon-kv.py` leg D.
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
Revert the swap commit (`scripts/frank-archon.env` MODEL_FILE line) +
|
||||||
|
redeploy: `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad`
|
||||||
|
(116,416 MiB) stays on disk, KLD-gated, same shape. The fp8-native era is
|
||||||
|
rebuild-only (see the cutover record). The w4dn8 runbook keeps every shape
|
||||||
|
law, the KV dtype table, the slab ladder, and the tuning levers:
|
||||||
|
[[frank-archon-radiance-swift15-w4dn8]].
|
||||||
|
|
||||||
|
## Related
|
||||||
|
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — the replaced payload and the
|
||||||
|
full shape-law library (slab slope, KV dtype arithmetic, TTM law)
|
||||||
|
- [[frank-archon-radiance-qwen38-native-fp8]] — the fp8-native era
|
||||||
|
(rebuild-only now; owns the descriptor-selection law)
|
||||||
|
- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
|
||||||
|
- [[deployment]] — the service-scoped deploy this cutover rides
|
||||||
@@ -145,11 +145,18 @@ dtype`) so a future resize cannot silently under-provision the pool.
|
|||||||
|
|
||||||
### Why the weights slab pays — and why that is not pure loss
|
### Why the weights slab pays — and why that is not pure loss
|
||||||
|
|
||||||
The two VRAM budgets are one budget: card 31,856 MiB, minus the 1,710 MiB a
|
The two VRAM budgets are one budget, and the ceiling is the engine's own — not a
|
||||||
boot consumes outside both (driver context, the 990 MiB activation arena,
|
hand-picked reserve. Card 32,624 MiB, and the table that prints at the pair
|
||||||
hardware queues, headroom, step staging), minus 1,024 MiB kept free ⇒
|
24,000/5,120 says `3.42 GiB is neither budgeted nor accounted — the driver's
|
||||||
**`weights + kv ≤ 29,120 MiB`**. Paying for a 6,912 MiB pool therefore cuts the
|
context, the activation arena and --gpu-headroom-mib come out of it; anything
|
||||||
resident slab 26,071 → 22,208 MiB (−14.8 %), and the evicted experts spill
|
past those is throughput on the floor`, with `card 0 settled at 2.01 GiB free
|
||||||
|
against 209.19 MiB (the 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the
|
||||||
|
static region)`. Claimable reserve above the ENGINE's floor:
|
||||||
|
2,058 − 209 = 1,849 MiB ⇒ **`weights + kv ≤ 30,969 MiB`**. (The 29,120 bound
|
||||||
|
written on 2026-10-11 was my own 1 GiB "keep free" reserve, measured off a card
|
||||||
|
figure of 31,856 MiB; the engine never asked for that reserve and the seam now
|
||||||
|
derives the number from its table.) Paying for a 6,912 MiB pool therefore cuts
|
||||||
|
the resident slab 26,071 → 22,208 MiB (−14.8 %), and the evicted experts spill
|
||||||
into the pinned host pool (40,960 → 44,928 MiB per rank; ledger: pinned pool
|
into the pinned host pool (40,960 → 44,928 MiB per rank; ledger: pinned pool
|
||||||
slots 7,190 → 10,375). The pinned-RAM ledger rises to ≈147 GiB, so the TTM
|
slots 7,190 → 10,375). The pinned-RAM ledger rises to ≈147 GiB, so the TTM
|
||||||
floor moved 140,000 → 150,000 MiB (the kernel cap is 204,800).
|
floor moved 140,000 → 150,000 MiB (the kernel cap is 204,800).
|
||||||
@@ -253,6 +260,240 @@ from the scored configuration — bf16 can only reduce the attention-state
|
|||||||
error, and a re-score is a drained-window job if the gate agreement is ever
|
error, and a re-score is a drained-window job if the gate agreement is ever
|
||||||
needed in the record.
|
needed in the record.
|
||||||
|
|
||||||
|
## 2026-10-13 — the power failure, the ttm reset, and 4 × 200,000 fp8 KV
|
||||||
|
|
||||||
|
### The outage: a refusal, not a crash
|
||||||
|
|
||||||
|
Dankserver lost power on 2026-10-12 ~23:17. The box came back; frank-archon
|
||||||
|
never did. The unit sat in `activating (auto-restart)`, exiting 1 about 50 ms
|
||||||
|
after starting, with one journal line on repeat:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ERROR: ttm pages cap is 125988 MiB, the pinned ledger needs >= 150000
|
||||||
|
frank-archon: raise /sys/module/ttm/parameters/pages_limit …
|
||||||
|
```
|
||||||
|
|
||||||
|
**6,919 refusals in ~20 hours.** No container was created (`podman ps -a` showed
|
||||||
|
the name with a zero start time), so there was no engine, no port, and no engine
|
||||||
|
log — the signature of a preflight refusal, which every deployer/monitor view
|
||||||
|
outside the journal renders as an unexplained crash loop.
|
||||||
|
|
||||||
|
The cause is a class, not an incident. The pinned-RAM ledger (~147 GiB: 2 × 44 GiB
|
||||||
|
expert pools + the 47.7 GiB PLE table + the 8 GiB prefix host tier) rides TTM's
|
||||||
|
shared pages, capped by `/sys/module/ttm/parameters/pages_limit`. **That knob is
|
||||||
|
runtime state, and its default is half of RAM** — so the manual raise of
|
||||||
|
2026-10-05 (52,428,800 pages = 200 GiB) died with the power failure and the cap
|
||||||
|
fell back to 32,253,039 pages = 125,988 MiB, 24 GiB below the floor the serve
|
||||||
|
script enforces. The fp8-native era had planned the persistent half of this
|
||||||
|
("add `ttm.pages_limit=52428800` to the kernel cmdline at the next scheduled
|
||||||
|
reboot", [[frank-archon-radiance-qwen38-native-fp8]]); it was never applied, and
|
||||||
|
a power failure is not scheduled.
|
||||||
|
|
||||||
|
### The repair: the gate raises the knob itself
|
||||||
|
|
||||||
|
`scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) reads the cap as
|
||||||
|
the MIN over the knobs the running kernel exposes, and when it is below
|
||||||
|
`FRANK_ARCHON_TTM_MIN_MIB` it writes `FRANK_ARCHON_TTM_TARGET_MIB` (204,800 MiB =
|
||||||
|
52,428,800 pages) into every writable knob, **re-reads**, and only then refuses.
|
||||||
|
The unit is root and the knob is a 0644 root-owned module parameter, so the write
|
||||||
|
is legal; a kernel that refuses the move produces the same named refusal as
|
||||||
|
before, and a reboot is now a boot rather than a visit to the machine.
|
||||||
|
|
||||||
|
| Knob | Power-failure default | After the gate |
|
||||||
|
|---|---|---|
|
||||||
|
| `/sys/module/ttm/parameters/pages_limit` | 32,253,039 pages = 125,988 MiB | 52,428,800 pages = 204,800 MiB |
|
||||||
|
|
||||||
|
Seam-tested, not inspection-tested: *the ttm gate raises a reset pages cap and
|
||||||
|
refuses when the write fails* and *the ttm cap counts only the knobs the kernel
|
||||||
|
exposes* drive the gate against a temp knob directory — a below-floor writable
|
||||||
|
knob ends at the target and the start passes; a read-only knob leaves the cap
|
||||||
|
untouched and the start refuses by name; an absent `pages_max` is skipped, never
|
||||||
|
read as zero (a zero reading would refuse a healthy host).
|
||||||
|
|
||||||
|
### The shape: 4 slots × 200,000, fp8 KV (same day, superseded by the re-cuts below)
|
||||||
|
|
||||||
|
Answers the open call of 2026-10-11 from the other side: Frank asked for **four**
|
||||||
|
concurrent 200,000-token conversations with quantized KV.
|
||||||
|
|
||||||
|
fp8 is not a preference at four slots, it is the only dtype that affords them.
|
||||||
|
The pool must back `seqs × max-model-len` (the engine's hard limit), so the
|
||||||
|
fourth slot is bought in KV, out of the one per-card budget
|
||||||
|
(`weights + kv ≤ 29,120 MiB`, the bound as it was understood then):
|
||||||
|
|
||||||
|
| dtype | pool for 4 × 200,000 | weights slab left |
|
||||||
|
|---|---|---|
|
||||||
|
| bf16 | 10,791 + 440 + 448 = 11,679 MiB | 17,441 MiB (−33 % vs the fp8-era 26,071) |
|
||||||
|
| **fp8 (deployed)** | 5,713 + 440 + 448 = 6,601 MiB | **22,208 MiB — unchanged** |
|
||||||
|
|
||||||
|
So `--vram-weights-mib 22208` / `--vram-kv-mib 6912` stay exactly where the
|
||||||
|
2026-10-11 change put them (pair sum = the measured 29,120 ceiling), the
|
||||||
|
44,928 MiB/rank pinned host pool is untouched, and the RAM/TTM ledger does not
|
||||||
|
move. What the fp8 pool buys: `(6912 − 110×4 − 448) × 1024 / 7.3125` =
|
||||||
|
**843,566 predicted tokens** = four full-height conversations with 5.4 % slack
|
||||||
|
(the bf16 pool backed 462,752 = two; the 2026-10-08 fp8 pool backed 386,904 =
|
||||||
|
1.93).
|
||||||
|
|
||||||
|
Terms that move with the dtype and the slot count:
|
||||||
|
|
||||||
|
- **Per-slot recurrent state** is dtype-independent and lives inside the pool:
|
||||||
|
+110 MiB per slot, 440 MiB for four (already in the arithmetic above).
|
||||||
|
- **The fused TP all-reduce** is declared over `seqs × (1 + spec)` rows: 4 × 3 =
|
||||||
|
**12 rows**, still inside the 25-row bound at which the `wht6` wire is
|
||||||
|
bit-exact — so four slots cost nothing in wire exactness.
|
||||||
|
- **Prefix-cache entries shrink** with the attention plane (110.50 KiB at bf16 →
|
||||||
|
58.50 KiB at fp8), so the 8 GiB host tier parks ~5.8 sessions of 200k where at
|
||||||
|
bf16 it parked ~2.9 — four live slots plus a parked set fit with no re-size.
|
||||||
|
- **`models.yml` is untouched**: `contextWindow` quotes the per-request window
|
||||||
|
(200,000), which did not change; only the concurrency did.
|
||||||
|
|
||||||
|
Expectation carried from the 2026-10-11 A/B, whose arms bracket this shape
|
||||||
|
(same slab 22,208): prefill is set by the slab, not the dtype, so ~3.1k t/s
|
||||||
|
cold / ~3.3k warm; decode sits in the 94–118 t/s band with draft acceptance
|
||||||
|
~0.73–0.75. The 4,379 t/s fp8 prefill of the 2026-10-08 control belongs to the
|
||||||
|
26,071 MiB slab and does not come back by quantizing KV.
|
||||||
|
|
||||||
|
### Boot ledger (2026-10-13 19:21, four slots, fp8 — superseded same evening)
|
||||||
|
|
||||||
|
The sizing law's prediction, checked against the engine's own arithmetic:
|
||||||
|
predicted 843,566 tokens, ledger **843,276** (0.03 % block rounding).
|
||||||
|
|
||||||
|
| Group | Page / state | Size | Tokens |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `kv_attn` | 13 layers, 4-token blocks, **26.00 KiB** (fp8 page) | 5.23 GiB | **843,276** |
|
||||||
|
| `kv_qsa_bkey` | 13 layers, 4-token blocks, **3.25 KiB** (dtype-fixed) | 669.10 MiB | **843,276** |
|
||||||
|
| `kv_gdn_state` | 36 layers, 54.00 MiB, **8 slots** (4 × 2 copies) | 432.00 MiB | per-slot |
|
||||||
|
| `kv_gdn_conv` / `kv_ple_*` | 36 + 1 + 1 layers, 4 slots | 7.91 MiB | per-slot |
|
||||||
|
|
||||||
|
`kv cache 6.75 GiB … backs 4 sequences all at 200000 tokens at once, which is
|
||||||
|
the worst case` — the capacity contract, in the engine's words. The rest of the
|
||||||
|
boot is the fp8-era picture: `vram_weights` 21.69 GiB filled to 147 KiB (rank 0)
|
||||||
|
/ 355 KiB (rank 1) of slack, `unclaimed` 2.03 GiB, cards settled at 1.99 /
|
||||||
|
2.00 GiB free, `mover: 4967 slab slots (64 shadow), 10375 pinned pool slots,
|
||||||
|
4.24 GiB static vram`, and the half-refund in plain sight: `4967 slots can be
|
||||||
|
lent out of the KV cache (5.87 GiB)` (the fp8-era 3,543 MiB pool lent 2.69 GiB).
|
||||||
|
|
||||||
|
Two ledger lines that confirm the derived consequences: the sampler prints
|
||||||
|
**12 rows** (4 sequences × (1 + 2 speculative) — inside the 25-row bound where
|
||||||
|
`wht6` is bit-exact), and the prefix/tier arithmetic runs on the fp8 entry size
|
||||||
|
(58.50 KiB, half of the bf16 entry).
|
||||||
|
|
||||||
|
### The same-day re-cuts: 4 → 3 slots, then the slab takes the card's slack
|
||||||
|
|
||||||
|
Frank's call, twice in one evening. Both are one-env-pair changes, and both were
|
||||||
|
read off the boot ledger rather than assumed.
|
||||||
|
|
||||||
|
**4 → 3 slots (22208/6912 → 24000/5120).** Three slots need
|
||||||
|
3 × 200,000 at fp8 = 4,285 + 330 + 448 = **5,063 MiB** of pool; the 6,912 pool
|
||||||
|
was backing 858,969 tokens — 43 % more window than the contract asks for.
|
||||||
|
Cutting it to 5,120 (predicts 608,027; ledger **607,740**, 0.05 %) hands
|
||||||
|
1,792 MiB to the slab and drops the all-reduce to 9 rows.
|
||||||
|
|
||||||
|
**24,000 → 25,600 (the card's settling slack).** The 24,000 boot printed
|
||||||
|
`card 0 settled at 2.01 GiB free against 209.19 MiB (the 96.00 MiB
|
||||||
|
--gpu-headroom-mib and 113.19 MiB kept for the static region)` — 1,849 MiB above
|
||||||
|
the ENGINE's own floor, with the budget table calling the surplus `throughput on
|
||||||
|
the floor`. Claiming 1,600 MiB of it puts the pair at 30,720 and the ledger at
|
||||||
|
`unclaimed 477.30 MiB` (cards ~336 MiB free, floor 209.19).
|
||||||
|
|
||||||
|
| pair | slab slots (rank 0) | pinned pool slots | A-cold 43.5k | concurrent 197k |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| 22208/6912, seqs 4 | 14,753 | 10,375 | 3,397 t/s | 3,307 t/s (4-way) |
|
||||||
|
| 24000/5120, seqs 3 | 16,235 | 8,893 | 3,290 t/s | 3,051 t/s (3-way) |
|
||||||
|
| **25600/5120, seqs 3** | **17,548** | 7,580 | **3,584 t/s** | **3,589 t/s** (3-way) |
|
||||||
|
| 26071/3543, seqs 3 (2026-10-11 control) | — | 7,190 | 4,379 t/s | — |
|
||||||
|
|
||||||
|
The claim shipped with the 25,600 arm — that the slab is a THRESHOLD below
|
||||||
|
~26 GiB so the extra 1.6 GiB buys ~0 — is **falsified by its own measurement**:
|
||||||
|
+8.9 % single-stream, +17.6 % concurrent, both outside the ±3 % noise floor. Two
|
||||||
|
slab points that land inside noise of *each other* (22,208 vs 24,000) are not a
|
||||||
|
plateau; the pinned host pool is the slow path, so every MiB moved from RAM onto
|
||||||
|
the card is throughput (the ledger shows the transfer directly: slab
|
||||||
|
+1,313 slots, pinned pool −1,313).
|
||||||
|
|
||||||
|
The cliff is real, it just sits above 25,600: 26,071 measured 4,379 t/s, +22 %
|
||||||
|
for +471 MiB. At three slots that is out of reach (26,112 + 5,120 = 31,232 > the
|
||||||
|
30,969 ceiling). At **two** slots the pool falls to 3,072 and the slab reaches
|
||||||
|
**27,897** — the ceiling exactly, slab above the cliff. That arm is untested and
|
||||||
|
it is a capacity decision, not a tuning one.
|
||||||
|
|
||||||
|
Capacity contract, live-verified on the 3-slot shape: three concurrent prompts
|
||||||
|
of **199,984 tokens** each (599,952 tokens) were accepted and completed on
|
||||||
|
`/v1/completions`; a fourth full-height request is refused as "too large to ever
|
||||||
|
fit" — radiance never truncates.
|
||||||
|
|
||||||
|
### The decode ladder (2026-10-13): slots are paid in concurrency, not prefill
|
||||||
|
|
||||||
|
Frank's challenge — "why are you not tracking decode?" — was right, and the
|
||||||
|
answer is a number now. 512 generated tokens (the bench law), greedy, cold
|
||||||
|
prompts, this shape (3 x 200,000, fp8, slab 25,600):
|
||||||
|
|
||||||
|
```
|
||||||
|
1-stream 43.5k: decode 99.0 t/s (prefill 4,042, accept 0.952)
|
||||||
|
1-stream 197k: decode 93.3 t/s (prefill 4,826, accept 0.938)
|
||||||
|
3-stream 43.5k: decode 113.9 / 43.3 / 10.9 t/s wall 83.4s
|
||||||
|
3-stream 197k: decode 88.9 / 16.3 / 64.6 t/s wall 122.4s
|
||||||
|
```
|
||||||
|
|
||||||
|
Two laws came out of it. **Decode is nearly window-free** — 99.0 at 43.5k,
|
||||||
|
93.3 at 197k, −6 % for 4.5x the depth — so a long conversation costs
|
||||||
|
prefill, not tokens-per-second. **Concurrency is where the slot count is
|
||||||
|
paid**: three simultaneous streams give 18.4 t/s aggregate at 43.5k and 12.5
|
||||||
|
at 197k against 99 single-stream, and the split is a lottery
|
||||||
|
(113.9/43.3/10.9), not a fair share — the fused-step schedule hands the step
|
||||||
|
to whoever it reached first, so one stream finishes at 113 t/s while its
|
||||||
|
sibling crawls at 10.9. That is what "3 slots" owes per token, and no prefill
|
||||||
|
number shows it.
|
||||||
|
|
||||||
|
Corpus caveat, stated at the claim: this probe's filler is a repetitive
|
||||||
|
ladder, so MTP acceptance is 0.93–0.97 against the 0.740 of the 2026-10-11
|
||||||
|
table below; these rows flatter decode relative to that table and are the
|
||||||
|
baseline for *this* corpus, not a replacement for those rows.
|
||||||
|
`probe-frank-archon-kv.py` now carries this leg as **D** (leg B generates 24
|
||||||
|
tokens, which is why a day of shape work priced only prefill).
|
||||||
|
|
||||||
|
### Recovery evidence
|
||||||
|
|
||||||
|
```text
|
||||||
|
Oct 13 19:21:08 dankserver ttm-preflight.sh[1623778]: frank-archon: ttm pages cap
|
||||||
|
is 125988 MiB, below the pinned-ledger floor 150000 — raising the knob to
|
||||||
|
204800 MiB (52428800 pages)
|
||||||
|
Oct 13 19:21:08 dankserver ttm-preflight.sh[1623778]: frank-archon: ttm pages cap
|
||||||
|
204800 MiB clears the pinned-ledger floor 150000
|
||||||
|
Oct 13 19:21:08 dankserver systemd[1]: Started frank-archon.service
|
||||||
|
```
|
||||||
|
|
||||||
|
`cat /sys/module/ttm/parameters/pages_limit` → 52,428,800 (204,800 MiB) after
|
||||||
|
the boot, i.e. the root `ExecStartPre` hook did the write the service process
|
||||||
|
(User=dank) cannot.
|
||||||
|
|
||||||
|
### Capacity probe (2026-10-13, live service)
|
||||||
|
|
||||||
|
`scripts/probe-frank-archon-kv.py` now reads the slot count, window and KV dtype
|
||||||
|
from the deployed env on the host, so the probe cannot drift from the service:
|
||||||
|
it printed `shape: 4 x 200,000 tokens, kv fp8` and fired **four concurrent chats
|
||||||
|
of 196,960 prompt tokens each** (646,299 chars, 98.5 % of the window), each
|
||||||
|
carrying a marker whose first word is unique.
|
||||||
|
|
||||||
|
| Stream | prompt | wall | prefill | retrieval |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| B1 | 196,961 | 238.3 s | 1,423 t/s | exact, HIT |
|
||||||
|
| B2 | 196,960 | 98.0 s | 2,043 t/s | exact, HIT |
|
||||||
|
| B3 | 196,961 | 201.1 s | 993 t/s | exact, HIT |
|
||||||
|
| B4 | 196,960 | 149.9 s | 1,327 t/s | exact, HIT |
|
||||||
|
| **total** | **787,842** | **238.4 s** | **3,305 t/s aggregate** | 4/4 |
|
||||||
|
|
||||||
|
All four answered their own marker; nothing was preempted, refused, or truncated
|
||||||
|
(`finish=stop`, `cache_n=0`, so no prefix cache padded the result). Aggregate
|
||||||
|
prefill equals the single-stream rate measured in the same probe (3,397 t/s at
|
||||||
|
43.5k), i.e. the four streams share one prefill pipeline at the slab-limited
|
||||||
|
ceiling and the engine never ran short of KV. Prefix cache intact: the same
|
||||||
|
43.5k prompt repeated answered in 1.0 s with `cache_n=43008`.
|
||||||
|
|
||||||
|
The per-stream `decode` column is NOT a decode rate here — 8 generated tokens
|
||||||
|
make it first-token dominated (the bench law: generate ≥ 128 tokens to read a
|
||||||
|
rate). This probe answers one question only: does the endpoint owe four
|
||||||
|
full-height conversations at once, and answer each one exactly. It does.
|
||||||
|
|
||||||
## Gates (measured at cutover)
|
## Gates (measured at cutover)
|
||||||
|
|
||||||
- **KLD gate: PASS (2026-10-08 04:12)** — `kld-gate-swift15.sh`,
|
- **KLD gate: PASS (2026-10-08 04:12)** — `kld-gate-swift15.sh`,
|
||||||
|
|||||||
@@ -2,14 +2,18 @@
|
|||||||
|
|
||||||
**Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
|
**Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
|
||||||
(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
|
(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
|
||||||
driving `podman run` directly, root systemd unit, :7770): the OFFICIAL
|
driving `podman run` directly, root systemd unit, :7770):
|
||||||
`Qwen/Qwen3.8-Flash-Next-FP8` checkpoint at NATIVE block-FP8 precision
|
the **publisher's** 4-bit-codebook container of the official
|
||||||
(converted no-recipe to `qwen4exp-fp8-native.rad` — see the runbook for
|
`Qwen/Qwen3.8-Flash-Next` (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
the descriptor-staging law), on BOTH R9700s TP=2 (`--tp-wire wht6`),
|
sha256-verified at fetch — w4nl codebook experts + int8 trunk, PLE n-gram,
|
||||||
experts in a 40 GiB/rank pinned host pool + VRAM cache (ratio 0.82), the
|
MTP head and vision tower in-container), on BOTH R9700s
|
||||||
47.7 GiB PLE table pinned to host RAM, fp8 KV, MTP spec decode depth 3,
|
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
|
||||||
2 × 200,000 first boot. Rollback tags frank-archon-freetoken-pr587-20261005
|
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
|
||||||
and frank-archon-vllm-shapeG-20261003),
|
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
|
||||||
|
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
|
||||||
|
froggeric v22.5 template. Rollback: the in-house Swift1.5 w4dn8 `.rad` (on
|
||||||
|
disk); tags frank-archon-freetoken-pr587-20261005 and
|
||||||
|
frank-archon-vllm-shapeG-20261003),
|
||||||
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
||||||
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
|
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
|
||||||
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
|
||||||
@@ -36,10 +40,10 @@ graph LR
|
|||||||
gfx1201, 32 GiB each]
|
gfx1201, 32 GiB each]
|
||||||
B2[radiance StillDeadcode, digest-pinned image
|
B2[radiance StillDeadcode, digest-pinned image
|
||||||
podman run, frank-archon, port 7770]
|
podman run, frank-archon, port 7770]
|
||||||
C[Qwen3.8-Flash-Next FP8 OFFICIAL, native block-FP8
|
C[Publisher qwen3.8-next-flash-fp8-iq4r-moe .rad
|
||||||
2x R9700 TP=2 wht6, experts 40 GiB/rank pinned pool + VRAM,
|
2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM,
|
||||||
PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 3,
|
PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
|
||||||
2 x 200000, froggeric v22.5]
|
3 x 200000, froggeric v22.5]
|
||||||
end
|
end
|
||||||
|
|
||||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||||
@@ -152,15 +156,42 @@ frank-zealot, port 7771]
|
|||||||
target; payload on disk; owns the measured boot-pair table)**: the same
|
target; payload on disk; owns the measured boot-pair table)**: the same
|
||||||
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
|
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
|
||||||
104.8 tok/s, 3,571 MiB true free
|
104.8 tok/s, 3,571 MiB true free
|
||||||
- [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**:
|
- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE since 2026-10-15**:
|
||||||
|
the engine author's published container of the OFFICIAL (base-alignment)
|
||||||
|
Qwen3.8-Flash-Next — `qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
|
Hub-sha256-verified at fetch: 4-bit codes into a 16-level non-uniform
|
||||||
|
codebook after a Walsh–Hadamard rotation (GPTQ + 3 coordinate-descent
|
||||||
|
sweeps, 10M-token calibration), ten bf16-protected experts per layer,
|
||||||
|
int8 trunk/lm_head, fp8 HC + PLE, MTP head, bf16 vision tower; card KL
|
||||||
|
0.0789 all-tokens / top-1 92.85 % vs bf16. **MODEL_FILE is the only line
|
||||||
|
that moved** — the 2026-10-13 shape (25600/5120 pair, 44928 pool, fp8 KV
|
||||||
|
3 × 200,000, MTP 2, wht6, MNBT 2048, froggeric v22.5) carries verbatim.
|
||||||
|
Rollback: revert the swap commit (w4dn8 `.rad` stays on disk). The
|
||||||
|
fp8-native `.rad` and the raw Swift1.5 BF16 source were DELETED from
|
||||||
|
dankserver 2026-10-15 (Frank's one-time rm, volume for this payload;
|
||||||
|
rebuild via the fetch + build scripts named in the fp8-native runbook)
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — **RETIRED 2026-10-08 → 10-15
|
||||||
|
(superseded by the publisher iq4r swap; payload on disk as the rollback)**:
|
||||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
|
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
|
||||||
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
|
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
|
||||||
the FP8-native payload remains the rollback. **2026-10-11: the KV cache is
|
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
|
||||||
UNQUANTIZED (bf16) at 2 x 200,000** — pool 3543 → 6912 MiB (386,904 →
|
pair re-cut twice** — `--kv-cache-dtype fp8` first at
|
||||||
462,752 tokens), weights slab 26071 → 22208, host pool 44928,
|
`--max-num-seqs 4`/6912 MiB pool (843,566 tokens, slab 22208), then **3 slots
|
||||||
`--max-num-seqs 2`. bf16 costs 1.89x fp8 per token (the QSA block key is
|
with the pool cut to the contract (5120 MiB, 608,027 tokens) and the slab
|
||||||
dtype-fixed, so it is not a clean 2x); measured price is prefill (-29%),
|
raised 22208 → 24000 → 25600**, the last 1,600 MiB claimed out of the card's
|
||||||
NOT decode, and spec-decode acceptance rose 0.661 → 0.740.
|
settling slack: the engine's ceiling is **30,969 MiB**, not the 29,120 my
|
||||||
|
1 GiB reserve invented. fp8 is what funds the slots — the pool must back
|
||||||
|
`seqs × window`, and at bf16 four slots cost 11,679 MiB (slab 17,441, −33 %).
|
||||||
|
Measured on the slab: A-cold 43.5k prefill 22208 → 3,397, 24000 → 3,290,
|
||||||
|
**25600 → 3,584 t/s**; 3-way 197k aggregate 3,051 → **3,589 t/s**. A shipped
|
||||||
|
prediction that the slab was a threshold below 26 GiB died on its own
|
||||||
|
measurement — it is a slope, and 26,071 (4,379 t/s) is the cliff. The bf16
|
||||||
|
arm (2 × 200,000, 462,752 tokens, prefill −29 %, acceptance 0.661 → 0.740)
|
||||||
|
and the 4-slot fp8 shape are the documented rollbacks.
|
||||||
|
**2026-10-13 also: the pinned-ledger preflight now raises `ttm.pages_limit`
|
||||||
|
itself** — the 2026-10-12 power failure
|
||||||
|
reset the knob to its default (125,988 MiB) and the endpoint refused to boot
|
||||||
|
6,919 times in ~20 h; `scripts/ttm-preflight.sh` is the repair.
|
||||||
- [[frank-archon-radiance-qwen38-native-fp8]] — **RETIRED 2026-10-08 (the
|
- [[frank-archon-radiance-qwen38-native-fp8]] — **RETIRED 2026-10-08 (the
|
||||||
w4dn8 swap replaced it; its `.rad` stays on disk as the rollback)**
|
w4dn8 swap replaced it; its `.rad` stays on disk as the rollback)**
|
||||||
(root systemd unit, port 7770): the StillDeadcode **radiance** engine
|
(root systemd unit, port 7770): the StillDeadcode **radiance** engine
|
||||||
|
|||||||
@@ -182,12 +182,20 @@ The LLM inference fleet on the two hosts. Full index:
|
|||||||
### Deployment and Topology
|
### Deployment and Topology
|
||||||
|
|
||||||
- [[deployment]] — unified deploy and update process for both servers
|
- [[deployment]] — unified deploy and update process for both servers
|
||||||
- [[frank-archon-radiance-swift15-w4dn8]] — STAGED frank-archon runbook:
|
- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE frank-archon runbook
|
||||||
|
(2026-10-15)**: the PUBLISHER's container of the official
|
||||||
|
Qwen3.8-Flash-Next (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
|
||||||
|
w4nl 4-bit-codebook experts + int8 trunk, Hub-sha256-verified at fetch)
|
||||||
|
swapped in as a MODEL_FILE-only change — the 2026-10-13 shape (slab
|
||||||
|
25600 / fp8-KV 5120, 3 × 200,000, host pool 44,928, MTP depth 2) carries
|
||||||
|
verbatim; rollback = revert the swap commit (the w4dn8 `.rad` stays on
|
||||||
|
disk; the fp8-native `.rad` was deleted 2026-10-15, rebuildable)
|
||||||
|
- [[frank-archon-radiance-swift15-w4dn8]] — RETIRED 2026-10-08 → 10-15 frank-archon runbook:
|
||||||
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
|
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
|
||||||
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
|
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
|
||||||
activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
|
activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
|
||||||
= revert that line (the FP8-native `.rad` stays on disk)
|
= revert that line (the FP8-native `.rad` stays on disk)
|
||||||
- [[frank-archon-radiance-qwen38-native-fp8]] — the ACTIVE frank-archon
|
- [[frank-archon-radiance-qwen38-native-fp8]] — the RETIRED 2026-10-05 → 10-08 frank-archon
|
||||||
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
|
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
|
||||||
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
|
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
|
||||||
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
|
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
|
||||||
|
|||||||
@@ -1,5 +1,99 @@
|
|||||||
# Bundle Change Log
|
# Bundle Change Log
|
||||||
|
|
||||||
|
## 2026-10-13
|
||||||
|
- **frank-archon outage: a kernel knob, not the model.** The 2026-10-12 23:17
|
||||||
|
power failure on dankserver reset `/sys/module/ttm/parameters/pages_limit`
|
||||||
|
from the manual 200 GiB raise back to its default (half of RAM = 125,988 MiB),
|
||||||
|
below the ~147 GiB pinned-RAM ledger the engine needs; the serve script's
|
||||||
|
preflight named the refusal and exited 1, and `Restart=always` re-ran it
|
||||||
|
**6,919 times in ~20 h** — `activating (auto-restart)`, exit 1 in 50 ms, no
|
||||||
|
container, no engine log. Law adopted: **a sysfs raise is a countdown to the
|
||||||
|
next reboot**, so the gate raises the knob itself. New
|
||||||
|
`scripts/ttm-preflight.sh` is sourced by the serve script (verify, raise,
|
||||||
|
re-verify, then refuse by name) and is also the unit's
|
||||||
|
`ExecStartPre=+…--raise` root hook — `+` because the service is `User=dank`
|
||||||
|
and the knob is root-owned 0644, which is what the first attempt (a plain
|
||||||
|
write inside the serve script) got wrong. `FRANK_ARCHON_TTM_TARGET_MIB=204800`
|
||||||
|
records the host ceiling.
|
||||||
|
- **Shape: 4 × 200,000 with fp8 KV (Frank's call).** `--max-num-seqs 4`,
|
||||||
|
`--kv-cache-dtype fp8`, pool and slab untouched (6,912 / 22,208 MiB = the
|
||||||
|
measured 29,120/card ceiling): the pool must back `seqs × max-model-len`, and
|
||||||
|
at bf16 four slots cost 11,679 MiB, which would cut the resident expert slab
|
||||||
|
to 17,441 (−33 %) — fp8 costs 6,601 and leaves it alone. 4 × (1 + spec 2) = 12
|
||||||
|
fused-all-reduce rows, inside the 25-row bound where `wht6` is bit-exact.
|
||||||
|
`models.yml` needed no change: the per-request window is unchanged, only the
|
||||||
|
concurrency.
|
||||||
|
- **Deployed at `bae78e7`, gates green.** Boot ledger: `kv cache 6.75 GiB …
|
||||||
|
backs 4 sequences all at 200000 tokens at once` — 843,276 tokens against the
|
||||||
|
sizing law's 843,566 (0.03 %), sampler at 12 rows, weights filled to 147 KiB
|
||||||
|
of slack, cards ~2 GiB free, KV lending 5.87 GiB to the slab. Live probe
|
||||||
|
(now reading the shape from the deployed env): four concurrent 196,960-token
|
||||||
|
chats, **4/4 exact marker retrieval**, 787,842 tokens in 238.4 s = 3,305 t/s
|
||||||
|
aggregate prefill, no preemption; prefix cache verified (repeat answered in
|
||||||
|
1.0 s, `cache_n=43008`). Playwright e2e 32 passed + 1 flaky (the known Plex
|
||||||
|
homepage retry); archon bats seam 21/21 including three new ttm-gate tests.
|
||||||
|
- **Re-cut twice the same evening (Frank's calls), each read off the ledger.**
|
||||||
|
4 → 3 slots, pool 6,912 → 5,120: three slots need 5,063 MiB and the 6,912 pool
|
||||||
|
was funding 858,969 tokens — 43 % over the contract. Then slab
|
||||||
|
24,000 → 25,600, claiming 1,600 of the 1,849 MiB the engine leaves above its
|
||||||
|
OWN settle floor (`card 0 settled at 2.01 GiB free against 209.19 MiB (the
|
||||||
|
96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)`): the
|
||||||
|
real ceiling is 30,969 MiB, not the 29,120 my 1 GiB "keep free" reserve
|
||||||
|
invented on 2026-10-11. The pair now spends 30,720 and settles
|
||||||
|
`unclaimed 477.30 MiB` (cards ~336 MiB free).
|
||||||
|
- **The prediction shipped with that arm was falsified by its own measurement,
|
||||||
|
which is the whole point of banking the A/B.** Claim: the slab is a THRESHOLD
|
||||||
|
below ~26 GiB (22,208 → 3,397 and 24,000 → 3,290 t/s are inside noise of each
|
||||||
|
other), so 1.6 GiB buys ~0. Measured at 25,600: **3,584 t/s** single-stream
|
||||||
|
cold at 43.5k (+8.9 %) and **3,589 t/s** 3-way aggregate at 197k (wall
|
||||||
|
193.7 → 164.5 s, +17.6 %) — slab 16,235 → 17,548 slots while the pinned host
|
||||||
|
pool hands the same experts back (8,893 → 7,580). The slab is a SLOPE; two
|
||||||
|
points inside noise of each other are not a plateau. The cliff is above
|
||||||
|
25,600: 26,071 measured 4,379 t/s (+22 % for +471 MiB), unreachable at three
|
||||||
|
slots (26,112 + 5,120 > the ceiling) and free at two.
|
||||||
|
- **Capacity verified live at 3 slots:** three concurrent 199,984-token prompts
|
||||||
|
accepted and completed (599,952 tokens resident), ledger
|
||||||
|
`backs 3 sequences all at 200000 tokens at once` at 607,740 tokens against the
|
||||||
|
law's 608,027. A fourth full-height request is refused as too large to ever
|
||||||
|
fit — radiance never truncates. Probe law: the oversized-prompt counter is
|
||||||
|
FREE (the refusal fires when PROMPT tokens alone exceed the window —
|
||||||
|
`prompt: 202,777 prompt tokens does not fit the 200000 token context`) and the
|
||||||
|
window is ~514,286 characters at the measured 2.5714 chars/token, so a
|
||||||
|
calibration probe below that pays a real prefill instead of reporting.
|
||||||
|
- **Open call for Frank:** two slots (pool 3,072 + slab 27,897 = the ceiling
|
||||||
|
exactly) crosses the cliff for the +22 %, and costs the third concurrent 200k
|
||||||
|
conversation. Untested arm; capacity decision, not tuning.
|
||||||
|
- Seam: the pair ceiling is derived from the engine's budget table (30,969) and
|
||||||
|
owns that arithmetic alone; the KV-pool contract (seqs × window at the dtype's
|
||||||
|
measured rate + 110 MiB a sequence + 448 MiB of prefix checkpoints) keeps its
|
||||||
|
own test. 371/371 tests and e2e 33/33 on both deploys.
|
||||||
|
- **Power-failure casualties, NOT fixed (outside the archon window):**
|
||||||
|
`mnt-md0.mount` failed at boot, so `/mnt/md0` is an empty mountpoint (the
|
||||||
|
raid6 array itself is healthy, 7/7; fstab has `nofail` so boot continued) —
|
||||||
|
media libraries on it read empty. `frank-job-digest.service` (frankbook
|
||||||
|
job-hunt digest) is in `list-units --state=failed`.
|
||||||
|
- **The slab is a threshold, not a slope** — measured, same method (probe A
|
||||||
|
leg: 43,512 tokens, greedy, cold, disk prefix cache clean on every boot):
|
||||||
|
22,208 → 3,397 t/s, 24,000 → 3,290, **25,600 → 3,290**. The third step
|
||||||
|
bought nothing; the +22 % belongs to the 2026-10-11 arm at slab 26,071
|
||||||
|
(4,379 t/s), so the cliff is real and sits at ~26 GiB. The 2-slot arm that
|
||||||
|
would clear it (pool 3,072 + slab 27,897) remains untested — and the
|
||||||
|
paragraph above is amended: that arm's payoff is +22 % prefill at 2 slots,
|
||||||
|
not the +8.9 % the deployed pair ever showed (24,000 → 25,600 = 0 %).
|
||||||
|
- **Frank's decode challenge, answered with numbers**: 512 tokens, greedy,
|
||||||
|
cold — **decode 99.0 t/s single-stream at 43.5k and 93.3 at 197k** (decode
|
||||||
|
is nearly window-free: −6 % for 4.5× depth), while **3 concurrent streams
|
||||||
|
give 113.9/43.3/10.9 t/s = 18.4 t/s aggregate** at 43.5k and 12.5 at 197k.
|
||||||
|
The slot count is paid in decode concurrency — a fused-step lottery, not a
|
||||||
|
fair share — which is exactly the axis the shape hunt was blind to (probe
|
||||||
|
leg B generates 24 tokens). Corpus caveat at the claim: the ladder's filler
|
||||||
|
is repetitive, acceptance 0.93–0.97 vs the 0.740 of the 2026-10-11 decode
|
||||||
|
table, so these rows are the baseline for this corpus, not a replacement.
|
||||||
|
- **Seam:** `probe-frank-archon-kv.py` gains **leg D** (N-stream decode, 512
|
||||||
|
tokens, per-stream + aggregate) so no future shape change can be priced on
|
||||||
|
prefill alone.
|
||||||
|
|
||||||
|
|
||||||
## 2026-10-10
|
## 2026-10-10
|
||||||
- **Fleet service rename: the StarCraft names became the SERVICE names.**
|
- **Fleet service rename: the StarCraft names became the SERVICE names.**
|
||||||
The 2026-10-09 wire alias moved onto every repo artifact: units, serve
|
The 2026-10-09 wire alias moved onto every repo artifact: units, serve
|
||||||
@@ -1651,3 +1745,57 @@ every restart since the cutover.
|
|||||||
[[frank-archon-radiance-swift15-w4dn8]], and the LLM index entry corrected
|
[[frank-archon-radiance-swift15-w4dn8]], and the LLM index entry corrected
|
||||||
(the w4dn8 entry said STAGED, the fp8-native one said ACTIVE).
|
(the w4dn8 entry said STAGED, the fp8-native one said ACTIVE).
|
||||||
|
|
||||||
|
## 2026-10-15 — frank-archon: publisher iq4r payload swap (MODEL_FILE-only)
|
||||||
|
|
||||||
|
- Ask: replace the served model with
|
||||||
|
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` — the engine author's
|
||||||
|
published rad-convert of the OFFICIAL Qwen3.8-Flash-Next (w4nl codebook
|
||||||
|
experts, ten bf16-protected/layer, int8 trunk, PLE/MTP/vision in-container,
|
||||||
|
card KL 0.0789 / top-1 92.85 %) — **no other argument changes**, mindful
|
||||||
|
the operator session runs on the endpoint being swapped.
|
||||||
|
- Disk: Frank authorized one `rm` — the spent Swift1.5 BF16 source (336 GiB)
|
||||||
|
and the two-era-old `qwen4exp-fp8-native.rad` (173 GiB) left
|
||||||
|
`/home/dank/models`; 150 GB → 658 GB free. fp8-native is now rebuild-only
|
||||||
|
(fetch + build scripts); rollback for THIS swap is the w4dn8 `.rad`
|
||||||
|
(on disk) by reverting the commit.
|
||||||
|
- Fetch: new `scripts/download-frank-archon-iq4r.sh` (hfget, exact bytes
|
||||||
|
121,969,901,568 + Hub LFS sha256, `--verify-only` default) as transient
|
||||||
|
unit `iq4r-fetch`; verified, staged sidecar files kept as provenance.
|
||||||
|
- Shape carried VERBATIM (25600/5120 pair, 44928 pool, fp8 KV 3 × 200,000,
|
||||||
|
MTP 2, wht6, MNBT 2048, froggeric v22.5). Tests re-keyed
|
||||||
|
(`frank_archon_unit.bats` pins the new payload id + sha256, keeps the
|
||||||
|
w4dn8 rollback provenance), homepage card + trio lore + both indexes
|
||||||
|
updated; new runbook [[frank-archon-radiance-qwen38-iq4r]]. Commits
|
||||||
|
`6dcf4a5` (swap) + `f44c398` + `c5ac072` (docs), full bats green.
|
||||||
|
- Cutover self-severed by design (detached `deploy.sh` survived the session
|
||||||
|
death when frank-archon restarted); engine healthy in ~100 s. First-boot
|
||||||
|
ledger: w4nl loads on pin 1.0.13 (no pin bump); kv_attn 26.00 KiB fp8 @
|
||||||
|
607,740 tokens (identical geometry); slab 17,610/17,658 slots per rank
|
||||||
|
(> w4dn8's 17,548 — fewer bf16-protected experts pack more 4-bit table);
|
||||||
|
sampler 9 rows; MTP 2/step; prefix tier RECOVERED 900,176 blocks across
|
||||||
|
the swap (arch+tokenizer unchanged). Vision probe green (reasoning read
|
||||||
|
the solid-red PNG; the 64-token budget went all to thinking — fleet law).
|
||||||
|
- e2e gate went red on three dankclaw specs — NOT the swap: dankclaw rebooted
|
||||||
|
01:41 and the gluetun boot-race dependency-failed `searxng`, `marreta`,
|
||||||
|
and `marreta-browser` (jobs consumed while gluetun's start job retried;
|
||||||
|
`Restart=always` never armed because the services never ran). Cascade
|
||||||
|
restarted; skill codified (`gluetun-dependent-boot-race-recovery`).
|
||||||
|
Gate rerunning against the repaired fleet; frank-archon's own health gate
|
||||||
|
passed every deploy. Open: decode/prefill ladders vs the w4dn8 record,
|
||||||
|
probe leg D — scheduled probe window.
|
||||||
|
|
||||||
|
## 2026-10-15 — authentik: two more human accounts (Dankclaw)
|
||||||
|
- **Live — mitch (pk 22) and eric (pk 23) provisioned via `ak shell` ORM, no
|
||||||
|
repo objects.** Each `<user>@franksdank.cloud` in the human-user template
|
||||||
|
shape (`is_active=True`, display name, password via `set_password` on stdin
|
||||||
|
only — never argv, so nothing lands in `ps`), groups `Non-Admin Users` +
|
||||||
|
`jellyfin_users` + `openwebui_users`; no admin group granted. Verified
|
||||||
|
through the exact PasswordStage code path (`authenticate()` accepted each
|
||||||
|
password and rejected a wrong one), `app_entitlements(open-webui)` →
|
||||||
|
`Open WebUI Users` for both (identical to the ridley precedent), and
|
||||||
|
`is_superuser=False` / `path=users`. Zero `proxysession` rows for either —
|
||||||
|
groups were set before any login, so no claim purge was due. Downstream:
|
||||||
|
Jellyfin visibility rides the LDAP outpost cache (expect lag); OpenWebUI
|
||||||
|
mints their accounts as role `pending` on first login.
|
||||||
|
- `/serverAppConfigs-docs/Infrastructure/Authentik.md` — human-user template
|
||||||
|
census extended with pk 22–23.
|
||||||
@@ -366,13 +366,13 @@
|
|||||||
"model": {
|
"model": {
|
||||||
"unit": "archon",
|
"unit": "archon",
|
||||||
"role": "Frank's strongest and fastest model runtime",
|
"role": "Frank's strongest and fastest model runtime",
|
||||||
"hf": "https://huggingface.co/frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700",
|
"hf": "https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe",
|
||||||
"tagline": "...Power Overwhelming...",
|
"tagline": "...Power Overwhelming...",
|
||||||
"sfx": [
|
"sfx": [
|
||||||
"archon-power.ogg"
|
"archon-power.ogg"
|
||||||
],
|
],
|
||||||
"equiv": "Equivalent to Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).",
|
"equiv": "Equivalent to Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).",
|
||||||
"spec": "Our own build: a GPTQ-w4 quant of Swift1.5-Qwen3.8-Flash-Next, quantized and tuned in-house (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token context, three submissions at a time; sees images.",
|
"spec": "The engine author's published quant: 4-bit codebook experts + int8 trunk of Qwen3.8-Flash-Next (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token context, three submissions at a time; sees images.",
|
||||||
"inputs": [
|
"inputs": [
|
||||||
"text",
|
"text",
|
||||||
"image"
|
"image"
|
||||||
|
|||||||
+133
-28
@@ -3,14 +3,17 @@
|
|||||||
#
|
#
|
||||||
# This file asserts the RADIANCE shape (StillDeadcode engine, digest-pinned
|
# This file asserts the RADIANCE shape (StillDeadcode engine, digest-pinned
|
||||||
# stilldeadcode/radiance image, BOTH R9700s TP=2 on the wht6 wire, serving
|
# stilldeadcode/radiance image, BOTH R9700s TP=2 on the wht6 wire, serving
|
||||||
# the Swift1.5 Qwen3.8-Flash-Next GPTQ-w4 .rad built by
|
# the publisher's 4-bit-codebook .rad of the official Qwen3.8-Flash-Next
|
||||||
# scripts/convert-swift15-w4dn8.sh). It replaces the FreeToken seam of
|
# (fetched by scripts/download-frank-archon-iq4r.sh; the in-house
|
||||||
|
# Swift1.5 w4dn8 .rad of scripts/convert-swift15-w4dn8.sh is the on-disk
|
||||||
|
# rollback). It replaces the FreeToken seam of
|
||||||
# 2026-10-05 (git tag frank-archon-freetoken-pr587-20261005) and pins what
|
# 2026-10-05 (git tag frank-archon-freetoken-pr587-20261005) and pins what
|
||||||
# flags alone do not say: the served window clears the harness worst case,
|
# flags alone do not say: the served window clears the harness worst case,
|
||||||
# the KV pool backs seqs x max-model-len at the declared cache dtype (an
|
# the KV pool backs seqs x max-model-len at the declared cache dtype (an
|
||||||
# engine HARD limit, not a ratio), the per-card VRAM pair stays inside the
|
# engine HARD limit, not a ratio), the per-card VRAM pair stays inside the
|
||||||
# measured card ceiling, the pinned-host ledger has a ttm pages cap floor and
|
# the pinned-host ledger has a ttm pages cap floor, the serve script ENFORCES
|
||||||
# the serve script ENFORCES it, the served window is what models.yml must
|
# it and RAISES the knob itself when a reboot reset it (the cap's default is
|
||||||
|
# half of RAM), the served window is what models.yml must
|
||||||
# quote, the host-networking contract (no --publish), the image gate on a
|
# quote, the host-networking contract (no --publish), the image gate on a
|
||||||
# name@sha256 ref, the .rad payload gate with its size floor, the
|
# name@sha256 ref, the .rad payload gate with its size floor, the
|
||||||
# chat-template override mount, the self-healing law (the run returns before
|
# chat-template override mount, the self-healing law (the run returns before
|
||||||
@@ -24,16 +27,28 @@ load helpers
|
|||||||
|
|
||||||
UNIT="$REPO_ROOT/scripts/frank-archon.service"
|
UNIT="$REPO_ROOT/scripts/frank-archon.service"
|
||||||
SERVE="$REPO_ROOT/scripts/serve-frank-archon.sh"
|
SERVE="$REPO_ROOT/scripts/serve-frank-archon.sh"
|
||||||
|
TTM_LIB="$REPO_ROOT/scripts/ttm-preflight.sh"
|
||||||
ENV_FILE="$REPO_ROOT/scripts/frank-archon.env"
|
ENV_FILE="$REPO_ROOT/scripts/frank-archon.env"
|
||||||
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||||
DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-swift15.sh"
|
DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-iq4r.sh"
|
||||||
BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh"
|
BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh"
|
||||||
HFGET="$REPO_ROOT/scripts/hfget.sh"
|
HFGET="$REPO_ROOT/scripts/hfget.sh"
|
||||||
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
||||||
|
|
||||||
|
# A PATH with `cat` (the gate reads knobs with it) and NO `sudo`, so the
|
||||||
|
# privileged fallback cannot rescue a write the test means to fail.
|
||||||
|
nosudo_bin() {
|
||||||
|
local bin="$BATS_TEST_TMPDIR/nosudo-bin"
|
||||||
|
mkdir -p "$bin"
|
||||||
|
ln -sf "$(command -v cat)" "$bin/cat"
|
||||||
|
ln -sf "$(command -v bash)" "$bin/bash"
|
||||||
|
echo "$bin"
|
||||||
|
}
|
||||||
|
|
||||||
@test "frank-archon unit, serve script, env, fetch and build scripts and template exist" {
|
@test "frank-archon unit, serve script, env, fetch and build scripts and template exist" {
|
||||||
[ -f "$UNIT" ]
|
[ -f "$UNIT" ]
|
||||||
[ -x "$SERVE" ]
|
[ -x "$SERVE" ]
|
||||||
|
[ -f "$TTM_LIB" ]
|
||||||
[ -f "$ENV_FILE" ]
|
[ -f "$ENV_FILE" ]
|
||||||
[ -f "$DOWNLOAD" ]
|
[ -f "$DOWNLOAD" ]
|
||||||
[ -x "$BUILD" ]
|
[ -x "$BUILD" ]
|
||||||
@@ -120,37 +135,40 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
grep -q '^FRANK_ARCHON_MAX_MODEL_LEN=200000$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MAX_MODEL_LEN=200000$' "$ENV_FILE"
|
||||||
ml=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
|
ml=$(awk -F= '/^FRANK_ARCHON_MAX_MODEL_LEN=/{print $2}' "$ENV_FILE")
|
||||||
[ "$ml" -ge 177356 ]
|
[ "$ml" -ge 177356 ]
|
||||||
# Concurrency: 2 slots x the served window is the capacity contract, and
|
# Concurrency: 3 slots x the served window is the capacity contract, and
|
||||||
# the fused TP all-reduce is declared over seqs x (1 + spec) = 6 rows,
|
# the fused TP all-reduce is declared over seqs x (1 + spec) = 9 rows,
|
||||||
# under the 25-row bound at which the wht6 wire stays bit-exact. The
|
# under the 25-row bound at which the wht6 wire stays bit-exact. The
|
||||||
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
|
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork
|
||||||
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
|
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted
|
||||||
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
|
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3
|
||||||
# tied inside the +-3% noise floor.
|
# tied inside the +-3% noise floor — slots buy concurrency, not speed.
|
||||||
grep -q '^FRANK_ARCHON_MAX_SEQS=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MAX_SEQS=3$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_SPEC_TOKENS=2$' "$ENV_FILE"
|
||||||
# expert_tiered placement: vram_weights + the pinned host pool serve the
|
# expert_tiered placement: vram_weights + the pinned host pool serve the
|
||||||
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
|
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
|
||||||
# KV pool moves the pool with it. KV is UNQUANTIZED bf16 (the engine takes
|
# KV pool moves the pool with it. KV is QUANTIZED fp8 (the engine takes
|
||||||
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned
|
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned
|
||||||
# in host RAM per Frank's law.
|
# in host RAM per Frank's law.
|
||||||
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_VRAM_KV_MIB=[1-9]' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_VRAM_KV_MIB=[1-9]' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_KV_DTYPE=bf16$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
||||||
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
|
||||||
grep -q '^FRANK_ARCHON_PREFIX_HOST_MIB=8192$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PREFIX_HOST_MIB=8192$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_PREFIX_DISK_MIB=65536$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PREFIX_DISK_MIB=65536$' "$ENV_FILE"
|
||||||
# The pinned ledger (2 x pool + 47.7 GiB PLE + prefix host) must be
|
# The pinned ledger (2 x pool + 47.7 GiB PLE + prefix host) must be
|
||||||
# under the ttm pages cap, and the env states the floor the serve
|
# under the ttm pages cap, and the env states the floor the serve
|
||||||
# script enforces (>= 140,000 MiB — the current cap ~123 GiB fails it,
|
# script enforces (>= 140,000 MiB — the kernel's own default cap, half of
|
||||||
# which is the point).
|
# RAM, fails it, which is the point) plus the value it raises the knob to
|
||||||
|
# when a reboot reset it (the target must clear the floor with slack).
|
||||||
grep -q '^FRANK_ARCHON_TTM_MIN_MIB=' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_TTM_MIN_MIB=' "$ENV_FILE"
|
||||||
ttmm=$(awk -F= '/^FRANK_ARCHON_TTM_MIN_MIB=/{print $2}' "$ENV_FILE")
|
ttmm=$(awk -F= '/^FRANK_ARCHON_TTM_MIN_MIB=/{print $2}' "$ENV_FILE")
|
||||||
|
ttmt=$(awk -F= '/^FRANK_ARCHON_TTM_TARGET_MIB=/{print $2}' "$ENV_FILE")
|
||||||
|
[ "$ttmt" -gt "$ttmm" ]
|
||||||
pool=$(awk -F= '/^FRANK_ARCHON_HOST_POOL_MIB=/{print $2}' "$ENV_FILE")
|
pool=$(awk -F= '/^FRANK_ARCHON_HOST_POOL_MIB=/{print $2}' "$ENV_FILE")
|
||||||
ph=$(awk -F= '/^FRANK_ARCHON_PREFIX_HOST_MIB=/{print $2}' "$ENV_FILE")
|
ph=$(awk -F= '/^FRANK_ARCHON_PREFIX_HOST_MIB=/{print $2}' "$ENV_FILE")
|
||||||
# 2 pools + PLE 47.7 GiB (48,824 MiB) + prefix host, rounded up.
|
# 2 pools + PLE 47.7 GiB (48,824 MiB) + prefix host, rounded up.
|
||||||
@@ -188,15 +206,20 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
}
|
}
|
||||||
|
|
||||||
@test "the per-card VRAM pair stays inside the measured card ceiling" {
|
@test "the per-card VRAM pair stays inside the measured card ceiling" {
|
||||||
# Measured on Dankserver 2026-10-11: card 31,856 MiB, a boot consumes
|
# The engine's OWN budget table, not a hand-picked reserve. Boot of
|
||||||
# 1,710 MiB OUTSIDE both budgets (driver context, the 990 MiB activation
|
# 2026-10-13 at the pair 24000/5120: card 32,624 MiB (31.86 GiB), budgets
|
||||||
# arena, hardware queues, headroom, step staging) and the fp8 pair settled
|
# claimed 29,120, `3.42 GiB is neither budgeted nor accounted — the
|
||||||
# at 1,379 MiB free. Claiming more than 29,120 MiB in total is an
|
# driver's context, the activation arena and --gpu-headroom-mib come out of
|
||||||
# allocation failure at graph capture, not a warning — so the KV pool and
|
# it; anything past those is throughput on the floor`, and `card 0 settled
|
||||||
# the weights slab are one budget, and this seam is the guard.
|
# at 2.01 GiB free against 209.19 MiB (the 96.00 MiB --gpu-headroom-mib and
|
||||||
|
# 113.19 MiB kept for the static region)`. Claimable reserve above the
|
||||||
|
# engine's floor: 2,058 - 209 = 1,849 MiB, so the ceiling is 30,969 MiB.
|
||||||
|
# The 2026-10-11 bound of 29,120 ("keep >= 1 GiB free", card read as 31,856
|
||||||
|
# MiB) was MY reserve, which the engine never asked for; the shipped pair
|
||||||
|
# spends the difference and lands ~458 MiB above the floor.
|
||||||
w=$(awk -F= '/^FRANK_ARCHON_VRAM_WEIGHTS_MIB=/{print $2}' "$ENV_FILE")
|
w=$(awk -F= '/^FRANK_ARCHON_VRAM_WEIGHTS_MIB=/{print $2}' "$ENV_FILE")
|
||||||
kv=$(awk -F= '/^FRANK_ARCHON_VRAM_KV_MIB=/{print $2}' "$ENV_FILE")
|
kv=$(awk -F= '/^FRANK_ARCHON_VRAM_KV_MIB=/{print $2}' "$ENV_FILE")
|
||||||
[ $(( w + kv )) -le 29120 ]
|
[ $(( w + kv )) -le 30969 ]
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the serve script wires the engine flags from the env" {
|
@test "the serve script wires the engine flags from the env" {
|
||||||
@@ -272,6 +295,83 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
grep -q -- '--device /dev/dri' "$SERVE"
|
grep -q -- '--device /dev/dri' "$SERVE"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@test "the ttm gate raises a reset pages cap and refuses when the write fails" {
|
||||||
|
# The cap is a RUNTIME knob: the kernel default is half of RAM, so every
|
||||||
|
# reboot resets it (measured: the 2026-10-12 23:17 power failure on
|
||||||
|
# dankserver took it 200 GiB -> 125,988 MiB, and the preflight refusal was
|
||||||
|
# re-run 6,919 times by Restart=always over ~20 h). Behaviour required of
|
||||||
|
# the gate, in both directions:
|
||||||
|
# knob below the floor + writable -> cap ends at the target, start passes
|
||||||
|
# knob below the floor + unwritable -> cap stays low, start REFUSES by name
|
||||||
|
local dir="$BATS_TEST_TMPDIR/ttm"
|
||||||
|
mkdir -p "$dir"
|
||||||
|
export FRANK_ARCHON_TTM_SYS_DIR="$dir"
|
||||||
|
export FRANK_ARCHON_TTM_MIN_MIB=150000 FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||||
|
|
||||||
|
echo 32253039 > "$dir/pages_limit" # 125,988 MiB: the kernel default here
|
||||||
|
run bash -c '. "$1"; ttm_ensure_cap' _ "$TTM_LIB"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ "$(cat "$dir/pages_limit")" = "52428800" ] # 204,800 MiB in 4 KiB pages
|
||||||
|
|
||||||
|
echo 32253039 > "$dir/pages_limit"
|
||||||
|
chmod 444 "$dir/pages_limit" # the write refused: cap cannot move
|
||||||
|
# The refusal case must not have `sudo` on PATH: the gate's privileged
|
||||||
|
# fallback runs `sudo -n`, and on a passwordless host root can write a 444
|
||||||
|
# file, which would silently test nothing.
|
||||||
|
run env PATH="$(nosudo_bin)" bash -c '. "$1"; ttm_ensure_cap' _ "$TTM_LIB"
|
||||||
|
[ "$status" -eq 1 ]
|
||||||
|
[[ "$output" == *"ERROR: ttm pages cap is 125988 MiB"* ]]
|
||||||
|
[ "$(cat "$dir/pages_limit")" = "32253039" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the ttm cap counts only the knobs the kernel exposes" {
|
||||||
|
# An absent knob is not a zero: a zero-cap reading would refuse a healthy
|
||||||
|
# host. The effective cap is the MIN over the knobs that exist; pages_max,
|
||||||
|
# when a kernel exposes one, is the binding one.
|
||||||
|
local dir="$BATS_TEST_TMPDIR/ttm2"
|
||||||
|
mkdir -p "$dir"
|
||||||
|
export FRANK_ARCHON_TTM_SYS_DIR="$dir"
|
||||||
|
export FRANK_ARCHON_TTM_MIN_MIB=150000 FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||||
|
|
||||||
|
echo 51200000 > "$dir/pages_limit" # 200,000 MiB, no pages_max knob
|
||||||
|
run bash -c '. "$1"; echo "$(ttm_cap_mib)"' _ "$TTM_LIB"
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ "$output" = "200000" ]
|
||||||
|
|
||||||
|
echo 30000000 > "$dir/pages_max" # 117,187 MiB: now the binding knob
|
||||||
|
run bash -c '. "$1"; echo "$(ttm_cap_mib)"' _ "$TTM_LIB"
|
||||||
|
[ "$output" = "117187" ]
|
||||||
|
|
||||||
|
chmod 444 "$dir/pages_limit" "$dir/pages_max" # both writes refused
|
||||||
|
run env PATH="$(nosudo_bin)" bash -c '. "$1"; ttm_ensure_cap' _ "$TTM_LIB"
|
||||||
|
[ "$status" -eq 1 ]
|
||||||
|
[[ "$output" == *"ERROR: ttm pages cap is 117187 MiB"* ]]
|
||||||
|
[ "$(cat "$dir/pages_max")" = "30000000" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
@test "the gate runs as a root pre-start hook, where the knob write is legal" {
|
||||||
|
# The knob is root-owned 0644 and the service runs as User=dank, so the
|
||||||
|
# gate CANNOT raise it from the service process. systemd's `+` prefix on
|
||||||
|
# ExecStartPre runs as root even under User= — the canonical privileged
|
||||||
|
# path — and the gate's own --raise mode is what that line calls.
|
||||||
|
grep -q '^ExecStartPre=+/home/dank/serverAppConfigs/scripts/ttm-preflight.sh --raise$' "$UNIT"
|
||||||
|
|
||||||
|
# Behaviour of that entry point: sourced env values, knob raised, exit 0.
|
||||||
|
local dir="$BATS_TEST_TMPDIR/ttm3" envf="$BATS_TEST_TMPDIR/ttm3.env"
|
||||||
|
mkdir -p "$dir"
|
||||||
|
printf 'FRANK_ARCHON_TTM_MIN_MIB=150000\nFRANK_ARCHON_TTM_TARGET_MIB=204800\n' > "$envf"
|
||||||
|
echo 32253039 > "$dir/pages_limit"
|
||||||
|
run env FRANK_ARCHON_ENV_FILE="$envf" FRANK_ARCHON_TTM_SYS_DIR="$dir" \
|
||||||
|
bash "$TTM_LIB" --raise
|
||||||
|
[ "$status" -eq 0 ]
|
||||||
|
[ "$(cat "$dir/pages_limit")" = "52428800" ]
|
||||||
|
|
||||||
|
# Usage is a named failure, not a silent no-op.
|
||||||
|
run bash "$TTM_LIB"
|
||||||
|
[ "$status" -eq 2 ]
|
||||||
|
[[ "$output" == *"--raise"* ]]
|
||||||
|
}
|
||||||
|
|
||||||
@test "the serve script clears stale containers and self-heals via podman wait" {
|
@test "the serve script clears stale containers and self-heals via podman wait" {
|
||||||
# A crashed unit can leave the container name registered forever after;
|
# A crashed unit can leave the container name registered forever after;
|
||||||
# the legacy identities hold the same GPUs and die here too. Then block
|
# the legacy identities hold the same GPUs and die here too. Then block
|
||||||
@@ -283,18 +383,23 @@ TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
|
|||||||
grep -q 'exec podman wait "$FRANK_ARCHON_CONTAINER"' "$SERVE"
|
grep -q 'exec podman wait "$FRANK_ARCHON_CONTAINER"' "$SERVE"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "the model id matches the payload's build and fetch scripts" {
|
@test "the model id matches the payload's fetch script (and the rollback's build script)" {
|
||||||
# The payload dir/file are the convert script's output layout; the fetch
|
# The deployed payload is the PUBLISHER's .rad: the fetch script names
|
||||||
# script must name the repo the convert stages from. (FP8-native era,
|
# the repo, the file, and the Hub's LFS sha256; nothing is converted on
|
||||||
# the rollback payload, keeps its own trio: build-frank-archon-rad.sh +
|
# the host for it. The in-house w4dn8 rollback keeps its own trio
|
||||||
# download-frank-archon.sh.)
|
# (convert-swift15-w4dn8.sh + download-frank-archon-swift15.sh, and the
|
||||||
|
# fp8-native era keeps build-frank-archon-rad.sh +
|
||||||
|
# download-frank-archon.sh).
|
||||||
grep -q '^FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad$' "$ENV_FILE"
|
||||||
grep -q 'qwen4exp-swift15-w4dn8-i8-hc8m.rad' "$BUILD"
|
grep -q 'StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe' "$DOWNLOAD"
|
||||||
grep -q 'ukisai/Swift1.5-Qwen3.8-Flash-Next' "$DOWNLOAD"
|
grep -q 'qwen3.8-next-flash-fp8-iq4r-moe.rad' "$DOWNLOAD"
|
||||||
|
grep -q '0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20' "$DOWNLOAD"
|
||||||
grep -q 'hfget.sh' "$DOWNLOAD"
|
grep -q 'hfget.sh' "$DOWNLOAD"
|
||||||
# The repo is UNGATED — no token may be required on the fetch path.
|
# The repo is UNGATED — no token may be required on the fetch path.
|
||||||
! grep -q 'HF_TOKEN' "$DOWNLOAD"
|
! grep -q 'HF_TOKEN' "$DOWNLOAD"
|
||||||
|
# Rollback provenance: the w4dn8 build script names its payload.
|
||||||
|
grep -q 'qwen4exp-swift15-w4dn8-i8-hc8m.rad' "$BUILD"
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "no bare HIP-reserved env var is defined" {
|
@test "no bare HIP-reserved env var is defined" {
|
||||||
|
|||||||
Reference in new issue
Block a user