Compare commits

..
Author SHA1 Message Date
Frank 2e86445fc5 comments: replace law idiom with the mechanism it encoded
Every comment that cited a named law, an incident date, or an
internal nickname now states the underlying mechanism and why it
matters, generically: curl exit-0 artifacts and byte-count gates,
argv-in-/proc secret exposure, tag mutability vs digest pinning,
shape-specialized kernels refusing uncompiled slot counts, MTP heads
as file tensors no flag can supply, drafter/slot coupling as an
engine property, pkill -f matching its invoker, pinned-RAM pairing
with LimitMEMLOCK, one-encoding-per-table, the short-run first-token
noise floor, and the protected-expert rule. No behavior change:
comment/string text only; suite green (371 tests).
2026-10-14 13:25:41 -07:00
Frank cf6319aad7 docs(operations): LLM usage-attribution research catalog
Deep-research reference for per-consumer tracking of the fleet endpoints:
measured baseline (no identity anywhere; frank-zealot /metrics 501), the
seven identity layers, engine-native/gateway/tracing/network catalogs with
citations, and a phased adoption plan.
2026-10-14 13:25:41 -07:00
31 changed files with 380 additions and 503 deletions

No files matched your search

+27 -30
View File
@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
| Target | Service | Runtime | Model | Port | | Target | Service | Runtime | Model | Port |
|--------|---------|---------|-------|------| |--------|---------|---------|-------|------|
| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: revert the swap commit (w4dn8 `.rad` on disk); the fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 | | Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | ukisai/Swift1.5-Qwen3.8-Flash-Next BF16 (335.3 GiB, 397/397 sha256-verified) converted by `scripts/convert-swift15-w4dn8.sh` to `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB**): **all 49,152 experts GPTQ w4** (10M-token calibration on engine 1.2.0) + **24 whole bf16-protected experts/layer** (energy-share law), int8 trunk/lm_head, fp8 HC, PLE n-gram fp8 fixed-scale, on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **44 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the **30,969 MiB** the engine's own budget table makes claimable: card 32,624 MiB, settle floor 209 MiB), **47.7 GiB n-gram copy pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV** (pool 5,120 MiB = the 3-slot contract + 1.3 % — the only dtype that funds 3 × 200,000 without gutting the slab, since bf16 bills 13.8 KiB/token), **MTP spec decode depth 2**, `--max-num-seqs 3` (the capacity contract is 3 × 200,000; 3 × (1+2) = 9 all-reduce rows, inside the 25-row wht6 exactness bound), `--max-model-len 200000`, MNBT 2048, Froggeric v22.5 template. KLD-gated (mean KL 0.0131, top-1 97.92 % vs BF16); decode ~190 t/s live (gate bar 30.8); prefill is **slab-limited and pays for it** — 22,208 → 3,397, 24,000 → 3,290, 25,600 → **3,584** t/s at 43.5k single-stream, 3-way 197k aggregate 3,051 → **3,589** t/s, with a cliff at ~26 GiB (26,071 → 4,379). Cutover 2026-10-08 from fp8-native (payload on disk = rollback); 3-slot fp8-KV shape 2026-10-13, re-cut twice in the slab's favour (22208/6912 at 4 slots → 24000/5120 → 25600/5120); the 4-slot shape (22208/6912, `--max-num-seqs 4`) is the documented rollback. | 7770 |
| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770| | Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
| Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 | | Dankclaw | `frank-zealot` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
@@ -498,33 +498,32 @@ Rollback: revert the swap commit — the Gemma4 payload
(`/home/dank/models/Ornith-Q8/`) all remain on disk. (`/home/dank/models/Ornith-Q8/`) all remain on disk.
### frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, TP=2 (Dankserver, ACTIVE) ### frank-archon — Radiance (StillDeadcode), Swift1.5-Qwen3.8-Flash-Next GPTQ w4, TP=2 (Dankserver, ACTIVE)
The **radiance engine** (StillDeadcode, Apache-2.0; image The **radiance engine** (StillDeadcode, Apache-2.0; image
`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13** `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
for serve; `libr4d` gfx1201 kernels, plain `podman run`, root for serve/convert/KLD; `libr4d` gfx1201 kernels, plain `podman run`, root
systemd unit, container name = unit name) serving the systemd unit, container name = unit name) serving
**publisher's container of the official Qwen/Qwen3.8-Flash-Next**: **ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 335.3 GiB, 131 shards,
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → 397/397 sha256-verified)** converted by `scripts/convert-swift15-w4dn8.sh`
`qwen3.8-next-flash-fp8-iq4r-moe.rad` (116,322 MiB, Hub-LFS sha256 with recipe `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` to
verified at fetch by `scripts/download-frank-archon-iq4r.sh`; recipe `qwen4exp-swift15-w4dn8-i8-hc8m.rad` (~114 GiB): **all 49,152 experts GPTQ
`qwen4exp-w4nl64-i8-hc8m`): **routed experts as 4-bit codes into a w4** (act-order, damp 0.05, 10,000,000-token calibration captured on engine
16-level non-uniform codebook** after a 128-point Walsh–Hadamard rotation **1.2.0** via `RADIANCE_CALIB_DIR`/`RADIANCE_CALIB_TOKENS` — the only pin
(E4M3 scale per 64 weights; codes by GPTQ on a 10M-token calibration with with the token knob; the RADGRAM2 grams are format-identical across pins and
per-expert down-proj Hessians, refined by 3 coordinate-descent sweeps), cross the boundary by design), **24 whole experts per layer cast bf16** (the
**10 whole bf16-protected experts/layer**, **int8 W8A8 trunk/lm_head** PROTECTED set, derived from the calibration grams' energy share),
(scale per 128 columns searched), **fp8 HC mixtures**, PLE n-gram table at **int8 trunk/lm_head**, **fp8 HC mixtures** (`--hc-mix-dtype`), PLE n-gram
fp8 fixed scale (47.7 GiB), **MTP head and bf16 vision tower in the table at fp8 fixed scale (47.7 GiB in the container, from 95.4 GiB bf16).
container**. Card quality (full-vocab teacher-forced KL vs bf16, ~75K The plugin's **one-table-one-encoding law** (measured at rad-convert
positions): mean 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %. declare): one expert table takes ONE encoding; only whole bf16 experts are
**MODEL_FILE is the only serving argument that moved** (2026-10-15); every legal beside it — no w4/fp8 split of the down/gate_up pair (24-value
number of the 2026-10-13 shape below carries verbatim. The plugin's `rad_arch_quant_desc` array, one per layer). KLD gate vs the BF16
**one-table-one-encoding law** (fleet-measured at rad-convert declare): reference (recorded on the same 1.0.13 pin) is the precision referee;
one expert table takes ONE encoding; only whole bf16 experts are legal the throughput gate (decode ≥ 30.8 t/s) was passed at 4.5× the bar
beside it. Rollback: revert the swap commit — the in-house w4dn8 `.rad` (122–173 t/s at 44.5k depth). Measured decode/prefill in the runbook.
(KLD-gated Swift1.5 conversion) stays on disk; the fp8-native `.rad` was Rollback: revert the swap commit — the fp8-native `.rad`
deleted 2026-10-15 (rebuild: `scripts/download-frank-archon.sh` + (`qwen4exp-fp8-native.rad`) and every prior payload remain on disk.
`scripts/build-frank-archon-rad.sh`).
**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`, **Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
`--max-model-len 200000` (the harness contextWindow must quote this), `--max-model-len 200000` (the harness contextWindow must quote this),
@@ -631,9 +630,8 @@ chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
`--mm-max-patches`); `/metrics` carries vLLM-compatible names + `--mm-max-patches`); `/metrics` carries vLLM-compatible names +
`radiance:*` families — the Dankclaw Prometheus scrape is re-added. `radiance:*` families — the Dankclaw Prometheus scrape is re-added.
Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md`. Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
Retired Swift1.5 w4dn8 era (payload on disk = rollback): `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`. Retired fp8-native era: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
Retired fp8-native era (payload deleted 2026-10-15, rebuildable): `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`. Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`. Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`. Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
@@ -1093,7 +1091,7 @@ Active:
|---------|---------| |---------|---------|
| frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` | | frank-dragoon (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md` |
| frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` | | frank-zealot (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` | | frank-archon (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md` |
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` | | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
| Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` | | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -1101,7 +1099,6 @@ Retired (rollback targets and history):
| Runtime | Host | Window | Runbook | | Runtime | Host | Window | Runbook |
|---------|------|--------|---------| |---------|------|--------|---------|
| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` on disk; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` | | llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` | | ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` | | tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
+2 -1
View File
@@ -27,7 +27,8 @@ CALIB=/home/dank/corpora/calib
OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad OUT=/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad
LOG=/home/dank/logs/swift15_convert.log LOG=/home/dank/logs/swift15_convert.log
# gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table; # gate_up ~37.6 + down ~18.8 + PLE 47.7 + trunk/embeddings/head ~10 GiB (all-w4 expert table;
# the plugin's one-encoding law dropped the block-fp8 down tier -- see the recipe header). # the format assigns ONE encoding per expert table, so a per-tensor block-fp8
# down tier cannot be served beside a w4 gate_up tier -- see the recipe header).
MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000} MIN_MIB=${FRANK_SWIFT15_RAD_MIN_MIB:-108000}
# Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a # Gate 1: inputs. A missing Gram directory converts UNCALIBRATED with a
+10 -12
View File
@@ -947,7 +947,7 @@ deploy_plex() {
# accepted and then answered with nothing (curl 52, Empty reply) against a # accepted and then answered with nothing (curl 52, Empty reply) against a
# perfectly healthy server. Measured 2026-10-02 — the gate burned its full # perfectly healthy server. Measured 2026-10-02 — the gate burned its full
# 600s and failed a plex whose https://127.0.0.1:32400/identity answered # 600s and failed a plex whose https://127.0.0.1:32400/identity answered
# 200 in 44 ms the whole time. The same law is why the Caddyfile dials the # 200 in 44 ms the whole time. The same requirement is why the Caddyfile dials the
# plex upstream over TLS (tests/caddyfile.bats pins it). # plex upstream over TLS (tests/caddyfile.bats pins it).
# -k: PMS presents its own self-signed cert on 32400. # -k: PMS presents its own self-signed cert on 32400.
# #
@@ -1207,8 +1207,9 @@ deploy_llm_dragoon() {
# on the FULL container name, which appears in the argv of a manual # on the FULL container name, which appears in the argv of a manual
# `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in # `podman run --name frank-dragoon-halogen` / `podman wait` but NOT in
# this deployer's own command line (`bash deploy-remote.sh # this deployer's own command line (`bash deploy-remote.sh
# frank-dragoon`) — so it cannot shoot itself in the foot (the fleet # frank-dragoon`) — so the sweep can never match the deployer's own argv
# pkill law), and it can never match frank-zealot's llama-server. # (`pkill -f` kills the invoker when the pattern appears in the invoker's
# own command line), and it can never match frank-zealot's llama-server.
run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true run_privileged systemctl stop frank-dragoon.service 2>/dev/null || true
pkill -f 'frank-dragoon-halogen' 2>/dev/null || true pkill -f 'frank-dragoon-halogen' 2>/dev/null || true
podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true podman rm -f frank-dragoon frank-dragoon-halogen >/dev/null 2>&1 || true
@@ -1277,14 +1278,11 @@ deploy_llm_zealot() {
} }
# deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode, # deploy_llm_archon — frank-archon is the radiance engine (StillDeadcode,
# digest-pinned stilldeadcode/radiance image) serving the PUBLISHER's # digest-pinned stilldeadcode/radiance image) serving the OFFICIAL
# 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next # Qwen/Qwen3.8-Flash-Next-FP8 at native block-FP8 precision from the
# (qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by # rad-convert container qwen4exp-fp8-native.rad, on BOTH R9700s (TP=2) as a
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2) as a
# ROOT unit whose serve script drives `podman run` directly (the engine is # ROOT unit whose serve script drives `podman run` directly (the engine is
# the image entrypoint; the image lives in dank's rootless podman storage). # the image entrypoint; the image lives in dank's rootless podman storage).
# 2026-10-15 payload swap from the in-house w4dn8 (payload on disk =
# rollback).
# 2026-10-05 cutover from FreeToken (pre-cutover tag # 2026-10-05 cutover from FreeToken (pre-cutover tag
# frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM # frank-archon-freetoken-pr587-20261005); before that, the tcclaviger vLLM
# fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the # fork (tag frank-archon-vllm-shapeG-20261003). Same port and alias; the
@@ -1293,7 +1291,7 @@ deploy_llm_zealot() {
# Gates: the image must exist under its digest ref in dank's rootless # Gates: the image must exist under its digest ref in dank's rootless
# storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the # storage (FRANK_ARCHON_IMAGE), the .rad container file must exist at the
# env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB — # env's size floor (FRANK_ARCHON_MODEL_FILE >= FRANK_ARCHON_RAD_MIN_MIB —
# a truncated fetch is a guaranteed long-timeout boot), and the unit env # a truncated convert is a guaranteed long-timeout boot), and the unit env
# must be complete. Per-file sha256 verification is the fetch script's job # must be complete. Per-file sha256 verification is the fetch script's job
# at download time and rad-convert/rad-info's job at build time, never # at download time and rad-convert/rad-info's job at build time, never
# re-hashed here. # re-hashed here.
@@ -1328,13 +1326,13 @@ deploy_llm_archon() {
fi fi
if [[ ! -f "$model_dir/$model_file" ]]; then if [[ ! -f "$model_dir/$model_file" ]]; then
echo "ERROR: .rad container missing: $model_dir/$model_file" >&2 echo "ERROR: .rad container missing: $model_dir/$model_file" >&2
echo "frank-archon: run scripts/download-frank-archon-iq4r.sh --fetch" >&2 echo "frank-archon: run scripts/build-frank-archon-rad.sh" >&2
return 1 return 1
fi fi
mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 )) mib=$(( $(stat -c %s "$model_dir/$model_file") / 1048576 ))
if [[ "$mib" -lt "$rad_min" ]]; then if [[ "$mib" -lt "$rad_min" ]]; then
echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2 echo "ERROR: .rad container too small: $model_dir/$model_file is ${mib} MiB, want >= $rad_min" >&2
echo "frank-archon: re-fetch it (scripts/download-frank-archon-iq4r.sh --fetch; delete the staged file first if its sha256 failed)" >&2 echo "frank-archon: rebuild it (scripts/build-frank-archon-rad.sh --force)" >&2
return 1 return 1
fi fi
+1 -1
View File
@@ -2,7 +2,7 @@
"""derive-protected-experts.py — rewrite the recipe's protected-expert section """derive-protected-experts.py — rewrite the recipe's protected-expert section
from a calibration recording. RUNS ON DANKSERVER (venv: numpy). from a calibration recording. RUNS ON DANKSERVER (venv: numpy).
The law (docs/MOE-W4.md "Protected experts", restated in the recipe): The protection rule (docs/MOE-W4.md "Protected experts", restated in the recipe):
an expert whose share of its layer's down-input energy is at least 3% AND an expert whose share of its layer's down-input energy is at least 3% AND
at least three times its share of the routing is kept bf16; the count must at least three times its share of the routing is kept bf16; the count must
be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest be EVEN on each rank's half of the layer (the four-bit GEMM takes the rest
-83
View File
@@ -1,83 +0,0 @@
#!/usr/bin/env bash
# download-frank-archon-iq4r.sh — payload fetch/verify for the frank-archon
# swap to the PUBLISHER's 4-bit-codebook container:
# StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe — the official
# Qwen/Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's own
# author into ONE .rad: routed experts as 4-bit codes into a 16-level
# non-uniform codebook (Walsh–Hadamard rotation, GPTQ + coordinate-descent,
# 10M-token calibration), ten bf16-protected experts per layer, int8 trunk +
# lm_head, fp8 HC mixtures, the PLE n-gram table at fp8 fixed scale, the MTP
# head, and the bf16 vision tower. File:
# qwen3.8-next-flash-fp8-iq4r-moe.rad — 121,969,901,568 B (113.6 GiB,
# 116,322 MiB), card KL 0.0789 all-tokens / top-1 92.85 % vs bf16.
# Repo is UNGATED — no token needed (never put one on the command line).
#
# bash scripts/download-frank-archon-iq4r.sh --fetch resume/complete
# bash scripts/download-frank-archon-iq4r.sh verify only
#
# Fetch uses the vendored scripts/hfget.sh (curl+jq only): per-file size
# checks against the Hub listing, `-C -` resume, disk-fit gate. The side
# files (LICENSE, README, the w4nl recipe) stay in the staging dir as
# provenance; the serve gate mounts ONLY /home/dank/models/rad, so the
# verified .rad alone moves into it.
#
# Verify (default) is the AGENTS.md law in two parts: the file lands at the
# final path with the Hub's EXACT byte count, and its sha256 equals the
# Hub's LFS oid. A truncated transfer is a guaranteed long-timeout boot
# without this gate.
set -euo pipefail
REPO="StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe"
RAD_NAME="qwen3.8-next-flash-fp8-iq4r-moe.rad"
STAGE="/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe"
DEST="/home/dank/models/rad/$RAD_NAME"
SIZE=121969901568
SHA256="0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
mode="verify"
[[ "${1:-}" == "--fetch" ]] && mode="fetch"
if [[ "$mode" == "fetch" ]]; then
# HFGET_JOBS=4 is hfget's default (a few streams saturate the link).
bash "$SCRIPT_DIR/hfget.sh" "$REPO" "$STAGE" main
staged="$STAGE/$RAD_NAME"
if [[ ! -f "$staged" ]]; then
echo "FETCH FAILED: $staged absent after hfget" >&2
exit 1
fi
got=$(stat -c %s "$staged")
if [[ "$got" != "$SIZE" ]]; then
echo "FETCH FAILED: staged .rad is $got B, want $SIZE B" >&2
exit 1
fi
echo "staged .rad byte-exact ($SIZE B); hashing (one full read, minutes)"
got_sha=$(sha256sum "$staged" | cut -d' ' -f1)
if [[ "$got_sha" != "$SHA256" ]]; then
echo "FETCH FAILED: sha256 $got_sha, want $SHA256 — DELETE the staged" \
"file before re-fetching (curl -C - cannot repair a corrupt body)" >&2
exit 1
fi
mkdir -p "$(dirname "$DEST")"
mv "$staged" "$DEST"
echo "payload placed: $DEST"
fi
echo "verifying deployed payload: $DEST"
if [[ ! -f "$DEST" ]]; then
echo " BAD payload absent: $DEST" >&2
exit 1
fi
got=$(stat -c %s "$DEST")
if [[ "$got" != "$SIZE" ]]; then
echo " BAD payload is $got B, want $SIZE B" >&2
exit 1
fi
echo "byte-exact ($SIZE B); hashing (one full read, minutes)"
got_sha=$(sha256sum "$DEST" | cut -d' ' -f1)
if [[ "$got_sha" != "$SHA256" ]]; then
echo " BAD sha256 $got_sha, want $SHA256" >&2
exit 1
fi
echo "payload verified: $RAD_NAME, size and sha256 match the Hub LFS oid"
+1 -1
View File
@@ -14,7 +14,7 @@
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0 # Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
# listing/resume downloader, curl+jq only): per-file size checks against # listing/resume downloader, curl+jq only): per-file size checks against
# the Hub listing, `-C -` resume, pickle-format duplicates skipped. # the Hub listing, `-C -` resume, pickle-format duplicates skipped.
# Verify (default) is the AGENTS.md law in three parts: every listed file # Verify (default) enforces the download contract in three parts: every listed file
# present with the listed size, the shard count matches the weight-map # present with the listed size, the shard count matches the weight-map
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs # index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
# only the public API. # only the public API.
+1 -1
View File
@@ -12,7 +12,7 @@
# Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0 # Fetch uses the vendored scripts/hfget.sh (StillDeadcode's Apache-2.0
# listing/resume downloader, curl+jq only): per-file size checks against # listing/resume downloader, curl+jq only): per-file size checks against
# the Hub listing, `-C -` resume, pickle-format duplicates skipped. # the Hub listing, `-C -` resume, pickle-format duplicates skipped.
# Verify (default) is the AGENTS.md law in three parts: every listed file # Verify (default) enforces the download contract in three parts: every listed file
# present with the listed size, the shard count matches the weight-map # present with the listed size, the shard count matches the weight-map
# index, and every shard's sha256 equals the Hub's LFS oid. Verify needs # index, and every shard's sha256 equals the Hub's LFS oid. Verify needs
# only the public API. # only the public API.
+5 -3
View File
@@ -4,9 +4,10 @@
# .hgn, the REQUIRED lookup table and the optional vision tower from # .hgn, the REQUIRED lookup table and the optional vision tower from
# peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model # peonist-ai/halogen-qwen3.8-flash-next (both at the revisions the model
# card pins), and the flat tokenizer the two repos share. All public, no # card pins), and the flat tokenizer the two repos share. All public, no
# token — and per the fleet secret law, none would ever touch argv. # token — and a token never rides a command line: argv is readable by every
# process on the host through /proc, so a key placed there is a key exposed.
# #
# The fetch law, same as every sibling script: curl -f (an access error # The fetch discipline, same as every sibling script: curl -f (an access error
# must not become a "successful" 153-byte artifact), exact byte count from # must not become a "successful" 153-byte artifact), exact byte count from
# the Hub tree API, LFS sha256 verification, --verify-only to re-check an # the Hub tree API, LFS sha256 verification, --verify-only to re-check an
# existing payload without downloading. # existing payload without downloading.
@@ -26,7 +27,8 @@
# tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer # tokenizer/* @e053f488 — byte-identical to the 27B-era tokenizer
# already on disk (tokenizer.json LFS # already on disk (tokenizer.json LFS
# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3); # 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3);
# fetched fresh rather than symlinked, per the no-symlink law. # fetched fresh rather than symlinked, so the payload dir is
# self-contained — every gate and every mount sees real files inside it.
set -euo pipefail set -euo pipefail
DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}" DEST="${FRANK_DRAGOON_MODEL_DIR:-$HOME/models/halogen-flashnext-v2}"
+5 -4
View File
@@ -1,10 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload: # download-frank-dragoon-halogen.sh — fetch frank-dragoon's halogen payload:
# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the # the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet # PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and a token never
# secret law, none would ever touch argv). # rides a command line: argv is readable by every process on the host
# # through /proc, so a key placed there is a key exposed).
# The fetch law, same as every sibling script: curl -f (an access error #
# The fetch discipline, same as every sibling script: curl -f (an access error
# must not become a "successful" 153-byte artifact), exact byte count from # must not become a "successful" 153-byte artifact), exact byte count from
# the model card, LFS sha256 verification, --verify-only to re-check an # the model card, LFS sha256 verification, --verify-only to re-check an
# existing payload without downloading. # existing payload without downloading.
+2 -2
View File
@@ -11,8 +11,8 @@ Tensor extents are DERIVED from consecutive GGUF header offsets — no quant
type-size table (those are easy to get subtly wrong; a wrong per-element size type-size table (those are easy to get subtly wrong; a wrong per-element size
silently under-covers each tensor and dumps tail samples into a junk bucket). silently under-covers each tensor and dumps tail samples into a junk bucket).
Fleet law this instrument established (2026-09-26, 75 s window, 4 mixed What this instrument shows (75 s window, 4 mixed streams, 2.7M
streams, 2.7M expert-tensor samples): per-BLOCK read volume is FLAT expert-tensor samples): per-BLOCK read volume is FLAT
(0.93x-1.08x of mean across all 40 CPU-resident blocks) because MoE top-k (0.93x-1.08x of mean across all 40 CPU-resident blocks) because MoE top-k
routing fires every block every token — there are no "most active layers" at routing fires every block every token — there are no "most active layers" at
`-ot` granularity. The real skew is per-EXPERT (top 10% of (block,expert) `-ot` granularity. The real skew is per-EXPERT (top 10% of (block,expert)
+44 -64
View File
@@ -1,15 +1,7 @@
# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving the # frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving
# PUBLISHER's 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next # ukisai/Swift1.5-Qwen3.8-Flash-Next converted to GPTQ w4 (MODEL_FILE below,
# (MODEL_FILE below, fetched by scripts/download-frank-archon-iq4r.sh) on # built by scripts/convert-swift15-w4dn8.sh) on BOTH R9700s (TP=2),
# BOTH R9700s (TP=2), dankserver :7770. # dankserver :7770.
# 2026-10-15 PAYLOAD SWAP: the model moved from the in-house w4dn8 conversion
# of the Swift1.5 finetune to the engine author's published container
# qwen3.8-next-flash-fp8-iq4r-moe.rad (116,322 MiB, sha256-verified at fetch —
# the w4dn8 measured 116,416 MiB, so this shape carries over with 94 MiB to
# spare). NO argument changed: every number below is the 2026-10-13 shape,
# kept verbatim. The swap commit is the rollback (the w4dn8 payload stays on
# disk).
# #
# SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV, # SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV,
# with the per-card budget spent on resident experts: 25,600 MiB of slab + # with the per-card budget spent on resident experts: 25,600 MiB of slab +
@@ -21,11 +13,9 @@
# +25 % at the full window) while KV beyond the contract buys nothing. # +25 % at the full window) while KV beyond the contract buys nothing.
# Consequence, deliberate: a FOURTH full-height conversation does not fit this # Consequence, deliberate: a FOURTH full-height conversation does not fit this
# pair, and the engine refuses it — it never truncates. See the KV block for the # pair, and the engine refuses it — it never truncates. See the KV block for the
# arithmetic and the measured dtype law. The w4dn8 payload is the swap # arithmetic and the measured per-dtype cost. The fp8-native payload this file was
# rollback and keeps its trio under the w4dn8 runbook; the fp8-native payload # written for is the rollback target and keeps its own trio under the
# that came before it was DELETED from disk 2026-10-15 (Frank's call, to free # fp8-native runbook.
# the volume for this download — rebuildable via the fetch + build scripts
# named in the fp8-native runbook).
# #
# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in # 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the # pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
@@ -40,30 +30,20 @@
# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram # (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring # table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram." # reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
# THE 4-bit-codebook publisher shape is the DEPLOYED payload since # The 4-bit-codebook publisher shape (qwen3.8-next-flash-fp8-iq4r-moe.rad,
# 2026-10-15. The card's own run line (--host-pool-mib 12288, # --host-pool-mib 12288) remains the documented fallback if the native
# --expert-vs-cache-ratio 0.82, seqs 8, spec 3) is the publisher's # container proves unbootable.
# suggestion on 32 GB cards; this file's shape is the fleet's measured one.
# #
# PAYLOAD: /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad — the # PAYLOAD: /home/dank/models/rad/qwen4exp-fp8-native.rad — a rad-convert
# official Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's # NO-RECIPE pure copy of the official 131-shard checkpoint (every weight
# own author (recipe qwen4exp-w4nl64-i8-hc8m: experts as 4-bit codes into a # keeps its block-FP8 encoding; ~161 GiB). The convert runs against a
# 16-level non-uniform codebook after a 128-point Walsh–Hadamard rotation, # staging dir whose config.json drops `quantization_config`: the descriptor
# GPTQ + 3 coordinate-descent sweeps on a 10M-token calibration, ten # string is only the plugin SELECTION key (spec §2.4 — exact match, no
# bf16-protected experts per layer, int8 W8A8 trunk + lm_head, fp8 HC # fallback), and no shipped plugin claims (qwen4exp, "fp8"); the shipped
# mixtures, PLE n-gram fp8 fixed-scale, MTP head, bf16 vision tower; card KL # qwen4exp plugin claims (qwen4exp, "") which is what recipe-built
# 0.0789 all-tokens / top-1 92.85 % vs bf16). Fetched byte-exact + sha256 by # containers carry. Per-weight encodings are data, not the descriptor —
# scripts/download-frank-archon-iq4r.sh; the recipe/LICENSE stay in the # removing the key changes nothing about what the weights hold. Built by
# staging dir beside it as provenance. ROLLBACK: revert the swap commit — the # scripts/build-frank-archon-rad.sh (staging + convert + rad-info).
# in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad (scripts/convert-swift15-w4dn8.sh,
# KLD-gated) stays on disk. Behind it, the fp8-native
# qwen4exp-fp8-native.rad was DELETED from disk 2026-10-15 (Frank's call,
# one-time rm, to free the volume for this payload); rebuild =
# scripts/download-frank-archon.sh + scripts/build-frank-archon-rad.sh — a
# NO-RECIPE rad-convert copy of the official 131-shard checkpoint under the
# descriptor law (no shipped plugin claims (qwen4exp, "fp8"), so the staging
# config.json drops quantization_config; per-weight encodings are data, not
# the descriptor).
# #
# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml): # SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
# - The HTTP listener binds AFTER the full load (main.cpp: engine.start() # - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
@@ -79,12 +59,12 @@
# enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) — # enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
# models.yml's extraBody set is fully inside it. Unknown params are # models.yml's extraBody set is fully inside it. Unknown params are
# refused (400), which is why nothing unverified rides the wire. # refused (400), which is why nothing unverified rides the wire.
# - MTP: the .rad carries the model's mtp.* head (the publisher's recipe # - MTP: the official checkpoint carries the mtp.* head (3,101 tensors,
# names the mtp.fc_hidden/fc_embedding/draft_head rules; card default # mtp_num_hidden_layers=1) and the plugin probes draft depth 3 — the
# depth 3). Served at the depth named by FRANK_ARCHON_SPEC_TOKENS below — # publisher's throughput peak for this model. Stated explicitly with
# the depth this shape's decode ladder was measured at. # --num-speculative-tokens 3.
# - The fused TP all-reduce is declared over max-num-seqs x # - The fused TP all-reduce is declared over max-num-seqs x
# (1 + --num-speculative-tokens) rows; 3 x (1+2) = 9 rows, under the 25-row # (1 + --num-speculative-tokens) rows; 2 x 3 = 6 rows, under the 25-row
# exactness bound of the wht6 wire. # exactness bound of the wht6 wire.
# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13, # Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb # index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
@@ -108,7 +88,7 @@ FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd7
# .rad files; the raw HF download and its staging dir live beside it and # .rad files; the raw HF download and its staging dir live beside it and
# are never mounted. # are never mounted.
FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad
FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
# The state dir's tune/ subdirectory holds rad-tune's kernel-selection # The state dir's tune/ subdirectory holds rad-tune's kernel-selection
# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune # cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
@@ -144,13 +124,12 @@ FRANK_ARCHON_TP=2
FRANK_ARCHON_TP_WIRE=wht6 FRANK_ARCHON_TP_WIRE=wht6
# Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three # Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three
# concurrent conversations of --max-model-len fit the KV pool (see the KV # concurrent conversations of --max-model-len fit the KV pool (see the KV
# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the # block below). A runtime that compiles its kernels specialized to the exact
# tcclaviger vLLM fork's torch.compile shape specialization on # batch shape can refuse a slot count it has never compiled for; a
# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired # shape-agnostic engine accepts any count that fits memory. Radiance is the
# with that runtime). Radiance booted seqs 2 at the fp8-native first boot, # latter: seqs 2 boots, and measured decode at seqs 2 and 3 is statistically
# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied # tied (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys
# (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys concurrency, # concurrency, not speed, so a slot costs KV, never decode.
# not speed, so a slot costs KV, never decode.
# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV # Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows — # pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
# 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT # 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT
@@ -262,10 +241,9 @@ FRANK_ARCHON_VRAM_KV_MIB=5120
# ledgers' `kv tier transfer` lines), so the host tier below parks twice as # ledgers' `kv tier transfer` lines), so the host tier below parks twice as
# many conversations as it did at bf16 — no re-size needed to park three slots. # many conversations as it did at bf16 — no re-size needed to park three slots.
FRANK_ARCHON_KV_DTYPE=fp8 FRANK_ARCHON_KV_DTYPE=fp8
# Spec decode: the checkpoint's own MTP head, served at depth 2 — the # Spec decode: the checkpoint's own MTP head at the publisher's depth-3
# publisher's card ships depth 3 as its peak; this fleet's ladder was # operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
# measured at 2 and the swap keeps every argument, so 2 stays (rows # flag states it so the ledger records the intent).
# 3 x (1+2) = 9, inside the wht6 25-row exactness bound).
FRANK_ARCHON_SPEC_TOKENS=2 FRANK_ARCHON_SPEC_TOKENS=2
FRANK_ARCHON_MAX_BATCHED_TOKENS=2048 FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES, # Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
@@ -276,9 +254,10 @@ FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
# available with the service loaded). # available with the service loaded).
FRANK_ARCHON_PREFIX_HOST_MIB=8192 FRANK_ARCHON_PREFIX_HOST_MIB=8192
FRANK_ARCHON_PREFIX_DISK_MIB=65536 FRANK_ARCHON_PREFIX_DISK_MIB=65536
# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host # The PLE n-gram table (47.7 GiB) serves from pinned host RAM the cards
# RAM the cards read. `ram` REFUSES the start if the memlock limit is too # read directly, so it cannot be evicted or page-faulted mid-token. `ram`
# small — the unit's LimitMEMLOCK=infinity is the other half of this law. # placement REFUSES the start if the memlock limit is too small, and the
# unit's LimitMEMLOCK=infinity is the matching half of that pairing.
FRANK_ARCHON_NGRAM_PLACEMENT=ram FRANK_ARCHON_NGRAM_PLACEMENT=ram
# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB # TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's # prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
@@ -299,8 +278,9 @@ FRANK_ARCHON_TTM_TARGET_MIB=204800
# multi-flag values as one string; the serve script expands it unquoted on # multi-flag values as one string; the serve script expands it unquoted on
# purpose. # purpose.
FRANK_ARCHON_EXTRA_ARGS= FRANK_ARCHON_EXTRA_ARGS=
# The .rad's minimum size in MiB (deployer gate; a truncated fetch is a # The .rad's minimum size in MiB (deployer gate; a truncated convert is a
# guaranteed long-timeout boot instead of a fast failure). The deployed # guaranteed long-timeout boot instead of a fast failure). The converted
# publisher container measures 116,322 MiB (121,969,901,568 B); the # container measured 177,109 MiB (1,857,127,997,440 B = 173.0 GiB; the
# in-house w4dn8 rollback measured 116,416 MiB. Floor sits just under both. # plan-only estimate 172.76 GiB covers the data planes but not the metadata
# tail) - floor sits just under the payload.
FRANK_ARCHON_RAD_MIN_MIB=110000 FRANK_ARCHON_RAD_MIN_MIB=110000
+6 -6
View File
@@ -1,9 +1,9 @@
# frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint # frank-archon.service — radiance (StillDeadcode, Apache-2.0) endpoint
# (the publisher's 4-bit-codebook container of the official # (OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 at NATIVE block-FP8 precision, the
# Qwen/Qwen3.8-Flash-Next, qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by # rad-convert container qwen4exp-fp8-native.rad) on BOTH R9700s (TP=2,
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2, wht6 wire), # wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (git tag
# dankserver :7770. 2026-10-15 payload swap from the in-house w4dn8 (the # frank-archon-freetoken-pr587-20261005 is the pre-cutover state; the
# swap commit is the revert; its .rad stays on disk). # retired unit/env/serve trio and Dockerfile are in that tag).
# #
# Same port, same served alias, same container identity as the unit name; # Same port, same served alias, same container identity as the unit name;
# the harness models.yml id is unchanged. The serve script drives # the harness models.yml id is unchanged. The serve script drives
@@ -19,7 +19,7 @@
# unit grants LimitMEMLOCK=infinity to the dank process via PID1; the # unit grants LimitMEMLOCK=infinity to the dank process via PID1; the
# engine still runs out of dank's rootless podman storage. # engine still runs out of dank's rootless podman storage.
[Unit] [Unit]
Description=frank-archon — radiance Qwen3.8-Flash-Next (publisher iq4r) on 2x R9700 TP=2 Description=frank-archon — radiance Qwen3.8-Flash-Next-FP8 (native) on 2x R9700 TP=2
After=network-online.target After=network-online.target
Wants=network-online.target Wants=network-online.target
# Never let the start-rate limiter give up after a crash loop (a failed LLM # Never let the start-rate limiter give up after a crash loop (a failed LLM
+7 -6
View File
@@ -42,10 +42,10 @@
# form of the engine's default, so the seam test can pin the product. # form of the engine's default, so the seam test can pin the product.
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB. # KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP # HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
# drafter is NOT slot-locked on this engine (batching and speculation # drafter is NOT slot-locked on this engine: some engines silently
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own # disable drafting the moment a second slot is enabled, this one does not
# quickstart ships 2 slots WITH the head), so the second slot is # (the card's own quickstart ships 2 slots WITH the head), so the second
# concurrent capacity, not a speculation tax. # slot is concurrent capacity, not a speculation tax.
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is # Memory ledger against the box's 121.2 GiB unified (the n-gram table is
# mmap'd page cache beside the checkpoint, not a pinned charge — the # mmap'd page cache beside the checkpoint, not a pinned charge — the
# card's own quickstart arithmetic does not bill it): # card's own quickstart arithmetic does not bill it):
@@ -88,8 +88,9 @@
# before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and # before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
# amdgpu.noretry=0 (check at deploy). # amdgpu.noretry=0 (check at deploy).
# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest # Image: digest-pinned (a tag is a mutable pointer — the same tag can
# is what was qualified). 0.16.2 = the version the model card names as its # silently deliver different bits — so the digest pins exactly the bits
# that were qualified). 0.16.2 = the version the model card names as its
# quickstart pin; RepoDigest captured at pull on 2026-10-06. # quickstart pin; RepoDigest captured at pull on 2026-10-06.
FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
# Container identity: kept from the 27B era (unit ExecStop names it). # Container identity: kept from the 27B era (unit ExecStop names it).
+1 -1
View File
@@ -16,7 +16,7 @@
# custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so # custom-quant lock-in like the ROCmFPX era). NO nextn/mtp tensors exist, so
# in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload. # in-file MTP is impossible here; SPEC_TYPE=none is forced by the payload.
# #
# KV rate law: 8 full layers × 4 KV heads × 256 dim = 8,192 elements per # KV rate arithmetic: 8 full layers × 4 KV heads × 256 dim = 8,192 elements per
# token per cache. f16 K + f16 V = 32,768 B/token = 32 KiB/token, so the # token per cache. f16 K + f16 V = 32,768 B/token = 32 KiB/token, so the
# 401,408-token pool is 12.25 GiB. The 24 DeltaNet layers cost a fixed # 401,408-token pool is 12.25 GiB. The 24 DeltaNet layers cost a fixed
# recurrent state per SLOT (not per token) — that is why two 200k slots fit # recurrent state per SLOT (not per token) — that is why two 200k slots fit
@@ -29,7 +29,8 @@
# its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and # its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
# getattr_static()s each one off the instance, breaking on the first miss. # getattr_static()s each one off the instance, breaking on the first miss.
# So BOTH wrappers need all three attributes. # So BOTH wrappers need all three attributes.
# * MRO law: INHERITING SupportsEagle3 as a base dies at class creation with # * Method-resolution order forbids the obvious inheritance: adding
# SupportsEagle3 as a base dies at class creation with
# "Cannot create a consistent method resolution order (MRO) for bases ..." # "Cannot create a consistent method resolution order (MRO) for bases ..."
# (GLM's wrappers already carry four/five Protocol-derived bases). The # (GLM's wrappers already carry four/five Protocol-derived bases). The
# attribute route avoids the MRO entirely. Also proven by simulation: # attribute route avoids the MRO entirely. Also proven by simulation:
+2 -1
View File
@@ -9,7 +9,8 @@ No scratch container, no mask: the service under test is the service.
Four probes, in increasing cost: Four probes, in increasing cost:
A decode at depth — one ~51k-token chat completion, 512 tokens, greedy. A decode at depth — one ~51k-token chat completion, 512 tokens, greedy.
Reports the engine's own `timings` (prefill, decode, draft acceptance). Reports the engine's own `timings` (prefill, decode, draft acceptance).
The bench law applies: >=128 generated tokens, greedy, real text. Measurement discipline: generate at least 128 tokens (a short run
measures first-token noise, not the server), greedy, real text.
B the capacity contract — as many CONCURRENT chats of ~197k prompt tokens as B the capacity contract — as many CONCURRENT chats of ~197k prompt tokens as
the deployed `FRANK_ARCHON_MAX_SEQS` (read from the repo env file on this the deployed `FRANK_ARCHON_MAX_SEQS` (read from the repo env file on this
host), each carrying a unique marker at its head. All must answer their own host), each carrying a unique marker at its head. All must answer their own
@@ -1,9 +1,9 @@
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next # qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official # (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down -- # base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law # through GPTQ into the w4nl64a8h table. The format assigns exactly ONE encoding per table
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served # ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 + # beside it", qwen4exp_fp8 rad_arch_declare — so the original split tier (down block-fp8 +
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it. # gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers # The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head. # and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
@@ -26,8 +26,8 @@
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE # The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only. # CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites # The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's # this section from Swift's OWN calibration recording before this recipe is ever run, by the
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three # protection rule -- an expert whose share of its layer's down-input energy is at least 3% and at least three
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM # times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert. # takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py) # PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
+6 -8
View File
@@ -1,13 +1,11 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode, # serve-frank-archon.sh — frank-archon endpoint: radiance (StillDeadcode,
# Apache-2.0) serving the PUBLISHER's 4-bit-codebook container of the # Apache-2.0) serving the OFFICIAL Qwen/Qwen3.8-Flash-Next-FP8 checkpoint at
# official Qwen/Qwen3.8-Flash-Next (the engine author's own rad-convert of # NATIVE block-FP8 precision from a rad-convert container
# the bf16 checkpoint: qwen3.8-next-flash-fp8-iq4r-moe.rad, fetched by # (/home/dank/models/rad/qwen4exp-fp8-native.rad), on BOTH R9700s (TP=2,
# scripts/download-frank-archon-iq4r.sh) on BOTH R9700s (TP=2, wht6 wire), # wht6 wire), dankserver :7770. 2026-10-05 cutover from FreeToken (PR #587,
# dankserver :7770. 2026-10-15 payload swap from the in-house w4dn8 of the # one card; git tag frank-archon-freetoken-pr587-20261005 is the
# Swift1.5 finetune (payload on disk = rollback; the swap commit is the # pre-cutover state).
# revert); 2026-10-05 radiance cutover from FreeToken (git tag
# frank-archon-freetoken-pr587-20261005 is the pre-radiance state).
# #
# The run line below IS the engine argv: the image's entrypoint is the # The run line below IS the engine argv: the image's entrypoint is the
# engine and its tools sit beside it on PATH. Unlike every predecessor # engine and its tools sit beside it on PATH. Unlike every predecessor
+3 -1
View File
@@ -70,7 +70,9 @@ fi
# Size-exact gates on all three payload files: a truncated curl exits 0 on # Size-exact gates on all three payload files: a truncated curl exits 0 on
# some failures and a partial .hgn loads far enough to look alive, so the # some failures and a partial .hgn loads far enough to look alive, so the
# byte counts from the Hub tree API are the contract (fleet fetch law). # byte counts from the Hub tree API are the contract: curl exits 0 on an
# HTTP error and writes the error body as the file, so a truncated download
# is caught by size here and never boots.
ck() { ck() {
# ck <env-name> <path> <expected-bytes> # ck <env-name> <path> <expected-bytes>
if [ ! -f "$2" ]; then if [ ! -f "$2" ]; then
+3 -2
View File
@@ -63,8 +63,9 @@ fi
: "${CACHE_TYPE_K:=f16}" : "${CACHE_TYPE_K:=f16}"
: "${CACHE_TYPE_V:=f16}" : "${CACHE_TYPE_V:=f16}"
# Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so # Speculative decoding is OFF: this Q6_K file carries NO nextn/mtp tensors, so
# there is no in-file head (the law: MTP is a property of the file, not of the # there is no in-file head — this kind of drafter exists only as tensors
# flags). SPEC_TYPE=none strips BOTH speculative flags (flags move as a set); # inside the checkpoint file, and no flag can supply one the file lacks.
# SPEC_TYPE=none strips BOTH speculative flags (they move as a set);
# naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4 # naming a drafter requires SPEC_DRAFT_FILE alongside SPEC_TYPE — the Gemma4
# assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35. # assistant on disk is arch gemma4-assistant and cannot drafter-match qwen35.
: "${SPEC_TYPE:=none}" : "${SPEC_TYPE:=none}"
+1 -1
View File
@@ -298,7 +298,7 @@ def main():
and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128]) and list(dssh) == [(dsh[0] + 127) // 128, (dsh[1] + 127) // 128])
if not scale_law: if not scale_law:
log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} " log(f"ERROR: unexpected expert shapes: gate {gdt}{gsh} scale {gsd}{gssh} "
f"down {ddt}{dsh} scale {dsd}{dssh} (scale law 128x128)") f"down {ddt}{dsh} scale {dsd}{dssh} (128x128 block scales)")
return 2 return 2
shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],), shapes = {"ffn2_h": (gsh[0] * 2, gsh[1]), "ffn": (dsh[1],),
"gs": tuple(gssh), "ds": tuple(dssh)} "gs": tuple(gssh), "ds": tuple(dssh)}
@@ -452,8 +452,7 @@ access is decided entirely by GROUP MEMBERSHIP consumed downstream —
- `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and - `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
every human account). every human account).
- **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic, - **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
debo, freya, mitch, eric — mitch (pk 22) and eric (pk 23) created debo, freya — kirby/ry/nic/debo/freya (pk 17–21) created 2026-10-09, kassie
2026-10-15, kirby/ry/nic/debo/freya (pk 17–21) created 2026-10-09, kassie
and alex created 2026-10-08, artem (pk 16) created 2026-10-07): `username` + and alex created 2026-10-08, artem (pk 16) created 2026-10-07): `username` +
`<user>@franksdank.cloud` + display name + `<user>@franksdank.cloud` + display name +
`is_active=True` + the password set via `set_password`, membership in `is_active=True` + the password set via `set_password`, membership in
@@ -1,144 +0,0 @@
# frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, 2× R9700 TP=2
**Status: LIVE — deployed 2026-10-15.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
The in-house w4dn8 conversion of the Swift1.5 finetune
([[frank-archon-radiance-swift15-w4dn8]]) is replaced by the **engine
author's published container**:
[`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`](https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe)
— the official `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint rad-converted by
StillDeadcode. Same port, alias, container name, systemd unit, digest-pinned
engine pin **1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…`), and
**every serving argument carried verbatim** from the 2026-10-13 shape.
Rollback: revert the swap commit — the w4dn8 `.rad` stays on disk.
## The payload
`qwen3.8-next-flash-fp8-iq4r-moe.rad` — **121,969,901,568 B = 116,322 MiB**,
sha256 `0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20`
(Hub LFS oid), fetched byte-exact by `scripts/download-frank-archon-iq4r.sh`
(hfget resume + size + sha256; staging dir
`/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe/` keeps the
card, LICENSE, and recipe `qwen4exp-w4nl64-i8-hc8m` as provenance).
| Plane | Encoding (publisher recipe `qwen4exp-w4nl64-i8-hc8m`) |
| --- | --- |
| Routed experts | 4-bit codes into a 16-level **non-uniform** codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes by GPTQ (10M-token calibration, per-expert down-proj Hessians) + 3 coordinate-descent sweeps |
| Protected experts | **ten** per layer, whole-expert bf16 (down-projection energy) |
| Trunk + lm_head | int8 W8A8, scale per 128 columns searched; HC mixtures E4M3 |
| PLE n-gram | fp8 fixed-scale, in-container (served `--ngram-placement ram`, same 47.7 GiB pin law) |
| Speculator | the model's own MTP head (card default depth 3; served at 2) |
| Vision | bf16 vision tower, in-container |
**Base model: the OFFICIAL (censored) Qwen3.8-Flash-Next** — the fleet's
only frontier model at base alignment; the abliterated family lives on
frank-dragoon. Card quality: full-vocab teacher-forced KL vs bf16 over
~75K positions — mean 0.0789 all-tokens / 0.0239 assistant turns, top-1
agreement 92.85 %.
## Why this swap
- **Publisher-graded precision**: the w4nl codebook + coordinate-descent
refinement is the engine author's own quality ceiling for this
checkpoint; the card's KL ladder (p99 1.65, p99.9 5.33) is published and
reproducible, where our w4dn8 gate measured against our own reference.
- **Formal parity**: the served file is the one the engine author supports
— decode/prefill behavior on 1.0.13 is their tested path (the card's run
line speaks the 1.0.13-era flags; the `--vram-weights-mib`/`--vram-kv-mib`
pair we ship is accepted by pin 1.0.13 per the 2026-10-06 A/B, so the
shape needed no translation).
- **Cost**: the Swift1.5 finetune's refusal-bending behavior leaves this
endpoint (it returns to the official base alignment); the 4-slot/200k
economics and all shape laws carry over unchanged.
## Shape (unchanged from 2026-10-13)
All of it, verbatim — the payload moved 116,416 → 116,322 MiB, inside the
same budget:
- `--tp 2 --tp-wire wht6`, `--max-num-seqs 3`, `--max-model-len 200000`
(capacity contract 3 × 200,000), `--kv-cache-dtype fp8`
- `--placement expert_tiered --host-pool-mib 44928` + explicit
`--vram-weights-mib 25600 --vram-kv-mib 5120` (30,720 of the engine's
30,969 MiB ceiling; pool = contract + 1.3 %)
- `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`
(MNBT 2048 law), prefix cache 8 GiB host + 64 GiB disk,
`--ngram-placement ram`, froggeric v22.5 `--override-chat-template`
- TTM preflight floor 150,000 / target 204,800 MiB; pinned ledger
2 × 44 GiB pool + 47.7 GiB PLE + 8 GiB prefix ≈ 147 GiB (see
[[frank-archon-radiance-swift15-w4dn8]] for the ledger arithmetic and
the KV dtype law — both are engine properties, payload-independent)
The publisher's card run line (`--host-pool-mib 12288`,
`--expert-vs-cache-ratio 0.82`, seqs 8, spec 3, ~32 GB host RAM) is their
shape on stock 32 GB cards; the fleet ships its own measured shape against
the same file.
## Cutover record — 2026-10-15
1. `rm` (Frank's one-time authorization): the raw
`ukisai/Swift1.5-Qwen3.8-Flash-Next` BF16 checkpoint (336 GiB, spent
conversion source) and `qwen4exp-fp8-native.rad` (173 GiB, two eras
back) left `/home/dank/models` — 150 GB free → 658 GB free. The fp8-native
payload is **no longer on disk**: rebuild =
`scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh`.
2. Fetch: `scripts/download-frank-archon-iq4r.sh --fetch` as transient unit
`iq4r-fetch` (detached-job protocol; log
`/home/dank/logs/iq4r-fetch.log`, rc sentinel `…rc`).
3. Deploy: `scripts/deploy.sh dankserver --service frank-archon` — launched
detached (the swap restarts the very endpoint the operator session runs
on; the harness timeout catches the session on return).
4. **Boot ledger (2026-10-15, first boot, pin 1.0.13 — the w4nl codebook
loaded, no pin bump)**: `kv_attn` fp8 page **26.00 KiB** at **607,740
tokens** (151,935 blocks = 3.77 GiB) — byte-identical to the w4dn8 boot,
the sizing law carries; `kv_qsa_bkey` 3.25 KiB at the same token count;
`kv_gdn_state` 54 MiB × 6 slots, `kv_gdn_conv` 1.76 MiB × 3,
`kv_qsa_tail` 19.5 KiB × 6; pool **5.00 GiB**, prefix checkpoints 8 ×
56 MiB; the sizing line reads `kv cache 5.00 GiB … backs 3 sequences all
at 200000 tokens at once, which is the worst case`; samplers print **9
rows** per rank (3 × (1+2), inside the wht6 25-row bound);
`speculation: 'mtp' drafts 2 token(s) a step in 2 serial pass(es)` +
prompt-lookup fallback; slab **17,610 (rank0) / 17,658 (rank1) slots**
(64 shadow) — above the w4dn8 record of 17,548 (ten bf16-protected
experts per layer replace twenty-four, so more of the 4-bit table
resides), pinned pool 7,532/7,484 slots; KV lending **4.22 GiB**
(3,571/3,568 mover slots); arenas 896.02/988.17 MiB charged before the
pair split; prefix cache RECOVERED 900,176 blocks + 126 checkpoint slots
from the 64 GiB disk tier (same arch + tokenizer as w4dn8 — the tier
survived the payload swap); froggeric template 28,234 B; vision encoder
live (≤ 4,194,304 px, 16,384 patches/pass). Health gate passed in ~100 s
(radiance binds the port after the full load).
5. **Vision probe (2026-10-15)**: 300 px solid-red PNG via `image_url` —
reasoning stream: "a solid fill of a deep, saturated red hue". Content
came back EMPTY at `max_tokens 64`: the thinking budget consumed all 64
(`reasoning_tokens: 64`, finish `length`) — fleet law, probe thinking
models with ≥ a few hundred tokens.
6. **e2e gate**: deploy exited DEPLOY INCOMPLETE — 30/33 Playwright green,
3 red (`12-crawl`, `13-sso` search SSO, `16-marreta`) on DANKCLAW
surfaces (vhosts intermittently returned `ERR_CONNECTION_RESET` from the
workstation); frank-archon's own health gate passed. The swap touched no
dankclaw runtime file (only homepage services.json text). Census of the
Caddy/auth stack pending; see `log.md`.
7. **Post-boot measurements (pending)**: decode ladder (99.0 t/s single
@ 43.5k, 93.3 @ 197k, 3-way 113.9/43.3/10.9 = 18.4 t/s aggregate —
these are the **w4dn8** numbers, the record to beat, not this file's),
slab-limited prefill (22208/3,397 → 25600/3,584 t/s A-cold 43.5k), draft
acceptance, and `scripts/probe-frank-archon-kv.py` leg D.
## Rollback
Revert the swap commit (`scripts/frank-archon.env` MODEL_FILE line) +
redeploy: `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad`
(116,416 MiB) stays on disk, KLD-gated, same shape. The fp8-native era is
rebuild-only (see the cutover record). The w4dn8 runbook keeps every shape
law, the KV dtype table, the slab ladder, and the tuning levers:
[[frank-archon-radiance-swift15-w4dn8]].
## Related
- [[frank-archon-radiance-swift15-w4dn8]] — the replaced payload and the
full shape-law library (slab slope, KV dtype arithmetic, TTM law)
- [[frank-archon-radiance-qwen38-native-fp8]] — the fp8-native era
(rebuild-only now; owns the descriptor-selection law)
- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
- [[deployment]] — the service-scoped deploy this cutover rides
+7 -25
View File
@@ -3,17 +3,14 @@
**Dankserver runs the radiance engine (StillDeadcode) on frank-archon** **Dankserver runs the radiance engine (StillDeadcode) on frank-archon**
(digest-pinned `stilldeadcode/radiance` image, engine as entrypoint, (digest-pinned `stilldeadcode/radiance` image, engine as entrypoint,
driving `podman run` directly, root systemd unit, :7770): driving `podman run` directly, root systemd unit, :7770):
the **publisher's** 4-bit-codebook container of the official `ukisai/Swift1.5-Qwen3.8-Flash-Next` converted to **GPTQ w4**
`Qwen/Qwen3.8-Flash-Next` (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB, (`qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB, KLD-gated), on BOTH R9700s
sha256-verified at fetch — w4nl codebook experts + int8 trunk, PLE n-gram,
MTP head and vision tower in-container), on BOTH R9700s
TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache TP=2 (`--tp-wire wht6`), experts in a 44 GiB/rank pinned host pool + VRAM cache
(explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the (explicit `--vram-weights-mib 25600 --vram-kv-mib 5120` = 30,720 of the
30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE 30,969 MiB the engine's own budget table makes claimable), the 47.7 GiB PLE
table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2, table pinned to host RAM, **fp8 KV at 3 × 200,000**, MTP spec decode depth 2,
froggeric v22.5 template. Rollback: the in-house Swift1.5 w4dn8 `.rad` (on froggeric v22.5 template. Rollback tags frank-archon-freetoken-pr587-20261005
disk); tags frank-archon-freetoken-pr587-20261005 and and frank-archon-vllm-shapeG-20261003),
frank-archon-vllm-shapeG-20261003),
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2 and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
(frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned (frank-dragoon, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
@@ -40,10 +37,10 @@ graph LR
gfx1201, 32 GiB each] gfx1201, 32 GiB each]
B2[radiance StillDeadcode, digest-pinned image B2[radiance StillDeadcode, digest-pinned image
podman run, frank-archon, port 7770] podman run, frank-archon, port 7770]
C[Publisher qwen3.8-next-flash-fp8-iq4r-moe .rad C[Swift1.5-Qwen3.8-Flash-Next GPTQ w4 .rad
2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM, 2x R9700 TP=2 wht6, experts 44 GiB/rank pinned pool + VRAM,
PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2, PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 2,
3 x 200000, froggeric v22.5] 4 x 200000, froggeric v22.5]
end end
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04] subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
@@ -156,22 +153,7 @@ frank-zealot, port 7771]
target; payload on disk; owns the measured boot-pair table)**: the same target; payload on disk; owns the measured boot-pair table)**: the same
runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower — runtime and the same 2 x 200,704 + f16 KV shape one quant rung lower —
104.8 tok/s, 3,571 MiB true free 104.8 tok/s, 3,571 MiB true free
- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE since 2026-10-15**: - [[frank-archon-radiance-swift15-w4dn8]] — **ACTIVE since 2026-10-08**:
the engine author's published container of the OFFICIAL (base-alignment)
Qwen3.8-Flash-Next — `qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
Hub-sha256-verified at fetch: 4-bit codes into a 16-level non-uniform
codebook after a Walsh–Hadamard rotation (GPTQ + 3 coordinate-descent
sweeps, 10M-token calibration), ten bf16-protected experts per layer,
int8 trunk/lm_head, fp8 HC + PLE, MTP head, bf16 vision tower; card KL
0.0789 all-tokens / top-1 92.85 % vs bf16. **MODEL_FILE is the only line
that moved** — the 2026-10-13 shape (25600/5120 pair, 44928 pool, fp8 KV
3 × 200,000, MTP 2, wht6, MNBT 2048, froggeric v22.5) carries verbatim.
Rollback: revert the swap commit (w4dn8 `.rad` stays on disk). The
fp8-native `.rad` and the raw Swift1.5 BF16 source were DELETED from
dankserver 2026-10-15 (Frank's one-time rm, volume for this payload;
rebuild via the fetch + build scripts named in the fp8-native runbook)
- [[frank-archon-radiance-swift15-w4dn8]] — **RETIRED 2026-10-08 → 10-15
(superseded by the publisher iq4r swap; payload on disk as the rollback)**:
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` on radiance
1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container); 1.0.13 (recipe + 10M-token calibration + KLD gate vs the BF16 container);
the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the the FP8-native payload remains the rollback. **2026-10-13: fp8 KV, and the
@@ -0,0 +1,199 @@
---
type: reference
title: LLM Usage Attribution
description: Research catalog of every viable way to answer "who is using the fleet LLMs" — engine-native identity, gateways, auth proxies, tracing platforms, network/host attribution — with the measured fleet baseline (2026-10-13) and a phased adoption plan.
tags: [llm, observability, auth, gateway, litellm, tracking, identity]
timestamp: 2026-10-14T00:00:00-07:00
---
# LLM Usage Attribution
Deep-research catalog (2026-10-13/14): solutions for tracking WHO uses the
fleet endpoints (`frank-archon`, `frank-dragoon`, `frank-zealot`). Full web
citations; live-fleet measurements marked **(measured)**.
## Baseline: what the fleet answers today
- All three endpoints are `auth: none`, plain HTTP, LAN-only
(`models.yml` providers; no vhost on 7770/7771).
- **(measured)** Live `/metrics` label census: `frank-archon` exports
`vllm:*`+`radiance:*` with labels only `{model_name, engine, le,
finished_reason, position, class}` — zero identity labels.
`frank-dragoon` exports `halogen:*`+`llamacpp:*`.
`frank-zealot` answers **501** — `serve-frank-zealot.sh` never passes
`--metrics`, so the Prometheus job is structurally never-green
(`LlmEndpointDown` fires). Fix: add `--metrics` to the ARGS array.
- **(measured)** frank-zealot verbosity-4 journal lines carry slot/task/timing
only — no client IP, no user. Alloy ships journal + podman ctr.log to Loki,
so what lands there has no identity either.
- Caddy has no `log` directive anywhere → no per-request access logs on the
OpenWebUI vhost.
- Net: fleet-level token telemetry exists per ENDPOINT; per-CONSUMER
identity exists nowhere.
## The identity layers (pick the layer, get its resolution)
| # | Layer | Identity | Answers |
|---|---|---|---|
| L0 | DHCP/ARP | MAC, hostname, lease | which device was on the LAN |
| L1 | DNS (Pi-hole v6) | device → queries | which device resolved/used a name |
| L2 | L3/L4 (nftables/conntrack, JA3/JA4) | IP, TLS client lib | bytes per client, client software |
| L3 | Transport (API keys, mTLS, SPIFFE, Tailscale) | key/cert/node | which credential connected |
| L4 | App gateway (virtual keys → user/team/end_user) | named consumer | tokens/spend/cost per who |
| L5 | Trace/content (Langfuse, Helicone, halogen records) | user_id, session, prompt text | what they asked, per session |
| L6 | Client host (auditd, eBPF, netdata) | pid/uid/comm per connect | which process on which host |
Keys/headers/`user` fields are CLIENT-DECLARED — they group traffic, they
prove nothing about a human (halogen's own docs warn this verbatim:
"they group requests, and they prove nothing about who sent them").
## Catalog
### A. Engine-native (per-consumer identity inside the inference server)
| Engine | AuthN | Identity in logs | Per-identity metrics | OTel | Notes |
|---|---|---|---|---|---|
| llama.cpp | `--api-key` (multi, comma) / `--api-key-file` **(measured in the deployed build's `--help`)** | request `user` field drives slot assignment/eviction; `--log-requests` dumps bodies | `--metrics` exports `llamacpp:*`, NO key/user label | none upstream | [server README](https://github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md) |
| vLLM | `--api-key` (multi) | request log; no user persistence | `vllm:*`; NO `api_key` label (cardinality, issue #53926) | native (`--otlp-traces-endpoint`) | [metrics design](https://github.com/vllm-project/vllm/blob/main/docs/design/metrics.md) |
| radiance | no public auth docs **[unverified]** | — | **(measured)** no identity labels on live /metrics | none found | closed deployment surface; treat authN as absent until probed |
| halogen-flash-server | `HALOGEN_API_KEY` / `HALOGEN_API_KEY_FILE` (multi, comma) | **`HALOGEN_RECORD_DIR` request records**: `turns/*.jsonl` per request — `client_family` (User-Agent), tokens, timings, finish, tool calls, plus HASHES of `user` / `prompt_cache_key` / `safety_identifier` / `metadata.user_id` (`.salt`-keyed); `units/` content-addressed text; caps `HALOGEN_RECORD_GIB`=2, `HALOGEN_RECORD_DAYS`=90; `HALOGEN_RECORD_TEXT=none` for metadata-only; `stats`/`prune` tools in-container | `halogen:*`+`llamacpp:*`, no per-key label | none | [configuration.md](https://github.com/peonist-ai/halogen-flash-server/blob/main/docs/configuration.md) — verbatim-verified 2026-10-13. Best native ledger of the fleet. |
| SGLang | `--api-key` (single) | `--log-requests` | `sglang:*`, no key label | native (`--enable-trace`) | for any future swap |
| Ollama | single key | request log | `ollama:` no key label | none | not in fleet |
Wire fields usable from clients today (all client-declared): OpenAI
`user`, `safety_identifier`, `prompt_cache_key`, Messages
`metadata.user_id`; LiteLLM `end_user_id`; gateway header conventions
(`X-Forwarded-User`, `Helicone-User-Id`, `cf-aig-metadata`).
### B. LLM gateways (front door with virtual keys — the only layer with
per-consumer TOKEN counts)
| Solution | Identity | Usage lands | License | Footprint |
|---|---|---|---|---|
| **LiteLLM Proxy** | virtual keys → `user_id`/`team_id`/`end_user`; budgets, rate limits | Postgres spend ledger; `/metrics` labels `api_key_alias,user,team,end_user,model,client_ip,user_agent` (`litellm_*_tokens_metric`, spend, budget gauges); OTel/Langfuse callbacks | MIT core (SSO/RBAC/audit-log enterprise) | ~150–300 MiB + Postgres |
| Kong AI Gateway | Consumer + key-auth/JWT/OIDC/mTLS; AI consumer groups | Postgres; `kong_*` per-consumer; file-log/zipkin/otel | OSS core; token-aware AI plugins enterprise | 0.5–1 GiB |
| Apache APISIX | Consumer + key-auth/JWT | `apisix_llm_prompt_tokens`/`completion_tokens` with `consumer` label; http-logger/kafka | Apache-2.0 | 200–400 MiB |
| Bifrost (maximhq) | virtual keys `sk-bf-*`/`x-bf-vk`, budgets | Prometheus/OTel, config store (SQLite/Postgres) | Apache-2.0, Go | ~150–300 MiB |
| Portkey OSS gateway | virtual keys | logs/OTel | MIT | 200–400 MiB |
| Higress AI GW | keys (+OIDC ent) | `higress_*`, ai-logging | Apache-2.0 CE | Istio-based, 0.5 GiB+ |
| Envoy AI GW / Gateway API Inference Extension / LLM Gatekeeper | delegated auth | Envoy metrics/access logs | Apache-2.0 | K8s-native |
| Gloo / KrakenD / Gravitee AI gateways | keys, OIDC | metrics/logs | OSS core, AI features enterprise | heavy |
| Cloudflare AI Gateway | Access JWT `cf.user_id`, `cf-aig-metadata` | Cloudflare-managed analytics/logs | SaaS (exotic for LAN; Tunnel possible) | edge |
([LiteLLM virtual keys](https://docs.litellm.ai/docs/proxy/virtual_keys),
[prometheus](https://docs.litellm.ai/docs/proxy/prometheus),
[APISIX ai-proxy](https://apisix.apache.org/docs/apisix/plugins/ai-proxy/),
[Bifrost](https://github.com/maximhq/bifrost),
[Kong AI](https://developer.konghq.com/index/ai-gateway/))
### C. Auth proxies / identity infrastructure (identity without token counts)
- **Ory Oathkeeper** (MIT, ~50–100 MiB): per-consumer api-key/JWT/
client-credentials → header mutation (`X-User: {{ .Subject }}`) +
decision logs.
- **oauth2-proxy** (Apache-2.0): OIDC (Authentik) incl. client_credentials,
`--pass-user-headers` → `X-Forwarded-User`; ~100 MiB.
- **Authentik itself**: proxy-provider custom headers (already injects
`X-Authentik-Username`/`-Groups`, spoof-stripped — the /whoami seam proves
the pattern on this host); event log + webhook forwarding.
- **Caddy `tls client_auth`** (mTLS per client cert) + access-log fields —
Caddy already fronts both hosts.
- **Smallstep step-ca**: short-lived per-device certs, ACME-style.
- **SPIFFE/SPIRE**: per-workload SVIDs (k8s-flavored; heavy for quadlets).
- **Tailscale / NetBird**: per-node identity, ACL + network-flow logs,
headscale for self-hosted control. Best for off-LAN laptops.
- **ApiAuth** (nginx auth_request): the minimal per-consumer apikey sidecar.
### D. LLM observability / tracing (prompt-level per-user audit)
| Platform | Identity | Content | Self-host | Shape |
|---|---|---|---|---|
| **Langfuse** | `user_id`,`session_id`, tags | full input/output, masking hook | ✅ ClickHouse, Apache-2.0 | fed via LiteLLM callback / OTLP / SDK |
| **Helicone** | api-key→user, `Helicone-User-Id` | full, tiered redaction | ✅ compose (ClickHouse+PG+Redis+Jawn), Apache-2.0 | PROXY — can be the front door itself |
| Arize Phoenix | `user.id`,`session.id` | opt-in `gen_ai.*` | ✅, Apache-2.0 | OTLP collector |
| OpenLIT | `user.id`,`session.id` | 3-tier capture | ✅, Apache-2.0 | OTLP+ClickHouse |
| SigNoz LLM | OTel attrs | opt-in | ✅, Apache-2.0 | OTLP |
| OpenLLMetry/Traceloop | assoc props | opt-in | ✅ Helm | OTel |
| Lunary / W&B Weave / LangSmith | user/session | full | enterprise-gated | — |
OTel GenAI semconv (Development-stability): `gen_ai.operation.name`,
`gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.usage.input_tokens`/
`output_tokens`, `gen_ai.conversation.id`, content attrs opt-in; `user.id`
from the registry. vLLM and SGLang emit OTLP natively; llama.cpp only via
external SDK — **the fleet's engines are OTel-silent at request level**,
which is why the gateway callback (LiteLLM→Langfuse) is the practical feed.
### E. Network/host passive attribution (no client cooperation)
- **Pi-hole v6** (already on Dankclaw): query log API per device,
`pihole6_exporter` + query-logger→Loki (GPLv3 exporter).
- **Caddy access logs**: add a `log` directive (JSON) → Alloy→Loki; with the
auth chain it carries `X-Authentik-Username` for browser users.
- **OpenWebUI admin**: per-user daily token stats, events/logs, analytics API
(`/api/v1/analytics/*`) — per-USER identity exists here already (MIT).
- **auditd** connect-syscall rules (uid/pid/comm per connect to 7770/7771),
ship via audisp→Loki.
- **eBPF**: Tetragon (process+socket events, policies), Coroot, DeepFlow,
bpftrace `tracepoint:syscalls:sys_enter_connect` one-liners. K8s-shaped
for single hosts, but agents.plugin-style tools work.
- **nftables named counters** per consumer IP (bytes; reload resets).
- **netdata apps.plugin**: per-process network bytes, auto-discovered.
- **JA3/JA4 TLS fingerprinting**: client library only, TLS-only; the LAN is
plaintext HTTP today → use the HTTP **User-Agent census** instead (free
from any access log).
- **DHCP/ARP census** (leases + `ip neigh`): device roster.
## Phased plan for this fleet
**Phase 0 — repo-only, no new services**
1. frank-zealot: add `--metrics` (fixes the measured 501 + the
structurally-red Prometheus job + the firing `LlmEndpointDown`).
2. frank-dragoon: set `HALOGEN_API_KEY_FILE` (multi-key per consumer) +
mount `HALOGEN_RECORD_DIR` (start `HALOGEN_RECORD_TEXT=none`; flip to
`all` if prompt audit is wanted) + Alloy ship of `turns/`.
3. frank-zealot: `--api-key-file` (multi-key, verified in the deployed
build); radiance: probe for `--api-key` support (contract facts are
silent — add to `fleet-probes`).
4. models.yml: per-consumer bearer keys (`auth: none` → key) — the provider
schema supports apiKey (pi-era conversion note).
5. Caddy: `log` on `ai.franksdank.cloud` (OpenWebUI browser census, JSON →
Loki).
Limits: keys gate access, and only the engines' request records (halogen)
see them per-request; llama.cpp/radiance/vLLM metrics stay identity-blind.
**Phase 1 — LiteLLM Proxy quadlet on Dankclaw = the front door.**
One virtual key per consumer (omp-<workstation> ×N, openwebui, hermes,
agents, bench). Per-key/user/team Prometheus labels into the existing
Grafana+Alertmanager; Postgres spend ledger; budgets + rate limits per
key. This is the ONLY layer that yields tokens-per-who without engine
cooperation. (Bifrost is the lighter Apache-2.0 Go alternative; Helicone
the proxy-native-observability alternative.)
**Phase 2 — prompt-level audit.** Langfuse self-hosted fed by the LiteLLM
callback: per-user traces, sessions, cost; masking + retention knobs.
**Phase 3 — ambitious identity infra.** step-ca + Caddy mTLS per device;
Tailscale (+flow logs) for off-LAN; auditd/eBPF process attribution on the
agent hosts; Pi-hole query logger; nft counters per consumer IP. Browser
users stay owned by OpenWebUI analytics + its per-user API keys; a pipeline
can forward `X-Forwarded-User` so the gateway sees them too.
## Cautions
- Client-declared identity (keys, `user`, headers) groups traffic; it does
not authenticate humans. Authenticated identity = Authentik (browser) /
mTLS (device) / Tailscale (node).
- Plain-HTTP LAN = bearer keys sniffable by any tenant; treat keys as
identity labels on the LAN, not secrets, until TLS/mTLS at the door.
- Prompt-content logging stores a plaintext copy of everything — set
retention caps (halogen GIB/days; Langfuse masking; Loki retention) and
gitignore the record dirs.
- Household-scale cardinality is safe for per-key Prometheus labels;
the engines' refusal of key labels is a multi-tenant concern, not ours.
## Related
- [[Fleet Audit and Posture]] — monitoring gaps census
- [Observability](../Infrastructure/Observability.md) — the Loki/Prometheus
pipes every sink above uses
- [LLM index](../LLM/index.md) — endpoint runbooks (env trios the Phase-0
edits touch)
@@ -23,6 +23,9 @@ Procedures for operating the fleet. The LLM-specific pages live under
posture, whole-fleet audit, bulk redaction, stash audits posture, whole-fleet audit, bulk redaction, stash audits
- [[SSH and Remote Access]] — passwordless bootstrap without sshpass, fish - [[SSH and Remote Access]] — passwordless bootstrap without sshpass, fish
login shells, GUI-hang diagnosis over ssh login shells, GUI-hang diagnosis over ssh
- [[LLM Usage Attribution]] — who-is-using-the-fleet-LLMs research catalog:
engine-native identity, gateways, tracing, network/host attribution, and
the phased plan
## Related ## Related
- [Observability Provisioning](../Infrastructure/Observability%20Provisioning.md) - [Observability Provisioning](../Infrastructure/Observability%20Provisioning.md)
+5 -10
View File
@@ -134,6 +134,9 @@ skill-cluster. See [log.md](log.md) for the wave plan and results.
posture, whole-fleet audit, bulk redaction posture, whole-fleet audit, bulk redaction
- [[SSH and Remote Access]] — passwordless bootstrap, fish login shells, - [[SSH and Remote Access]] — passwordless bootstrap, fish login shells,
GUI-hang diagnosis over ssh GUI-hang diagnosis over ssh
- [[LLM Usage Attribution]] — who-is-using-the-fleet-LLMs research catalog:
engine-native identity, gateways, tracing, network/host attribution,
phased plan
**Architecture:** **Architecture:**
@@ -182,20 +185,12 @@ The LLM inference fleet on the two hosts. Full index:
### Deployment and Topology ### Deployment and Topology
- [[deployment]] — unified deploy and update process for both servers - [[deployment]] — unified deploy and update process for both servers
- [[frank-archon-radiance-qwen38-iq4r]] — **ACTIVE frank-archon runbook - [[frank-archon-radiance-swift15-w4dn8]] — STAGED frank-archon runbook:
(2026-10-15)**: the PUBLISHER's container of the official
Qwen3.8-Flash-Next (`qwen3.8-next-flash-fp8-iq4r-moe.rad`, 116,322 MiB,
w4nl 4-bit-codebook experts + int8 trunk, Hub-sha256-verified at fetch)
swapped in as a MODEL_FILE-only change — the 2026-10-13 shape (slab
25600 / fp8-KV 5120, 3 × 200,000, host pool 44,928, MTP depth 2) carries
verbatim; rollback = revert the swap commit (the w4dn8 `.rad` stays on
disk; the fp8-native `.rad` was deleted 2026-10-15, rebuildable)
- [[frank-archon-radiance-swift15-w4dn8]] — RETIRED 2026-10-08 → 10-15 frank-archon runbook:
Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance Swift1.5 Qwen3.8-Flash-Next BF16 → GPTQ-calibrated W4Dn8 `.rad` (radiance
1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s; 1.0.13), gated on KLD ≤ 0.03 vs the BF16 container and decode ≥ 30.8 t/s;
activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback activates by flipping `MODEL_FILE` in `scripts/frank-archon.env`, rollback
= revert that line (the FP8-native `.rad` stays on disk) = revert that line (the FP8-native `.rad` stays on disk)
- [[frank-archon-radiance-qwen38-native-fp8]] — the RETIRED 2026-10-05 → 10-08 frank-archon - [[frank-archon-radiance-qwen38-native-fp8]] — the ACTIVE frank-archon
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL 287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert `Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
-54
View File
@@ -1745,57 +1745,3 @@ every restart since the cutover.
[[frank-archon-radiance-swift15-w4dn8]], and the LLM index entry corrected [[frank-archon-radiance-swift15-w4dn8]], and the LLM index entry corrected
(the w4dn8 entry said STAGED, the fp8-native one said ACTIVE). (the w4dn8 entry said STAGED, the fp8-native one said ACTIVE).
## 2026-10-15 — frank-archon: publisher iq4r payload swap (MODEL_FILE-only)
- Ask: replace the served model with
`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` — the engine author's
published rad-convert of the OFFICIAL Qwen3.8-Flash-Next (w4nl codebook
experts, ten bf16-protected/layer, int8 trunk, PLE/MTP/vision in-container,
card KL 0.0789 / top-1 92.85 %) — **no other argument changes**, mindful
the operator session runs on the endpoint being swapped.
- Disk: Frank authorized one `rm` — the spent Swift1.5 BF16 source (336 GiB)
and the two-era-old `qwen4exp-fp8-native.rad` (173 GiB) left
`/home/dank/models`; 150 GB → 658 GB free. fp8-native is now rebuild-only
(fetch + build scripts); rollback for THIS swap is the w4dn8 `.rad`
(on disk) by reverting the commit.
- Fetch: new `scripts/download-frank-archon-iq4r.sh` (hfget, exact bytes
121,969,901,568 + Hub LFS sha256, `--verify-only` default) as transient
unit `iq4r-fetch`; verified, staged sidecar files kept as provenance.
- Shape carried VERBATIM (25600/5120 pair, 44928 pool, fp8 KV 3 × 200,000,
MTP 2, wht6, MNBT 2048, froggeric v22.5). Tests re-keyed
(`frank_archon_unit.bats` pins the new payload id + sha256, keeps the
w4dn8 rollback provenance), homepage card + trio lore + both indexes
updated; new runbook [[frank-archon-radiance-qwen38-iq4r]]. Commits
`6dcf4a5` (swap) + `f44c398` + `c5ac072` (docs), full bats green.
- Cutover self-severed by design (detached `deploy.sh` survived the session
death when frank-archon restarted); engine healthy in ~100 s. First-boot
ledger: w4nl loads on pin 1.0.13 (no pin bump); kv_attn 26.00 KiB fp8 @
607,740 tokens (identical geometry); slab 17,610/17,658 slots per rank
(> w4dn8's 17,548 — fewer bf16-protected experts pack more 4-bit table);
sampler 9 rows; MTP 2/step; prefix tier RECOVERED 900,176 blocks across
the swap (arch+tokenizer unchanged). Vision probe green (reasoning read
the solid-red PNG; the 64-token budget went all to thinking — fleet law).
- e2e gate went red on three dankclaw specs — NOT the swap: dankclaw rebooted
01:41 and the gluetun boot-race dependency-failed `searxng`, `marreta`,
and `marreta-browser` (jobs consumed while gluetun's start job retried;
`Restart=always` never armed because the services never ran). Cascade
restarted; skill codified (`gluetun-dependent-boot-race-recovery`).
Gate rerunning against the repaired fleet; frank-archon's own health gate
passed every deploy. Open: decode/prefill ladders vs the w4dn8 record,
probe leg D — scheduled probe window.
## 2026-10-15 — authentik: two more human accounts (Dankclaw)
- **Live — mitch (pk 22) and eric (pk 23) provisioned via `ak shell` ORM, no
repo objects.** Each `<user>@franksdank.cloud` in the human-user template
shape (`is_active=True`, display name, password via `set_password` on stdin
only — never argv, so nothing lands in `ps`), groups `Non-Admin Users` +
`jellyfin_users` + `openwebui_users`; no admin group granted. Verified
through the exact PasswordStage code path (`authenticate()` accepted each
password and rejected a wrong one), `app_entitlements(open-webui)` →
`Open WebUI Users` for both (identical to the ridley precedent), and
`is_superuser=False` / `path=users`. Zero `proxysession` rows for either —
groups were set before any login, so no claim purge was due. Downstream:
Jellyfin visibility rides the LDAP outpost cache (expect lag); OpenWebUI
mints their accounts as role `pending` on first login.
- `/serverAppConfigs-docs/Infrastructure/Authentik.md` — human-user template
census extended with pk 22–23.
+2 -2
View File
@@ -366,13 +366,13 @@
"model": { "model": {
"unit": "archon", "unit": "archon",
"role": "Frank's strongest and fastest model runtime", "role": "Frank's strongest and fastest model runtime",
"hf": "https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe", "hf": "https://huggingface.co/frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700",
"tagline": "...Power Overwhelming...", "tagline": "...Power Overwhelming...",
"sfx": [ "sfx": [
"archon-power.ogg" "archon-power.ogg"
], ],
"equiv": "Equivalent to Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).", "equiv": "Equivalent to Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).",
"spec": "The engine author's published quant: 4-bit codebook experts + int8 trunk of Qwen3.8-Flash-Next (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token context, three submissions at a time; sees images.", "spec": "Our own build: a GPTQ-w4 quant of Swift1.5-Qwen3.8-Flash-Next, quantized and tuned in-house (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token context, three submissions at a time; sees images.",
"inputs": [ "inputs": [
"text", "text",
"image" "image"
+19 -25
View File
@@ -3,10 +3,8 @@
# #
# This file asserts the RADIANCE shape (StillDeadcode engine, digest-pinned # This file asserts the RADIANCE shape (StillDeadcode engine, digest-pinned
# stilldeadcode/radiance image, BOTH R9700s TP=2 on the wht6 wire, serving # stilldeadcode/radiance image, BOTH R9700s TP=2 on the wht6 wire, serving
# the publisher's 4-bit-codebook .rad of the official Qwen3.8-Flash-Next # the Swift1.5 Qwen3.8-Flash-Next GPTQ-w4 .rad built by
# (fetched by scripts/download-frank-archon-iq4r.sh; the in-house # scripts/convert-swift15-w4dn8.sh). It replaces the FreeToken seam of
# Swift1.5 w4dn8 .rad of scripts/convert-swift15-w4dn8.sh is the on-disk
# rollback). It replaces the FreeToken seam of
# 2026-10-05 (git tag frank-archon-freetoken-pr587-20261005) and pins what # 2026-10-05 (git tag frank-archon-freetoken-pr587-20261005) and pins what
# flags alone do not say: the served window clears the harness worst case, # flags alone do not say: the served window clears the harness worst case,
# the KV pool backs seqs x max-model-len at the declared cache dtype (an # the KV pool backs seqs x max-model-len at the declared cache dtype (an
@@ -16,7 +14,7 @@
# half of RAM), the served window is what models.yml must # half of RAM), the served window is what models.yml must
# quote, the host-networking contract (no --publish), the image gate on a # quote, the host-networking contract (no --publish), the image gate on a
# name@sha256 ref, the .rad payload gate with its size floor, the # name@sha256 ref, the .rad payload gate with its size floor, the
# chat-template override mount, the self-healing law (the run returns before # chat-template override mount, the self-healing requirement (the run returns before
# the engine is up, so the serve script must wait on the container for # the engine is up, so the serve script must wait on the container for
# Restart=always to mean anything), the deployer's false-green MainPID check, # Restart=always to mean anything), the deployer's false-green MainPID check,
# the vendored hfget fetch path (no token for this ungated repo), and the # the vendored hfget fetch path (no token for this ungated repo), and the
@@ -30,7 +28,7 @@ SERVE="$REPO_ROOT/scripts/serve-frank-archon.sh"
TTM_LIB="$REPO_ROOT/scripts/ttm-preflight.sh" TTM_LIB="$REPO_ROOT/scripts/ttm-preflight.sh"
ENV_FILE="$REPO_ROOT/scripts/frank-archon.env" ENV_FILE="$REPO_ROOT/scripts/frank-archon.env"
REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh" REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-iq4r.sh" DOWNLOAD="$REPO_ROOT/scripts/download-frank-archon-swift15.sh"
BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh" BUILD="$REPO_ROOT/scripts/convert-swift15-w4dn8.sh"
HFGET="$REPO_ROOT/scripts/hfget.sh" HFGET="$REPO_ROOT/scripts/hfget.sh"
TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja" TEMPLATE="$REPO_ROOT/scripts/qwen38-froggeric-v22.5.jinja"
@@ -138,10 +136,11 @@ nosudo_bin() {
# Concurrency: 3 slots x the served window is the capacity contract, and # Concurrency: 3 slots x the served window is the capacity contract, and
# the fused TP all-reduce is declared over seqs x (1 + spec) = 9 rows, # the fused TP all-reduce is declared over seqs x (1 + spec) = 9 rows,
# under the 25-row bound at which the wht6 wire stays bit-exact. The # under the 25-row bound at which the wht6 wire stays bit-exact. The
# "seqs 2 cannot boot" law belongs to the retired tcclaviger vLLM fork # "seqs 2 cannot boot" is the failure mode of a runtime that compiles
# (torch.compile query_start_loc constraint, 2026-10-02); radiance booted # kernels specialized to the batch shape — it refuses a slot count it has
# seqs 2 in the fp8-native era and the 2026-10-06 sweep had seqs 2 and 3 # never compiled for. This engine is shape-agnostic: seqs 2 boots, and
# tied inside the +-3% noise floor — slots buy concurrency, not speed. # measured decode ties seqs 2 and 3 inside the +-3% noise floor — slots
# buy concurrency, not speed.
grep -q '^FRANK_ARCHON_MAX_SEQS=3$' "$ENV_FILE" grep -q '^FRANK_ARCHON_MAX_SEQS=3$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE" grep -q '^FRANK_ARCHON_TP=2$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE" grep -q '^FRANK_ARCHON_TP_WIRE=wht6$' "$ENV_FILE"
@@ -149,8 +148,8 @@ nosudo_bin() {
# expert_tiered placement: vram_weights + the pinned host pool serve the # expert_tiered placement: vram_weights + the pinned host pool serve the
# per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the # per-rank resident set TOGETHER, so cutting the VRAM slab to pay for the
# KV pool moves the pool with it. KV is QUANTIZED fp8 (the engine takes # KV pool moves the pool with it. KV is QUANTIZED fp8 (the engine takes
# only bf16|fp8 — no per-tensor K/V split), the PLE n-gram table is pinned # only bf16|fp8 — no per-tensor K/V split), and the PLE n-gram table is
# in host RAM per Frank's law. # pinned in host RAM so the cards read it without page-faulting mid-token.
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE" grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE" grep -q '^FRANK_ARCHON_HOST_POOL_MIB=44928$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE" grep -q '^FRANK_ARCHON_VRAM_WEIGHTS_MIB=[1-9]' "$ENV_FILE"
@@ -383,23 +382,18 @@ nosudo_bin() {
grep -q 'exec podman wait "$FRANK_ARCHON_CONTAINER"' "$SERVE" grep -q 'exec podman wait "$FRANK_ARCHON_CONTAINER"' "$SERVE"
} }
@test "the model id matches the payload's fetch script (and the rollback's build script)" { @test "the model id matches the payload's build and fetch scripts" {
# The deployed payload is the PUBLISHER's .rad: the fetch script names # The payload dir/file are the convert script's output layout; the fetch
# the repo, the file, and the Hub's LFS sha256; nothing is converted on # script must name the repo the convert stages from. (FP8-native era,
# the host for it. The in-house w4dn8 rollback keeps its own trio # the rollback payload, keeps its own trio: build-frank-archon-rad.sh +
# (convert-swift15-w4dn8.sh + download-frank-archon-swift15.sh, and the # download-frank-archon.sh.)
# fp8-native era keeps build-frank-archon-rad.sh +
# download-frank-archon.sh).
grep -q '^FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE" grep -q '^FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad$' "$ENV_FILE" grep -q '^FRANK_ARCHON_MODEL_FILE=qwen4exp-swift15-w4dn8-i8-hc8m.rad$' "$ENV_FILE"
grep -q 'StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe' "$DOWNLOAD" grep -q 'qwen4exp-swift15-w4dn8-i8-hc8m.rad' "$BUILD"
grep -q 'qwen3.8-next-flash-fp8-iq4r-moe.rad' "$DOWNLOAD" grep -q 'ukisai/Swift1.5-Qwen3.8-Flash-Next' "$DOWNLOAD"
grep -q '0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20' "$DOWNLOAD"
grep -q 'hfget.sh' "$DOWNLOAD" grep -q 'hfget.sh' "$DOWNLOAD"
# The repo is UNGATED — no token may be required on the fetch path. # The repo is UNGATED — no token may be required on the fetch path.
! grep -q 'HF_TOKEN' "$DOWNLOAD" ! grep -q 'HF_TOKEN' "$DOWNLOAD"
# Rollback provenance: the w4dn8 build script names its payload.
grep -q 'qwen4exp-swift15-w4dn8-i8-hc8m.rad' "$BUILD"
} }
@test "no bare HIP-reserved env var is defined" { @test "no bare HIP-reserved env var is defined" {
+11 -8
View File
@@ -11,8 +11,9 @@
# vision file names AND byte counts inside the /models mount, and the # vision file names AND byte counts inside the /models mount, and the
# tokenizer's flat-file shape (a HF cache snapshot's symlinks dangle inside # tokenizer's flat-file shape (a HF cache snapshot's symlinks dangle inside
# the container). A serve gate and a deployer gate that disagree is the # the container). A serve gate and a deployer gate that disagree is the
# 2026-10-01 crash-loop; both sides are pinned here, plus the one engine # crash-loop class this file exists to kill: both sides are pinned here,
# law the card is emphatic about: HALOGEN_SPEC_ADAPT=0 — the default # plus the one engine rule the card is emphatic about: HALOGEN_SPEC_ADAPT=0 —
# the default
# adaptive policy silently disables the MTP head per request and HALVES # adaptive policy silently disables the MTP head per request and HALVES
# decode. # decode.
# #
@@ -111,9 +112,10 @@ good_env() {
@test "the head stays on: adaptive spec must stay 0 and the window shape pinned" { @test "the head stays on: adaptive spec must stay 0 and the window shape pinned" {
# The card is emphatic: the default adaptive policy turns the MTP head # The card is emphatic: the default adaptive policy turns the MTP head
# off for whole requests after low acceptance and HALVES decode. The # off for whole requests after low acceptance and HALVES decode. The
# two-slot law does NOT carry over from the 27B engine — on this line # two-slot coupling of the 27B engine — which disabled drafting whenever a
# batching and speculation coexist (the card's quickstart ships 2 # second slot existed — does NOT carry over: on this line batching and
# slots WITH the head), so 2 is concurrent capacity, not a tax. # speculation coexist (the card's quickstart ships 2 slots WITH the head),
# so 2 is concurrent capacity, not a tax.
grep -qE '^HALOGEN_SPEC_ADAPT=0$' "$ENV_FILE" grep -qE '^HALOGEN_SPEC_ADAPT=0$' "$ENV_FILE"
grep -qE '^HALOGEN_KV_SLOTS=2$' "$ENV_FILE" grep -qE '^HALOGEN_KV_SLOTS=2$' "$ENV_FILE"
# The cap/timeout pair is coupled (FLAGS.md table): a request at the # The cap/timeout pair is coupled (FLAGS.md table): a request at the
@@ -178,8 +180,9 @@ good_env() {
} }
@test "the serve script refuses a checkpoint of the wrong byte count" { @test "the serve script refuses a checkpoint of the wrong byte count" {
# A truncated curl exit-0 artifact must not boot; the Hub's byte # A truncated curl exit-0 artifact must not boot: curl exits 0 on an HTTP
# count is the contract (fleet fetch law). # error and writes the error body as the file, so the Hub's declared byte
# count is the contract that catches it.
podman_stub podman_stub
good_env good_env
run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \ run env PATH="$TEST_TMP/bin:$PATH" "${GOOD_ENV[@]}" \
@@ -280,7 +283,7 @@ $(cat "$TEST_TMP/serve.log")"
! grep -q 'PODMAN wait' "$TEST_TMP/serve.log" ! grep -q 'PODMAN wait' "$TEST_TMP/serve.log"
} }
@test "the shipped env keeps the harness fit law inside the ceiling" { @test "the shipped env keeps the harness fit budget inside the engine ceiling" {
# 262144 (the checkpoint's native window) is the per-slot hard ceiling # 262144 (the checkpoint's native window) is the per-slot hard ceiling
# (a longer prompt is a 400 naming the limit, never a silent # (a longer prompt is a 400 naming the limit, never a silent
# truncation). The harness clamps max_tokens = min(45056, 217088 − est # truncation). The harness clamps max_tokens = min(45056, 217088 − est