fix(frank-archon): declare two slots, revert the inert host-pool raise

FRANK_ARCHON_MAX_SEQS 3 -> 2. At 1.8.1 the budget resolver sizes the KV cache
from the cache bound (--max-model-len x --max-num-seqs), --gpu-headroom-mib and
--host-pool-mib, and the shipped CLI has no KV-size flag (the
--vram-weights-mib/--vram-kv-mib pair was removed at 1.3.0). The live boot
ledger reads that cache at 4.20 GiB = 556,348 tokens = 2.8 sessions at 200,000,
so a third full-height conversation is refused while /health answers green and
the harness advertises capacity the engine cannot honour. Two slots is what the
shape actually serves at the full window; a slot costs KV, never decode, and the
fused TP all-reduce drops from 9 rows to 2 x (1 + spec) = 6, still inside
wht6's 25-row bit-exact bound.

FRANK_ARCHON_HOST_POOL_MIB back to 44928 and FRANK_ARCHON_TTM_MIN_MIB back to
150000, reverting PR #30. That raise bought cache the resolver never hands out:
the two boot ledgers differ in exactly one line -- host_pool 44.47 -> 45.09 GiB
-- while vram_weights (25.81 GiB) and vram_kv (4.20 GiB / 556,348 tokens, both
ranks) are byte-identical, so the extra 640 MiB sat unclaimed in the host pool
while the TTM floor moved out from under the preflight's arithmetic. The
compute-bound cap on --host-pool-mib is unchanged either way (45,568 -> 22,222
MiB of experts either side of the 22,222 clamp).

The bats seam pins the new contract: MAX_SEQS=2, pool 44,928, and the ledger
floor that computes to 148,920 MiB at that pool against the 150,000 minimum.

AGENTS.md

@@ -23,15 +23,17 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 
 ## Fleet topology (ACTIVE)
 
-| Target | Service | Runtime | Model | Port |
-|--------|---------|---------|-------|------|
-| Dankserver | `frank-archon` | **radiance** (StillDeadcode, **pin 1.0.13** digest-pinned `stilldeadcode/radiance@sha256:4c3f484564…`, plain `podman run`, root systemd unit) | **Publisher payload** (swap 2026-10-15, MODEL_FILE-only): `StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` → **`qwen3.8-next-flash-fp8-iq4r-moe.rad`** (121,969,901,568 B = **116,322 MiB**, Hub-LFS sha256 `0af5e962…` verified at fetch by `scripts/download-frank-archon-iq4r.sh`) — the OFFICIAL Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's author: routed experts as **4-bit codes into a 16-level non-uniform codebook** (128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent sweeps, E4M3 scales per 64 weights), **10 whole bf16-protected experts/layer**, int8 W8A8 trunk/lm_head, fp8 HC mixtures, fp8 fixed-scale PLE n-gram (47.7 GiB), **MTP head + bf16 vision tower in the container**; card KL 0.0789 mean all-tokens / top-1 92.85 % vs BF16. **Base-alignment (censored) model** — the abliterated family lives on frank-dragoon. Shape carried VERBATIM from the 2026-10-13 record: **3 × 200,000 fp8-KV** (`--max-num-seqs 3`), **25,600 MiB slab + 5,120 MiB pool = 30,720 of the 30,969 MiB claimable**, 44 GiB/rank pinned host pool, PLE pinned to RAM, MTP depth 2, MNBT 2048, Froggeric v22.5 template. Decode/prefill ladders in the runbook are the **w4dn8 record** (the bar to beat; new-payload probes pending). Rollback: re-download the w4dn8 `.rad` from the HF mirror `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` (LFS sha256 `09dd9306…`) + revert the swap commit — the local copy was **purged 2026-10-17** (it was the last non-iq4r model on the host; the GLM-4.7 GGUF, the radiance kvcache tiers, and every stale compile cache went in the same purge, 763 GiB free). The fp8-native `.rad` + the raw Swift1.5 BF16 source were DELETED from disk 2026-10-15 (Frank's one-time rm, volume for this payload; rebuildable via the fetch/build scripts). | 7770 |
-| Dankclaw | `frank-dragoon` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-dragoon.sh`, root systemd unit) | Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-abliterated.hgn`** (66,687,678,432 B ≈ 62.1 GiB resident, Hub LFS sha256 `b5bd925f…` verified at fetch — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned BF16, abliterated, NOT the writer transplant that carried this filename; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea) **+ vision tower** (897,916,416 B, @e053f488) — both **hardlinked** from the swift15 payload dir (the release publishes only the main file; the engine's own `niah` 18/18 + `ppl` 3.968 with mean KL 0.155 vs the swift15 logits certified the stock companions), **vision ON** (`image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset), **TWO slots over ONE `HALOGEN_KV_POOL_POSITIONS=401408` pool (= 2 × ctx) at `HALOGEN_CTX=200704`** (pool RAISED from 200,704 and DEPLOYED 2026-10-18 so BOTH slots can hold a maximum-length request at once; the parity play was frank-zealot's then-401,408 total in-flight context — zealot moved to 786,432 the same evening and dragoon's pool was NOT raised again, because every position is page cache the n-gram table loses; **11.0 GiB reserved, measured on that deploy** (the preflight formula bills 11.3) where 200,704 cost 5.5, 262,144 cost 7.2 and the 2026-10-06 pool measured 14.6; the ceiling does NOT move with it — 262,144 is the native wall and the engine refuses more without `HALOGEN_ROPE_YARN`), harness window 155,648 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (peonist engine recipe — Quat3rnion publishes none; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold), **n-gram table disk-backed** (mmap page cache, `Pin::None`); measured vs the swift15 boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking tokens ×1.9|7770|
-| Dankclaw | `frank-zealot` | **llama.cpp PrismML fork build** (vendor release `0.2.0-dev` build **10777**, commit `e3de71522`; root systemd unit, relinked — the ONLY loader for this payload, and since 2026-10-18 also the minimum build that pairs MTP with a quantized V cache) | `prism-ml/Ternary-Bonsai-2-9B-gguf` **PQ2_0 carrying the MTP head in the file** (2,675,410,720 B, Hub-LFS sha256 `c79fcdfa…` verified at fetch by `scripts/download-frank-zealot-bonsai9.sh`; `qwen35` hybrid: `block_count 33` = **32 trunk layers** (8 full-attention + 24 Gated DeltaNet at `full_attention_interval 4`) + the `blk.32` MTP block, 4 KV heads × 256 dim, native 262,144; `general.file_type 141` with **202 custom-type PQ2_0 tensors of 442** + `prism.hadamard.*` dequant metadata, which is what locks the runtime) + the publisher's own **Q8_0 mmproj** (624,229,664 B, sha256 `e0cb6f69…`; **Q8_0 is the publisher's requirement** for concurrent image requests — the BF16 tower can crash the server, and this endpoint runs three slots) for **vision**, on the RTX 3090, **THREE 262,144-token slots** (`--ctx-size 786432 --parallel 3` = every conversation at the trained window), **q8_0 K + q8_0 V** under flash attention (17 KiB/token over the 8 full layers = 13,056 MiB pool, ledger-exact; **f16 KV needs 26,112 MiB and cannot boot this card**; +151 MiB of recurrent state for the three DeltaNet cells — the SPEC-ARMED arms print 452 MiB, because MTP depth 2 bills three seqs per slot), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF — measured, not because the head is absent**: the draft context bills its own f16 KV over the WHOLE pool (3,072 MiB) and at ubatch 2048 that squeeze halves decode (167.2 → 109.2 t/s); the documented MTP arm is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` → 213.5 t/s. Production boot 17,922 used / 6,203 free; **168.7 t/s** single-stream greedy decode (three runs, 168.4–168.7) and **~300 t/s** aggregate across three concurrent streams (102.8 each); 2,877 t/s prefill at 106,418 was the trial arm. Cutover 2026-10-18 from Ternary-Bonsai-2-**27B** PQ2_0 at 2 × 200,704 (payload + build 10754 tree on disk as rollback). | 7771 |
+| Target | Service | Runtime | Payload | Port |
+|--------|---------|---------|---------|------|
+| Dankserver | `frank-archon` | radiance, digest-pinned, root unit, TP=2 | publisher `qwen3.8-next-flash-fp8-iq4r-moe.rad` | 7770 |
+| Dankclaw | `frank-dragoon` | halogen-flash-server 0.16.2, digest-pinned, root unit | `qwen38-flash-next-v2-abliterated.hgn` + n-gram + vision tower | 7770 |
+| Dankclaw | `frank-zealot` | llama.cpp PrismML fork build 10777, root unit — the only loader for this payload | Ternary-Bonsai-2-9B PQ2_0 + Q8_0 mmproj | 7771 |
 
-Retired endpoints (`frank-npu`, and every prior runtime) are in
-the [runbook index](#runbook-index). The full swap history lives in
-`serverAppConfigs-docs/LLM/index.md` — do not grow it here.
+Payload identity, shape, eras, laws and rollback live in ONE page each:
+[[frank-archon]], [[frank-dragoon]], [[frank-zealot]]. `scripts/<service>.env`
+is the authority for every number; a page that disagrees with it is the bug.
+A retired payload is a row in that page's **Eras** table — there is no page
+per attempt and no separate swap-history file.
 
 ## Repository Layout
 
@@ -246,71 +248,106 @@ Never `rm`. Move deletions into trash so they stay recoverable:
 `git * --no-verify` is forbidden. The trash is a temporary hold, not an
 archive.
 
-## Benchmarking Policy (MANDATORY)
+## Model Swaps (fleet law, 2026-10-18)
 
-**STOP the production service BEFORE benchmarking** — these servers serve
-real users, and a bench beside a live service starves the GPU and corrupts
-results. Use the runtime's own harness: `llama-bench` for the llama.cpp
-endpoints; a live llama.cpp server (frank-zealot) also reports honest
-per-request numbers in its `/completion` `timings` object (prefill/decode
-rates, `draft_n`, `draft_n_accepted`), which is how frank-zealot was measured.
-halogen has NO `/completion` path at all — frank-dragoon's rates come from
-wall-clock around `/v1/chat/completions` plus the boot ledger. Rolling your own
-bench requires asking first.
-Scratch ports are not used on LLM hosts — a
-test process cannot share the single GPU anyway; validate against the service
-itself.
+**Swap the model. Do not instrument it.** A payload change is: edit the trio
+(env / serve script / unit) → `scripts/deploy.sh <host> --service <svc>` →
+`/health` answers → one chat turn, one tool call, and one vision probe on an
+endpoint that serves images. That is the entire procedure.
 
-(A gufo-era law; frank-dragoon ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
-halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the
-rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode
-settings, and will invert the ranking.**
-Measured 2026-09-28 on frank-dragoon at depth 32768: its corpus
-reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
-draft widths, i.e. the same highly-predictable text every run, and 93 % of
-every proposal accepted. It reported 29–37 t/s decode where real traffic
-gives ~15 t/s, and it made `--draft-tokens 16` look 6 % faster than 7 when
-real text made it 14 % *slower*. Use it for shape, not for A/B. For an A/B,
-replay **real** text through the live server and read the `Server-Timing`
-header — but note the unit rides in the **name** half (`ttft;dur=402.485`),
-so splitting on `=` alone keys the dict `ttft;dur` and every lookup silently
-returns nothing. **That header is a llama.cpp feature**: halogen flash-server
-0.16.2 answers `/v1/chat/completions` with only `date`, `server` and
-`content-*` (verified 2026-10-18), and `/completion`, `/ping`, `/props` and
-`/tokenize` are all 404 on it — on frank-dragoon the A/B instrument is
-wall-clock around the POST plus `usage.prompt_tokens`. Generate **at least 128
-tokens** for the measurement: at 8 tokens, `inter_token` is first-token noise
-and the run-to-run spread swamps every real difference.
+- **No benchmarking unless Frank orders it.** No pre-swap baseline battery,
+  no A/B arms, no decode/prefill ladders, no acceptance tables, no numbers
+  written into any page afterwards. Data exists only when he asks for it,
+  and then only in the file he names.
+- **It doesn't work → roll back.** Revert the commit, redeploy, done. No
+  post-mortem page, no investigation log, no trial-drop-in forensic tour. If
+  the fault is not obvious, one line in the endpoint page's *Laws* section is
+  the whole record.
+- What still gates a change: `./tests/run.sh` before commit, the deployer's
+  per-service health gate, and the Playwright e2e gate. Those are the fleet's
+  evidence. A bench is not evidence of correctness and buys nothing a swap
+  needs.
 
-This covers **any GPU experiment, not just benchmarks** — including a bare
-2-rank RCCL/`torch.distributed` probe. On 2026-09-16 one such probe was run
-against the live service "because it only needs 200 MB of VRAM"; the host went
-hard down ~20 minutes later (no ARP reply, 100% ICMP loss from two subnets, no
-recovery — a power cycle was required). A hung AQL dispatch wedging the GPU
-driver is the plausible mechanism, and this host has precedent for going
-unreachable until a power cycle. If an experiment needs GPUs, stop and mask
-first, or use a host that serves nobody. Telling an off machine from a wedged
-one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
-merely wedged kernel usually still answers ARP.
+### When a bench IS ordered
+
+**STOP the production service first** — these servers serve real users, and a
+bench beside a live service starves the GPU and corrupts the result. That
+covers **any GPU experiment, not just benchmarks**, including a bare
+2-rank RCCL/`torch.distributed` probe: on 2026-09-16 one was run against a
+live service "because it only needs 200 MB of VRAM" and the host went hard
+down ~20 minutes later (no ARP reply, 100 % ICMP loss, power cycle required;
+a hung AQL dispatch wedging the driver is the plausible mechanism). Telling
+an off machine from a wedged one: `ip neigh show <ip>` from a peer — FAILED
+means off or NIC dead; a merely wedged kernel usually still answers ARP.
 
 1. Stop and mask (self-healing would restart it):
-   - Dankclaw `frank-dragoon` (root unit, halogen podman on the iGPU):
-     `sudo systemctl stop frank-dragoon && sudo systemctl mask frank-dragoon`
-   - Dankclaw `frank-zealot` (root unit, llama.cpp on the 3090):
-     `sudo systemctl stop frank-zealot && sudo systemctl mask frank-zealot`
-  - Dankserver `frank-archon` (root unit, radiance on both R9700s, TP=2):
-     `sudo systemctl stop frank-archon && sudo systemctl mask frank-archon`
-2. Bench.
-   NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
-   `/etc/systemd/system/` is itself a symlink into the checkout — mask
-   refuses to replace it). Stop is enough for the bench window; the deploy
-   restores the unit, so re-deploy after benching instead of unmasking.
-3. Unmask and start; verify `curl http://192.168.0.<host>:7770/health`.
+   `sudo systemctl stop frank-dragoon frank-zealot` on Dankclaw,
+   `sudo systemctl stop frank-archon` on Dankserver, then `mask` each.
+   `mask` FAILS on a relinked unit (the file in `/etc/systemd/system/` is
+   itself a symlink into the checkout); stop alone is enough for the window,
+   and the deploy restores the unit — re-deploy after benching rather than
+   unmasking.
+2. Bench against the service itself. No scratch ports on an LLM host — one
+   process owns the card. Use the runtime's own harness (`llama-bench`; a
+   live llama.cpp server also reports honest `timings` per request, including
+   `draft_n`/`draft_n_accepted` — but on frank-zealot's PrismML fork build
+   10777 that object only reaches you on **`/v1/chat/completions`**: a greedy
+   `ignore_eos` run on the raw `/completion` path there answers **500 "The
+   model produced output that does not match the expected Content-only
+   format"** (measured 2026-10-18). halogen has no `/completion` path at all,
+   so its rates are wall-clock around `/v1/chat/completions` plus the boot
+   ledger. Generate ≥128 tokens: at 8, `inter_token` is first-token noise and
+   run-to-run spread swamps every real difference.
+3. Unmask, start, verify `curl http://192.168.0.<host>:7770/health`.
+   Retire every bench drop-in the same session
+   (`systemctl show <svc> -p DropInPaths` must come back empty) and write the
+   numbers into the file Frank named — `/tmp` and the journal are not records.
+
+Instrument traps that survive from the era when this was routine:
+`gufo bench` cannot rank speculative decode and **inverts the ranking** (its
+corpus reports `acceptance=0.933 ± 0.000` with an identical trace sha256
+across draft widths — the same predictable text every run); llama.cpp's
+`Server-Timing` carries the unit in the **name** half (`ttft;dur=402.485`),
+so splitting on `=` keys the dict `ttft;dur` and every lookup silently
+returns nothing; halogen 0.16.2 answers `/completion`, `/ping`, `/props` and
+`/tokenize` with 404 and sends no `Server-Timing`; and the first request
+after a boot is a warmup artifact (measured 2.6 t/s prefill then 1,258 t/s on
+the next), so never report request #1.
 
 **CRITICAL:** never `systemctl disable` a system unit on these hosts — it
 deletes the unit file from `/etc/systemd/system/`, not just the symlink.
 `mask` symlinks to `/dev/null` and is safe/reversible.
 
+## Documentation Policy (fleet law, 2026-10-18)
+
+**One fact, one owner.** A page owns exactly one thing — an endpoint, a
+service, or a technique — and every other file links to it with `[[Page]]`.
+Restating a shape, a digest, a port list, or a measured number in a second
+file is a defect: it is the next thing to go stale.
+
+- **One page per endpoint, forever:** `serverAppConfigs-docs/LLM/<endpoint>.md`
+  — current shape, an **Eras** table (`window | payload | runtime | shape |
+  replaced because | rollback`), the **Laws** the endpoint taught, the wire
+  contract, and the files that own it. A payload swap edits the shape block
+  and appends one Era row. It never creates a page, and the retired era's
+  page goes to `.AGENTTRASH/` once its rollback path and its laws are in the
+  table.
+- **`log.md` records incidents and laws** — anything with a causal story
+  (why the host went down, why a knob resets at reboot, why the RAM was the
+  fault). A model swap is not an incident: it is an Era row and nothing else.
+- **`AGENTS.md` carries operator-critical numbers only** — port, context
+  ceiling, slot shape, unit/env/serve trio — plus the cross-cutting laws.
+- **Index pages are locators:** one row, one clause, no numbers.
+- **A measured number is written once:** in the env-file header beside the
+  serve script that consumes it, and in the endpoint page. Every other page
+  cites it by link.
+- Technique pages (spec decode, KV math, offload budgets, PP bandwidth) stay
+  technique pages: they stop carrying a per-endpoint throughput table or a live
+  shape, but a campaign ledger that exists nowhere else is owned by the page that
+  measured it. The defect is two pages carrying one figure, not a number living on a
+  technique page — move a figure only when an endpoint page or an env header is its
+  home; otherwise leave it in place and name the era it measured.
+
 ## Model Config (harness)
 
 The harness reaches the fleet through `.omp/agent/models.yml` in the
@@ -350,11 +387,12 @@ and `frank-zealot` through
 the publisher's own **Q8_0** mmproj passed as `--mmproj` — Q8_0 is the
 publisher's requirement for concurrent image requests (the card states the
 BF16 tower can crash `llama-server` when several image requests run at once,
-and this endpoint runs three slots). `frank-zealot`'s `contextWindow` is
-**262144** — the PER-SLOT window (`--ctx-size 786432 --parallel 3` since the
-2026-10-18 Bonsai-2-**9B** swap), which since that swap EQUALS the model's
-native window; llama.cpp answers a longer prompt with a 400 naming the limit,
-never a truncation, so the harness must compact before it.
+and this endpoint runs several slots). `frank-zealot`'s `contextWindow` is
+**200704** — the PER-SLOT window (`--ctx-size 602112 --parallel 3` since the
+2026-10-19 reslice that funded a third slot AND an armed MTP drafter),
+deliberately below the model's native window; llama.cpp answers a longer prompt
+with a 400 naming the limit, never a truncation, so the harness must compact
+before it.
 Note also
 that
 the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
@@ -366,406 +404,72 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
 audio encoder, and left again with the Gemma4 retirement on 2026-10-01
 (neither the MiMo nor the Qwen3.5 tower carries audio).
 
-### frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 NATIVE abliterated (Dankclaw, ACTIVE)
+Each endpoint has ONE page in the vault: shape, Eras, Laws, wire contract,
+rollback. The numbers below are only what a consumer or a deploy gate needs;
+the env file beside the serve script is the authority, and a page that
+disagrees with it is the bug.
 
-The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
-digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
-driven by `scripts/serve-frank-dragoon.sh` under a **root systemd unit**
-(`scripts/frank-dragoon.service`, `User=dank`, `Restart=always`,
-`ExecStop=podman stop -t 60 frank-dragoon-halogen`, the script runs `podman
-run` FOREGROUND — the container's exit code is the service's exit so
-`Restart=always` stays real, and the engine ledger lands on disk) on port
-7770 from the Strix Halo iGPU. Engine stdout goes to
-`/home/dank/logs/frank_dragoon.log` — **the boot ledger (weights resident, KV
-pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
-Cutover 2026-10-17 from the Swift1.5-finetune shape
-([[frank-dragoon-flash-server-v2-swift15]], retired the same window; it
-itself cut over the halogen 27B-dense era on 2026-10-06).
+### frank-dragoon — halogen flash-server, Dankclaw iGPU, :7770
 
-Serving `Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`:
-`qwen38-flash-next-v2-abliterated.hgn` (66,687,678,432 B ≈ 62.1 GiB, sha256
-`b5bd925f…` — the **NATIVE 2026-10-11 v2 conversion** of orcarouter's pinned
-BF16 checkpoint, abliterated; NOT the writer transplant that carried this
-filename, which measured 66,687,678,464 B — 32 bytes apart, same filename
-family, different model) + the **REQUIRED** n-gram lookup table
-`qwen38-flash-next-ngram.hgn` (51,200,246,144 B, peonist @5cc17cea) + the
-vision tower `qwen38-flash-next-vision.hgn` (897,916,416 B, @e053f488) + the
-flat `tokenizer/` directory (6 files) — the three companions **hardlinked**
-out of the swift15 payload dir, because the release publishes only the main
-file and no source-matched pair exists. Cutover 2026-10-17 from the Swift1.5
-finetune, chosen by Frank for the base checkpoint itself plus the card's
-memory-saving pool.
+- Owns: `scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
+  `scripts/frank-dragoon.service`, container `frank-dragoon-halogen`, image
+  digest-pinned `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`.
+  Engine stdout (the boot ledger: weights, KV pool, clamp) goes to
+  `/home/dank/logs/frank_dragoon.log`, NOT the journal.
+- Shape: `HALOGEN_CTX=200704` is the **per-request ceiling** (a longer prompt
+  is a 400, never a truncation); `HALOGEN_KV_SLOTS=2` over ONE
+  `HALOGEN_KV_POOL_POSITIONS=401408` pool — pool positions buy concurrency,
+  the per-request window never moves; `HALOGEN_SPEC_ADAPT=0` keeps the MTP
+  drafter always armed (the adaptive default disarms it and halves decode);
+  server sampler 1.0/0.95/20.
+- Harness contract: `models.yml` `contextWindow: 155648` = ceiling − 45,056.
+  The clamp is `min(45056, window − est − 4096)` and the chars/4 estimate
+  under-counts, so a harness window equal to the ceiling makes every long
+  request a guaranteed 400. Reasoning arrives in `reasoning_content`; the
+  template rejects `reasoning_effort` outside `xhigh|medium|low`; images are
+  OpenAI `image_url` parts (probe ≥256 px); `response_format` unsupported;
+  `/completion`, `/ping`, `/props`, `/tokenize` are 404.
+- Page: [[frank-dragoon]].
 
-**The format is the lock.** The v2 `.hgn` loads ONLY in the
-flash-server line — the 27B-era halogen 0.1.4 container refuses it, so the
-engine moved with the model. The MTP head is INSIDE the checkpoint
-(`HALOGEN_MTP_HEAD` stays unset; the separate `qwen38-flash-next-mtp.hgn`
-sidecar is deliberately not fetched).
+### frank-zealot — llama.cpp (PrismML fork), RTX 3090, :7771
 
-**Product facts vs the 27B era, both regressions FIXED:** the base is
-**abliterated** (the 27B was a censored-base finetune; refusals were an
-accepted trade there) and **vision is ON** (the 27B engine silently
-ignored its encoder — frank-zealot owned images for exactly one day at this
-endpoint).
+- Owns: `scripts/frank-zealot.env`, `scripts/serve-frank-zealot.sh`,
+  `scripts/frank-zealot.service`; fork build `llama-prism-b10777` is the ONLY
+  loader (the payload carries the fork's custom PQ2_0 tensor type plus
+  `prism.hadamard.*` dequant metadata — stock llama.cpp refuses it, and a
+  loader ignoring that metadata dequantizes wrong); payload
+  `Bonsai2-9B-MTP/…PQ2_0.gguf` + the publisher's **Q8_0** mmproj (the BF16
+  tower can crash the server with concurrent image requests).
+- Shape: `--ctx-size 602112 --parallel 3` = **three conversations at
+  200,704 each**, `CACHE_TYPE_K=f16` + `CACHE_TYPE_V=q8_0` (asymmetric: 25,088
+  B/token, 14,406 MiB of pool), batch 6144 / ubatch 1024,
+  `SPEC_TYPE=draft-mtp` (depth 2), `--flash-attn on`, `--jinja`, `--metrics`.
+- Harness/gate contract: `contextWindow: 200704` is the **per-slot** window
+  (`n_ctx_slot = CTX_SIZE / PARALLEL`), and llama.cpp answers a longer prompt
+  with a 400 rather than truncating. `--metrics` is mandatory or Prometheus
+  reads 501 = DOWN beside a healthy service. Presence penalty has no GGUF
+  metadata key, so it must reach argv. This endpoint is the fleet's
+  compaction hub.
+- Page: [[frank-zealot]].
 
-**Engine laws that shape the config** (all in `scripts/frank-dragoon.env`
-with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
-policy disables the MTP head for whole requests after low acceptance and
-HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
-for it (accepted). `HALOGEN_CTX=200704` is the PER-REQUEST hard ceiling (400
-naming the limit, never truncation) — the checkpoint's native window is
-262,144 and was served until 2026-10-17, when Frank lowered it for RAM. The
-**400-zone law is invariant to the value**: the harness clamps max_tokens to
-`min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts
-the rendered prompt by ~3 % (measured: est 152,938 → server 157,388,
-request 43,158 vs room 42,804), so a harness window == engine ceiling
-makes every long request a guaranteed 400. `models.yml`
-`contextWindow: 155648` (= ceiling − 45056) keeps that gap (worst-case
-request ≈ 156.2k ≤ 200,704); compaction fires at 85 % of the harness window
-= 132,301 est ≈ 136k measured (the 180k target died with the 262,144 window).
-`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
-law does NOT carry over; batching and speculation coexist (the card's
-quickstart ships 2 slots with the head), so 2 is concurrent capacity.
-`HALOGEN_KV_POOL_POSITIONS=401408` — **2 × ctx, RAISED from 200,704 on 2026-10-18**
-(Frank: dragoon's total in-flight context was HALF frank-zealot's 401,408
-pool — doubling the POOL is what fixes that, not doubling the window) — is
-**ONE pool behind both slots**, ledger form `2 slots over ONE <pool>-position KV
-pool, each request up to HALOGEN_CTX`, and at pool == slots × ctx two
-MAXIMUM-length conversations DO coexist. The PER-REQUEST ceiling stays
-`HALOGEN_CTX=200704`: the checkpoint's native window is 262,144 and the engine's
-own 400 text refuses to exceed it without `HALOGEN_ROPE_YARN`, so "more context"
-bought here is CONCURRENCY, not a longer single conversation. ~28 KiB/position ⇒
-the 2026-10-18 deploy RESERVES **11.0 GiB** (`reserving the KV pool, 401408
-positions (about 11.0 GiB)`; the preflight formula bills 11.3 with block scratch
-+ ~115 MiB/slot of O(1) state), where 200,704 cost 5.5, 262,144 cost 7.2 and
-the 2026-10-06 shape's 524,288 measured 14.6 — quote the reservation.
-`HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
-harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
-`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
-**A server-side sampler EXISTS** (`HALOGEN_TEMPERATURE/TOP_P/TOP_K` =
-1.0/0.95/20, the card's thinking recipe) — an absent temperature is no
-longer greedy; `models.yml` extraBody carries the same values, consistent
-not load-bearing. Prompt cache is ON by default (`PROMPT_CACHE=2`): a warm
-100k follow-up costs ~2 s vs ~88 s cold — the 27B engine's manual
-`HALOGEN_CACHE_MB` pin era is over. `response_format` is unsupported.
-Thinking lands in **`reasoning_content`**; the v2 chat template rejects
-`reasoning_effort` outside `xhigh|medium|low`, so the `models.yml` fold
-stays mandatory. Thinking tokens COUNT against the budget (empty content +
-`finish_reason: "length"` when exhausted mid-thought). Vision: OpenAI
-`image_url` content parts only; ≥256 px probe images or the model answers
-"video".
+### frank-archon — radiance, two R9700s TP=2, Dankserver :7770
 
-**Memory ledger** (121.2 GiB unified; the referee is the boot ledger): weights
-62.9 (trunk 62.11 + vision 0.84) + KV **11.0** + working 5.7
-(`3.8 + 8.4 × MAX_TOK/16384`) = **79.6 GiB in all** (boot ledger, the
-401,408-position deploy of 2026-10-18), the **1.3 GiB** prompt cache (12 entries
-× 111 MiB at ctx 200704) booked separately by the ledger, + `HALOGEN_HOST_
-RESERVE_GIB=20` → ~20 GiB margin over baseline co-tenancy (`MemAvailable` read
-22 GiB right after the raise; the pool-equals-ctx boot printed 74.1 GiB with
-~30 GiB of margin). The 51.2 GB n-gram table rides as mmap'd page cache beside
-the checkpoint (ledger: `mapped 51.2 GB, NOT pinned (Pin::None)`) — which is why
-every pool position is a double cost: **RAM the pool takes is RAM that table
-loses**, so the warm prompt-cache ratio is re-measured after a raise.
-`HALOGEN_FLASH_PIN_TRUNK=0` is REJECTED by the v2 loader, so disk-backing the
-trunk is not a lever on this format. READ the boot ledger and re-size if the
-engine prints a clamp.
-
-Fetch: `scripts/download-frank-dragoon-v2abl.sh` into
-`/home/dank/models/halogen-v2-abliterated/` (`-f` + exact byte counts + LFS
-sha256 + `--verify-only`, no token on the command line; resume goes against
-the RESOLVED CDN URL because curl drops `Range` across HF's 302) — the main
-file is fetched, the three companions are HARDLINKED from the verified
-swift15 dir (a hardlink is a regular file, so the no-symlink serve gate
-passes without a 47.7 GiB copy). The serve gates mirror the deployer EXACTLY
-(byte-exact checkpoint + n-gram + vision tower, flat tokenizer with all six
-files as regular files) — the 2026-10-01 crash-loop lesson.
-
-Runbook:
-`serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md`.
-Rollback: revert the swap commit and redeploy — the Swift1.5 payload
-(`/home/dank/models/halogen-flashnext-v2/`, re-verified by
-`scripts/download-frank-dragoon-flashnext.sh`), the 27B payload
-(`/home/dank/models/halogen-qwen3.8-27b/`) and the halogen 0.1.4 image
-digest all remain on disk; the ROCmFPX door behind that
-(`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
-
-### frank-zealot — llama.cpp (PrismML fork) on the RTX 3090 (Dankclaw, ACTIVE)
-
-**The PrismML fork build of llama.cpp** (vendor release, `--version` =
-`0.2.0-dev (build 10777, commit e3de71522)`;
-`/home/dank/GitPackages/llama-prism-b10777/llama-server`) as a **root systemd
-unit** (`scripts/frank-zealot.service`, `relink_unit`ed into
-`/etc/systemd/system/`), serving `prism-ml/Ternary-Bonsai-2-9B-gguf` at
-**PQ2_0 with the MTP head IN the file** (2,675,410,720 B, Hub-LFS sha256
-`c79fcdfa…` verified at fetch) **plus the publisher's own Q8_0 mmproj**
-(624,229,664 B, sha256 `e0cb6f69…`), both `/home/dank/models/Bonsai2-9B-MTP/`,
-on port 7771. Cutover 2026-10-18 from the Bonsai-2-**27B** PQ2_0 era
-(2 × 200,704, same card, same fork family).
-
-**The runtime is part of the payload identity — twice over.** The file is
-`general.file_type 141` — **202 of the 442 tensors carry the fork's PQ2_0 type**
-(the rest are f32 norms/scalars plus 8 q8_0 in the MTP block) — beside
-`prism.hadamard.*` metadata (normalized Sylvester–Walsh transform): stock
-llama.cpp refuses the custom type at load, and a loader that
-ignored the Hadamard metadata would dequant WRONG — same lock class as the
-ROCmFPX era. On top of that lock, **the fork gates MTP beside a quantized V
-cache on build `prism-b10776-d00e252` or newer**, which is why the prebuilt
-moved 10754 → 10777; the 10754 tree stays on disk as the 27B rollback runtime.
-Retargeting the payload without the fork ships an endpoint that cannot load,
-so the seam pins both paths.
-
-**Architecture: qwen35 hybrid** — read from the GGUF's own header: `block_count`
-**33 = 32 trunk layers + the `blk.32` MTP block** (the MTP block has its own
-attention + FFN in q8_0, which is why the draft context bills real KV), and
-`qwen35.full_attention_interval = 4` ⇒ **8 full-attention + 24 Gated
-DeltaNet**, 4 KV heads × 256 dim, native 262,144. **KV accounting:** the pool
-scales with the 8 full layers — 8 × 4 × 256 = 8,192 elements/token per cache,
-so **q8_0 K + q8_0 V bills 17,408 B/token ≈ 17 KiB** = **13,056.00 MiB** at the
-786,432 pool (ledger-exact) — exactly HALF the 27B's 34,816 B/token, which is
-the whole reason three native windows fit where the 27B afforded only two
-shortened ones. **f16 K would bill 26,112 MiB and cannot boot on a 24,123 MiB
-card.** The 24 DeltaNet layers bill a fixed recurrent state per SLOT — the
-production ledger prints `RS buffer size = 150.75 MiB (3 cells, 32 layers,
-3 seqs)`, while the SPEC-ARMED arms print 452.25 MiB (MTP depth 2 bills three
-seqs per slot, so 9 cells) — so pool arithmetic alone under-prices the shape.
-
-**The shape (2026-10-18):** `--ctx-size 786432 --parallel 3` = THREE
-conversations of 262,144, every one at the model's TRAINED window (Frank: "if
-you can possibly fit a third context, do it"), `--flash-attn on`, **q8_0 K +
-q8_0 V**, batch 4096 / **ubatch 2048** (the measured CUDA optimum carried from
-the MiMo/Swift A/Bs on this card; it also sets the compute-buffer size, which
-is what decides whether MTP fits), `--log-verbosity 4`, `--metrics`,
-`--jinja`, `--mmproj` for vision, **speculative decoding OFF**
-(`SPEC_TYPE=none`). Boot: **17,876 MiB used / 6,249 free**; compute 1,409 MiB
-CUDA + 1,201 MiB CUDA_Host, projector 595 + 248 MiB. Measured on the same
-boot: **167.2 t/s** greedy decode, 158.9 t/s single-stream chat, **280.2 t/s
-aggregate across three concurrent streams**, **2,877 t/s prefill at 106,418**
-tokens — the 27B record (76.6 decode / 109.4 aggregate / 691.7 prefill at 166k)
-beaten on every axis. The **first request after boot is a warmup artifact**
-(37.7 t/s class) — never report it.
-
-**Why MTP is off even though the head is in the file.** The drafter is GOOD —
-acceptance 0.798 / 0.617 per position, mean accepted length 2.41 — and the card
-recommends depth 2. It still loses AT THIS SHAPE because the draft context
-bills its OWN **f16** KV over the WHOLE pool (4,096 B/position × 786,432 =
-**3,072.00 MiB**, 4,830 MiB with its compute buffers) and ubatch 2048 refuses to
-give it up. Booted arms, same card, same traffic (2026-10-18):
-
-| slots | spec | ubatch | used / free | decode |
-|---|---|---|---|---|
-| 3 × 262,144 | off | 2048 | 17,876 / 6,249 | **167.2** |
-| 3 × 262,144 | on | 2048 | 22,706 / 1,419 | 109.2 |
-| 3 × 262,144 | on | 1024 | 21,400 / 2,725 | **213.5** |
-| 2 × 262,144 | on | 2048 | 17,178 / 6,947 | 213.2 |
-| 1 × 262,144 | on | 2048 | 11,652 / 12,473 | 214.2 |
-
-**MTP is worth +28 % decode whenever the card has room**; the squeeze, not the
-drafter, is what halves it at three windows. The arm that WOULD run it at three
-windows is `SPEC_TYPE=draft-mtp` + `UBATCH_SIZE=1024` (213.5 t/s single-stream,
-250.8 three-stream aggregate) and pays prefill −17 % and 2.7 GiB of headroom on
-a card whose projector worst-case lands AFTER the engine's own fit check. This
-endpoint is the fleet's compaction hub — prefill-bound, 2–3 concurrent
-subagents — so the shipped arm takes the +17 % prefill (2,877 vs 2,395 t/s) and
-the +12 % three-stream aggregate (280.2 vs 250.8) over single-stream speed, and
-keeps 6.2 GiB free. `SPEC_TYPE=none` strips the WHOLE spec set (`--spec-type`
-and every `--spec-draft-*`) — flags move as a set or the arm measures a crash
-instead of an absence.
-
-**Sampler**: the recipe lives in the checkpoint (`general.sampling.*` =
-temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 / repeat_penalty 1.0,
-confirmed in the engine's own `/props`) and llama.cpp reads it itself.
-**Presence penalty is the one recipe value with no GGUF metadata key**, so
-`PRESENCE_PENALTY=1.0` must reach the argv; `""` is the documented opt-out, and
-the serve script therefore uses `${PRESENCE_PENALTY=1.0}` (NOT `:=`, which
-would silently re-add the flag on an explicit empty value).
-
-**Vision**: the publisher's own **Q8_0** mmproj via `--mmproj` — Q8_0 is
-MANDATORY, not a preference: the card says the BF16 tower "can crash
-llama-server when several image requests run at once", and `--parallel 3` is
-exactly that shape. Probe traps carry over: ≥256 px probe images or the model
-answers "video", only the OpenAI `image_url` content part reaches the model,
-and thinking tokens count against `max_tokens` — budget ≥512 or the probe
-returns EMPTY `content` with `finish_reason: "length"` and the endpoint looks
-blind while it is only mid-thought.
-
-**Base alignment**: Bonsai-2 is the censored BASE lineage. Unlike the 27B, the
-CRACK re-export on disk (`/home/dank/models/Bonsai2-CRACK/`) is byte-identical
-geometry to the 27B it matched, NOT to this 9B file — if a refusal ever
-surfaces there is no one-line `MODEL_FILE` swap; a 9B dealigned export would
-have to be fetched first.
-
-**Compaction hub**: `models.yml` routes `compactionModel` for
-`frank-archon`, `frank-dragoon` and `frank-zealot` (the 3090 endpoint) here —
-262,144 (since the 2026-10-18 reslice) is the widest window in the fleet and
-the only one that comfortably fits a worst-case compaction input (~170k
-tokens). With `--parallel 3` a compaction request no longer serializes behind a
-live generation (the Gemma4 era's `--parallel 1` law). Retrieval through a
-25,960-token prompt was verified exact after the swap.
-
-**Contract law learned at verbosity 4:** a raw `/completion` request prints
-`failed to parse reasoning content (… parse_reasoning=true)` whenever the
-model closes a bare prompt with `` — cosmetic
-(`probe_request.cpp:789`); the `/v1/chat/completions` path parses
-`reasoning_content` correctly, which is the only path consumers use. Do not
-lower `VERBOSITY` to silence it: the KV/buffer ledger is what VRAM forensics
-read. `--metrics` is required or the Prometheus scrape reads 501 = DOWN.
-
-Runbook:
-`serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md`.
-Rollback: revert the swap commit — the 27B payload
-(`/home/dank/models/Bonsai2-PQ2_0/`) with the 10754 fork tree
-(`~/GitPackages/llama-prism/`), the Qwen3.5-9B Q6_K payload
-(`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`) with the master CUDA build
-(`~/GitPackages/llama-master/build-cuda/`), the Gemma4 payload
-(`/home/dank/models/Gemma4-26B-A4B/`), the MiMo Q6_K payload
-(`/home/dank/models/MiMo-Q6K/`), the Swift payload
-(`/home/dank/models/Swift15-UncIQ3XXS/`) and the Ornith Q8_0 payload
-(`/home/dank/models/Ornith-Q8/`) all remain on disk.
-
-
-### frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, TP=2 (Dankserver, ACTIVE)
-
-The **radiance engine** (StillDeadcode, Apache-2.0; image
-`stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned, **pin 1.0.13**
-for serve; `libr4d` gfx1201 kernels, plain `podman run`, root
-systemd unit, container name = unit name) serving the
-**publisher's container of the official Qwen/Qwen3.8-Flash-Next**:
-`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe` →
-`qwen3.8-next-flash-fp8-iq4r-moe.rad` (116,322 MiB, Hub-LFS sha256
-verified at fetch by `scripts/download-frank-archon-iq4r.sh`; recipe
-`qwen4exp-w4nl64-i8-hc8m`): **routed experts as 4-bit codes into a
-16-level non-uniform codebook** after a 128-point Walsh–Hadamard rotation
-(E4M3 scale per 64 weights; codes by GPTQ on a 10M-token calibration with
-per-expert down-proj Hessians, refined by 3 coordinate-descent sweeps),
-**10 whole bf16-protected experts/layer**, **int8 W8A8 trunk/lm_head**
-(scale per 128 columns searched), **fp8 HC mixtures**, PLE n-gram table at
-fp8 fixed scale (47.7 GiB), **MTP head and bf16 vision tower in the
-container**. Card quality (full-vocab teacher-forced KL vs bf16, ~75K
-positions): mean 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %.
-**MODEL_FILE is the only serving argument that moved** (2026-10-15); every
-number of the 2026-10-13 shape below carries verbatim. The plugin's
-**one-table-one-encoding law** (fleet-measured at rad-convert declare):
-one expert table takes ONE encoding; only whole bf16 experts are legal
-beside it. Rollback: revert the swap commit AND re-download the in-house
-w4dn8 `.rad` (KLD-gated Swift1.5 conversion) — it is **no longer on disk**
-(purged 2026-10-17); the byte-exact copy is published as
-`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`
-(LFS sha256 `09dd9306…`). The fp8-native `.rad` was deleted 2026-10-15
-(rebuild: `scripts/download-frank-archon.sh` +
-`scripts/build-frank-archon-rad.sh`).
-
-**Shape (both R9700s, TP=2)**: `--tp-wire wht6`, `--max-num-seqs 3`,
-`--max-model-len 200000` (the harness contextWindow must quote this),
-`--placement expert_tiered --host-pool-mib 44928` + **explicit VRAM
-partition `--vram-weights-mib 25600 --vram-kv-mib 5120`** (the vram-pair
-replaced `--expert-vs-cache-ratio`; the two are ONE budget —
-`weights + kv ≤ 30,969 MiB`, derived from the engine's own budget table: card
-32,624 MiB, the pair at 29,120 settling 2,058 MiB free against the engine's
-209.19 MiB floor (`--gpu-headroom-mib 96` + 113.19 static), so 1,849 MiB is
-claimable and the old 29,120 bound was my hand-picked 1 GiB reserve, not a
-hardware wall), `--kv-cache-dtype` **`fp8` — quantized KV, restored 2026-10-13
-because it is the only dtype that funds the slot count at this window**,
-`--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`, prefix cache
-8 GiB pinned host + 64 GiB disk at `/home/dank/radiance-state`,
-**`--ngram-placement ram`**, Froggeric v22.5 chat template vendored
-(`scripts/qwen38-froggeric-v22.5.jinja`, `--override-chat-template`). Boot
-ledger (2026-10-13, pair 25600/5120): `kv cache 5.00 GiB … backs 3 sequences
-all at 200000 tokens at once, which is the worst case` — `kv_attn` page
-26.00 KiB (fp8) at 607,740 tokens (the sizing law predicted 608,027),
-`kv_qsa_bkey` 3.25 KiB at the same token count, `kv_gdn_state` 54 MiB × 6 slots
-(3 sequences × 2 copies) = 324 MiB; the sampler prints **9 rows**
-(3 × (1+2), inside the 25-row `wht6` exactness bound); the slab filled to
-563 KiB of slack with **17,548 slots** (16,235 at 24,000, 14,753 at 22,208 —
-every 1,792 MiB buys ~1,300 resident experts, and the pinned host pool hands
-the same experts back: 8,893 → 7,580 slots); `unclaimed 477.30 MiB`, cards
-settled ~336 MiB free against the engine's 209.19 MiB floor, KV lending
-4.22 GiB to the expert cache. Prefill tracks this slab only up to a
-**threshold**: 22,208 → 3,397 and 24,000 → 3,290 t/s at 43.5k are one band
-(±3 %), and the 25,600 arm measured **3,290** — the third step bought nothing
-(the 3,584 arm did not reproduce at 3 slots); the cliff is at ~26 GiB
-(26,071 → 4,379 on 2026-10-11's arm), so below that the slab is a threshold,
-not a slope.
-
-**Decode ladder (2026-10-13, 512 tokens, greedy, this shape)**: 99.0 t/s
-single-stream at 43.5k, 93.3 at 197k — decode is nearly window-free (−6 % for
-4.5× depth); **3 concurrent streams give 113.9/43.3/10.9 t/s = 18.4 t/s
-aggregate**, so the slot count is paid in decode concurrency (a fused-step
-lottery), not in prefill; leg D of `probe-frank-archon-kv.py` measures it.
-
-Shape notes carried from the fp8-native era and unchanged: wht6 gains,
-MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
-prefix cache, and the rule that a prompt over `--max-model-len` is
-refused, never truncated. What changed: the fp8-native era's NVMe expert
-streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the
-host pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s
-on the fp8 shape (512-token streamed probe). The campaign also
-measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
-accept 0.7246).
-
-**The KV dtype law (measured 2026-10-11, applied at 4 slots 2026-10-13)**:
-`--kv-cache-dtype` takes ONLY `bf16|fp8` — one dtype for the whole cache, so
-llama.cpp's exact-K/quantized-V pair (the 9B-era frank-zealot ran it; the
-Bonsai-2 swap runs q8_0/q8_0) has NO radiance
-equivalent; the rung under bf16 is fp8 on both halves. Measured per rank from
-the two boot ledgers: `kv_attn` (13 layers, 4-token blocks) pages 26.00 → 52.00
-KiB, while `kv_qsa_bkey` (13 layers) stays 3.25 KiB at BOTH dtypes (the QSA
-block key is an index, not attention state) — so a token costs 7.3125 KiB at fp8
-and 13.8125 KiB at bf16: **1.89×, not 2×**. Pool sizing must also pay the
-dtype-independent terms that live in the same pool (~110 MiB a sequence of
-recurrent state + 448 MiB of prefix checkpoints):
-`(vram_kv − 110·seqs − 448) MiB / rate` — bf16 at 6,912 gives 462,946 where the
-engine reported 462,752 (0.04 % page rounding); fp8 at 6,912 with four slots
-gives 843,566, i.e. the deployed capacity contract 4 × 200,000. **That is why
-the dtype is fp8: the pool must back `seqs × max-model-len` as a hard limit, and
-bf16 at four slots costs 11,679 MiB — the slab would fall to 17,441 MiB (−33 %),
-where fp8 leaves it at 22,208.** The cost is paid twice on this engine, and
-half-refunded: idle KV VRAM is also the expert cache's spillover, and the bigger
-pool *lends* 6.08 GiB to the slab where the 3,543 MiB fp8 pool lent 2.69 GiB.
-`--max-num-seqs` rides 4 × (1 + spec) = 12 fused-all-reduce rows, under the
-25-row bound where wht6 is bit-exact (the "seqs 2 cannot boot" law belongs to
-the retired tcclaviger fork's torch.compile, not radiance). Disk-tier sizing
-law: the engine forces ~9.6 GiB of disk-tier weight buffers + arena onto the
-weights side, so `weights + kv` is capped near 29.6 GiB/card; boot with the
-pair, never `--kv-cache-size-mib`.
-
-Shape notes carried from the fp8-native era and unchanged: wht6 gains,
-MNDT 2048 law (2560 collapses decode to 2.6–7.7 t/s), the 64 GiB disk
-prefix cache, and the rule that a prompt over `--max-model-len` is
-refused, never truncated. What changed: the fp8-native era's NVMe expert
-streaming (the 3.5 GB/s wall) is gone — the w4 resident set fits the
-host pool, so decode ran 122–173 t/s in the campaign arms and ~190 t/s
-on the fp8 shape (512-token streamed probe). The campaign also
-measured the int8-KV + spec-3 arm (136.0 t/s median at 44.5k depth,
-accept 0.7246).
-
-**The pinned-RAM ledger is why this is a root unit**: 2 × 44 GiB expert
-pools + the 47.7 GiB n-gram copy (`--ngram-placement ram` is pinned RAM
-the card reads) + 8 GiB prefix host ≈ **147 GiB** charged against
-`RLIMIT_MEMLOCK` — the TTM floor is set just above it.
-`ttm.pages_limit`/`pages_max` cap GPU-mappable pinned pages, and **the knob is
-runtime state whose default is half of RAM**: `scripts/ttm-preflight.sh` (sourced
-by the serve script) therefore RAISES it to `FRANK_ARCHON_TTM_TARGET_MIB=204800`
-before refusing, because the 2026-10-12 power failure reset it to 125,988 MiB
-and the preflight re-refused 6,919 times. A manual sysfs write in the stop
-window is a convenience, not the repair; kernel cmdline persistence
-(`ttm.pages_limit=52428800`) is the belt to it.
-
-**Contract facts** (source-verified): the HTTP listener binds AFTER the
-full load, so a bare `curl /health` gate is honest (`/health` == `/ping`,
-200 serving / 503 stopping); sampling params temperature…preserve_thinking
-accepted, **unknown params 400** (models.yml's extraBody set is fully
-inside it); `response_format` supported; `logprobs` REFUSED; reasoning in
-`reasoning_content` and tool calls in the qwen3.8 dialect via the auto
-chat format — no parser flags; vision native (the 27-layer Qwen-VL tower,
-`--mm-max-patches`); `/metrics` carries vLLM-compatible names +
-`radiance:*` families — the Dankclaw Prometheus scrape is re-added.
-
-Runbook: `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md`.
-Retired Swift1.5 w4dn8 era (payload purged 2026-10-17; rollback = HF re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`): `serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md`.
-Retired fp8-native era (payload deleted 2026-10-15, rebuildable): `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md`.
-Retired FreeToken era: `serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md`.
-Retired vLLM era: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
-Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md`.
+- Owns: `scripts/frank-archon.env`, `scripts/serve-frank-archon.sh`,
+  `scripts/frank-archon.service`, `scripts/ttm-preflight.sh`, template
+  `scripts/qwen38-froggeric-v22.5.jinja`; image digest-pinned
+  `stilldeadcode/radiance@sha256:4c3f4845…`; container name = unit name.
+- Shape: TP=2 over `wht6`, `--max-num-seqs 3`, `--max-model-len 200000`,
+  `expert_tiered` placement, 44,928 MiB pinned host pool per rank, VRAM pair
+  `--vram-weights-mib 25600 --vram-kv-mib 5120` (ONE budget: weights + kv ≤
+  30,969 MiB/card), `fp8` KV, MTP depth 2, MNBT 2048, prefix cache 8 GiB host
+  + 64 GiB disk, n-gram table pinned to RAM.
+- Harness/gate contract: `models.yml` `contextWindow` MUST quote 200,000
+  (over-length is refused, never truncated); the ~147 GiB pinned-RAM ledger
+  is why this is a root unit, and `ttm-preflight.sh` raises
+  `ttm.pages_limit` itself because the kernel resets that knob to half of RAM
+  on every boot; `/health` binds only after the full load, so a bare health
+  gate is honest.
+- Page: [[frank-archon]].
 
 ## Key Lessons Learned
 Durable operational lessons from the fleet's history. Full detail in the
@@ -859,15 +563,13 @@ linked runbooks.
 
 **ROCm / GPU**
 
-- **Layer split was the deployed shape on both multi-device llama.cpp services,
-  for two different reasons.** On Dankserver it was the *only* way both R9700s
-  served one model (no collective could be dispatched on card #2) until the
-  2026-09-20 board swap; frank-medium now runs radiance TP=2 instead. On a
-  RCCL-less fork build it is also the fast one: tensor parallel falls back to
-  a butterfly allreduce and collapses prefill ~7x (measured 271.8 vs 1991 t/s).
-  Pin the spec-decode drafter to one device — an unpinned draft under tensor
-  mode aborts the KV build, and on a layer split the natural home is the card
-  that ends up holding the target's last layers.
+- **Layer split needs no collective** — layers go in order and only the boundary
+  activation crosses PCIe, so it is the fallback wherever a collective cannot be
+  dispatched (and on an RCCL-less fork it beats tensor mode, which falls back to
+  a butterfly allreduce). Pin the spec-decode drafter to one device: an unpinned
+  draft under tensor mode aborts the KV build, and on a split its home is the
+  card holding the target's last layers. Deployed history:
+  [[frank-medium-llamacpp-dflash2]].
 - **Bundling a ROCm runtime next to a llama.cpp binary: `cp -r SRC DST/` is a
   trap.** With a missing `DST`, cp creates `DST` as a copy of `SRC` and drops
   one directory level. rocBLAS and hipBLASLt look for their device code at a
@@ -1022,76 +724,44 @@ linked runbooks.
   so acceptance drops on ALL traffic even where the matcher fires. Measure
   chains per architecture, never by analogy.
 - **A drafter's KV scales with the TARGET's context, and its window policy
-  decides how much that costs.** The draft context is built with
-  `cparams.n_ctx = llama_n_ctx(ctx_tgt)` (`common/speculative.cpp`) — there is
-  no flag to cap it — so a drafter that declares NO sliding window caches the
-  whole window on whatever device `--spec-draft-device` names. Swapping DFlash2
-  (`dflash.attention.sliding_window = 2048`, ~40 MiB of draft KV) for DSpark
-  (no window; 5 layers x 2 x 8 kv heads x 128 dim x 2 B = 20 KiB per position,
-  so **5.0 GiB at 262144**) put ROCm1 at 33.9 GB of 34.2 GB and killed the
-  FIRST draft graph capture with `ROCm error: out of memory` — after the model
-  had loaded and `/health` had answered. Read `sliding_window` out of the
-  drafter's GGUF header before sizing any spec-decode deployment, and prefer a
-  skewed `-ts` over a context cut when the full window matters. The pair was
-  swapped DFlash2 -> DSpark -> DFlash2 in one day (2026-09-16) and the round
-  trip moved the balance point FOUR layers (~4.3 GiB of headroom, 40:25 ->
-  36:29), which is the practical size of the effect. See
-  [[frank-medium-llamacpp-dflash2]].
+  decides the price.** The draft context is built at `llama_n_ctx(ctx_tgt)` with
+  no flag to cap it, so a drafter declaring NO sliding window caches the whole
+  window on `--spec-draft-device` and can OOM the first draft graph capture
+  AFTER `/health` has answered. Read `sliding_window` out of the drafter's GGUF
+  header before sizing, and prefer a skewed `-ts` over a context cut. Worked
+  example: [[frank-medium-llamacpp-dflash2]].
 - **A draft context bills its own f16 KV over the WHOLE POOL, and on a
-  single-card endpoint that squeeze can cost more than the drafter wins.** The
-  draft context is built at the target's `n_ctx`, so with `--parallel N` the
-  drafter pays the POOL, not the slot: on frank-zealot's 786,432-position
-  qwen35 pool, `--spec-type draft-mtp` added **3,072.00 MiB of f16 draft KV**
-  (4,096 B/position — the draft cache is never quantized) plus its own compute
-  buffers, 4,830 MiB all told. Booted arms, same card, same traffic
-  (2026-10-18): 3 slots + ubatch 2048 → **167.2 t/s decode with spec OFF,
-  109.2 with spec ON**; dropping `UBATCH_SIZE` to 1024 hands the buffers back
-  and spec ON gives **213.5** (+28 % over the spec-OFF arm), and at 1–2 slots
-  the SAME card runs MTP at ubatch 2048 and still gains ~28 %. So the MTP
-  question is never "does the file carry the head" — it is "what do draft KV
-  + buffers cost against whatever the batch knobs keep for compute buffers",
-  and that answer is per-shape. Measure the arm, never the flag. See
-  [[frank-zealot-llama-prism-bonsai9-mtp]].
-- **`--ubatch-size` has an OPTIMUM on the 2x R9700 layer split, not a plateau —
-  and the curve is BACKEND-SPECIFIC, so never carry it across (next bullet). Its
-  compute buffers land about TWICE as heavy on the card that holds the output
-  layers and the drafter.** Measured on frank-medium 2026-09-16 at the deployed
-  shape (262144 over two slots, a 64k prefill measured at 64k depth,
-  `scripts/bench-batch-ubatch.sh`): prefill runs 1046 tok/s at ubatch 256, 1068
-  at 384, **1193 at 512 (the peak)**, 1135 at 768, 1152 at 1024, 1151 at 1280,
-  1165 at 1536, 1148 at 1792 and 1078 at 2048. Both ends lose — below 512 the
-  SMs are not saturated, and 2048 is 10 % off the peak — and 512 -> 1536 costs
-  ROCm0 1.30 GiB and ROCm1 2.41 GiB to go 2.4 % SLOWER. The logical
-  `--batch-size` does not matter: 2048 through 8192 land within 2 % of each
-  other at a fixed ubatch and use identical VRAM. Decode never moves at all
-  (15.88 tok/s at depth 65536 on every cell). Because the buffers are
-  asymmetric, a split balanced at one batch is unbalanced at another: 512 ->
-  1536 shifted frank-medium's balance by a layer, so 36:29 became 35:30.
-  Re-measure the balance after changing the batch. (8192/2048 failed to allocate
-  its 2020 MiB pp buffer here under the DSpark drafter, when ROCm1 had 0.08 GiB
-  free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
-  model — the sweep's or the seam's — can see it.)
-- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
-  frank-zealot ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
-  live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
-  ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
-  2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
-  279-token prompt, which is below either ubatch, came back identical — that is
-  the mechanical proof the knob was the only variable, since a prompt shorter
-  than the ubatch cannot see it. The logical `--batch-size` was again free. And
-  the predecessor's "ubatch 2048 failed the scratch boot" was a shape
-  observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
-  it by restarting the SERVICE through a transient `FRANK_ZEALOT_ENV_FILE`
-  drop-in, never a scratch port — one card holds one server. See
-  [[frank-zealot-llama-mimo-q6k-2x200704]].
+  single-card endpoint that squeeze can cost more than the drafter wins.** With
+  `--parallel N` the drafter pays the pool, not the slot, plus its own compute
+  buffers. So the MTP question is never "does the file carry the head" but "what
+  do draft KV + buffers cost against what the batch knobs leave for compute" —
+  a per-shape answer. Measure the arm, never the flag; the booted arm table is
+  on [[frank-zealot]].
+- **A quant's scale overhead belongs in its byte rate; doubling a measured
+  figure to "get the f16 number" invents 6.25 %.** q8_0 is 1.0625 B/element
+  (32 codes + one 2-byte scale per 32-block), f16 is exactly 2.0 ⇒ the ratio is
+  **1.882×**, not 2×. Derive byte rates from element count × block layout, then
+  check them against the boot ledger — this repo carried an invented 2× number
+  for weeks. Current KV math: [[frank-zealot]].
+- **`--ubatch-size` has an OPTIMUM, not a plateau, and the curve is
+  BACKEND-SPECIFIC — never carry it across hosts.** Its compute buffers land
+  about TWICE as heavy on the card that holds the output layers and the drafter,
+  so a split balanced at one batch is unbalanced at another: re-measure the
+  balance after any batch change. The logical `--batch-size` is nearly free.
+  A large-batch allocation can fail on BUFFER CONTIGUITY while totals still fit
+  — no totals model can see that. Ladders: [[frank-medium-llamacpp-dflash2]]
+  (R9700 split) and [[frank-zealot]] (CUDA 3090).
+- **On the CUDA 3090 that same knob has no interior optimum up to 2048.** A
+  prompt shorter than the ubatch cannot see the knob at all — that is the
+  control that proves the knob, not the traffic, moved the number. When a bench
+  IS ordered, A/B it by restarting the SERVICE through a transient
+  `FRANK_ZEALOT_ENV_FILE` drop-in, never a scratch port: one card holds one
+  server. Numbers: [[frank-zealot]].
 - **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
-  size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
-  worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one
-  (0.45 GiB of weights plus ~1.0 GiB of its own f16 KV at 262144) — both
-  measured on frank-medium. Balance by measurement, never by interpolating
-  between two splits; an affine fit reproduces the deployed point and overstates
-  the gaps in between. `tests/frank_medium_unit.bats` encodes that fit as a
-  gate. See [[frank-medium-llamacpp-dflash2]].
+  size** (a DeltaNet boundary and a full-attention boundary can differ by ~1 GiB
+  each with its own KV), so balance by measurement. An affine fit reproduces the
+  deployed point and overstates the gaps in between.
+  [[frank-medium-llamacpp-dflash2]].
 - **ngram-mod prints no startup banner.** The load log shows the drafter's
   `adding speculative implementation 'draft-dflash'` and nothing at all for the
   matcher, so a chain can look unengaged while it is working. Identify it
@@ -1102,32 +772,17 @@ linked runbooks.
   without acceptance.
 - **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the
   per-slot number is the contract — not `ctx-size`.** `n_ctx_slot =
-  CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
-  (`400 exceed_context_size_error: request (184001 tokens) exceeds the
-  available context size (180224 tokens)`) even when the pool has room. So
-  frank-zealot's 2 → 3 reslice moved each conversation from the model's native
-  262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
-  the harness would discover the ceiling as a 400 on a long conversation. Side
-  effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
-  100.4 tok/s) — less attention to scan and one fewer DeltaNet state resident.
-  The 2026-10-03 reslice went the other way (3 → 2 slots, 180,224 → 200,704,
-  pool 401,408) to fund unquantized f16 V, and `contextWindow` followed again;
-  decode landed 104.8 tok/s — the ~2 % off 106.8 is the exact-V tax. The same
-  day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung
-  itself is a further measured −10 %, weight-bandwidth, not slot geometry).
-  All moves measured; both slot directions cost or buy window, never pool.
-  See [[frank-zealot-llama-mimo-q6k-2x200704]].
+  CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT with
+  a 400 even when the pool has room, so `models.yml` `contextWindow` must follow
+  every reslice. Both slot directions cost or buy window, never pool; shorter
+  slots decode slightly faster (less attention to scan, one fewer recurrent
+  state). Measured reslices: [[frank-zealot]].
 - **Price a VRAM budget from two boots, never from a formula — and an mmproj
-  costs ~1.5x its file size.** The frank-zealot 3090 fit was solved from two
-  measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
-  **non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
-  14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
-  that bills ~1.4 GiB, because the encoder's activation and compute buffers
-  ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
-  "a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
-  solve `KV ≤ card − non-KV − headroom` and round to a multiple of
-  `PARALLEL × 4096`. Recorded in `scripts/frank-zealot.env` so the next resize
-  starts from measurement.
+  costs ~1.5× its file size** (the encoder's activation and compute buffers ride
+  with it). Subtracting two booted rows isolates it exactly; extrapolating
+  weights+KV+"a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7.
+  Then solve `KV ≤ card − non-KV − headroom` and round to a multiple of
+  `PARALLEL × 4096`. The two-boot table: [[frank-zealot]].
 - **`images` and `image_url` are NOT interchangeable: only the content part
   reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
   model still invents an answer if the client used the legacy top-level
@@ -1289,68 +944,34 @@ linked runbooks.
   already does all three; ad-hoc fetches tend not to. Gated repos (e.g.
   `orcarouter/Qwen3.8-Flash-Next-Uncensored-GGUF`, same 104.09 GiB Q4_K_S)
   return 403 even with the host's HF token until that account accepts the gate.
+- **A scout verdict is a lead, never a finding.** In the 2026-10-19 vault de-dup
+  campaign, four read-only scouts worked the same pages: one classified era pages as
+  stale *because* they document retired payloads (that is an era page's content), one
+  reported files that do not exist, one reported duplications that grep disproved
+  (`457.9 t/s` and the ping-RTT table are solo-held by `Ethernet Bandwidth Math`),
+  and one cited a line number past the end of the file it judged. Re-verify every
+  claimed duplicate and every claimed staleness against the file and today's fleet
+  state before editing; a scout yield buys no evidence.
 
 ## Runbook Index
 
-Active:
+One page per live endpoint; every payload swap it has ever run is a row in that
+page's **Eras** table, with the rollback path in the same row. There is no
+per-attempt page and no per-swap ledger anywhere else.
 
-| Service | Runbook |
-|---------|---------|
-| frank-dragoon (flash-server 0.16.2, NATIVE v2 abliterated .hgn + MTP always-on, 2 slots over ONE 401,408 pool at ctx 200704, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md` |
-| frank-zealot (llama.cpp PrismML fork build 10777, Bonsai-2-**9B** PQ2_0 **with the MTP head**, 3×262,144 q8_0 KV, MTP off by measurement, vision ON via Q8_0 mmproj, Dankclaw) | `serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md` |
-| frank-archon (radiance StillDeadcode, publisher Qwen3.8-Flash-Next iq4r .rad, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md` |
-| Dankclaw kernel posture (`iommu=pt` since 2026-10-18, was `amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
+| Service | Page |
+|---------|------|
+| `frank-archon` — radiance, 2× R9700 TP=2, Dankserver | `serverAppConfigs-docs/LLM/frank-archon.md` |
+| `frank-dragoon` — halogen flash-server, Dankclaw iGPU | `serverAppConfigs-docs/LLM/frank-dragoon.md` |
+| `frank-zealot` — llama.cpp (PrismML fork), RTX 3090 | `serverAppConfigs-docs/LLM/frank-zealot.md` |
+| `frank-tts` — cloned-voice speech, Dankclaw | `serverAppConfigs-docs/Dankclaw/Frank TTS.md` |
+| Dankclaw kernel posture (`iommu=pt` since 2026-10-18) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
 
-Retired (rollback targets and history):
-
-| Runtime | Host | Window | Runbook |
-|---------|------|--------|---------|
-| llama.cpp PrismML fork build 10754, Ternary-Bonsai-2-**27B** PQ2_0, 2×200,704 q8_0 K+V, MTP off (no head in the file), vision ON (`frank-zealot`) | Dankclaw | 2026-10-17 → 10-18 (rollback target — payload + 10754 fork tree on disk; owns the 34,816 B/token 16-full-layer KV law, the 25,088 MiB f16 OOM wall and the two-boot VRAM pricing) | `frank-zealot-llama-prism-bonsai2-pq20.md` |
-| radiance Swift1.5-Qwen3.8-Flash-Next GPTQ w4dn8, 3×200,000 fp8-KV slab 25600 (`frank-archon`) | Dankserver | 2026-10-08 → 10-15 (rollback target — `.rad` purged 2026-10-17, byte-exact re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`; owns the slab-slope ladder, KV dtype law, KLD campaign, and every shape law the iq4r swap carried verbatim) | `frank-archon-radiance-swift15-w4dn8.md` |
-| llama.cpp CUDA (master) Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-17 (rollback target — payload + master CUDA build on disk; owns the 32 KiB/token 8-full-layer KV law and the 2→3-slot reslice law) | `frank-zealot-llama-qwen35-9b-q6k-2x200704.md` |
-| llama.cpp CUDA Gemma4-26B-A4B QAT Q4_K_M, 1×200,704 f16-K/q8_0-V MTP-off (`frank-zealot`) | Dankclaw | 2026-10-07 → 10-07 (same-day swap; rollback target — payload on disk; owns the `attention_k_eq_v` pair-billing KV law and the gemma4-assistant drafter-cost boots) | `frank-zealot-llama-gemma4-q4km-200704.md` |
-| halogen-flash-server 0.16.2 abliterated Swift1.5 v2 `.hgn` + MTP always-on, TWO slots at ctx 262,144 over a 524,288 pool (`frank-dragoon`) | Dankclaw | 2026-10-06 → 10-17 (rollback target — payload + fetch script on disk; owns the 400-zone law and the sampler/prompt-cache contract) | `frank-dragoon-flash-server-v2-swift15.md` |
-| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md` |
-| tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-archon`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-archon-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
-| FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-archon`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-archon-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-archon-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-archon-freetoken-pr587.md` |
-| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-03 → 10-05 (rollback target — payload on disk; owns the ubatch-2048 A/B; the Ornith swap of 10-05 was committed but never deployed, so this was the LIVE payload Swift replaced) | `frank-zealot-llama-mimo-q6k-2x200704.md` |
-| llama.cpp Ornith-1.5-9B Q8_0, ONE 262,144 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-05 (committed 77decb5→380aafa, never deployed — superseded by Swift the same evening; payload on disk) | trail in `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
-| llama.cpp CUDA Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×114,688 draft-mtp q8_0 KV (`frank-zealot`) | Dankclaw | 2026-10-06 → 2026-10-07 (rollback target — payload on disk; owns the 3090 ubatch ladder A/B and the 34 KiB/token qwen35 KV law) | `frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md` |
-| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-dragoon`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-dragoon-rocmfpx-q8agent-mtp.md` |
-| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-dragoon-gufo-qwen38-27b.md` |
-| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-dragoon`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-dragoon-vllm-mimo-v26-bf16.md` |
-| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-zealot`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-zealot-llama-mimo-q6k-2x200704.md`) | `frank-zealot-llama-mimo-q6k.md` |
-| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-zealot`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-zealot-llama-mimo-q5km.md` |
-| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-zealot`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-zealot-sglang-mimo-v26.md` |
-|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-archon`)|Dankserver|2026-09-30 → 10-01|`frank-archon-glm53-flash-w4a16.md`|
-| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-dragoon`) | Dankclaw | 2026-09-21 → 09-28 | `frank-dragoon-unsloth-qwen38-27b-q8xl.md` |
-| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-archon`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-archon-r9v-radiance-qwen38-uncensored.md` |
-| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-archon`) | Dankserver | 2026-09-26 → 09-27 | `frank-archon-llamacpp-qwen38-flash-next-q8.md` |
-| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-dragoon`) | Dankclaw | 2026-09-15 → 09-16 | `frank-dragoon-ik-uncensored.md` |
-| vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` |
-| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-archon`) | Dankserver | 2026-09-26 (same-day swap) | `frank-archon-lucebox-ds4.md` |
-| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
-| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-dragoon`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
-| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-dragoon`) | Dankclaw | 2026-09-19 → 09-21 | `frank-dragoon-ciru-orca.md` |
-| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-unsloth-qwen38-flash-next.md` |
-| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-dragoon`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-dragoon-kairic-edge-qwen38-27b.md` |
-| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-zealot`) | Dankclaw | 2026-09-16 → 09-17 | `frank-zealot-qwen38-gsq-rco.md` |
-| vLLM Gemma4-12B w4a16 (`frank-zealot`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-zealot-vllm-gemma4.md` |
-| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-zealot`) | Dankclaw | 2026-09-17 → 10-01 | `frank-zealot-gemma4-12b.md` |
-| agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
-| eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
-| Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |
-| CIRU ActiveFPX v3 / PromptForge | Dankclaw | through 2026-08-22 | `ciru-activefpx-promptforge.md` |
-| DFlash2 Vulkan | Dankclaw | 2026-08-21 (same-day revert) | `dflash2-large-vulkan.md` |
-| FastFlowLM NPU (`frank-npu`) | Dankclaw | twice, retired 2026-09-11 | `frank-npu.md`, `FastFlowLM NPU Inference.md` |
-| FreeToken (official + Huihui FP8, `frank-zealot`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
-| FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` |
-| ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` |
-| llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` |
-
-Retired runbooks are under `serverAppConfigs-docs/LLM/`. Retired scripts live
-in `old/` and `.AGENTTRASH/`.
+Retired **endpoints** (not payloads) keep their own pages — `frank-medium`,
+`frank-npu`, the pre-name Dankclaw/Dankserver eras — because no live page owns
+them; `serverAppConfigs-docs/index.md` is the locator for every page in the
+vault. Retired scripts live in `old/` and `.AGENTTRASH/`.
 
 ## Why
 
@@ -1368,7 +989,9 @@ history.
 | Unstaged local changes | **Commit or discard** — check `git status` before starting work; ask if unsure. |
 | Deploying a committed change | **`scripts/deploy.sh <target>`** — pushes, pulls ff-only, plans, dispatches, health-checks. |
 | Container without a Quadlet unit | **Give it one** — boot-persistent, self-healing `.container` via git. |
-| Benchmarking | **Stop and mask the service first**, bench, unmask, restart, verify health. |
+| Changing a model/payload | **Swap it** — edit the trio, deploy, `health` + one chat/tool/vision probe. No benches unless Frank orders one; if it misbehaves, revert. |
+| Writing a doc | **One owner per fact** — edit the endpoint/service page; everywhere else links. Never restate a number. |
+| Benchmarking (only when ordered) | **Stop and mask the service first**, bench, unmask, restart, verify health. |
 | Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
 | Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
 | Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |

dankclaw/voiceservice/app.py

@@ -7,7 +7,10 @@ Consumers (the omp harness, OpenWebUI, open-notebook) speak the OpenAI
 directory under ``$VOICES_DIR`` and ``reference.wav`` inside it is what makes
 the name real. A name that has no reference is not a voice — the shim answers
 400 with the list that is, instead of silently narrating in a fallback voice
-(silent fallback is how a wrong-voice endpoint stays wrong for weeks).
+(silent fallback is how a wrong-voice endpoint stays wrong for weeks). A request
+that names NO voice at all takes ``$DEFAULT_VOICE``: that is a spoken choice in
+the env file rather than a surprise in the code, and the journal marks it
+``(default)`` so a name nobody asked for is never mistaken for one that was.
 
 Two backends sit behind the one surface, chosen by ``TTS_BACKEND``:
 
@@ -70,7 +73,7 @@ class Config:
 
     def __init__(self, voices_dir, backend, upstream, upstream_model, llama_bin,
                  llama_model, llama_mmproj, llama_lang, llama_threads, port,
-                 max_chars, timeout):
+                 max_chars, timeout, default_voice=""):
         self.voices_dir = Path(voices_dir)
         self.backend = backend
         self.upstream = upstream
@@ -83,6 +86,7 @@ class Config:
         self.port = port
         self.max_chars = max_chars
         self.timeout = timeout
+        self.default_voice = default_voice
 
     @classmethod
     def from_env(cls, environ=None):
@@ -100,6 +104,7 @@ class Config:
             port=_int(env, "SHIM_PORT", 7772),
             max_chars=_int(env, "SHIM_MAX_CHARS", 15000),
             timeout=float(_int(env, "SHIM_TIMEOUT", 600)),
+            default_voice=(env.get("DEFAULT_VOICE") or "").strip(),
         )
 
 
@@ -271,8 +276,13 @@ class SpeechHandler(BaseHTTPRequestHandler):
                                    "endpoint serves wav and mp3" % fmt))
             return
 
-        # `voice_id` is the alias the omp harness forwards for `voice`.
-        voice = body.get("voice") or body.get("voice_id")
+        # `voice_id` is the alias the omp harness forwards for `voice`. A request
+        # that names nothing takes the declared default — the harness has no
+        # setting that feeds a voice, so voice-less calls are the normal shape
+        # there, not a client bug.
+        requested = body.get("voice") or body.get("voice_id")
+        defaulted = not requested and bool(cfg.default_voice)
+        voice = requested or cfg.default_voice
         reference = resolve_voice(cfg.voices_dir, voice)
         if reference is None:
             available = list_voices(cfg.voices_dir)
@@ -297,8 +307,9 @@ class SpeechHandler(BaseHTTPRequestHandler):
 
         content_type = MP3_CONTENT_TYPE if fmt == "mp3" else WAV_CONTENT_TYPE
         self._bytes(200, audio, content_type)
-        self.log_message("speech voice=%s format=%s chars=%d -> %d bytes %s",
-                         voice, fmt, len(text), len(audio), content_type)
+        self.log_message("speech voice=%s%s format=%s chars=%d -> %d bytes %s",
+                         voice, " (default)" if defaulted else "", fmt,
+                         len(text), len(audio), content_type)
 
     # -- backends ----------------------------------------------------------
     def _speech_http(self, cfg, model, text, voice, reference):

scripts/autoresearch-frank-archon.sh

@@ -29,10 +29,10 @@
 # The scratch mirrors the production container contract exactly (serve-frank-
 # primary.sh): --network host, --init, seccomp=unconfined (io_uring for the
 # n-gram table), memlock unbounded (the ~132 GiB pinned ledger), /dev/kfd +
-# /dev/dri, the .rad ro, the state dir rw, the tune cache mount. The ONLY
-# differences: the container name (arbench, so prod's identity is untouched)
-# and a FRESH prefix-cache disk tier (arbench/, wiped per run) so the cold
-# prefill is cold — the prod tier at kvcache/frank-archon is left intact.
+# /dev/dri, the .rad ro, the tune cache mount. The ONLY differences: the
+# container name (arbench, so prod's identity is untouched) and the prefix cache
+# switched OFF (--prefix-cache-mib 0), so every prefill is cold — the tier is
+# host RAM now, there is no per-run disk directory to wipe.
 set -uo pipefail
 
 ENV_FILE="${1:-/home/dank/serverAppConfigs/scripts/frank-archon.env}"
@@ -60,9 +60,8 @@ set -a; . "$ENV_FILE"; set +a
 sudo systemctl stop frank-archon >/dev/null 2>&1 || true
 podman rm -f "$SCRATCH" >/dev/null 2>&1 || true
 
-# --- fresh prefix-cache tier for a cold prefill (prod's tier untouched) ------
-FRESH_KV="$FRANK_ARCHON_STATE_DIR/kvcache/$SCRATCH"
-rm -rf "$FRESH_KV"; mkdir -p "$FRESH_KV"
+# --- the prefix cache is switched off below, so there is no scratch KV tree ---
+# to wipe; the cold prefill comes from --prefix-cache-mib 0.
 mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
 
 # --- boot the scratch: the production run line, scratch name + fresh KV ------
@@ -73,7 +72,6 @@ podman run -d --name "$SCRATCH" \
   --ulimit memlock=-1:-1 \
   --device /dev/kfd --device /dev/dri \
   -v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-  -v "$FRANK_ARCHON_STATE_DIR":/data \
   -v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
   -v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
   "$FRANK_ARCHON_IMAGE" \
@@ -87,12 +85,9 @@ podman run -d --name "$SCRATCH" \
   --placement "$FRANK_ARCHON_PLACEMENT" \
   --host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
   --gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
-  --vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
-  --vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
   --kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
-  --prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
-  --prefix-cache-dir "/data/kvcache/$SCRATCH" \
-  --prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
+  --prefix-cache-mib 0 \
+  --busy-prefill-chunk "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:-auto}" \
   --ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
   --num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
   --max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \

scripts/frank-archon.env

@@ -1,306 +1,131 @@
-# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving the
-# PUBLISHER's 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next
-# (MODEL_FILE below, fetched by scripts/download-frank-archon-iq4r.sh) on
-# BOTH R9700s (TP=2), dankserver :7770.
-# 2026-10-15 PAYLOAD SWAP: the model moved from the in-house w4dn8 conversion
-# of the Swift1.5 finetune to the engine author's published container
-# qwen3.8-next-flash-fp8-iq4r-moe.rad (116,322 MiB, sha256-verified at fetch —
-# the w4dn8 measured 116,416 MiB, so this shape carries over with 94 MiB to
-# spare). NO argument changed: every number below is the 2026-10-13 shape,
-# kept verbatim. The swap commit is the rollback (the w4dn8 payload stays on
-# disk).
-
+# frank-archon.env — radiance (StillDeadcode) on BOTH R9700s (TP=2),
+# Dankserver :7770, publisher's 4-bit-codebook container of the official
+# Qwen3.8-Flash-Next.
 #
-# SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV,
-# with the per-card budget spent on resident experts: 25,600 MiB of slab +
-# 5,120 MiB of pool = 30,720 MiB, inside the 30,969 MiB ceiling the engine's own
-# budget table allows (card 32,624 MiB, settle floor 209 MiB). The pool is sized
-# to the contract and nothing more (3 x 200,000 needs 5,063 MiB at fp8; 5,120
-# predicts 608,027 tokens, 1.3 % slack), because every MiB taken from the slab
-# is prefill and TTFT (2026-10-11 A/B: slab -14.8 % cost prefill -29 %, TTFT
-# +25 % at the full window) while KV beyond the contract buys nothing.
-# Consequence, deliberate: a FOURTH full-height conversation does not fit this
-# pair, and the engine refuses it — it never truncates. See the KV block for the
-# arithmetic and the measured dtype law. The w4dn8 payload is the swap
-# rollback and keeps its trio under the w4dn8 runbook; the fp8-native payload
-# that came before it was DELETED from disk 2026-10-15 (Frank's call, to free
-# the volume for this download — rebuildable via the fetch + build scripts
-# named in the fp8-native runbook).
-#
-# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
-# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
-# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
-# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
-# frank-archon-vllm-shapeG-20261003).
-#
-# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
-# gfx1201, drives `podman run` directly from the digest-pinned
-# stilldeadcode/radiance image (287 MiB; host needs only the amdgpu driver),
-# and its qwen4exp plugin reads the modelopt FP8 encoding NATIVELY
-# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
-# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
-# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
-# THE 4-bit-codebook publisher shape is the DEPLOYED payload since
-# 2026-10-15. The card's own run line (--host-pool-mib 12288,
-# --expert-vs-cache-ratio 0.82, seqs 8, spec 3) is the publisher's
-# suggestion on 32 GB cards; this file's shape is the fleet's measured one.
-#
-# PAYLOAD: /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad — the
-# official Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's
-# own author (recipe qwen4exp-w4nl64-i8-hc8m: experts as 4-bit codes into a
-# 16-level non-uniform codebook after a 128-point Walsh–Hadamard rotation,
-# GPTQ + 3 coordinate-descent sweeps on a 10M-token calibration, ten
-# bf16-protected experts per layer, int8 W8A8 trunk + lm_head, fp8 HC
-# mixtures, PLE n-gram fp8 fixed-scale, MTP head, bf16 vision tower; card KL
-# 0.0789 all-tokens / top-1 92.85 % vs bf16). Fetched byte-exact + sha256 by
-# scripts/download-frank-archon-iq4r.sh; the recipe/LICENSE stay in the
-# staging dir beside it as provenance. ROLLBACK: revert the swap commit — the
-# in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad (scripts/convert-swift15-w4dn8.sh,
-# KLD-gated) stays on disk. Behind it, the fp8-native
-# qwen4exp-fp8-native.rad was DELETED from disk 2026-10-15 (Frank's call,
-# one-time rm, to free the volume for this payload); rebuild =
-# scripts/download-frank-archon.sh + scripts/build-frank-archon-rad.sh — a
-# NO-RECIPE rad-convert copy of the official 131-shard checkpoint under the
-# descriptor law (no shipped plugin claims (qwen4exp, "fp8"), so the staging
-# config.json drops quantization_config; per-weight encodings are data, not
-# the descriptor).
-#
-# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
-# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
-#   then engine.serve()) — during load the port refuses connections, so a
-#   bare curl /health gate is honest. /health == /ping: 200 JSON while
-#   serving, 503 only while stopping (core/server/admin.cpp:385).
-# - Reasoning: --reasoning-format defaults to auto, emitting thinking in
-#   `reasoning_content` (config.cpp). Tool calls: the qwen3.8 dialect, from
-#   the chat format. Both need no parser flags.
-# - Sampling: the OpenAI endpoint accepts temperature, top_p, top_k, min_p,
-#   typical_p, presence_penalty, frequency_penalty, repetition_penalty,
-#   repeat_last_n, chat_template_kwargs, reasoning_effort,
-#   enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
-#   models.yml's extraBody set is fully inside it. Unknown params are
-#   refused (400), which is why nothing unverified rides the wire.
-# - MTP: the .rad carries the model's mtp.* head (the publisher's recipe
-#   names the mtp.fc_hidden/fc_embedding/draft_head rules; card default
-#   depth 3). Served at the depth named by FRANK_ARCHON_SPEC_TOKENS below —
-#   the depth this shape's decode ladder was measured at.
-# - The fused TP all-reduce is declared over max-num-seqs x
-#   (1 + --num-speculative-tokens) rows; 3 x (1+2) = 9 rows, under the 25-row
-#   exactness bound of the wht6 wire.
-# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
-# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
-# 25558e07fc60d20328f1346f96f3c6e00845eb763f01fa, tag 1.2.0, commit
-# e95d397) was benched 2026-10-06 on this exact shape (fp8-native rad,
-# vram pair, spec 2, seqs 3, 43.6k greedy probe): decode 29.21 t/s and
-# draft acceptance 0.7123 vs the 1.0.13 control's 30.08 and 0.7549.
-# Acceptance on the greedy harness is deterministic (the control
-# reproduced the recorded 0.755 exactly), so the drop is a real numerics
-# change — the 1.2.0 MoE weight-rotation change (92b8aaf) is the named
-# suspect; decode loss tracks the acceptance ratio (0.943 vs 0.945).
-# The vram-pair translation itself is exact (1.2.0 boot ledger took
-# vram_weights 25.46 GiB / vram_kv 3.46 GiB per card byte-identical to
-# the 1.0.13 ledger at EVC 0.82). Re-test 1.2.0+ when the rotation change
-# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
-# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
-# file already uses the pair, which BOTH pins accept (rollback symmetry).
-FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
-# The .rad container FILE (mounted under /models) and the state dir for the
-# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
-# .rad files; the raw HF download and its staging dir live beside it and
-# are never mounted.
+# ONE OWNER: shape, payload identity, eras, laws and every measured number live
+# in serverAppConfigs-docs/LLM/frank-archon.md. This file carries the knobs plus
+# only the reason a reader editing THIS line cannot recover from the page.
+# Where the two disagree, this file wins on values; the boot ledger arbitrates
+# the engine shape.
+FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed
+# 1.8.1 (2026-10-18), pinned by digest. Two CLI breaks sit between it and the
+# 1.0.13 line this endpoint ran for two eras, and both are boot refusals, not
+# warnings: --vram-weights-mib/--vram-kv-mib were REMOVED at 1.3.0 (the card is
+# now split by the engine's budget resolver — see below), and the prefix cache
+# lost its disk tier. Draft acceptance has to be re-scored on any pin past 1.2.0:
+# 1.2.0's MoE weight rotation measured 0.7123 against the 0.7549 control, and the
+# suspect is untangled (the page's Dead ends section names it).
+# The .rad file mounts under /models; the tune cache mounts separately below.
+# The model dir holds ONLY .rad files — the HF download and staging stay beside it.
 FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
 FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
 FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
-# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
-# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
-# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
-# cache must ride a host mount — serve script mounts it, see the runbook's
-# "Tuning levers").
-# Container identity is the SERVICE name (Frank's call, 2026-10-05): one
-# name across systemctl and podman ps. The runtime changes; the name does
-# not. The legacy list retires every predecessor identity that could hold
-# the GPUs or the port.
+# state dir's tune/ subdir carries rad-tune's kernel-selection cache; the engine
+# reads $RADIANCE_HOME/tune/<machine>.tune and that defaults to /opt/radiance
+# (inside the image), so the cache only survives on a host mount.
+# Container identity is the SERVICE name: one name across systemctl and
+# `podman ps`; the runtime changes, the name does not. The legacy list retires
+# every predecessor identity that could hold the GPUs or the port.
 FRANK_ARCHON_CONTAINER=frank-archon
 FRANK_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
-# Served model id — Protoss name (wire rename 2026-10-09). The container,
-# unit, scripts and this file keep the frank-archon name above; only the
-# alias on the wire moved. Radiance validates the requested model name, so
-# every client (harness, bench scripts, probes) MUST send this value.
+# Served id. Radiance VALIDATES the requested model name, so every client must
+# send exactly this value (unlike llama.cpp and halogen, which ignore it).
 FRANK_ARCHON_SERVED_NAME=frank-archon
 FRANK_ARCHON_PORT=7770
-# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
-# repo's root chat_template.jinja, vendored here; the checkpoint's own
-# template is baked into the .rad by rad-convert and this overrides it).
-# Radiance derives the reasoning/tool-call reply format from it.
+# Frank's chat-template call: froggeric's fixed Qwen3.8 template, vendored in
+# the repo; it overrides the template baked into the .rad, and radiance derives
+# the reasoning and tool-call reply format from it.
 FRANK_ARCHON_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
 
 # --- engine shape (the boot ledger arbitrates every number below) ---------
-# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
-# x8/x8 on root ports) made collectives legal on this host; radiance's TP
-# wire is libr4d's own, not RCCL.
+# TP=2 needs the 2026-09-20 board (x8/x8 on root ports): a collective cannot be
+# dispatched on a card whose PCIe chain lacks atomics. Radiance's wire is
+# libr4d's own, not RCCL.
 FRANK_ARCHON_TP=2
-# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
-# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
-# engine default.
+# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice). It
+# stays EXACT for single-sequence decode and any step of <= 25 rows — see
+# MAX_SEQS x (1 + SPEC_TOKENS) below.
 FRANK_ARCHON_TP_WIRE=wht6
-# Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three
-# concurrent conversations of --max-model-len fit the KV pool (see the KV
-# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
-# tcclaviger vLLM fork's torch.compile shape specialization on
-# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
-# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
-# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
-# (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys concurrency,
-# not speed, so a slot costs KV, never decode.
-# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
-# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
-# 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT
-# (single-sequence decode and any step of <= 25 rows is bit-exact; only busier
-# steps are lossy).
-# The pool is sized to this contract and nothing more (see the KV block):
-# 3 x 200,000 at fp8 = 5,063 MiB, pool 5,120, and the freed 1,792 MiB went into
-# the resident expert slab. The trade is explicit and it is Frank's call:
-# a FOURTH full-height conversation no longer fits (5,120 MiB of pool backs
-# ~608k tokens; 4 x 200,000 needs 6,601), so a fourth 200k request is refused
-# as too large to fit — it is NOT squeezed, paged, or truncated. Third-party
-# traffic that wants four full-height sessions is a --vram-kv-mib change, not
-# a code change.
-FRANK_ARCHON_MAX_SEQS=3
-# Served window. Native is 262,144; 200,000 is the publisher's production
-# value (compose) and what models.yml's contextWindow must quote — a longer
-# prompt is refused by the engine, never truncated. The engine treats
-# `kv group tokens >= max-model-len` as a HARD limit ("ONE request longer
-# than N tokens is accepted by the API and then refused as too large to ever
-# fit — this is a hard limit, not a backing ratio"), so the pool is sized
-# for seqs x maxlen, not for one sequence.
+# Decode slots. A slot costs KV, never decode; the pool is sized to this
+# contract and nothing more. At 1.8.1 the budget resolver caps the cache at
+# 556,348 fp8 tokens — 2.8 sessions at 200,000 — so a THIRD full-height
+# conversation answers 400 while /health and every shorter chat work fine
+# (3 x 50k streams run at 77.6 t/s aggregate). Frank's trade: declare the two
+# slots the engine can hold and let the freed ~1.1 GiB go to the expert plane,
+# instead of advertising a third slot that only exists at shorter prompts.
+FRANK_ARCHON_MAX_SEQS=2
+# Served window. The engine treats `kv group tokens >= max-model-len` as a HARD
+# limit, and models.yml contextWindow MUST quote this number (over-length is a
+# refusal, not a truncation). The pool is sized for seqs x maxlen, not one
+# sequence.
 FRANK_ARCHON_MAX_MODEL_LEN=200000
 FRANK_ARCHON_PLACEMENT=expert_tiered
-# Pinned host pool for the expert rows that do not fit VRAM. vram_weights and
-# this pool serve the per-rank resident set TOGETHER: at the fp8-KV pair
-# (26071/3543) the ledger showed the VRAM slab filled to 259 KiB of slack,
-# i.e. the pair was exactly sized. Cutting vram-weights by 3.8 GiB to pay for
-# the bf16 KV pool SPILLS ~3.8 GiB/rank here, so the pool moves with it —
-# under-sizing it is a boot failure (36864 refused the boot in the fp8 era).
-# The boot ledger's coverage line remains the referee, not this arithmetic.
+# Pinned host pool for expert rows that do not fit VRAM. Under the budget
+# resolver this is ALSO the lever that decides which experts stay on the card:
+# the card's leftover goes to the expert plane first and the KV cache takes the
+# rest, so growing the pool moves experts off the card and shrinking it spills
+# them onto VRAM the cache wanted. Under-sizing it is a boot failure (36,864
+# refused a boot). The ledger's coverage line is the referee, not this file.
+# Measured, not guessed: 45,568 (PR #30) moved host_pool to 45.09 GiB and left
+# vram_kv byte-identical at 4.20 GiB / 556,348 tokens on both ranks — extra
+# host RAM above what the expert plane's footprint needs buys no cache. Back at
+# 44,928, the size that boots clean.
 FRANK_ARCHON_HOST_POOL_MIB=44928
-# VRAM left unclaimed per card (publisher default 96 MiB).
+# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
+# floor is built from it.
 FRANK_ARCHON_GPU_HEADROOM_MIB=96
-# VRAM split of claimed memory per card, MiB (0 = derive from the card).
-# THE CEILING, read off this engine's own VRAM-budget table (boot 2026-10-13,
-# pair 24000/5120): card 32,624 MiB; budgets claimed 29,120; `3.42 GiB is
-# neither budgeted nor accounted — the driver's context, the activation arena
-# and --gpu-headroom-mib come out of it; anything past those is throughput on
-# the floor`; and `card 0 settled at 2.01 GiB free against 209.19 MiB (the
-# 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)`.
-# So the reserve above the ENGINE's floor is 2,058 - 209 = 1,849 MiB and the
-# ceiling is 29,120 + 1,849 = 30,969 MiB. The 2026-10-11 bound of 29,120
-# ("31,856 - 1,710 outside both budgets - keep 1 GiB free") was MY reserve, not
-# the engine's; the pair below spends the difference and lands 458 MiB above
-# the engine's floor. `weights + kv <= 30,969` is the gate.
-# THE PRICE, and it is paid twice: on this engine idle KV VRAM is ALSO the
-# expert cache's spillover (`mover: 2275 slots can be lent out of the KV
-# cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <= 256 tokens`),
-# so a bigger KV pool steals weight residency from the slab AND from the
-# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
-# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
-# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. The 2026-10-11
-# exact-KV A/B measured the slab's own price (22208 vs the fp8-era 26071):
-# prefill -29 %, TTFT at the full window +25 %, decode inside the noise band on
-# 2 of 3 passes, draft acceptance UP 0.661 -> 0.740.
-# 2026-10-13 (late): THREE slots, so the pool is cut to the contract (5,120)
-# and every spare MiB goes to the slab: 22208 -> 24000 (the third slot's
-# surplus KV), then 24000 -> 25600 (the card's settling slack). The direction
-# the A/B data points: prefill is slab-limited, KV beyond the contract is not.
-# The margin is thin by design (608,027 predicted tokens against the 600,000
-# contract), so the boot ledger's `backs N sequences all at 200000 tokens at
-# once` line is the referee for THIS pair and the concurrent probe is its
-# regression test.
-# VERDICT on the 25,600 arm (measured 2026-10-13, scripts/probe-frank-archon-kv.py):
-# the threshold claim is FALSIFIED — the slab is a SLOPE at this end. A-cold
-# 43.5k single-stream: 22,208 -> 3,397, 24,000 -> 3,290, 25,600 -> 3,584 t/s;
-# the 3-way 197k arm's aggregate prefill 3,051 -> 3,589 t/s (wall
-# 193.7 -> 164.5 s). So +1,600 MiB of slab = +8.9 % single-stream and
-# +17.6 % concurrent, both outside the +-3 % noise floor. The 22,208/24,000
-# pair landed inside noise of EACH OTHER, which is what made it read as a
-# plateau; two points are not a curve.
-# The cliff that remains is between 25,600 (3,584) and 26,071 (4,379, the
-# 2026-10-11 fp8 control at seqs 3 / pool 3,543) — +22 % for +471 MiB. At THREE
-# slots that is only reachable by breaking the ceiling (26,112 + 5,120 = 31,232
-# vs 30,969, ~214 MiB above the engine's floor); at TWO slots it is free:
-# pool 3,072 + slab 27,897 = exactly the ceiling, slab above the cliff.
-FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
-FRANK_ARCHON_VRAM_KV_MIB=5120
-# KV cache QUANTIZED (fp8). This engine's flag takes ONLY bf16|fp8 — there is
-# ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
-# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
-# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
-# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
-# 2026-10-08, bf16 boot 2026-10-11):
-#   kv_attn     13 layers, 4-token blocks, page 26.00 KiB (fp8) -> 52.00 KiB
-#   kv_qsa_bkey 13 layers, 4-token blocks, page  3.25 KiB at BOTH dtypes —
-#               the QSA block key is an index, not attention state, and does
-#               NOT follow --kv-cache-dtype
-#   => 7.3125 KiB/token fp8, 13.8125 KiB/token bf16: bf16 costs 1.89x fp8,
-#               not 2x, precisely because the bkey plane is dtype-fixed.
-# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
-# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
-# checkpoint region 8 x 55.99 = 448 MiB. So:
-#   pool_mib >= tokens x 7.3125 KiB + 110 x seqs + 448
-#   3 x 200,000 at fp8 needs 4,285 + 330 + 448 = 5,063 MiB; the pool is 5,120,
-#   and it predicts (5120 - 330 - 448) x 1024 / 7.3125 = 608,027 tokens: the
-#   three-slot contract with 1.3 % slack, every MiB above 5,063 having been
-#   handed to the expert slab instead. The ledger line is the referee ("backs N
-#   sequences all at 200000 tokens at once") — a pair this tight is read off the
-#   boot, never assumed.
-#   At bf16 the same three slots cost 8,093 + 778 = 8,871 MiB, which would cut
-#   the slab to 20,249 MiB (-9 %); at FOUR slots bf16 costs 10,791 + 888 =
-#   11,679 MiB and the slab falls to 17,441 (-33 %). fp8 keeps 22,208 at both
-#   slot counts. The 2026-10-11 bf16 shape (6,912 / seqs 2) backed 462,752
-#   tokens = 2 x 200,000; the historical fp8 pool of 3,543 gave 386,904 (1.93
-#   sessions), and the four-slot fp8 boot of 2026-10-13 19:21 backed 843,276.
-# Secondary saving: the prefix-cache entry shrinks with the attention plane
-# (one entry 110.50 KiB at bf16 -> 58.50 KiB at fp8, measured on the two boot
-# ledgers' `kv tier transfer` lines), so the host tier below parks twice as
-# many conversations as it did at bf16 — no re-size needed to park three slots.
+# There is no VRAM pair any more. --vram-weights-mib/--vram-kv-mib were removed
+# at 1.3.0: the engine takes capacity - already-held - GPU_HEADROOM - arena,
+# puts the static (non-expert) weights off the top, then divides the rest between
+# the expert plane and the KV cache — experts first when the cache has no idle
+# blocks to lend, and never less than ONE MAX_MODEL_LEN sequence for the cache.
+# What moves the split from here is HOST_POOL_MIB (which experts stay resident),
+# MAX_MODEL_LEN/MAX_SEQS (the cache bound) and GPU_HEADROOM_MIB. The slab size
+# the prefill-slope law cares about is a boot-ledger line now, not a knob: read
+# it after every resize instead of asserting it here.
+# fp8 is the ONLY rung below bf16 on this engine: one kv_dtype for the whole
+# cache, so llama.cpp's exact-K/quantized-V split (frank-zealot) is unservable
+# here. bf16 costs 1.89x fp8, not 2x — the QSA block-key plane is dtype-fixed.
 FRANK_ARCHON_KV_DTYPE=fp8
-# Spec decode: the checkpoint's own MTP head, served at depth 2 — the
-# publisher's card ships depth 3 as its peak; this fleet's ladder was
-# measured at 2 and the swap keeps every argument, so 2 stays (rows
-# 3 x (1+2) = 9, inside the wht6 25-row exactness bound).
+# The checkpoint's own MTP head at depth 2: this fleet's ladder was measured at
+# 2, and 3 x (1+2) = 9 rows stays inside the wht6 25-row exactness bound.
 FRANK_ARCHON_SPEC_TOKENS=2
 FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
-# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
-# and one entry tracks the cache dtype (58.50 KiB at fp8, 117 KiB at bf16, per
-# the ledger's `kv tier transfer` line): at fp8, 8192 MiB parks ~5.8 sessions
-# of 200k (at bf16 the same tier parked ~2.9), so four live slots plus a full
-# parked set fits with the tier as it stands — at RAM the host has (104 GiB
-# available with the service loaded).
-FRANK_ARCHON_PREFIX_HOST_MIB=8192
-FRANK_ARCHON_PREFIX_DISK_MIB=65536
-# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
-# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
-# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
+# Step sharing. A prefill chunk that shares a step with decoding sequences holds
+# the device for the chunk's whole time, and every decode row in that step waits
+# that long for its next token (the census: [[LLM Performance Diagnosis]] §9).
+# `auto` leaves each decode row behind the chunk a row plus its draft window and
+# keeps a share for whatever is waiting to be admitted; a number pins the chunk's
+# cap; `off` is the pre-1.4 behaviour this endpoint ran for two eras, where the
+# chunk took the step and admission never ran. Interleave inserts N decode-only
+# steps after a chunk step taken beside decoders: it protects their cadence and
+# holds a waiting request out of those steps, so it stays 0 until the post-migration
+# decode-vs-prefill buckets say it is wanted.
+FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto
+FRANK_ARCHON_PREFILL_INTERLEAVE=0
+# Prefix cache: pinned host memory only. The disk tier was removed outright
+# (--prefix-cache-dir and --prefix-cache-disk-mib are refusals), so what used to
+# be restored from NVMe is now re-prefilled. The tier is sized in ENTRIES and one
+# entry tracks the cache dtype (58.50 KiB at fp8). Watch prefix_hit_rate after
+# this change: a drop is extra prefill, and prefill is exactly the traffic that
+# costs decode its cadence.
+FRANK_ARCHON_PREFIX_CACHE_MIB=8192
+# Frank's pin law: the PLE n-gram table serves from pinned host RAM the cards
+# read. `ram` REFUSES the start if memlock is too small — the unit's
+# LimitMEMLOCK=infinity is the other half of this law.
 FRANK_ARCHON_NGRAM_PLACEMENT=ram
-# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
-# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
-# pages_limit (the current kernel exposes NO pages_max; the cap function
-# takes the min over the knobs that exist). Below the plan the load dies
-# mid-pin with an allocator error instead of this named refusal.
-# THE CAP IS A RUNTIME KNOB — the kernel default is half of RAM, so every
-# raise dies with a reboot. Measured the hard way: the 2026-10-05 manual raise
-# to 52,428,800 pages (200 GiB) was gone after the 2026-10-12 23:17 POWER
-# FAILURE, the cap fell back to 125,988 MiB, and the preflight refused the
-# start 6,919 times in ~20 h while Restart=always re-ran it. The gate in
-# scripts/ttm-preflight.sh now RAISES the knob itself before refusing (the
-# root unit is privileged; the knob is a 0644 root module parameter), so the
-# recovery is a boot, not a visit to the machine.
+# TTM preflight. The pinned ledger rides TTM's shared pages, and the kernel
+# RESETS pages_limit to half of RAM on every boot; below the plan the load dies
+# mid-pin with an anonymous allocator error. ttm-preflight.sh raises the knob
+# itself before refusing by name, so recovery is a boot, not a visit.
+# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT
+# per the bats ledger test; at the 44,928 pool that computes to 148,920 MiB.
 FRANK_ARCHON_TTM_MIN_MIB=150000
 FRANK_ARCHON_TTM_TARGET_MIB=204800
-# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
-# multi-flag values as one string; the serve script expands it unquoted on
-# purpose.
+# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
+# this unquoted on purpose, so keep multi-flag values as one string.
 FRANK_ARCHON_EXTRA_ARGS=
-# The .rad's minimum size in MiB (deployer gate; a truncated fetch is a
-# guaranteed long-timeout boot instead of a fast failure). The deployed
-# publisher container measures 116,322 MiB (121,969,901,568 B); the
-# in-house w4dn8 rollback measured 116,416 MiB. Floor sits just under both.
+# Deployer gate: a truncated fetch must be a fast refusal, not a long-timeout
+# boot. Floor sits under the live payload and under every rollback payload
+# (see the page's Eras table for what is on disk and what needs a re-download).
 FRANK_ARCHON_RAD_MIN_MIB=110000

scripts/frank-dragoon.env

@@ -1,209 +1,82 @@
 # frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
 # Strix Halo iGPU (gfx1151), host port 7770.
 #
-# RUNTIME (2026-10-17, Frank): halogen-flash-server 0.16.2 — the
-# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
-# digest-pinned below) serving Quat3rnion/halogen-qwen3.8-flash-next-v2-
-# abliterated — the NATIVE v2 conversion (2026-10-11) of orcarouter's pinned
-# BF16 Qwen3.8-Flash-Next, abliterated — as a podman container under a root
-# unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
-# 2026-10-06 Swift1.5-finetune shape; rollback = revert the swap commit and
-# redeploy: the swift15 payload stays on disk at
-# /home/dank/models/halogen-flashnext-v2/, the 27B-era payload beside it.
+# ONE OWNER: payload identity, shape, eras, laws and every measured number live
+# in serverAppConfigs-docs/LLM/frank-dragoon.md. This file carries the knobs
+# plus only the reason a reader editing THIS line cannot recover from the page.
+# Where the two disagree, this file wins on values; the boot ledger arbitrates
+# the memory figures.
 #
-# MEASURED A/B, both boots on this box, identical battery traffic (engine
-# tools with the service stopped, behavior probes against it, 2026-10-17):
-# NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls and the
-# red-PNG vision probe identical, needle hit at both depths of a 21.5k-token
-# haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s,
-# wikitext-2 PPL 3.968 vs 3.877 (the finetune's edge is gone: +2.3 %) and
-# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
-# base thinks ~1.9x longer per answer.
-#
-# The A/B ran at the card's ctx/pool of 262,144. Afterwards Frank lowered ctx to
-# 200,704 (RAM + the harness's compaction budget) and, on 2026-10-18, raised the
-# pool to 2 x ctx = 401,408 so BOTH slots can carry a full-length request at
-# once; the rates above are the ceiling-262,144 record and the production ledger
-# is the ctx-200,704 one.
-#
-# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
-# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
-# .hgn loads ONLY in the flash-server line (the old halogen engine is for
-# the 27B p1w4d file; the v2 quickstart pins flash-server). What the move
-# buys (card + live probe evidence): MTP spec decode (53-57 t/s greedy,
-# 3-10x serial), ~1,640 t/s prefill at 34k, ~72% fewer thinking tokens than
-# the censored 27B (905 mean), the ABLITERATED base (refusals should NOT
-# return), IMAGE INPUT (the v2 vision tower loads beside the checkpoint —
-# red-PNG probe verified on the card; the 27B engine ignored its encoder),
-# tool calls verified, and the native 262,144 context — served at 200,704 by
-# choice from 2026-10-17 (see SHAPE: the ceiling is a RAM decision, not a
-# capability limit).
-#
-# SHAPE — the knobs that are NOT free, with the arithmetic:
-#   HALOGEN_CTX=200704 is the PER-REQUEST ceiling and the per-conversation
-#   hard limit — a prompt over it is a 400 naming the limit, never a silent
-#   truncation. LOWERED from 262,144 on 2026-10-17 at Frank's direction (the
-#   checkpoint's native window is 262,144 and the engine serves anything
-#   below it): every position costs KV pool RAM and shrinks the prompt-cache
-#   entry (111 MiB at 262,144, ~85 MiB here). The 400-zone law is INVARIANT
-#   to the value — it says the harness window must sit ~45k UNDER the engine
-#   ceiling, because the harness clamps max_tokens to contextWindow − est −
-#   4096, its chars/4 est UNDER-counts the rendered prompt by ~3 % on
-#   code/agent traffic, and compaction only fires at 85 % (measured 2026-10-07
-#   at the old 200,192 ceiling: est 152,938, server 157,388, over by 354).
-#   Harness contract (models.yml): contextWindow 155648 (= ceiling − 45056,
-#   the gap that absorbs the estimate error), maxTokens 45056.
-#   Compaction fires at 85 % = 132,301 est ≈ 136k measured — the 180k target
-#   of the 262,144 era is unreachable at this ceiling: the window itself is
-#   155,648. Worst-case request: prompt + budget <= 155648 − 4096 + 3 %
-#   est-error ≈ 156.2k <= 200,704 — ~9.5× the 3 % error of margin (robust to a
-#   ~25 % under-count). Compaction rides on frank-zealot: input ≈ threshold −
-#   keepRecent(20k) ≈ 112k, inside its 200,704 slot.
-#   HALOGEN_KV_POOL_POSITIONS=401408 = SLOTS x CTX (2 x 200,704), the top of the
-#   legal band (the seam enforces ctx <= pool <= slots * ctx). Still ONE pool
-#   behind HALOGEN_KV_SLOTS=2 (ledger form: "2 slots over ONE <pool>-position KV
-#   pool, each request up to HALOGEN_CTX"), but sized so BOTH slots can hold a
-#   MAXIMUM-length request at the same time — the parity with frank-zealot's
-#   401,408-position pool that Frank ordered on 2026-10-18. The PER-REQUEST
-#   ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the
-#   checkpoint's native window is 262,144 and the engine refuses to go past it
-#   without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position
-#   (FLAGS.md) ⇒ the DEPLOYED 2026-10-18 boot RESERVES 11.0 GiB (`reserving the
-#   KV pool, 401408 positions (about 11.0 GiB)`), where pool == ctx (200,704)
-#   cost 5.5, 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6.
-#   The preflight formula bills 11.3 (block scratch + ~115 MiB/slot of O(1)
-#   state) — quote the RESERVATION. FLAGS.md law: "RAM the pool
-#   takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this
-#   much page cache, so the warm prompt-cache ratio is what to re-measure.
-#   HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
-#   drafter is NOT slot-locked on this engine (batching and speculation
-#   coexist — DFlash-era slots>1 law does NOT carry over; the card's own
-#   quickstart ships 2 slots WITH the head), so the second slot is
-#   concurrent capacity, not a speculation tax.
-#   Memory ledger against the box's 121.2 GiB unified (the n-gram table is
-#   mmap'd page cache beside the checkpoint, not a pinned charge — the boot
-#   ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
-#     weights 62.9 GiB locked (0.8 of them the vision tower) + KV 11.0 +
-#     working 5.7 (HALOGEN_MAX_TOK=8192) = 79.6 GiB engine-side (the boot
-#     ledger of the 401408-position deploy, 2026-10-18), with the 1.3 GiB
-#     prompt cache (12 entries x 111 MiB at ctx 200704) booked separately by
-#     the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host honest. Against
-#     the 121.2 GiB box that leaves ~20 GiB over the reserve (MemAvailable
-#     read 22 GiB right after the raise) for baseline co-tenancy
-#     (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot ledger is the
-#     referee — read it after deploy and re-size if the engine prints a
-#     clamp.
-#   HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
-#   turns the MTP head off for whole requests after low acceptance and
-#   HALVES decode as a result. Frank accepted the head always-on trade
-#   (temp 1.0 traffic costs acceptance; the card still ships the recipe).
-#
-# CLIENT CONTRACT deltas vs the 27B shape (harness side lives in
-# ~/.omp/agent/models.yml):
-#   * SERVER-SIDE SAMPLER EXISTS NOW: HALOGEN_TEMPERATURE/TOP_P/TOP_K set
-#     the card's thinking recipe (1.0 / 0.95 / 20) as engine defaults, so
-#     an absent temperature is no longer greedy. models.yml extraBody
-#     carries the same values — consistent, not load-bearing.
-#   * HALOGEN_MAX_TOK=8192 is only the DEFAULT for requests that omit
-#     max_tokens; the harness sends at most its maxTokens 45056. Requests
-#     over HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
-#     queue timeout are COUPLED (FLAGS.md table): 65536 at ~10 t/s serial
-#     worst case needs > 6550 s; 7200 honors it. Do not move one alone.
-#   * max_tokens INCLUDES thinking; a reasoning reply over a small budget
-#     returns empty content with finish_reason "length".
-#   * Prompt cache is ON by default (PROMPT_CACHE=2): a follow-up turn at
-#     100k context re-prefills ~2 s instead of ~88 s cold. No pin needed —
-#     unlike the 27B engine, auto-sizing fits this box's ledger.
-#   * Thinking lands in reasoning_content (same as the 27B engine); the
-#     v2 chat template rejects reasoning_effort outside xhigh|medium|low,
-#     so the models.yml fold (high->xhigh) still applies.
-#   * Vision: OpenAI image_url content parts only (the fleet's standard
-#     contract). >=256 px probe images or the model answers "video".
-#   * Concurrency at depth is tight now: the KV pool is ONE 200,704-position
-#     budget behind 2 slots — exactly one maximum-length request fills it. Two
-#     conversations coexist only while they stay under ~100k each. The
-#     PER-REQUEST ceiling moved WITH the pool (262144 -> 200704), so the harness
-#     contract moves too: contextWindow 155648 = ceiling − 45k, never higher.
-#   * response_format json_object/json_schema unsupported in this line
-#     (no constrained decoding) — unchanged from the 27B engine.
-#   * The engine's token protocol has no auth and binds loopback INSIDE
-#     the container; only the frontend port (8731) is published. Same as
-#     before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
-#     amdgpu.noretry=0 (check at deploy).
-
-# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
-# is what was qualified). 0.16.2 = the version the model card names as its
-# quickstart pin; RepoDigest captured at pull on 2026-10-06.
+# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
+# engine moves with the payload, never ahead of it.
+# Image digest-pinned (a tag pull silently re-points; the digest is what was
+# qualified — 0.16.2 is the version the model card names as its quickstart pin).
 FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
-# Container identity: kept from the 27B era (unit ExecStop names it).
+# Container identity kept from the 27B era: the unit's ExecStop names it.
 FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
-# Old quadlet-era container identity; the serve script clears stale names.
+# Predecessor identity the serve script clears if it is still holding the GPUs.
 FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
-# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
-# default); the engine port (8730) is never published.
+# host:container — the frontend listens on 8731 inside; the engine's own port
+# (8730) has no auth and is NEVER published.
 FRANK_DRAGOON_PUBLISH=7770:8731
 
 FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
-# The checkpoint: Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated
-# qwen38-flash-next-v2-abliterated.hgn — the NATIVE 2026-10-11 v2 conversion
-# of orcarouter's pinned BF16 Qwen3.8-Flash-Next, abliterated. NOT the writer
-# transplant this filename used to carry: 66,687,678,432 B where the swift15
-# r2 measured 66,687,678,464 B — same filename family, different artifact.
-# Hub LFS sha256 verified at fetch by scripts/download-frank-dragoon-v2abl.sh.
+# Same filename family, DIFFERENT artifact: this is the native v2 conversion,
+# 32 bytes smaller than the writer transplant that used to carry the name.
+# Never infer the payload from the filename — this size and sha256 are the gate.
 FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
 FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
 FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
-# The n-gram lookup table: REQUIRED (the engine reads it beside the
-# checkpoint; the card's quickstart names it explicitly). From
-# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
-# The card does NOT establish this stock table as matched to the NATIVE
-# conversion and no source-matched table is published, so the engine's own
-# tools are the referee: NIAH 18/18 and PPL 3.968 on it (2026-10-17). It is
-# HARDLINKED from the swift15 payload dir (a hardlink is a regular file; the
-# serve gate refuses symlinks), i.e. byte-identical to the file that era
-# served.
+# The n-gram table is REQUIRED (the engine reads it beside the checkpoint) and
+# disk-backed: mmap'd page cache, never pinned. It is a stock peonist companion
+# hardlinked from the previous payload dir — a hardlink is a regular file, and
+# the serve gate refuses symlinks — with the engine's own NIAH/PPL verdicts on
+# the page as the referee.
 FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
 FRANK_DRAGOON_NGRAM_BYTES=51200246144
 FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
-# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
-# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
+# Vision tower: optional on the engine, ENABLED here (image input).
 FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
 FRANK_DRAGOON_VISION_BYTES=897916416
 FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
-# The tokenizer is a FLAT DIRECTORY, $FRANK_DRAGOON_MODEL_DIR/tokenizer,
-# mounted whole at /tokenizer. The two repos ship the same one; fetched
-# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
-# the 27B-era copy; the rest are plain git blobs — size-checked only). The
-# serve gate and the deployer refuse a missing file or ANY symlink: an HF
-# cache snapshot symlinks each file into blobs/, and those links dangle
+# The tokenizer is a FLAT DIRECTORY ($FRANK_DRAGOON_MODEL_DIR/tokenizer) mounted
+# whole. The serve gate and the deployer refuse a missing file or ANY symlink:
+# an HF cache snapshot symlinks each file into blobs/ and those links dangle
 # inside the container mount.
 FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
 
 # --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
 # stays at the image default, which is what was qualified) ------------------
-# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
-# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
-# model_ids for the :7770 backend MUST name this value.
+# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
 HALOGEN_MODEL_ID=frank-dragoon
+# PER-REQUEST ceiling: a longer prompt is a 400 naming the limit, never a
+# truncation. The harness window must sit ~45k UNDER this (the 400-zone law on
+# the page) — a window equal to the ceiling makes every long request a 400.
 HALOGEN_CTX=200704
+# ONE pool behind both slots, sized so BOTH can carry a maximum-length request
+# at once. Pool positions buy concurrency; the per-request window never moves
+# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it), and
+# RAM the pool takes is RAM the n-gram table's page cache loses.
 HALOGEN_KV_POOL_POSITIONS=401408
+# Two concurrent conversations: the MTP drafter is not slot-locked on this
+# engine, so the second slot is capacity, not a speculation tax.
 HALOGEN_KV_SLOTS=2
 HALOGEN_HOST_RESERVE_GIB=20
-# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
-# depth 2 is the documented "best all-round" default. HALOGEN_DRAFTER /
-# HALOGEN_DFLASH_* from the 27B engine DO NOT EXIST here.
+# The MTP head is INSIDE the checkpoint — do NOT set HALOGEN_MTP_HEAD.
+# 0 is mandatory: the adaptive default disarms the head for whole requests after
+# low acceptance and halves decode. Depth 2 is the card's best-all-round default.
 HALOGEN_SPEC_ADAPT=0
-# Coupled pair (FLAGS.md table): the cap bounds how long ONE request holds
-# the GPU, the timeout bounds how long the NEXT client waits. Do not move
-# one alone (see the client-contract note above).
+# Coupled trio: MAX_TOK is only the default for requests that omit max_tokens;
+# the cap bounds how long ONE request holds the GPU and the queue timeout bounds
+# how long the NEXT client waits. Move all three together or none.
 HALOGEN_MAX_TOK=8192
 HALOGEN_MAX_TOKENS_CAP=65536
 HALOGEN_QUEUE_TIMEOUT=7200
-# Server-side sampler = the card's thinking recipe (the engine now HAS one;
-# requests may still override per-call). Matches models.yml extraBody.
+# The engine has a server-side sampler (an absent temperature is NOT greedy
+# here); models.yml extraBody carries the same recipe.
 HALOGEN_TEMPERATURE=1.0
 HALOGEN_TOP_P=0.95
 HALOGEN_TOP_K=20
-# Card's belt-and-braces: the engine pins weights by default
-# (HALOGEN_FLASH_PIN_TRUNK=1; only the retired w4b build honored 0).
+# Weights stay pinned: the engine pins by default and only the retired w4b build
+# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
 HALOGEN_WEIGHTS_LOCK=1

scripts/frank-tts.env

@@ -57,6 +57,16 @@ TTS_BACKEND=omni-http
 TTS_UPSTREAM=http://frank-tts-engine:8880
 TTS_UPSTREAM_MODEL=voxcpm2
 
+# THE DEFAULT VOICE — what a request that names NO voice speaks. The omp harness
+# has no setting that feeds a voice to its speech device (`voice_id` is a
+# per-call argument; `speech.voice` belongs to the TUI's Kokoro speak-aloud and
+# never reaches an HTTP request), so a harness voiceover arrives voice-less and
+# would otherwise be a 400 for every caller that does not name one. An UNKNOWN
+# name still 400s with the list that exists — this covers the absent case only —
+# and the journal marks such a line `(default)` so an unrequested voice never
+# reads as a requested one. Any pack directory in /home/dank/voices is legal.
+DEFAULT_VOICE=narrator1
+
 # CLI-backend knobs, inert while TTS_BACKEND=omni-http. Empty means "not
 # configured": the endpoint answers a 500 that says so instead of running
 # `nice env -m ...` with an empty binary name.

scripts/frank-zealot.env

@@ -1,141 +1,72 @@
-# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) serving
-# prism-ml/Ternary-Bonsai-2-9B-gguf at PQ2_0 **with the MTP head in the file**
-# on the RTX 3090, :7771.
+# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) on the RTX 3090,
+# :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file.
 #
-# 2026-10-18 (Frank): swap from the Bonsai-2-**27B** PQ2_0 to the 9B MTP export
-# at THREE 262,144-token conversations (Frank: "if you can possibly fit a third
-# context, do it"). The 9B bills HALF the KV of the 27B per token, which is what
-# funds 3 x the native window on one 24,123 MiB card. Speculative decoding ships
-# OFF — measured, not assumed; see the MTP law below, which is the whole story
-# and the reason the publisher's "depth 2 works best" line does not transfer.
-#
-# THE RUNTIME IS A LOCK, NOT A PREFERENCE. The file is general.file_type 141
-# carrying custom GGML type 142 tensors plus prism.hadamard.* metadata
-# (normalized-Sylvester-Walsh transform); stock llama.cpp refuses type 142 at
-# load and a loader that ignored the Hadamard metadata would dequant WRONG. On
-# top of that lock, MTP beside a quantized V cache needs fork build >= b10776,
-# so the vendor prebuilt moved from 10754 (commit 2459f68b5, the 27B era) to
-# 10777 (commit e3de71522). The 10754 tree stays on disk as the rollback
-# runtime for the 27B payload.
+# ONE OWNER: shape, payload identity, eras, laws and every measured number
+# live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the
+# knobs and only the reason a reader editing THIS line cannot recover
+# elsewhere. Where the two disagree, this file wins on values.
 LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server
-# 2,675,410,720 B, Hub LFS sha256 c79fcdfaf6b7… (verified at fetch by
-# scripts/download-frank-zealot-bonsai9.sh). This IS the MTP build: the draft
-# head ships INSIDE the checkpoint, so MTP availability is a property of the
-# FILE, never of a flag. The plain PQ2_0 sibling could not run it at any flag
-# value; loading this file instead of that one is the only change that matters.
+# The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.*
+# metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard
+# metadata dequantizes wrong, silently.
 MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf
-# 624,229,664 B, Hub LFS sha256 e0cb6f69… — Q8_0 IS MANDATORY, not a preference:
-# the publisher's card says the BF16 mmproj "can crash llama-server when several
-# image requests run at once; Q8_0 is not affected", and this endpoint runs
-# --parallel 3, i.e. exactly that shape. The projector's worst case lands AFTER
-# the engine's own fit check, so it is never inside a projection number.
+# Q8_0 IS MANDATORY: the publisher states the BF16 tower can crash llama-server
+# under concurrent image requests, which is exactly --parallel 3.
 MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
-# Base alignment: Bonsai-2 is the censored BASE lineage. The 27B CRACK re-export
-# (/home/dank/models/Bonsai2-CRACK/) is byte-identical geometry to the 27B it
-# matched, NOT to this 9B file — if a refusal ever surfaces there is no
-# one-line swap here; the 9B dealigned export would have to be fetched first.
-#
-# Served model id — the harness/family name. The SERVICE, unit, scripts and
-# this env file keep the frank-zealot name; only the alias on the wire is the
-# model id. Consumers that key on the served id: OpenWebUI (model row,
-# access_grant, api_configs model_ids) and the harness models.yml.
+# Served id — the harness/family name, not a fact about the payload.
+# Id-keyed consumers: OpenWebUI (model row, access_grant, api_configs), models.yml.
 ALIAS=frank-zealot
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7771
-# CTX_SIZE is the POOL llama.cpp divides across slots: 786,432 / 3 = 262,144 per
-# conversation — each slot at the model's TRAINED window, which is the point of
-# the swap (the 27B era had to cut each conversation to 200,704 to fit two).
-# MUST stay divisible by PARALLEL (the seam enforces it) and the harness
-# contextWindow quotes the PER-SLOT number — a longer prompt is answered with a
-# 400, never truncated. The ceiling cannot go past 3 x 262,144: that is the
-# native wall (no YaRN in this path) and the arithmetic agrees — a 4th slot
-# would push each below the compaction contract.
-CTX_SIZE=786432
+# CTX_SIZE is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
+# conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
+# models.yml contextWindow quotes the PER-SLOT number — a longer prompt is a
+# 400, never a truncation. The pool is bought, not granted: K+V at 25,088
+# B/token is 14,406.00 MiB here and the MTP draft context bills its own f16 KV
+# over the WHOLE pool (4,096 B/position ⇒ 2,352.00 MiB), so the old 786,432
+# pool cannot carry a drafter at any KV dtype — 18,816 + 3,072 of cache alone
+# against the 24,123 MiB the driver leaves addressable.
+CTX_SIZE=602112
 PARALLEL=3
-BATCH_SIZE=4096
-# Ubatch is a prefill knob (decode never moves) with NO interior optimum on this
-# CUDA card up to 2048 — 2048 is the measured optimum (+4.6 % prefill). It has a
-# second job here: it sets the compute-buffer size, and the compute buffers are
-# what decide whether MTP fits (see the MTP law). 1024 is the documented MTP arm.
-UBATCH_SIZE=2048
-# KV law (header + boot ledger, both confirmed): qwen35 hybrid, 32 layers with
-# qwen35.full_attention_interval = 4 ⇒ 8 full-attention + 24 Gated DeltaNet;
-# 4 KV heads × 256 dim = 8,192 elements per token per cache. q8_0 bills 8,704 B
-# per cache ⇒ 17,408 B/token = 17 KiB, so the 786,432 pool is 13,056.00 MiB —
-# the ledger said exactly that. HALF the 27B's 34,816 B/token, which is the
-# whole reason three native windows fit. f16 would bill 26,112 MiB at this pool
-# (against a 24,123 MiB card) and dies at cudaMalloc; the 24 DeltaNet layers
-# bill a fixed recurrent state per SLOT instead of per token — the PRODUCTION
-# ledger prints "RS buffer size = 150.75 MiB (3 cells, 32 layers, 3 seqs)"
-# (~50 MiB a cell), while the SPEC-ARMED arms print 452.25 MiB, because MTP
-# depth 2 bills three seqs per slot (9 cells) — which is why pool arithmetic
-# alone under-prices the shape.
-CACHE_TYPE_K=q8_0
+# The logical batch is VRAM- and throughput-neutral at a fixed ubatch (the law
+# on [[frank-zealot]]); 1024×6 keeps it a clean multiple of the ubatch.
+BATCH_SIZE=6144
+# Ubatch is a prefill knob (decode never moves with it) and it sets the
+# compute-buffer size, which decides whether a drafter fits. 1024 is the rung
+# every measured MTP arm here ran on (213.5 t/s against 109.2 at 2048). Its
+# VRAM credit is SHAPE-specific: the 786,432-pool arm census showed ~1,306 MiB
+# back, this boot measured 2,089.88 MiB of target compute against 2,496.13 at
+# ubatch 2048 — 406 MiB back, with the split moving (CUDA0 +257, CUDA_Host
+# −663). The drafter is funded by CACHE_TYPE_V below, not by this knob.
+UBATCH_SIZE=1024
+# K exact, V in the 8-bit class: 16,384 + 8,704 = 25,088 B/token (8,192
+# elements per token per cache × 2 B for f16, × 1.0625 for q8_0 — there is no
+# true fp8 KV dtype in this build). That 7,680 B/token saved against exact V is
+# what funds BOTH the third conversation and the drafter. Quantized V is
+# illegal without --flash-attn, which the serve script pins. Rate law and the
+# arithmetic: [[frank-zealot]] → Laws.
+CACHE_TYPE_K=f16
 CACHE_TYPE_V=q8_0
-# SPECULATIVE DECODING: OFF, MEASURED. The head is in the file and the drafter
-# is GOOD (acceptance 0.798/0.617 per position, mean accepted length 2.41) — it
-# still loses, because the draft context bills its OWN f16 KV over the WHOLE
-# pool: 4,096 B/position × 786,432 = 3,072.00 MiB, plus its compute buffers,
-# 4,830 MiB all told. Booted arms, same card, same traffic (2026-10-18):
-#   3 × 262,144  spec OFF  ub 2048 → 17,876 used / 6,249 free, decode 167.2 t/s
-#   3 × 262,144  spec ON   ub 2048 → 22,706 used / 1,419 free, decode 109.2 t/s
-#   3 × 262,144  spec ON   ub 1024 → 21,400 used / 2,725 free, decode 213.5 t/s
-#   2 × 262,144  spec ON   ub 2048 → 17,178 used / 6,947 free, decode 213.2 t/s
-#   1 × 262,144  spec ON   ub 2048 → 11,652 used / 12,473 free, decode 214.2 t/s
-# i.e. MTP is worth +28 % decode whenever the card has room, and at the 3-slot
-# pool it costs 1.3 GiB of compute buffers that ubatch 2048 refuses to give up —
-# the squeeze, not the drafter, is what halves decode. The arm that WOULD run
-# MTP at three windows is SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024 (decode 213.5,
-# 3-stream 250.8 aggregate) and pays prefill −17 % and 2.7 GiB of headroom on a
-# card whose projector worst-case lands after the fit check. This endpoint's
-# load is the fleet's COMPACTION HUB (prefill-bound, ~170k in) plus 2-3
-# concurrent subagents, so the deployed arm takes the +17 % prefill
-# (2,877 vs 2,395 t/s at 106,418 tokens) and the +12 % three-stream aggregate
-# (280.2 vs 250.8) over single-stream speed, and keeps 6.2 GiB free.
-# SPEC_TYPE=none strips the WHOLE spec set (--spec-type, every --spec-draft-*):
-# flags move as a set or the arm measures a crash instead of an absence.
-SPEC_TYPE=none
+# Speculation ON (Frank, 2026-10-19): the head is in the file
+# (qwen35.nextn_predict_layers = 1, blk.32.nextn.*) and rides the MAIN model's
+# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
+# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
+# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
+# beside MTP needs fork build >= b10776; LLAMA_BIN is 10777.
+# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
+# every --spec-draft-*); half a set has segfaulted at load.
+SPEC_TYPE=draft-mtp
 SPEC_DRAFT_N_MAX=2
 SPEC_DRAFT_N_MIN=0
 SPEC_DRAFT_P_MIN=0
-# The card's sampler recipe lives in the checkpoint (general.sampling.* = temp
-# 1.0 / top_p 0.95 / top_k 20 / min_p 0 / repeat_penalty 1.0 — confirmed in the
-# engine's own /props defaults) and llama.cpp reads it itself. Presence penalty
-# is the ONE recipe value with no GGUF metadata key, so it must reach the argv.
-# PRESENCE_PENALTY="" is the documented opt-out.
+# The sampler recipe lives in the checkpoint (general.sampling.*) and
+# llama.cpp reads it itself. Presence penalty is the one recipe value with no
+# GGUF metadata key, so it must reach argv. "" is the documented opt-out.
 PRESENCE_PENALTY=1.0
 NGL=99
-# Boot ledger (2026-10-18 PRODUCTION boot under the unit, verbosity 4):
-#   17,922 MiB used / 6,203 MiB free of 24,123 (nvidia-smi settled)
-#   CUDA0 model 2,036.71 MiB + CPU_Mapped 257.66 MiB
-#   CUDA0 KV 13,056.00 MiB (q8_0 K + q8_0 V, 786,432 positions)
-#   CUDA0 RS 150.75 MiB (3 cells, 32 layers, 3 seqs)
-#   compute 1,409.02 MiB CUDA + 1,088.87 MiB CUDA_Host (+2.84 MiB output buf)
-#   mmproj worst-case estimate 868.32 MiB (248.10 CUDA + 24.93 CPU compute
-#   meta) — charged AFTER the engine's own fit check, as always
-#   no clamp line, no cudaMalloc error
-# Measured on this boot (/completion timings, greedy, ignore_eos, 512 tokens):
-# 168.7 t/s decode (three runs 168.4-168.7) and ~300 t/s aggregate across
-# THREE concurrent streams (102.8 each). The trial arm's record (167.2 decode,
-# 280.2 aggregate, 2,877 t/s prefill at 106,418) stands; the 27B it replaces
-# gave 76.6 / 109.4 / 691.7 at 166k. The FIRST request after a boot is a
-# warmup artifact (37.7 t/s class) and a natural generation that stops early
-# makes a wall-clock aggregate read 4x low - measure with ignore_eos and read
-# the engine's own timings.
-# Fallback rung if the margin is ever eaten (desktop, co-tenant): q4_0 KV on the
-# 27B era measured 16,392 used / 7,733 free at 2 × 200,704; the same drop here
-# frees ~6.5 GiB at three windows. UBATCH_SIZE=1024 is the cheaper lever
-# (−1,306 MiB measured on this shape).
-# Log verbosity threshold (llama.cpp -lv). 4 = debug: KV layout, buffer sizes
-# and graph capture land in the journal, which is what the VRAM-fit forensics
-# read. Levels above 4 flood the journal.
+# 4 = debug: KV layout, buffer sizes and graph capture land in the journal, and
+# that is what VRAM forensics read. Above 4 floods the journal.
 VERBOSITY=4
-
-# Serve /metrics. llama.cpp answers 501 without --metrics, and Dankclaw's
-# Prometheus scrapes /metrics on every fleet endpoint — a 501 reads as DOWN, so
-# this service has held the critical LlmEndpointDown alert for as long as the
-# metric has history (verified 2026-10-18: the oldest retained sample of
-# up{job="frank-zealot"}, 2026-10-11, is already 0; scrapeUrl
-# http://host.containers.internal:7771/metrics, lastError "server returned HTTP
-# status 501 Not Implemented"). 0 opts out and re-breaks the scrape.
+# Without --metrics llama.cpp answers 501 on /metrics, which Prometheus reads
+# as DOWN while /health stays 200. 0 opts out and re-breaks the scrape.
 METRICS=1

scripts/frank-zealot.service

@@ -1,5 +1,5 @@
 [Unit]
-Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF (qwen35 hybrid, 3x262144 slots over one 786432 pool, q8_0 K+V, MTP off, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
+Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF (qwen35 hybrid, 2x262144 slots over one 524288 pool, unquantized f16 K+V, MTP off, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
 After=network-online.target
 Wants=network-online.target
 

scripts/serve-frank-archon.sh

@@ -15,12 +15,13 @@
 # log states every budget before anything is allocated (weights vs KV
 # split, the expert host pool's coverage of the worst case, the PLE pin,
 # the P2P answer per pair) — read it after every resize.
-# Deployed shape (2026-10-13): THREE decode slots x 200,000 tokens, fp8
-# (quantized) KV in a 5,120 MiB pool (contract 3 x 200,000 + 1.3 %) and a
-# 25,600 MiB resident expert slab — 30,720 MiB of the 30,969 MiB the engine's
-# own budget table makes claimable, spent on the slab because prefill is
-# slab-limited. 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache host
-# 8 GiB + disk 64 GiB, PLE pinned in RAM.
+# Deployed shape (2026-10-19, radiance 1.8.1): THREE decode slots x 200,000
+# tokens, fp8 KV, 44 GiB/rank pinned expert pool, MTP depth 2, prefix cache in
+# pinned host RAM, PLE pinned in RAM. The per-card split between the expert slab
+# and the KV pool is NO LONGER an operator number — 1.3.0 removed the VRAM pair
+# and the budget resolver derives it from headroom, the cache bound and the host
+# pool, so the slab size and the pool's token count are LEDGER lines: after every
+# boot read them out of the log and check the pool still backs 3 x 200,000.
 #
 # The root unit (scripts/frank-archon.service, User=dank,
 # LimitMEMLOCK=infinity) grants the pin budget: 2 x 44 GiB expert pools +
@@ -56,13 +57,11 @@ set +a
 : "${FRANK_ARCHON_PLACEMENT:?FRANK_ARCHON_PLACEMENT must be set}"
 : "${FRANK_ARCHON_HOST_POOL_MIB:?FRANK_ARCHON_HOST_POOL_MIB must be set}"
 : "${FRANK_ARCHON_GPU_HEADROOM_MIB:?FRANK_ARCHON_GPU_HEADROOM_MIB must be set}"
-: "${FRANK_ARCHON_VRAM_WEIGHTS_MIB:?FRANK_ARCHON_VRAM_WEIGHTS_MIB must be set}"
-: "${FRANK_ARCHON_VRAM_KV_MIB:?FRANK_ARCHON_VRAM_KV_MIB must be set}"
+: "${FRANK_ARCHON_BUSY_PREFILL_CHUNK:?FRANK_ARCHON_BUSY_PREFILL_CHUNK must be set (auto|off|N)}"
 : "${FRANK_ARCHON_KV_DTYPE:?FRANK_ARCHON_KV_DTYPE must be set}"
 : "${FRANK_ARCHON_SPEC_TOKENS:?FRANK_ARCHON_SPEC_TOKENS must be set}"
 : "${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
-: "${FRANK_ARCHON_PREFIX_HOST_MIB:?FRANK_ARCHON_PREFIX_HOST_MIB must be set}"
-: "${FRANK_ARCHON_PREFIX_DISK_MIB:?FRANK_ARCHON_PREFIX_DISK_MIB must be set}"
+: "${FRANK_ARCHON_PREFIX_CACHE_MIB:?FRANK_ARCHON_PREFIX_CACHE_MIB must be set}"
 : "${FRANK_ARCHON_NGRAM_PLACEMENT:?FRANK_ARCHON_NGRAM_PLACEMENT must be set}"
 : "${FRANK_ARCHON_TTM_MIN_MIB:?FRANK_ARCHON_TTM_MIN_MIB must be set}"
 : "${FRANK_ARCHON_TTM_TARGET_MIB:?FRANK_ARCHON_TTM_TARGET_MIB must be set}"
@@ -103,19 +102,12 @@ if [ ! -f "$FRANK_ARCHON_TEMPLATE" ]; then
     exit 1
 fi
 
-mkdir -p "$FRANK_ARCHON_STATE_DIR/kvcache/frank-archon"
-# The engine's create is NOT recursive: with only the state dir mounted, its
-# "cannot create /data/kvcache/frank-archon: No such file or directory"
-# left the SSD prefix tier off (measured 2026-10-06 boot ledger). Pre-create
-# the full path on the host so the tier opens on first boot.
-
 mkdir -p "$FRANK_ARCHON_STATE_DIR/tune"
 # rad-tune's kernel-selection cache (see the runbook's "Tuning levers"):
 # the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune at boot;
 # $RADIANCE_HOME defaults to /opt/radiance, which lives in the image, so
 # the cache must ride a host mount or it dies with the --rm container
-# that wrote it. The dir is pre-created for the same reason as kvcache
-# (the engine's create is not recursive).
+# that wrote it. The engine's create is not recursive, so the dir is pre-created.
 
 if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
     echo "ERROR: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
@@ -148,6 +140,15 @@ done
 LOG=/home/dank/logs/frank_archon.log
 mkdir -p "$(dirname "$LOG")"
 
+# Interleave is opt-in: 0 is the engine's default and passing it anyway would put
+# a flag in argv that changes nothing but the record. The sharing cap is always
+# explicit because `auto` vs `off` is the difference between decode waiting one
+# chunk and waiting the whole step.
+PREFILL_ARGS=()
+if [ "${FRANK_ARCHON_PREFILL_INTERLEAVE:-0}" -gt 0 ]; then
+    PREFILL_ARGS=(--prefill-interleave "$FRANK_ARCHON_PREFILL_INTERLEAVE")
+fi
+
 # Publisher-vetted container contract (docs/DOCKER.md, compose/common.yaml):
 # --network host (the server binds the host's addresses directly, no NAT),
 # --init (the engine releases the cards on SIGTERM), seccomp=unconfined
@@ -156,8 +157,9 @@ mkdir -p "$(dirname "$LOG")"
 # makes for host memory), memlock unbounded (the pinned ledger — legal
 # because the root unit grants LimitMEMLOCK=infinity to this dank process),
 # /dev/kfd + /dev/dri (render/video groups are moot for a root container).
-# The template rides a single-file mount; the .rad dir is ro; /data holds
-# the prefix-cache disk tier.
+# The template rides a single-file mount and the .rad dir is ro. The prefix cache
+# has no disk tier any more, so the only host state this container writes is the
+# rad-tune cache under /opt/radiance/share/radiance/tune.
 podman run -d --name "$FRANK_ARCHON_CONTAINER" \
   --network host \
   --init \
@@ -165,7 +167,6 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
   --ulimit memlock=-1:-1 \
   --device /dev/kfd --device /dev/dri \
   -v "$FRANK_ARCHON_MODEL_DIR":/models:ro \
-  -v "$FRANK_ARCHON_STATE_DIR":/data \
   -v "$FRANK_ARCHON_STATE_DIR/tune":/opt/radiance/share/radiance/tune \
   -v "$FRANK_ARCHON_TEMPLATE":/templates/template.jinja:ro \
   "$FRANK_ARCHON_IMAGE" \
@@ -179,15 +180,13 @@ podman run -d --name "$FRANK_ARCHON_CONTAINER" \
   --placement "$FRANK_ARCHON_PLACEMENT" \
   --host-pool-mib "$FRANK_ARCHON_HOST_POOL_MIB" \
   --gpu-headroom-mib "$FRANK_ARCHON_GPU_HEADROOM_MIB" \
-  --vram-weights-mib "$FRANK_ARCHON_VRAM_WEIGHTS_MIB" \
-  --vram-kv-mib "$FRANK_ARCHON_VRAM_KV_MIB" \
   --kv-cache-dtype "$FRANK_ARCHON_KV_DTYPE" \
-  --prefix-cache-host-mib "$FRANK_ARCHON_PREFIX_HOST_MIB" \
-  --prefix-cache-dir /data/kvcache/frank-archon \
-  --prefix-cache-disk-mib "$FRANK_ARCHON_PREFIX_DISK_MIB" \
+  --prefix-cache-mib "$FRANK_ARCHON_PREFIX_CACHE_MIB" \
   --ngram-placement "$FRANK_ARCHON_NGRAM_PLACEMENT" \
   --num-speculative-tokens "$FRANK_ARCHON_SPEC_TOKENS" \
   --max-num-batched-tokens "$FRANK_ARCHON_MAX_BATCHED_TOKENS" \
+  --busy-prefill-chunk "$FRANK_ARCHON_BUSY_PREFILL_CHUNK" \
+  "${PREFILL_ARGS[@]}" \
   ${FRANK_ARCHON_EXTRA_ARGS:+$FRANK_ARCHON_EXTRA_ARGS} \
   > "$LOG" 2>&1
 

scripts/serve-frank-zealot.sh

@@ -3,33 +3,47 @@
 # in the file) as frank-zealot on Dankclaw's RTX 3090 via llama.cpp (PrismML
 # vendor build 10777, CUDA sm_86).
 #
-# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export at THREE
-# 262,144-token conversations (--ctx-size 786432 --parallel 3), q8_0 K + q8_0 V.
-# The 9B bills 17,408 B/token against the 27B's 34,816 (8 full-attention layers
-# instead of 16), and that halved rate is what funds three NATIVE windows where
-# the 27B could only afford two cut to 200,704. Verbosity 4 ships so the
-# VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in the
-# journal.
+# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same
+# evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over
+# an UNQUANTIZED f16 KV cache.
+#
+# 2026-10-19 (Frank): THREE 200,704-token conversations over a 602,112 pool
+# (--ctx-size 602112 --parallel 3), K left at the engine's own default (f16),
+# V dropped to q8_0, batch 1024×6 / ubatch 1024, and the MTP drafter ARMED
+# (--spec-type draft-mtp). The 9B bills 8,192 elements per token per cache (8
+# full-attention layers instead of the 27B's 16), so f16 K is 16,384 B/token and
+# q8_0 V is 8,704: 25,088 B/token = 14,406.00 MiB at the 602,112 pool. That
+# 7,680 B/token against exact V is exactly what funds the third conversation
+# AND a draft context that bills f16 KV over the whole pool. Verbosity 4 ships
+# so the VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in
+# the journal.
 #
 # Architecture (engine ledger, not the marketing page): qwen35 hybrid, 32
 # layers with qwen35.full_attention_interval = 4 ⇒ 8 full-attention + 24 Gated
 # DeltaNet, 4 KV heads × 256 dim, native 262,144. The KV pool scales with the 8
-# full layers (13,056.00 MiB at the 786,432 pool, ledger-exact); the DeltaNet
-# layers cost a fixed recurrent state per SLOT — 452.25 MiB for 3 cells, ~151
-# MiB each. The runtime is a hard lock, not a preference: custom GGML type 142
-# plus prism.hadamard.* metadata, which only the vendor build registers, and
-# MTP beside quantized V needs build ≥ b10776 — so the prebuilt moved 10754 →
-# 10777 with the payload. Vision comes from the publisher's Q8_0 projector (the
-# BF16 one can crash the server on concurrent images, and this endpoint runs
-# three slots), so --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is
-# the documented text-only opt-out.
+# full layers (14,406.00 MiB of f16 K + q8_0 V at the 602,112 pool); the
+# DeltaNet layers cost a fixed recurrent state per SLOT — ~50.25 MiB a cell, so
+# 150.75 MiB for three, and an armed depth-2 drafter triples that again. The
+# runtime is a hard lock, not a preference: custom GGML type 142 plus
+# prism.hadamard.* metadata, which only the vendor build registers — and the
+# b10776 gate that moved the prebuilt 10754 → 10777 (MTP beside a quantized V)
+# is LOAD-BEARING again on this shape, not moot as it was under exact KV.
+# Vision comes from the publisher's Q8_0 projector (the BF16 one can crash the
+# server on concurrent images, and this endpoint runs three slots), so --mmproj is
+# what gives this endpoint eyes; MMPROJ_FILE="" is the documented text-only
+# opt-out.
 #
-# Speculative decoding ships OFF, measured: the head loads and accepts well
-# (0.798/0.617 per position, mean length 2.41) but the draft context bills its
-# own f16 KV over the WHOLE pool — 4,096 B/position × 786,432 = 3,072 MiB plus
-# buffers — and at ubatch 2048 that squeeze HALVES decode (167.2 → 109.2 t/s).
-# frank-zealot.env carries the six-arm table and the arm that turns it back on
-# (--ubatch-size 1024, decode 213.5 t/s) with its price.
+# Speculative decoding ships ON (Frank, 2026-10-19). The head is IN the file and
+# accepts well (0.798/0.617 per position, mean length 2.41, measured 2026-10-18);
+# what made it lose that evening was the bill, not the drafts — the draft context
+# is built against the target's WHOLE pool, so blk.32 adds 4,096 B/position
+# (2,352.00 MiB at 602,112) plus 480.97 MiB of its own compute buffers on the
+# card, and it triples the recurrent state per slot (452.25 MiB for three cells
+# against 50.25 a cell with the drafter off). CACHE_TYPE_V=q8_0 is what pays
+# that: 7,680 B/token cheaper than exact V, 4,410 MiB at this pool. The ubatch
+# drop to 1024 contributes ~406 MiB here, NOT the ~1,306 the 786,432-pool census
+# showed — that figure does not transfer between shapes. SPEC_TYPE=none is
+# the rollback and strips the whole flag set.
 #
 # Sampling: the checkpoint carries the publisher's thinking recipe as GGUF
 # metadata (general.sampling.* = temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
@@ -56,43 +70,46 @@ fi
 : "${ALIAS:=frank-zealot}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-# ctx-size is the POOL llama.cpp divides across slots: 786,432 / 3 = 262,144
-# per conversation — every slot at the model's trained window. The harness
-# contextWindow MUST quote the PER-SLOT number — a longer prompt is answered
-# with a 400, never truncated.
-: "${CTX_SIZE:=786432}"
+# ctx-size is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
+# conversation. The harness contextWindow MUST quote the PER-SLOT number — a
+# longer prompt is answered with a 400, never truncated. The pool is priced by
+# the KV bill, not by the trained window: at 25,088 B/token (f16 K + q8_0 V) the
+# 602,112 pool costs 14,406.00 MiB and the armed drafter adds its own f16 KV over
+# the whole pool (2,352.00 MiB). Exact V at this pool (18,816 MiB) or the old
+# 786,432 pool at any dtype cannot boot inside the 24,123 MiB the driver leaves
+# addressable.
+: "${CTX_SIZE:=602112}"
 : "${PARALLEL:=3}"
 # Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
-# 2048 is the measured CUDA optimum (MiMo era, same arch/shape: +4.6 % prefill
-# at 8.9k depth, decode flat, 616 MiB). The logical batch is VRAM- and
-# throughput-neutral at a fixed ubatch. It ALSO sets the compute-buffer size,
-# which is the term that decides whether an MTP draft context fits beside a
-# 786,432-position pool: 1024 hands back 1,306 MiB and is the arm that runs the
-# drafter at three slots (frank-zealot.env carries the table).
-: "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=2048}"
-# Quantized KV: q8_0 K + q8_0 V — the only pair that funds 3 × 262,144 on this
-# card. 17 KiB/token at this geometry (8 full-attention layers × 4 KV heads ×
-# 256 dim); 13,056.00 MiB at the 786,432 pool, ledger-exact. f16 would bill
-# 26,112 MiB against a 24,123 MiB card and dies at cudaMalloc; q4_0 is the
-# fallback rung. Explicit either way so the contract is visible in the argv,
-# and --flash-attn stays required for quantized V (no true fp8 KV dtype exists
-# in this build).
-: "${CACHE_TYPE_K:=q8_0}"
+# The logical batch is VRAM- and throughput-neutral at a fixed ubatch, so 6144
+# (1024×6) is free; the ubatch is what sets the compute-buffer size, and 1024 is
+# the rung every measured MTP arm here ran on (213.5 t/s at 1024 against 109.2
+# at 2048) — worth ~406 MiB at THIS shape, not the ~1,306 the old pool showed.
+: "${BATCH_SIZE:=6144}"
+: "${UBATCH_SIZE:=1024}"
+# Asymmetric KV: f16 K (the engine's own default, spelled out so the contract is
+# visible in argv) + q8_0 V. 8,192 elements per token per cache (8 full-attention
+# layers × 4 KV heads × 256 dim) ⇒ 16,384 B/token for K and 8,704 for V (q8_0 is
+# 1.0625 B/element — llama.cpp has no true fp8 KV dtype), 25,088 B/token and
+# 14,406.00 MiB at the 602,112 pool. That is the halfway door between exact KV
+# (32,768 B/token, which costs the third conversation and the drafter both) and
+# the fully quantized pair (17,408 B/token, which costs retrieval fidelity in the
+# keys); --flash-attn stays pinned because quantized V is illegal without it.
+: "${CACHE_TYPE_K:=f16}"
 : "${CACHE_TYPE_V:=q8_0}"
-# Speculative decoding defaults OFF, measured on this card and shape: the head
-# IS in the checkpoint (qwen35.nextn_predict_layers = 1 — MTP is a property of
-# the FILE, so the plain PQ2_0 sibling could not run it at any flag value), it
-# loads, and it accepts well (mean accepted length 2.41) — but the draft context
-# bills its own f16 KV over the WHOLE pool (4,096 B/position, 3,072 MiB here)
-# and at ubatch 2048 that squeeze halves decode: 167.2 → 109.2 t/s. There is no
-# --spec-draft-cache-type knob in this build to make the draft cheap, and no
-# --spec-draft-model either: the draft rides the main model's weights. Card gate:
-# quantized V alongside MTP needs fork build >= prism-b10776, which is why
-# LLAMA_BIN moved to the 10777 prebuilt; on an older build the pair is illegal.
-# SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024 is the documented MTP arm (213.5 t/s);
-# SPEC_TYPE=none strips the whole set (flags move together).
-: "${SPEC_TYPE:=none}"
+# Speculative decoding defaults ON: the head IS in the checkpoint
+# (qwen35.nextn_predict_layers = 1 — MTP is a property of the FILE, so the plain
+# PQ2_0 sibling could not run it at any flag value) and the draft context rides
+# the MAIN model's weights, so there is no --spec-draft-model. Its bill is the
+# target's whole pool at 4,096 B/position plus ~0.45 GiB of buffers at this
+# ubatch and a tripled recurrent state; CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024
+# above are what pay for it. The draft cache dtype IS a further knob
+# (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this
+# build's --help) and stays at its default here. Card gate: quantized V alongside
+# MTP needs fork build >= prism-b10776, which is why LLAMA_BIN is the 10777
+# prebuilt; the type-142 loader lock keeps it pinned regardless.
+# SPEC_TYPE=none is the rollback and strips the whole set (flags move together).
+: "${SPEC_TYPE:=draft-mtp}"
 : "${SPEC_DRAFT_N_MAX:=2}"
 : "${SPEC_DRAFT_N_MIN:=0}"
 : "${SPEC_DRAFT_P_MIN:=0}"
@@ -104,7 +121,7 @@ fi
 : "${NGL:=99}"
 # Log verbosity threshold (llama.cpp -lv). 3 is the engine default (info);
 # 4 adds debug detail (KV layout, buffer sizes, graph capture) which is what
-# the VRAM-fit forensics need — the q8_0 KV contract and the draft-KV bill are
+# the VRAM-fit forensics need — the exact-KV size and the draft-KV bill are
 # only provable from this log. Levels above 4 flood the journal.
 : "${VERBOSITY:=4}"
 # /metrics needs the flag: llama.cpp returns 501 unless the server starts with
@@ -169,11 +186,11 @@ fi
 # open after boot; make sure it exists before the server needs it.
 nvidia-modprobe -u -c=0 2>/dev/null || true
 
-# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
-# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
-# the q8_0 KV contract in frank-zealot.env is a real path — without this
-# flag or that build option the engine silently falls back and the
-# contract becomes a lie.
+# Flash attention stays pinned on. Exact f16 KV runs fine with or without it,
+# but the documented fallback rung (quantized V) is ILLEGAL without it, and the
+# build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17) so that rung is a
+# real path — dropping the flag here would leave the retreat route silently
+# falling back to something the env file does not describe.
 ARGS=(
     --model "$MODEL_FILE"
     --alias "$ALIAS"

serverAppConfigs-docs/Agent/OMP Harness.md

@@ -14,16 +14,16 @@ registry; single source of truth — there is no copy on the servers) and
 this file is repo-managed; edits are harness-side only.
 
 **Model ids are the StarCraft unit names (official 2026-10-08, on the wire
-since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
-`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
-:7771, the 3090 compaction hub) — formerly `frank-archon`, `frank-dragoon`,
+since 2026-10-09):** `frank-archon` (Dankserver), `frank-dragoon` and
+`frank-zealot` (Dankclaw) — engine, port and card behind each id live on
+[[frank-archon]], [[frank-dragoon]] and [[frank-zealot]] — formerly `frank-archon`, `frank-dragoon`,
 `frank-zealot`. The SERVICES keep the old names (units, scripts, env
 files, runbook pages); pages written in the old era name the service,
 the harness names the model. The wire now matches: each endpoint's
 `/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
 picker/grants, the homepage docs and the raw API all use one id per model
-(see [[OpenWebUI]] for the grant re-key and the [[frank-archon-radiance-swift15-w4dn8]]
-note on radiance echoing the requested name).
+(see [[OpenWebUI]] for the grant re-key and the [[frank-archon]]
+note on the endpoint echoing the requested name).
 
 Verified 2026-10-18: `/v1/models` on 192.168.0.235:7770, 192.168.0.143:7770 and
 192.168.0.143:7771 each answer with the unit name (`frank-archon`,
@@ -50,10 +50,11 @@ Where the contract lives:
   `~/.omp/plugins/node_modules/@earendil-works/pi-coding-agent/dist/core/compaction/compaction.js`
 
 The arithmetic (the coupled contract every endpoint's maxTokens comment
-encodes). Recomputed from the live files 2026-10-18 — contextWindows
-200,000 (archon) / 155,648 (dragoon) / 200,704 (zealot) at
-`models.yml:48,131,206`; maxTokens 16,384 / 45,056 / 16,384 at
-`models.yml:56,143,212`:
+encodes). The authoritative `contextWindow`/`maxTokens` pair per model is on the
+endpoint page ([[frank-archon]], [[frank-dragoon]], [[frank-zealot]]); what
+follows is a **worked example** recomputed from the live files on 2026-10-18 —
+a snapshot, not fleet state — with windows 200,000 / 155,648 / 200,704 and
+maxTokens 16,384 / 45,056 / 16,384 (archon / dragoon / zealot):
 
 1. **Source trigger** = `thresholdPercent × source contextWindow`; at the live
    95 % that is 190,000 (archon), 147,866 (dragoon), 190,669 (zealot). The
@@ -141,28 +142,25 @@ Use **wire names**, not OMP camelCase. Two traps:
 2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
    also sets it — `Object.assign` is shallow and replaces the whole object,
    silently dropping the `reasoning_effort` kwarg the compat policy set.
-   Top-level `preserve_thinking` is honoured by halogen. Live state 2026-10-18:
-   frank-archon's radiance entry carries `chat_template_kwargs:
-   {enable_thinking, preserve_thinking}` in `extraBody` (`models.yml:89-91`)
-   and nothing else sets that object at that endpoint, so the shallow-merge
-   hazard is latent rather than firing; the FreeToken-era "verify-and-prune at
-   the next swap" note is retired with that swap (2026-10-05,
-   `scripts/serve-frank-archon.sh:9-10`).
+   Top-level `preserve_thinking` is honoured by frank-dragoon
+   ([[frank-dragoon]]). Live state 2026-10-18: frank-archon's `extraBody`
+   carries `chat_template_kwargs: {enable_thinking, preserve_thinking}`
+   (`models.yml:89-91`) and nothing else sets that object at that endpoint, so
+   the shallow-merge hazard is latent rather than firing.
 
-**Gotcha (retired 2026-10-17): silent greedy decode.** Earlier flash-server
+**Gotcha (retired 2026-10-17): silent greedy decode.** Earlier frank-dragoon
 builds carried **no server-side sampler defaults**, so an absent
 `temperature` decoded greedy while the endpoint looked configured. That era is
-over: `HALOGEN_TEMPERATURE/TOP_P/TOP_K` now set the card's thinking recipe
-(1.0 / 0.95 / 20) as engine defaults (`scripts/frank-dragoon.env:98-103`,
-exported at `scripts/serve-frank-dragoon.sh:186-188`), so an absent temperature
-is no longer greedy. frank-dragoon's `extraBody` still carries the same trio —
-consistent, not load-bearing. Treat any older "do not drop it or you get
-greedy" wording as history.
+over — the endpoint now ships its own defaults ([[frank-dragoon]]), so an
+absent temperature is no longer greedy. frank-dragoon's `extraBody` still
+carries the same trio — consistent, not load-bearing. Treat any older "do not
+drop it or you get greedy" wording as history.
 
-**Reasoning-effort folding.** Qwen3.8's template accepts only
-`{xhigh (default), medium, low}` for `reasoning_effort` and 400s on anything
-else ("Unexpected reasoning effort high"). Every endpoint's
-`reasoningEffortMap` folds all harness levels onto that set; a role whose
+**Reasoning-effort folding.** The set `reasoning_effort` accepts is a property
+of the endpoint, not of the harness: [[frank-dragoon]] and [[frank-zealot]]
+raise on anything outside `xhigh|medium|low`, while [[frank-archon]] does not
+gate the set. Every endpoint's `reasoningEffortMap` folds all harness levels
+onto whichever set that endpoint ships; a role whose
 level is unmapped fails on its FIRST request, and the failure names Jinja,
 not the model.
 

serverAppConfigs-docs/Architecture/Deploy State Triage.md

@@ -22,8 +22,10 @@ marker. So `marker != HEAD` means "not dispatched via deploy.sh", NOT
 "behavior pending".
 
 **Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
-whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
-a 100+ GiB payload — and frank-archon (dankserver:7770) is also the
+whose marker lags. On the big-payload endpoints that is a full model reload
+measured in tens of minutes — each endpoint's boot budget is on its own page
+([[frank-archon]], [[frank-dragoon]], [[frank-zealot]]) — and frank-archon
+(dankserver:7770) is also the
 harness's own model backend, so a blind deploy is a self-inflicted outage for
 possibly zero behavioral change.
 
@@ -46,9 +48,8 @@ possibly zero behavioral change.
    - `podman inspect <ctr> --format '{{.Args}}' | tr ' ' '\n' | grep -i <flag>`
    - engine env: `podman exec <ctr> sh -c 'for f in /proc/[0-9]*/environ; do
      tr "\0" "\n" <$f 2>/dev/null; done | grep -i <VAR>'`
-   - confirm the launcher maps each tunable to its flag
-     (`grep scripts/launch.sh`), e.g. `R9V_ENABLE_PREFIX_CACHING=1 →
-     --enable-prefix-caching`.
+  - confirm the launcher maps each tunable to its flag
+    (`grep scripts/launch.sh`).
 
 **Decision:** live argv+env == target config AND `/health` 200 ⇒ behavior is
 live — **do NOT restart**; reconcile the marker (below). Only if the live

serverAppConfigs-docs/Architecture/Deployment Flow.md

@@ -46,7 +46,7 @@ graph LR
 ./scripts/deploy.sh dankserver
 ./scripts/deploy.sh frankbook-pro --service qbittorrent   # podman-machine target
 ./scripts/deploy.sh dankclaw --service searxng
-./scripts/deploy.sh dankserver --service frank-medium
+./scripts/deploy.sh dankserver --service frank-archon
 ./scripts/deploy.sh dankclaw --dry-run
 ```
 
@@ -62,13 +62,13 @@ service on the other host.
 |---|---|---|
 | Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
 | SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
-| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
-| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
-| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
+| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the Dankclaw iGPU inference service; waits up to 20 minutes for `/health`. Shape, gates and boot budget: [[frank-dragoon]]. |
+| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the Dankclaw 3090 inference service; waits up to 20 minutes for `/health`. Shape, gates and boot budget: [[frank-zealot]]. |
+| `frank-archon` | systemd + `systemctl` (root unit) | `deploy_llm_archon` plans from repo paths and gates on the port; it does NOT see host-side unit drift (see [[frank-archon]] → Deploy / verify). |
 | Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
 
 The shared chat template maps to the inference service on the selected target.
-The `frank-npu` endpoint is retired (2026-09-11) and has no deploy handler;
+The `frank-npu` endpoint is retired ([[frank-npu]]) and has no deploy handler;
 an unrelated large-model deployment never touches it.
 
 ## Caddy safety

serverAppConfigs-docs/Architecture/Stale Signal Audit.md

@@ -34,8 +34,9 @@ port; a "probe" job scraping `/` (HTML → `unsupported Content-Type`); a
 `/metrics` that is disabled (Gitea 404).
 
 - A scrape cannot express "is this HTTP service answering?" — the right probe
-  is the **blackbox exporter** in probe mode (which also gives guarded vhosts
-  a real uptime signal, since Prometheus cannot solve an Anubis challenge).
+  is the **blackbox exporter** in probe mode
+  (`/probe?target=…&module=http_2xx`), which also gives guarded vhosts
+  a real uptime signal, since Prometheus cannot solve an Anubis challenge.
 - **Fix in the direction of honesty, never of silence:** unreachable target →
   mark the homepage card `wip: true` with the reason in `description` (the
   checker skips `wip`; `wip` is the existing convention for "real service,
@@ -43,8 +44,8 @@ port; a "probe" job scraping `/` (HTML → `unsupported Content-Type`); a
   database port on the LAN to make a badge green, and never delete a scrape
   job — an absent scrape and a healthy one are indistinguishable.
 - Alert rules must not depend on always-red signals — scope them to services
-  that can actually be healthy (`LlmEndpointDown` is scoped to `frank-*` for
-  exactly that reason).
+  that can actually be healthy; the `frank-*` scope `LlmEndpointDown` carries
+  is recorded on [[Prometheus]].
 - A target's failure STRING can prove a fix: a `/metrics` scrape flipping from
   `connection refused` to `unsupported Content-Type "text/html"` means the
   service came up and only the scrape is wrong. Read the error, not just

serverAppConfigs-docs/Dankclaw/Dankclaw Stack.md

@@ -13,11 +13,10 @@ timestamp: 2026-08-26
 # Dankclaw Stack
 
 Dankclaw (192.168.0.143) runs the AI, search, development, and observability
-stack. Its LLM inference endpoint `frank-dragoon` (`:7770`) is a ROCmFPX
-llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
-the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
-FLM v1.0.1) serves sight+sound models. The completion endpoint
-`frank-medium` (ROCmFPX llama.cpp, `:7770`) runs on Dankserver (192.168.0.235).
+stack. Its LLM inference endpoints are bare-metal host services, not part of
+the container stack; each has its own page — shape, payload, eras and wire
+contract live there, nowhere else. See [[frank-dragoon]] and [[frank-zealot]]
+(both on this host) and [[frank-archon]] (Dankserver, 192.168.0.235).
 
 ```mermaid
 graph LR
@@ -28,9 +27,8 @@ graph LR
     C --> F[ComfyUI]
     C --> D[Grafana]
     S --> V[Gluetun VPN]
-    O --> L[llama.cpp dankserver :7770]
+    O --> L[LLM endpoints]
     O --> H[Hermes]
-    O --> E[frank-dragoon :7770]
 ```
 
 ## Services
@@ -48,18 +46,17 @@ graph LR
 | Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
 | Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
 | Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
-| frank-dragoon (host service) | — | `7770` | [[deployment]] |
-| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
-| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
+| frank-dragoon (host service) | — | `7770` | [[frank-dragoon]] |
+| frank-zealot (host service) | — | `7771` | [[frank-zealot]] |
+| frank-archon (host service, Dankserver) | — | `7770` | [[frank-archon]] |
 
-The LLM rows are bare-metal systemd services, not containers. This
-repo owns their scripts, units, and knowledge base (see
-[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
-`frank-dragoon` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
-on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
-`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
-`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
-kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF on Dankserver.
+The LLM rows are bare-metal systemd services, not containers, and this repo
+owns their scripts, units and knowledge base (see
+[LLM fleet knowledge base](../LLM/index.md)). Which payload and runtime sits on
+each port is era state and lives on the endpoint page, not here. `frank-npu`
+(the XDNA2 NPU endpoint) and `frank-medium` (the predecessor of `frank-archon`
+on the Dankserver port) are both retired — see [[frank-npu]] and
+[[frank-medium-radiance-mxfp4]].
 
 ## Boot Persistence
 

serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — repo-owned, not yet deployed."
+description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — deployed 2026-10-18, and it publishes no port on purpose."
 tags:
   - dankclaw
   - tts
@@ -14,7 +14,7 @@ timestamp: 2026-10-18
 
 `frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** in RAM once and synthesizing from there. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.
 
-**Nothing here is deployed.** This page documents repo-owned configuration — the Quadlet unit and network, the image build, the weight fetcher, the boot warmer, the deployer path — and the measured facts those files were written against. No container, image, weight file or network exists on Dankclaw yet, and the engine publishes no port at all, so there is nothing for the homepage checker or a browser to reach. Deploying it is a deliberate next step, in the order at the bottom of this page.
+**Deployed 2026-10-18.** The chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified, 18 s), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy 31 s after the build, **4.0 GiB** resident on a host that kept 17 GiB available, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user.
 
 **Host:** Dankclaw (`192.168.0.143`), rootless Podman, user `dank` — **Port:** 8880, **never published** — **Network:** the private `systemd-frank-tts-engine` bridge (10.89.5.0/24) and no other — **Deployer:** `frank-tts-engine` — **Consumers:** `frank-tts` (7772) and, through it, every voice request in the fleet including [[Open Notebook]]'s podcast renders.
 
@@ -96,25 +96,27 @@ The **façade** on 7772 is the pair's public surface and carries the probe. Note
 |---|---|
 | `dankclaw/voiceservice/Containerfile.engine` | two-stage build of the pinned commit, proven cmake line, digest-pinned base, ships both scripts |
 | `dankclaw/voiceservice/tts-engine-start.sh` | image entrypoint (`/app/start.sh`): env → flags, refuses to boot without weights |
-| `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav`, speak one sentence, report cold/warm |
+| `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav` (body via a temp file), speak a fixed sentence twice, report cold/warm |
 | `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, health probe, warm hook, `Restart=always` |
 | `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the private 10.89.5.0/24 bridge, with the subnet registry |
-| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the cold-voice law in its header |
+| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the registry law in its header |
 | `scripts/download-frank-tts-engine-qwentts.sh` | weight fetcher: no resume, byte count + LFS oid check, `--verify-only` |
 | `scripts/deploy-remote.sh` → `deploy_frank_tts_engine` | weights gate → build → quadlet → restart → container health → warm |
 | `tests/frank_tts_engine.bats` | the seam: generator output, entrypoint argv, warmer against a stub engine, planner and routing maps |
 
-## Deploy order (host steps, in order)
+## Deploy record (host steps, in the order they ran)
 
-Nothing below has been run. Steps 1–5 are this service; 6–7 are what unblocks the podcast pipeline.
+Run 2026-10-18 as `./scripts/deploy.sh dankclaw --service frank-tts-engine`, then
+`--service frank-tts`. Steps 1–6 are this service and the façade; only step 7 is
+still open.
 
 1. **Fetch the weights** — `scripts/download-frank-tts-engine-qwentts.sh` (1.4 GiB, byte + sha256 verified, idempotent; `--verify-only` is what the deployer calls).
-2. **Deploy the engine** — `./scripts/deploy.sh dankclaw --service frank-tts-engine` (`--service` is the only way to name one service; `deploy-remote.sh` is the remote half and takes `<target> <git-revision> [service]...`, so never call it by hand with a bare service name). Builds the image (needs github.com once), installs the unit and the network file, restarts, gates on the container's health status, then warms.
+2. **Deploy the engine** — `./scripts/deploy.sh dankclaw --service frank-tts-engine` (`--service` is the only way to name one service; `deploy-remote.sh` is the remote half and takes `<target> <git-revision> [service]...`, so never call it by hand with a bare service name). It builds the image (needs github.com once), installs the unit and the network file, gates on the container's health status, then warms.
 3. **Confirm the endpoint is alive** — `podman exec frank-tts-engine curl -fsS http://127.0.0.1:8880/health` → `{"status":"ok"}`, and `systemctl --user status frank-tts-engine` showing the unit `active` with the warm pass behind it.
-4. **Confirm the voices are known** — the warmer's journal lines must name every pack; once the façade is up, `curl http://127.0.0.1:7772/v1/audio/voices` is the authoritative list (the façade has exactly three routes: `/health`, `/v1/audio/voices`, `/v1/audio/speech`).
-5. **Prove the cold-voice law is paid at boot, not by a listener** — two identical synthesis requests for one voice, back to back, from inside the engine's namespace; the **second** must answer well under a second (`warm=` in the journal is already that number; the manual pair is the confirmation on the real artifact).
-6. Then the façade on **7772** (its own page): nothing may be published there until the engine answers, because open-notebook's render fails the moment the façade is up and the engine is not.
-7. Nothing on 7772 exists yet — that is why open-notebook's podcast render cannot run. See [[Open Notebook]].
+4. **Confirm the voices are known** — the warmer's journal lines name every pack, and once the façade is up `curl http://127.0.0.1:7772/v1/audio/voices` is the authoritative list (the façade has exactly three routes: `/health`, `/v1/audio/voices`, `/v1/audio/speech`).
+5. **Measure the ladder instead of trusting the trial** — two identical requests for one voice back to back. The design expected 6.2-8.0 s then 0.5 ms; the deployed container gave **1,883 ms then 1,226-1,283 ms**, so the hook buys ~0.6 s and stays only for the registry reason (law 1). The retired numbers must not be quoted.
+6. **Then the façade on 7772** (its own page), after the engine answered — that order is the whole point of step 6: publishing the façade in front of a dead engine would make every consumer fail at the first voice line.
+7. **Open:** [[Open Notebook]] is still not installed, so nothing renders a podcast yet; the voice leg it waits on now exists.
 
 ## Troubleshooting
 

serverAppConfigs-docs/Dankclaw/Frank TTS.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; repo-owned, not yet deployed."
+description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; deployed 2026-10-18 and wired as the harness speech role."
 tags:
  - dankclaw
  - tts
@@ -16,20 +16,26 @@ audio in that cloned voice. Everything else in the voice stack sits behind it
 the engine ([[Frank TTS Engine]]) publishes no port on purpose, and consumers
 never learn a path.
 
-**Nothing here is deployed.** No host was touched while this was written: no
-image built, no container started, nothing listening on 7772. That is also why
-[[Open Notebook]] cannot render a podcast episode yet — its pipeline calls this
-endpoint.
+**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`,
+both units `active`, no drop-ins, and the homepage card's `wip` gate came off in
+the same change. Probes on the deployed endpoint: `/health` answers
+`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`,
+and two different voices rendered the same sentence into **different** bytes
+(24 kHz mono PCM16, 4.24 s, healthy RMS) — so name resolution is real, not a
+pass-through of one default. The operator harness routes its `speech` role here;
+see [[#The harness speech role]]. What still waits is [[Open Notebook]]: its
+render needs the app installed, not this endpoint.
 
 | | |
 | --- | --- |
 | Host | Dankclaw (`192.168.0.143`), rootless podman, user unit `frank-tts.service` |
-| Port | **7772**, published (`PublishPort=7772:7772`); the family is 7770 dragoon, 7771 zealot, 7772 tts |
+| Port | **7772**, published (`PublishPort=7772:7772`) — the rest of the `frank-*` port family is in [[Dankclaw Stack]] |
 | Routes | `GET /health`, `GET /v1/audio/voices`, `POST /v1/audio/speech`; everything else 404, and speech is POST-only |
 | Image | `localhost/franksdank/frank-tts:latest`, built from `dankclaw/voiceservice/Containerfile`, `Pull=never` |
 | Networks | `dankclaw_default` first (it owns the default route and the consumers), then `frank-tts-engine.network` (10.89.5.0/24, the bridge to the engine) |
 | Voice packs | `/home/dank/voices` mounted read-only at `/voices` — host state, never repo state |
 | Upstream | `http://frank-tts-engine:8880` (`scripts/frank-tts.env`, `TTS_UPSTREAM`) |
+| Default voice | `DEFAULT_VOICE` (`scripts/frank-tts.env`) answers a request that names none; the journal marks that line `(default)`, and an **unknown** name is still a 400 |
 | Rollback | `TTS_BACKEND=llama-tts-cli` runs a CLI binary per request and needs no upstream at all |
 
 ## Naming a voice
@@ -37,7 +43,9 @@ endpoint.
 A voice **is** a directory name. `voice=narrator1` resolves to
 `/voices/narrator1/reference.wav`, and that file — not a manifest, not a config
 entry — is what makes the name real. omp's TTS device may send `voice_id`; the
-shim accepts either.
+shim accepts either. A call that sends **neither** — the harness's normal shape,
+because no harness setting carries a voice for this transport — speaks
+`$DEFAULT_VOICE`; empty default makes it a 400 again.
 
 The usable shape is `^[a-z0-9][a-z0-9._-]*$`: lowercase, starting with a letter
 or digit, then letters, digits, `.`, `_`, `-`, at most 64 characters. No slash,
@@ -109,7 +117,7 @@ shows up as a failed synthesis, never as a down probe.
 | `dankclaw/voiceservice/Containerfile` | python 3.12-alpine pinned by digest + ffmpeg, uid 10000 |
 | `dankclaw/voiceservice/quadlet/frank-tts.container` | the unit: two networks, `Restart=always`, `HealthCmd`, `[Install]` |
 | `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the shared bridge, named after the engine unit and attached by both |
-| `scripts/frank-tts.env` | backend, upstream, port, limits — paths and numbers, no credentials |
+| `scripts/frank-tts.env` | backend, upstream, port, limits, and the default voice — paths and numbers, no credentials |
 | `tests/frank_tts_unit.bats` | the shim against stub engines over real sockets |
 | `tests/frank_tts_engine.bats` | the pair's wiring: generator output, planner, homepage, deployer |
 
@@ -134,12 +142,55 @@ types: [[Prometheus]] records that the existing HTML probes (`comfyui`, the two
 `services` targets) sit **down** with
 `received unsupported Content-Type … and no fallback_scrape_protocol specified`.
 A `/health` probe job would therefore be structurally never-green, and because
-`LlmEndpointDown` is scoped `up{job=~"frank-.*"} == 0`, a permanently red
-`frank-tts` target would page on Signal every few minutes for an endpoint that
-is fine — the frank-archon lesson pointed the other way. The engine cannot be
+`LlmEndpointDown` is scoped across the whole `frank-*` job family (the rule set
+lives on [[Prometheus]]), a permanently red `frank-tts` target would page on
+Signal every few minutes for an endpoint that is fine. The engine cannot be
 scraped at all: it publishes nothing. Status lives in `systemctl --user status`,
-`podman healthcheck run`, the two unit journals, and the homepage card, which is
-gated `wip: down` until the deploy and probes `127.0.0.1:7772`.
+`podman healthcheck run`, the two unit journals, and the homepage card, which
+probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
+2026-10-18 deploy.
+
+## The harness speech role
+
+The operator harness reaches this endpoint through
+`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`,
+`baseUrl: http://192.168.0.143:7772/v1`, `auth: none`, and the runner api
+`openai-speech` on the **model** row (naming a runner api at *provider* level
+fails the whole file's validation and disables every custom provider, which is
+how the fleet's chat endpoints disappear) — plus `config.yml`
+`modelRoles.speech: frank-tts (dankclaw)/frank-tts`. Four laws, read out of the
+harness bundle and measured the day of the flip:
+
+- **The only voice the harness can send is the `voice_id` argument of the call.**
+  The transport builds `{model, input, response_format, voice?}` and adds `voice`
+  **only** when the caller passed `voice_id`; no setting feeds it.
+  `tts.localVoice` is the *local Kokoro backend's* voice (it is what answered
+  `voice=kokoro/bm_george, backend=local` in the control test), and
+  `speech.voice` is the TUI's **speak-aloud** voice — its enum is Kokoro's list,
+  and it never reaches an HTTP request. So a harness call arrives voice-less by
+  default, which is exactly why the endpoint grew `DEFAULT_VOICE`: the declared
+  pack answers a voice-less request and the journal marks that line `(default)`,
+  while an **unknown** name still 400s with the list that exists.
+- **`speech.voice` must stay a Kokoro name.** Setting it to a pack name does not
+  select a pack; it hands the vocalizer a value outside its enum. (The first
+  draft of this section said the opposite and was reverted the same day, after a
+  fresh session's call came back `unknown voice None`.)
+- **There is no silent fallback.** An explicitly configured `modelRoles.speech`
+  yields exactly one candidate, so a dead 7772 fails the speech device with
+  *"No available speech model matches the speech role"* instead of quietly
+  degrading to Kokoro. That is the price of `auth: none` (which makes a dead
+  endpoint read as available) and it is the wanted shape: a narration job should
+  fail loudly rather than ship a voice Frank never asked for.
+- **A session older than the models.yml row cannot see it.** `models.yml` loads
+  at process start while `config.yml` is read live, so a long-lived session
+  carrying the flipped role fails exactly like a dead endpoint while a fresh
+  process succeeds in the same minute. Measured 2026-10-18: the open session
+  failed twice (provider/id form, then the bare id), `local/kokoro` succeeded in
+  that same session, and a fresh `omp -p --tools tts` returned `Saved 180524
+  bytes (model=frank-tts (dankclaw)/frank-tts, voice=narrator1, codec=wav)`
+  while the façade logged the matching `speech voice=narrator1 format=wav
+  chars=51 -> 180524 bytes` from the podman bridge. Rule the stale registry out
+  before blaming the service.
 
 ## Troubleshooting
 
@@ -150,3 +201,4 @@ gated `wip: down` until the deploy and probes `127.0.0.1:7772`.
 | Endpoint answers, synthesis fails | the engine: `podman healthcheck run frank-tts-engine`, then `systemctl --user status frank-tts-engine.service` |
 | First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
 | The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
+| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |

serverAppConfigs-docs/Dankclaw/Hermes.md

@@ -78,9 +78,9 @@ is not durable. The tracked, sanitized configuration baseline is
   `/data/config.yaml` is not readable without `sudo`):
   `dankclaw-large` reaches `http://192.168.0.143:7770/v1` and exposes
   `frank-dragoon` — verified from the host with `curl
-  http://127.0.0.1:7770/v1/models` → single id `frank-dragoon`, `max_model_len
-  200704` (2026-10-18; write `127.0.0.1`, not `localhost`, because that port is
-  a rootless pasta publish and is IPv4-only);
+  http://127.0.0.1:7770/v1/models` → a single served id (2026-10-18; write
+  `127.0.0.1`, not `localhost`, because that port is a rootless pasta publish
+  and is IPv4-only);
   `dankserver-small` reaches `http://192.168.0.235:7770/v1` and, **as of
   2026-10-18, that endpoint serves only `frank-archon`** (`GET
   http://192.168.0.235:7770/v1/models` → `[{"id":"frank-archon"}]`). The name
@@ -88,7 +88,8 @@ is not durable. The tracked, sanitized configuration baseline is
   2026-09-15 — is retired-era: nothing on that endpoint answers to it today, so
   if the live `config.yaml` still names `frank-medium` the provider is dead
   weight and requests will 404 on a nonexistent model. Diff the live config
-  before trusting either name.
+  before trusting either name. Which payload sits behind each port, and the
+  served id it answers to, is on [[frank-dragoon]] and [[frank-archon]].
 - The live `config.yaml` contains secrets and user state and is not committed;
   the tracked example is the sanitized backup of its non-secret settings.
 
@@ -227,9 +228,10 @@ resp = call_llm(task="title_generation", messages=[...], max_tokens=64,
 
 What actually breaks, and the fix:
 
-- **`response_format` (json_object / json_schema) is a hard 400 on every
-  local backend here** — neither FreeToken nor halogen implements constrained
-  decoding. Config cannot override it: `call_llm` merges the task's
+- **`response_format` (json_object / json_schema) is a hard 400 on the local
+  backends this task was written against** — what each endpoint accepts on the
+  wire is on [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]]. Config
+  cannot override it: `call_llm` merges the task's
   `extra_body` first and then `update()`s the caller's, so the hardcoded one
   wins. Only temperature and max_tokens have reactive retry paths.
 - **Fix: move that one task to the Responses transport**, which never
@@ -246,9 +248,8 @@ What actually breaks, and the fix:
           effort: none                 # mandatory, see below
   ```
 
-  Before copying this: as of 2026-10-18 `frank-zealot` is served on **Dankclaw
-  `:7771`**, and `http://192.168.0.235:7770/v1` — the endpoint this provider
-  name is recorded against — answers only `frank-archon`. The provider/model
+  Before copying this: which id is served on which port is endpoint state — see
+  [[frank-zealot]] and [[frank-archon]]. The provider/model
   pairing above is the shape that was landed in 2026; re-derive it from the live
   config before reusing it.
 

serverAppConfigs-docs/Dankclaw/Open Notebook.md

@@ -59,9 +59,9 @@ podman exec open-notebook curl -sS http://frank-tts:7772/health
 podman exec open-notebook curl -sS http://host.containers.internal:7771/v1/models
 ```
 
-**The blocker to state plainly:** frank-tts is built, named and tested but **not deployed** — nothing answers on 7772 (`ss -ltn`, Dankclaw, 2026-10-18; [[Local TTS and Voice Cloning]] records the same census). open-notebook deployed today writes scripts and fails at the first voice line. frank-tts ships first, or podcast generation waits.
+**The blocker moved; it did not vanish.** frank-tts and frank-tts-engine are **deployed** (2026-10-18) and answer on 7772, so the voice leg works end to end — `podman exec open-notebook curl -sS http://frank-tts:7772/health` is the check to run the day the app exists. What still blocks a render is open-notebook itself: not installed on this host yet. When it is, the two provider rows above are the only wiring, and the seeded speaker profiles carrying OpenAI voice names are the first trap.
 
-**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). llama.cpp's OpenAI-compatible endpoint is a known-good target for that; frank-dragoon's halogen runtime has **unverified** structured-output behaviour, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
+**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). The decision rests on each endpoint's documented request surface ([[frank-zealot]], [[frank-dragoon]]): zealot's is a known-good target for that call shape, dragoon's structured-output behaviour is **unverified**, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
 
 ## Speakers and voices: 1–4, and the IDs are free text
 

serverAppConfigs-docs/Dankclaw/OpenWebUI.md

@@ -39,28 +39,26 @@ As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`** *(unve
 `frank-dragoon`/`frank-zealot`/`frank-archon`, per the cards in
 `shared/homepage/services.json`)*.
 
-| Connection (`openai.api_configs` idx) | URL as recorded 2026-10-09 | Enabled | `model_ids` | Runtime | Host |
-|---|---|---|---|---|---|
-| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
-| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | radiance TP=2 (2× R9700) | Dankserver |
-| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090; production shape 3×262,144 per `scripts/frank-zealot.env`) | Dankclaw |
+| Connection (`openai.api_configs` idx) | URL as recorded 2026-10-09 | Enabled | `model_ids` | Endpoint page |
+|---|---|---|---|---|
+| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | [[frank-dragoon]] |
+| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | [[frank-archon]] |
+| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | [[frank-zealot]] |
 
 The URLs in the first column are the DB's recorded values and predate the 2026-08-22
 bridge move; on this network they are wrong — see the reachability gotcha below, which
-is the corrected law. Note also that the 2026-10-18 **bench window** on `:7771`
-closed the same day (production unit stopped 03:38 per the repo Benchmarking
-Policy while candidate arms held the port): the candidate — Ternary-Bonsai-2-**9B**
-PQ2_0 with the MTP head, 3 × 262,144 — was **deployed as production**, so the
-picker row below was re-keyed to it (see the next paragraph).
+is the corrected law. The picker row for `frank-zealot` was re-keyed on 2026-10-18 after
+that day's payload swap (see [[frank-zealot]] for what was deployed and when).
 
 The `model` table carries exactly two decoration rows, `frank-zealot` and
 `frank-dragoon`. Both the context length and the description live **inside
 JSON columns** — `params` holds `{"context": {"length": …}}` and `meta` holds
 `{"description": …}`; there is no `context_length` or `description` column, so
-`sqlite3` one-liners against those names fail. Re-keyed 2026-10-18:
-`frank-zealot` → 262,144 with the Bonsai-2-9B description, and `frank-dragoon`
-→ 200,704 (its `params` had kept the pre-lowering 262,144 window from
-2026-10-17 while its own description already said 200,704). These rows are
+`sqlite3` one-liners against those names fail. `params.context.length` is a copy
+of the endpoint's per-slot window, so the authoritative number lives on the
+endpoint page ([[frank-zealot]], [[frank-dragoon]]) and the row must follow it —
+the 2026-10-18 re-key existed because `frank-dragoon`'s row still carried the
+window its payload had before that day's swap. These rows are
 live DB state, so a payload swap must re-key the database too, not only the
 repo. There is no `sqlite3` binary on Dankclaw — use `python3 -c` with the
 stdlib `sqlite3` module, and take a `Connection.backup()` snapshot first (the
@@ -104,10 +102,10 @@ Two gotchas make the wiring non-obvious:
 The Ollama connection is disabled (`ollama.enable=false`).
 
 > All three endpoints are bare-metal systemd services managed by this
-> repository. Model swaps, context sizing, and spec tuning happen there. See
-> the [LLM knowledge base](../LLM/index.md). Disabling a connection here only
-> hides it from this picker — the endpoint itself keeps serving every other
-> client (the harness, Hermes, API consumers).
+> repository; their shape, swaps and tuning live on
+> [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]]. Disabling a
+> connection here only hides it from this picker — the endpoint itself keeps
+> serving every other client (the harness, Hermes, API consumers).
 
 ## Container introspection (DB and backend python)
 
@@ -143,9 +141,9 @@ The Ollama connection is disabled (`ollama.enable=false`).
    models. Response key is `items`, not `data`.
 2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
    with exactly one word: pong") → each returns content `pong`. Reasoning
-   models need a generous budget: frank-zealot (Qwen3.5 template) returns
-   empty content + `finish_reason: "length"` at `max_tokens 200` (the
-   thinking block eats it) and answers `pong` at 1500.
+   endpoints need a generous budget: thinking consumes `max_tokens`, so a short
+   one returns empty content + `finish_reason: "length"` (see
+   [[frank-zealot]]); `frank-zealot` answers `pong` at 1500.
 3. A 500 "Server Connection Error" → `podman logs open-webui` for
    `ClientConnectorError`: a network-reachability problem (host LAN IP vs
    `host.containers.internal`), not a model problem.
@@ -292,6 +290,42 @@ carries base64 image data or a URL.
 Failure modes: connection refused ⇒ OpenWebUI or ComfyUI is down; timeouts
 on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
 
+## Text-to-speech (frank-tts)
+
+TTS is the fleet's one OpenAI-compatible speech endpoint —
+`http://host.containers.internal:7772/v1` (`audio.tts.openai.api_base_url`
+in the config DB, 2026-10-18), model `frank-tts`, voice `narrator1`. That
+port is the rootless quadlet `frank-tts` (this repo:
+`dankclaw/voiceservice/quadlet/frank-tts.container`,
+`scripts/frank-tts.env`), and its shim resolves `voice=<name>` against
+`/home/dank/voices/<name>/reference.wav` — a name without a reference is a
+400 listing the voices that exist, never a silent fallback.
+
+**The 29-voice `af_* … bm_fable` list is NOT frank-tts.** That enum belongs
+to **Kokoro.js running in the browser**: the deployed front end (0.11.4)
+ships an in-page Kokoro implementation (chunk carrying a frozen voice table
+`{af_heart:{name:"Heart",…}}` and `_validate_voice`, model
+`onnx-community/Kokoro-82M-v1.0-ONNX`, plus a personal *Kokoro.js dtype*
+setting). Its engine select takes `""` (Default) or `browser-kokoro`; when
+`browser-kokoro` is chosen the page synthesizes locally and rejects any
+voice outside its fixed pack — so `narrator1` fails with
+`Voice: "narrator1" not found. Should be one of: af_heart, …` **without a
+single request reaching the server**. Personal audio settings are per
+browser: no `user.settings` row and no container env carries an audio
+override, yet the toast still appears ⇒ the choice lives in that browser's
+own storage. Fix in the UI: Settings → Audio → TTS Engine → **Default**,
+which hands synthesis to the admin config (frank-tts); picking `af_heart`
+instead keeps Kokoro, and Kokoro cannot clone — fixed pack only.
+
+Verified 2026-10-18 from inside the container with the exact OpenWebUI body
+(`{"model":"frank-tts","input":…,"voice":"narrator1",
+"response_format":"mp3","speed":1.0}`) → **200, `audio/mpeg`, 7,629 B**,
+while `podman logs open-webui` recorded **zero** `audio/speech` requests for
+the failing click — that contrast is the instrument: a server-path failure
+leaves a log line and a frank-tts error, a `browser-kokoro` failure leaves
+neither.
+
+
 ## Retired instances
 
 - The compose `open-webui` service (port 3000) was never run: 3000 is Gitea's HTTP port, so it could not bind.
@@ -302,8 +336,8 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
 
 - The image tracks upstream `main`; pin a digest in the quadlet unit if reproducible rollback matters more than tracking. (The old "CPU-only host" framing is stale: this APU runs the gfx1151 ROCm build — see [[ComfyUI]].)
 - The default admin account is created on first login; secure it immediately.
-- If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready.
-- The on-host LLM services (Dankclaw `:7770`, `:7771`) are reached via the **host LAN IP `192.168.0.143`**, which is exactly what the deployed unit does (`open-webui.container:32`). The older "use `host.containers.internal`" advice is retired for this container: it is a slirp4netns/pasta-only name and the unit has been on the pinned bridge `systemd-open-webui` since 2026-08-22, where that name does not resolve (`open-webui.network:15-19`). Both endpoints are published on all interfaces today (`ss -ltn` → `*:7770`, `0.0.0.0:7771`; `curl http://192.168.0.143:7771/health` → 200, 2026-10-18). The Dankserver backend is a different host, reached via its LAN IP `192.168.0.235:7770`.
+- If the backend is loading, OpenWebUI shows connection errors until the endpoint is ready; each endpoint's boot budget is on its own page ([[frank-dragoon]], [[frank-zealot]], [[frank-archon]]).
+- Repeated from *The LLM backends*: the on-host endpoints are reached via the **host LAN IP**, never `host.containers.internal`.
 
 ## History
 
@@ -349,6 +383,7 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
 ## See Also
 
 - [[Dankclaw Stack]]
+- [[Frank TTS]] — the speech endpoint behind *Text-to-speech (frank-tts)*
 - [[Hermes]] — the agent OpenWebUI can talk to
 - [[SearXNG]] — the search engine on this host
-- [LLM knowledge base](../LLM/index.md) — the vLLM and llama.cpp endpoints
+- [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]] — the endpoints behind the picker

serverAppConfigs-docs/Dankclaw/Swap Configuration.md

@@ -36,14 +36,14 @@ it is not invisible.
 ## Why 64 GiB
 
 Dankclaw is shared. The 2026-09-12 decision measured `frank-dragoon` alone holding
-**103.5 GiB of the 121.2 GiB** host (`68.0` weights + `14.4` KV pool + `21.1`
-working), leaving roughly **7.1 GiB** for the then-28 co-tenant containers whose
-working set was ~4.3 GiB. That is a decision-time reading, not today's: on
+**103.5 GiB of the 121.2 GiB** host, leaving roughly **7.1 GiB** for the then-28
+co-tenant containers whose working set was ~4.3 GiB. How that total is split
+between weights, KV pool and working memory is the engine's own ledger — it lives
+on [[frank-dragoon]]. That is a decision-time reading, not today's: on
 2026-10-18 the host reports **91 GiB used / 29 GiB available of 121 GiB**
 (`free -h`), 30 containers are up (`podman ps -q | wc -l`), and the
 `frank-dragoon-halogen` container accounts for **70.33 GiB**
-(`podman stats --no-stream`, same day) on the current flash-server payload —
-a different model and KV shape than the one measured in September.
+(`podman stats --no-stream`, same day).
 The engine itself warned at the time that when the headroom runs out,
 frank-dragoon and its neighbours "can stall for minutes at a time at full CPU with
 no disk activity and no output". Swap is the valve that turns an OOM kill into slow
@@ -59,7 +59,7 @@ like this, **never `swapoff` before the replacement is online.** The old 8 GiB
 held 4 GiB of live pages, and `MemAvailable` read 73 GiB while only ~5 GiB was
 genuinely free — the pinned model weights are counted as reclaimable page
 cache, so the headline number is a lie by roughly the size of the model
-(see [[halogen-flash-server]]). Swapping off into that apparent space, with
+(see [[frank-dragoon]]). Swapping off into that apparent space, with
 nothing else to receive the pages, is how you invite the OOM killer.
 
 The ordering that avoids it — create and enable the new swap **first**, so the
@@ -113,10 +113,10 @@ frank-dragoon `health=200` throughout.
 
 ## Caveats
 
-- **Swap cannot help the pinned weights.** frank-dragoon's ~68 GiB of weights are
-  held deliberately; paging them is a stall, not a relief. More swap does not
-  make the doubled KV pool affordable — see the reserve/pool ceiling in
-  [[halogen-flash-server]].
+- **Swap cannot help the pinned weights.** frank-dragoon's weights are pinned
+  deliberately; paging them is a stall, not a relief. More swap does not make a
+  larger KV pool affordable — see the reserve/pool ceiling in
+  [[frank-dragoon]].
 - **`vm.swappiness=60` is untested against this workload.** Under real
   pressure the kernel will happily page anonymous memory, and frank-dragoon's KV
   pool and working set *are* anonymous. Swapping the KV pool would be far worse
@@ -151,6 +151,6 @@ The old swapfile has therefore been purged; nothing on the host still holds it.
 
 ## See Also
 
-- [[halogen-flash-server]] — the memory model that motivated this
+- [[frank-dragoon]] — the memory model that motivated this
 - [[Dankclaw System Specs]] — the Strix Halo box
 - [[Dankclaw Improvements]] — other host-level changes

serverAppConfigs-docs/Dankclaw/Voice Packs.md

@@ -233,12 +233,12 @@ loud, not a bug. Work through it in order:
   they have to be copied there once before a build.
 - `VOICE_PACK_TOOL` points at the trial's build tree
   (`/home/dank/voice-audition/qwentts/qwentts.cpp/build/qwen-codec`, checked out
-  at exactly `5916bf7c`), and the two GGUFs at
-  `/home/dank/models/Qwen3-TTS-GGUF/`, which is where the engine slice's
-  download script puts them — that directory does not exist yet, so until it
-  does the models have to come from the trial's `models/` directory.
-- Nothing is deployed: nothing listens on 7772 or 8880 today, so building packs
-  is possible on that host while serving them is not.
+  at exactly `5916bf7c`), and the two GGUFs live at
+  `/home/dank/models/Qwen3-TTS-GGUF/`, the engine slice's fetch target — that
+  path and its fetch state belong to [[Frank TTS Engine]].
+- Whether the façade (7772) and the engine (8880) are up and serving is deploy
+  state on [[Frank TTS]] / [[Frank TTS Engine]] — read it there, not from this
+  census.
 
 ## Known gaps, on purpose
 

serverAppConfigs-docs/Dankserver/Plex.md

@@ -30,7 +30,7 @@ public traffic can be fronted by Caddy like every other vhost.
 | Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover |
 | Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration |
 | Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` |
-| GPU | none by decision — CPU decode only; the two R9700s stay dedicated to `frank-medium` |
+| GPU | none by decision — CPU decode only; the two R9700s stay dedicated to [[frank-archon]] |
 | Deploy | `./scripts/deploy.sh dankserver --service plex` |
 | Docs | https://support.plex.tv |
 

serverAppConfigs-docs/Infrastructure/Homepage.md

@@ -159,10 +159,9 @@ Sonarr, Radarr, Lidarr, Readarr, qBittorrent) is out of the family docs
 entirely — admin tooling; Seerr is presented as the front door for
 requests. Each fleet model gained a Hugging Face source link: card field
 `model.hf` rendered as the cyan `model on huggingface ↗` line under the
-role, and the docs `Model:` bullets link the checkpoint names (zealot →
-HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive, dragoon →
-Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated, archon →
-Qwen/Qwen3.8-Flash-Next-FP8).
+role, and the docs `Model:` bullets link the checkpoint names — which
+checkpoint each endpoint runs is payload state on
+[[frank-dragoon]], [[frank-archon]] and [[frank-zealot]].
 
 **Navy Protoss console, maximal cornering, font swap (2026-10-08, sixth
 wave):** the amber CRT palette is retired. The page is now deep space
@@ -235,11 +234,10 @@ house-DNS advice; the bullet now just says the names resolve on the
 home network.
 
 **Archon card runs the fleet's own model (2026-10-08):** the
-`frank-archon` homepage card and docs Model bullet now say the payload
-is our in-house GPTQ-w4 quant of Swift1.5-Qwen3.8-Flash-Next and link
-our HF publication (`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`)
-in place of the FP8-era base-model link; capacity line says three
-submissions (`--max-num-seqs 3`).
+`frank-archon` homepage card and docs Model bullet swapped the FP8-era
+base-model link for our own HF publication, and the capacity line for the
+endpoint's own concurrency figure; the payload, its publication and the slot
+count all live on [[frank-archon]].
 
 ## Design decision — 2026-08-22
 

serverAppConfigs-docs/Infrastructure/Observability Provisioning.md

@@ -130,13 +130,14 @@ the blackbox exporter. See [[Prometheus]] → Targets.
 **A llama.cpp endpoint is invisible unless it opts in.** Until 2026-10-18,
 `up{job="frank-zealot"}` was 0 for the entire retained history of the metric —
 the oldest sample, 2026-10-11, is already 0 — while the service itself answered
-`/health` 200 the whole time. Cause: llama.cpp answers **501 Not Implemented**
-on `/metrics` unless the server is started with `--metrics`, so the scrape was
-refused at the protocol layer and `LlmEndpointDown` (critical) stayed firing:
+`/health` 200 the whole time. Cause: llama.cpp only exports `/metrics` when it
+is started with the flag, so the scrape was refused at the protocol layer (the
+flag and its opt-out are on [[frank-zealot]]) and `LlmEndpointDown` (critical)
+stayed firing:
 the relay re-sent it on the 4 h `repeat_interval` for days (`journalctl --user
 -u signal-relay` — firing sends at 01:37, 05:42, 09:47, 13:52, 17:57 and 22:02
 on 2026-10-12). It finally resolved at `2026-10-18T01:53:21-07:00`, after the
-`METRICS=1` knob added to `scripts/frank-zealot.env` that day; the target
+metrics opt-in landed that day; the target
 reports up since (`curl -s http://127.0.0.1:9090/api/v1/targets` 2026-10-18).
 How many payload generations it spanned is not readable from any host
 artifact. *(unverified 2026-10-18)*
@@ -151,7 +152,8 @@ notification log — `dankclaw/observability/quadlet/alertmanager.container:39-4
 `/opt/alertmanager/data`.)
 
 Remaining known downs are the structural ones above (HTML-scraped probes, Gitea
-metrics, ComfyUI), and halogen serves `/metrics` natively, which is why
+metrics, ComfyUI), and `frank-dragoon` exports `/metrics` natively
+([[frank-dragoon]]), which is why
 `frank-dragoon` has always been green. The active-alert count is a
 moment-in-time value and should never be documented as a constant:
 `/api/v1/alerts` was empty at the 2026-10-18 morning census, held one firing
@@ -167,8 +169,8 @@ arm (`systemctl is-active frank-zealot` → `failed` after
 `stop-sigterm timed out. Killing`, which is llama.cpp ignoring SIGTERM past
 `TimeoutStopSec`, the known shutdown shape for these engines). So
 `up{job="frank-zealot"} == 1` that day measured the trial, not the production
-shape in `scripts/frank-zealot.env` (Ternary-Bonsai-27B-PQ2_0, `CTX_SIZE=401408
-PARALLEL=2` = 2 × 200,704 per slot, q8_0 K+V), which remains the repo contract.
+shape in `scripts/frank-zealot.env` — that shape is [[frank-zealot]]'s to state,
+and it is the repo contract.
 The unit stayed `enabled`; `./scripts/deploy.sh dankclaw --service frank-zealot`
 reinstalls and starts it, restoring boot persistence and self-healing. Never
 quote a candidate arm's flags or VRAM as fleet state — the process and its shape

serverAppConfigs-docs/Infrastructure/Prometheus.md

@@ -25,7 +25,7 @@ Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
 | Config | `/opt/prometheus/config/prometheus.yml`, installed from `shared/prometheus/prometheus.yml` on every deploy, together with `/opt/prometheus/config/rules/llm-fleet.yml` from `shared/prometheus/rules/llm-fleet.yml` (`scripts/deploy-remote.sh:219-227`) |
 | Data | `/opt/prometheus/data` (TSDB, `prometheus.container:52`) |
 | Retention | `--storage.tsdb.retention.time=30d` (`prometheus.container:49`) |
-| Alerting | **4 alert rules + 2 recording rules** (`shared/prometheus/rules/llm-fleet.yml`: alerts `LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`, `LlmEndpointDown`; recordings `job:llm_prefix_cache_reuse:rate5m`, `job:llm_spec_accept_length:rate5m`), confirmed against the running server: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18) lists 4 alerting + 2 recording. Evaluated **and delivered** since 2026-09-17: the `alerting:` block ships to `alertmanager:9093` (`prometheus.yml:19-22`) → Signal relay → signal-cli (see [[Alertmanager]]) |
+| Alerting | **4 alert rules + 2 recording rules** (`shared/prometheus/rules/llm-fleet.yml`: alerts `LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`, `LlmEndpointDown`; recordings `job:llm_prefix_cache_reuse:rate5m`, `job:llm_spec_accept_length:rate5m`), confirmed against the running server: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18) lists 4 alerting + 2 recording. `LlmEndpointDown` is scoped `up{job=~"frank-.*"} == 0`, so only the `frank-*` LLM jobs can page on it — any other job that is structurally never-green must stay out of that family ([[Stale Signal Audit]]). Evaluated **and delivered** since 2026-09-17: the `alerting:` block ships to `alertmanager:9093` (`prometheus.yml:19-22`) → Signal relay → signal-cli (see [[Alertmanager]]) |
 
 ## Targets
 
@@ -37,10 +37,10 @@ Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
 | `node-dankclaw` | `host.containers.internal:9100` | host metrics from [[Node-Exporter]] — a net=host bind, so via the gateway (the LAN address is unreachable from this bridge container) |
 | `node-dankserver` | `192.168.0.235:9100` | host metrics from [[Node-Exporter]] — a different machine, so the LAN address works |
 | `grafana`, `loki` | `grafana:3000`, `loki:3100` | container names on the `dankclaw_default` network; both serve real `/metrics` |
-| `frank-archon` | `192.168.0.235:7770` | the LLM fleet — the radiance engine since 2026-10-05, exporting vLLM-compatible names plus `radiance:*` (`prometheus.yml:122-131`); a different machine, so the LAN address works |
-| `frank-dragoon`, `frank-zealot` | `host.containers.internal:7770`, `:7771` | the LLM fleet — both export the `llamacpp:` namespace (halogen included: `curl -s http://127.0.0.1:7770/metrics` → `llamacpp:prompt_tokens_total`, `text/plain; version=0.0.4`, 2026-10-18). They are addressed through the gateway because they are **same-host** — `frank-zealot` is a host-process bind, `frank-dragoon` a container publishing `0.0.0.0:7770->8731`, not a net=host pair (`prometheus.yml:96-99`, `:108-120`) |
+| `frank-archon` | `192.168.0.235:7770` | the LLM fleet — a different machine, so the LAN address works (`prometheus.yml:122-131`); engine, metric families and their meaning live on [[frank-archon]] |
+| `frank-dragoon`, `frank-zealot` | `host.containers.internal:7770`, `:7771` | the LLM fleet. They are addressed through the gateway because they are **same-host** — a host-process bind and a published container port are equally invisible to a bridge container at the host's LAN address (`prometheus.yml:96-99`, `:108-120`); what each endpoint exports is on [[frank-dragoon]] and [[frank-zealot]] |
 
-The `frank-medium` job and its `vllm:` namespace are **retired**: that endpoint left the fleet, `frank-archon` briefly had no exporter at all (the FreeToken-era job scraped 404s forever) and re-joined on radiance the same day, 2026-10-05 (`prometheus.yml:101-107`). No `frank-medium` job exists in the config or in `/api/v1/targets` (2026-10-18).
+The `frank-medium` job is **retired**: no `frank-medium` job exists in the config or in `/api/v1/targets` (2026-10-18), and the endpoint itself left the fleet — its history is on [[frank-archon]], which holds the port today.
 
 **Three jobs read down today, and they are not outages** (re-audited against `curl -s http://127.0.0.1:9090/api/v1/targets`, 2026-10-18; first audited 2026-09-16):
 
@@ -53,15 +53,14 @@ The `frank-medium` job and its `vllm:` namespace are **retired**: that endpoint
 
 Gitea's exporter stays off until `[metrics] ENABLED = true` in its own config (no metrics env is set in the container, and `curl -s http://127.0.0.1:3000/metrics` → 404, 2026-10-18); ComfyUI and the two `services` probes are HTML pages, which Prometheus 3 refuses as metrics. Everything else is green: `prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`, `frank-zealot`, `frank-dragoon`, `frank-archon`.
 
-**`frank-zealot` reads up since 2026-10-18, and did not before.** llama.cpp answers **501 Not Implemented** on `/metrics` unless started with `--metrics`; the endpoint had no such flag, so `up{job="frank-zealot"}` was 0 across the whole retained history (the oldest sample, 2026-10-11, is already 0) while the service answered `/health` 200 the whole time. The `METRICS=1` knob in `scripts/frank-zealot.env` is the fix, and the target is up as measured 2026-10-18. Read `lastError` before believing an outage: a target can read down beside a healthy service.
+**`frank-zealot` reads up since 2026-10-18, and did not before.** llama.cpp only answers `/metrics` when started with `--metrics`, so `up{job="frank-zealot"}` was 0 across the whole retained history (the oldest sample, 2026-10-11, is already 0) while the service answered `/health` 200 the whole time; the flag and its opt-out are on [[frank-zealot]]. Read `lastError` before believing an outage: a target can read down beside a healthy service.
 
 Alert state is a live property, not a page fact: `curl -s http://127.0.0.1:9090/api/v1/alerts` returned no alerts at the 2026-10-18 morning census, one firing `LlmPrefixCacheCollapse` (info, `frank-zealot`, `activeAt` 11:10:48Z) minutes later, and none again by 11:56Z.
 
-A scrape cannot express "is this HTTP service answering?" — that is what a
-probe is for, and the remedy is the **blackbox exporter** in probe mode
-(`/probe?target=…&module=http_2xx`), which also gives the guarded vhosts a real
-uptime signal since Prometheus cannot solve an Anubis challenge itself. Until it
-is deployed, these three stay red for a structural reason: deleting them would
+The remedy for a probe-shaped question is the **blackbox exporter**; the
+classification rule and the honesty law for it live in [[Stale Signal Audit]].
+Until the exporter is deployed, these three stay red for a structural reason:
+deleting them would
 hide a real service, and fixing them by publishing databases or enabling metrics
 would trade a wrong signal for a real exposure. Targets are `host:port`, never
 URLs.

serverAppConfigs-docs/LLM/DFlash DSpark Speculative Decoding.md

@@ -14,7 +14,7 @@ timestamp: 2026-09-03
 
 DFlash and DSpark are the two newest speculative decoding methods. They draft many tokens in one parallel pass instead of one token at a time. This reduces decode latency on memory-bound workloads.
 
-DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both have draft models for the Qwen3.6 models that Dankserver and Dankclaw run.
+DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both publish draft models for the Qwen3.5/Qwen3.6 families (table below); what any of them pairs with on this fleet is an era on [[frank-dragoon]] or [[frank-zealot]].
 
 ## How DFlash Works
 
@@ -115,12 +115,12 @@ unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
 - Internal order is fixed (ngram types first, then drafters); CLI order is
   irrelevant.
 - **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod`
-  on the qwen35 hybrid (frank-dragoon era) measured −15 % prose / −7 %
-  block-copy (2026-10-03): adding the ngram layer logs `recurrent state
-  rollback is not compatible with 'ngram-mod' - disabling rollback support`,
-  and MTP verification depends on that rollback, so acceptance drops on ALL
-  traffic even where the matcher fires. Measure chains per architecture,
-  never by analogy.
+  on the qwen35 hybrid loses on both traffic shapes (2026-10-03): adding the
+  ngram layer logs `recurrent state rollback is not compatible with
+  'ngram-mod' - disabling rollback support`, and MTP verification depends on
+  that rollback, so acceptance drops on ALL traffic even where the matcher
+  fires. The measured size of each loss is [[frank-dragoon]]'s (Laws). Measure
+  chains per architecture, never by analogy.
 - ngram-mod prints NO startup banner — the load log shows only the drafter's
   `adding speculative implementation` line. Never conclude "chain not
   engaged" from the load log; identify behaviourally (`draft_n` > 0 only
@@ -198,7 +198,10 @@ Compatible MTP-only draft GGUFs on Hugging Face:
 
 Speculative decoding is lossless under rejection sampling: the target verifies every draft token. A draft head from the base model works with a derivative target; wrong proposals just lower the acceptance rate. The accepted MTP draft lengths are 2-3 tokens; the "MTP Defaults Are a Trap" writeup (thefrontierlab.ai) documents the tuning pitfalls.
 
-On Dankclaw, `draft-dflash` remains the benchmarked winner (20.2 t/s at 4k vs 16.9 for DFlash+ngram and 8.7 for ngram-only). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
+On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
+at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
+three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
+the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
 
 ## Draft Models
 
@@ -213,13 +216,12 @@ DFlash draft model sizes for the models on these servers:
 
 The DFlash draft is not a standalone model. It must pair with its target model. The model cards recommend block size 8 for higher concurrency and 16 for single-stream throughput.
 
-## Deployment on Dankserver (NVIDIA RTX 3090)
+## DFlash on an RTX 3090 (Ampere) — the archived vLLM path
 
-The Dankserver llama.cpp deployment (`frank-zealot`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
-
-> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-zealot` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
-
-The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
+DFlash was tried on the 3090 line, lost there to the ngram chain, and was never
+kept deployed; that card's payloads, shapes and rates are [[frank-zealot]]'s
+(its Eras rows are the llama.cpp arms the draft was tested against). What
+transfers is the Ampere arithmetic — compute capability 8.6, so:
 
 - FP8 KV cache is valid on CUDA 12.
 - FP8 weights compute through a fallback path; no native FP8 tensor cores.
@@ -248,11 +250,15 @@ docker run --rm --name vllm-qwen36-dflash \
     --gpu-memory-utilization 0.95
 ```
 
-## Deployment on Dankclaw (AMD Strix Halo gfx1151)
+## DFlash and DSpark on Strix Halo (gfx1151) — what killed them
 
-`frank-dragoon` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[kingjones-strix-rocmfp4]].
-
-History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
+Both context-backed drafters died the same way on the iGPU: under parallel load.
+DSpark took `ROCm error: an illegal memory access` inside the shared draft
+context with two concurrent 16k prompts; DFlash corrupted multi-slot into
+premature EOS and was retired first. The timeline, the crash class, the
+"concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
+are on [[qwen3.8-spec-matrix]]; the shape they were measured on is
+[[frank-dragoon]]'s mid-August 2026 era.
 
 ## A/B protocol on a live llama.cpp endpoint
 
@@ -315,7 +321,7 @@ not switch without re-measuring".
 
 The A/B vehicle for ANY single-GPU llama.cpp knob (batch/ubatch, KV type,
 slot split, quant rung — not just spec flags) is the transient systemd
-drop-in procedure in [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]].
+drop-in procedure in [[frank-dragoon]] and [[frank-zealot]].
 
 ## Is it worth it? (arithmetic before deployment)
 
@@ -359,16 +365,18 @@ pay for K sequential forwards.
 Trained heads are target-specific: EAGLE/DFlash consume the TARGET's hidden
 states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.
 
-**Step 4 — acceptance reality checks.** Fleet datum: a trained z-lab DFlash2
-at depth 7 accepted ~41 % on its matched target (`draft-dflash 7 → 43.5
-tok/s`); treat 40–60 % as the honest band for a matched pair. Cheapest proxy
-for `p` at temp 0 when both models are greedy and share a tokenizer: greedy
-continuation agreement on identical prompts, two API calls, no weights moved.
-llama.cpp can also synthesize the projection without any draft artifact:
+**Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
+matched pair; the arm that anchors the bottom of it on this fleet is the
+depth-7 `draft-dflash` row of [[frank-medium-llamacpp-dflash2]]'s spec table
+(accepted share, rate and VRAM for every depth are in that table). Cheapest
+proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
+greedy continuation agreement on identical prompts, two API calls, no weights
+moved. llama.cpp can also synthesize the projection without any draft artifact:
 `--spec-synth-rates P0,P1,...` / `--spec-synth-len L`. Speculation pays IN
-PROPORTION TO HOW SLOW THE TARGET IS — the same 41 % gave 2.3x on an
-18.8 tok/s layer-split target and a loss on a 25 tok/s one; fast targets are
-usually sparse MoE, the worst case for drafting.
+PROPORTION TO HOW SLOW THE TARGET IS — that band's midpoint was worth 2.3× on
+the slow two-card layer-split target and a LOSS on the 25 tok/s target of the
+worked example above; fast targets are usually sparse MoE, the worst case for
+drafting.
 
 **Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
 engines refuse external drafters by design: halogen-flash-server's
@@ -403,7 +411,7 @@ measurement that would settle it.
 
 - Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
 - DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
-- The old `--swap-space` flag no longer exists in current vLLM. Use `--kv-offloading-size`.
+- The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by [[RAM CPU Offloading in VLLM]].
 - `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
 - Use `rejection_sample_method` default `strict` for lossless output.
 
@@ -419,4 +427,4 @@ measurement that would settle it.
 - [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
 - [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
 - [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
-- [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — the single-GPU env drop-in A/B vehicle
+- [[frank-dragoon]] and [[frank-zealot]] — the single-GPU env drop-in A/B vehicle

serverAppConfigs-docs/LLM/Dankclaw Improvements.md

@@ -17,66 +17,59 @@ Dates: researched 2026-08. Hardware is verified on the live machine unless
 marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
 
 **Live today (2026-10-18).** The era described below — llama.cpp on the 8060S
-iGPU behind :7770 — is retired. Dankclaw now serves
-[[frank-dragoon-flash-server-v2-native]] (halogen-flash-server 0.16.2, port
-7770) and [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork on
-the RTX 3090, port 7771 — drift today: that unit is `failed` since 2026-10-18
-03:40 and a hand-run 9B-MTP process answers the port), and boots `iommu=pt` since 2026-10-18
-([[dankclaw-iommu-translation-off]]). The numbers below are the measurements of
-the era they were taken in; the `ttm` ceiling and the NVMe scheduler rows were
-re-checked 2026-10-18 and still hold.
+iGPU — is retired; what runs on this box now (both endpoints, their engines and
+ports) is [[frank-dragoon]]'s and [[frank-zealot]]'s pages, and both run under their
+repo units today — no drift open as of 2026-10-19 (`frank-zealot` active since
+2026-10-18 21:06, the deployer's 2-slot reslice; `frank-dragoon` active). The OS facts
+are this page's: `iommu=pt` since 2026-10-18 ([[dankclaw-iommu-translation-off]]), and
+the `ttm` ceiling and NVMe scheduler rows below were re-checked 2026-10-18 and
+still hold. Every benchmark number below is a measurement of the era it was
+taken in.
 
 ## Corrections (re-verified 2026-08-15/16)
 
-- **Backend**: that era's production was **Vulkan0**, not ROCm. Measured on this box
-  (b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx
-  Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs
-  302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not
-  hold on the 8060S. See [[frank-dragoon-benchmark]].
+- **Backend**: that era's production was **Vulkan0**, not ROCm. On this box ROCm
+  wins prefill and Vulkan wins decode, and the decode gap widens with context:
+  the 64k A/B is [[qwen3.8-strix-optimizations]] §4, the 262k A/B is
+  [[qwen3.8-spec-matrix]] §High-Context Backend Comparison. The "HIP
+  out-performs Vulkan for decode" claim did not hold on the 8060S.
 - **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
   does not exist on this machine (checked 2026-08-15). GPU DPM is already
   `performance`, forced level `auto`. This item is N/A.
-- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). The law
-  still holds on `frank-zealot` (`CACHE_TYPE_K=q8_0`, `CACHE_TYPE_V=q8_0` —
-  `scripts/frank-zealot.env:55-56`).
+- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). What
+  `frank-zealot` runs now is its own Current shape — this row is not it.
 - **Spec**: DFlash/DSpark are retired. That era's production was `--spec-default`
-  (ngram-mod) with draft-mtp first. See [[qwen3.8-spec-matrix]]. Today `frank-zealot`
-  is configured with speculation **off** (`SPEC_TYPE=none` — `scripts/frank-zealot.env:67`).
-- **TTM**: raised to 114 GiB on 2026-08-25, set live (no reboot) and
-  persisted in `/etc/modprobe.d/ttm.conf`. The effective VRAM ceiling that
-  once held the service at ~15 GiB is gone.
+  (ngram-mod) with draft-mtp first — timeline on [[qwen3.8-spec-matrix]]; the
+  spec setting each endpoint runs today is on its own page.
+- **TTM**: the `pages_limit` raise that removed the ~15 GiB ceiling is recorded
+  once on [[Dankclaw System Specs]] (Memory).
 
 ## Verified Memory Picture (measured 2026-08-11)
 
 | Figure | Value | Source |
 | ------ | ----- | ------ |
 | OS physical RAM | 30.5 GiB | `/proc/meminfo` |
-| TTM pages_limit | 114 GiB (29,883,536 x 4KiB) — raised 2026-08-25 | `/sys/module/ttm/parameters/pages_limit` |
 | ROCm VRAM total | 96.0 GiB | `rocm-smi --showmeminfo vram` under the then firmware VGM carve |
 | ROCm VRAM used | 76.2 GiB | `rocm-smi` while serving 3x200700 |
 
 That 2026-08 reading is the firmware-VGM era: the BIOS carved ~96 GiB as
 "dedicated" iGPU VRAM, so the kernel saw only ~30 GiB (the same split is
-recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT). The OS now accounts
-**121 GiB** (`MemTotal 127108080 kB`, 2026-10-18) and `rocm-smi
---showmeminfo vram` reports **2.0 GiB** (2,147,483,648 B) of dedicated VRAM —
-the carve is gone and the GPU budget is the GTT aperture, not the SMI figure
-([[Dankclaw System Specs]]).
-The TTM row above is still live: `pages_limit = 29883536` (114 GiB) and
-`/etc/modprobe.d/ttm.conf` still carries the `options ttm` line (both read
-2026-10-18).
+recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT; the 76.2 GiB row is an
+era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`,
+the ~114 GiB GTT aperture that is the real GPU budget, the 2.0 GiB dedicated
+carve and the `ttm` page cap — belongs to [[Dankclaw System Specs]], re-read
+2026-10-18.
 
 This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
-limits documented in [[frank-dragoon-benchmark]].
+limits documented in [[frank-dragoon]].
 
 ## 1. Self-Hosted Agentic Tools
 
-All of these connect to the local OpenAI-compatible endpoint
-(`http://127.0.0.1:7770/v1`). Today that port is **halogen-flash-server**, not a
-llama.cpp server, and the address must be the IPv4 literal: `curl
-http://localhost:7770/v1/models` returns nothing while
-`http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18 — the rootless
-podman publish is IPv4-only). None need NVIDIA.
+All of these connect to the local OpenAI-compatible endpoint on
+`:7770` (which engine owns that port is [[frank-dragoon]]'s to say), and the
+address must be the IPv4 literal: `curl http://localhost:7770/v1/models` returns
+nothing while `http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18
+— the rootless podman publish is IPv4-only). None need NVIDIA.
 
 | Tool | Use | Endpoint config | Priority |
 | ---- | --- | --------------- | -------- |
@@ -90,12 +83,13 @@ podman publish is IPv4-only). None need NVIDIA.
 | Qdrant | Vector store for RAG | Standalone, CPU-native | Low |
 
 The pragmatic first installs are Open WebUI, Cline, and Aider. They all work
-with the endpoint on 7770 and need no extra GPU. The 7770 endpoint today is
-halogen-flash-server, not llama.cpp.
+with the endpoint on 7770 and need no extra GPU.
 
 ## 2. Inference Runtime Tuning (llama.cpp on gfx1151)
 
-Verified against the running service and llama.cpp source.
+Verified against that era's running service and llama.cpp source; the era is
+retired ([[frank-dragoon]]), so this table is history and a method, not a live
+TODO list.
 
 | Change | Recommendation | Priority |
 | ------ | -------------- | -------- |
@@ -104,8 +98,8 @@ Verified against the running service and llama.cpp source.
 | Flash attention | Leave `--flash-attn auto`. It helps prompt processing, not decode. | Keep |
 | Load mode | Start with mmap (default). Enable `--mlock` only if page-fault stutter appears. | Optional |
 | Threads | Use `--threads 12 --threads-batch 16 --threads-http 1` for 16 Zen5 cores + 3 slots. Frees cores for the ROCm driver. | Medium |
-| DFlash n_max | Current 8 (block 16) is high. Try `--spec-draft-n-max 5`; it may reduce rejection cost at the same throughput. | Verify |
-| ngram layering | Measured slower than DFlash alone at 4k (16.9 vs 20.2 t/s). Not enabled. | Keep off |
+| DFlash n_max | Depth 8 (block 16) was high; a shallower draft cuts rejection cost at the same throughput. Sweep depth per the A/B protocol in [[DFlash DSpark Speculative Decoding]]. | Verify |
+| ngram layering | Layering ngram on the DFlash draft measured slower than the draft alone at 4k — figures on [[DFlash DSpark Speculative Decoding]]. | Keep off |
 
 ## 3. Server / OS / Hardware Settings
 
@@ -113,7 +107,7 @@ All require root and a reboot for the modprobe change.
 
 | Setting | Action | Effect | Priority |
 | ------- | ------ | ------ | -------- |
-| TTM GPU pool | `/etc/modprobe.d/ttm.conf` → `options ttm pages_limit=29883536 page_pool_size=29883536` (applied 2026-08-25, set live then persisted) | Lifts the 15 GiB cap to 114 GiB; the full 262k KV + Q8 AGENT payload fit in one slot | Done |
+| TTM GPU pool | Raised 2026-08-25; the knob, its value and its persistence file are [[Dankclaw System Specs]]'s (Memory row) | Lifted the ~15 GiB kernel-page ceiling so a full 262k KV plus a ~30 GB payload fits one slot | Done |
 | Power profile | **N/A on this machine** (2026-08-15; re-checked 2026-10-18: no `power_profile` under `/sys/class/drm/card0/device/` or `card1/device/`); GPU DPM already `performance` | none | ~~Medium~~ |
 | NVMe scheduler | Already `none` (`[none] mq-deadline` in `/sys/block/nvme0n1/queue/scheduler`, re-read 2026-10-18) | No change | Done |
 | BIOS VGM | Set Variable Graphics Memory / UMA to "High" in the EVO-X2 BIOS | Max dedicated VRAM for the GPU carve-out; verify in the live BIOS | Verify |
@@ -130,7 +124,7 @@ All require root and a reboot for the modprobe change.
 
 ## Sources
 
-- [[frank-dragoon-benchmark]] — measured backend/spec numbers
+- [[frank-dragoon]] — measured backend/spec numbers
 - [[deployment]] — deploy and update process
 - AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
 - llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
@@ -139,7 +133,7 @@ All require root and a reboot for the modprobe change.
 
 ## See Also
 
-- [[frank-dragoon-benchmark]]
+- [[frank-dragoon]]
 - [[DFlash DSpark Speculative Decoding]]
 - [[deployment]]
 - [[index]]

serverAppConfigs-docs/LLM/Dankclaw System Specs.md

@@ -10,7 +10,7 @@ timestamp: 2026-09-03
 
 ## Server Overview
 
-Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[ciru-activefpx-promptforge]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon-flash-server-v2-native]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
+Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
 ## Hardware Specifications
 
 ### CPU
@@ -46,7 +46,7 @@ Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp
 | Runtime | None installed. FastFlowLM 0.9.45 + XRT (installed 2026-08-16) was **retired 2026-09-11** with the `amd_iommu=off` decision — see [[dankclaw-iommu-translation-off]]; the package remains at `/home/dank/GitPackages/fastflowlm_0.9.45_ubuntu26.04_amd64.deb` |
 | Status | Not in service — no NPU endpoint is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. See [[FastFlowLM NPU Inference]] |
 
-The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see [[frank-zealot-llama-prism-bonsai2-pq20]]); the Gemma4-12B QAT rung it once pointed at is retired history ([[frank-zealot-gemma4-12b]], 2026-09-17 → 10-01).
+The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see [[frank-zealot]]); the Gemma4-12B QAT rung it once pointed at is retired history ([[frank-zealot]], 2026-09-17 → 10-01).
 
 ### ROCm
 
@@ -66,6 +66,7 @@ The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name
 | System RAM | 121 GiB (shared CPU + GPU; `free -g` total 121, `MemTotal 127108080 kB`, 2026-10-18) |
 | GPU-Visible (ROCm) | ~114 GiB GTT aperture (`mem_info_gtt_total` 122,402,963,456 B — the budget) |
 | Swap | 64 GB `/swap.img` (`swapon`, 2026-10-18; `free` reports 63 GiB) |
+| TTM page cap | `ttm.pages_limit = 29883536` (114 GiB of 4 KiB pages) and `page_pool_size=29883536`, persisted in `/etc/modprobe.d/ttm.conf`, applied live 2026-08-25 and re-read 2026-10-18 (`/sys/module/ttm/parameters/pages_limit`). The kernel default is half of RAM, so in the firmware-VGM era (~30 GiB visible) the cap was ~15 GiB and a 28 GB payload plus a 262k KV cache could not load — this is the row every other page should cite |
 | /dev/shm | 61 GiB tmpfs (half of RAM, `df /dev/shm`, 2026-10-18) |
 
 ### Storage
@@ -95,21 +96,21 @@ cable between claw and server is required for 10G.
 | Kernel | 7.0.0-38-generic (`uname -r`, 2026-10-18) |
 | Boot line | `iommu=pt` since 2026-10-18 (`/proc/cmdline`, no `amd_iommu` token); eras before: `amd_iommu=off` 2026-09-11 → 10-18, IOMMU default-on before that — [[dankclaw-iommu-translation-off]] |
 | Python | 3.14.4 (system, `python3 --version`, 2026-10-18). The 3.12.13 CIRU venv left with that runtime (retired 2026-08-22) |
-| CIRU runtime | ciru-ai/ROCmFPX — **retired 2026-08-22** ([[ciru-activefpx-promptforge]]); not the serving runtime |
+| CIRU runtime | ciru-ai/ROCmFPX — **retired 2026-08-22** ([[frank-dragoon]]); not the serving runtime |
 | uv | 0.12.8 (`~/.local/bin/uv --version`, 2026-10-18) |
 
 ## Runtime Relevance
 
 - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
 - The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
-- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon-flash-server-v2-native]]) — and `frank-zealot` on 7771 ([[frank-zealot-llama-prism-bonsai2-pq20]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon-gufo-qwen38-27b]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
+- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
 - The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either.
 - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
 - 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage.
 
 ## See Also
 
-- [[ciru-activefpx-promptforge]]
+- [[frank-dragoon]]
 - [[Shared Memory Architecture Dankclaw]]
 - [[frank-redo-2026-08-17]]
 - [[FastFlowLM NPU Inference]]

serverAppConfigs-docs/LLM/Dankserver System Specs.md

@@ -16,7 +16,7 @@ timestamp: 2026-09-20
 ## Server Overview
 
 Dankserver is a media/library server and the `frank-archon` LLM host (radiance
-1.0.13, 2×R9700 tensor-parallel — [[frank-archon-radiance-qwen38-iq4r]]; the
+1.0.13, 2×R9700 tensor-parallel — [[frank-archon]]; the
 `frank-medium` name this page carried was retired 2026-09-20, see §Active
 Services). The host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that
 served the FreeToken/CUDA `frank-zealot` era was removed on 2026-09-15 and
@@ -101,7 +101,7 @@ RCCL-backed rebuild needs no new host packages.
 | Kernel | 7.0.0-38-generic, `iommu=pt` (`uname -r`, `/proc/cmdline`, read 2026-10-18) |
 | Python | 3.14.4 |
 | ROCm userspace | Ubuntu 7.1 packages in /usr (hipcc 7.1.1, rocminfo, rocm-smi 7.1.1) |
-| Serving stack | radiance 1.0.13 container (`stilldeadcode/radiance@sha256:4c3f484564…`), TP=2 across both R9700s — [[frank-archon-radiance-qwen38-iq4r]]. The llama.cpp `fb27a525` build at `~/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714` is the retired frank-medium artifact |
+| Serving stack | radiance 1.0.13 container (`stilldeadcode/radiance@sha256:4c3f484564…`), TP=2 across both R9700s — [[frank-archon]]. The llama.cpp `fb27a525` build at `~/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714` is the retired frank-medium artifact |
 
 The stale AMD-repo ROCm 6.3.3 packages (167, `~24.04`-suffixed) that used to
 hijack `libhsa-runtime64` were purged on 2026-09-15 — see
@@ -113,7 +113,7 @@ The active inference service is `frank-archon.service` (`systemctl is-active`
 `active`, `is-enabled` → `enabled`, read 2026-10-18): radiance 1.0.13 in a
 host-network podman container answering `curl 127.0.0.1:7770/health` →
 `{"status":"ok","model":"frank-archon",…}`. See
-[[frank-archon-radiance-qwen38-iq4r]].
+[[frank-archon]].
 
 The services this page once called active are gone: `frank-medium.service`
 (llama-server, 32:33 layer split + DFlash2 drafter) is `not-found` — that rung
@@ -146,7 +146,7 @@ unverified. See [[deployment]].
 
 ## See Also
 
-- [[frank-archon-radiance-qwen38-iq4r]] — the active serving runbook
+- [[frank-archon]] — the active serving runbook
 - [[Dual-R9700-Cooperation]] — why the PCIe path matters, and what the swap changed
 - [[deployment]] — the deploy flow
 - [[Shared Memory Architecture Dankclaw]] — for contrast, Dankclaw's unified memory

serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md

@@ -16,7 +16,7 @@ timestamp: 2026-09-16
 
 Dankserver's two R9700s run **one tensor-parallel engine across both cards** —
 today that is `frank-archon` under radiance 1.0.13, TP=2
-([[frank-archon-radiance-qwen38-iq4r]]). When this page was written the endpoint
+([[frank-archon]]). When this page was written the endpoint
 was `frank-medium`: first llama.cpp's layer split
 ([[frank-medium-llamacpp-dflash2]], 2026-09-16 → 09-20), then from 2026-09-20
 one vLLM-radiance process at `tensor_parallel_size 2`
@@ -51,7 +51,7 @@ What that bought:
   decode and 3.6x at 4-way concurrency, and it replaced the layer split on the day
   of measurement. The payload has moved since: the MXFP4 `frank-medium` shape is
   retired 2026-09-26, and what runs today is the iq4r publisher under radiance
-  1.0.13 ([[frank-archon-radiance-qwen38-iq4r]]).
+  1.0.13 ([[frank-archon]]).
 - **The drafter, not the collective, is what llama.cpp lost.** Its tensor mode
   could not carry the DFlash2 drafter, which is the honest caveat on the
   comparison below.
@@ -147,10 +147,10 @@ LAN/SATA/WiFi:
   they were in separate IOMMU groups (15 and 20 — re-read 2026-10-18: still 15
   and 20), and both root ports had ACS `ReqRedir+`, so this page inferred P2P
   would hairpin at the root complex. **That inference is superseded:** the live
-  engine says the opposite about the same pair — "I device: the 2 cards reach
-  each other's memory directly (peer to peer)" (`podman logs frank-archon`,
-  2026-10-15 boot). Plain P2P traffic measured 45.27 GB/s bidirectional on the
-  *old* board — DMA was never the gate, atomics were.
+  engine reports the two cards reaching each other's memory directly, which
+  [[frank-archon]] and [[Dankserver System Specs]] record. Plain P2P traffic
+  measured 45.27 GB/s bidirectional on the *old* board — DMA was never the gate,
+  atomics were.
 - Kernel 7.0.0-31-generic with `iommu=pt` then; this host runs 7.0.0-38-generic
   with `iommu=pt` today (`/proc/cmdline`, read 2026-10-18). Both were ruled out
   as causes ([above](#ruled-out-empirically-do-not-re-derive)) and this host's
@@ -163,10 +163,9 @@ launched since the swap. Until the 2-rank probe runs — or a `--split-mode row`
 boot that does *not* log `internal AllReduce init failed ... falling back to
 meta-backend butterfly` — "collectives work here" was a strong expectation, not a
 measurement. **RCCL never got that measurement**; what settled the operational
-question instead is production traffic: radiance's TP=2 wire moves cross-rank
-all-reduces over the direct P2P path and reports its own precision — "tp wire:
-LOSSY (--tp-wire wht6) at 128 KiB a message and above … Steps of at most 25 rows
-are exact" (`podman logs frank-archon`, 2026-10-15 boot).
+question instead is production traffic — radiance's TP=2 wire carries cross-rank
+all-reduces over the direct P2P path and prints its own precision, and the
+measured `wht6` banner with its exact-step bound is on [[frank-archon]].
 
 ## Measured matrix
 
@@ -233,35 +232,34 @@ landed, at x8/x8, clears the arithmetic with room to spare.
 
 Both cooperation modes were run on this host on 2026-09-20 — the same salted
 10-prompt battery (`scripts/bench-conversation.py`), the service stopped and
-masked between arms. Decode/prefill are geometric means over the ten rows;
-burst-4 is four concurrent copies of one prompt, reported as aggregate tokens/s.
+masked between arms. The arm table is [[frank-medium-radiance-mxfp4]]'s bench of
+record (arms R = radiance TP=2 + DFlash2 as deployed, I = the llama.cpp layer
+split that preceded it, L = llama.cpp `--split-mode tensor` against RCCL); this
+page keeps only the verdict. **TP=2 won on every axis — roughly 2.1× decode and
+3.6× at 4-way concurrency over the layer split — and replaced the layer split the
+day of measurement.**
 
-| Shape | Cross-device path | Decode | TTFT | Prefill | burst-4 |
-|---|---|---|---|---|---|
-| **radiance TP=2 + DFlash2 — deployed** | R4D one-shot allreduce | **95.8 t/s** | **0.21 s** | **1493.6 t/s** | **259.4 t/s (65.6/stream)** |
-| llama.cpp layer split 32:33 + DFlash2 — predecessor | none, boundary activation copy | 45.7 t/s | 0.72 s | ~530 t/s | 71.7 t/s |
-| llama.cpp `--split-mode tensor` 1:1 | RCCL allreduce | 30.5 t/s | 0.71 s | n/r | 54.3 t/s |
-
-Row 3 carries a caveat that no flag can remove: llama.cpp's tensor mode could not
-load the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
+Arm L carries a caveat no flag can remove: llama.cpp's tensor mode could not load
+the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
 pre-allocated on the non-primary device has no backend once `n_copies == 1`), so
-it decoded **without** speculation while the other two rows decoded with it. The
-comparison that decides anything is row 1 against row 2, and some of that gap is
-the 4-bit MXFP4 checkpoint rather than the parallelism.
+it decoded **without** speculation while the other arms decoded with it; and part
+of R's margin over I is the 4-bit MXFP4 checkpoint, not the parallelism.
 
 Two numbers elsewhere on this page are therefore not interchangeable:
 
 - The 272 t/s recorded 2026-09-16 is **prefill**, on a **butterfly** allreduce,
-  with RCCL off, on the old board. It was never a TP decode number.
-- 26.3 t/s from the same table is that butterfly tensor's decode. Row 3's 30.5 is
-  an RCCL tensor decode a day later: same shape family, different collective path.
+  with RCCL off, on the old board. It was never a TP decode number, and its
+  paired decode figure is in "The third path" below.
+- The butterfly tensor arm and the RCCL tensor arm a day later are the same shape
+  family on different collective paths; the measured pair lives with
+  [[frank-medium-radiance-mxfp4]]'s arms.
 
 RCCL needs PCIe atomics on *every* participating GPU — the gate the old board
 failed. vLLM's R4D one-shot allreduce dispatches no collective at all, which is
 why TP=2 runs today rather than merely "being unblocked": the image ships TheRock
 RCCL, and TheRock carries no gfx1201 device code (`invalid device function`), so
 radiance could not have used RCCL on this host even with atomics routed. Ubuntu's
-RCCL 2.27 does carry gfx1201 kernels, and that is what row 3 linked against.
+RCCL 2.27 does carry gfx1201 kernels, and that is what arm L linked against.
 
 **Rule for any future multi-GPU collective experiment: stop and mask the
 production service first**, per the benchmarking policy, or run it on a host that
@@ -292,12 +290,11 @@ on the second card, so the atomics gate never comes up.
 
 That is what `frank-medium` ran from 2026-09-16 to 09-20
 ([[frank-medium-llamacpp-dflash2]]): one process, one model, both cards,
-262144 context. Measured against tensor split
-on this host: 1991 t/s prefill / 18.8 tok/s decode layer-split, against 272 /
-26.3 tensor (butterfly fallback, no RCCL). **That tensor number is not a
-prediction for an RCCL-backed build** — it is what tensor split costs when the
-allreduce degrades to a meta-backend butterfly, and it is the number the
-board-swap path exists to retire.
+262144 context. Its measured layer-split rates are that page's; this host's
+tensor arm under the butterfly fallback measured 272 t/s prefill / 26.3 tok/s
+decode. **That tensor number is not a prediction for an RCCL-backed build** — it
+is what tensor split costs when the allreduce degrades to a meta-backend
+butterfly, and it is the number the board-swap path exists to retire.
 
 The ranking therefore changed twice. On 2026-09-16: software already gets both
 cards onto one model, so the hardware fix stopped being a capability question

serverAppConfigs-docs/LLM/FastFlowLM NPU Inference.md

@@ -24,7 +24,7 @@ timestamp: 2026-09-03
 > usable again since the 2026-10-18 `iommu=pt` boot (`amdxdna` bound,
 > `/dev/accel/accel0` present — [[dankclaw-iommu-translation-off]]), but no
 > service runs on it. `frank-zealot` today is the llama.cpp PrismML endpoint on
-> the RTX 3090 ([[frank-zealot-llama-prism-bonsai2-pq20]]). See [[frank-npu]]
+> the RTX 3090 ([[frank-zealot]]). See [[frank-npu]]
 > for the retirement record. Everything below is the 2026-08 research record.
 
 # FastFlowLM NPU Inference
@@ -57,7 +57,10 @@ Key properties, per the project README and docs:
 
 ## NPU Hardware on Dankclaw
 
-Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU rated at 50 TOPS (vendor spec: "NPU TOPS — Up to 50 TOPS", [amd.com product page](https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html)). It sits beside the Radeon 8060S iGPU and shares the same unified memory pool — 30 GiB exposed to the OS in the 2026-08 reading below; the box presents **121 GiB** today (`free -g` total 121, read 2026-10-18).
+Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU
+with 50 TOPS of compute and, crucially, **no dedicated VRAM** - it shares system memory
+via the IOMMU SVA mapping. The budget this page reasons about - 29 GiB usable at
+4800 MT/s LPDDR5X - is the 2026-08 host state; the live figure is [[Dankclaw System Specs]].
 
 Live verification on Dankclaw, 2026-08-16:
 
@@ -89,10 +92,9 @@ Budget math for this fleet:
 | Dankserver (Ryzen 7 7700X) | 62 GB | no NPU |
 | FLM bench box (Kraken Point) | 32 or 96 GB | ~16 or ~48 GB |
 
-2026-10-18 re-read: Dankclaw's OS now reports **121 GiB** (`free -g`) and
-Dankserver **246 GiB** (`free -g`), so the half-of-RAM rule would price the NPU
-budget at ~60 GiB and the 7700X box still has no NPU. The 30 GiB / 62 GB rows
-are the 2026-08-16 state; the table is moot while no NPU endpoint is deployed.
+The half-of-RAM gate scales with host RAM - live figures in [[Dankclaw System Specs]]
+and [[Dankserver System Specs]]. The live gate on Dankclaw is still `flm validate`
+(it owns the current number).
 
 Weight sizes (approximate, Q4-class):
 
@@ -159,7 +161,10 @@ Qwen3.6-35B-A3B (Q4_KS), FLM v0.9.45, Kraken Point **96 GB DRAM**:
 | Decode t/s | 13.65 | 13.41 | 13.09 | 12.51 | 11.24 | 9.51 |
 | Prefill t/s | 78.98 | 118.04 | 156.43 | 197.93 | 218.84 | 221.96 |
 
-Context: the current PP split (head + 3090 worker) decodes at 14-18 t/s for the 27B dense model. A 35B-A3B at ~10-14 t/s on an NPU with 48 GB of RAM budget is not a speed win over the existing topology.
+- PP and NPU together cost 2.2 GiB + NPU model + ~2 GiB of headroom. The then-current
+  PP split (head + 3090 worker, 2026-08 era - see [[frank-redo-2026-08-17]]) decoded at
+  14-18 t/s for the 27B dense model, so an NPU offload would free the head or upgrade
+  the worker's model.
 
 The vendor also claims "over 10x more power-efficient" versus GPU. That is a marketing claim. We have not measured it.
 
@@ -219,7 +224,12 @@ Notes:
 
 ## Promotion (2026-08-16)
 
-`frank-zealot` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
+`frank-zealot` then ran the NPU endpoint on Dankclaw (2026-08; the name was reused
+twice after this window - see the banner - and the NPU unit today is `frank-npu`,
+masked). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`),
+wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`.
+`LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap
+`FLM_MODEL` in the env to switch.
 
 ### Boot policy (2026-08-17)
 
@@ -256,11 +266,13 @@ The 4-stream wall times form a near-perfect 10.7 s ladder. Each stream self-repo
 Consequence: the NPU endpoint is a single lane. It suits low-frequency background work (triage, summarization, agent side-tasks). It does not suit interactive chat for several simultaneous users. The 27B llama.cpp service, by contrast, runs PARALLEL=2 with true concurrent slots.
 
 ## Test Runbook
-
 This runbook targets a retired endpoint; keep it as the procedure if the NPU is
-ever revived. Do not run large model loads while the production service holds
-most of the unified pool — 30 GiB exposed to the OS in the 2026-08 reading,
-**121 GiB** today (`free -g`, read 2026-10-18). Check free RAM first, or stop
+ever revived.
+
+**Host RAM availability.** The gate is half of system RAM by default - a relic of the
+install: the endpoint ran on a machine that presented 30 GiB of system RAM; on today's
+box that ceiling is far larger (live figure: [[Dankclaw System Specs]]). Check free RAM
+first, or stop
 the service the way the bench policy requires (`sudo systemctl stop
 frank-dragoon && sudo systemctl mask frank-dragoon` — self-healing would
 otherwise restart it). The NPU shares the unified pool with everything else on

serverAppConfigs-docs/LLM/GGUF Compatibility.md

@@ -119,5 +119,6 @@ re-download.
 
 - [[Llama Cpp Runtimes]] — which fork ships which custom types
 - [[Llama Cpp Code Diff]] — the enum tables and code-level differences
-- [[frank-dragoon-rocmfpx-q8agent-mtp]] — the ftype-115 lock-in in production
+* [[kingjones-q8agent-rocmfpx]] — the era that served the ftype-115 file, and
+  [[frank-dragoon]] for what runs there now
 - [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads

serverAppConfigs-docs/LLM/Hybrid CPU GPU Processing Dankserver.md

@@ -10,7 +10,7 @@ timestamp: 2026-08-28
 
 ## Overview
 
-Dankserver has **no NVIDIA GPU**: the RTX 3090 left the box on 2026-09-15 and the host now runs two AMD Radeon AI PRO R9700s (`lspci | grep -ci nvidia` → 0, read 2026-10-18 — [[Dankserver System Specs]]). This page's vLLM-era hybrid guidance is reference material, not the serving path: the live service is radiance 1.0.13 as `frank-archon`, and its hybrid mechanism is a pinned host pool (43.88 GiB per rank, `scripts/frank-archon.env:184`) plus the CPU-resident 47.68 GiB n-gram/PLE table, not vLLM KV swapping ([[frank-archon-radiance-qwen38-iq4r]]).
+Dankserver has **no NVIDIA GPU**: the RTX 3090 left the box on 2026-09-15 and the host now runs two AMD Radeon AI PRO R9700s (`lspci | grep -ci nvidia` → 0, read 2026-10-18 — [[Dankserver System Specs]]). This page's vLLM-era hybrid guidance is reference material, not the serving path: the live service is radiance 1.0.13 as `frank-archon`, and its hybrid mechanism is a pinned host pool (43.88 GiB per rank, `scripts/frank-archon.env:184`) plus the CPU-resident 47.68 GiB n-gram/PLE table, not vLLM KV swapping ([[frank-archon]]).
 
 ## Architecture
 
@@ -56,7 +56,10 @@ Lower values leave more GPU memory for KV cache. This reduces swap frequency.
 
 ### Swap Space / KV Offload
 
-Recent vLLM releases removed the `--swap-space` engine argument — `CacheConfig` on `main` has no `swap_space` field ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/cache.py), read 2026-10-18). Use `--kv-offloading-size` (`kv_offloading_size`, same file) to move KV cache to CPU RAM. In Podman, mount the host cache read-write for KV offload writes. The table below shows historical `--swap-space` guidance as a sizing reference.
+Current vLLM has no `--swap-space` engine argument, and the CPU KV-offload knob pair
+is owned by [[RAM CPU Offloading in VLLM]] (that page carries the upstream
+`CacheConfig` verification). In Podman, mount the host cache read-write for KV offload
+writes. The table below shows historical `--swap-space` guidance as a sizing reference.
 
 | System RAM | Recommended Swap |
 |------------|------------------|

serverAppConfigs-docs/LLM/LLM Performance Diagnosis.md

@@ -20,8 +20,8 @@ Sweep decode across context depth. **Flat** ⇒ attention over history is not
 the cost; the problem is per-token expert movement and KV/attention tuning is
 ruled out entirely. A genuinely context-bound server shows a clear downward
 slope. Record cold vs warm separately — a small gap means the cache is
-working. Measured on the fp8 expert-offload shape: 12.11 tok/s FLAT from 304
-to 16,095 tokens.
+working. The canonical flat run (fp8 expert-offload shape, a few hundred to
+sixteen thousand tokens of context) is in [[Tcclaviger Fork Ops]]'s profile.
 
 ## 2. Duty-cycle arithmetic (the "GPU not fully utilized" answer)
 
@@ -161,7 +161,7 @@ of the same RAM, not extra memory.
 ## 7. IBS model-access attribution (AMD CPUs)
 
 Real CPU access patterns for mmap'd weights, live, no stop (established on
-frank-archon, 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
+[[frank-archon]], 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
 `scripts/frank-archon-expert-activity.py` (self-contained dump + sample +
 analyze; `--analyze-only`).
 

serverAppConfigs-docs/LLM/Llama Cpp Serving Sizing.md

@@ -23,8 +23,9 @@ guesswork.
    each near the target; record `nvidia-smi` used/total plus the boot log's
    `n_slots`/`n_ctx_slot`: shape A = current/cheap, shape B = new demand.
    `non-KV = used − (tokens × KiB/token)`; the two rows separate
-   weights/buffers/graphs from the new demand (a 918 MB projector bills
-   ~1.4 GiB).
+   weights/buffers/graphs from the new demand; a projector is billed after the
+   fit at roughly 1.5x its file size ([[frank-zealot]] Contract has that
+   measurement).
 3. **Solve the band:** `KV_tokens ≤ (card − non-KV − headroom_low) /
    KiB/token`, round the pool DOWN to a multiple of `PARALLEL × 4096` so every
    slot gets an equal integer window. Boot and measure; if outside the band,
@@ -54,8 +55,8 @@ First decide which kind of memory is in question:
 | device | kind | read it with |
 |---|---|---|
 | RTX 3090 (Dankclaw) | discrete GDDR6X | `nvidia-smi --query-gpu=memory.total,memory.used,memory.free` |
-| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` (114 GiB) vs `mem_info_vram_*` (2 GiB carve-out) |
-| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram` → 34,208,743,424 B = 32,624 MiB (31.86 GiB) per card, read 2026-10-18 — **not** the 64 GiB "SVM-backed" figure this row once carried ([[Dankserver System Specs]]) |
+| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` vs `mem_info_vram_*`; the aperture and carve-out values are [[Dankclaw System Specs]]'s |
+| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram`; the per-card byte/MiB value (and the stale "64 GiB SVM-backed" figure it replaced) is [[Dankserver System Specs]]'s |
 
 Per-process truth on NVIDIA:
 `nvidia-smi --query-compute-apps=pid,process_name,used_memory`. Never add the
@@ -63,9 +64,10 @@ iGPU's GTT to host RAM as a separate pool; `free -g` is the arbiter for
 co-tenants.
 
 - **Engine ledger first.** Halogen prints weights pinned (+ remaining 2 MiB
-  contiguous blocks — the honest fragmentation signal: 240 blocks/12.3 GiB
-  after reboot vs 475/14.7 GiB before means the model did not come back into
-  the same hole), device budget, KV pool/working split, "left for everything
+  contiguous blocks — the honest fragmentation signal: a block census that drops
+  across a restart means the model did not come back into the same hole; the
+  before/after census this was learned on is that era's, on [[frank-dragoon]]),
+  device budget, KV pool/working split, "left for everything
   else". llama.cpp prints them at `llama-model.cpp:1827` (per-backend model buffer)
   and `llama-context.cpp:692` (compute buffer) on the era's build; on today's
   `master` the same `LLAMA_LOG_INFO` lines sit at `llama-model.cpp:2009`
@@ -83,13 +85,13 @@ co-tenants.
   graph/compute pool).
 - **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
   "move layers to the card" converts reclaimable cache into device memory and
-  frees nothing for co-tenants. A served model's footprint ≠ weight size
-(a retired frank-zealot shape: 22.6 of 24.1 GiB for an 11.29 GiB model; the card
-today reads 24,576 MiB = 24.0 GiB total, `nvidia-smi --query-gpu=memory.total`,
-2026-10-18, and the deployed payload is the PQ2_0 pair
-[[frank-zealot-llama-prism-bonsai2-pq20]]). Do the capacity
-  subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
-  28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
+  frees nothing for co-tenants. A served model's footprint is not its weight
+  size: the MiMo Q5_K_M era booted a 6.40 GiB payload to **20,530 MiB used** of the
+  3090's addressable 24,576 — the Eras row and its addressable ≠ advertised law are on
+  [[frank-zealot]]. What the card reports used is weights + KV + compute buffers +
+  projector, never weights alone. Do
+  the capacity subtraction before designing, on the card number the owner page
+  quotes, not the advertised one.
 - **Context lengths: derive, never remember.** `llama-bench` computes
   `n_ctx = n_prompt + n_gen + n_depth` (defaults 512/128/0, type_k/v F16,
   flash_attn AUTO, split_mode LAYER). `-p 512,16384 -n 64` = contexts
@@ -112,10 +114,10 @@ them from the binary (`llama-server --help`) before sizing on them.
 **State law** (compute before trusting any cap): a state is the serialized KV
 of the whole sequence + constant recurrent state + the draft context's KV:
 `per_token = 2 × n_full_attn_layers × n_kv_heads × head_dim × bytes (K and V) +
-draft_per_token`. Measured on frank-medium (qwen3.5 hybrid, f16, MTP draft):
-16×4×256×2×2 = 64 KiB/token + 4 KiB draft = **68 KiB/token**; a full
-196,608-token slot state ≈ 12.8 GiB. Cross-check against the journal's
-`prompt state size` lines — the two must agree.
+draft_per_token`. The worked qwen35-hybrid instance of that formula (full layer
+count, f16 KV, MTP draft) and the slot-size table it prices are on
+[[frank-medium-llamacpp-dflash2]]; cross-check any such arithmetic against the
+journal's `prompt state size` lines — the two must agree.
 
 **The journal is the instrument:** `exceeds cache size limit … skipping` =
 cap binding in production; `making room … removing oldest` / `cache size
@@ -162,13 +164,13 @@ Only the charlie12345/ROCmFPX build-qwen38 (PR #27742 qwen4exp merged) takes
 it; CIRU, upstream and stock builds do not. Validate live: load with
 `--flash-attn on`, health 200, a real completion finishing `"stop"`, journal
 clean of `GGML_ASSERT`/`cudaMalloc`/`missing tensor`, and flash-attn engaged
-in the timings. Context cap for full-model residency on that shape stayed
-32,768.
+in the timings. The context cap that build reached for full-model residency is
+part of that era's record on [[frank-dragoon]].
 
 ## Related
 
 - [[GGUF Compatibility]] — what a payload is before it is priced
-- [[frank-zealot-llama-mimo-q6k-2x200704]] — the two-boot priced shape of the retired MiMo era (live shape today: [[frank-zealot-llama-prism-bonsai2-pq20]])
-- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
+- [[frank-zealot]] — the endpoint whose Eras rows are the two-boot priced shapes this method produced
+- [[frank-medium-llamacpp-dflash2]] — the state-law worked example behind section 3
 - [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
 - [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side

serverAppConfigs-docs/LLM/Model Swap and Rollback.md

@@ -40,9 +40,9 @@ curl -s <health>                                      # then a REAL completion
 ```
 
 Deploy caveat: "healthy after Ns" = systemd ACTIVE, not model LOADED —
-confirm with a real `/v1/chat/completions`. RTX 3090 (Ampere) has no
-FP8/FP4 tensor cores: FP8 weights run via upcast but must still fit; choose
-quant by size fit, then accept the compute path the card supports. Before
+confirm with a real `/v1/chat/completions`. On Ampere an FP8 file buys memory
+and no compute ([[Quantization by Hardware Architecture]]): choose the quant by
+size fit, then accept the compute path the card supports. Before
 choosing a smaller quant, state WHICH of the two you are buying — headroom,
 or a design change (a smaller quant does not change what the runtime can
 express; per-tensor cross-device placement stays refused under
@@ -60,8 +60,8 @@ express; per-tensor cross-device placement stays refused under
 ## 3. Fetch and verify (HF payloads)
 
 **Inventory by QUANT NAME, never by extension** — large quants ship as
-`.partNofM` fragments (Q4_K_M = 37.00+37.00+36.97 GiB), so an
-`endswith(".gguf")` filter makes Q4_K_S look nonexistent. Check both
+`.partNofM` fragments, so an `endswith(".gguf")` filter makes a whole rung look
+nonexistent (AGENTS.md carries the worked fragment ledger). Check both
 mradermacher repos (`…-i1-GGUF` imatrix, `…-GGUF` plain) and the model's own
 repo; the tree API's `blobs=true` gives `lfs.oid` per file. Gated mirrors
 return 403 even with a valid token until the account accepts the gate —
@@ -150,7 +150,7 @@ recoverable, drift is not.
    entry in `serverAppConfigs-docs/log.md`.
 2. `~/.omp/agent/models.yml`: `contextWindow` must equal the REAL per-slot
    context — llama.cpp logs `n_ctx_slot` per prompt (`--parallel N` divides
-   the pool; e.g. 262144 native / 2 slots = 131072 per slot); vLLM reports
+  the pool; e.g. a 524288 pool over 2 slots = 262144 per slot); vLLM reports
    `max-model-len` and its concurrency line. Overselling causes silent
    client-side overflow or a discovered-wall 400. Also `maxTokens` vs the
    engine's cap, `input` capabilities (text/image), and
@@ -252,12 +252,10 @@ small/fast + image roles at the new endpoint; move the vision role WITH the
 vision tower (one owner per capability). `shared/homepage/services.json`: a
 card per endpoint, probed from the monitor's own host. Prometheus: only add
 a job if the runtime actually exposes Prometheus text — and re-measure rather
-than repeat, because that answer moves. This bullet once said halogen serves
-`/health`, `/cache`, `/v1/*` "and nothing else"; measured 2026-10-18 on
-Dankclaw, `curl -s http://127.0.0.1:7770/metrics` returns Prometheus text
-(`# HELP llamacpp:prompt_tokens_total …`, content-type
-`text/plain; version=0.0.4`) while an invented path 404s — so the surface has
-grown and a scrape target is expressible again.
+than repeat, because that answer moves: halogen's HTTP surface has grown twice
+since this page was written, and the endpoint list that matters for a scrape
+target — including whether `/metrics` answers — is [[frank-dragoon]]'s wire
+contract.
 
 **4. Cutover, then verify by driving it.** `deploy.sh <target> --service
 <large> --service <small>` (order matters); per endpoint: `/health`,
@@ -265,10 +263,10 @@ grown and a scrape target is expressible again.
 `reasoning_content`, a `tools` request returns `tool_calls`, an image
 request is answered (or refused with a named reason), the runtime's own
 speculation counters move, and the unit is `enabled` and `active`. All three LLM
-endpoints are **root** units today (`User=dank` + `WantedBy=multi-user.target` —
-`scripts/frank-archon.service:30,71`, `scripts/frank-dragoon.service:8,31`,
-`scripts/frank-zealot.service:8,26`), so linger is not what gates them; linger
-matters for rootless quadlets. Read the engine's boot ledger for a silent clamp. Finally spend one
+endpoints are **root** units today (`scripts/frank-archon.service`,
+`scripts/frank-dragoon.service`, `scripts/frank-zealot.service`), so linger is
+not what gates them; linger matters for rootless quadlets. Read the engine's boot
+ledger for a silent clamp. Finally spend one
 real request through the harness per changed role — routing labels lie; the
 servers' own journals do not.
 
@@ -285,4 +283,5 @@ predecessor "stopped" (the port race above).
 - [[Llama Cpp Serving Sizing]] — pricing the shape once it loads
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — vLLM-side fit arithmetic
 - [[LLM Bench Windows and Measurement]] — measuring the new deployment
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — the retired tcclaviger vLLM endpoint (2026-09-29 → 10-05); the live Dankserver endpoint is [[frank-archon-radiance-qwen38-iq4r]]
+* [[frank-archon]] — the live Dankserver endpoint; the tcclaviger vLLM era
+  (2026-09-29 → 10-05) is a row in its Eras table

serverAppConfigs-docs/LLM/PCIe Link and Atomics.md

@@ -65,13 +65,11 @@ LPDDR5X on the Strix Halo — 128 GB LPDDR5x-8000 on a 256-bit bus,
 streaming loses by construction and no tuning rescues it; doubling the link
 only doubles a losing number.**
 
-Worked example (Dankclaw 2026-09-16): Gen4 x4 measured H2D **6.72 / D2H 6.60
-GB/s** (85 % of spec) — pinned H2D ≈ async, pageable within 1 %, i.e.
-link-limited not staging-limited. A 3 MiB slab (the expert-streaming
-granularity) = 474 µs. A3B MoE at Q4_K_M (~1.70 GiB active/token) → 272
-ms/token ⇒ **3.7 tok/s** (4.6 with 20 % resident) vs measured **20.46 tok/s**
-with experts computed on the CPU, 25–26.5 on the iGPU. The streaming design
-loses 3–5x.
+Worked example: the Dankclaw Gen4 x4 run of 2026-09-16 — measured H2D/D2H
+rates, the 3 MiB slab cost, and the streaming-vs-compute comparison it decided
+— is recorded on [[x4-pair-experiment]] §The link, measured. Read it once and
+apply the rule above: on that link the streaming design lost by 3–5x, and the
+x8 ceiling would only have halved the loss.
 
 ## 3. Measure the link: two probes, same mechanics
 
@@ -150,9 +148,9 @@ int main(int argc, char **argv) {
   (the usual "pageable costs 2x" rule does not apply at this link).
 - Small-transfer penalty (2.345 MiB: ~106 µs vs ~78 µs ideal, ~21 %) is
   per-transfer overhead — recoverable only by BATCHING transfers, not by a
-  faster link. (Dankserver 2026-09-29: bulk 22.7 GB/s at the 2.344 MiB slab =
-  91 % of Gen5 x8, while the engine's effective read path ran ~9.5 GB/s — the
-  gap is the fork's gather software path, not the wire.)
+  faster link. The engine-vs-wire version of that reading on Dankserver
+  (bulk slab rate against the engine's effective gather rate, same
+  granularity) is [[frank-archon]]'s tcclaviger-era record.
 - Both GPUs symmetric ⇒ no cross-device rebalance is available; measure both.
 - Don't quote a derived line from inside a probe without hand-checking it —
   a printf format/argument mismatch in one of these printed a nonsense
@@ -186,7 +184,8 @@ sudo lspci -vv -s <bridge> | grep AtomicOpsCap                   # want Routing+
 `Routing-` is a read-only hardware capability. Typical AM5 layout: one x16
 slot on CPU root ports (atomics fine), the rest behind the chipset (usually
 not). Bandwidth for TP is small anyway: ~10 MB per token-step for a 27B at
-batch 4 ≈ 1.4 ms at Gen4 x4 — compute-bound, not interconnect-bound.
+batch 4 is a millisecond-scale figure at x4 — compute-bound, not
+interconnect-bound (full derivation: [[Dual-R9700-Cooperation]] §Bandwidth).
 
 **Probe matrix (bare 2-rank `torch.distributed`, one process per GPU, 25 s PG
 timeout) — one probe answers several design questions:**
@@ -215,13 +214,12 @@ reporting an outage (`gitea` vs `gitea-server`).
 
 1. CPU-attached M.2 → PCIe x4 riser (~$30) — puts the card on the CPU root
    complex; x4 suffices for TP.
-2. Board whose CPU x16 bifurcates x8/x8 (done 2026-09-20 on dankserver — both
-   R9700s on root ports `00:01.1`/`00:01.3`, every bridge `Routing+`, both
-   endpoints `ReqEn+`). Re-checked 2026-10-18 without root: both root ports
-   report `current_link_width=8` at `32.0 GT/s`
+2. Board whose CPU x16 bifurcates x8/x8 (done 2026-09-20 on dankserver — the
+   per-device chain census, including both endpoints reading `ReqEn+`, is
+   [[Dual-R9700-Cooperation]]'s). Re-checked 2026-10-18 without root: both root
+   ports report `current_link_width=8` at `32.0 GT/s`
    (`/sys/bus/pci/devices/0000:00:01.{1,3}/current_link_width`), and the live
-   engine states direct P2P between the cards
-   ([[Dual-R9700-Cooperation]]).
+   engine states direct P2P between the cards.
 3. llama.cpp `--split-mode layer` — no collectives at all ([[Llama Cpp Multi-Device Splits]]); costs the GGUF conversion plus paged attention /
    continuous batching / spec decode.
 4. Independent TP=1 replicas + routing proxy — cards never share one model.
@@ -253,4 +251,4 @@ kernel's declared args (`roc-obj-extract` + `llvm-readelf --notes`), look for
 
 - [[Llama Cpp Multi-Device Splits]] — the layer-split path around the wall
 - [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — measured history
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — the gather economics of the retired tcclaviger vLLM era (2026-09-29 → 10-05)
+- [[frank-archon]] — the gather economics of the retired tcclaviger vLLM era (2026-09-29 → 10-05)

serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.8 PP Deployment.md

@@ -17,7 +17,7 @@ timestamp: 2026-09-03
 > removed from Dankserver on 2026-09-15 (it now sits in Dankclaw) and the server
 > runs 2× R9700. The live multi-GPU scheme on Dankserver is tensor parallelism
 > inside one process ([[Dual-R9700-Cooperation]],
-> [[frank-archon-radiance-qwen38-iq4r]]).
+> [[frank-archon]]).
 
 # Qwen3.8-27B Pipeline-Parallel Deployment (Dankclaw + Dankserver)
 
@@ -43,8 +43,16 @@ state, verified against the live journal and output smoke tests
   the mismatch (`resolve_fused_ops: Flash Attention enabled`) via the
   GGML_CUDA_FA_ALL_QUANTS rebuild — see "FA-all-quants rebuild" below.
 - Pool `262144`, `PARALLEL=1`, live `n_ctx_slot = 262144`.
-- MTP: standalone draft head, acceptance 0.83 then 0.75, mean len
-  4.0-4.3, per-position ~0.86/0.71/0.71/0.71.
+* MTP: standalone draft head, acceptance 0.83 then 0.75, mean len 4.0-4.3,
+  per-position ~0.86/0.71/0.71/0.71 — stronger at every position than the old bundled
+  head, which measured 0.739/0.554/0.427/0.340.
+- Spec chain as deployed: `--spec-draft-n-max 4` on the standalone draft, with
+  the `--spec-default` ngram-mod fallback at n_match 24 / n_min 48 / n_max 64
+  (the preset that survived the DSpark/DFlash retirements —
+  [[qwen3.8-spec-matrix]]).
+- Batch 8192 / ubatch 256, `--direct-io`, `--fit off`, `--gpu-layers all`. The
+  pi `models.json` declared 262144 (`contextWindow = maxTokens`, the house
+  convention of that era).
 - Head GPU side: **ROCm0** (no `--device` flag; HIP registers before
   Vulkan).
 - Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but
@@ -199,7 +207,7 @@ exists** (checkout read 2026-10-18: `frank-pp-worker.env`,
 `serve-pp-worker.sh`, `serve-pp-head.sh`, `frank-pp-worker.service`,
 `pp-test.sh`, `pp-bench.py` and `prune-rpc-cache.sh` are all absent), and
 `scripts/frank-dragoon.env` is today the halogen-flash-server env, not a PP
-head env ([[frank-dragoon-flash-server-v2-native]]).
+head env ([[frank-dragoon]]).
 
 ## Gotchas (all verified in b10433 source / live tests, 2026-08-15)
 
@@ -215,8 +223,11 @@ head env ([[frank-dragoon-flash-server-v2-native]]).
    faster prefill, ~3% slower decode (measured on this box).
 3. **Pin `--tensor-split 1,1`.** The default split is proportional to
    *reported free memory*: the Strix reports up to 96 GiB (GTT) vs the
-   3090's 24 GB, so the default lands ~20/80 on the slow device (verified
-   with a 4B smoke test). 1,1 = 50/50 = balanced per-layer latency.
+   3090's 24 GB, so the default lands ~20/80 on the slow device. 1,1 = 50/50 =
+   balanced per-layer latency. The 2026-08-15 4B smoke test that proved it
+   (production untouched): at a 0.2/0.8 split only 4 of 36 layers reached the
+   3090, and at a real split worker VRAM rose ~819 MiB with 95 t/s prefill /
+   58.8 t/s decode on the tiny model; link RTT ~0.3–0.5 ms.
 4. **`-dio` on the head.** Large model + RPC hangs at load without direct
    I/O.
 5. **RPC is unauthenticated and unencrypted.** LAN only. The worker binds

serverAppConfigs-docs/LLM/Pipeline Parallelism/index.md

@@ -65,6 +65,6 @@ graph LR
 ## See Also
 
 - [[DFlash DSpark Speculative Decoding]]
-- [[frank-zealot-benchmark]]
+- [[frank-zealot]]
 - [[Dankserver System Specs]]
 - [[Dankclaw System Specs]]

serverAppConfigs-docs/LLM/Quantization Production.md

@@ -147,12 +147,14 @@ cut to GPTQ-w4 for the 3090 is a bench experiment that needs no new
 hardware; EXL3 (`convert.py` on the 3090, exllama serve) is the
 quality-per-bit extreme of the same menu.
 Caveats before any swap: the serving contract (OpenAI server,
-`reasoning_content`, vision mmproj, compaction-hub window
-200,704 ×2 slots) must be met by the challenger runtime, and the
-24 GB card runs ONE server — benching follows the stop-and-mask law.
+`reasoning_content`, vision mmproj, and the compaction-hub window the endpoint
+declares in its Current shape — [[frank-zealot]]) must be met by the challenger
+runtime, and the 24 GB card runs ONE server — benching follows the stop-and-mask
+law.
 
 #### Worked example — the frank-archon w4dn8 recipe ported to the 3090
-(audit 2026-10-08, target `Qwen3.6-35B-A3B`)
+(audit 2026-10-08, target `Qwen3.6-35B-A3B`; sized at that date's hub shape —
+the window history is [[frank-zealot]]'s Eras table)
 **The methodology ports; the encoding must be translated.** llama.cpp
 decodes no GPTQ — the recipe re-expresses on GGUF-native terms:
 imatrix on our corpora (replaces the radiance grams; runs the 65 GiB
@@ -253,8 +255,6 @@ top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
 - Repo: `scripts/convert-swift15-w4dn8.sh`,
   `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
   `scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
-  `scripts/build-kairic-edge-gfx1151.sh`, [[agentionai-fast-imatrix]],
-  [[kingjones-strix-rocmfp4]], [[qwen3.8-strix-optimizations]],
-  [[frank-archon-radiance-swift15-w4dn8]],
-  [[frank-zealot-llama-qwen35-9b-q6k-2x200704]],
-  [[frank-dragoon-flash-server-v2-swift15]], [[GGUF Compatibility]].
+  `scripts/build-kairic-edge-gfx1151.sh`; endpoints [[frank-archon]],
+  [[frank-dragoon]], [[frank-zealot]]; method [[qwen3.8-strix-optimizations]]
+  and [[GGUF Compatibility]].

serverAppConfigs-docs/LLM/Quantization by Hardware Architecture.md

@@ -108,9 +108,10 @@ Use ROCmFP4 only with the ROCmFPX fork of llama.cpp. Upstream llama.cpp cannot r
 
 ## Choices for This Project
 
-### Dankserver (RTX 3090, 24 GB, Ampere)
+### The RTX 3090 (24 GB, Ampere) — frank-zealot's card
 
-The RTX 3090 has no native FP8 or FP4 tensor cores. Use:
+The 3090 has no native FP8 or FP4 tensor cores. It has sat in Dankclaw since
+2026-09-16; [[frank-zealot]]'s Eras table records the move off Dankserver. Use:
 
 | Ranking | Format | Why |
 |---------|--------|-----|
@@ -121,13 +122,17 @@ The RTX 3090 has no native FP8 or FP4 tensor cores. Use:
 
 ### Dankclaw (Strix Halo gfx1151, RDNA 3.5)
 
-RDNA 3.5 has no native FP8 tensor cores. The APU shares memory. The active Ling-3.0-flash service runs vLLM-ROCm with the AMD Quark INT4 (W4A16) format (the jcbtc CIRU Strix-native bundle). INT4 weights keep the 124B model inside the 96 GiB frame buffer with a full-precision KV cache.
+RDNA 3.5 has no native FP8 tensor cores and the APU shares its memory, so the
+GPU budget is the GTT aperture and every live figure on the box is
+[[Dankclaw System Specs]]. What serves here today is [[frank-dragoon]]: a
+vendor `.hgn` checkpoint, a format this page cannot rank because no public
+quantizer exists for it ([[Quantization Production]]).
 
 | Ranking | Format | Why |
 |---------|--------|-----|
-| 1 | INT4 W4A16 (Quark/vLLM) | active; fits 96 GiB; native ROCm kernels; concurrency |
-| 2 | MXFP4 (Quark) | same-bit, higher accuracy; beta on gfx1151 (A/B experiment) |
-| 3 | GGUF Q4_K_M / Q8_0 | works with upstream llama.cpp only |
+| 1 | `.hgn` (deployed) | the halogen engine's own format; production door is closed |
+| 2 | GGUF K/I quants, K_P blends | the only self-produced option on this card ([[Quantization Production]]) |
+| 3 | INT4 W4A16 (Quark) / MXFP4 (Quark) | the retired Ling-3.0-flash vLLM era, measured under the firmware-VGM frame buffer: [[ling-3.0-flash-trial]] |
 | 4 | FP8 (vLLM) | software emulation; memory savings only; no compute win |
 
 ## See Also

serverAppConfigs-docs/LLM/RAM CPU Offloading in VLLM.md

@@ -11,9 +11,19 @@ timestamp: 2026-08-28
 
 ## Overview
 
-vLLM uses CPU RAM and swap space to extend GPU memory capacity. This allows larger models to run on GPUs with limited VRAM. vLLM does not offload model weights to CPU. Instead, vLLM moves KV cache data between GPU and CPU.
+Stock vLLM offloads **KV pages**, not weights: the scheduler moves KV cache between GPU
+and CPU. That is the upstream behaviour and it is NOT the whole story on our hosts — the
+tcclaviger/R4D fork ran a real **weight** offload (`--enable-expert-offload`, tiered
+expert placement, a pinned host pool per rank), which is how a 147 GiB MoE served from a
+246 GiB box: [[RDNA4 MoE Expert Offload VRAM Budget]] and the tcclaviger era row of
+[[frank-archon]]. First-hand in-repo evidence outranks a generic upstream claim about our
+own hosts ([[Model Swap and Rollback]] §10).
 
-> **Current vLLM note:** Recent vLLM releases removed the `--swap-space` engine argument. CPU KV-cache offloading now uses `--kv-offloading-size`. See the parameters below for details.
+> **Verified 2026-10-19 against `vllm/config/cache.py` on `main`:** there is no
+> `swap_space` field — `--swap-space` is gone from current engine args. CPU KV offload is
+> `--kv-offloading-size`, and it activates **only** together with `--kv-offloading-backend`
+> (default `native`); both fields live in that one file. This page is the fleet's owner of
+> that knob contract — other pages link here instead of restating it.
 
 ## How CPU Offloading Works
 
@@ -42,9 +52,12 @@ This parameter controls the fraction of GPU memory used for KV cache.
 
 Lower values leave more GPU memory for model weights. Higher values allow more concurrent requests but increase swap frequency.
 
-### swap_space (removed) / kv-offloading-size (current)
+### swap_space (removed) / kv-offloading-size + kv-offloading-backend (current)
 
-The `--swap-space` argument was removed from current vLLM engine args. Use `--kv-offloading-size` instead. The table below describes the historical `--swap-space` values for reference.
+`--swap-space` no longer exists in current engine args (verified above); the live pair is
+`--kv-offloading-size` with `--kv-offloading-backend`, and offload stays off until the size
+is set. The table below is the historical `--swap-space` sizing guidance, kept as a sizing
+reference only.
 
 | Value | Description |
 |-------|-------------|
@@ -103,6 +116,9 @@ This parameter sets the KV cache data type.
 
 ## Memory Configuration Examples
 
+All three examples carry the retired `--swap-space` syntax and are kept as sizing
+references; the live knob pair is named above.
+
 ### Small Model on Single GPU
 
 ```
@@ -157,7 +173,7 @@ WARNING scheduler.py: Sequence group 0 is preempted by PreemptionMode.RECOMPUTE
 If you see frequent preemptions:
 
 1. Increase `gpu_memory_utilization`
-2. Increase `swap_space`
+2. Raise the KV offload budget (`--kv-offloading-size`, with `--kv-offloading-backend`)
 3. Decrease `max_num_seqs`
 4. Increase `tensor_parallel_size`
 5. Increase `max_model_len` to match actual usage
@@ -169,7 +185,8 @@ If you see frequent preemptions:
 - Reduce `max_num_seqs` to limit concurrent requests
 - Disable CUDA graphs with `--enforce-eager` to save memory
 - Set `compilation_config.cudagraph_capture_sizes` to smaller values
-- Use `--kv-cache-memory` to skip memory profiling
+- Set `--kv-cache-memory-bytes` to pin the KV budget instead of letting the profiler guess
+  (the flag our own `scripts/patches/gpu_worker-kv-cap.patch` reads)
 - Limit multi-modal inputs with `limit_mm_per_prompt`
 
 ## Environment Variables

serverAppConfigs-docs/LLM/RDNA4 Container Serving.md

@@ -8,12 +8,12 @@ timestamp: 2026-10-05T00:00:00-07:00
 
 # RDNA4 Container Serving
 
-Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201, 31.86 GiB each =
-**63.7 GiB real VRAM**, 246 GiB RAM, 7700X) with
+Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201 — card count,
+per-card VRAM and host RAM are [[Dankserver System Specs]]'s) with
 `docker.io/tcclaviger/vllm` (vLLM 0.29.0.dev, GLM/Qwen-specialised RDNA4 build
 with native MoE quant kernels and a first-class expert offload planner). The
 tuning history and current deployed shape live in
-[[dankserver-qwen38-fp8-expert-offload-tuning]]; this page is the bring-up law.
+[[frank-archon]]; this page is the bring-up law.
 
 ## 0. Know the box — and its phantom third GPU
 
@@ -155,7 +155,9 @@ podman run -d --name <ctr> --ipc=host --ulimit memlock=-1:-1 \
 
 ## 5. Memory budgeting: the levers are not where they look
 
-The planner builds `budget = total × util`, subtracts a KV reserve, and gives
+Every figure in this section is a tcclaviger-era boot on the port that is
+[[frank-archon]]'s today; the laws they teach are general. The planner builds
+`budget = total × util`, subtracts a KV reserve, and gives
 the REMAINDER to the expert arena. Consequences, all measured:
 
 - **`--gpu-memory-utilization` grows the ARENA, not the KV pool** (KV stayed
@@ -163,16 +165,19 @@ the REMAINDER to the expert arena. Consequences, all measured:
 - **Lowering `--max-model-len` is the WRONG lever and backfires:** a smaller
   context shrinks the reserve, handing the space to the arena — measured 65536
   → 0.64 GiB KV, 32768 → 0.34 GiB KV.
-- **Hybrid-attention undercount:** the planner's per-token KV figure (9190
-  B/token for Qwen4Exp) misses the large fixed per-sequence Mamba state
+- **Hybrid-attention undercount:** the planner's per-token KV figure for
+  Qwen4Exp is an undercount — the printed figure and the true one are both on
+  [[frank-archon]] — because it misses the large fixed per-sequence Mamba state
   (~2x), so the reserve is always short and the arena silently covers it.
-  Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (2 GiB →
-  113,595 tokens / 3.47x concurrency at 32k). For long context,
+  Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (era reading: a
+  2 GiB pin = 113,595 tokens ≈ 3.47x concurrency at 32k — that pair is this
+  page's record of the tcclaviger boots, not [[frank-archon]]'s). For long
+  context,
   `--kv-offloading-size` (GiB) spills KV to host.
 - **`--expert-offload-mem` caps the HOST side, not the VRAM arena**
   (`cap = int(x * GIB)` against `host_memory_available()`). Do not reach for
   it to fix a VRAM problem; to grow the arena, shrink the KV pin or raise
-  util (see [[dankserver-qwen38-fp8-expert-offload-tuning]] for the
+  util (see [[frank-archon]] for the
   util=1.0 unbootability law).
 
 **Read the plan — it is the real acceptance test:**
@@ -201,18 +206,21 @@ host RAM = `host-backed set + layers still loading`, not the whole model. At
 runtime `apply_hot_cold` sends arena-resident experts down a hot GEMM and the
 rest straight over PCIe via UVA views — no staging copy in the critical path.
 
-**Emulation check:** a dequantize-to-compute path doubles real host RAM (118
-GiB offload request → ~238 GiB) and is invisible in the weight count. Read the
-log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".
+**Emulation check:** a dequantize-to-compute path doubles real host RAM — the
+measured pair (offload request vs host footprint) is on [[frank-archon]] — and
+the doubling is invisible in the weight count. Read the log for `Using
+'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".
 
 ## 6. Performance profile and honest measurement
 
-Decode on this pair is **PCIe-gather-bound, context-independent**: 12.11
-tok/s FLAT from 304 to 16,095 context tokens — KV attention is not the cost.
-With ~29 % of experts arena-resident, ~0.84 GB crosses PCIe per token at the
-engine's effective read rate; cold-vs-warm differing ~10 % confirms the LRU
-works; growth is linear in hit rate, and only speculative decode cuts
-sequential STEPS rather than making each cheaper.
+Decode on this pair is **PCIe-gather-bound, context-independent**: the rate is
+flat from a few hundred to sixteen thousand context tokens, so KV attention is
+not the cost (the flat figure and its MTP/gather deltas are
+[[Tcclaviger Fork Ops]]'s profile of the same shape). With ~29 % of experts
+arena-resident, under a gigabyte crosses PCIe per token at the engine's
+effective read rate; cold-vs-warm differing ~10 % confirms the LRU works;
+growth is linear in hit rate, and only speculative decode cuts sequential STEPS
+rather than making each cheaper.
 
 **Benchmark traps that produced fake numbers here:**
 
@@ -238,13 +246,13 @@ sequential STEPS rather than making each cheaper.
 boolean (default 0; at depth 1 only ~10 % cold-vs-warm difference — the cache
 already works); `CLAV_TUNABLEOP_SWEEP=1` autotunes GEMM shapes (silently
 skipped under expert offload unless `CLAV_TUNABLEOP_WITH_OFFLOAD=1` too —
-see [[dankserver-qwen38-fp8-expert-offload-tuning]] for why it was rejected);
+see [[frank-archon]] for why it was rejected);
 `--expert-precise-memory` swaps the fitted overhead figure in.
 
 ## Related
 
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — deployed shape, plan law,
+- [[frank-archon]] — deployed shape, plan law,
   measured campaigns
 - [[PCIe Link and Atomics]] — why decode is gather-bound and what cannot fix it
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — arena/plan arithmetic deep-dive
-- [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — window discipline on live hosts
+- [[frank-dragoon]] and [[frank-zealot]] — window discipline on live hosts

serverAppConfigs-docs/LLM/RDNA4 MoE Expert Offload VRAM Budget.md

@@ -8,10 +8,12 @@ timestamp: 2026-10-05T00:00:00-07:00
 
 # RDNA4 MoE Expert Offload VRAM Budget
 
-For VRAM-bound vLLM on the tcclaviger RDNA4 fork (frank-archon: Qwen3.8-
-Flash-Next FP8, TP=2, 2x R9700 at 31.86 GiB each, expert offload to host RAM).
-Tuning history and the deployed shape live in
-[[dankserver-qwen38-fp8-expert-offload-tuning]]; boot-failure capture lives in
+For VRAM-bound vLLM on the tcclaviger RDNA4 fork. Every boot ledger, arena
+figure and rate below is from [[frank-archon]]'s tcclaviger era (2026-09-29 →
+10-05: Qwen3.8-Flash-Next FP8, TP=2, 2x R9700 at 31.86 GiB each, expert offload
+to host RAM) — that page owns the endpoint's shape, tuning history and era
+record; this page owns only the plan arithmetic, the ladder method and the audit
+procedure. Boot-failure capture lives in
 [[vLLM Boot Forensics and Config Interrogation]].
 
 ## 1. The plan law
@@ -215,5 +217,5 @@ at graph capture; 6 seqs fp8 → 10.70 GiB ask with 9.18 GiB free (fails) —
 
 - [[RDNA4 Container Serving]] — the bring-up path and container traps
 - [[vLLM Boot Forensics and Config Interrogation]] — capture and contracts
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — measured history
+- [[frank-archon]] — measured history
 - [[Llama Cpp Serving Sizing]] — the llama.cpp equivalent

serverAppConfigs-docs/LLM/Running Qwen3.6 27B FP8 on Dankclaw.md

@@ -7,6 +7,12 @@ timestamp: 2026-09-03
 ---
 
 # Running Qwen3.6 27B FP8 on Dankclaw
+> **RETIRED - vLLM era (before the 2026-08-17 redo).** Nothing on this page serves
+> today: no vLLM or ROCm process runs on Dankclaw, and the iGPU endpoint at
+> `192.168.0.143:7770` is [[frank-dragoon]] (halogen flash-server). The page stays as
+> the era record of the Qwen3.6-27B-FP8 vLLM bring-up and the Strix Halo ROCm
+> recipes ([[frank-redo-2026-08-17]]). The hardware table below carries the era's
+> host figures; live ones are [[Dankclaw System Specs]].
 
 ## Model Information
 
@@ -29,6 +35,7 @@ Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. It uses 27
 | Vision Encoder | Yes (image-text-to-text) |
 
 ## Hardware: Dankclaw
+*Era figures (2026-08). The host has changed since - [[Dankclaw System Specs]].*
 
 | Property | Value |
 |----------|-------|
@@ -55,7 +62,11 @@ The FP8 variant requires approximately 16-17 GB. ThinkingCap-FP8 requires ~29 GB
 
 ## Prerequisites
 
-> **Production note:** Dankclaw's primary service now runs vLLM-ROCm with the Ling-3.0-flash CIRU INT4 model (see `scripts/frank-dragoon.service`). This page documents the alternative vLLM FP8 deployment of Qwen3.6-27B.
+> **Era note (2026-08, superseded twice):** at writing time Dankclaw's primary
+> service was the vLLM-ROCm Ling-3.0-flash CIRU INT4 trial ([[ling-3.0-flash-trial]],
+> retired 2026-08-17), and this page documented the alternative vLLM FP8 deployment
+> of Qwen3.6-27B. Both paths are dead; `scripts/frank-dragoon.service` today is the
+> halogen flash-server unit ([[frank-dragoon]]).
 
 Install vLLM for ROCm. Version 0.24.0+ is required for Qwen3.6 with speculative decoding:
 

serverAppConfigs-docs/LLM/Shared Memory Architecture Dankclaw.md

@@ -36,7 +36,7 @@ In unified memory:
 
 ### ROCm Version
 
-vLLM supports AMD GPUs with ROCm 6.3 or above *(unverified 2026-10-18)*. Dankclaw has ROCm 7.2.4 from AMD repos (`/etc/apt/sources.list.d/rocm.list` → `deb https://repo.radeon.com/rocm/apt/7.2.4 noble main`; `rocm-core 7.2.4.70204-93~24.04`, read 2026-10-18). No vLLM endpoint runs on this host today — 7770 is halogen-flash-server and 7771 is llama.cpp ([[Dankclaw System Specs]]) — so this page is reference material, not the serving path.
+vLLM supports AMD GPUs with ROCm 6.3 or above *(unverified 2026-10-18)*. Dankclaw's ROCm version and which engine owns which port today are [[Dankclaw System Specs]]'s — so this page is reference material, not the serving path.
 
 Check ROCm version:
 
@@ -103,7 +103,7 @@ vllm serve model \
 
 - `gpu_memory_utilization`: Set lower (0.8-0.85) to leave room for CPU operations
 - `max_model_len`: Match actual context needs
-- The old `--swap-space` argument was removed from current vLLM — `CacheConfig` on `main` carries no `swap_space` field ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/cache.py), read 2026-10-18). Use `--kv-offloading-size` (`kv_offloading_size`, same file) for CPU KV offload
+- `--swap-space` is retired from current vLLM; the CPU KV-offload knob pair, with the upstream `CacheConfig` verification, is owned by [[RAM CPU Offloading in VLLM]]
 
 ### NUMA Binding
 
@@ -166,7 +166,7 @@ Unified memory has lower bandwidth than dedicated VRAM:
 
 ### ROCm Overcommit
 
-ROCm reports ~114 GiB of GPU memory on Strix Halo APU (the GTT aperture: `mem_info_gtt_total` = 122402963456 bytes = 114 GiB, read 2026-10-18 under `/sys/class/drm/card1/device/`; `llama-server --list-devices` shows 116,732 MiB — the 2026-08-17 "96 GiB" figure was stale). Physical RAM visible to the OS is **121 GiB** today (`free -g` total 121, read 2026-10-18; 64 GiB in the earlier reading on this page, upgraded from 32 GB). The GPU can address more than physical RAM exists because the GTT aperture maps over the shared LPDDR. vLLM will see ~114 GiB and try to allocate accordingly. Set `--gpu-memory-utilization` to 0.80 or lower to prevent OOM.
+ROCm reports the GTT aperture as "GPU memory" on this APU; the aperture size, the physical `MemTotal` behind it and the firmware carve-out are [[Dankclaw System Specs]]'s numbers, and that page also carries the 2026-08-17 "96 GiB" reading this page used to quote (stale). One reporting quirk worth keeping here: `llama-server --list-devices` prints 116,732 MiB for the same device the sysfs file calls 114 GiB — engines round the aperture differently, so read the sysfs value when sizing. The GPU can address more than physical RAM exists because the GTT maps over the shared LPDDR. vLLM will see the aperture and try to allocate accordingly; set `--gpu-memory-utilization` to 0.80 or lower to prevent OOM.
 
 Lower bandwidth means slower inference. Plan for lower throughput.
 
@@ -232,9 +232,8 @@ FP8 quantization depends on GPU architecture:
   the matmul claim as the settled half.)
 
 For FP4 on Dankclaw the ROCmFPX llama.cpp fork (`Q2_0_ROCMFP4`) provided native
-ROCm FP4 kernels; that runtime line is **retired** (CIRU/Kairic eras,
-[[ciru-activefpx-promptforge]], [[frank-dragoon-kairic-edge-qwen38-27b]]) and no
-ROCmFPX build serves this host today.
+ROCm FP4 kernels; that runtime line is **retired** — the CIRU and Kairic eras on
+[[frank-dragoon]] — and no ROCmFPX build serves this host today.
 
 ## Example Commands
 

serverAppConfigs-docs/LLM/Tcclaviger Fork Ops.md

@@ -60,6 +60,8 @@ TUNABLE=0`, stock gather 8×16 → 26.80/25.15/26.81 tok/s @ 166/1087/4064 ctx.
 
 Decode on this stack is PCIe-gather-bound (~30% GPU duty cycle), so
 batch-shaped compute knobs move ~nothing.
+Measured with no spec: **12.11 tok/s FLAT from 304 to 16,095 context tokens** — KV
+attention is not the cost, so a context-slope hunt on this stack is null by construction.
 
 The `R4D_*/CLAV_*` knob family (QSA_*, MHC, ROUTE, ACT_QUANT, AR_QUANT,
 SILU_QUANT) are kill-switches defaulting ON, not perf dials; real levers are
@@ -120,95 +122,36 @@ exactly SEQS×(n+1) (`suboptimal` warning) → try 8192.
 
 ## Serving a model larger than VRAM (dankserver recipe)
 
-Dankserver: 2× R9700 (gfx1201), 31.86 GiB each = 63.7 GiB real VRAM, 246 GiB
-RAM. `torch.cuda.device_count()` returns 3 and summing `total_memory` gives
-a bogus 186.7 GiB (the Raphael iGPU is in the list — see
-[[Llama Cpp Serving Sizing]] for the reading discipline).
+Dankserver's card count, per-card VRAM and host RAM are in [[Dankserver System Specs]];
+`torch.cuda.device_count()` returns 3 on this box and summing `total_memory` overstates it
+badly (the Raphael iGPU is in the list — reading discipline: [[Llama Cpp Serving Sizing]]).
 
-**Kernel gate — this decides which checkpoint can work at all.**
-`/app/r4dhip/r4d.so` exports `gemm_moe_mxfp4a8_*` and `gemm_moe_dsfp4a8_*`
-but NO `gemm_moe_w4a16_*`. Expert offload requires an experts class with
-BOTH a compiled kernel AND an `offload_recipe`; the planner aborts startup
-otherwise. Usable: MXFP4, DSFP4, FP8-HIP (`R4dMxfp4MoEExperts`,
-`R4dDsfp4MoEExperts`, `Fp8HipMoEExperts`). `R4dW4a16MoEExperts` has no
-kernel (the class exists, the r4d.so entry does not); RFA has neither. Check
-the checkpoint's quant against this list BEFORE downloading anything.
+Everything else about running this image — the quant-kernel gate that decides
+which checkpoints can serve at all, the entrypoint contract, the known-good
+invocation, the `/dev/dri` and memlock traps, the util and max-model-len arena
+traps, the `--ipc=host` / `-d` stdout traps and the gated-fetch discipline — is
+written once on [[RDNA4 Container Serving]], and the plan arithmetic behind the
+arena/KV traps is [[RDNA4 MoE Expert Offload VRAM Budget]]'s. Three findings
+stay here because this harness produced them:
 
-Entrypoint is `/app/tools/image_entrypoint.sh`, ending `exec vllm serve
-"$@"` — pass the model path as the FIRST argument and do NOT pass `serve`.
-Known-good invocation:
+- **The KV reserve undercounts hybrid-attention models ~2×** (the planner's
+  B/token figure against ~18,000 real: the Mamba state is a fixed per-sequence
+  cost). Fix by pinning the pool: `--kv-cache-memory-bytes`.
+- **`--expert-offload-mem` caps the HOST side**, not the VRAM arena —
+  structurally orthogonal to `--expert-precise-memory` (verified in-container
+  2026-10-05: precise-memory only swaps the GPU overhead term,
+  expert_offload_mem only sets the host bound; they rejoin only inside
+  `place_experts`, and the plan rewrites `expert_offload_mem` back to the cap
+  the plan met, so plan and allocation cannot drift).
+- **Check whether the stack supports the model natively before porting an
+  overlay.** The image already registered `Glm5NextForConditionalGeneration`,
+  `Qwen4ExpForCausalLM` and both MTP variants — chasing four missing symbols
+  through two image tags to bring an overlay up was entirely wasted work.
 
-```bash
-sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
-podman run -d --name <ctr> --ipc=host --ulimit memlock=-1:-1 \
-  --cap-add SYS_PTRACE \
-  --device /dev/kfd --device /dev/dri --device /dev/mem \
-  -e HIP_VISIBLE_DEVICES=0,1 -e VLLM_ROCM_USE_AITER=0 \
-  -v /home/dank/models/<model>:/models:ro \
-  docker.io/tcclaviger/vllm:latest /models \
-  --served-model-name <name> --port 7770 \
-  --tensor-parallel-size 2 \
-  --max-model-len 32768 \
-  --kv-cache-memory-bytes 2147483648 \
-  --max-num-seqs 4 \
-  --gpu-memory-utilization 0.95 \
-  --enable-prefix-caching --trust-remote-code
-```
-
-`-p 7770:7770` is not included, so the API is only reachable via
-`podman exec` — add `-p` if host-side curl is needed. Load ~5 min;
-`/health` answers connection-refused the whole time; watch for
-`Application startup complete` in `podman logs <ctr>`. Verified working:
-`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (gated; 172.8 GiB, 131
-shards, `Qwen4ExpForConditionalGeneration`, 48 layers, 512 experts top-10,
-multimodal, native 262,144).
-
-Traps that each cost a boot:
-
-1. **Mount the whole `/dev/dri`, never individual render nodes.** Render
-   nodes without the primary `card*` nodes → HIP enumerates two agents and
-   refuses a context on either (`RuntimeError: No CUDA GPUs are available`)
-   while `device_count()` cheerfully returns 2. Confirmed by direct A/B.
-2. **memlock must be raised** (done on Dankserver via
-   `/etc/security/limits.d/99-memlock.conf`). Host default 8 MiB makes any
-   large pinned allocation fail as a misleading
-   `torch.AcceleratorError: CUDA error: out of memory` from `pin_memory()`;
-   it cannot be raised from inside a container or root's own shell — only
-   PAM applies it at login. Matters because `--expert-host-mlock` defaults
-   True.
-3. **`--gpu-memory-utilization` does NOT buy KV under expert offload** —
-   the planner sizes the arena against the budget and leaves KV the
-   remainder; raising util 0.95→0.98 grew the arena and the KV pool stayed
-   at 0.64 GiB.
-4. **Lowering `--max-model-len` is the WRONG lever and backfires** — the
-   budget subtracts the KV reserve first, so a smaller context shrinks the
-   reserve and hands the space to the arena (measured: 65536 → 0.64 GiB KV;
-   32768 → 0.34 GiB).
-5. **The KV reserve undercounts hybrid-attention models ~2×** (9190 B/token
-   planned vs ~18,000 real: the Mamba state is a fixed per-sequence cost).
-   Fix by pinning the pool: `--kv-cache-memory-bytes`.
-6. **`--expert-offload-mem` caps the HOST side**, not the VRAM arena —
-   structurally orthogonal to `--expert-precise-memory` (verified
-   in-container 2026-10-05: precise-memory only swaps the GPU overhead term,
-   expert_offload_mem only sets the host bound; they rejoin only inside
-   `place_experts`, and the plan rewrites `expert_offload_mem` back to the
-   cap the plan met, so plan and allocation cannot drift).
-7. `--ipc=host` and `--shm-size` are mutually exclusive in podman; pick one.
-8. `podman run -d` without `-a` sends no stdout to a redirect — the log file
-   only receives the container ID; use `podman logs`.
-
-Status reads: use `scripts/dankstatus.sh`, never hand-rolled
-`/proc/meminfo` parsing — `/proc/meminfo` separates key from value with a
-TAB, so `grep MemAvailable | tr -s " " | cut -d" " -f2` returns empty and
-awk prints `0.0` — a false "host out of memory" that invites an unnecessary
-abort. Gated checkpoints: token via a mode-600 `curl -K` config, never argv
-(`pgrep -a curl` prints it); always `--fail`; compare each file's byte count
-to the Hub's declared size; `-C -` resumes truncated shards.
-
-**Check whether the stack supports the model natively before porting an
-overlay.** The image already registered `Glm5NextForConditionalGeneration`,
-`Qwen4ExpForCausalLM` and both MTP variants — chasing four missing symbols
-through two image tags to bring an overlay up was entirely wasted work.
+The checkpoint this recipe was verified on is
+`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (gated; 172.8 GiB, 131 shards,
+`Qwen4ExpForConditionalGeneration`, 48 layers, 512 experts top-10, multimodal,
+native 262,144) — an [[frank-archon]] era payload.
 
 ## Compile-cache persistence (survives `podman rm -f`)
 
@@ -240,20 +183,15 @@ cold, of which 173 s was one graph).
 
 ## Plan-line verification checklist (after any fork deploy)
 
-```
-ssh dankserver 'sudo journalctl CONTAINER_NAME=frank-archon-qwen -b -o cat --no-pager \
-  | grep -aoE "(budget \(total x util\) +[0-9.]+|expert cache space +[0-9.]+ GiB +[0-9,]+ slots|Estimated CUDA graph memory.{0,80}|Graph capturing finished.{0,40}|GPU KV cache size.{0,90})"'
-```
-
-`grep -oE` with a digit-anchored tail — plain `grep`/journalctl truncate
-these columns to `17.`. Verify: new budget/graphs/arena in the plan;
-`GPU KV cache size` matches the requested token count; `Estimated CUDA graph
-memory` vs the capture line (raise `_PER_GRAPH_MB` toward 52 if capture
-exceeds the charge); `rocm-smi --showmeminfo vram` high-water per card = the
-real margin; `systemctl show frank-archon -p NRestarts --value` = 0;
-`VLLM_CUDAGRAPH_*` present in `podman exec … env`; then a real chat
-completion from a PEER — `curl localhost:7770/health` inside an ssh session
-on the host can return `000` and looks like an outage that isn't.
+The journal one-liner that reads the plan lines (digit-anchored `grep -oE`,
+because plain grep truncates those columns to `17.`) and the post-deploy
+verification order — plan vs captured graph memory, `GPU KV cache size` vs the
+ask, per-card `rocm-smi` high-water, `NRestarts` = 0, `VLLM_CUDAGRAPH_*` present
+in the container env, then a real completion from a PEER because
+`curl localhost:7770/health` inside an ssh session on the host can return `000`
+and look like an outage — are step 3 of
+[[RDNA4 MoE Expert Offload VRAM Budget]] §7. Run that; do not re-derive it
+here.
 
 ## Related
 

serverAppConfigs-docs/LLM/VLLM Index.md

@@ -9,7 +9,7 @@ timestamp: 2026-08-28
 
 # vLLM Index
 
-> **RETIRED (2026-08-18).** The vLLM service on Dankclaw was replaced by the CIRU ROCmFPX llama.cpp runtime (v3). These pages are historical reference. See [[ciru-activefpx-promptforge]].
+> **RETIRED (2026-08-18).** The vLLM service on Dankclaw was replaced by the CIRU ROCmFPX llama.cpp runtime (v3). These pages are historical reference. See [[frank-dragoon]].
 
 ## Documentation Overview
 
@@ -60,7 +60,7 @@ vllm serve meta-llama/Llama-3.1-70B-Instruct --tensor-parallel-size 4
 vllm serve Qwen/Qwen2.5-1.5B-Instruct --quantization fp8
 
 # With GPU memory control
-vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --swap-space 8
+vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --kv-offloading-size 8 --kv-offloading-backend native
 ```
 
 ### Common API Endpoints
@@ -79,7 +79,7 @@ vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --swap-space
 |-----------|-------------|---------------|
 | tensor_parallel_size | Number of GPUs for model sharding | 1-8 |
 | gpu_memory_utilization | Fraction of GPU memory for KV cache (default 0.92) | 0.7-0.95 |
-| swap_space | Removed in current vLLM; use `--kv-offloading-size` | n/a |
+| swap_space | Retired; live knob pair in [[RAM CPU Offloading in VLLM]] | n/a |
 | max_model_len | Maximum context length | auto |
 | max_num_batched_tokens | Max tokens per batch | 2048-8192 |
 | max_num_seqs | Max sequences per batch | 128-512 |

serverAppConfigs-docs/LLM/VLLM Overview.md

@@ -140,7 +140,7 @@ curl http://localhost:8000/v1/chat/completions \
 | `tensor_parallel_size` | 1 | Number of GPUs for tensor parallelism |
 | `pipeline_parallel_size` | 1 | Number of GPUs for pipeline parallelism |
 | `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache |
-| `swap_space` | removed | Removed in current vLLM; use `--kv-offloading-size` |
+| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] |
 | `enforce_eager` | false | Use eager mode (no CUDA graphs) |
 | `max_num_batched_tokens` | 2048 | Max tokens per batch |
 | `max_num_seqs` | 128 | Max sequences per batch |
@@ -167,7 +167,7 @@ curl http://localhost:8000/v1/chat/completions \
 | Argument | Default | Description |
 |----------|---------|-------------|
 | `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache |
-| `swap_space` | removed | Removed in current vLLM; use `--kv-offloading-size` |
+| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] |
 | `kv_cache_dtype` | auto | KV cache data type (auto, fp8, fp8_e5m2) |
 | `max_model_len` | auto | Maximum context length |
 | `num_gpu_blocks` | auto | Number of GPU blocks |

serverAppConfigs-docs/LLM/agentionai-fast-imatrix.md

@@ -1,217 +0,0 @@
----
-type: reference
-title: agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix on Dankclaw
-description: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF served by the charlie12345/ROCmFPX llama.cpp runtime (build-qwen38). 87.06 GiB Q4_0_ROCMFP4_FAST imatrix (v2 joined PLE layout), own MTP draft head (ROCmFP4-FAST, 2.28 GiB). frank-dragoon.
-tags:
-  - llama.cpp
-  - rocm
-  - dankclaw
-  - gfx1151
-  - agentionai
-  - rocmfp4
-  - strix-halo
-  - qwen4exp
-  - mtp
-timestamp: 2026-08-29
----
-
-# agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix on Dankclaw
-
-> **RETIRED 2026-08-30** — replaced by the kingjones STRIX model on the
-> card's pinned kingjones30/ROCmFPX runtime; see
-> [[kingjones-strix-rocmfp4]] for the active runbook. This page stays as
-> the deployment record: the v2-joined-PLE layout requirement, the MTP
-> head incompatibility (verified 3x), and the QSA F32-mask patch
-> (`scripts/patches/qwen4exp-qsa-f32-mask.patch`) that made
-> `--flash-attn on` work on the charlie build-qwen38 tree — all still
-> true for that binary + model pair, which remains the rollback target.
-
-`frank-dragoon` (port 7770) served
-`agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF` (main,
-`Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf`, the `v2/` joined layout)
-with the author's own MTP draft head
-(`Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf`, never loaded) on the
-charlie12345/ROCmFPX `build-qwen38` runtime, 2026-08-29 → 2026-08-30.
-
-## Why this model
-
-- 87.06 GiB at 4.23 bpw — everything on the GPU of the 128 GB unified-memory
-  box, including the 51.2 B-parameter n-gram (PLE) table. REPO `v2/` layout
-  is used (joined single `per_layer_token_embd.weight` tensor) because the
-  charlie12345/ROCmFPX loader requires it; the root `-ple16` file's per-head
-  split only loads in the LaurentZuijdwijk Vulkan fork (see Known issues #2).
-- imatrix-calibrated on 1540 chunks; +2.48% PPL vs reference on wikitext-2
-  (better than AesSedai IQ4_XS / IQ3_S at smaller size).
-- 18-25% faster prompt processing than a mainline quant of similar size
-  (356 vs 285 t/s at 2048 tok; 301 vs 256 at 8k depth).
-- Ships the model's own f16 vision tower (`mmproj/`).
-- The MTP head is quantized to match the target (ROCmFP4-FAST) — a Q8_0
-  draft measured worse on this head family — but it does not load on
-  build-qwen38 (Known issues #3).
-
-## Runtime
-
-charlie12345/ROCmFPX `llama-server` built at
-`/home/dank/GitPackages/ROCmFPX/build-qwen38` (HIP, ROCm 7.2.4, gfx1151,
-PR #27742 qwen4exp merged). Upstream llama.cpp cannot load the ROCmFP4
-quant types (`TYPE_100`/`TYPE_101`/FAST variants) — verified:
-
-```
-strings build-qwen38/bin/llama-server | grep Q4_0_ROCMFP4_FAST
-```
-
-CIRU binaries (ciru-ai v1.1) do **not** carry the FAST types; do not point
-this model at the CIRU runtime. The build previously served the kingjones
-STRIX / STRIX_LEAN models; the agentionai-FAST pair is the first use of the
-author's own matched MTP head on this binary.
-
-## Files / download
-
-| Item | Path on Dankclaw | Size |
-|---|---|---|
-| Main (v2 joined) | `/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/v2/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf` | 93,484,242,816 B (87.06 GiB) |
-| Main (root ple16, unused) | `/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2-ple16.gguf` | 93,484,237,760 B (87.06 GiB) |
-| Vision | `…/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf` | 904,004,128 B |
-| Draft head | `/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf` | 2,444,519,296 B |
-
-Draft head sha256 (verified against HF LFS):
-`5046d69571bb35c699e19c7a00b36633c2fdfdd06d70482551744ec523a6b590`
-
-Downloaded with a resumable plain `curl -L -C -` (Hugging Face `resolve/main`
-URLs; no auth token needed for public repos). The mmproj lives in an `mmproj/`
-subdirectory of the repo.
-
-## Serving profile
-
-Card sampling table (instruct, non-thinking) mixed with the validated
-build-qwen38 profile:
-
-```
-llama-server \
-  --model Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf \
-  --mmproj mmproj-Qwen3.8-Flash-Next-f16.gguf \
-  --host 0.0.0.0 --port 7770 --jinja \
-  --device ROCm0 --gpu-layers 999 \
-  --split-mode none --fit off --flash-attn on \
-  --ctx-size 32768 --parallel 1 \
-  --batch-size 4096 --ubatch-size 4096 \
-  --cache-type-k f16 --cache-type-v f16 \
-  --threads 16 --threads-batch 16 \
-  --ctx-checkpoints 16 --cache-ram 8192 \
-  --cache-prompt --cache-idle-slots --metrics \
-  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0 --presence-penalty 1.5 \
-  --reasoning on --reasoning-format deepseek --reasoning-budget -1 \
-  --chat-template-file chat-template.jinja \
-  --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}' \
-  --spec-type ngram-mod
-```
-
-Key choices and deviations from the CIRU deployment:
-
-- **`--fit off`** — auto-fit segfaults on qwen4exp hyper-connection config
-  (same as every Flash-Next deployment).
-- **`--checkpoint-min-step` removed** — build-qwen38 does not implement it
-  (verified via `--help`); only `--ctx-checkpoints` is passed.
-- **No PLE sidecar args** — the n-gram table is in-GGUF (joined
-  `per_layer_token_embd.weight` from the `v2/` layout), not a `ple/`
-  CIRUPLE1 sidecar. `--ple-sidecar`/`--ple-cache-mib` and the
-  `GGML_QWEN4EXP_PLE_*` / `GGML_CUDA_Q41_MOE_FORCE_J` env knobs are CIRU-
-  runtime-only and are gone.
-- **`--mmproj` active** — vision tower included (f16, lossless).
-- **`--flash-attn on` (restored 2026-08-29).** fa on previously aborted graph
-  build: the QSA indexer path (layers with attention compression) called
-  `ggml_fill` on the F16-cast kq mask (`GGML_ASSERT(a->type == GGML_TYPE_F32)`,
-  ggml.c:5371, backtrace through `build_attn_qsa`). Fixed in the fork source
-  by the QSA F32-mask patch — vendored at
-  `scripts/patches/qwen4exp-qsa-f32-mask.patch` (sha256
-  `f2f4a486e21cfad80bfa42683281d18e1cba22fa43c5e0e2aa0cfdbe141e925f`);
-  the binary at `build-qwen38/bin/llama-server` (rebuilt 2026-08-29) already
-  carries it. A clean rebuild must re-apply
-  `git apply scripts/patches/qwen4exp-qsa-f32-mask.patch` inside
-  `/home/dank/GitPackages/ROCmFPX` before `make`. Verified live at
-  CTX_SIZE=32768: model loads, health 200, real completion.
-- Context capped at **32768 × 1 slot — the author's own quick-start**. This
-  model is fully GPU-resident (87.06 GiB): the PLE n-gram table sits inside
-  the GGUF as one tensor, unlike STRIX_LEAN which streamed it from RAM.
-  HIP heap is 119.3 GiB; the context buffer is 50.8 GiB at 131072 and
-  103 GiB at 262144, so either window OOMs (`cudaMalloc … failed: out of
-  memory`, verified live on first graph alloc). At 32768 the buffer is
-  ~13 GiB (~100 GiB total) and loads clean.
-- MTP is disabled (ngram-mod only) — see the MTP section below.
-
-## MTP draft head — INCOMPATIBLE on build-qwen38
-
-The author's own matched head (`agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-
-FAST-GGUF`, 2.28 GiB, `qwen4exp` arch) does **not** load on this binary:
-
-```
-error loading model: missing tensor 'blk.0.hc_attn_norm.weight'
-```
-
-The build-qwen38 qwen4exp loader requires a full per-layer
-`blk.<il>.hc_attn_norm.weight` set in the draft (the graph reads all 48
-layers); the head only carries `blk.48.hc_attn_norm.weight` (count 1 — all
-three Flash-Next-era heads examined on this fork have that same partial
-layout). Verified three times, not a cross-vendor artifact:
-
-1. EasiiX Q8_0 head vs kingjones STRIX_LEAN main (2026-08-28) — failed.
-2. agentionai FAST head vs kingjones STRIX_LEAN main (2026-08-28) — failed.
-3. agentionai FAST head vs agentionai v2 main (2026-08-29, this deployment)
-   — failed identically.
-
-The 2.28 GiB head stays on disk; the moment a binary or head appears that
-carries the full per-layer norm set, re-enable with
-`SPEC_TYPE=draft-mtp,ngram-mod`, `--spec-draft-n-max 3` (author-measured
-best: 32.4 t/s, 0.612 acceptance; adaptive p-split measured worse).
-
-## Config files
-
-- Env: `scripts/frank-dragoon.env`
-- Serve: `scripts/serve-frank-dragoon.sh`
-- Chat template: `scripts/chat-template.jinja` (unchanged, Qwen-Sharp)
-- Fork patch: `scripts/patches/qwen4exp-qsa-f32-mask.patch` (re-apply after
-  any clean fork rebuild — see Serving profile)
-
-## Swap procedure
-
-```bash
-sudo systemctl stop frank-dragoon
-# edit scripts/frank-dragoon.env + scripts/serve-frank-dragoon.sh, commit, push
-# on Dankclaw: git pull --ff-only && scripts/deploy.sh dankclaw --service frank-dragoon
-sudo systemctl start frank-dragoon   # or via deploy.sh
-curl http://192.168.0.143:7770/health
-```
-
-Model load is slow (87 GiB + 2.3 GiB draft): health returns 503 while
-loading; `deploy.sh` waits up to 20 minutes (`LLM_WAIT`). Do not kill it.
-
-## Known issues
-
-1. **ROCmFP4 types require the ROCmFPX fork.** Stock llama.cpp and the CIRU
-   runtime reject these files.
-2. **Use the `v2/` joined layout, NOT the root `-ple16` split.** This binary's
-   qwen4exp loader (qwen4exp.cpp:104-108) unconditionally requires a single
-   `per_layer_token_embd.weight` tensor and GGML_ASSERTs "qwen4exp is missing
-   the PLE n-gram table" against the per-head split. The root file only loads
-   in the LaurentZuijdwijk Vulkan fork (`vulkan-qwen4exp-rocmfpx`). The v2
-   file is byte-identical in weights, ~5 KB larger header, loads fully into
-   SVM unified memory (no `--ngram-on-disk` needed — kingjones STRIX ran a
-   50 GiB joined PLE table the same way).
-3. **No MTP draft head loads on build-qwen38.** The qwen4exp loader
-   requires a full per-layer `blk.<il>.hc_attn_norm.weight` set; every
-   Flash-Next-era head checked (EasiiX Q8_0, agentionai FAST, CIRU) carries
-   only `blk.48`. Spec decoding is ngram-mod only. See the MTP section.
-4. **`--flash-attn on` (restored 2026-08-29).** fa on aborts graph build
-   without the QSA F32-mask patch — see Serving profile (fix + sha256
-   vendored there).
-5. **`SVM mapping failed` kernel spam** is a documented symptom, not root
-   cause — investigate before blaming memory pressure.
-
-## Citations
-
-- [agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF)
-- [agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF)
-- [Qwen/Qwen3.8-Flash-Next base model](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)
-- [charlie12345/ROCmFPX](https://github.com/charlie12345/ROCmFPX) — fork with PR #27742 (qwen4exp architecture) and the ROCmFP4 types
-- [llama.cpp PR #27742](https://github.com/ggml-org/llama.cpp/pull/27742) — qwen4exp support (merged upstream)
-- [llama.cpp PR #27739](https://github.com/ggml-org/llama.cpp/pull/27739) — MTP draft head support

serverAppConfigs-docs/LLM/ciru-activefpx-promptforge.md

@@ -1,266 +0,0 @@
----
-type: reference
-title: CIRU ActiveFPX PromptForge on Dankclaw
-description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (v3) served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-dragoon service.
-tags:
-  - llama.cpp
-  - rocm
-  - dankclaw
-  - gfx1151
-  - ciru
-  - promptforge
-timestamp: 2026-08-22
----
-
-# CIRU ActiveFPX PromptForge on Dankclaw
-
-> **RETIRED 2026-08-24.** `frank-dragoon` served this v3 model from
-> 2026-08-22 (restored after a same-day Kairic Edge IU4 experiment was
-> reverted — operator decision, commit `c60156f`; see [[kairic-edge]])
-> until [[eemin-rocmi4-mtp]] took over on 2026-08-24. Kept as the
-> historical record of the v3 deployment.
-
-`frank-dragoon` (port 7770, moved from 52625 on 2026-08-22) serves
-`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU
-ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that
-previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21.
-
-## What This Is
-
-The model is a **GGUF**, not a vLLM checkpoint. v3 is the consolidated
-quality-and-speed release of Qwen3.8-27B for AMD ROCm: an FFN-equalized
-compact body, 50 retained late-writer Q6 tensors, full-Q8 output
-projection, full-W8 PromptForge view (FFN + Gated DeltaNet), the
-small-row W8 route, a draft-only Output-K8 shortlist proxy with Q8
-reranking, native depth-4 MTP (no ngram), and the BF16 Qwen3.8 vision
-tower + 5120-wide merger as a separate projector.
-
-| Artifact | Size |
-|----------|------|
-| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3-Q8-FFNEQ-LateQ6.gguf` | 15.44 GiB |
-| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-FFN.pfs` | 15.95 GiB |
-| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-GDN.pfs` | 3.75 GiB |
-| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-Output-K8.pfs` | 0.67 GiB |
-| `Qwen3.8-27B-mmproj-BF16.mmproj` | 0.87 GiB |
-| Full v3 payload (GGUF + 3 .pfs + mmproj) | ~36.7 GiB |
-
-The `.pfs` companions and the Output-K8 proxy are optional. Omit them to
-run the GGUF alone and save ~20.6 GiB of memory. The GGUF remains
-independently usable.
-
-## Requirements
-
-- **ROCm 7.15 is required** for the advertised Strix Halo performance.
-  The system ROCm (`/opt/rocm-7.2.4`) selects slower or fallback kernels.
-- The pinned runtime is `ciru-ai/ROCmFPX` tag
-  `qwen3.8-activefpx-promptforge-v2.3` (commit `103af8b41`) **plus the
-  v3 runtime patch** `runtime/qwen38-v3-output-k8-runtime.patch`. On
-  Dankclaw the patch is commit `af0eac6` on top of the v2.3 checkout.
-- Composable Kernel at `fdf4bb7fcc984811cef48ce817d89aac064b984a`.
-- Validated GPU: AMD Radeon 8060S / gfx1151.
-
-## ROCm 7.15 Toolchain
-
-The retired `ling3-ciru` package left a complete TheRock ROCm 7.15 SDK
-(`7.15.0a20260728`) inside its venv:
-
-```
-/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/
-  _rocm_sdk_devel/      # hipcc, amdclang 23, cmake, headers
-  _rocm_sdk_libraries/  # hipblas, hipblaslt, rocroller
-  _rocm_sdk_core/       # libamdhip64, libhsa-runtime64
-  rocm-sdk-device-gfx1151
-```
-
-The compiler is amdclang 23. `rocm-sdk version` reports `7.15.0a20260728`.
-The binary is built against and run with these 7.15 libraries.
-
-## Build Runbook (2026-08-18, v2.3; +v3 patch on 2026-08-21)
-
-Step 1-4 below built v2.3 on 2026-08-18. On 2026-08-21 the v3
-Output-K8 runtime patch was applied on top and the server rebuilt:
-
-0. Apply the v3 runtime patch to the v2.3 checkout and rebuild:
-   ```
-   % verify the patch hash: 8ba7d721ea380b060aeb0c7595b8b05f1582c0f7bf7a4b89e0910bc8e72465d5
-   git am runtime/qwen38-v3-output-k8-runtime.patch   # -> commit af0eac6
-   cmake --build build-promptforge --target llama-server -j$(nproc)
-   ```
-   The pre-built binary is kept as `build-promptforge/bin/llama-server.v2.3-bak`
-   for rollback.
-
-1. Install host build tools (ninja-build was missing): `sudo apt install ninja-build`.
-2. Clone the runtime and CK at their pinned revisions:
-   ```
-   git clone --depth 1 --branch qwen3.8-activefpx-promptforge-v2.3 \
-     https://github.com/ciru-ai/ROCmFPX.git
-   git clone --depth 1 https://github.com/ROCm/composable_kernel.git
-   git -C composable_kernel checkout fdf4bb7fcc984811cef48ce817d89aac064b984a
-   ```
-3. CK headers are generated at CK configure time (`ck/config.h`,
-   `ck/version.h`). The llama.cpp build includes the CK source tree, so run
-   CK's configure once and copy the headers into `include/ck/`:
-   ```
-   cmake -S composable_kernel -B build-ck -DAMDGPU_TARGETS=gfx1151 -DCK_ENABLE_WMMA=ON
-   cp build-ck/include/ck/config.h build-ck/include/ck/version.h composable_kernel/include/ck/
-   ```
-4. Configure the llama.cpp build against the venv ROCm 7.15 SDK:
-   ```
-   export ROCM_PATH=$SDK/_rocm_sdk_devel
-   cmake -S ROCmFPX -B build-promptforge -G Ninja \
-     -DCMAKE_BUILD_TYPE=Release \
-     -DCMAKE_C_COMPILER=$ROCM_PATH/llvm/bin/clang \
-     -DCMAKE_CXX_COMPILER=$ROCM_PATH/llvm/bin/clang++ \
-     -DCMAKE_HIP_COMPILER=$ROCM_PATH/llvm/bin/clang++ \
-     -DCMAKE_PREFIX_PATH=$ROCM_PATH \
-     -DGGML_HIP=ON -DGGML_CUDA=OFF -DGGML_VULKAN=OFF \
-     -DGGML_HIP_FORCE_MMQ=ON -DGGML_HIP_GRAPHS=ON \
-     -DGGML_HIP_MMQ_MFMA=ON -DGGML_HIP_NO_VMM=ON \
-     -DGGML_HIP_ROCWMMA_FATTN=OFF -DGGML_NATIVE=ON \
-     -DAMDGPU_TARGETS=gfx1151 -DGPU_BUILD_TARGETS=gfx1151 \
-     -DPROMPTFORGE_CK_ROOT=$CK_ROOT \
-     -DLLAMA_BUILD_SERVER=ON \
-     -DLLAMA_BUILD_WEBUI=OFF
-   cmake --build build-promptforge --target llama-server -j$(nproc)
-   ```
-   `LLAMA_BUILD_WEBUI=OFF` is required. The embedded browser UI downloads
-   assets from an HF bucket (`llama-ui`) that returns 404; the service is
-   consumed through the OpenAI-compatible API, so the UI is not needed.
-
-Build output: `build-promptforge/bin/llama-server`.
-
-```mermaid
-graph TD
-    A[Clone ROCmFPX v2.3 + CK pinned] --> B[Configure CK, copy generated headers]
-    B --> C[Configure llama-server vs ROCm 7.15 SDK]
-    C --> D[Build llama-server]
-    D --> E[Verify --list-devices shows gfx1151]
-    E --> F[Download v3 GGUF + FFN/GDN/Output-K8 + mmproj]
-    F --> G[Deploy frank-dragoon]
-    G --> H[Health check + chat completion]
-```
-
-## Model Download
-
-With the HF token to the v3 staging directory
-`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. The v3 files live there
-so the v2 artifacts in `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX/` stay
-intact for rollback. Download ran on Dankclaw with the background script
-`/tmp/download-v3.sh`; all five files + the patch passed SHA-256 checks
-against the model card before the service restart.
-
-## Service
-
-- Unit: `frank-dragoon.service`, port 7770
-- Script: `scripts/serve-frank-dragoon.sh`, env `scripts/frank-dragoon.env`
-- The script exports the ROCm 7.15 SDK libraries in `LD_LIBRARY_PATH` before
-  the system ROCm. `ROCM_PATH` points at the devel SDK.
-
-### Launch profile
-
-- 262144 context (native max), 1 slot (user request 2026-08-19; parallel
-  slots degraded performance), f16 KV (~16 GiB), context checkpoints (16),
-  prompt cache. Concurrent requests queue on the single slot; the server
-  never preempts a running generation
-- Native MTP draft, depth 4, device ROCm0
-- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`
-  puts `<think>` reasoning in `message.reasoning_content`)
-- v3 runtime: Output-K8 draft proxy (top-8 shortlist, Q8 rerank) +
-  small-row W8 prompt route, full-W8 FFN/GDN views
-- PromptForge env: `PROMPTFORGE_MODE=m2048_fused_tail1476`,
-  `GGML_CUDA_GRAPH_OPT=0`, `PROMPTFORGE_MTP_OUTPUT_K8=1`,
-  `PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE=0`,
-  `PROMPTFORGE_ENABLE_SMALLM_W8=1`
-
-### Required environment
-
-```
-export PROMPTFORGE_SIDECAR=...FFN.pfs
-export PROMPTFORGE_GDN_SIDECAR=...GDN.pfs
-export PROMPTFORGE_MODE=m2048_fused_tail1476
-export GGML_CUDA_GRAPH_OPT=0
-export PROMPTFORGE_ENABLE_SMALLM_W8=1
-export PROMPTFORGE_MTP_OUTPUT_K8=1
-export PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE=0
-export PROMPTFORGE_MTP_OUTPUT_K8_PROXY=...Output-K8.pfs
-# v3 qualified profile: disable the CPU argmax draft fast path and any
-# IU4/ngram lane. The Output-K8 proxy + backend sampling replace them.
-unset LLAMA_MTP_CPU_ARGMAX_FASTPATH LLAMA_MTP_BACKEND_GREEDY
-unset PROMPTFORGE_IU4_SIDECAR PROMPTFORGE_GDN_IU4_SIDECAR
-unset PROMPTFORGE_ENABLE_NGRAM_MOD PROMPTFORGE_ENABLE_NGRAM_M65_IU4
-```
-
-The v3 qualified profile runs the MTP draft through the Output-K8 proxy,
-not the CPU argmax fast path. The old `LLAMA_MTP_CPU_ARGMAX_FASTPATH=1`
-profile is retired for this slot. A TOP_K sampler warning may still appear
-in the journal once per start; the Output-K8 draft path does not depend on
-it and generation proceeds.
-
-## Operational Notes
-
-- Health: `curl http://127.0.0.1:7770/health`. Returns 503 while loading.
-- Model load is slow (v3 five-file payload ~36.7 GiB); do not kill the
-  process during warmup. The page cache makes a repeat load fast right after
-  download, but a later cold start takes several minutes.
-- `deploy.sh dankclaw` waits 20 minutes for health.
-- The systemd unit must not set `HOST` or `TEMP` env vars (HIP collision
-  rule). `--host`/`--port`/`--temp` are CLI flags only.
-- v3 adds the BF16 vision tower over v2. The service reports `multimodal`
-  capability; mmproj loads with the language model (CLIP on ROCm0 backend).
-
-## ngram-mod + MTP Stacking (benchmarked 2026-08-21)
-
-This page records a research question: does ngram-mod add value when
-stacked under the native MTP draft? The v3 profile runs MTP alone; the
-benchmark below compares that against the stacked pair.
-
-### Feasibility
-
-ngram-mod DOES stack with MTP in this CIRU fork. The v3 build compiles in
-both `ngram-mod` and `draft-mtp` (`common/speculative.cpp`). Pass
-`--spec-type draft-mtp,ngram-mod` (or `ngram-mod,draft-mtp`) and the fork
-initializes both implementations in the same process. The boot log shows:
-
-```
-adding speculative implementation 'ngram-mod'
-initialized ngram_mod with n_match=24, size=4194304 (16.000 MB)
-adding speculative implementation 'draft-mtp'
-```
-
-The model card's "no ngram subsystem" line refers to the CIRU PromptForge
-env lanes (`PROMPTFORGE_*_NGRAM_*`, `PROMPTFORGE_ENABLE_NGRAM_MOD`), not
-the upstream llama.cpp spec chain. Those env lanes must stay unset; the
-`--spec-type ngram-mod` chain is separate and works.
-
-### Ordering — the CLI order does not matter
-
-The `--spec-type` string order is discarded. `common_speculative_init`
-collapses the requested types into a bitmask, then rebuilds the chain in a
-hard-coded priority: ngram types first, then draft-simple, then MTP.
-So `draft-mtp,ngram-mod` and `ngram-mod,draft-mtp` produce the same
-internal order (`ngram-mod` then `draft-mtp`) and the same runtimes.
-Verified: both orders decode ~25.3 t/s at 8k.
-
-### Results (decode generation t/s, 200 tokens, prose prompt)
-
-| ctx | MTP only | MTP + ngram-mod | verdict |
-|-----|----------|-----------------|---------|
-| 8k | 29.42 | 25.27 | ngram hurts -4.15 |
-| 16k | 26.42 | 21.13 | ngram hurts -5.29 |
-| 64k | 19.72 | 18.00 | ngram hurts -1.72 |
-
-At every depth, adding ngram-mod under MTP REDUCED decode throughput. The
-ngram draft costs its own forward pass / occupancy without adding enough
-accepted tokens on top of MTP. Baseline (no ngram) remains the production
-config (`--spec-type draft-mtp` only). Keep ngram-mod off.
-
-The historical harness was moved to `.AGENTTRASH/bench-scripts-20260901/` and is retained for audit only.
-it boots the v3 stack on a scratch port and reads authoritative
-`print_timing` lines. It requires the production service to be stopped.
-
-## See Also
-
-- [[frank-redo-2026-08-17]] — the fleet topologies this slot has served.
-- [[Shared Memory Architecture Dankclaw]] — unified memory sizing.
-- [[qwen3.8-strix-optimizations]] — llama.cpp ROCm0 baselines on this iGPU.
-- [[VLLM Overview]] — vLLM reference (retired for this slot).

serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md

@@ -205,7 +205,7 @@ answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
 slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the
 boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total
 is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working),
-see [[frank-dragoon-flash-server-v2-native]].)
+see [[frank-dragoon]].)
 
 **The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
 Krackan/Strix Halo NPU rev 11), firmware `amdnpu/17f0_11/npu_7.sbin`, `Initialized
@@ -229,8 +229,8 @@ lives in `~/.local/bin`.
 ## Related
 
 - [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
-- [[halogen-flash-server]] — the runtime bought by the prefill figure
-- [[kingjones-strix-rocmfp4]] — the runtime it replaced (rollback target)
+- [[frank-dragoon]] — the runtime bought by the prefill figure
+- [[frank-dragoon]] — the runtime it replaced (rollback target)
 
 ## Citations
 

serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md

@@ -166,7 +166,7 @@ Not advisable.
 | Runtime | gfx1201 | Architectures | Note |
 |---|---|---|---|
 | **Affinity** | yes, built here | **V4-Flash only** — "the geometry is compiled in; there is no architecture abstraction" | 110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model |
-| **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[dankserver-qwen38-fp8-expert-offload-tuning]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels |
+| **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[frank-archon]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels |
 | **Lucebox** | yes, image on host | Qwen 3.8, DS4; **V4.1 on the unmerged `feat/ds41-engine` branch** | The only V4.1 path. **Refuses layer split and layer-major prefill for V4.1** |
 | **radiance-vLLM** (drwolfen) | yes — served frank-archon 2026-09-27 → 09-30 | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture |
 | **FreeToken-ROCm** | partial | block-FP8 safetensors only (modelopt); known `CpuMoeExecutor` issue under ROCm | cannot load a GGUF |
@@ -221,7 +221,7 @@ gfx1201+gfx1201. hip:0 takes dense weights and the hot expert stack;
 
 ## See Also
 
-- [[frank-archon-lucebox-ds4]] — the retired Lucebox frank-archon runbook
+- [[frank-archon]] — the retired Lucebox frank-archon runbook
 - [[Dual-R9700-Cooperation]] — the PCIe/atomics facts behind the board swap
 - [[Dankserver System Specs]] — the host (RAM corrected 2026-09-28)
-- [[frank-archon-r9v-radiance-qwen38-q8]] — the ACTIVE frank-archon
+- [[frank-archon]] — the ACTIVE frank-archon

serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md

@@ -1,724 +0,0 @@
----
-type: research
-title: Qwen3.8-Flash-Next FP8 on tcclaviger — expert-offload decode tuning
-description: VERIFIED 2026-09-29 by measurement. The flag-level decode-tuning campaign on the ad hoc qwen-fp8 container (orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8, tcclaviger RDNA4 vLLM fork, 2x R9700): the >35 tok/s target is MET at 37.4-40.6 tok/s by turning prefetch OFF — telemetry measured its precision at 36%, wasting ~half of all gather traffic — while the original 26.8 tok/s incumbent (prefetch on) survived every other challenger. Residual upside is the fork's gather read path (effective ~6.5-9.5 GB/s vs 22.7 GB/s the link delivers at the same slab size).
-tags:
-  - dankserver
-  - r9700
-  - gfx1201
-  - qwen3.8
-  - fp8
-  - expert-offload
-  - speculative-decoding
-  - tuning
-timestamp: 2026-09-29
----
-
-# Qwen3.8-Flash-Next FP8 — expert-offload decode tuning
-
-> **Question this page answers.** On the tcclaviger RDNA4 vLLM fork serving
-> `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` across both R9700s, what
-> runtime flags actually move decode at a 32k working point?
->
-> **Answer in one line.** **37.44–40.57 tok/s — the >35 target is met — by
-> deleting `--expert-prefetch 1`.** Expert-cache telemetry measured the
-> prefetcher at **36% precision** (36,720 slabs prefetched, 13,369 served):
-> ~half of all gather traffic was useless copies, and the copies it "saved"
-> would have been installs anyway. The prefetch-on incumbent (26.8 tok/s)
-> survived every other challenger measured: gather geometry, spec depth,
-> TunableOp, scheduler batch knobs, concurrency, ngram-only drafting.
-
-This is the ad hoc `qwen-fp8` container on Dankserver (rootless podman,
-image `docker.io/tcclaviger/vllm:latest`, engine
-`v0.29.0.dev0+g2bdbbc8080`), NOT the `frank-archon` systemd service —
-the R9V production unit is untouched, so no stop-and-mask was required for
-any of this. All numbers are decode tok/s measured from the server's own
-`usage.completion_tokens / (wall - ttft)` — SSE chunk counting is invalid
-under speculative decoding because rejected drafts change the accounting.
-
-## Deployment shape (the tuned incumbent)
-
-Serve script on the host (`/home/dank/serve_qwen_fp8.sh`), env hooks in
-CAPS. **PREFETCH must be unset** (omitting the flag; the fork's default is
-off):
-
-```
-MODEL=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8   (~172.8 GiB, 131 shards)
-IMAGE=docker.io/tcclaviger/vllm@sha256:da4029666668b90a0386c1ed23ea370c79916ad4e1a6ad1dd9db098b9fa069ea
-# image ID 3322dbbbc96d; local alias qwen-fp8-runtime:tuned-20260929;
-# the floating :latest tag moved once already — always launch by digest.
-# repo tag of this state: qwen-fp8-tuned-20260929
---tensor-parallel-size 2  --max-model-len 32768  --gpu-memory-utilization 0.98
---kv-cache-memory-bytes 2147483648  --max-num-seqs 4  --enable-prefix-caching
---speculative-config '{"method":"mtp","num_speculative_tokens":4}'
-# NO --expert-prefetch: measured 36% precision, ~50% wasted gather traffic
-```
-
-Published to the host LAN via the serve script's `PORTMAP` hook
-(`PORTMAP=7773:7770`): health and completions answer on
-`http://192.168.0.235:7773` (verified host-side 2026-09-29; 256k config,
-decode 36.49/34.55/35.98 through the published path). Host port 7773 was
-chosen over 7770 because `frank-archon` (7770) is stopped but ENABLED —
-any future start would collide with a published 7770.
-
-Device notes carried from the bring-up: `--device /dev/dri` must be the
-WHOLE directory (individual renderD nodes enumerate but refuse contexts —
-"No CUDA GPUs are available" while device_count() reports 2); the image
-entrypoint execs `vllm serve` itself.
-
-The memory plan this produces (from the engine's own `expert_plan.py` log):
-
-| region | size |
-|---|---:|
-| expert cache space | 15.05 GiB = 6,574 slots x 2.344 MiB |
-|   VRAM-only resident layers | 3.52 GiB (layers 0-2, 1,536 experts) |
-|   swap region | 11.53 GiB (5,038 slots, 111-112/layer) |
-| host-backed experts | 52.74 GiB per rank, 23,040 experts |
-| KV reserve (one max-len request +8%) | 0.36 GiB of the 2 GiB pool |
-| CUDA graphs | 1.93 GiB (21 graphs; 1.32 GiB at `--max-num-seqs 2`) |
-| runtime overhead | 7.6 GiB (non-torch + buffers + activation) |
-
-## Measured arms (2026-09-29)
-
-Decode tok/s at 166 / 1087 / 4064 prompt tokens, median of 3 runs after a
-64-token warm pass, temperature 0, max_tokens 256:
-
-| Arm | 166 | 1087 | 4064 | Acceptance | Verdict |
-|---|---:|---:|---:|---:|---|
-| **MTP4, stock 8x16, PREFETCH OFF** | **40.57** | **37.44** | **37.57** | 54.0% | **NEW INCUMBENT — target met** |
-| MTP4, stock 8x16, prefetch ON | 26.80 | 25.15 | 26.81 | 60.7% | old incumbent (+51%/+40% worse) |
-| MTP4, prefetch ON (re-check) | 25.80 | 24.38 | 26.63 | 47.9% | reproduces (noise band ±1) |
-| MTP4, prefetch ON, gather 16x32 | 24.30 | 22.77 | 24.11 | 50.3% | −9% — ruled out |
-| MTP4, prefetch ON, gather 32x64 | 26.58 | 24.94 | 26.34 | 49.6% | parity — ruled out |
-| MTP3, prefetch ON | 24.47 | 24.41 | 24.44 | 66.7% | −9%, flat in ctx — ruled out |
-| MTP6, prefetch ON | 19.07 | 17.81 | 13.54 | 17.4% | collapses; crosses UVA threshold — ruled out |
-| MTP4, prefetch ON, MNBT 8192 | 29.32 | 24.12 | 23.65 | — | neutral — ruled out |
-| MTP4, prefetch ON, PREFETCH depth 2 | 25.83 | 25.06 | 25.87 | 56.0% | neutral — ruled out |
-| MTP4, prefetch ON, TunableOp w/ offload | 23.59 | (79.82*) | (79.89*) | (96.5%*) | **rejected — corrupts** |
-| 2-way concurrency, prefetch ON (2 streams) | — | — | 15.06 + 23.11 = **25.49 summed** | — | no gain vs 26.8 single |
-| ngram-only drafter | — | — | — | — | fork bug: PLE inputs unprepared |
-| 256k ctx (MAXLEN=262144, KV 4.5 / 4.03 GiB, prefetch off) | 36.65 / 36.48 | 34.21 / 34.13 | 34.77 / **35.02** | 53.0 / 39.4% | boots and serves; band 34.1-36.5 straddles 35 within noise; arena identical (5,648) at both KV sizes |
-
-\* The TunableOp numbers are invalid by construction: the tuned GEMMs
-changed generated behavior (repetition-loop, the server's
-"repetition detected" halt firing at ~140-146 tokens instead of the full
-256, acceptance jumping 60.7% → 96.5%), the 79.8 tok/s figures were
-measured on those halted degenerate completions, and the text-only check on
-a fresh prompt came back 20.01 tok/s — *below* baseline. GEMM autotuning
-that changes output text is broken tuning.
-
-The decode curve without spec decode is dead flat 12.11 tok/s from 304 to
-16,095 tokens — decode is context-independent on this shape, bound per-step
-by the expert gather, not by KV depth.
-
-## Telemetry — the measurement that found the win
-
-`R4D_LRU_TELEMETRY=1` arms per-layer counters; a torch-profiler window
-(`/start_profile` → traffic → `/stop_profile`) makes the fork log and write
-`expert_cache_rank<N>.json` beside the traces. **`torch_profiler_with_stack=false`
-is mandatory in the profiler config** — the default (true) grew a 6.5-min
-window's trace buffer ~40 GB and the kernel OOM-killed the engine core
-(anon-rss 95.2 GB, `oom-kill` in `journalctl -k`). The short window at
-166 ctx (~15 s of traffic) completed cleanly with `with_stack=false`.
-
-The fork's own numbers (102 steps, both ranks identical):
-
-```
-hit rate 0.8245 (137,084/166,262)   host installs 15,814
-read-through 13,364                 prefetched 36,720
-served by prefetch (migrations) 13,369
-```
-
-- **The LRU is good**: 82.45% hit rate overall; layers 0-2 pinned at
-  100%; swap-region layers 78-81%. The "arena holds only 27% of experts
-  → 73% miss" arithmetic was wrong — temporal reuse carries it.
-- **The prefetcher is the poison**: 36,720 slabs moved to serve 13,369
-  (36% precision). The 23,351 unused copies are ~0.7 GB/step/rank of pure
-  waste — roughly half of all gather traffic — and the 13,369 "savings"
-  would have been installs anyway. Depth-1 prefetch also cannot cover the
-  ~2,400-slab per-step working set with its 16 slots/layer (768 total).
-- Per-layer-step at a swap layer: ~30.6 routed, 24.8 hits, 3.3 installs,
-  2.6 read-through, 8.0 prefetched of which 1.7-3.3 useful.
-
-## The wall, measured
-
-- **PCIe link (pcie-link-probe):** GPUs `03:00.0`/`07:00.0` LnkCap/LnkSta
-  32GT/s x16; root ports `00:01.1`/`00:01.3` cap x8. Effective **Gen5 x8 =
-  31.5 GB/s/direction**. `rocm-smi pcie_link_speed: 25` is the ASPM idle
-  downshift, not the loaded rate.
-- **H2D (HIP probe, pinned):** 2.345 MiB (the engine's exact expert slab)
-  **22.74 GB/s**; 16 MiB 28.19; 1 GiB 28.74 (91% of ceiling). Pinned ≈
-  pageable — staging is not the lever. Single-transfer latency at slab size
-  105.7 µs vs 78 µs ideal → ~21% per-transfer overhead.
-- **Engine's effective gather rate:** ~6.5-9.5 GB/s = **30-42% of what the
-  link delivers at the same size**. The bottleneck is the fork's read path
-  (chunk/lane geometry, per-transfer cost), not the wire.
-- **Duty cycle:** peak GPU util 100%, smooth ~29% with prefetch on — each
-  step is ~100 ms of compute then hundreds of ms waiting on expert
-  streaming. rocm-smi reports *utilization*, which reads ~0 while weights
-  load during an arm's reload; VRAM is the honest panel there.
-
-## What the remaining flags cannot do (all measured)
-
-- **Gather geometry** — 16x32 and 32x64 measured no better despite the
-  fork's "tuned at the Gen4 ceiling" comment; the constants are already at
-  parity on Gen5 x8. The read-path gap is structural, not a tuning knob.
-- **Spec depth** — 4 is the optimum both ways (with prefetch on). Depth 6
-  additionally proves the UVA read-through path (`step tokens >
-  R4D_LRU_UVA_TOKENS=5`) is *slower* than copy-in.
-- **Prefetch anything** — depth 2 neutral, depth 1 the new negative; the
-  tier is 3x too small for the working set and the predictor is 36%
-  precise. `expert_prefetch_k/budget/slots` exist (defaults 20/14/16) but
-  the precision result makes tuning them pointless.
-- **Arena growth via shrink** — `UTIL=0.99` fails boot (`Free memory on
-  device cuda:1 (31.29/31.86)`); `--max-num-seqs 2` fails CUDA-graph
-  capture (`Constraints violated (L['query_start_loc'].size()[0])`); and
-  the MNBT 8192 arm already proved a ~1 GiB arena shift is decode-neutral
-  in both directions.
-- **Concurrency** — under TP=2 both streams' tokens are computed on both
-  cards, so each rank gathers the union of both streams' misses; the links
-  do not add capacity and the shared arena thrashes. Summed 2-way (25.49)
-  measured *below* single-stream (26.8) with prefetch on.
-- **ngram-only drafting** — fails at engine init:
-  `RuntimeError: PLE inputs were not prepared`; the fork prepares this
-  model's PLE inputs only on the MTP spec path. Needs a fork patch, not a
-  config.
-- **V-only q8 KV** — stock `--kv-cache-dtype` offers fp8 variants only
-  (K+V together); a V-only q8 needs a fork patch.
-
-## Fork bugs / traps found (for the engine authors or a future patch)
-
-1. `CLAV_TUNABLEOP_SWEEP=1` is silently skipped whenever expert offload is
-   active unless `CLAV_TUNABLEOP_WITH_OFFLOAD=1` is also set
-   (`tunableop_warmup.py`).
-2. TunableOp-with-offload, once forced on: +6.7 min warmup, no persistent
-   cache (container-local), and corrupt numerics (repetition loops,
-   early-EOS halts, invalid measurements). Rejected outright.
-3. The torch-profiler stop path hangs the workers on ROCm when
-   `torch_profiler_with_stack` is true (default): >10 min hangs on two
-   attempts while the AsyncLLM wrapper reports "stopped successfully" in
-   1 s; the default stack-tracing grew a 6.5-min window's buffer ~40 GB
-   and OOM-killed the engine core. With `with_stack=false` the stop
-   completes and the telemetry report runs. The telemetry counters remain
-   reachable ONLY through that profiler stop — no direct RPC trigger
-   exists (`telemetry_report` has exactly one caller, `gpu_worker.py`).
-4. ngram spec method does not prepare PLE inputs (above).
-
-## Where the headroom lives now
-
-The 35 tok/s target is met (37.4-40.6) with prefetch off. What is left:
-
-1. **Fork patch to the gather read path** — the ~6.5-9.5 → 22.7 GB/s
-   software gap at slab size. If the gather reached wire rate, decode
-   lands in the 60+ tok/s band before acceptance effects. Code work in
-   `expert_lru.py`'s gather (`R4D_LRU_GATHER_*` is read at import; the
-   copy-in path itself is compiled — see `/app/r4dhip/r4d.so`).
-2. **256k ctx — measured, final**: boots at `MAXLEN=262144` with
-   `KV_BYTES=4831838208` or `4328521728` alike — a 262,144-token window
-   needs **4.0 GiB** KV (hybrid attention + mamba pages ≈ 16 KiB/token,
-   not the 32k plan's 9,190 B/token; 3 GiB boot-refuses and the engine
-   estimates max len 184,224 at that pool; the exact-4.0 override loses
-   ~20 MB to accounting and boots as 3.98 < 4.0). Decode band
-   **34.1-36.5** across both KV sizes — the arena is **identical (5,648
-   slots)** at 4.5 and 4.03 GiB (the planner buckets; the trim went to
-   margin, not slots), so the window costs ~10% decode vs the 32k config
-   and no KV trim recovers it. The alternative that keeps the full
-   37.4-40.6 band is `MAXLEN=184224` at 3 GiB KV. Correction: the planner
-   sizes the arena AFTER the KV override (the earlier "before" reading was
-   wrong) — the arena is what remains.
-3. **2+ slots at long ctx** — the KV pool post-spec holds ~66k tokens at
-   2 GiB (~2x 32k streams); concurrency summed *below* parity at 4064 ctx,
-   so slots are a capacity feature, not a throughput one. Two full 256k
-   streams need ~8-10 GiB of KV — that fits the arithmetic only by cutting
-   the arena below usefulness; slots-at-256k is a fork-patch or
-   smaller-payload question.
-
-## Reproducing
-
-Host-side scratch scripts (2026-09-29): `/home/dank/serve_qwen_fp8.sh` (the
-serve script with all hooks), `/home/dank/mtp_measure.py` (usage-counter
-decode measure, 3 depths), `/home/dank/mtp_measure_short.py` (single depth,
-for profile windows), `/home/dank/bench2way.py` (concurrency), plus the arm
-scripts `arm_*.sh` / `sweep_*.sh` and the detached runner `/home/dank/job.sh`
-(outputs under `/home/dank/jobs/NAME.out|.rc|.done`). The measure scripts
-are `podman cp`'d into the container after every recreate (the serve script
-re-copies them; a recreate wipes container-local files).
-
-## 2026-10-01 — production cutovers (qwen FP8+MTP4, then bf16 no-spec)
-
-Two same-day cutovers moved this page's findings into production:
-
-1. **Qwen FP8 + MTP4 replaced the failed GLM/DFlash2 experiment** (the GLM
-   window is documented in [[frank-archon-glm53-flash-w4a16]]; git tag
-   `frank-archon-fp8-mtp-20261001` is the last commit of that shape). The
-   production trio took MAXLEN 200000 x 3, fp8 KV, MTP4, prefetch off —
-   measured 37.4-40.6 tok/s at ~54% acceptance.
-2. **The overnight redo campaign re-measured EVERY lever at matched
-   geometry** (prefetch OFF everywhere, MAXLEN 200000/SEQS 3/fp8 baseline)
-   and concluded the day: the incumbent cutover to **bf16 KV, no
-   speculative decode** (Frank's call).
-
-### What the redo campaign measured (all at shallow depth, 166/1087/4064)
-
-| lever | verdict |
-|---|---|
-| prefetch (re-confirmed) | the whole night's +40-51% lives in deleting it; the old 12.11 "no-spec baseline" was itself prefetch-taxed |
-| gather geometry | **R4D_LRU_GATHER_CHUNKS=32 / LANES=64 = +18.7%** (49.25 @4064), the win GROWS with depth (39.5@166 -> 54.1@1087); g8x16 mild, g64x128 past peak |
-| KV dtype | bf16 (`auto`) +4% decode on fp8-era numbers but halves attention-KV capacity (measured 14,047 vs 7,926 B/token; mamba states unaffected) |
-| spec family | no-spec fastest once the tax was gone (41.46/41.29/41.47); MTP4's drafter reserves ~2.6 GiB/rank beside the pool; untaxed MTP re-measures never completed |
-| MNBT | 512 doubles TTFT (2.46 s vs 1.36 s @4064); 1024-4096 interchangeable; 12288 -2.5% + arena; **locked 1024** |
-| UVA | 2048 POISON (-39%); 8192 ~parity |
-| arena | flat ±1.4% from ~6,830 down to ~5,500 slots; knee below |
-| six slots | every SEQS-6 KV sizing refused at engine init |
-
-Depth caveat: every number above is at a near-empty KV. Deep-context
-behavior (the gather win growing with depth, bf16's decode cost at depth)
-is only sampled at 3 depths; a 64k-192k-depth probe is the flagged gap.
-
-### The deployed shape (2026-10-03 shape G: 3 slots x 200704, fp8, precise memory, MNBT 2048, arena at the flag ceiling)
-
-```
-FRANK_ARCHON_MAXLEN=200704   200704-token window
-FRANK_ARCHON_SEQS=3          --max-num-seqs 3 (SEQS=2 CANNOT boot - below)
-FRANK_ARCHON_KV_BYTES=3166500000   PER RANK (fp8, 7,892 B/token -> 400,700 tokens)
-FRANK_ARCHON_KV_DTYPE=fp8    --kv-cache-dtype fp8 (quantized)
-FRANK_ARCHON_MNBT=2048       --max-num-batched-tokens 2048 (arena-free, see shape F)
-FRANK_ARCHON_UTIL=0.982      budget = total x util, the law (1.0 unbootable, see below)
-FRANK_ARCHON_CUDAGRAPH_BASE_MB=0 / _PER_GRAPH_MB=12  -e VLLM_CUDAGRAPH_MEM_*
-FRANK_ARCHON_PRECISE_MEMORY=1      --expert-precise-memory (overhead 7.60 -> 5.25 GiB)
-FRANK_ARCHON_EXPERT_OFFLOAD_MEM=120  --expert-offload-mem (host-region cap, explicit)
-FRANK_ARCHON_GPU_MAX_HW_QUEUES=2   -e GPU_MAX_HW_QUEUES=2 (HSA queues/GPU)
-FRANK_ARCHON_CACHE_DIR=/home/dank/.cache/frank-archon   mounted at /root/.cache
-R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64  (container env)
-NO --speculative-config, NO --expert-prefetch
-FRANK_ARCHON_NO_PREFIX_CACHING=1  --no-enable-prefix-caching (2026-10-05, the QFN align-mode GDN state fault — below)
-```
-
-### 2026-10-05 — prefix caching OFF: the fork's QFN align-mode GDN state fault
-
-**Symptom (Frank, 2026-10-04):** output turned repetitive/corrupted at
-~40k tokens into a long conversation; short sessions stayed clean.
-
-**Evidence (live `podman logs frank-archon-qwen`, the 2026-10-04 18:19
-boot):** one 0.6 s burst at 18:46:07 PDT — 5.6 h into service, one
-running request, and the engine's own 18:46:08 log line reading
-"Prefix cache hit rate: 48.1%". 48 `gdn_hip` faults, then the
-process-lifetime silence cap:
-
-```
-[gdn_hip] ERROR gdn_decode_r: state out of ssm-cache range: max|S|=0
-  with non-finite values (slot=42 head=3 limit=3.40282e+38) fault #1-23
-[gdn_hip] ERROR gdn_decode_r: ... (slot=42 head=8 ...) fault #24-46
-[gdn_hip] ERROR: state fault report limit (32) reached; further faults
-  are counted silently
-```
-
-One physical SSM slot (42), two heads (3 and 8 — one per TP rank), 23
-GDN layers each, the state non-finite (NaN). The guard's own text in
-this build: *"known Qwen3.8-Flash-Next (QFN) flaw under prefix caching
-and is being actively worked; output quality for this request may
-degrade. Mitigation: --no-enable-prefix-caching."*
-
-**Mechanism.** With prefix caching on, the fork defaults QFN to
-`mamba_cache_mode=align` (boot log: "Mamba cache mode is set to 'align'
-for Qwen4ExpForConditionalGeneration … when prefix caching is enabled";
-`config/cache.py`: "none" is the no-prefix-cache mode, "all" is
-deprecated — align is the only sound mode WITH the flag). Align mode
-checkpoints each sequence's GDN (Gated DeltaNet) recurrent state into a
-mamba page per block and RE-SEEDS the state from that page when a
-request resumes from a cached prefix. The block size is raised to
-**3136 tokens** for the page-size equality (boot log: "Setting
-attention block size to 3136 tokens to ensure that attention page size
-is >= mamba page size" + "Padding mamba page size by 0.13% … exactly
-equal"). The corrupt state landed at the **13th boundary: 13 × 3136 =
-40,768 tokens** — the reported "~40k" mark. The NaN persisted in the
-mamba page, so every later turn of that conversation re-seeded the same
-non-finite state for those two heads (silently — the fault-report cap
-was already spent) while fresh short sessions looked fine.
-
-**RAM ruled out the same day** (the 2026-10-04 17:17 host reboot is when
-the current memory speed took effect): zero MCE/EDAC kernel records
-since boot; an 8 GiB in-band pattern check stable over 60 s under live
-load; 4800 MT/s is the JEDEC DDR5 floor on 5600 MT/s-rated DIMMs (not an
-overclock); and a two-head, single-slot NaN state is a state-path fault,
-not scattered finite bit corruption.
-
-**Fix.** `FRANK_ARCHON_NO_PREFIX_CACHING=1` → `--no-enable-prefix-caching`
-(the fork-named mitigation; the flag renders in both directions from the
-knob so a fork default change cannot silently flip the contract). Cost:
-no prefix reuse — every turn re-prefills the whole history instead of at
-most 3135 tail tokens. A silently corrupted conversation is worse than a
-slower first token.
-
-**Verified 2026-10-05 (deploy b820d34):**
-
-- Engine argv carries `--no-enable-prefix-caching`; health 200; warm boot
-  `init engine … took 76.57 s` (the mounted compile cache is intact).
-- Boot plan unchanged, as predicted: expert cache **17.77 GiB / 7,762
-  slots** (the flag touches no plan term), CUDA-graph estimate 0.07 GiB
-  vs 0.08 GiB captured (inside the 12 MiB/graph charge).
-- `GPU KV cache size: 424,678 tokens` — **up** from 401,408 at the same
-  3,166,500,000 B ask: the pool now prices ~7,456 B/token (was 7,891.8),
-  the mamba/side per-token term dropping once the per-block checkpoint
-  pages are gone; the 3136 attention block-size raise nonetheless stays
-  (the page-size invariant is enforced in mode "none" too). Frank's
-  400,700-token floor is cleared with ~24k of headroom.
-- **Zero `gdn_hip` lines** in the boot-and-traffic window (only the two
-  INFO module banners mention the name).
-- Boundary probe (single request, no prefix reuse): prompt 39,280 +
-  completion 1,491 = **40,771 total tokens — across the 40,768 line**;
-  the task (write 1–400, one per line) completed coherently: 399 integers
-  parsed, 0.997 monotone, tail `… 398 399 400`. Pre-fix, this boundary is
-  exactly where the conversation degraded (the 13th 3136-page re-seed).
-- e2e: 32/33 pass; the one failure is the admin spec's
-  192.168.0.112:8989 leg with FrankBook-Pro asleep (environmental,
-  unrelated to this change; re-run `tests/e2e/run.sh` once the mini is up).
-
-**RESULT — REVERTED THE SAME DAY.** With every mechanical check green
-(zero faults, coherent 40,771-token probe), live long conversations
-STILL degraded at ~40k on the running endpoint (Frank, direct
-observation). The container serving that window (`--no-enable-prefix-
-caching`, booted 2026-10-05 12:47 PDT) logged ZERO `gdn_hip` faults
-while the corruption reproduced — so this recurrence is **fault-free**:
-the NaN-state guard did not fire, which the re-seed theory requires.
-Open hypotheses, in order: (1) the poisoned transcript is being re-fed —
-corrupted assistant turns already in the history keep producing
-corrupted continuations even with perfect state, so the same
-conversation cannot distinguish fixed from unfixed; (2) a second
-long-context mechanism with no guard coverage — fp8 KV degradation at
-depth is the next suspect (a bf16-KV boot costs ~2× the pool per token
-and would settle it). Prefix caching is **back ON**
-(`FRANK_ARCHON_NO_PREFIX_CACHING` commented out — the cost bought no
-fix). The knob and the serve branch stay for a future fork build with a
-fixed GDN path. Next diagnostic: reproduce on a FRESH conversation; if
-it still breaks at ~40k with zero faults, A/B the KV dtype.
-
-**RESOLVED (2026-10-05 evening): it was the RAM.** Frank dropped the
-memory from 4800 to **4600 MT/s** (in effect since the 2026-10-05 12:21
-PDT host boot) and the corruption stopped: zero `gdn_hip` ERROR lines in
-the container since, zero MCE records, and the live agent session on
-this endpoint past 140k context, "comfy cozy" (Frank). Marginal DDR5 on
-four DIMMs is a common cause for BOTH eras: the 2026-10-04 NaN burst
-(corrupted weight/activation traffic poisoning a GDN state write — the
-guard-visible path) and the fault-free ~40k degradation (silent flips in
-KV/expert data, which no guard covers on non-ECC memory). The re-seed
-theory is therefore UNPROVEN, not disproven — the entire caching
-experiment ran on marginal RAM and was confounded on both arms.
-Post-mortem on the rule-out: consumer non-ECC RAM raises no MCE for a
-silent flip; a 60 s / 8 GiB pattern sample covers ~0.5 % of 246 GiB;
-"4800 is the JEDEC floor on 5600-rated DIMMs" is a rating, not a
-stability proof at four-DIMM loading. The revert test was the decisive
-instrument and it pointed at the RAM all along. Prefix caching stays ON
-(zero faults under heavy re-send traffic since the speed drop); the knob
-stays for a future fork fix. The fp8-KV A/B is NOT needed.
-
-### Shape F/G (2026-10-03): where the free VRAM actually is, and what forcing it cost
-
-`rocm-smi` showed **3.57 GiB free per R9700** at the deployed shape E (28.28 GiB
-used of 31.86 GiB, after hours of real traffic — so it is the high-water mark,
-not an idle artifact). Growing the expert arena into it is NOT a matter of
-raising `--gpu-memory-utilization`, because the arena is
-
-```
-arena = budget - non-expert - KV reserve - CUDA graphs - runtime overhead
-budget = device_total x gpu_memory_utilization          # expert_plan.py:567
-```
-
-and every term on the right is either fixed or a lie:
-
-| term | plan charges | measured reality | reachable from our side? |
-|---|---|---|---|
-| `budget` | 31.22 GiB (0.98) | field is `le=1` so 31.86 GiB is the field max, but 1.0 is UNBOOTABLE (below) | **yes** — shipped 0.982 |
-| `non-expert weights` | 6.28 GiB | checkpoint census | no |
-| `KV reserve` | 1.91 GiB (registry 9,190 B/token x 200704 +8%) | real pool bills 2.93 GiB at 7,892 B/token | no (registry constant) |
-| `CUDA graphs` | 0.55 GiB = 256 MiB + 52 MiB x 6 | `Graph capturing finished in 3 secs, took 0.31 GiB` | **yes** — `VLLM_CUDAGRAPH_MEM_BASE_MB` / `_PER_GRAPH_MB`; `VLLM_CUDAGRAPH_MEM_ESTIMATE_MB` pins the whole term outright (checked first, pair ignored when set) |
-| `runtime overhead` | 5.25 GiB (`--expert-precise-memory`) | far lower at idle | no (registry constant) |
-
-Shape F took the graph term (0.55 -> 0.32 GiB) and nudged util to 0.982:
-**arena 17.52 GiB / 7,653 slots measured**, +131 slots. Shape G takes the
-graph term further (0.32 -> 0.07 GiB) at the same util.
-
-**Shape G takes the graph term to the measured floor, util to the law.** The
-graph base pool is the actual over-charge: shape F's own boot reported
-`Graph capturing finished in 4 secs, took -0.06 GiB` — a **negative** capture
-delta, so the 256 MiB base was fiction. Charging `0 base + 12 MiB/graph`
-(0.07 GiB for the shipped 6-graph plan, and still per-graph so an 18-graph
-spec-decode shape pays 0.21 GiB) lands the plan at budget 31.29 / graphs 0.07 /
-**arena 17.77 GiB / 7,762 slots measured**. The KV ask went to
-`3,166,500,000` B/rank, which the engine's own 7,891.8 B/token price lands at
-**401,408 tokens measured / 2.00x concurrency** — the first shape holding two
-full-length conversations with real spare. (The ask/landed delta is block
-alignment: 196 x 2048. The price is shape-dependent, 7,701 B/token at 262144.)
-
-**UTIL 1.00 is not a knife-edge on this fork, it is UNBOOTABLE — measured
-2026-10-03.** One deploy took the field maximum to buy the last 0.57 GiB of
-arena; both workers refused init BEFORE loading a single weight, twice:
-`ValueError: Free memory on device cuda:N (31.29/31.86 GiB ...)` — the worker
-demands free > budget and budget == total, so `le=1` on the CLI field is a
-trap: the engine dies in a 2-restart crash loop with zero payload progress.
-So `0.982` is a LAW, not a preference, and the "why 0.982" answer now has two
-proofs: this ValueError, and the historical ceiling deaths (the 2x262144 bf16
-boot at 0.13 GiB free; the 5x262144 boot at "0 bytes free" on a real
-995-token request). Everything beyond ~7,912 slots needs the registry edited
-inside the image (the fork's log line says exactly that: *"Update the
-registry entry if this drifts"*), not a deploy.
-
-**The host region was the last arena-plan term still autofitting per boot —
-now it is a literal (2026-10-04).** With no `--expert-offload-mem` the planner
-binds the host region to `MemAvailable` AT BOOT minus `PLE 47.68 - 2 x 7.0
-process - 5.0 api/engine - 5% margin`: the 2026-10-05 boot read 237.97 GiB and
-offered 159.39 GiB usable, while the same host reads **75.4 GiB available while
-running** — the bound tracks whatever the co-tenants hold at that moment. The
-demand is 105.48 GiB (52.74 GiB/rank x 2, 23,040 swap-target experts), and a
-boot where the bound drops near it silently converts next-hottest experts to
-VRAM-only (the `pinned_extra` warning is the only trace).
-`FRANK_ARCHON_EXPERT_OFFLOAD_MEM=120` pins it: measured demand + 14% headroom,
-on a host with 246.1 GiB total. Semantics verified in the shipped image
-(`offloader/expert_plan.py`): the cap binds as the bound, an exceedance only
-warns, and the plan rewrites the field to the cap it met (+1 MiB/rank).
-
-**MNBT 1024 -> 2048 is free on every axis the plan can see.** The registry
-overhead model is `base + 0.22 GiB x max(0, (nbt - 4096)/1024)` anchored at
-nbt 4096, so any rung up to 4096 charges the identical 5.25 GiB; the fitted 5.25
-was itself fitted at nbt 4096, which means 2048 is covered by an activation
-figure measured at a **larger** batch. SEQS 3 boots at both 1024 and 2048
-(measured 2026-10-02 — the `query_start_loc` `ConstraintViolationError` is a
-SEQS property). The costs are a one-time recompile (the compile **range** is
-part of the cache key, so `FRANK_ARCHON_CACHE_DIR` pays once) and a wider
-prefill chunk, which the 09-29 sweep found decode-interchangeable across
-1024-4096 while 512 doubled TTFT at 4064 ctx (2.46 s vs 1.36 s).
-
-**`--expert-precise-memory` is the arena lever that does not touch the window.**
-It attacks the planner's overhead term (shape F above attacks the graph term —
-the third and last term this repo can reach), and the flag swaps the default
-registry figure for the fitted one. Measured
-2026-10-02 by a boot probe on the shape-D base (production stopped, candidate
-booted AS the service; numbers are the engine's own plan lines):
-
-| plan line | shape D | **shape E** |
-|---|---|---|
-| `runtime overhead (non-torch + buffers + activation)` | 7.60 GiB | **5.25 GiB** (`precise (gfx1201 TP2 nbt4096/seqs16/len262144)`) |
-| `expert cache space` | 14.87 GiB | **17.22 GiB** |
-| slots | 6,496 | **7,522** (+1,026, +16%) |
-| `swap region` | - | 13.7 GiB / 5,986 slots, 133-134 per layer |
-| boot | - | healthy in 420 s, no OOM |
-
-`'expert_precise_memory': True` also appears in `non-default args`, so a silent
-argparse drop is visible in the boot log. The flag lands **in the arena, not the
-pool** - `--kv-cache-memory-bytes` is untouched at 3.15e9/rank (option A), because
-the pool is not an arena lever and cutting it further would only cost
-concurrency: a 2.2e9 pool was probed in the same window and reports 277,686
-tokens / **1.38x**, so it is the conservative fallback if free VRAM ever binds,
-not the shipped value.
-
-**The compile cache is a boot-cost fix, and it was worth 173 s per boot.**
-Measured 2026-10-02: the image ships **no `/root/.cache` at all** (a fresh
-container: `No such file or directory`), and after a boot that directory holds
-`vllm` at **1,154,714,333 B in 4,665 files** plus `comgr` at 229 MB, all inside
-the container's ephemeral layer. Every
-`podman rm -f` - i.e. every deploy, restart and reboot - discards it, and the
-engine re-pays:
-
-```
-Using cache directory: /root/.cache/vllm/torch_compile_cache/b265bd396a/rank_0_0/backbone
-Dynamo bytecode transform time: 4.99 s
-Compiling a graph for compile range (1, 1024) takes 173.19 s
-saved AOT compiled function to /root/.cache/vllm/torch_compile_cache/torch_aot_compile/…
-torch.compile took 181.20 s in total
-init engine (profile, create kv cache, warmup model) took 217.26 s (compilation: 181.20 s)
-```
-
-Mounting a host directory at `/root/.cache` makes every later boot load the AOT
-artifact instead. The mount is deliberately the **parent**, not
-`/root/.cache/vllm`: `VLLM_CACHE_ROOT` defaults to the `vllm` child, and
-`comgr` is the HIP-kernel JIT cache, so one mount persists both — and it hides
-nothing, because the image ships no `/root/.cache`. The cache key
-includes the compile range, so the MNBT choice costs exactly one compilation.
-Entries are version- and config-hashed: a stale entry is a miss, never a wrong
-answer.
-
-**Proven live on the deployed endpoint, 2026-10-02, consecutive boots of this
-exact shape:**
-
-| boot | engine log |
-|---|---|
-| cold (deploy) | `Compiling a graph for compile range (1, 1024) takes 179.24 s` → `torch.compile took 188.26 s in total` |
-| warm (restart) | `Directly load the compiled graph(s) for compile range (1, 1024) from the cache, took 2.143 s` → `torch.compile took 3.97 s in total` → `init engine (profile, create kv cache, warmup model) took 41.13 s (compilation: 3.98 s)` |
-
-The host directory held **1.4 GB in 4,707 files** after the cold boot. Time to
-`/health` is dominated by the 172.8 GiB payload load (the warm boot reached
-health in 490 s), so the win is the ~184 s of inductor/AOT build rather than a
-proportional cut in boot time.
-
-`GPU_MAX_HW_QUEUES=2` caps the HSA hardware queues per GPU. Verified a real knob
-rather than a no-op - the literal is read out of
-`/opt/rocm/core-10.0/lib/libhsa-runtime64.so` - and shipped deliberately
-alongside the 32x64 gather geometry. It is **not** A/B measured.
-
-**The pool, not the slot count, sets how many conversations live at once.**
-Measured live at this shape: `GPU KV cache size: 398,658 tokens, Maximum
-concurrency for 200,704 tokens per request: 1.99x` (3,146,129,408 B reserved).
-The per-token price is **shape-dependent**: the 262144 shape priced at 7,701
-B/token (6,100,000,000 B -> 792,069 tokens), this pool at 7,892 - ~2.5% dearer,
-so a concurrency figure never extrapolates across geometries. The endpoint
-advertises 3 slots: a third LONG request queues rather than being refused, and
-shorter requests use the spare blocks. Verified: three concurrent 67-token
-requests answered in parallel, ~4.1 s each. Two contexts at 200704 are a
-capacity contract, not a throughput one (2-way concurrency summed BELOW parity
-at 4064 ctx).
-
-#### `--max-num-seqs 2` is impossible on this model (measured 2026-10-02)
-
-Four single-variable boot probes against the then-deployed shape, production
-stopped for each:
-
-| MAXLEN | SEQS | MNBT | result |
-|---|---|---|---|
-| 262144 | 2 | 1024 | **Engine core initialization failed** |
-| 262144 | 2 | 2048 | **Engine core initialization failed** |
-| 262144 | 3 | 1024 | healthy - 792,069 tokens, 3.02x, arena 6,248 |
-| 262144 | 3 | 2048 | healthy - the then-deployed incumbent |
-
-The failure is `torch.fx.experimental.symbolic_shapes.ConstraintViolationError:
-Constraints violated (L['query_start_loc'].size()[0]) ... specialized it to be
-a constant (3)`, raised from AOT `build_guards(strict_error=True)` during
-`determine_available_memory -> profile_run -> _dummy_run`. `model_runner.py`
-sizes that input buffer `max_num_reqs + 1` and `_dummy_run` sets
-`num_reqs = min(max_num_batched_tokens, max_num_reqs)`, so the width is
-`SEQS + 1` for ANY `MNBT >= SEQS`: 3 at SEQS 2, 4 at SEQS 3. **MNBT is not a
-lever here** - it only moves the compile RANGE ((1,1024) vs (1,2048)), which
-is why the identical error appears at both rungs. `--enforce-eager` bypasses
-the path at the cost of CUDA graphs. The same failure class killed the
-2026-10-01 bf16 2 x 262144 shape.
-
-#### The arena lever is `max_model_len`, not `--kv-cache-memory-bytes`
-
-The fork's planner sizes expert cache as `budget - non-expert - KV reserve -
-graphs - overhead`, and the KV reserve is registry-priced at **9190 B/token x
-max_model_len** - not from the `--kv-cache-memory-bytes` value:
-
-| MAXLEN | KV reserve | arena |
-|---|---|---|
-| 262144 | 2.48 GiB | 14.30 GiB / **6,248 slots** |
-| 200704 | 1.91 GiB | 14.87 GiB / **6,496 slots** |
-
-D(reserve) 0.57 GiB = D(arena) 0.57 GiB = +248 slots. The 6.1e9 -> 3.15e9
-pool cut (2.75 GiB) bought **zero** arena slots. It buys runtime HEADROOM,
-which is still worth having - the shape it replaced died with `0 bytes free`
-on a real 995-token request. Grow the arena by lowering `max_model_len`, or by
-taking the overhead term with `--expert-precise-memory` (below); the pool is the
-headroom lever only.
-
-#### The other arena lever: `--expert-precise-memory` (overhead 7.60 -> 5.25 GiB)
-
-The flag is now ON. It was refused on 2026-10-01 for a reason that does not
-generalize: the 2 x 262144 bf16 shape was already at the ceiling (weights 6.28 +
-pool 6.80 + graphs 0.55 + arena 16.65 + non-torch ~1.45 = 31.73 of 31.86 GiB,
-NRestarts=3 at ~6.7 min each), so the 2.35 GiB the flag moves into the arena was
-the last stack over the line, not the cause. The plain bf16 2 x 262144 boot also
-did not stay up: 0 bytes free at runtime, and a real client request (995-token
-prompt, `max_tokens=261149`) killed the worker with `Tried to allocate 50.00 MiB
-... 0 bytes free`.
-
-At 200704 x 3 the flag fits with room and is worth **+1,026 slots (+16%)**. What
-made it affordable is the window trim, not a pool cut: the arena is
-DEVICE-resident, the pool is not part of the budget's arithmetic, and the 200704
-window's KV reserve (1.91 GiB) plus a smaller attention footprint is what the
-plan needed. The pool stays at 3.15e9/rank deliberately (option A) - cutting it
-further would cost concurrency (2.2e9 -> 1.38x) and buy no slots.
-
-Boot-probed 2026-10-02 (transient systemd units at the production uid +
-memlock, full-log capture; the rootless bench probe had masked the production
-memory environment). The night searched two concurrency contracts:
-
-| rung | result |
-|---|---|
-| 3 x 262144, bf16, KV 11,047,010,304 | **OOM at graph capture** — `Tried to allocate 76.00 MiB ... 0 bytes free`, 30.43 GiB of 31.86 GiB already allocated by PyTorch |
-| 3 x 245760, bf16 | **OOM identically** |
-| 3 x 237568, bf16 | PASS — KV 718,606 tokens |
-| 2 x 262144, bf16 + `--expert-precise-memory` | **DEPLOYED then crashed** — 0.13 GiB free at boot, NRestarts=3 (the ceiling, not the flag) |
-| 2 x 262144, bf16 (no precise flag) | did not stay up either: 0 bytes free at runtime, worker killed by a real request |
-| 3 x 200704, fp8, pool sized for 2 contexts | **DEPLOYED 2026-10-02 (shape D)**: 398,658 tokens = 1.99x concurrency, arena 14.87 GiB / 6,496 slots, the 2.75 GiB/rank pool cut returned as runtime headroom |
-| **3 x 200704, fp8, + `--expert-precise-memory`** | shape E (2026-10-02): same 1.99x pool, overhead 5.25 GiB, arena 17.22 GiB / 7,522 slots, boot healthy in 420 s |
-| 3 x 200704, fp8, MNBT 2048, graph term 0.32 GiB, util 0.982 | shape F (2026-10-03): arena 17.52 GiB / **7,653 slots** measured, budget 31.29, `Estimated CUDA graph memory: 0.32 GiB`, KV unchanged 398,658 |
-| 3 x 200704, fp8, graph term 0.07 GiB, util **1.00**, KV 3.1665e9 | **DID NOT BOOT (2026-10-03)**: `ValueError: Free memory on device cuda:N (31.29/31.86 GiB)` on both ranks, twice, zero weights loaded — the worker demands free > budget and budget == total. UTIL 1.0 is unbootable, full stop |
-| **3 x 200704, fp8, graph term 0.07 GiB, util 0.982, KV forced to 400,700** | **DEPLOYED 2026-10-03 (shape G)**: arena **17.77 GiB / 7,762 slots** measured (+109 over F, +240 over E), KV **401,408 tokens / 2.00x** concurrency measured, capture 0.31/0.10 GiB vs the 0.07 charge (under-charged, boots and holds), driver used 30.97 GiB -> **0.89 GiB real free per card** held through a live 4,096-token generation (~38-39 tok/s single-request, +13% vs the 33.99 shape-E baseline); rollback is UTIL or KV_BYTES down a notch |
-
-Six full-length contexts cannot fit at any KV dtype: the 6-seq graph/activation
-footprint leaves only 9.18 GiB for the pool, and 6 × 262144 needs 12.47 GB even
-at fp8.
-
-The old "KV_BYTES is PER RANK, 9/16 GiB OOM" law was the MTP4 WORKSPACE
-reservation, not a VRAM wall. Failures are clean and diagnostic: the engine
-reports `torch.OutOfMemoryError: Tried to allocate X GiB. GPU N has ... of
-which Y GiB is free`, and the fork's giant ERROR lines are split by journald —
-read them via `journalctl -o json`, not `-o cat` (and `podman logs`, which is
-only available while the container lives).
-
-Rollback to the 2026-09-29 fp8+MTP4 shape: git tag
-`frank-archon-fp8-mtp-20261001` (env trio + seam re-pinned there).
-
-## Parallelism options on this box (2026-10-01 survey)
-
-Surveyed from the running image's own CLI surface. Verdicts are specific to
-dankserver: 2x R9700 (gfx1201), a ~172.8 GiB expert-offload MoE, and a model
-with only 2 KV heads.
-
-| option | flag | verdict on this box |
-|---|---|---|
-| TP | `--tensor-parallel-size` | **DEPLOYED, =2.** All-reduce dispatch shows `R4D_AR` then `PYNCCL` |
-| DCP | `--decode-context-parallel-size` | **REJECTED at TP=2** — see below |
-| PP | `--pipeline-parallel-size` | supported; untried (stage handoff plus a TP-keyed expert plan) |
-| DP | `--data-parallel-size` (+`-rank`, `-start-rank`) | impossible: every replica holds a full model copy (172.8 GiB vs 31.86 GiB/card) |
-| EP | `--enable-expert-parallel` + `--all2all-backend` (+`--enable-eplb`, `--enable-dbo`, `--enable-elastic-ep`) | supported (`naive`, `allgather_reducescatter`, `deepep_high_throughput`, `deepep_low_latency`, `pplx`, `mori`, `nixl`); would replace the fork's tiered-offload LOCAL gather with a per-layer PCIe all-to-all |
-| multi-node | `--nnodes`, `--node-rank`, `--master-addr/port`, `--distributed-executor-backend` | supported; the actual path to more cards |
-
-### DCP is structurally unavailable here — do not re-test on 2 GPUs
-`--decode-context-parallel-size 2` fails `VllmConfig` validation at boot:
-
-```
-Decode context parallelism for GQA/MQA requires `--tensor-parallel-size` (2) to be
-greater than the model's total number of KV heads (2). Increase
-`--tensor-parallel-size` or set `--decode-context-parallel-size 1`.
-```
-
-DCP shards the KV along the *sequence* dimension after KV heads have been spread
-across the TP ranks, so it requires **TP > num_kv_heads**. This model has 2 KV
-heads (24 Q heads, GQA 12:1), so TP must be >= 4 — four GPUs, not two. Related
-fork facts: DCP *reuses* the TP ranks rather than multiplying them (it is not
-`TP x DCP`), and with PCP enabled the valid DCP sizes are `{1, pcp, tp x pcp}`
-(`vllm/config/parallel.py`). The KV memory a DCP win would have bought is real —
-each rank would hold half the sequence's KV — but it is unreachable without two
-more cards.
-
-### Measuring decode: the harness lives INSIDE the container
-`/home/dank/mtp_measure.py` imports `/bench_iter.py` by absolute path, so it must
-run in the engine's namespace and the files must be copied in first:
-
-```
-podman cp /home/dank/mtp_measure.py frank-archon-qwen:/
-podman cp /home/dank/bench_iter.py  frank-archon-qwen:/
-podman exec frank-archon-qwen python3 /mtp_measure.py
-```
-
-Without the copy the exec dies with `can't open file '/mtp_measure.py': [Errno 2]
-No such file or directory` — measured 2026-10-01, which silently cost a whole
-arm's numbers. A window script must do the copy, not assume the path exists.
-
-### The then-deployed shape, measured again (2026-10-01 A/B window)
-A clean production-environment boot of the config deployed at that time (4 x 262144 fp8, MNBT
-2048 in that window) reported **`GPU KV cache size: 1,079,582 tokens, Maximum
-concurrency for 262,144 tokens per request: 4.12x`** — the same figures as the
-deployed MNBT-1024 shape, i.e. MNBT does not move the pool, and the capacity is
-reproducible across boots.
-
-## See Also
-
-- [[dankserver-frontier-moe-feasibility]] — the sibling feasibility page
-  (which frontier MoE fits this box; the same PCIe-expert-stream wall at
-  V4.1 scale)
-- [[frank-archon-r9v-radiance-qwen38-q8]] — RETIRED 2026-09-30 (the R9V
-  fork era; production has been the tcclaviger fork since)
-- [[x4-pair-experiment]] — the same residency law measured on a narrower
-  link: streaming over PCIe loses, computing in RAM does not care
-- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here

serverAppConfigs-docs/LLM/dankserver-vram-growth.md

@@ -14,6 +14,10 @@ timestamp: 2026-09-03
 
 > **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
 > See [[frank-redo-2026-08-17]] and [[deployment]] for the active configuration.
+> The `frank-zealot` below is the 2026-08 service: llama.cpp on the RTX 3090
+> **while that card sat in Dankserver**. Today's [[frank-zealot]] is a different host
+> (Dankclaw), payload and runtime build, though the same card class - this crash
+> signature is not a live fault.
 
 # Dankserver VRAM and RAM Growth Under Load
 
@@ -116,7 +120,7 @@ Verified 2026-08-14 after deploy: flash attention enabled, warmup clean, decode
 
 ## Mitigation Options
 
-The documentation for the benchmark ([[frank-zealot-benchmark]])
+The documentation for the benchmark ([[frank-zealot]])
 prescribes the first option for this crash class. None of these change
 the model.
 
@@ -135,6 +139,6 @@ PR #25649 (slot checkpoints freed on prompt clear, merged), issue #26119
 
 ## See Also
 
-- [[frank-zealot-benchmark]]
+- [[frank-zealot]]
 - [[Dankserver System Specs]]
 - [[deployment]]

serverAppConfigs-docs/LLM/dflash2-large-vulkan.md

@@ -1,174 +0,0 @@
----
-type: reference
-title: DFlash2 on Dankclaw (frank-dragoon)
-description: ARCHIVED - unsloth Qwen3.8-27B-UD-Q8_K_XL served by upstream llama.cpp PR #27342 with a z-lab DFlash2 drafter on the Vulkan backend. Retired 2026-08-21 for speed; kept as a runbook.
-tags:
-  - llama.cpp
-  - vulkan
-  - dankclaw
-  - gfx1151
-  - dflash
-  - dflash2
-  - speculative-decoding
-timestamp: 2026-08-21
----
-
-# DFlash2 on Dankclaw (frank-dragoon) — ARCHIVED
-
-> **Archived 2026-08-21.** This DFlash2 Vulkan deployment was retired the
-> same day it shipped because it was slower than the CIRU runtime it
-> replaced. `frank-dragoon` is back on CIRU. This page stays as the
-> runbook and provenance for the DFlash2 attempt.
-
-`frank-dragoon` (port 52625) briefly served
-`unsloth/Qwen3.8-27B-UD-Q8_K_XL` with a DFlash2 drafter from z-lab. The
-runtime was upstream llama.cpp PR #27342 built with `GGML_VULKAN=ON`.
-
-## Why Vulkan
-
-DFlash2 selects its draft candidates with a lightweight path selector
-built on the `TOP_K` and `ARGMAX` graph ops. The HIP backend does not
-implement `TOP_K`. That is why the retired CIRU setup routed the MTP
-draft around it with `LLAMA_MTP_CPU_ARGMAX_FASTPATH`.
-
-The PR reference fork (Nathanw1014/strix-halo-llamacpp) validated DFlash2
-on this exact gfx1151 through Vulkan (RADV). Its `test-backend-ops` passed
-the new graph ops:
-
-| Op | Cases |
-|----|-------|
-| TOP_K | 445 |
-| CONCAT | 193 |
-| PAD | 28 |
-| REPEAT | 18 |
-| FILL | 4 |
-
-The whole stack (target + drafter) runs on `Vulkan0`.
-
-## What Runs
-
-| Artifact | Repo / File | Size |
-|----------|-------------|------|
-| Target | `unsloth/Qwen3.8-27B-GGUF` `Qwen3.8-27B-UD-Q8_K_XL.gguf` | 31.4 GB |
-| Drafter | `z-lab/Qwen3.8-27B-DFlash2-GGUF` `Qwen3.8-27B-DFlash2-Q4_K_M.gguf` | 2.0 GB |
-| Runtime | upstream llama.cpp PR #27342, GGML_VULKAN=ON | - |
-
-The target was already on disk. The drafter was added for this deploy.
-Both live in `/home/dank/models/`.
-
-## Drafter Hyperparameters
-
-Read from the drafter GGUF metadata:
-
-| Parameter | Value |
-|-----------|-------|
-| `dflash.block_count` | 5 |
-| `dflash.block_size` | 8 |
-| `dflash.conv_kernel_size` | 2 |
-| `dflash.conv_group_size` | 16 |
-| `dflash.selector_rank` | 256 |
-| `dflash.selector_top_k` | 16 |
-| `dflash.context_length` | 262144 |
-| `dflash.embedding_length` | 5120 |
-| architecture | `dflash` |
-
-The target and drafter share the Qwen3.8 architecture
-(`qwen35`, dense 27B, vocab 248320), so the drafter inherits the target
-tokenizer and LM head.
-
-## Build Runbook (2026-08-21)
-
-Performed on Dankclaw.
-
-1. In `/home/dank/llama.cpp`, register the `ggml` remote and fetch the PR
-   head:
-   ```
-   git -C /home/dank/llama.cpp remote add ggml https://github.com/ggml-org/llama.cpp.git
-   git -C /home/dank/llama.cpp fetch ggml pull/27342/head:pr-27342
-   ```
-2. Add a detached worktree that does not disturb the running build:
-   ```
-   git -C /home/dank/llama.cpp worktree add --detach /home/dank/llama.cpp-dflash2 pr-27342
-   ```
-3. Configure with Vulkan only (glslang and `glslc` must be installed):
-   ```
-   cmake -B /home/dank/llama.cpp-dflash2/build-dflash2 -G Ninja \
-     -DGGML_VULKAN=ON -DGGML_HIP=OFF -DGGML_CUDA=OFF -DGGML_METAL=OFF \
-     -DGGML_NATIVE=ON -DCMAKE_BUILD_TYPE=Release \
-     -DLLAMA_BUILD_SERVER=ON -DLLAMA_BUILD_WEBUI=OFF
-   ```
-4. Build the server:
-   ```
-   cmake --build /home/dank/llama.cpp-dflash2/build-dflash2 --target llama-server -j16
-   ```
-5. Verify the Vulkan device:
-   ```
-   /home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server --list-devices
-   ```
-   Output: `Vulkan0: AMD Radeon Graphics (RADV STRIX_HALO)`.
-
-```mermaid
-graph TD
-    A[Fetch PR #27342 head] --> B[Worktree on DFlash2 head]
-    B --> C[Configure Vulkan build]
-    C --> D[Build llama-server]
-    D --> E[Verify Vulkan0 device]
-    E --> F[Add DFlash2 drafter]
-    F --> G[Swap frank-dragoon]
-    G --> H[Health check + completion]
-```
-
-## Service
-
-- Unit: `frank-dragoon.service`, port 52625
-- Script: `scripts/serve-frank-dragoon.sh`, env `scripts/frank-dragoon.env`
-- Binary: `/home/dank/llama.cpp-dflash2/build-dflash2/bin/llama-server`
-
-### Launch profile
-
-- 262144 context (native max), 1 slot, f16 KV (~16 GiB), 16 context
-  checkpoints, 8 GiB prompt cache
-- DFlash2 draft, block size 8, device Vulkan0, drafter fully offloaded
-- `--spec-type draft-dflash --spec-draft-n-max 8`
-  (clamped to block size 8 - 1 = 7 at runtime)
-- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`)
-- `--no-mmap`, flash attention `-fa on`, `--metrics`
-
-### Required environment
-
-No PromptForge variables are needed. The `TEMP`, `HOST`, and
-`GPU_MAX_HEAP_SIZE` naming rule still applies for portability, even though
-this build uses Vulkan, not HIP.
-
-## Validation
-
-The smoke test on 2026-08-21 used a scratch port (52626) and confirmed:
-
-- Both models load and the server reports healthy.
-- A chat completion returns reasoning and content.
-- The DFlash2 drafter engaged:
-  `draft-dflash: #gen drafts = 11, #acc drafts = 10, #mean acc len = 5.27`.
-- The per-position acceptance curve decays along the block
-  `(0.909, 0.818, 0.727, 0.545, 0.545, 0.364, 0.364)`, as expected.
-
-The published mean acceptance for the Q4_K_M drafter is ~5.39, so 5.27
-is close.
-
-## Operational Notes
-
-- Health: `curl http://127.0.0.1:52625/health`. Returns 503 while loading.
-- Model load takes about a minute with a warm OS page cache (31 GB target)
-  and longer on a cold cache. Do not kill the process during warmup.
-- `deploy.sh dankclaw` waits 20 minutes for health.
-- The Vulkan device reports 113942 MiB. rocm-smi reports the 64 GiB
-  unified-memory budget. Trust the runtime for the addressable SVM pool.
-- The MTP head of the target GGUF is ignored (`--spec-type draft-dflash`
-  replaces it). The load log warns about unused MTP tensors; this is normal.
-
-## See Also
-
-- [[ciru-activefpx-promptforge]] — the active CIRU runtime on Dankclaw
-- [[DFlash DSpark Speculative Decoding]] — DFlash/DSpark design and history
-- [[deployment]] — the deployment and update process
-- [[qwen3.8-strix-optimizations]] — llama.cpp Optimizations on this iGPU
-- [[index]] — knowledge base index

serverAppConfigs-docs/LLM/eemin-rocmi4-mtp.md

@@ -1,188 +0,0 @@
----
-type: reference
-title: caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw
-description: caf-eemin/Qwen3.8-27B-ROCmI4-MTP-GGUF (Q4_0_ROCMI4, embedded NextN/MTP head) served by the charlie12345/ROCmFPX llama.cpp runtime (ROCmI4 W4A4) on the Strix Halo iGPU. frank-dragoon.
-tags:
-  - llama.cpp
-  - rocm
-  - dankclaw
-  - gfx1151
-  - rocmi4
-  - mtp
-  - iu4
-timestamp: 2026-08-25
----
-
-# caf-eemin Qwen3.8-27B ROCmI4 embedded MTP on Dankclaw
-
-> **RETIRED 2026-08-25.** `frank-dragoon` served this model from 2026-08-18
-> until 2026-08-25. It replaced the kingjones777 ROCmFP4 STRIX MTP payload
-> on the same port and the same Strix Halo iGPU, running the new
-> charlie12345/ROCmFPX ROCmI4 W4A4 build (see [[kingjones-strix-mtp]] for
-> the page it replaced), and was itself replaced by
-> [[kingjones-strix-rocmfp4]].
-
-`frank-dragoon` (port 7770) serves
-`cafonz/Qwen3.8-27B-ROCmI4-MTP-GGUF` (published under the public
-`eemin` namespace) from the **charlie12345/ROCmFPX** llama.cpp fork built with
-the opt-in `ROCmI4 W4A4` path. The model is `Q4_0_ROCMI4`: signed four-bit
-weights packed two per byte, quantized from the BF16 Qwen3.8-27B and carrying
-its **embedded NextN/MTP head** in the main GGUF — so there is no separate
-draft file. The native gfx1151 `v_wmma_i32_16x16x16_iu4` matrix instruction
-batch-verifies the MTP proposals for higher decode throughput.
-
-## What This Is
-
-**ROCmI4** is the GGUF weight format (signed 4-bit codes). **IU4** is the AMD
-WMMA instruction spelling that consumes them. **W4A4** means four-bit weights
-and four-bit activations during the accelerated matrix multiply. Ordinary
-one-token decode stays memory-bandwidth-bound (MMVQ); batched prompt processing
-and batched MTP target verification use the IU4 MMQ path. W4A4 is a lossy
-activation-quant speed path — treat it as a speed/quality tradeoff.
-
-The model is a conversion/quantization of `Qwen/Qwen3.8-27B` (Unsloth lineage,
-Apache-2.0), not a retrained model. It reports architecture `qwen35`
-(M-RoPE), ~27.32B params, native 262144 context, 866 tensors, ftype 108.
-
-## Model Files
-
-| File | Size | SHA-256 |
-|------|------|---------|
-| `Qwen3.8-27B-Q4_0_ROCMI4.gguf` | 13.50 GiB (14,534,384,384 B) | `ad51ecbee55930ababc629c377943a7bbe50f8aafcf32340c7cdae6fa6b7bb3c` |
-| `mmproj-BF16.gguf` (vision, optional) | 0.87 GiB (931,146,432 B) | `83ee4f4f205fa514161778c41df1ea14144faa0f713510893b63c2395f5c2d53` |
-
-Store under `/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/`. The vision
-projector is optional for text-only use.
-
-## Runtime Build
-
-The stock llama.cpp build does not understand `Q4_0_ROCMI4` or the qualified
-IU4/MTP path. Use charlie12345/ROCmFPX at the pinned commit, built HIP-only
-(so the combined HIP/Vulkan scheduler cannot route the ROCmI4 tensors to
-Vulkan, which has no shader for this type) with the W4A4 option on.
-
-| Artifact | Path |
-|----------|------|
-| Fork + pinned commit | `/home/dank/rocmi4-qwen38/ROCmFPX` @ `c49ebdbd5c9f01ec242369f9e7f7967855f80cba` |
-| Build | `cmake -S . -B build-rocmi4-w4a4 -G Ninja -DGGML_HIP=ON -DGGML_VULKAN=OFF -DCMAKE_HIP_ARCHITECTURES=gfx1151 -DGGML_HIP_ROCMI4_W4A4=ON -DLLAMA_BUILD_SERVER=ON -DLLAMA_BUILD_WEBUI=OFF` |
-| Binary | `/home/dank/rocmi4-qwen38/ROCmFPX/build-rocmi4-w4a4/bin/llama-server` (version 257, `c49ebdb`) |
-| ROCm | system `/opt/rocm-7.2.4`, amdclang 22 |
-
-Build a separate directory so the exact-int8 path (the default branch) stays
-available. Rollback: disable `-DGGML_HIP_ROCMI4_W4A4=ON` and rebuild — no model
-conversion or data change needed. The build prints
-`ROCmI4 W4A4: enabled for device 0` when the accelerated path is active.
-
-## Model Download
-
-Downloaded from the public
-`eemin/Qwen3.8-27B-ROCmI4-MTP-GGUF` repo (the card's download section still
-names the still-private `cafonz/…`; use `eemin`). Verify against the repo's
-`checksums.sha256` after download:
-
-```
-cd /home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP
-sha256sum -c checksums.sha256
-```
-
-## Launch Profile
-
-The qualified MTP profile (from the model card and PR #94, verified on the
-live service): strict Qwen MTP, backend sampling, `n_max 4`, `p_min 0.60`.
-
-```mermaid
-graph TD
-    A[llama-server charlie12345 ROCmFPX c49ebdb<br/>ROCmI4 W4A4 enabled] --> B[load Q4_0_ROCMI4 GGUF + mmproj<br/>-ngl 999 -dev ROCm0]
-    B --> C[draft-mtp on ROCm0<br/>--spec-draft-ngl all]
-    C --> D[strict Qwen MTP<br/>n_max 4 / p_min 0.60<br/>backend sampling]
-    D --> E[Sharp chat template<br/>reasoning deepseek]
-```
-
-Key flags (see `scripts/serve-frank-dragoon.sh`):
-
-```
---spec-type draft-mtp --spec-mtp-strict-qwen
---spec-draft-device ROCm0 --spec-draft-ngl all
---spec-draft-type-k f16 --spec-draft-type-v f16
---spec-draft-n-max 4 --spec-draft-n-min 0
---spec-draft-p-min 0.60 --spec-draft-backend-sampling
-```
-
-Because the MTP head is **embedded**, no `--model-draft` is passed and the
-draft uses all layers on ROCm0. `--spec-mtp-strict-qwen` preserves the target
-path's greedy decisions; it does not remove the W4A4 activation approximation.
-
-### Serving settings
-
-- 65536 context (192K below the native 262144 maximum), 1 slot, f16 KV,
-  context checkpoints, prompt cache. The lower limit leaves headroom for the
-  model plus KV cache in the host's unified-memory budget. Concurrent requests
-  queue; the server never preempts a running generation.
-- `-t 16 -tb 32`, batch 512 / ubatch 256 (the qualified W4A4 profile).
-- `--temp 0.7 --top-p 0.95`, reasoning on (`deepseek` puts `<think>` into
-  `message.reasoning_content`).
-- Qwen-Sharp chat template (`--chat-template-file`), `enable_thinking` +
-  `preserve_thinking` on, preserving the peg-native + deepseek shape the Pi
-  agent expects.
-- Strix Halo unified memory + native gfx1151 detection: `HSA_OVERRIDE_GFX_VERSION=11.5.1`,
-  `GGML_HIP_ENABLE_UNIFIED_MEMORY=1`.
-
-### Service
-
-- Unit: `frank-dragoon.service`, port 7770
-- Script: `scripts/serve-frank-dragoon.sh`, env `scripts/frank-dragoon.env`
-
-## Tuning MTP
-
-MTP gains are content-dependent: predictable code, structured data, and lists
-accept longer drafts than creative prose. The active conservative profile is
-`n4 / p0.60`; the previously qualified starting point was `n16 / p0.60`.
-
-## ubatch Benchmark
-
-Decode throughput depends on the ubatch (per-eval token batch). Run the sweep
-on 8k contexts, 512 → 4096 in steps of 512, with a fixed batch:
-
-```
-sudo systemctl stop frank-dragoon
-./scripts/bench-ubatch.sh
-sudo systemctl start frank-dragoon
-```
-
-The script stops `frank-dragoon` for a clean GPU, boots one server
-instance per ubatch on a scratch port, reads authoritative `print_timing` tok/s,
-and restarts the service. See [[frank-dragoon-benchmark]].
-
-## Known Issues
-
-1. **gfx1151 only** for the W4A4 path. Other GPUs fall back to exact int8 MMQ.
-2. **Memory pressure:** the model is 13.5 GiB. Under `--no-mmap` with full
-   offload it needs the whole tensor set at once; leave room in the 64 GiB
-   unified pool (rocm-smi figure). Do not run a bench with the live service up
-   or it OOMs at load.
-3. **`Q4_0_ROCMI4` + MTP flags need the ROCmFPX fork**; stock llama.cpp may
-   reject the file or drop the optimized path.
-4. **W4A4 is lossy** (+~5.4% perplexity vs exact int8). Opt-in speed path.
-
-## Measured Results (qualification)
-
-From the model card, gfx1151, pinned build, W4A4 + strict MTP-16:
-
-| Result | Value |
-|--------|-------|
-| HumanEval pilot, W4A4 IU4 (mean) | 49.40 tok/s |
-| Full HumanEval, W4A4 (mean / median) | 44.39 / 45.23 tok/s |
-| HumanEval / HumanEval+ pass@1 | 94.5% / 91.5% |
-| Non-speculative tg128 | ~13.8 tok/s |
-
-Local smoke test 2026-08-25 on the iGPU: model loaded, strict-MTP active,
-draft acceptance 0.74 (mean 7.67), generation ~25 tok/s at a small output.
-The ubatch sweep above gives the production ubatch choice.
-
-## See Also
-
-- [[deployment]] — deploy and update process
-- [[kingjones-strix-mtp]] — the retired ROCmFP4 predecessor on this slot
-- [[ciru-activefpx-promptforge]] — the retired v3 CIRU PromptForge page
-- [[Dankclaw System Specs]] — the box
-- [[frank-dragoon-benchmark]] — backend/spec benchmark history

serverAppConfigs-docs/LLM/frank-archon-freetoken-pr587.md

@@ -1,184 +0,0 @@
-# frank-archon — FreeToken PR #587 ROCm (Qwen3.8-Flash-Next-Uncensored-FP8, 1× R9700)
-
-**Status: RETIRED 2026-10-05 → (superseded the same day by the radiance
-cutover — see [[frank-archon-radiance-qwen38-native-fp8]], itself retired
-2026-10-08; the payload live today (2026-10-18) is the publisher iq4r container,
-[[frank-archon-radiance-qwen38-iq4r]]). Kept as the rollback runbook: the image
-is still in dank's rootless storage, the orcarouter payload is not on disk any
-more)** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 ·
-**GPU:** 1× Radeon AI PRO R9700 (gfx1201) — the second R9700 sits idle
-
-The tcclaviger RDNA4 vLLM fork was replaced by **FreeToken at PR #587 head
-`77a7201f98f00d93f618c6cf8b81ffe8360e0899`** (branch `rocm714_torch214`), built
-on the host into the local image `localhost/frank-primary-freetoken:pr587-77a7201`
-(that is the tag podman reports today — the 2026-10-10 rename never retagged it)
-by `scripts/docker/frank-primary-freetoken.Dockerfile`, which survives only in
-the rollback tag: `scripts/docker/` is not in the tree today. Same port, same
-served alias (the wire id was `frank-primary` until the 2026-10-09 rename), same
-131-shard payload (`/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8`, since
-deleted from the host). Rollback:
-git tag `frank-archon-vllm-shapeG-20261003` (the vLLM trio + its env lore),
-runbook [[dankserver-qwen38-fp8-expert-offload-tuning]].
-
-## Why FreeToken
-
-The vLLM fork planned a per-boot VRAM/host tiering from an estimator: util
-0.982 was a hard law (1.00 was **unbootable** — worker init raises
-`ValueError: Free memory on device cuda:N (31.29/31.86 GiB)` when budget ==
-total), the expert arena capped at 17.77 GiB / 7,762 slots, and live traffic
-still degraded ~40k tokens in (the GDN state-fault class, never fixed in the
-fork). FreeToken keeps **all expert weights in pinned host RAM** with an LRU
-GPU cache, sizes **KV as an explicit token pool**, and boots in one
-deterministic pass — no per-boot estimator to fight.
-
-## The build (on dankserver, rootless podman as dank)
-
-```bash
-git checkout frank-archon-freetoken-pr587-20261005   # scripts/docker/ exists only in the tag
-cd /home/dank/serverAppConfigs
-podman build -t frank-primary-freetoken:pr587-77a7201 \
-  -f scripts/docker/frank-primary-freetoken.Dockerfile scripts/docker/
-```
-
-(The names above are the ones the tag actually carries —
-`scripts/docker/frank-primary-freetoken.Dockerfile`, per
-`git ls-tree frank-archon-freetoken-pr587-20261005`. The 2026-10-10 rename
-swept the docs, not the tag or the image in storage.)
-
-Laws baked into the Dockerfile (each one cost a failure):
-
-- **Base digest-pinned**: `rocm/pytorch:rocm7.14_ubuntu24.04_py3.12_pytorch_release_2.11.0@sha256:a223aee1…` —
-  the PR doc's known-good stack (torch 2.11.0+rocm7.14, triton 3.7). No
-  rocm7.14+torch2.14 image exists on Docker Hub (the 2.14 images are
-  rocm10.1.0 — an untested pair for this PR).
-- **`--no-build-isolation --no-deps -e .`** — a plain
-  `pip install -e .` on ROCm can REPLACE the image's ROCm torch with a CUDA
-  build (the dependency set is authored for NVIDIA). Runtime deps are
-  curated explicitly; `flashlib==0.3.0 --no-deps` (its numba /
-  nvidia-cutlass-dsl deps are CUDA-only); ninja for the QSA JIT;
-  **flashinfer / sgl_kernel intentionally absent** (no ROCm builds;
-  FreeToken falls back to its Triton kernels — the PR's 12 pytest failures
-  are exactly those backends).
-- **`libhiprtc.so` dev symlink**: the pip-ROCm torch ships only the
-  versioned `libhiprtc.so.7` in
-  `/opt/venv/.../site-packages/_rocm_sdk_core/lib`, but the PR's `_cpu_moe`
-  extension links `-lhiprtc`. First build died `ld: cannot find -lhiprtc`;
-  the Dockerfile symlinks the unversioned name into the ALREADY-LINKED dir.
-- FreeToken source is checked out at the **pinned PR head**, not the branch
-  (a future push to `rocm714_torch214` cannot change the deployed engine).
-- The stale Docker Hub login (`docker login` creds at
-  `~/.docker/config.json`) poisons podman's ANONYMOUS pulls of public images
-  with "invalid username/password" — `docker logout` fixes the pull. Hit
-  2026-10-05 while pulling the base image.
-
-## Measured at cutover (2026-10-05/06)
-
-
-- **Boot**: ~2.3 min container-start → API-ready (131 shards page-in + PLE
-  pin + expert staging + CUDA graphs `[1,2]`). VRAM after graphs:
-  **1.80 GiB free** of 31.86. RAM: 122 GB used + **113 GB shared (the
-  pinned expert/PLE slabs)**, 123 GB still available.
-- **Decode: 17.1 tok/s** single-stream, temp 0, 400-token probe (clean
-  1→N counting). The vLLM incumbent measured ~30 tok/s on TWO cards;
-  FreeToken single-card is the deployed baseline.
-- **The single-card LRU ceiling**: the KV contract (harness worst case
-  177,356 tokens) + dense fp8 + the vision tower + graphs leave ~12 GiB for
-  the expert LRU at rate 0.10 — and only 1.8 GiB of live headroom, so
-  `ft ctl cache --moe` cannot raise it far on one card. The paths to a
-  better decode rate are TP=2 (below) or a smaller pool (contract
-  violation — the 177,356 floor is real traffic, not padding).
-- **TP=2 exists but does not boot on ROCm** (probed 2026-10-06,
-  zero-impact window): `--tensor-parallel-size 2 --gpu 0,1` dies in the TP
-  allreduce JIT — `pynccl.cu` compiles per-rank at boot and clang can't
-  resolve `cuda_runtime.h` from the hipified include path
-  (`1 error generated when compiling for gfx1201`, ninja stopped). The PR
-  author validated single-GPU only; this is FreeToken's untested-TP-on-ROCm
-  edge, not a board or driver fault (radiance TP=2 allreduces fine on this
-  pair since the 2026-09-20 swap). A future fix = patch the JIT include
-  path (HIP's cuda_runtime shim) or vendor an ROCm allreduce — re-probe
-  then.
-- **Verification gates passed**: /health ok; plain completion `FLEET-OK`
-  (finish stop); thinking split — `reasoning_content` non-empty AND
-  `content` carries the answer (17×24 → "408"); native `tool_calls` with
-  correct name+arguments (finish tool_calls); 256×256 solid-red PNG through
-  an `image_url` part answers exactly "Red" (prompt_tokens 127); stats:
-  native ctx 262,144, KV 4,687 pages × 64 = 299,968, modalities
-  [text,image], sampling defaults from generation_config.json
-  (temp 1.0 / top_p 0.95 / top_k 20).
-- **Thinking counts against max_tokens** (same as the vLLM era): a
-  `max_tokens: 10` probe returns EMPTY content with the budget spent in the
-  think block. Clients must budget for thinking or disable it.
-- **Prometheus scrapes /metrics → 404** (Dankclaw scrapes 192.168.0.235:7770
-  from the llm-fleet job; FreeToken has no /metrics endpoint). The target
-  will read down until the scrape config is repointed or a metrics shim
-  lands — same class as the retired HTML-probe targets
-  (see [[Observability]]).
-
-
-## The shape
-
-| Knob | Value | Why |
-|---|---|---|
-| `--gpu` / `HIP_VISIBLE_DEVICES` | 0 / 0 | ONE R9700; visibility pinned so in-container index 0 is the right card |
-| `--num-tokens` | **299,968** | KV pool in TOKENS; **must be a multiple of 64** (the auto attention backend is `qsa_sparse`, which overrides page size to 64 — 300,000 was refused: `nearest valid values: 299968 or 300032`) |
-| `--max-running-requests` | 2 | first-boot conservative; raise after real-traffic slot-occupancy look |
-| `--max-output-tokens` | 36,864 | harness maxTokens; requests that omit max_tokens get this default |
-| `--moe-cache-rate` | 0.10 | GPU expert LRU as fraction of all experts (~120 GiB fp8 → ~12 GiB VRAM); first-boot conservative |
-| parsers | AUTO | thinking → `reasoning_content` natively; tools in the qwen3.8 dialect; NO `--reasoning-parser`/`--tool-call-parser` flags |
-
-KV math: 12 full-attention layers × 2 KV heads × 256 head_dim × 2 (K+V) ×
-2 B = **24 KiB/token** (f16 KV; the 36 GDN layers use fixed per-seq state,
-not per-token KV) → 299,968 × 24 KiB ≈ 7.0 GiB. Clears the harness worst
-case (140,492-token prompt + 36,864 max_tokens = 177,356) and admits ONE
-native 262,144-token conversation + full budget. `models.yml`
-`contextWindow: 200704` stays (compacts early, never 400s).
-
-## RAM math (why the root unit + memlock)
-
-246 GiB total. FreeToken pins ~120 GiB of fp8 expert rows + the ~47.7 GiB
-PLE ngram table in host RAM → the unit carries
-`LimitMEMLOCK=infinity` (root unit; dank's login session caps at 8 MiB and
-a user quadlet cannot raise it). Weights+PLE+overhead ≈ 190 GiB — the vLLM
-container's ~170 GiB MUST be down first; `systemctl stop frank-archon`
-frees it (measured: 207 GB free right after the stop).
-
-## Resize live
-
-```bash
-ft ctl cache --kv <tokens>   # grows the KV pool LIVE (shrink needs restart)
-ft ctl cache --moe <rate>    # same for the expert LRU
-ft ctl stats                 # pool occupancy, VRAM, throughput
-```
-
-## Verification (the fleet's real-completion law)
-
-1. `/health` 200 — necessary, not sufficient.
-2. Plain completion returns the asked-for string.
-3. Thinking request: `reasoning_content` non-empty AND `content` carries the
-   answer (the models.yml `reasoningContentField: reasoning_content` flip
-   rides on this).
-4. Vision probe: a solid-color PNG through an `image_url` part answers the
-   exact color (the legacy top-level `images` field trap does not exist
-   here — FreeToken is OpenAI-native).
-5. Unit-managed: `systemctl show -p MainPID` behind the health answer (the
-   2026-09-26 stray-port false green).
-
-## Rollback
-
-```bash
-git checkout frank-archon-vllm-shapeG-20261003 -- \
-  scripts/frank-archon.env scripts/serve-frank-archon.sh \
-  scripts/frank-archon.service
-./tests/run.sh && git commit -m "frank-archon: rollback to vLLM shape G" \
-  && git push && ./scripts/deploy.sh dankserver --service frank-archon
-```
-
-The tcclaviger image and the FreeToken image coexist in dank's rootless
-storage; the deploy re-pulls the pinned digest if absent.
-
-## Index
-
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — the retired vLLM shape
-  (rollback target; its 362-line env lore lives in the tag)
-- [[freetoken-frank-small]] — the v0.1.2 CUDA precedent (ortobox, RTX 3090)
-- [[LLM/index]] — fleet index

serverAppConfigs-docs/LLM/frank-archon-glm53-flash-w4a16.md

@@ -1,808 +0,0 @@
----
-type: runbook
-title: frank-archon — GLM-5.3-Flash W4A16 int4 (tcclaviger, 2x R9700)
-description: VERIFIED 2026-09-30 by boot probe, bench, live expert-cache telemetry AND a torch-profiler kernel-time window. frank-archon cut over from Qwen3.8-Flash-Next-Uncensored-FP8 + MTP4 to Intel/GLM-5.3-Flash-W4A16-AutoRound, the first payload this fleet serves through the fork's W4A16 expert-offload path (INC scheme -> R4dW4a16MoEMethod, libr4d grouped W4A16 GEMM). Measured decode 9.9-10.0 tok/s and prefill 918 tok/s at 131,072 ctx. THE 35 tok/s TARGET IS NOT MET, and the profiler window FALSIFIED the first explanation: the expert gather is 1.05 % of device time, not the bound. 44 % of the 93 ms/token budget is pointwise glue (1,428 elementwise launches per token) because the fork's GLM trunk is NOT torch.compiled while its own Qwen trunk is; the second-largest lever, the fused router, is binary-blocked at 288 experts. The compile changelist is staged but incomplete (three patches, one blocker left).
-tags:
-  - dankserver
-  - r9700
-  - gfx1201
-  - glm5.3
-  - w4a16
-  - int4
-  - expert-offload
-  - expert-cache
-  - autoround
-  - vllm
-timestamp: 2026-09-30
----
-
-# frank-archon — GLM-5.3-Flash W4A16 int4 (tcclaviger, 2x R9700)
-
-> **RETIRED 2026-10-01** (era 2026-09-30 → 10-01, matching the AGENTS.md Runbook
-> Index). Cutover reversed to Qwen3.8-Flash-Next-Uncensored-FP8 + MTP4 — the
-> trio as of commit `2339fae`, restored by `d91379c` (2026-09-30). frank-archon
-> runs [[frank-archon-radiance-qwen38-iq4r]] (radiance) today, 2026-10-18. The
-> DFlash2 drafter experiment is a **failure**: after five patches the drafter
-> served and drafted (12.8 tok/s proposed, 65,536 ctx, both TP ranks capturing
-> FULL dflash2 cuda graphs) but **acceptance pinned at 1.00-1.38 on every text
-> class including verbatim repetition** — the signature of a mis-wired
-> aux-hidden-state capture, not a weak drafter. At ~1.2 acceptance the 8-token
-> verification tax makes decode 5x SLOWER than the 9.9-10.0 no-draft baseline
-> (1.6-1.8 tok/s measured). The untested alternative capture semantics: post-layer
-> output state (DeepSeek's `hidden_states + residual`) or the mHC `comb` stream.
-> Payload (170 GiB) + drafter (2.2 GiB) were staged to
-> `/home/dank/.AGENTTRASH/20261001-glm-purge/` and purged since — on 2026-10-18
-> Dankserver has no GLM tree under `/home/dank/models/` and `~/.AGENTTRASH` is
-> empty. KV-cache work that survives: `glm53-dflash-04-kv-groups-draft.patch` and
-> `glm53-dflash-04b-kv-registry-dflash.patch` are still in `scripts/patches/`;
-> patch 03 (tailpin) was deleted at the rollback (`d91379c`) and patch 02 was
-> never committed. None of these apply byte-exact today — the 2026-10-10 rename
-> (`55f87fc`) rewrote the retired service names inside their context lines.
-
-
-> **Question this page answers.** Can the fleet serve the 320B-class
-> `GLM-5.3-Flash` at int4 on two R9700s with expert offload, and how fast is it?
-> **Answer.** Yes — it serves, with vision, tool calls and 131,072 ctx. It runs
-> at **9.9-10.0 tok/s decode / 918 tok/s prefill**, and **35 tok/s is not
-> reachable** on this payload as the fork ships it. The bound is **not** the
-> model, the quant, the link, or the expert gather (measured: 1.05 % of device
-> time). It is the **unfused pointwise glue of an uncompiled trunk** — 44 % of a
-> 93 ms per-token budget spread over ~1,428 elementwise launches per token —
-> because the fork compiles its own Qwen trunk and not its GLM one. See
-> [The measured profile](#the-measured-profile-and-the-real-wall).
-
-## DFlash2 target-side port — the four laws and the KV ladder
-
-Distilled 2026-10-05 from the probe windows (skills `glm53-dflash-boot-ladder`
-/ `glm53-dflash-eagle3-port`); each law was proven by a real failed boot.
-
-**The four EAGLE3-port laws** (make the GLM trunk satisfy `SupportsEagle3`):
-
-1. **Patch the instance the runner calls.** `model_runner.py:425` calls
-   `set_eagle3_aux_hidden_state_layers(self.model)` on the *loaded* model —
-   GLM loads as `Glm5NextForConditionalGeneration` (multimodal wrapper;
-   `self.language_model = Glm5NextForCausalLM`). Members placed only on the
-   CausalLM fail the check on the conditional — same error text, wrong file.
-   Patch BOTH wrappers.
-2. **Presence-checked member set.** `SupportsEagle3` (interfaces.py:1659) is
-   `@runtime_checkable`; `isinstance` walks ALL `__protocol_attrs__`
-   (including base-protocol members) with `getattr_static`, breaking on the
-   first miss. Both wrappers need, as class attributes:
-   `supports_eagle3: ClassVar[Literal[True]] = True`,
-   `has_own_lm_head: bool = False`, `has_own_embed_tokens: bool = False`.
-   Verify in-image with stock typing — no boot needed.
-3. **MRO forbids inheriting the Protocol.** GLM wrappers carry four/five
-   Protocol-derived bases; adding `SupportsEagle3` anywhere dies with
-   "Cannot create a consistent method resolution order" (sim-proven: the
-   ordering rejects it in every base position). The attribute route avoids
-   the MRO, so the two methods must be written by hand: the CausalLM
-   unwraps to `self.model` (the `EagleModelMixin` trunk), the conditional
-   delegates to `self.language_model` (pixtral.py:498 pattern);
-   `get_eagle3_default_aux_hidden_state_layers` returns `(2, L//2, L-3)`,
-   with the drafter config's `target_layer_ids` winning when present.
-4. **Trunk capture point:** append `aux_hidden_states` BEFORE the layer call
-   — GLM's loop carries the single-stream state entering the layer.
-
-**The KV blocker ladder** (patches `scripts/patches/glm53-dflash-0*`):
-
-1. EAGLE3 gate → presence-checked attrs on both wrappers (above).
-2. KV page-size unifier: MLA layers cannot be padded (sparse MLA reads
-   whole token rows) but CAN grow block_size when the max page is an exact
-   multiple — bump `max_page_size` in `unify_kv_cache_spec_page_size` to
-   the `lcm` of all MLA-family pages first (patch 02).
-3. kpool tail assert `tail_kv_cache.shape[2] == pool_size`: the unifier's
-   divisible branch grew KpoolTailSpec's block; its kernel pins block ==
-   index_kpool. Add a KpoolTailSpec branch that pads `page_size_padded`
-   instead of growing (patch 03).
-4. KV capacity: the glm5 layout ALIASES mamba states INTO MLA pages (why
-   GLM fits 32 GiB cards); the generic path (taken when draft/hidden specs
-   exist) cannot alias → ~30 GiB demanded vs a 0.92 GiB reserve. Extend
-   `_get_kv_cache_groups_glm5_next` to partition
-   HiddenStateCacheSpec + drafter attention specs; the drafter forms its
-   own uniform group (per-group pages, NO unification → the MLA block
-   stays at the engine-auto-set 2,432 from interface.py:918, kpool guard
-   `block_size % 512 == 0` unstressed); layout returns a 10-tuple
-   (+draft_group, +hidden_groups); the builder sizes ONE shared backing
-   allocation (`allocate_kv_cache` asserts all KVCacheTensor sizes equal) =
-   aliased pool + appended draft/hidden tensors (patch 04). After 04 the
-   capacity requirement collapsed 31.76 → 1.88 GiB at 65,536 max len and
-   the engine SERVED (READY ~500 s, dflash2 fused prep ON, drafter FULL
-   graphs on both TP ranks).
-5. Reserve registry: `_glm53_kv_bytes_per_token` must add 4,096 (drafter
-   KV) + 5×hidden×2 (aux hidden cache) keyed on
-   `speculative_config.method in {dflash, dspark, extract_hidden_states}`.
-   NEVER key on `use_aux_hidden_state_outputs` — it lives on the
-   gpu_model_runner instance (gpu_model_runner.py:623/678), unset at plan
-   time, so the check silently fails and the reserve stays 13,600
-   (patch 04b). Proven reserve math at 65,536: 13,600 + 4,096 +
-   5×4096×2 ≈ 58,656 B/token → ~4.15 GiB reserve, arena ~10.3 GiB.
-
-**Verify cheaply before booting:** smoke-test the pure KV logic on CPU
-torch with the image's real spec classes — stub `vllm.logger`/`vllm.utils.*`,
-register the spec classes in `KVCacheSpecRegistry` with a dummy manager, add
-`from __future__ import annotations`, and run synthetic GLM+draft+hidden
-spec sets through the patched grouping/layout/builder functions. A 7-minute
-boot per hypothesis is the alternative.
-
-**Window discipline:** stop + mask `frank-archon` FIRST and gate on
-MemAvailable ≥ 230 GiB before booting the probe (the expert-offload planner
-reads MemAvailable; beside-prod boots die at plan time with `ExpertPlanError:
-host holds 5113 of 12096 swappable experts`). Save `podman logs glm-probe`
-to a file BEFORE teardown — a `podman logs | grep | head` pipeline died once
-under pipefail via SIGPIPE before log-save, losing the capture. The abort
-path must restore prod (unmask + daemon-reload + reset-failed + start),
-never bare-exit after stop+mask. Launcher knobs: `DFLASH_PATCH=1` (mounts
-the patched model.py), `DRAFT_MOUNT=1`, `SPECJSON_FILE=…` ({"method":
-"dflash","model":"/draft","num_speculative_tokens":8}) — note the launcher's
-echo prints `patch=$PATCH_MOUNT` (the COMPILE knob), not DFLASH_PATCH.
-Boot timeline: EAGLE3 gate fires ~4 min in at worker init; drafter
-torch.compile is the frontier (~380 s).
-
-**Open telemetry discrepancy:** the contaminated window read 1.5–2.1 tok/s
-with acceptance 1.00–1.25; Frank live-measured ~18 tok/s decode / ~370 tok/s
-prefill. Reconcile with a clean single-request measurement before trusting
-either.
-
-Related: [[dankserver-qwen38-fp8-expert-offload-tuning]] (the same fork, the same
-hardware — its conclusion was drawn from throughput arithmetic and is superseded
-by this page's kernel-time measurement),
-[[dankserver-frontier-moe-feasibility]] (the earlier fit analysis),
-[[VLLM Index]], [[LLM/index|LLM fleet]].
-
-## The deployed shape
-
-| term | value |
-| --- | --- |
-| payload | `Intel/GLM-5.3-Flash-W4A16-AutoRound` — AutoRound **symmetric int4, group 128**, packing `auto_round:auto_gptq`, 34 shards + 2 extra safetensors, 113,074 tensors, ~170 GiB |
-| runtime | `docker.io/tcclaviger/vllm@sha256:da4029666…` (same digest-pinned image as the qwen cutover) |
-| engine | TP=2, `--enable-expert-offload --expert-precise-memory`, max-model-len 131,072 x 2 seqs, bf16 KV, util 0.98, NBATCH 2048, prefix caching |
-| MoE kernel | `R4dW4a16MoEMethod` → libr4d grouped W4A16 GEMM, 6.234 MiB per expert-rank slot |
-| expert cache | 14.51 GiB = **2,383 slots** per rank; 12,096 host-backed experts, **73.64 GiB pinned per rank** |
-| unit | `scripts/frank-archon.service` (root unit, `LimitMEMLOCK=infinity` for the pin) |
-| measured | **decode 9.9-10.0 tok/s**, **prefill 918 tok/s** (warm, 2.6k-token prompt), KV cache 138,561 tokens |
-
-```mermaid
-graph LR
-    A[Intel AutoRound W4A16<br/>sym int4 g128, auto_gptq] -->|quant_method auto-round| B[inc.py override to inc]
-    B --> C[inc_wna16_scheme<br/>_resolve_gptq_moe]
-    C -->|sym int4 g128 + gfx1201 + libr4d| D[R4dW4a16MoEMethod<br/>grouped W4A16 GEMM]
-    D --> E[ExpertLRUCache<br/>2,383 VRAM slots / rank]
-    D --> F[73.64 GiB pinned host rows / rank]
-    E --> G[decode step]
-    F -->|host misses<br/>1.05% of device time| G
-```
-
-## Why this payload and not another
-
-The W4A16 fast path on this fork is reachable **only** through the INC
-(AutoRound) quant method. `compressed_tensors` checkpoints of the same model —
-for example `dealignai/GLM-5.3-Flash-CYBERSECURITY-W4A16`, 182 GiB — resolve
-their MoE through the WNA16 oracle to `TritonWNA16Experts`, which **has no
-offload recipe**, and the engine refuses to serve at all: 170 GiB of experts
-cannot fit 64 GiB of VRAM, so offload is mandatory, not optional.
-
-The chain that makes Intel's checkpoint work, with the exact evidence:
-
-```
-[expert_plan.py]  quantization=inc            ← inc.py:364 overrides "auto-round"
-[inc_wna16_scheme.py:249]  INC GPTQ MoE on ROCm: R4dW4a16MoEExperts (libr4d grouped W4A16 GEMM)
-[r4d_w4a16_moe.py:682]     Using MoEPrepareAndFinalizeNoDPEPModular / R4dW4a16MoEExperts
-[expert.py:450]            Expert offload: 42 MoE layers staged; 2,383 cache slots in 14.51 GiB planned per rank
-[expert.py:413]            42 layers split: 0 VRAM-only experts, 2,383 swap slots, arena 14.51 GiB,
-                           host 73.64 GiB pinned, host rows prepacked on the GPU
-```
-
-`R4dW4a16MoEExperts` is what the offload machinery wraps: the offload plan is
-format-agnostic but requires the experts class to expose the hot/cold split.
-
-## The numbers, and the wall
-
-Measured on the deployed engine, 2026-09-30 (`bench_glm.py`, real prose, greedy):
-
-| depth | prefill | decode |
-| --- | --- | --- |
-| 2,621 tok prompt, run 1 (cold cache) | 259 tok/s | 9.88 |
-| 2,621 tok prompt, runs 2-3 (warm) | 918 / 885 tok/s | 9.91 / 9.41 |
-| 493 tok prompt (telemetry window) | 124 tok/s | 8.90 |
-
-Decode is flat across context and across every arena change — the signature of a
-per-step cost that is not KV- or residency-driven at the margin.
-
-### The measured profile, and the real wall
-
-**This section replaces an earlier explanation, which the measurement falsified.**
-The first reading of this page bounded the step with throughput arithmetic: 42 MoE
-layers x 8 experts x 6.234 MiB = 2.045 GiB per step per rank, 0.82 GiB of it
-missed at a measured 0.53-0.66 hit rate, giving "0.82 GiB in ~100 ms = 8.2 GB/s =
-the gather is the wall". That arithmetic was circular — it *assumed* the gather
-consumed the whole step — and it is wrong. The engine, asked directly, disagrees:
-
-A torch-profiler window (`--profiler-config`, `POST /start_profile` /
-`stop_profile`, 496 decode tokens at the deployed shape, rank0 trace parsed for
-device-activity time) gives the true split. Total device kernel time **46.18 s /
-496 tokens = ~93 ms per token** against a measured ~104 ms step (9.6 tok/s with
-the profiler attached):
-
-| ms | % | launches | kernel |
-| --- | --- | --- | --- |
-| 16,074 | **34.8** | 708,500 | `vectorized_elementwise_kernel` (22.7 us each) |
-| 4,602 | 10.0 | 42,000 | `r4d_gemm_moe_w4a16_nt_b16_m64` — the MoE GEMM |
-| 3,108 | 6.7 | 174,708 | `elementwise_kernel_manual_unroll` |
-| 2,943 | 6.4 | 138,252 | `wvSplitK_hf_sml_` — split-K GEMM (projections / head) |
-| 2,829 | 6.1 | 83,160 | router: `sbtopk::gatherTopK` + `warpMergeSortTopK` + `bitonicSortKVInPlace` (1,208 + 1,051 + 559 ms; the 1,051 row is not in the list below) |
-| 3,015 | 6.5 | 133,920 | `r4d_mhc_pre_finish/pre_partial/post` — hyper-connections |
-| 2,782 | 6.0 | 127,916 | inductor `triton_poi_*` / `triton_per_*` (the fusions that DO happen) |
-| **487** | **1.05** | 20,664 | **`r4d_moe_lru_gather_kernel`** — the accused |
-| 250 | 0.54 | 20,832 | `_moe_lru_tele_kernel` — the telemetry that measured it (TELEM was on) |
-
-So the gather is **1.05 %** of device time, and `r4d_moe_lru_fused_kernel` (the
-LRU *manager*, not the copy) is a further 1,599 ms / 3.5 %. Residency work and the
-PCIe link are not the story: 44 % of the budget is **pointwise glue** — bf16<->fp32
-conversions, adds, `pow`/`rsqrt` from fp32 RMSNorm, `launch_clamp_scale` quant
-passes, `reduce_kernel` means — spread over **~1,428 elementwise launches per
-token** (3,734 kernel launches per token in total). The GPU is busy 86 % of the
-wall clock, so this is real device work, not launch-gap overhead.
-
-Two consequences that kill three whole families of planned work:
-
-- **Spec decode cannot help this shape, and the MTP measurement was not an
-  accident.** Pointwise cost scales with the token count, so a draft batch that
-  verifies 4 tokens pays ~4x the glue for the same per-token output — which is
-  exactly why MTP4 measured "acceptance 3.76-4.90, decode unchanged". Same reason
-  the gather was never the bound.
-- **Arena size and hit rate are not levers.** Confirmed twice: `--expert-precise-memory`
-  (arena 13.08 -> 14.51 GiB) moved decode by nothing, and the gather is 1 %.
-
-## Fork work: the compile changelist (WIP, does not boot yet)
-
-The fork compiles its own flagship trunk — `vllm/models/qwen4_exp/amd/model.py`
-carries `@support_torch_compile` — and **not** its GLM trunk. The engine says so on
-every boot:
-
-```
-Cudagraph mode FULL_AND_PIECEWISE requires piecewise capture, but the loaded model
-provides neither a compiled submodule nor breakable CUDA graphs.
-Overriding to FULL_DECODE_ONLY.
-```
-
-Inductor fusion is therefore the only lever that touches the 44 % pointwise mass.
-Three patches are staged in `scripts/patches/glm53-compile-0*.patch`, each carrying
-its own evidence header, base/patched sha256 and the next blocker:
-
-| patch | target | fixes |
-| --- | --- | --- |
-| `-01-model-decorator` | `models/glm5next/common/model.py` | decorates `Glm5NextModel` (env `R4D_GLM_COMPILE`, default on) |
-| `-02-trace-safe-probes` | `model_executor/kernels/r4d_lib.py` | freezes 10 capability probes that do `os.path.exists`/`.so` lookups **inside the traced forward** |
-| `-03-fullgraph-optout` | `compilation/wrapper.py` | makes `fullgraph=True` opt-out (env `VLLM_COMPILE_FULLGRAPH=0`) |
-
-What the boots established, in order (each one ~7 min):
-
-1. **01 alone dies** on the first cold probe call inside the traced region:
-   `Dynamo does not know how to trace the builtin 'posix.stat'` —
-   `r4d_mhc_supported -> r4d_lib.r4d_mhc_enabled -> available -> os.path.exists`.
-   **`functools.lru_cache` does not protect these probes**: dynamo re-traces the
-   wrapped body to build its guard even when the cache is warm (it warns about
-   exactly this), verified in-image — a *warm* lru_cache-calling function still
-   fails `fullgraph=True`, while a plain module-global read compiles.
-2. **01+02 get past it** (`libr4d capability probes frozen: 10 warm [...]` in all
-   four processes) and die on
-   `RuntimeError: Graph breaks are not supported with aot compile`, because this
-   build runs AOT compile by default and AOT refuses breaks.
-3. **01+02+03 with `VLLM_COMPILE_FULLGRAPH=0 VLLM_USE_AOT_COMPILE=0` traces the
-   trunk successfully** (`Dynamo bytecode transform time: 1.08 s`) and then dies on
-   the decode forward with `KeyError: 'in_proj_qkvbfg_a'` (`Glm5NextModel.forward`
-   line 748 calling the decoder layer at line 456). `in_proj_qkvbfg_a` is the KDA
-   layer's merged column-parallel projection (`kda.py:217`, called at :407) — a real
-   submodule attribute name, also present in the `load_weights` shard table
-   (`model.py:780-785`). Reading the full traceback (the reported line is inside
-   dynamo-rewritten code) traces the throw to dynamo's own name→module map:
-   `torch/_dynamo/output_graph.py` registers into `nn_modules` at :1259/:1384,
-   looks up with `self.nn_modules[node.target]` at :3000 (a bare-name `KeyError`
-   when absent), and documents stripping `_buffers[..]/_parameters[..]/_modules[..]`
-   from those names at :1225. So the compiler is failing to **resolve a submodule by
-   name** while tracing the GLM decoder layer; whether the root cause is the fork's
-   module tree or dynamo's registration is not yet settled. The next step is the one
-   torch itself suggests in that error — `TORCHDYNAMO_VERBOSE=1` for the internal
-   stack trace (and note the launcher resets `ENVARGS`, so it must be added inside
-   `serve_glm_autoround.sh`'s COMPILE_PATCH branch rather than in the outer
-   environment).
-
-4. **04 removes the fused-norm call from the traced region** (`R4D_GLM_FUSE_QKV_RMSNORM=0`:
-   the GLM MLA block stops calling the fork's fused-norm JIT kernel, whose kernel
-   object in `models/common/ops/fused_qk_rmsnorm.py` is bound *after* the function
-   that uses it — `_FUSED_Q_KV_RMSNORM_KERNEL` at line 223 vs its use at line 211).
-   **It removes a real break** (under `fullgraph=True` the fused call was the first
-   break dynamo reported; after this patch the report moves on to the workspace
-   acquisition) — but it is not by itself what blocks a boot, because with breaks
-   genuinely tolerated the blocker is the name-map `KeyError`. See the run history
-   below.
-5. **A launcher bug that made four boots misleading — worth knowing because the
-   failure kept moving.** The scratch launcher's compile branches were restructured
-   on 2026-09-30 and the `-e VLLM_COMPILE_FULLGRAPH=0` line was dropped, so four
-   consecutive boots mounted the wrapper patch while `_FULLGRAPH` still defaulted to
-   True. With `fullgraph=True` **every** break is an error, so those boots reported
-   whichever break came first:
-
-   | config | outcome |
-   | --- | --- |
-   | `fullgraph=False`, AOT off | breaks tolerated → dies later on the dynamo name-map `KeyError: 'in_proj_qkvbfg_a'` |
-   | `fullgraph=True`, AOT off | dies on the first break (cell → workspace → compiler-disable) |
-
-   So the reported breaks are real *as breaks*, but they are not what blocks a
-   break-tolerant compile — and an earlier version of this page blamed
-   `TORCH_LOGS=graph_breaks` for the difference. **That was wrong**; the flag only
-   logs. The launcher sets the env var again.
-
-**How to see the breaks (and its trap).** `TORCH_LOGS=graph_breaks` names each break
-with a user stack, and `VLLM_USE_AOT_COMPILE=0` is required because AOT compile
-hard-refuses breaks — but a fullgraph=True run reports the first break as fatal, so
-check which configuration you are actually in (`podman exec <ctr> printenv
-VLLM_COMPILE_FULLGRAPH`) before concluding a break blocks the boot. For the name-map
-KeyError there is a ready-made probe: put `/home/dank/dynamo-probe` on `PYTHONPATH`
-with `VLLM_DYNAMO_DUMP_ATTRS=1` (a `sitecustomize.py` — a `.pth` on `PYTHONPATH` is
-NOT executed, and the image's own `sitecustomize` is chained so its apport hook
-survives) and it prints, at the miss, the node target, the near-miss keys, and
-whether another live `OutputGraph` holds the key.
-
-**State of the changelist (2026-09-30, latest).** The trunk **traces**
-(`Dynamo bytecode transform time: 1.08 s`) and the boot dies on the dynamo name-map
-`KeyError: 'in_proj_qkvbfg_a'`. Established about it, in order:
-
-- It is raised **inside the traced `Glm5NextDecoderLayer.forward`** — the frame chain
-  is `profile_run → _dummy_run → execute_model → glm4_1v.py:2358 → the compile wrapper
-  → eval_frame.compile_wrapper → Glm5NextModel.forward:748 (layer(...)) → module
-  _call_impl → Glm5NextDecoderLayer.forward:456 → KeyError`. `TORCHDYNAMO_VERBOSE=1`
-  adds **no** internal stack trace, i.e. it is a plain Python KeyError in the traced
-  frame, raised in a callee dynamo **inlined** (the reported line is the `def`, which
-  is how inlined frames surface).
-- It is **not** raised from any of dynamo's four `nn_modules` lookup sites:
-  `/home/dank/dynamo-probe` (a `sitecustomize.py` on `PYTHONPATH`,
-  `VLLM_DYNAMO_DUMP_ATTRS=1`) wraps three of them and stayed silent across two runs
-  with the probe confirmed installed in both the API-server and worker processes.
-- It is **not** a string-keyed lookup of a literal: `in_proj_qkvbfg_a` appears in the
-  whole tree only as the KDA layer's attribute (`kda.py:217`, used at `:407`) and in
-  the `load_weights` shard table (`model.py:780-785`). The four
-  `proj_name = prefix.split(".")[-1]` sites in the quantization utils do compute
-  exactly this bare key, but every one of them is guarded (`if proj_name not in
-  fused_mapping: return`), so they cannot raise it. The pluggable-layer registry is
-  keyed by class name, not module path.
-- It reproduces on **every** run of this configuration (5 of 5), and it is
-  independent of patches 04/05: with breaks genuinely tolerated, those remove breaks
-  that were never fatal in the first place — the KeyError is the only thing standing
-  between this changelist and a boot.
-
-**Next instrument, precisely:** the inlined caller has to be named. Two cheap ways:
-`TORCH_LOGS=+dynamo` on a boot and grep the log for the key (dynamo prints inlined
-frames with their source), or wrap the inlining candidates in that forward — the mHC
-CustomOps (`hc_pre` / `hc_post` / `hc_fused_post_pre`, `layers/mhc.py`) and the
-indexer op — with a probe that reports on `KeyError`. Until that names the caller, this
-is a dynamo-side name-resolution failure and not a fixable fork bug.
-
-Two verified facts bound the rest of the shape: `suppress_errors` does **not** cover
-graph breaks (only `fullgraph=False` does), and the GLM trunk can never satisfy
-`fullgraph` because its indexer metadata syncs to host
-(`glm5next/amd/sparse_indexer.py:263,419`: `int(...max().item())`) — both are dead
-fallbacks on the decode path that the fork author already replaced, so they are not
-the blockers they appear to be at first glance.
-
-### Two structural constraints on any further fork work
-
-- **No new HIP kernels are possible.** The image ships `r4d.so` as a binary with no
-  sources. The obvious second lever — the 9 % router — is gated
-  `num_experts in {128, 256, 512}` in `router/r4d_route_router.py` and the binary
-  contains only `r4d_moe_route_kernel<128|256|512>` instantiations. GLM has **288**
-  experts, so it falls back to the generic sort-based top-k and **cannot be moved to
-  the fused router without rebuilding libr4d**. Only Python/config/inductor changes
-  (and runtime-JIT Triton) are available.
-- **`--kv-cache-dtype fp8` is a dead end for capacity too.** The plan's KV reserve
-  is dtype-blind (1.76 GiB at 131,072 either way), so fp8 buys no arena and no
-  window; it only removes the only MLA decode path gfx12 has (trap 1 above).
-
-### DFlash2 for GLM — what the drafter actually is
-
-`incoai/GLM-5.3-Flash-DFlash2` is **not** a GLM-shaped drafter and needs no MLA/DSA
-port: its `config.json` declares `architectures: ["DFlash2DraftModel"]` and
-`model_type: "qwen3"` — 5 dense layers, hidden 4096, 32 heads / 8 kv heads,
-head_dim 128, sliding attention (window 2048), `is_causal: false`,
-`use_cache: false`, `dflash_config` = block 8, conv group 16, selector rank 256 /
-top-k 16, and **`target_layer_ids: [5, 14, 24, 33, 42]` against
-`num_target_layers: 45`**.
-
-The fork already maps that architecture:
-`registry.py:636 "DFlash2DraftModel": ("qwen3_dflash2", "DFlash2Qwen3ForCausalLM")`,
-and the spec method is `"dflash"` (`config/speculative.py:67`). So the port is
-**target-side only**: the GLM trunk must expose auxiliary hidden states at those five
-layer indices, the same interface `deepseek_v2.py` (also MLA+MoE) implements —
-`set_aux_hidden_state_layers()` / `get_eagle3_default_aux_hidden_state_layers()` plus
-capture at `idx in self.aux_hidden_state_layers` in the layer loop. `glm5next`
-implements neither.
-
-**The load was tried, and the gap is exactly one interface.** Payload fetched to
-`/home/dank/models/GLM-5.3-Flash-DFlash2` and hash-verified (`model.safetensors`
-2,342,169,800 B, sha256 `b038e1d9…` = the Hub's LFS oid). Booting the engine with
-`--speculative-config '{"method":"dflash","model":"/draft","num_speculative_tokens":8}'`
-accepts the config and the drafter and dies at worker init with:
-
-```
-RuntimeError: Model does not support EAGLE3 interface
-```
-
-So nothing about the drafter, the method name, or the payload is wrong — the GLM
-trunk simply does not implement the aux-hidden-state contract the dflash runner
-requires. The reference is in this same image, twice: `deepseek_v2.py`
-(`set_aux_hidden_state_layers` / `get_eagle3_default_aux_hidden_state_layers` at :1948-1951,
-capture at :1554 and :1579) and the fork's own `qwen4_exp` trunk. The contract is
-small:
-
-- `self.aux_hidden_state_layers: tuple[int, ...]` on the trunk, default `()`, set
-  by the wrapper's `set_aux_hidden_state_layers()`;
-- in the layer loop, `if idx in self.aux_hidden_state_layers: aux_hidden_states.append(<single-stream state entering layer idx>)`
-  (DeepSeek appends `hidden_states + residual`, with a TP all-gather fixup when the
-  rank holds fewer tokens than `positions`);
-- `Glm5NextModel.forward` returns `(hidden_states, aux_hidden_states)` when the
-  list is non-empty;
-- `get_eagle3_default_aux_hidden_state_layers()` supplies a default triple when the drafter
-  config omits `target_layer_ids` (this drafter provides `[5, 14, 24, 33, 42]`, so
-  the runner derives trunk indices `[6, 15, 25, 34, 43]`).
-
-**The GLM-specific unknown** is what "the single-stream state entering layer k"
-means for a trunk whose layers carry mHC multi-stream state: DeepSeek's
-`(hidden_states, residual)` pair has no counterpart here — GLM's decoder layer
-returns `(hidden_states, residual, post, comb)` with `hc_expand`/`hc_pre` mixing
-`hc_mult` streams, so the plausible answer is the pre-mHC single-stream input `x`
-(4096-wide, matching the drafter's `hidden_size`), not a post-layer value. That is
-a semantic choice to validate by acceptance rate, not by inspection.
-
-**Budget no speedup.** Per the profile above, a verification pass pays pointwise
-cost proportional to its token count — the same reason MTP4 measured flat — and
-MTP4's acceptance was already excellent (3.76-4.90). The port is worth doing as
-*enabling work* (it is the prerequisite for any spec decode on this endpoint, and
-it is small), not as a performance fix, and the honest expectation on this shape is
-flat decode with a prefill regression like MTP's until the pointwise mass is fused.
-
-**The port exists now: `scripts/patches/glm53-dflash-01-aux-hidden-states.patch`** (WIP,
-independent of the compile changelist — it patches the same file as compile-01, so
-compile-01 goes first if both are applied; base `008eeaa2…` → patched `bc9913a1…`). It
-implements exactly the contract read out of this image: `EagleModelMixin` on
-`Glm5NextModel`, capture of the single-stream `hidden_states` ENTERING each requested
-layer in the loop, the conditional `(hidden_states, aux_hidden_states)` tuple return, and
-the three `SupportsEagle3` member attributes plus both interface methods on the
-wrappers (capture shape from `deepseek_v2.py:1551-1585`; tuple consumption at
-`v1/worker/gpu/cudagraph_utils.py:621`).
-
-**The patch went through five versions, and the decisive discovery was where the
-instance actually lives.** The traceback's `set_eagle3_aux_hidden_state_layers(self.model)`
-is called on the model the runner LOADED — and for GLM-5.3-Flash that is
-`Glm5NextForConditionalGeneration` (the multimodal wrapper; the APIServer log names it),
-which holds the text model as `self.language_model = Glm5NextForCausalLM`
-(`glm5next/common/model.py:1113`). Patching only `Glm5NextForCausalLM` (v4) still failed
-with the identical error because the `isinstance` ran against the wrapper. **v5 fixes
-both wrappers** and is the first version to clear the gate (both TP workers log
-`Using Eagle3 auxiliary layers from config: (6, 15, 25, 34, …)`).
-
-The full evolution, each an instructive trap:
-
-1. **v1** hand-wrote the two methods the traceback names → identical error. The methods
-   are not the whole contract.
-2. **v2** inherited `SupportsEagle3` as a base (the image's own idiom — `laguna.py:717`,
-   `mimo_v2.py:834`, …) → dies at class creation with `Cannot create a consistent method
-   resolution order (MRO) for bases …`. GLM's wrappers already carry four/five
-   Protocol-derived bases, and simulation confirmed the ForCausalLM's five-base ordering
-   cannot take `SupportsEagle3` in *any* position.
-3. **v3** declared the members as class attributes (MRO-safe) — but only on
-   `Glm5NextForCausalLM` → identical error (the wrong wrapper, see above).
-4. **v4** added all three members (`supports_eagle3`, `has_own_lm_head`,
-   `has_own_embed_tokens` — a runtime-checkable Protocol's `isinstance` walks ALL
-   `__protocol_attrs__` *including base-Protocol members* via `getattr_static`, verified
-   experimentally in-image with stock typing) on `Glm5NextForCausalLM` → identical error.
-5. **v5** adds the three attributes to BOTH wrappers + the two methods on both
-   (`Glm5NextForCausalLM` unwraps to `self.model` — the `EagleModelMixin` trunk;
-   `Glm5NextForConditionalGeneration` delegates to `self.language_model`, the
-   `pixtral.py:498` pattern for exactly this wrapper shape) → **gate passes**.
-
-**The lesson generalizes twice over**: when a runtime error names an interface, read the
-interface's declaration first — and establish WHICH INSTANCE the check runs against
-before patching anything: multimodal wrappers (`…ForConditionalGeneration`) are loaded
-models, not pass-throughs, and the text model behind `self.language_model` is invisible
-to a check run on the wrapper.
-
-**Retired diagnostic (do not resurrect as-is): the `/probe` startup hook.** An
-`op_probe.py` + `sitecustomize.py` pair was mounted to wrap vLLM module forwards and
-torch's name-map lookups. It produced no evidence and caused a **fork bomb** (see the
-hazard note under experiment-window discipline). Use `TORCH_LOGS` for dynamo questions
-instead — it needs no startup hook.
-
-Note also that the KV-cap patch's predecessor arithmetic in
-[[dankserver-qwen38-fp8-expert-offload-tuning]] should be read the same way: its
-"gather-bound" conclusion came from the same throughput arithmetic that this page's
-profiler window has now falsified.
-
-### The KV-cache grouping journey: clearing the unifier, then the kpool tail
-
-With patch 01 v5 the gate passes and the boot moves to KV-cache layout
-construction — where the next two blockers live, both in
-`v1/core/kv_cache_utils.py`:
-
-1. **The page-size unifier refuses GLM's MLA family.** With a draft model the
-   spec dict gains non-MLA drafter attention specs and `HiddenStateCacheSpec`
-   entries, so `_get_kv_cache_groups_glm5_next` (which requires ALL attention
-   specs to be `MLAAttentionSpec`) bails to the generic grouping path — whose
-   `unify_kv_cache_spec_page_size` raises
-   `NotImplementedError: Layer language_model.model.layers.3.self_attn.indexer.k_cache: page size is not divisible… cannot be padded. Padding is only supported for non-MLA attention layers.`
-   MLA layers cannot be *padded* (sparse MLA indexes whole token rows) — but
-   they CAN grow `block_size` when the max page is an exact multiple. The
-   indexer's page is just `tokens_per_state x` the MLA page, so
-   **`glm53-dflash-02-kv-unify-mla-multiple.patch`** bumps the unified max page
-   to a multiple of every `MLAAttentionSpec` page before the loop: all MLA-family
-   layers then take the legal divisible block-growth branch, drafter + mamba
-   layers pad. The 2026-09-30 window proved it: unify passes, memory profiling
-   passes, torch.compile passes, and the boot reaches cudagraph capture — the
-   NotImplementedError is gone.
-2. **The kpool tail kernel pins its block.** Cudagraph capture then dies in
-   `kpool_ops.kpool_decode_update_and_maybe_write_cache_batched`:
-   `assert tail_kv_cache.shape[2] == pool_size`. The tail cache must view as
-   `[num_blocks, 2, index_kpool, head_dim]` (`KpoolTailSpec(block_size=index_kpool)`,
-   `Glm5NextTailCache` in `glm5next/common/attention.py:159`), but the tail's
-   page (8192 B) *divided* the raised max page, so the unifier's divisible branch
-   grew its `block_size` by the ratio and the view became
-   `[num_blocks, 2, index_kpool x ratio, head_dim]`. Verified not the hybrid
-   promotion: the log carries no "Hybrid KV cache manager is disabled" warning,
-   so `disable_hybrid_kv_cache_manager` is unset and
-   `unify_hybrid_kv_cache_specs` never ran. **`glm53-dflash-03-kv-unify-tailpin.patch`**
-   (= 02 + a `KpoolTailSpec` branch that pads `page_size_padded` instead of ever
-   growing its block) **verified 2026-09-30**: the assert is gone and the boot
-   captures the DFlash2 drafter's FULL cuda graphs on both TP workers
-   (`Capturing model for DFlash2 speculator...`). Supersedes 02 in the repo.
-
-3. **The capacity wall: the generic path cannot alias the mamba states.**
-   Past capture, the boot dies at `_check_enough_kv_cache_memory`:
-   `Available KV cache memory: 0.92 GiB` against `31.76 GiB needed at max len
-   65536` (32.91 GiB at 131,072 — nearly flat, a fixed intercept, not
-   per-token KV). Two coupled causes, read out of the image:
-   - **The glm5 tensor layout aliases the mamba states INTO the MLA pages**
-     (`get_kv_cache_config_from_groups` places every mamba layer at the MLA
-     offsets — that is why GLM-5.3 fits 32 GiB cards at all, and why the
-     expert plan's KV registry is calibrated at 13,600 B/token). On the
-     generic path the mamba groups claim their OWN blocks, so ~30 GiB of
-     unaliased state pages appears.
-   - **The plan's KV reserve (`expert_activation_registry.py:_glm53_kv_bytes_per_token`,
-     a flat 13,600 B/token) knows nothing about the drafter.** The engine caps
-     KV at the plan's reserve, so even a correct generic-path estimate would
-     be starved.
-
-   Learned en route: the engine AUTO-SETS the attention block size to
-   **2,432 tokens** (`interface.py:918: Setting attention block size to 2432
-   tokens to ensure that attention page size is >= mamba page size`), so every
-   attention group's block counts scale from that floor — a 16-token drafter
-   block would need 4x the shared `num_blocks`; with the global 2,432 all
-   groups align.
-
-   **`glm53-dflash-04-kv-groups-draft.patch`** (+ **`-04b-kv-registry-dflash.patch`**
-   for the registry) keeps the aliased glm5 pool and extends it: the grouping
-   partitions the hidden-state specs (`HiddenStateCacheSpec` is an
-   `MLAAttentionSpec` subclass — excluded before the exact-type
-   `type(spec) is MLAAttentionSpec` partition) and the drafter attention
-   specs; the drafter layers form their own uniform group (per-group pages,
-   NO unification — the MLA block size stays at the engine-set 2,432 and the
-   kpool guard `block_size % 512 == 0` is never stressed); the layout returns
-   a 10-tuple with `draft_group`/`hidden_groups`; the builder sizes ONE
-   shared backing allocation (`allocate_kv_cache` asserts all tensor sizes
-   equal) as aliased pool + appended draft/hidden tensors with their own
-   strides; the memory estimator adds the extras. Patch 04b extends the
-   registry: 13,600 + 4,096 (drafter 8 kv heads x 128 x K+V x bf16) +
-   5 x hidden_size x 2 B (the aux hidden-state cache) per token when
-   `speculative_config.use_aux_hidden_state_outputs` is set.
-
-   The pure logic is smoke-tested locally (CPU torch + the image's real spec
-   classes against synthetic GLM+draft+hidden spec sets): grouping shape,
-   layout recognition, builder block math and memory accounting all pass.
-   Real-fit arithmetic: per-token ≈ 58,656 B → reserve ≈ 4.15 GiB at max len
-   65,536 (arena shrinks ~13.5 → ~10.3 GiB, ~1,650 slots), num_blocks ≈ 29 x
-   2,432 tokens ≈ 70,500 ≥ 65,536.
-
-All of these mount over site-packages via the scratch launcher's
-`KVUNIFY_PATCH=1` knob (host `/home/dank/dflash-patch2/new/…`, patched shas
-`5521e50b…` for `kv_cache_utils.py` and `1a623d5c…` for
-`expert_activation_registry.py`; apply-verified against the image's pristine
-files). The window script's decision pipeline was also fixed for a SIGPIPE
-death (`head -80` on a `podman logs` grep pipeline killed one run before
-log-save and restore; the log is now captured to a file FIRST, then grepped).
-
-
-## Measured arms (all 2026-09-30, one boot each)
-
-| arm | decode | prefill | verdict |
-| --- | --- | --- | --- |
-| baseline int4, MAXLEN 131,072, util 0.92, NBATCH 2048 | 10.03 / 10.08 / 9.41 | 877 / 878 / 885 | qualified incumbent |
-| + `--expert-precise-memory` (arena 13.08 → 14.51 GiB) | 9.88 / 9.91 | 918 | **neutral** — proves residency is no longer the binding term |
-| + MTP4 spec decode (draft loads, acceptance excellent) | 9.64-11.16 | **255** | **rejected** — see below |
-| + MTP4 + explicit `--kv-cache-memory-bytes` (patched cap) | boots, 151,767-token KV | — | enabler for the MTP arm only |
-
-### Why MTP is rejected here (it is not that it fails)
-
-MTP **works** on GLM in this fork: the head loads, the architecture resolves to
-`Glm5NextMTPModel`, and acceptance is excellent —
-
-```
-SpecDecoding metrics: Mean acceptance length: 3.83-4.90,
-Per-position acceptance rate: 0.917/0.833/0.583/0.500 … 0.962/0.846/0.692/0.615,
-Avg Draft acceptance rate: 69-97%
-```
-
-— and it buys **nothing**: decode 9.6-11.2 tok/s against 10.0 without it, while
-prefill **collapses 918 → 255 tok/s**. The reason is structural to offload: a
-verification forward gathers the **union** of the batch's experts, so spec
-decode amortizes compute but not a residency-bound gather. It also costs 2.6 GiB
-of arena. On a compute-bound (fully-resident) shape this would invert; here it
-does not.
-
-## Served surface (verified 2026-09-30 against the live engine)
-
-| capability | state | evidence |
-| --- | --- | --- |
-| `/v1/models` | `glm53-flash`, max_model_len 131,072 | model list probe |
-| completion | yes | `bench_glm.py` runs above |
-| **tool calls** | **yes** — needs `--enable-auto-tool-choice --tool-call-parser glm47` | a `get_weather` prompt returns `tool_calls: [{"id": "chatcmpl-tool-…", "type": "function", "function": {"name": "get_weather", "arguments": "{\"city\": \"Seattle\"}"}}]`, `finish_reason: tool_calls` |
-| **vision** | **yes** | a 64x64 solid-red PNG is answered "The image appears to be a solid red color…" |
-| reasoning channel | **yes** — needs `--reasoning-parser glm47` | without it the thinking leaks into `content` (measured: content began with the answer, then "Let me verify: … = 391", then a bare `</think`) with an empty reasoning field; with it, a train-time prompt answers in `content` ("**Step 1: Add the hours.** 14:20 + 3 hours = 17:20 …") and the chain of thought lands in **`reasoning`** — the vLLM field name the fleet documents (llama.cpp endpoints use `reasoning_content`) |
-
-Without the tool flags the engine answers **HTTP 400** — `"auto" tool choice
-requires --enable-auto-tool-choice and --tool-call-parser to be set` — which
-breaks every tool-using agent on the endpoint. Both parsers are `glm47`, the
-newest GLM dialect the fork ships (`glm45`/`glm47` exist for both).
-
-## Fork traps found (for the engine authors)
-
-1. **`--kv-cache-dtype fp8` is illegal on gfx12 with this model — and it buys
-   nothing even if fixed.** The `ROCM_AITER_MLA_SPARSE` backend's
-   `_use_rocm_sparse_triton()` requires a non-fp8 KV cache, and that rope-free
-   Triton path is the **only** MLA decode available on gfx12: aiter's MLA decode
-   op is registered under `@if_aiter_supported`, which is gated on **CDNA > 2**
-   (`is_aiter_found_and_supported()` → `get_cdna_version() > 2`). With fp8 KV
-   the MTP draft step demands an op that cannot exist here:
-   `RuntimeError: Worker failed with error ''_OpNamespace' 'vllm' object has no
-   attribute 'rocm_aiter_mla_decode_fwd''`. KV dtype therefore stays auto/bf16.
-   (`VLLM_ROCM_USE_AITER_MLA=1` does **not** help — the umbrella gate is the
-   CDNA check, not the env var.)
-   - **The capacity payoff is measured at zero.** The point of fp8 KV is fitting
-     more tokens in the same reserve, and this fork's reserve model is
-     **dtype-blind**: the fp8 boot printed the *identical* plan line as bf16 —
-     `KV reserve 1.76 GiB (13600 B/token x 131072 + mamba pages)`. The reserve is
-     dominated by the per-sequence KDA state (34 linear-attention layers) and the
-     DSA indexer's pools, not by the MLA KV bytes those 13,600 B/token barely
-     cover — so halving the KV dtype cannot move it.
-   - **What a fix would take, if it were ever wanted**: a new fp8 path in the
-     rope-free Triton kernels (`rocm_sparse_attn_prefill` / `rocm_sparse_attn_decode`
-     in `v1/attention/ops/rocm_aiter_mla_sparse.py`) that dequantizes pages
-     in-kernel, plus loosening `_use_rocm_sparse_triton()`. The fork's
-     `_r4d_fp8_paged_mqa_logits` shows the author already has an fp8 MQA-logits
-     path, but the sparse-MLA backend is not wired to it. Verdict 2026-09-30:
-     **do not spend effort here** — the payoff is capacity, and the reserve is
-     dtype-blind.
-2. **The plan's KV reserve under-models this model, and the cap enforces the
-   mistake.** The plan charges 13,600 B/token + a small mamba term, but GLM
-   carries a per-sequence KDA state (34 linear-attention layers) and, under MTP,
-   the draft's KV. `gpu_worker` then *caps* KV at that reserve, so a legal
-   `--max-model-len` becomes unbootable:
-   `ValueError: To serve at least one request with the model's max seq len
-   (131072), (2.07 GiB KV cache is needed, which is larger than the available KV
-   cache memory (1.74 GiB))`. Raising `max_model_len` only helps past ~440k
-   tokens (the cap's slope is ~13,845 B/token vs the need's ~12,700 + 0.52 GiB
-   fixed) — where the reserve would eat a third of the arena.
-   `scripts/patches/gpu_worker-kv-cap.patch` lets an explicit operator KV
-   request raise the cap; **it is not part of the deployed shape**, which boots
-   on the plan's own reserve.
-3. **A window is forced, not chosen.** The production `max_model_len 131072` is
-   what the reserve model requires for this payload (below ~92k the boot
-   refuses); it is not a capacity decision.
-4. **Expert prefetch remains poison** — unchanged from the qwen campaign
-   (36% precision, +40-51% decode from deleting it). No GLM arm re-opened it.
-
-## Operations
-
-```bash
-# rollback to the pre-cutover qwen FP8 + MTP4 shape. The trio is in git; the
-# qwen FP8 payload is NOT on disk any more, so re-fetch it first. The R9V state
-# dirs are still on the host (/home/dank/r9v-state/):
-git show 2339fae -- scripts/ | git apply   # or: git checkout 2339fae -- scripts/
-./scripts/deploy.sh dankserver --service frank-archon
-
-# payload inventory (34 shards + 2 extras, weight-map index) — in the GLM era
-# this was scripts/download-frank-primary.sh (REPO=Intel/GLM-5.3-Flash-W4A16-AutoRound,
-# EXPECTED_SHARDS=34, git show f2bcf84). The same-named script today fetches
-# Qwen/Qwen3.8-Flash-Next-FP8 (scripts/download-frank-archon.sh:21):
-bash scripts/download-frank-archon.sh --verify-only
-
-# live telemetry (needs the profiler window; with_stack=false is mandatory —
-# with stack the stop path hangs and the trace buffer grows to tens of GB):
-podman logs frank-archon-glm 2>&1 | grep "expert_lru.py"
-```
-
-**Experiment-window discipline (learned the hard way, 2026-09-30).** This engine
-**ignores SIGTERM**: after `systemctl stop` its container sits in `podman ps` state
-`Stopping` for 20+ minutes *while still answering `/health` with 200*. That makes a
-liveness probe a lying oracle for a restore — twice a restore script printed
-`PROD HEALTHY` while `systemctl is-active` said `failed` and the runtime mask was
-still in place. The restore oracle must be a **conjunction**: `is-active` ==
-`active` **and** `is-enabled` == `enabled` **and** no `/run/systemd/system/<unit>`
-mask symlink **and** the container is owned by the unit and not `Stopping`. The
-repair recipe, all four steps needed:
-
-```bash
-sudo mv /run/systemd/system/frank-archon.service /tmp/stale-mask-N.service  # runtime mask SURVIVES stop and blocks start
-sudo systemctl daemon-reload
-sudo systemctl reset-failed frank-archon.service                            # a failed unit refuses to start
-podman rm -f frank-archon-glm                                               # SIGKILL; `podman stop` hangs on the same SIGTERM
-sudo systemctl start frank-archon.service                                   # then ~7 min to health 200
-```
-
-Also: **never let two experiment jobs overlap.** A later job's `podman rm -f
-glm-probe` cleanup ran while an earlier job was still mid-flight and deleted its
-container, so one whole round produced no evidence at all (`no container with name
-glm-probe found`) — check `proc://` for running jobs before opening a window.
-
-**Never put a diagnostic hook on `PYTHONPATH` without a marker guard — it is a fork
-bomb.** On 2026-09-30 a `sitecustomize.py` mounted at `/probe` (to wrap dynamo's
-name-map lookups and vLLM module forwards) produced a self-replicating chain of
-`python3` processes 19+ levels deep inside the probe container: `sitecustomize` is
-imported by *every* interpreter, the hook imported torch/vLLM at startup, something in
-that import chain spawned a helper python, and the child inherited `PYTHONPATH` (hence
-the hook) and spawned again. It was still alive 18 minutes later with the container
-still showing `Up`. Symptoms: `ps -eo pid,ppid,comm --forest` shows a deep ladder of
-identical interpreters, each child's PPID being the previous one. Guard (now in the
-hook): arm once per environment via a marker in `os.environ`, which children inherit —
-
-```python
-if os.environ.get("VLLM_PROBE_ARMED") == "1":
-    pass                       # an ancestor already armed it; stay inert
-else:
-    os.environ["VLLM_PROBE_ARMED"] = "1"
-    <install probes>
-```
-
-Kill recipe: `podman rm -f <experiment-container>` (the container cgroup owns every
-generation), then verify with a precise count over `/proc/<pid>/environ` for the marker
-variable — **never** pattern-kill an interpreter name, because the media stack on this
-host (romm, jellyfin, plex, glances) runs python in production.
-
-```bash
-# boot: ~7 min warm page cache, longer cold; health returns 503 while loading
-curl -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:7770/health
-
-# experiment windows (production stopped + `systemctl mask --runtime`, which
-# self-clears at reboot) ran from the host-side scratch launcher
-# /home/dank/serve_glm_autoround.sh: env knobs MAXLEN SEQS NBATCH UTIL KVBYTES
-# KVDTYPE SPEC SPECJSON_FILE TOOLPARSER REASONINGPARSER TELEM GCHUNKS GLANES
-# PROFDIR GLM_PATCH_MOUNT COMPILE_PATCH GATHER_PATCH DRAFT_MOUNT ENVARGS.
-# DRAFT_MOUNT=1 mounts /home/dank/models/GLM-5.3-Flash-DFlash2 at /draft;
-# SPECJSON_FILE passes a whole --speculative-config JSON (JSON with no spaces,
-# word-split by the launcher's unquoted $EXTRA).
-bash /home/dank/serve_glm_autoround.sh
-```
-
-Boot gate: the engine refuses a legal-looking shape in two measured ways — the
-KV cap above, and the profile-run activation peak (`TritonMLA` indexer scores:
-64 heads x 2,048 topk x NBATCH tokens x 2 B = 2.06 GiB at NBATCH 8192), which is
-what killed the first two boots at util 0.95.
-
-`PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True` is part of the qualified
-shape, not a tuning: the plan fills VRAM to its modelled headroom by design, so
-the default allocator's fragmentation fails the profile/graph-capture phase
-(measured on the deployed unit: "Tried to allocate 662.00 MiB. GPU 0 … 240.00
-MiB is free").
-
-The `deploy.sh` e2e gate also drives the admin panel's open-links spec, which
-walks every guarded vhost — including `http://192.168.0.112:8989/` on
-FrankBook-Pro. When that Mac mini is asleep the spec fails on a `page.goto`
-timeout and the deploy reports `DEPLOY INCOMPLETE` even though the engine is
-healthy; wake the host (magic packet to `de:6b:6d:c1:52:23`) or re-run
-`tests/e2e/run.sh` once it is up. Seen 2026-09-30: 32 passed, this one failed,
-and the WoL packet did not wake it (macOS "Wake for network access" needs
-toggling on the mini itself).
-
-## Rollback targets
-
-| state | where |
-| --- | --- |
-| qwen FP8 + MTP4 (2026-09-29 → 09-30) | commit `2339fae`; payload `/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8` was deleted from the host (2026-10-18: `/home/dank/models/` has `StillDeadcode`, `rad`, `tpl.jinja`) — rollback = re-fetch |
-| R9V radiance Qwen3.8-Flash-Next Q8_0 | [[frank-archon-r9v-radiance-qwen38-q8]], state dirs `/home/dank/r9v-state/{q8xl,uncensored}` |

serverAppConfigs-docs/LLM/frank-archon-llamacpp-qwen38-flash-next-q8.md

@@ -1,270 +0,0 @@
----
-type: configuration
-title: frank-archon llama.cpp Qwen3.8-Flash-Next Q8_0 Runbook
-description: RETIRED 2026-09-26 → 09-27 frank-archon runbook — unsloth llama.cpp (the b11030-mix prebuilt, ROCm gfx120X, bundled runtime, root systemd unit) serving unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 (175.3 GiB across six split parts) on Dankserver's two Radeon AI PRO R9700s (gfx1201), with the shared Q8_0 MTP drafter (depth 2) chained behind ngram-mod, BF16 vision, and the routed experts + 51B-param n-gram embedding table computed in host RAM. Port 7770, one stream, full 262,144 ctx. Cutover 2026-09-26, replacing the same-day Lucebox DS4.1 deployment; superseded 2026-09-27, and frank-archon runs [[frank-archon-radiance-qwen38-iq4r]] today (2026-10-18).
-tags:
-  - frank-archon
-  - dankserver
-  - llamacpp
-  - unsloth
-  - qwen38-flash-next
-  - q8_0
-  - mtp
-  - ngram-mod
-  - spec-decode
-  - vision
-  - rocm
-  - gfx1201
-  - r9700
-timestamp: 2026-09-26
----
-
-# frank-archon llama.cpp Qwen3.8-Flash-Next Q8_0 Runbook
-
-The dankserver LLM endpoint (:7770, alias **frank-archon**) as it stood from
-2026-09-26 to 2026-09-27 (retired; the port today serves
-[[frank-archon-radiance-qwen38-iq4r]]):
-unsloth llama.cpp serving `unsloth/Qwen3.8-Flash-Next-GGUF` at **Q_8**
-(the 6-part `Q8_0/` main, 175.30 GiB) on **both** Radeon AI PRO R9700s,
-with the **shared Q_8 MTP drafter** (`draft-mtp`, depth 2) layered with
-**`ngram-mod`**, and **vision** through `mmproj-BF16.gguf`. It replaces the
-Lucebox DS4.1 quadlet that held the port for hours earlier the same day
-([[frank-archon-lucebox-ds4]]), which itself replaced
-[[frank-medium-radiance-mxfp4]].
-
-## Why this shape
-
-- **One runtime for the fleet.** All three endpoints are llama.cpp now —
-  one metric namespace (`llamacpp:*`), one reasoning contract
-  (`reasoning_content`), one tool contract (`--jinja`). The two-day
-  experiment with a single-vendor engine (lucebox) ended because Qwen
-  Flash-Next at Q8 on llama.cpp delivers better quality than the DS4.1
-  fp2/fp3 stream, plus vision, plus the full 262k window, at comparable
-  speed.
-- **The placement law, not a preference:** the routed experts and the
-  51B-param n-gram (PLE) embedding table are **computed in host RAM**
-  (`-ot exps=CPU,per_layer_token_embd.weight=CPU`) and never streamed over PCIe. The
-  x4-pair measurement on the same model shape: streaming loses (3.7 t/s),
-  computing in RAM does not care (20+ t/s). The dense weights, attention,
-  projector and MTP head sit on the two cards (~29 GiB); the ~100 GiB of
-  expert pages live in mmap page cache — `--no-mmap` is forbidden (it
-  forces the table into anonymous memory; the cgroup kill is silent) and
-  pinning the PLE table with a GPU target cost 23 → 13.4 t/s when tried.
-  The tensor names are verified, not guessed: the GGUF v3 header dump
-  (`dump-gguf.py`, run on the fetched parts 2026-09-26) shows the PLE
-  table is one tensor — `per_layer_token_embd.weight`, dims
-  `[160, 320001536]`, Q8_0, 54.4 GiB, exactly the whole of part 3 — and
-  the model is `qwen4exp` with 512 experts top-10
-  (`qwen4exp.ple.ngram_size=3`, `heads_per_ngram=8`). The same tensor is
-  the MTP head's borrow target, so the override is required even if the
-  placement were optional.
-- **Two cards, layer split** (`-ts 1,1`): both R9700s are on CPU root
-  ports with full PCIe atomics (the ProArt B850-Creator swap,
-  2026-09-20), but this deployment uses llama.cpp's layer split —
-  `-ot` tensor overrides do not compose with tensor-parallel mode, and
-  `exps=CPU` is the override the placement law needs. The 27B-dense
-  equivalent on this host measured 1991 t/s prefill / 18.8 t/s decode in
-  this shape; Flash-Next Q8 with CPU experts lands in the same class
-  (measured numbers below).
-- **Spec chain: `draft-mtp,ngram-mod`.** MTP is a property of the FILE:
-  the shared-Q8_0 head (`mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf`,
-  2,786,568,256 B) is loaded via `--model-draft` and borrows the target's
-  embedding tensors — which is why `per_layer_token_embd.weight` must be
-  CPU-resident in the override set (a GPU-resident borrow crashes the
-  draft load with `invalid vector subscript`). `ngram-mod` is draftless,
-  so it chains legally behind the drafter. Depth 2: each longer
-  verification round re-batches the CPU expert reads, so depth buys less
-  than bandwidth here. **The drafter must sit on ROCm1** — the head also
-  borrows the target's `output.weight`, which a `-ts 1,1` split places on
-  the last-layers card; with tensor overrides set the draft scheduler runs
-  `n_copies==1`, and a draft context elsewhere aborts at load
-  (`ggml-backend.cpp:941`, "pre-allocated tensor (output.weight) in a
-  buffer (ROCm1) that cannot run the operation"). The 2026-09-26 cutover
-  crash-looped 4× on `ROCm0` before the probe confirmed `ROCm1` loads —
-  the frank-medium precedent (drafter on the last-layers card) was right.
-- **Vision, explicitly.** This build never autoloads projectors — without
-  `--mmproj` the server comes up healthy and text-only and answers image
-  requests with 500 (the frank-medium 2026-09-17 trap). The BF16 projector
-  (907,542,944 B) is named in the serve script.
-
-## Deployment
-
-|Piece|Value|
-|---|---|
-|Unit|`scripts/frank-archon.service` — root systemd unit, `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity`, relinked to `/etc/systemd/system/`|
-|Serve script|`scripts/serve-frank-archon.sh` — validates binary + all 6 parts + drafter + mmproj before exec|
-|Env tunables|`scripts/frank-archon.env` — paths and numbers only; bind is `BIND_HOST` (never a bare `HOST`: HIP reads it)|
-|Runtime|`/home/dank/GitPackages/unsloth-llama-b11030/llama-server` — unsloth b11030-mix prebuilt (commit 6ba30d05b), ROCm gfx120X, bundled runtime via `LD_LIBRARY_PATH`|
-|Payload|`/home/dank/models/Qwen3.8-Flash-Next-GGUF/` — `Q8_0/` (6 parts), `MTP/mtp-...-shared-Q8_0.gguf`, `mmproj-BF16.gguf`; fetched by `scripts/download-frank-archon.sh` (sha256 + byte-count verified)|
-|Port / alias|7770 / `frank-archon` — unchanged from the lucebox and radiance eras; no client moves|
-|Context|262,144 × 1 slot (`PARALLEL_SLOTS=1`, one stream, the model's full native window)|
-|Sampling|Greedy server default (deterministic e2e); Qwen-recommended sampling rides `models.yml` `compat.extraBody` per request|
-|Template|The GGUF's embedded template via `--jinja`; reasoning → `reasoning_content` (`--reasoning-format deepseek`)|
-|Observability|`frank-archon` Prometheus job at `192.168.0.235:7770` — cross-host LAN scrape (the netavark published-port quirk that forces `host.containers.internal` is same-host only); exports `llamacpp:*`|
-
-### Cutover mechanics
-
-The deployer (`deploy_llm_archon` in `scripts/deploy-remote.sh`) gates on
-every artifact at its declared byte count (`stat -c %s`, never `wc -c` —
-which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
-retires the lucebox user quadlet to dated home trash
-(`~/.AGENTTRASH/20260926-llamacpp-cutover/`), waits for port 7770 to be
-truly free, relinks + enables + restarts the root unit, and waits up to
-1800 s for `/health` (cold start page-ins ~175 GiB and builds the graphs).
-
-### Preflight the GGUF without downloading it
-
-GGUF metadata lives in the header and HF honours ranged GETs, so a few MB
-answers every sizing question before a 13 GB transfer
-(`Range: bytes=0-8000000`). Parse: magic `GGUF`, u32 version, u64
-n_tensors, u64 n_kv, then key/value pairs (string = u64 len + bytes; ARRAY
-= u32 elem type + u64 count; BOOL is 1 byte — the type hand-written parsers
-forget). Walk arrays by byte size even for a summary: a naive parser
-desynchronises on `tokenizer.ggml.tokens` and lands gigabytes off. Read:
-`general.architecture`, `<arch>.block_count`, `<arch>.context_length`,
-`<arch>.nextn_predict_layers` (MTP is a property of the FILE — the `-mtp`
-build carries `blk.<n>.nextn.*`; `--spec-type draft-mtp` then loads the
-head against the MAIN model's weights, and `--model-draft` would load the
-file twice), `<arch>.head_count_kv`, `<arch>.key_length`/`value_length`,
-`<arch>.full_attention_interval`, `<arch>.ssm.state_size`/`group_count`/
-`time_step_rank`, and for the projector `clip.projector_type` +
-`clip.has_vision_encoder`.
-
-KV arithmetic for the hybrid (16 × (3 Gated DeltaNet + 1 Gated Attention)):
-only full-attention layers take KV
-(`attn_layers × n_head_kv × head_dim × 2 × bytes_per_elem`; f16 = 2 B,
-q8_0 = 1.0625 B, q4_0 = 0.5625 B); DeltaNet state is a fixed per-slot
-buffer (~0.16 GiB/sequence at 48 layers/128 d_state/48 v-heads f32); the
-MTP draft context costs ≈ 4 KiB/token. `--parallel N` DIVIDES `--ctx-size`
-between slots — it does not multiply.
-
-## Verification (2026-09-26 deploy)
-
-- `/v1/models` reports `frank-archon`.
-- Chat completion returns `content` + `reasoning_content`.
-- Tool call returns well-formed `tool_calls` (`--jinja`).
-- Real image request answered from the projector (not a health check —
-  health cannot see vision).
-- Spec chain live: `draft acceptance` lines in the unit journal; ngram-mod
-  has no startup banner (identify behaviourally: `draft_n` > 0 where text
-  repeats).
-- `reasoning_effort: high` → 200 (the template's `{xhigh, medium, low}`
-  gate is folded by `reasoningEffortMap` in `models.yml`).
-- VRAM/RAM headroom: `rocm-smi` per card + `free -g` after the first long
-  completion.
-
-_**Measured (2026-09-26 battery, 6/6 pass).** Load: ~7 s warm (mmap from
-page cache; the fetch's own verify pass pre-warms it — a cold first load
-pages in 175 GiB). Decode: **11.8 t/s** `predicted_per_second` at
-depth-0 prose, 128-token generation, draft_n 109 / accepted 72 (0.66 on
-reasoning-heavy prose; **0.864 mean len 2.73** on the tool-call task).
-Vision: the probe image (green square centered on a red field) described
-exactly. VRAM: GPU0 29% / GPU1 36% of 32 GiB (~9.3 / ~11.5 GiB — the
-output-head card carries the drafter, as designed; ~22 GiB free per
-card). RAM: ~11 GiB used, ~226 GiB page cache (the payload itself),
-available ~235 of 246 GiB. `reasoning_effort: high` → 200. Two cutover
-traps for the record: a LAST-ACK tail from a LAN client held
-`wait_port_free` until it cleared (a foreign process on the port gates
-the deploy correctly — but the deployer's `/health` wait cannot tell WHO
-answers; after any aborted deploy, confirm the responder is the unit's
-own MainPID), and llama-server's SIGTERM cleanup hung after
-spec-decode service (logged `cleaning up before exit` and never exited —
-`kill -9` was needed; `systemctl stop` relies on TimeoutStopSec)._
-**KV law (from the -lv 4 load log, cross-checked by the GGUF header):**
-29,696 B/token total ctx-scaled VRAM — 24,576 B main KV (only 12 of 48
-layers carry attention, `full_attention_interval=4`; 2 KV heads x 256,
-f16) + 3,072 B indexer + 2,048 B draft KV. Everything else (weights
-~6.7 GiB, recurrent state 0.33 GiB, compute buffers ~4.1 GiB) is
-ctx-independent. At 262,144: 7.25 GiB scaled, 18.75 GiB total, 9.37
-GiB/card. At 100,000: 2.77 GiB scaled, 14.3 GiB total, 7.1 GiB/card.
-`--parallel N` splits the same pool — slots do not multiply KV.
-
-**Reshape (later the same day): four full slots, V q8_0, hot expert
-blocks on-GPU.** `CONTEXT_SIZE=1048576` / `PARALLEL_SLOTS=4` — four
-conversations, each the FULL native 262,144 window (per-slot window
-unchanged, so no client or models.yml implication; slots divide the
-pool, never multiply it). `CACHE_TYPE_V=q8_0` with K held at f16: main
-KV drops 24,576 → 18,816 B/token (−23.4%), pool law 23,936 B/token.
-The shared MTP drafter's KV ignores the V type (measured: draft pool
-2,048 MiB = the f16 law at 1,048,576 cells — one layer's worth, the
-miss costs ~0.5 GiB on ROCm1). At 4×262,144 the pool is ~23 GiB across
-both cards: main 9,408 MiB + indexer 1,536 MiB per card, draft
-2,048 MiB on ROCm1.
-
-Expert blocks stay GPU-resident — and the measurement below shows it
-cannot matter *which* ones. `-ot` granularity is the whole
-per-block expert tensor (512 experts share one tensor), so "critical
-experts" = blocks co-located on the card that holds their layers:
-blocks 0-4 on ROCm0, 45-47 on ROCm1 (measured 2.42/2.49 GiB per
-block). The placement law that makes it work: b11030 wants ONE
-comma-joined `--override-tensor` (repeats log DEPRECATED), and the
-loader applies the FIRST matching pattern per tensor — the
-block-specific patterns MUST precede the broad `exps=CPU` sweep, or
-the sweep silently swallows them (verified live: broad-first left
-every expert on CPU with no warning anywhere in the log).
-
-Measured headroom, deployed fill **5/3** (blocks 0-4 on ROCm0, 45-47
-on ROCm1 — the operator directed the fill past the 3-4 GB checkpoint
-floor the same day, "shift a few more experts"): card0 2.53 GiB free
-at boot, 2.23 GiB under load with the vision graph exercised; card1
-3.31 GiB at boot, 3.15 GiB under load. Card1 cannot take another
-block — 3.31 − 2.49 = 0.82 GiB, and the vision graph (~0.43 GiB) plus
-boot drift alone exceed that — so 5/3 is the physical limit at the
-4×262144 / V-q8_0 shape. Earlier 4/3 state for reference: card0 5.02
-GiB boot free. Live numbers (4/3 era): content decode 13.05 t/s
-(acceptance 0.58, mean len 2.16), reasoning phase 7.24 t/s
-(acceptance 0.74, mean len 2.48), two concurrent full generations both
-delivered, prompt cache reusing 42/64 tokens across requests. All
-three Prometheus fleet targets up; the 5/3 deploy re-verified chat,
-vision and the target on the same battery class.
-
-**Which blocks, measured (same day):** asked whether the `-ot` regex
-could target the *most active* blocks, I instrumented the live server
-instead of guessing — `scripts/frank-archon-expert-activity.py`
-samples llama-server with AMD IBS (`perf record -e ibs_op// --data`)
-while 4 mixed concurrent generations run, maps each sampled load
-address through `/proc/<pid>/maps` back to its backing tensor (extents
-derived from consecutive GGUF header offsets — never from a quant
-type-size table, whose errors silently mis-bucket sample tails), and
-aggregates per block and per expert. Verdict, 75 s window / 2.7M
-expert-tensor samples: **per-block read volume is FLAT, 0.93×–1.08× of
-mean across all 40 CPU-resident blocks** (0-4 and 45-47 absent —
-GPU-resident). MoE top-k routing fires every block on every token, so
-"most active layers" do not exist at `-ot` granularity: the regex can
-express any 8-block set, and any set is equivalent to any other — the
-lever is COUNT, already at the 5/3 VRAM ceiling. The real skew is
-per-EXPERT (top 10% of the 48×512 (block,expert) cells carry 40.5% of
-samples; hottest cell 0.1%), below the reach of a tensor-level
-override. Not a benchmark: it instruments production serving in place
-at negligible IBS overhead; re-measure any future checkpoint or
-traffic mix with `frank-archon-expert-activity.py <pid>` (or
-`--analyze-only` against the saved sample file in `/tmp`).
-
-## Rollback
-
-The lucebox DS4.1 image `localhost/lucebox-rocm:gfx1201` is still in dank's
-rootless storage; the 378 GiB payload is not (Dankserver's `/home/dank/models/`
-holds only `StillDeadcode`, `rad` and `tpl.jinja` as of 2026-10-18), and the
-quadlet-era repo files were deleted by `8500600` rather than trashed — there is
-no `dankserver/.AGENTTRASH/20260926-llamacpp-cutover/` (see
-[[frank-archon-lucebox-ds4]]). Restore = recover the `.container` from
-`git show 8500600^`, stop the root unit (`systemctl stop frank-archon`; the
-fleet law forbids `systemctl disable` on these hosts — it removes the unit
-entry, which here is the symlink into the checkout), `daemon-reload`, start the
-user quadlet. The Qwen payload (~179 GiB) is no longer on disk either, so a
-forward re-swap is a re-fetch plus a redeploy.
-
-## Fleet context
-
-- [[frank-dragoon-unsloth-qwen38-27b-q8xl]] — the era's Dankclaw :7770, the same
-  b11030-mix binary family on gfx1151 (retired 2026-09-28; frank-dragoon runs
-  [[frank-dragoon-flash-server-v2-native]] today).
-- [[frank-zealot-gemma4-12b]] — the era's Dankclaw :7771, mainline llama.cpp on
-  the 3090, and then the fleet's only audio-capable endpoint. Both are history:
-  frank-zealot serves [[frank-zealot-llama-prism-bonsai2-pq20]] today and no
-  fleet endpoint takes audio (the audio path left with Gemma4 on 2026-10-01).
-- The harness contract lives in `/home/blank/.omp/agent/models.yml`. The era's
-  `llamacpp (dankserver)` numbers (`contextWindow 262144`, `maxTokens 65536`,
-  `input: [text, image]`) are long gone — that provider reads
-  `contextWindow: 200000`, `maxTokens: 16384`, `input: [text, image]` as of
-  2026-10-18.

serverAppConfigs-docs/LLM/frank-archon-lucebox-ds4.md

@@ -1,222 +0,0 @@
----
-type: configuration
-title: frank-archon Lucebox DeepSeek V4 Runbook
-description: RETIRED 2026-09-26 frank-archon runbook (active for hours on 2026-09-26) — the Lucebox runtime (github.com/Luce-Org/lucebox, local image localhost/lucebox-rocm:gfx1201) serving DeepSeek-V4.1-Flash-ROCMFP23 (377.2 GiB GGUF, 43-layer MoE, 256 routed experts, top-6) on Dankserver's R9700 with the routed experts computed on CPU from the mmapped page cache. Replaced the same day by [[frank-archon-llamacpp-qwen38-flash-next-q8]] (llama.cpp Qwen3.8-Flash-Next Q8_0); the image is still in rootless storage but the payload is gone from the host, and frank-archon runs [[frank-archon-radiance-qwen38-iq4r]] today (2026-10-18).
-tags:
-  - frank-archon
-  - dankserver
-  - lucebox
-  - deepseek-v4
-  - ds4
-  - rocmfp23
-  - gfx1201
-  - r9700
-  - quadlet
-  - moe
-timestamp: 2026-09-26
----
-
-# frank-archon Lucebox DeepSeek V4 Runbook
-
-> [!warning] RETIRED 2026-09-26 (same day it went live)
-> Replaced by [[frank-archon-llamacpp-qwen38-flash-next-q8]] — unsloth
-> llama.cpp serving Qwen3.8-Flash-Next Q8_0 on both R9700s. This runtime
-> served for hours; the image `localhost/lucebox-rocm:gfx1201` is still in
-> dank's rootless storage, but the 378 GiB payload was deleted from the host
-> (`ls /home/dank/models/` on Dankserver, 2026-10-18: `StillDeadcode`, `rad`,
-> `tpl.jinja` only), and the quadlet-era repo files were deleted by `8500600`
-> (2026-09-26) — they survive only in git history, not in a `.AGENTTRASH`
-> directory (`dankserver/.AGENTTRASH/` holds only `20260916-vllm-radiance`).
-
-The dankserver LLM endpoint (:7770, alias **frank-archon**) as it stood on
-2026-09-26: the
-[Lucebox](https://github.com/Luce-Org/lucebox) runtime serving
-`Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF` (377.2 GiB, 405,349,859,328
-bytes, LFS oid
-`2daea6ca1715bc987aae201e63445d3f7abdc2337c05f5f569f5e50237ce8be2`) on a
-single Radeon AI PRO R9700 (gfx1201). It replaced
-[[frank-medium-radiance-mxfp4]] (vllm-radiance TP=2), retired earlier the
-same day.
-
-## Why this shape
-
-DeepSeek-V4.1-Flash is a 43-layer MoE: 256 routed experts + 1 shared expert
-per layer, top-6 routing. The ROCMFP23 quantization is fp2 gate/up with fp3
-down experts (Q2_0_ROCMFP2 type 107 + Q3_0_ROCMFPX type 104). At 377.2 GiB
-the model fits neither the 63.7 GiB of combined VRAM nor the 246 GiB of RAM,
-so it is served the way llama.cpp serves big models by default: the GGUF is
-mmapped, pages pull from disk/page cache on demand, dense work (attention,
-shared expert, embeddings, output projection) and the MLA KV cache run on
-the R9700, and the routed experts are computed on CPU. The 246 GiB RAM
-(upgraded for this deployment) holds the working set of active experts; the
-rest streams from the 1.8 T NVMe.
-
-Lucebox's `--ds4-fused-decode` requests a monolithic GPU load; on a model
-this size that allocation fails and the backend falls back to **hybrid
-expert placement + layered decode** — that fallback is the intended serving
-mode here, not an error.
-
-## Image
-
-`localhost/lucebox-rocm:gfx1201`, built on the host from
-`github.com/Luce-Org/lucebox` `Dockerfile.rocm`:
-
-```
-git clone --recurse-submodules https://github.com/Luce-Org/lucebox.git
-cd lucebox
-# Three local modifications (tracked at the time as
-# dankserver/frank-primary/lucebox-gfx1201.patch, deleted by 8500600):
-#   1. Dockerfile.rocm drops the ROCm *-dev apt packages (the 7.14.1-full
-#      base already ships them in /opt/rocm; the apt repo does not serve
-#      them) and the three runtime math libs.
-#   2. Dockerfile.rocm installs the official cmake 3.31.6 binary (jammy's
-#      3.22 predates the `block` command ROCm 7.14 hipblaslt-config.cmake
-#      requires, CMake >= 3.25).
-#   3. ggml-cuda.cu: `ggml_cuda_node_launches_nothing` moved above the
-#      `#ifdef USE_CUDA_GRAPH` guard — `ggml_cuda_node_is_noop` calls it
-#      unconditionally, but the macro is never defined by the build
-#      system, so the in-guard definition was dead and the use failed.
-git apply ../lucebox-gfx1201.patch
-podman build --build-arg LUCE_HIP_ARCHES=gfx1201 \
-             --build-arg ROCM_VERSION=7.14.1-full \
-             -t localhost/lucebox-rocm:gfx1201 -f Dockerfile.rocm .
-```
-
-- The prebuilt `ghcr.io/luce-org/lucebox-hub:rocm` is **not usable as-is**:
-  it is built for gfx1151 + ROCm 6.4.1, and gfx1200/gfx1201 are not
-  code-object compatible.
-- `ROCM_VERSION=7.14.1-full` matches the host driver's ROCm 7.x line (the
-  radiance image's core-7.14 userspace runs on this host). A 6.4.x userspace
-  on a 7.x driver can segfault at model load. Only `-full` tags exist for
-  7.14.x.
-- The binary lands at `/opt/lucebox-hub/server/build/luce_server`.
-
-## Model
-
-`/home/dank/models/DeepSeek-V4.1-Flash/DeepSeek-V4.1-Flash-ROCMFP23.gguf`,
-fetched with `huggingface_hub` (the xet-native path — plain `curl` full GETs
-400 against HF's xethub CDN for this repo; see the fetch note in
-`scripts/hf-fetch.sh`). Byte count and sha256 are deploy-gated in
-`deploy_llm_archon`.
-
-## Deployed flags (2026-09-26)
-
-From `dankserver/frank-primary/frank-primary-entrypoint.sh` (deleted by `8500600`;
-recover it with `git show 8500600^:dankserver/frank-primary/frank-primary-entrypoint.sh`) — the unit
-bypasses the image's entrypoint, which only forwards a fixed Qwen/DFlash
-flag set and cannot express the DS4 flags):
-
-| Setting | Value |
-|---|---|
-| Model | `DeepSeek-V4.1-Flash-ROCMFP23.gguf` (377.2 GiB) |
-| Device | `--target-device hip:0` (`HIP_VISIBLE_DEVICES=0`; the Raphael iGPU at index 2 stays out) |
-| Window | `--max-ctx 131072` (the qualified Strix Halo window; DS4's MLA KV is compressed, so the pool is small) |
-| Chunk | `--chunk 8192` (the qualified profile's chunk) |
-| KV dtype | `--cache-type-k q4_0 --cache-type-v q4_0` (the qualified launch) |
-| Decode | `--ds4-fused-decode` (falls back to hybrid expert placement on this host) |
-| Prefill | `--ds4-prefill sparse` (the batched prefill; `exact` runs at token-wise decode cost) |
-| Routing | `--ds4-expert-top-k 6` (the model default; do NOT reduce on adaptive qtype-105/106 artifacts — measured degenerate output at top-4, DS4.md 2026-08-05) |
-| Draft | none — no DSpark draft is published for V4.1-Flash; plain autoregressive decode |
-| Identity | `--model-name frank-archon`, `--host 0.0.0.0 --port 7770` |
-
-## API surface
-
-OpenAI Chat Completions (streaming + tool metadata), `/v1/models`,
-`/health`, OpenAI Responses, Anthropic Messages, and Open WebUI model
-metadata — the fleet contract is unchanged from frank-medium.
-
-## Cutover procedure
-
-1. Image built and model fetched (gates in `deploy_llm_archon`).
-2. `deploy.sh` → `deploy_llm_archon`: stops and **masks** the radiance
-   unit (`frank-medium`), waits for :7770 to free, installs
-   `dankserver/frank-primary/quadlet/frank-primary.container` (also deleted by
-   `8500600`; recover it from `git show 8500600^`), starts
-   `frank-archon.service`, waits for health (1800 s start period — first
-   start page-ins the dense weights and builds the graphs).
-3. Verify: `/health` 200 is not enough — a real chat completion, a tool
-   call, and an Open WebUI completion.
-4. Retire the radiance unit to the home trash (never `rm`); the MXFP6
-   checkpoint payload did not stay — it is gone from the host (see Rollback).
-
-## Rollback
-
-Unmask and start the radiance unit (`systemctl --user unmask frank-medium &&
-systemctl --user start frank-medium`) and stop/mask frank-archon. The
-`localhost/radiance-vllm:0.13.0-e060895` image is still in dank's rootless
-storage; the `/models/Qwen3.8-27B-PARO-MXFP6` checkpoint is not — Dankserver's
-`/home/dank/models/` holds only `StillDeadcode`, `rad` and `tpl.jinja` as of
-2026-10-18, so this rollback needs a re-fetch of the payload.
-
-## Verification (2026-09-26)
-
-_Filled in after the cutover: health, real completion, tool call, Open WebUI
-completion, measured prefill/decode, `systemctl --user is-enabled/is-active`
-state._
-
-## Dual-R9700 follow-up (2026-09-28 probe era)
-
-After this single-card runbook was retired, the DS41 engine got a dual-R9700
-attempt on the `feat/ds41-engine` branch (`main` rejects the checkpoint with
-`unsupported model architecture 'deepseek41'`). Findings that outlive the
-probe:
-
-**Budget both cards explicitly, or you waste half the box.** The engine
-sizes the streamed-expert cache LAST, after the model, so uncapped it absorbs
-every spare byte on the stream device — symptom: card1 at 33.0/34.2 GB
-looking busy while card0 sits at 14.2/34.2 GB; the busy card is holding a
-cache of things still streaming from NVMe. Pin all four:
-
-| env | card | meaning |
-|---|---|---|
-| `LUCE_EXPERT_BUDGET_MB` | hip:0 | primary hot experts (~11.96 MiB each) |
-| `LUCE_EXPERT_SECONDARY_BUDGET_MB` | hip:1 | secondary experts (needs `LUCE_DS4_MOE_TP*`) |
-| `LUCE_EXPERT_STREAM_CACHE_MB` | hip:1 | cap the auto-sized stream cache |
-| `LUCE_EXPERT_STREAM_DEVICE=1` | hip:1 | which card holds the slots and computes them |
-
-Working full-card shape: 21000 + 20000 + 8000 → card0 ~28.6/31.86 GiB,
-card1 ~27.3/31.86 GiB, resident experts 3,360 of 15,360.
-
-**Expert TP DOES work on gfx1201+gfx1201.** A retired quadlet recorded
-`LUCE_DS4_MOE_TP_INPROC` as "qualified for gfx1151+gfx1201, not
-gfx1201+gfx1201" — that is WRONG; peer access negotiates and the secondary
-tier comes up clean: `[deepseek4-moe-tp] enabled mode=in-process local=hip:0
-secondary=hip:1`. It is the only lever that materially raises resident
-expert count, and resident count is the entire bottleneck. Confirm every run
-with `expert ownership:` and `hybrid experts ready:` lines.
-
-**The placement file must live under the mounted models dir.** JSON
-`{"owners": [...]}`, 15,360 entries indexed `layer*384+expert`, values
-`primary`/`secondary`/`stream`. It is a ranking, not a promise: each device
-keeps what fits its budget and demotes the rest. Anywhere outside
-`/models/...` gives `cannot open expert placement` and a SILENT single-tier
-fallback — a silent-wrongness failure, not a crash.
-
-**V4.1 refuses layer split and layer-major prefill** — only `--ds4-prefill
-exact` (token-by-token) is legal on a 32 GiB card and `--ds4-fused-decode`
-is refused there. With ~22% of experts resident each token pulls ~187 cold
-experts × ~12 MiB; if `/proc/<pid>/io` `read_bytes` climbs into terabytes
-the model is larger than the box's RAM and you are in page-cache thrash at
-the NVMe ceiling — a capacity verdict, not a tuning problem. Low GPU util
-with a pegged link is the EXPECTED signature of PCIe-expert-stream-bound
-decode, not a broken run.
-
-**Harness traps, each of which produced a wrong number:** never `env -i` the
-podman call (rootless podman runs as `dank`; `env -i HOME=/root` makes
-PODMAN try `/root/.config` and die `stat /root/.config: permission denied`
-before the container exists — pass container env with `-e`); never put a
-large prompt on curl's argv (`Argument list too long` at ~128 KB — use
-`--data-binary @file`); size prompts by growing the file until `wc -c`
-reaches the byte target AND assert depth against the engine's own
-`usage.prompt_tokens` (nominal 4 chars/token was really 5.73; a chunk loop
-that appends a fixed ~152-byte line per nominal 200-byte chunk undershot
-30%); the server runs as the container's PID 1, so tear down with
-`podman rm -f`, never host `pkill`; drop page cache between arms
-(`sync; echo 3 > /proc/sys/vm/drop_caches`) or the first arm reads warm and
-every later one is a lie.
-
-Read the placement log — it is the whole diagnosis
-(`arch=deepseek41 layers=40 embd=5120 experts=384 used=6 ff=2304
-hash_layers=0` — hash_layers=0 is correct for V4.1; `engram: layers=[1,14]
-… 2 table(s) embedded (not mapped)` — the 189.13 GiB Engram tables are never
-mapped, so they do not evict the expert page cache).

serverAppConfigs-docs/LLM/frank-archon-r9v-radiance-qwen38-q8.md

@@ -1,296 +0,0 @@
----
-title: frank-archon — R9V radiance-vLLM Qwen3.8-Flash-Next Q8_0
-service: frank-archon
-host: dankserver
-port: 7770
-status: retired
-deployed: 2026-09-27
-supersedes: "[[frank-archon-r9v-radiance-qwen38-uncensored]]"
-tags: [llm, frank-archon, r9v, vllm, radiance, gfx1201, q8]
----
-
-# frank-archon — R9V radiance-vLLM Q8_0 (dankserver, RETIRED 2026-09-27 → 09-30)
-
-The Dankserver LLM endpoint **was** the **R9V radiance-vLLM fork** (retired
-2026-09-30, when [[frank-archon-glm53-flash-w4a16]] took the port; the engine
-live today, 2026-10-18, is radiance with
-[[frank-archon-radiance-qwen38-iq4r]])
-(`drwolfen/radiance-vllm-qwen4exp`, docker-pinned prebuilt image bundle
-`2dac17a2`) serving the **unsloth Qwen3.8-Flash-Next-GGUF Q8_0** payload
-(175.30 GiB, 6 GGUF shards, sha256-verified per part) with the blockfp8
-MTP4 head and the mmproj, on **both R9700s** (gfx1201, BDFs `0000:03:00.0`
-/ `0000:07:00.0`) as `frank-primary` on **:7770** (the `frank-archon` wire alias
-arrived with the 2026-10-09 rename).
-
-Cut over from the uncensored IQ4_XS payload on 2026-09-27 at Frank's
-direction. Rollback target at the time:
-[[frank-archon-r9v-radiance-qwen38-uncensored]] — its state dir
-(`/home/dank/r9v-state/uncensored/`, with `setup.json` and
-`restore-config-20260927.env`) is still on the host as of 2026-10-18, but the
-IQ4_XS payload itself is not (`/home/dank/models/` holds only `StillDeadcode`,
-`rad`, `tpl.jinja`), so that flip now needs a re-fetch.
-
-## Why the swap
-
-The uncensored IQ4_XS payload measured 59–96 tok/s decode with
-draft-acceptance swings; the Q8_0 payload trades VRAM-resident expert
-headroom for precision — quantization to Q8_0 doubles expert bytes per
-block, so the tiered-expert residency budget halves — the probe-era
-11 tok/s decode was residency-bound (expert streaming, not compute).
-The "Known costs" section below carries the accepted trade.
-
-## Production shape (repo-tracked)
-
-The repo carries a **thin** wiring; the engine shape lives on-host:
-
-| Piece | File | Law |
-|---|---|---|
-| Unit | `scripts/frank-archon.service` | Root unit, `User=dank`, `LimitMEMLOCK=infinity`, `ExecStop=podman stop -t 120 frank-archon-r9v`, `Restart=on-failure` |
-| Env | `scripts/frank-archon.env` | Production overrides ONLY: bind 0.0.0.0, :7770, alias `frank-archon`, container `frank-archon-r9v`, GPU BDFs, dev-overlay adapter, `R9V_FULL_MUTABLE_CACHE=0`, `R9V_PLE_READY_TIMEOUT=1800`, prefix caching ON, sharp chat template |
-| Serve | `scripts/serve-frank-archon.sh` | Layers setup state under repo env → execs the fork's own `scripts/launch.sh` via the docker→podman shim → `docker wait` |
-| Deployer | `scripts/deploy-remote.sh` `deploy_llm_archon` | Gates on setup state ready, image digest, PLE bytes, all six shard files + mmproj; MainPID false-green check |
-
-**Why the tuned shape lives on-host:** the q8 profile's setup run
-qualified the `R9V_*` config (tiered-expert placement, per-rank offload
-budgets, 80-slot tiered cache, PLE **pinned** residency 54.4 GB, MTP4,
-parser wiring, `MAX_MODEL_LEN=131072`). It is written to
-`/home/dank/r9v-state/q8xl/setup.json` and the serve script layers it
-UNDER the repo env (repo keys win). Copying those values into the repo
-would drift from the qualification — don't.
-
-## Spec decode: MTP4 only — the ngram-mod ask is NOT expressible
-
-llama.cpp chains drafter types (`--spec-type draft-mtp,ngram-mod` on
-frank-zealot/large). The fork's `speculative_config` is a **single dict**
-(one method per engine): `mtp`, `ngram`, `ngram_gpu`, `dflash`, … are
-mutually exclusive — no chaining seam exists in the fork. What IS active:
-
-- **MTP4** (`R9V_MTP_SPEC_TOKENS=4`, blockfp8 draft head, draft TP=2) —
-  measured mean acceptance length 3.19–5.00 during qualification.
-- **The PLE n-gram** — architectural (`ngram_embedding` module), rides
-  along automatically; it is not the llama.cpp `ngram-mod` matcher and
-  cannot be "added" as a second spec method.
-- `ngram_gpu` exists as an ALTERNATIVE single method (fork's GPU n-gram
-  proposer) — swapping MTP4 for it is an either/or, not a chain.
-
-## Sharp chat template (fleet parity with frank-dragoon)
-
-`scripts/qwen-sharp-v22.4.0.jinja` is served via the launcher's opt-in
-`R9V_CHAT_TEMPLATE_FILE` passthrough (fork commit `30d4a32`): the file is
-mounted read-only at `/opt/r9v/share/chat-template.jinja` and passed as
-`--chat-template`, which wins over the tokenizer dir's
-`metadata/chat_template.jinja`; `R9V_CHAT_TEMPLATE_KWARGS` carries
-`--default-chat-template-kwargs {"preserve_thinking":true}` (the template
-defaults `preserve_thinking` true natively; request-level kwargs still win
-per request). The template folds reasoning efforts (high/xhigh/max →
-xhigh, minimal/low → low), matching `models.yml`'s `reasoningEffortMap`.
-The sha-pinned `metadata/chat_template.jinja` in the package is NEVER
-edited in place — mutating it drifts the qualification.
-
-## Two q8-only runtime laws
-
-1. **Quantized inject shard** — the Q8_0 payload quantizes BOTH fused
-   hyper-connection shards (the IQ4_XS package ships inject as F32), so
-   the shard-uniformity check rejects the fused projection at load.
-   `R9V_DEV_QWEN4_EXP_ADAPTER_PY` (fork commit `363d606`) mounts a
-   dev-overlay adapter that dequantizes a quantized inject shard
-   in-stream; uncensored payloads pass through unchanged.
-2. **`R9V_FULL_MUTABLE_CACHE=0`** — the runtime overlay's `env.always`
-   injects the uncensored 160-slot full-mutable cache geometry, which the
-   q8 profile's 80-slot tiered cache fails with a baseline mismatch. The
-   launcher passes this key AFTER the overlay envs (docker last-wins).
-
-## Fork checkout + shim (on-host, outside the repo)
-
-- Checkout: `/home/dank/GitPackages/radiance-vllm-qwen4exp` (fork commits
-  `363d606` dev-overlay, `2fe6dbb`, `50a79ec` qualification doc,
-  `30d4a32` chat-template passthrough, `2de0357` PLE-timeout forwarding —
-  local-only, remote is GitHub)
-- State: `/home/dank/r9v-state/q8xl/setup.json` (0600, `ready: true`)
-- Model dir: `/home/dank/models/Qwen3.8-Flash-Next-Q8_0-R9V/` — 175.30 GiB
-  payload as six `Q8_0/Qwen3.8-Flash-Next-Q8_0-0000N-of-00006.gguf` shards
-  + mmproj; PLE derived at the pinned size 54,400,261,120 B
-- Docker→podman shim: `/home/dank/GitPackages/radiance-vllm-qwen4exp/.podman-shim/docker`
-  (PATH-prepended by the serve script). The R9V tooling is docker-pinned;
-  the shim carries three laws:
-  1. **Image ID law** — the bundle is a containerd/OCI docker-save; docker
-     keys the image by its OCI MANIFEST digest (`sha256:2dac17a2…`), podman
-     by config digest but preserves the manifest digest in **RepoDigests**.
-     The shim answers `image inspect sha256:X` from RepoDigests and rewrites
-     bare `sha256:X` run refs to `repo@sha256:X`.
-  2. **LogConfig law** — podman inspect answers
-     `HostConfig.LogConfig.Config: null` where docker returns the log-opts
-     map (the doctor crashes on `.get("Config", {}) == null`). The shim
-     records `run --log-opt` pairs and normalizes inspect output.
-  3. **docker info law** — podman lacks `SecurityOptions`/`ServerVersion`;
-     the shim answers rootless + `27.5.1` (liveness probe).
-- venv: `~/.r9v-venv` (huggingface_hub for fetch/verify)
-
-## Bring-up battery (qualification, 2026-09-27)
-
-- Setup `ready: true`; all 20 payload artifacts size+sha256 verified
-  (175.30 GiB total across the six shards).
-- Vision: 64×64 red PNG → "Red" (79 tok, 7.4 s end-to-end on the scratch
-  port). Prose: 100 tok in 9.1 s ≈ 11.0 tok/s wall **on a cold cache with
-  reasoning tokens included in the wall clock** — decode-rate context, not
-  a bench (no throughput bench was run, per Frank's direction).
-- MTP4: mean acceptance length 3.19–5.00 across the battery.
-- PLE pinned proof: `mapping_bytes = mapping_rss_bytes =
-  54,400,261,120` (fully RSS-resident); doubling for re-pin would need
-  108,800,522,240 ≤ available 140,876,808,192; registration took 446 s.
-
-## Memory shape (measured)
-
-- Load: 21.84 GiB / worker at ~158 s; cold start → serve ~16 min.
-- PLE pinned residency 54.4 GB in host RAM (vs the uncensored payload's
-  ssd-mmap residency).
-- Context envelope: `R9V_MAX_MODEL_LEN=131072`; harness `models.yml`
-  carries `contextWindow: 131072`, `maxTokens: 36864` (compaction-gap
-  math: 70% × 131,072 = 91,750 + 36,864 = 128,614 ≤ 131,072). The KV
-  envelope at 131,072 is smaller than the uncensored 196,608 shape.
-- Prefix caching OFF at cutover (`R9V_ENABLE_PREFIX_CACHING=0`), CACHE192
-  off — superseded 2026-09-28: prefix caching is ON (see bench matrix
-  below).
-
-## Prefill / decode bench matrix (2026-09-28, scratch port 7773)
-
-Production stopped for the window (fleet bench law); one matrix plus three
-follow-up runs, fork's own bench-pp + decode_speed harness, corpus-sliced
-prompts with the fork's canonical decode rounds. Baseline E0 = production
-shape (chunk 4096, 80-slot sync cache, MTP4):
-
-| Config | Override | PP8K tok/s | PP131K tok/s | Decode | Verdict |
-|---|---|---|---|---|---|
-| E0 | — | 954.6 | 883.0 | 141.9 ms/step ≈ 23.8 t/s eff | baseline |
-| E1R | chunk 8192 | **1201.4 (+25.9%)** | 844.6 (−4.4%) | 158.5 ms (−8.5% t/s) | short-prompt win, decode tax |
-| E2 | chunk 16384 | — | — | — | boots; 33.3/34.2 GiB VRAM (0.93 free) → requests hang/OOM |
-| E1/E3 | chunk 8192/32768 | — | — | — | PLE registration blew the 600 s engine timeout (see law below) |
-| E4 | cache async=1 | — | — | — | doctor: cache-prepare backend requires synchronous single-fill LRU |
-| E5/E6 | 96/112 slots | — | — | — | doctor: capacities (48, 64, 80) only; expert-budget [108, 198] rank maxima — **80 is the policy max, prod is at it** |
-| E7 | MTP depth 6 | 843.1 | 819.6 | 454–489 ms/step (3.7x worse) | depth 4 stays |
-
-Laws from the matrix:
-
-- **4096 is the prefill-chunk ceiling twice over**: the engine caps
-  scheduled tokens by the speculative config, and E2's VRAM (0.93 GiB free
-  at 16384) shows bigger chunks OOM at request time. 883–955 tok/s is what
-  this payload shape delivers at long context.
-- **Unqualified shapes fail mid-request, not at health** — E2 and E7 both
-  answered /health 200 and then died streaming.
-- **PLE registration can exceed the 600 s engine timeout**
-  (`VLLM_PLE_OFFLOAD_READY_TIMEOUT`) under page-cache churn — measured 740 s
-  after a bench teardown; a production boot failed on it the same morning
-  (unit "Deactivated successfully", no error in journal — the serve script
-  exits 0 on the wait). Fixed with `R9V_PLE_READY_TIMEOUT=1800` in
-  `frank-archon.env` (forwarded by launch.sh `2de0357`).
-
-Prefix caching (E8/E9, `R9V_ENABLE_PREFIX_CACHING=1`, same port): **zero
-VRAM cost** (hits reuse freed KV blocks; load VRAM identical to E0),
-cold path unchanged (first request ≈ cold bench), reuse hits large — 32k
-prompt TTFT 44.2 s → **3.6–4.3 s** (9x), 131k 439 s → **6.9 s** (63x).
-Hit pattern: rounds 1–2 miss, round 3+ hit (identical-prompt and
-conversation-growth probes both). For the harness's resend-every-turn
-traffic that is a ~40–440 s TTFT win per steady-state turn; the one-turn
-miss costs nothing extra. **DEPLOYED 2026-09-28** (Frank's go-ahead): the
-repo env sets `R9V_ENABLE_PREFIX_CACHING=1` (commit `41e9189`), verified
-on :7770 (`--enable-prefix-caching` in the container argv, health 200,
-live completion answering).
-
-## Authoring a new profile (the qualification pipeline)
-
-Order matters at every step:
-
-1. **Package manifest** `packages/models/<pkg>/package.json` — target shards
-   + aux artifacts with sha256. Verify every target's byte count AND sha256
-   after fetch (`curl -fL -C -` per shard; never trust a fetch without `-f`).
-2. **Placement bootstrap** from GGUF headers: parse all shards' tensor
-   tables (a KV-only shard1 is normal for split parts; each part carries its
-   own tensor table). Per-expert full per layer =
-   (`ffn_gate_exps` + `ffn_up_exps` + `ffn_down_exps`)/512; rank slices
-   rank0 = full×416//640, rank1 = full−rank0. Hot counts =
-   floor(measured_budget / per-expert total). **Anchor budgets to the
-   MEASURED hot_bytes of the qualified sibling profile, never invented.**
-3. **Trim + validate**: `tools/trim_experts.py --hot-counts N,M`, then
-   `expert_budget.validate_manifest(m)` +
-   `expert_budget.expert_memory(m, slots, {cache_rank}, runtime=<runtime.json>)`
-   (runtime/catalog are keyword-only). Declared hot_bytes must match
-   computed or it raises.
-4. **PLE derive** (containerized `tools/prepare_ple.py` with
-   `--expected-bytes/--expected-type/--expected-shape`); setup skips
-   re-derivation when the file exists with matching size.
-5. **profile.env**: set values BEFORE `source dual-r9700/profile.env`
-   (set-first-wins; an empty string does NOT survive `:=`).
-
-The canonical bootstrap.json wrapper schema is exactly
-`{schema: "r9v.placement.v1", id, model_package, hardware, artifact_path,
-artifact_sha256, scope}` with `hardware: "dual-r9700-32gb-asymmetric"`; the
-wrapper `id` MUST equal the id in the profile.json placement descriptor
-(setup rejects mismatches: "does not match profile placement"). Never
-fabricate qualification metadata — use
-`scope: "Unranked routing bootstrap only; does not certify fit or headroom"`.
-
-GGUF parsing laws for the bootstrap: expert tensor names are
-`blk.0.ffn_up_exps.weight` — split on "." gives FOUR parts, layer number is
-parts[1] (a 3-part assumption crashes `int("")`); outer-KV value-type sizes
-`{0:1,1:1,2:2,3:2,4:4,5:4,6:4,7:1,8:str,9:arr,10:8,11:8,12:8}`; tensor byte
-extents from consecutive OFFSETS, never a type-size table (mis-buckets tails
-silently); the PLE Q8_0 tensor on this arch is shape (160, 320001536),
-54,400,261,120 B, an entire shard.
-
-Setup/start: `./r9v setup <alias> --state-dir <dir> --model-dir <dir> --image
-sha256:<digest> --local-image --reuse-from <sibling-model-dir>
---accept-model-license` (aux artifacts reuse byte-identical files; F16 vs
-Q8 mmproj will not reuse). Start requires the SAME `--state-dir`; fixed
-placement via `R9V_EXPERT_MANIFEST_PATH` skips headroom/catalog/seed raises.
-The host-only doctor FAILs when another instance holds the cards (it wants
-3 GiB free per rank; production at ~30 GiB/card is a hard wall) — a second
-instance CANNOT coexist, so qualification requires a stop+mask window for
-the incumbent, then unmask+restart.
-
-**Qualification gate (unqualified → qualified):** first start runs the
-workload battery (PONG / tool_calls / vision / long-ctx), the PLE pinning
-proof (`VmLck` ≈ PLE bytes in `/proc/<container-pid>/status` when
-RESIDENCY_MODE=pinned), and a VRAM headroom snapshot (floor 3.3 GiB). If
-headroom is violated, lower hot counts via re-trim and re-derive
-MAX_EFFECTIVE. Only then record state `ready: true` and author the runbook.
-
-Remote-execution gotchas from the same windows: `nohup X &` inside an ssh
-command races ssh teardown — the script can die before writing a byte; run
-long remote scripts synchronously, or setsid + disown, or via a system
-unit. zsh login shells abort on unquoted `===` separators; grep globs need
-quoting through ssh+zsh layers.
-
-## Ops
-
-- **Restart**: `sudo systemctl restart frank-archon` (root unit; the serve
-  script re-gates everything, relaunch through the shim, waits).
-- **First start after boot** re-warms ~175 GiB of page cache; expect
-  minutes to /health; `TimeoutStartSec=1800`.
-- **Scratch/probe**: NEVER probe against the prod state dir. Copy it, set
-  `R9V_HOST_PORT` to a free port and `R9V_CONTAINER_NAME` to a scratch
-  name, and stop the probe before any deploy — a probe holding :7770
-  produces a false-green deploy (the 2026-09-26 lesson, guarded by the
-  MainPID check).
-- **Rollback to uncensored**: point
-  `scripts/frank-archon.env` `R9V_STATE_FILE` back at
-  `/home/dank/r9v-state/uncensored/setup.json`, drop the two q8-only keys
-  (adapter, FULL_MUTABLE_CACHE) and the template keys, redeploy — the
-  uncensored state dir is intact on disk; the IQ4_XS payload is not (it was
-  deleted in the 2026-10-17 cleanup), so this rollback re-fetches first.
-- **Models on the wire**: served alias `frank-primary` (the StarCraft wire
-  names arrived 2026-10-09, `7c77326`); thinking channel is `reasoning` (vLLM);
-  tool parser `qwen3_coder`.
-
-## Known costs (accepted at cutover)
-
-- **Residency-halved experts** — Q8_0 doubles expert bytes vs IQ4_XS, so
-  the tiered cache holds half as many experts in VRAM at the same budget;
-  the probe's 11 tok/s decode was residency-bound, and the production
-  number depends on the re-qualified budgets in the q8 state.
-- **128K envelope** — down from the uncensored 196,608 (which itself was
-  down from 262,144 on llama.cpp).
-- **Reference quant** — back to the unsloth Q8_0 weights (no
-  abliteration); answer-quality profile differs from the uncensored
-  incumbent by design.
-- **Docker-pinned fork** — updates flow through image bundles + the shim,
-  never a bare container run; the launcher stays the single argv owner.

serverAppConfigs-docs/LLM/frank-archon-r9v-radiance-qwen38-uncensored.md

@@ -1,138 +0,0 @@
----
-title: frank-archon — R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS
-service: frank-archon
-host: dankserver
-port: 7770
-status: retired
-retired: 2026-09-27
-superseded_by: "[[frank-archon-r9v-radiance-qwen38-q8]]"
-supersedes: "[[frank-archon-llamacpp-qwen38-flash-next-q8]]"
-tags: [llm, frank-archon, r9v, vllm, radiance, gfx1201, retired]
----
-
-# frank-archon — R9V radiance-vLLM (dankserver, RETIRED)
-
-> **RETIRED 2026-09-27** — superseded the same day by the Q8_0 cutover,
-> [[frank-archon-r9v-radiance-qwen38-q8]] (itself retired 2026-09-30; the live
-> frank-archon runbook today, 2026-10-18, is
-> [[frank-archon-radiance-qwen38-iq4r]]). The uncensored state dir and frozen
-> config are still on disk (`/home/dank/r9v-state/uncensored/` — `setup.json`,
-> `restore-config-20260927.env`); the IQ4_XS **payload is not** — it was deleted
-> in the 2026-10-17 cleanup. The rollback procedure lives in the Q8_0 page's Ops
-> section, with a re-fetch as step one.
-
-The Dankserver LLM endpoint **was** the **R9V radiance-vLLM fork**
-(`drwolfen/radiance-vllm-qwen4exp`, docker-pinned prebuilt) serving
-`Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS` — the abliterated
-Qwen3.8-Flash-Next at IQ4_XS (92.39 GiB, 3 GGUF shards) with the blockfp8
-MTP4 head and the F16 mmproj — on **both R9700s** (gfx1201, BDFs
-`0000:03:00.0` / `0000:07:00.0`) as `frank-primary` on **:7770** (the
-`frank-archon` wire alias arrived 2026-10-09).
-
-Replaced llama.cpp b11030 (Qwen3.8-Flash-Next Q_8) on 2026-09-27 at Frank's
-direction; downtime window ~1 h 20 min (killed 02:47, healthy 04:11 PDT).
-Rollback target: [[frank-archon-llamacpp-qwen38-flash-next-q8]] — the
-llama.cpp trio (unit/env/serve) is at commit `4992e5a`+ (the repo-side
-wiring; host unit was the relinked one); the GGUF payload is no longer on
-disk (Dankserver's `/home/dank/models/` holds only `StillDeadcode`, `rad`,
-`tpl.jinja` as of 2026-10-18), so that rollback is a re-fetch plus a redeploy.
-
-## Production shape (repo-tracked)
-
-The repo carries a **thin** wiring; the engine shape lives on-host:
-
-| Piece | File | Law |
-|---|---|---|
-| Unit | `scripts/frank-archon.service` | Root unit, `User=dank`, `LimitMEMLOCK=infinity`, `ExecStop=podman stop -t 120 frank-archon-r9v`, `Restart=on-failure` |
-| Env | `scripts/frank-archon.env` | Production overrides ONLY: bind 0.0.0.0, :7770, alias `frank-primary` (the wire id became `frank-archon` on 2026-10-09), container `frank-archon-r9v`, GPU BDFs |
-| Serve | `scripts/serve-frank-archon.sh` | Layers setup state under repo env → execs the fork's own `scripts/launch.sh` via the docker→podman shim → `docker wait` |
-| Deployer | `scripts/deploy-remote.sh` `deploy_llm_archon` | Gates on setup state ready, image digest, PLE bytes, shard files; MainPID false-green check |
-
-**Why the tuned shape lives on-host:** the profile's setup run qualified a
-104-key `R9V_*` config (tiered-expert placement, per-rank offload budgets
-146.25/78.75 GiB, CED on, MTP4, parser wiring, `MAX_MODEL_LEN=196608`). It
-is written to `/home/dank/r9v-state/uncensored/setup.json` and the serve
-script layers it UNDER the repo env (repo keys win). Copying those values
-into the repo would drift from the qualification — don't.
-
-**Self-healing law:** the launcher `--detach`es; the serve script therefore
-blocks on `docker wait`. Without the wait, a crashed engine is a green
-unit. `docker wait` cannot stop the container child, hence `ExecStop`.
-
-## Fork checkout + shim (on-host, outside the repo)
-
-- Checkout: `/home/dank/GitPackages/radiance-vllm-qwen4exp`
-- State: `/home/dank/r9v-state/uncensored/setup.json` (0600, `ready: true`)
-- Model dir: `/home/dank/models/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS/`
-  (92.39 GiB payload + `r9v-data/per_layer_token_embd.iq4_nl.bin` 26.82 GiB
-  PLE, sha256 `34fa36f8…`)
-- Docker→podman shim: `/home/dank/GitPackages/radiance-vllm-qwen4exp/.podman-shim/docker`
-  (PATH-prepended by the serve script). The R9V tooling is docker-pinned;
-  the shim carries three laws:
-  1. **Image ID law** — the bundle is a containerd/OCI docker-save; docker
-     keys the image by its OCI MANIFEST digest (`sha256:2dac17a2…`), podman
-     by config digest but preserves the manifest digest in **RepoDigests**.
-     The shim answers `image inspect sha256:X` from RepoDigests and rewrites
-     bare `sha256:X` run refs to `repo@sha256:X`.
-  2. **LogConfig law** — podman inspect answers
-     `HostConfig.LogConfig.Config: null` where docker returns the log-opts
-     map (the doctor crashes on `.get("Config", {}) == null`). The shim
-     records `run --log-opt` pairs and normalizes inspect output.
-  3. **docker info law** — podman lacks `SecurityOptions`/`ServerVersion`;
-     the shim answers rootless + `27.5.1` (liveness probe).
-- venv: `~/.r9v-venv` (huggingface_hub for fetch/verify)
-
-## Verified at cutover (2026-09-27)
-
-- Setup: **PASS=28 WARN=3 FAIL=0** — 20/20 artifacts size+sha256 verified
-  (~93 GiB), PLE derived at exact pinned size, CED projector verified.
-- First-start doctor battery: **PASS=34 WARN=8 FAIL=0**, including
-  `runtime-worker-transport`: **pinned UVA reads, H2D copies, TP
-  all-reduce verified inside serving workers** on both BDFs.
-- Wire: `/v1/models` → `frank-archon`; `/health` + `/metrics` 200
-  (Prometheus scraping); "Say PONG" (non-thinking) → `'PONG'`, stop;
-  thinking requests populate `reasoning` (vLLM channel).
-- Memlock: the full tiered-expert load ran to healthy **under a plain dank
-  session** before the unit existed — the pin path does NOT hard-require
-  `LimitMEMLOCK=infinity` on this host, but the root unit keeps fleet
-  parity and headroom for the pinned reserve.
-- e2e gate: 32 passed / 1 failed — the failure is `11-admin open links`,
-  which needs FrankBook-Pro (192.168.0.112) awake; the machine was down and
-  WOL from both hosts failed. Suite minus that spec: **31 passed** (the
-  one-time 12-crawl flake re-passes in isolation).
-
-## Memory shape (measured)
-
-- Both cards ~29–30 GiB used of 31.86; engine budget
-  `CPU_OFFLOAD_GB_BY_DEVICE=146.25,78.75` (UVA host pin ≈ 43 GiB on rank 0
-  + 16 GiB PLE pinned reserve; PLE residency mode `ssd` mmap+trim).
-- `MemAvailable` floor: 56.3 GiB (`R9V_MIN_HOST_AVAILABLE_BYTES`).
-- Context envelope: 192,608 native window served as `MAX_MODEL_LEN=196608`;
-  harness `models.yml` carries `contextWindow: 196608`, `maxTokens: 57344`
-  (compaction-gap math: 70% × 196,608 = 137,625 + 57,344 ≤ 196,608).
-
-## Ops
-
-- **Restart**: `sudo systemctl restart frank-archon` (root unit; the serve
-  script re-gates everything, relaunch through the shim, waits).
-- **First start after boot** re-warms ~93 GiB of page cache; expect several
-  minutes to /health; `TimeoutStartSec=1800`.
-- **Scratch/probe**: NEVER probe against the prod state dir. Copy it, set
-  `R9V_HOST_PORT` to a free port and `R9V_CONTAINER_NAME` to a scratch
-  name, and stop the probe before any deploy — a probe holding :7770
-  produces a false-green deploy (the 2026-09-26 lesson, now guarded by the
-  MainPID check).
-- **Qualification artifacts** (kept on-host):
-  `/home/dank/r9v-state/uncensored/qualification-*`,
-  failure bundles `failure-*` (review before sharing — raw logs).
-- **Models on the wire**: served alias `frank-archon`; thinking channel is
-  `reasoning` (vLLM), tool parser `qwen3_coder` (not qwen3_xml like the
-  vllm030 wiring had planned).
-
-## Known costs (accepted at selection)
-
-- **Abliterated weights** (Dyluhn uncensored build) — not the reference
-  model; answer-quality deltas are possible.
-- **192K envelope** — down from 262,144 on llama.cpp.
-- **Docker-pinned fork** — updates flow through image bundles + the shim,
-  never a bare container run; the launcher stays the single argv owner.

serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-iq4r.md

@@ -1,154 +0,0 @@
-# frank-archon — Radiance (StillDeadcode), publisher Qwen3.8-Flash-Next iq4r, 2× R9700 TP=2
-
-**Status: LIVE — deployed 2026-10-15.** · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
-
-The in-house w4dn8 conversion of the Swift1.5 finetune
-([[frank-archon-radiance-swift15-w4dn8]]) is replaced by the **engine
-author's published container**:
-[`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`](https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe)
-— the official `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint rad-converted by
-StillDeadcode. Same port, alias, container name, systemd unit, digest-pinned
-engine pin **1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…`), and
-**every serving argument carried verbatim** from the 2026-10-13 shape.
-Rollback: re-download the w4dn8 `.rad` from the HF mirror + point
-`MODEL_FILE` at it (the local copy was purged 2026-10-17, see Rollback).
-
-## The payload
-
-`qwen3.8-next-flash-fp8-iq4r-moe.rad` — **121,969,901,568 B = 116,322 MiB**,
-sha256 `0af5e96244e80c21ac8edca1719dae49b8a201b94a2f3994c3e24026ceaa4d20`
-(Hub LFS oid), fetched byte-exact by `scripts/download-frank-archon-iq4r.sh`
-(hfget resume + size + sha256; staging dir
-`/home/dank/models/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe/` keeps the
-card, LICENSE, and recipe `qwen4exp-w4nl64-i8-hc8m` as provenance).
-
-| Plane | Encoding (publisher recipe `qwen4exp-w4nl64-i8-hc8m`) |
-| --- | --- |
-| Routed experts | 4-bit codes into a 16-level **non-uniform** codebook after a 128-point Walsh–Hadamard rotation, one E4M3 scale per 64 weights; codes by GPTQ (10M-token calibration, per-expert down-proj Hessians) + 3 coordinate-descent sweeps |
-| Protected experts | **ten** per layer, whole-expert bf16 (down-projection energy) |
-| Trunk + lm_head | int8 W8A8, scale per 128 columns searched; HC mixtures E4M3 |
-| PLE n-gram | fp8 fixed-scale, in-container (served `--ngram-placement ram`, same 47.7 GiB pin law) |
-| Speculator | the model's own MTP head (card default depth 3; served at 2) |
-| Vision | bf16 vision tower, in-container |
-
-**Base model: the OFFICIAL (censored) Qwen3.8-Flash-Next** — the fleet's
-only frontier model at base alignment; the abliterated family lives on
-frank-dragoon. Card quality: full-vocab teacher-forced KL vs bf16 over
-~75K positions — mean 0.0789 all-tokens / 0.0239 assistant turns, top-1
-agreement 92.85 %.
-
-## Why this swap
-
-- **Publisher-graded precision**: the w4nl codebook + coordinate-descent
-  refinement is the engine author's own quality ceiling for this
-  checkpoint; the card's KL ladder (p99 1.65, p99.9 5.33) is published and
-  reproducible, where our w4dn8 gate measured against our own reference.
-- **Formal parity**: the served file is the one the engine author supports
-  — decode/prefill behavior on 1.0.13 is their tested path (the card's run
-  line speaks the 1.0.13-era flags; the `--vram-weights-mib`/`--vram-kv-mib`
-  pair we ship is accepted by pin 1.0.13 per the 2026-10-06 A/B, so the
-  shape needed no translation).
-- **Cost**: the Swift1.5 finetune's refusal-bending behavior leaves this
-  endpoint (it returns to the official base alignment); the 4-slot/200k
-  economics and all shape laws carry over unchanged.
-
-## Shape (unchanged from 2026-10-13)
-
-All of it, verbatim — the payload moved 116,416 → 116,322 MiB, inside the
-same budget:
-
-- `--tp 2 --tp-wire wht6`, `--max-num-seqs 3`, `--max-model-len 200000`
-  (capacity contract 3 × 200,000), `--kv-cache-dtype fp8`
-- `--placement expert_tiered --host-pool-mib 44928` + explicit
-  `--vram-weights-mib 25600 --vram-kv-mib 5120` (30,720 of the engine's
-  30,969 MiB ceiling; pool = contract + 1.3 %)
-- `--num-speculative-tokens 2`, `--max-num-batched-tokens 2048`
-  (MNBT 2048 law), prefix cache 8 GiB host + 64 GiB disk,
-  `--ngram-placement ram`, froggeric v22.5 `--override-chat-template`
-- TTM preflight floor 150,000 / target 204,800 MiB; pinned ledger
-  2 × 44 GiB pool + 47.7 GiB PLE + 8 GiB prefix ≈ 147 GiB (see
-  [[frank-archon-radiance-swift15-w4dn8]] for the ledger arithmetic and
-  the KV dtype law — both are engine properties, payload-independent)
-
-The publisher's card run line (`--host-pool-mib 12288`,
-`--expert-vs-cache-ratio 0.82`, seqs 8, spec 3, ~32 GB host RAM) is their
-shape on stock 32 GB cards; the fleet ships its own measured shape against
-the same file.
-
-## Cutover record — 2026-10-15
-
-1. `rm` (Frank's one-time authorization): the raw
-   `ukisai/Swift1.5-Qwen3.8-Flash-Next` BF16 checkpoint (336 GiB, spent
-   conversion source) and `qwen4exp-fp8-native.rad` (173 GiB, two eras
-   back) left `/home/dank/models` — 150 GB free → 658 GB free. The fp8-native
-   payload is **no longer on disk**: rebuild =
-   `scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh`.
-2. Fetch: `scripts/download-frank-archon-iq4r.sh --fetch` as transient unit
-   `iq4r-fetch` (detached-job protocol; log
-   `/home/dank/logs/iq4r-fetch.log`, rc sentinel `…rc`).
-3. Deploy: `scripts/deploy.sh dankserver --service frank-archon` — launched
-   detached (the swap restarts the very endpoint the operator session runs
-   on; the harness timeout catches the session on return).
-4. **Boot ledger (2026-10-15, first boot, pin 1.0.13 — the w4nl codebook
-   loaded, no pin bump)**: `kv_attn` fp8 page **26.00 KiB** at **607,740
-   tokens** (151,935 blocks = 3.77 GiB) — byte-identical to the w4dn8 boot,
-   the sizing law carries; `kv_qsa_bkey` 3.25 KiB at the same token count;
-   `kv_gdn_state` 54 MiB × 6 slots, `kv_gdn_conv` 1.76 MiB × 3,
-   `kv_qsa_tail` 19.5 KiB × 6; pool **5.00 GiB**, prefix checkpoints 8 ×
-   56 MiB; the sizing line reads `kv cache 5.00 GiB … backs 3 sequences all
-   at 200000 tokens at once, which is the worst case`; samplers print **9
-   rows** per rank (3 × (1+2), inside the wht6 25-row bound);
-   `speculation: 'mtp' drafts 2 token(s) a step in 2 serial pass(es)` +
-   prompt-lookup fallback; slab **17,610 (rank0) / 17,658 (rank1) slots**
-   (64 shadow) — above the w4dn8 record of 17,548 (ten bf16-protected
-   experts per layer replace twenty-four, so more of the 4-bit table
-   resides), pinned pool 7,532/7,484 slots; KV lending **4.22 GiB**
-   (3,571/3,568 mover slots); arenas 896.02/988.17 MiB charged before the
-   pair split; prefix cache RECOVERED 900,176 blocks + 126 checkpoint slots
-   from the 64 GiB disk tier (same arch + tokenizer as w4dn8 — the tier
-   survived the payload swap); froggeric template 28,234 B; vision encoder
-   live (≤ 4,194,304 px, 16,384 patches/pass). Health gate passed in ~100 s
-   (radiance binds the port after the full load).
-5. **Vision probe (2026-10-15)**: 300 px solid-red PNG via `image_url` —
-   reasoning stream: "a solid fill of a deep, saturated red hue". Content
-   came back EMPTY at `max_tokens 64`: the thinking budget consumed all 64
-   (`reasoning_tokens: 64`, finish `length`) — fleet law, probe thinking
-   models with ≥ a few hundred tokens.
-6. **e2e gate**: deploy exited DEPLOY INCOMPLETE — 30/33 Playwright green,
-   3 red (`12-crawl`, `13-sso` search SSO, `16-marreta`) on DANKCLAW
-   surfaces (vhosts intermittently returned `ERR_CONNECTION_RESET` from the
-   workstation); frank-archon's own health gate passed. The swap touched no
-   dankclaw runtime file (only homepage services.json text). Census of the
-   Caddy/auth stack pending; see `log.md`.
-7. **Post-boot measurements (pending)**: decode ladder (99.0 t/s single
-   @ 43.5k, 93.3 @ 197k, 3-way 113.9/43.3/10.9 = 18.4 t/s aggregate —
-   these are the **w4dn8** numbers, the record to beat, not this file's),
-   slab-limited prefill (22208/3,397 → 25600/3,584 t/s A-cold 43.5k), draft
-   acceptance, and `scripts/probe-frank-archon-kv.py` leg D.
-
-## Rollback
-
-Revert the swap commit (`scripts/frank-archon.env` MODEL_FILE line) +
-redeploy: `/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad`
-(116,416 MiB), KLD-gated, same shape — **no longer on disk**. The 2026-10-17
-disk purge (Frank's approval, trash-staged then emptied: 763 GiB free after)
-removed it; the byte-exact copy lives on the Hub as
-`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`
-(`qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`)
-and re-fetches through `scripts/download-frank-archon-iq4r.sh`'s
-hfget/verify pattern retargeted at that repo. The same purge deleted the
-`radiance-state` kvcache tiers, `~/.radiance-cache-w4a8-*`,
-`~/.cache/frank-primary`, and `/opt/vllm/moremodels` (GLM-4.7 GGUF) — all
-derived or retired. The fp8-native era is rebuild-only (see the cutover
-record). The w4dn8 runbook keeps every shape law, the KV dtype table, the
-slab ladder, and the tuning levers:
-[[frank-archon-radiance-swift15-w4dn8]].
-
-## Related
-
-- [[frank-archon-radiance-swift15-w4dn8]] — the replaced payload and the
-  full shape-law library (slab slope, KV dtype arithmetic, TTM law)
-- [[frank-archon-radiance-qwen38-native-fp8]] — the fp8-native era
-  (rebuild-only now; owns the descriptor-selection law)
-- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
-- [[deployment]] — the service-scoped deploy this cutover rides

serverAppConfigs-docs/LLM/frank-archon-radiance-qwen38-native-fp8.md

@@ -1,308 +0,0 @@
-# frank-archon — Radiance (StillDeadcode), Qwen3.8-Flash-Next-FP8 NATIVE, 2× R9700 TP=2
-
-**Status: RETIRED 2026-10-05 → 2026-10-08** — replaced by the Swift1.5 w4dn8
-conversion ([[frank-archon-radiance-swift15-w4dn8]]), itself retired 2026-10-15;
-the payload live today (2026-10-18) is the publisher iq4r container,
-[[frank-archon-radiance-qwen38-iq4r]] (`scripts/frank-archon.env:110-111`). ·
-**Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
-
-The FreeToken PR #587 endpoint was replaced by the **radiance engine** (C++/HIP,
-libr4d kernels for gfx1201, no Python in the serving path) driving the
-digest-pinned image `stilldeadcode/radiance@sha256:4c3f484564bb2872784e4…`
-(v1.0.7 line, 287 MiB, `libr4d@0.5.0`) as a plain `podman run` through
-`scripts/serve-frank-archon.sh` under the **root systemd unit**
-(`frank-archon.service`, `User=dank`, `LimitMEMLOCK=infinity`). Same port,
-same served alias (`frank-archon`), same container identity
-(`frank-archon` = the unit name). Rollback: git tag
-`frank-archon-freetoken-pr587-20261005` (FreeToken trio + Dockerfile; the
-Dockerfile lives in the tag as `scripts/docker/frank-primary-freetoken.Dockerfile`
-— `scripts/docker/` is not in the tree today), image
-`localhost/frank-primary-freetoken:pr587-77a7201` (that is the tag on the host —
-the 2026-10-10 rename never retagged it), payload
-`/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8`, which is **no longer on
-disk** (`ls /home/dank/models/` on Dankserver 2026-10-18: `StillDeadcode`, `rad`,
-`tpl.jinja` only).
-
-## Why radiance
-
-- **TP=2 is real here.** FreeToken's TP=2 dies in the `pynccl.cu` ROCm JIT
-  (`cuda_runtime.h` unresolvable for gfx1201 — probed 2026-10-06); the
-  second R9700 idled. Radiance's TP wire is libr4d's own one-shot
-  all-reduce (no RCCL), legal on this board since the 2026-09-20 swap
-  (ProArt B850, x8/x8 on CPU root ports, `Routing+`/`ReqEn+` everywhere).
-- **Native FP8 end to end** (Frank's call): the official
-  `Qwen/Qwen3.8-Flash-Next-FP8` checkpoint keeps its modelopt block-FP8
-  encoding — no requantisation, no calibration run, no 4-bit codebook.
-- **Every tier is explicit and measured**, not an estimator: placement
-  `expert_tiered` with a pinned host pool, `--ngram-placement ram` for the
-  47.7 GiB PLE table, `--expert-vs-cache-ratio 0.82` (the publisher's
-  measured 99 %-hit point; 0.87 already collapses to 0 %), fp8 KV.
-
-## The payload: native FP8, reshaped then converted (one recipe rule)
-
-- **Raw checkpoint**: `/home/dank/models/Qwen/Qwen3.8-Flash-Next-FP8`
-  (ungated repo, 172.8 GB, 131 shards + index + tokenizer + vision
-  configs). Fetched by `scripts/download-frank-archon.sh` (vendored
-  `hfget.sh`, curl+jq, no HF token needed) and **verified byte-for-byte**:
-  all 131 shards sha256-match the Hub LFS oids (2026-10-05, log
-  `/home/dank/logs/verify-qwen38-official-fp8.log`).
-- **The container**: `/home/dank/models/rad/qwen4exp-fp8-native.rad`
-  (177,109 MiB measured = 173.0 GiB; floor
-  `FRANK_ARCHON_RAD_MIN_MIB=177000` in the env gates a
-  truncated convert). Built by `scripts/build-frank-archon-rad.sh`:
-  1. stage `/home/dank/models/stage-qwen4exp-fp8` — `config.json` with
-     **`quantization_config` dropped** + relative symlinks into the raw
-     checkout;
-  2. `stack-qwen38-official-fp8.py` reshapes the staging dir — two
-     transforms, both value-preserving (script docstring carries the full
-     argument): the plugin maps the STACKED expert layout
-     (`mlp.experts.gate_up_proj` `[512,1280,2560]` + `_scale_inv` at the
-     same index) while the official repo ships per-expert unfused tensors,
-     so staging byte-stacks all 49 stacks' (48 trunk + MTP) 512 experts
-     with NO requantisation; and the PLE table ships as 128 **scaleless**
-     E4M3 shards + one bf16 scalar (1.9931793212890625e-04) while the
-     plugin's fp8 branch wants the codes beside a ONE-ELEMENT bf16 scale
-     plane — so staging decodes to a single bf16 `shard_0` with the
-     checkpoint's own scale (e4m3 decode and the scale are both exact in
-     bf16);
-  3. `rad-convert` with ONE `--quant` rule — pure copy for everything else
-     (every weight keeps the checkpoint's block-FP8 planes: `X.weight`
-     E4M3 codes + `X.weight_scale_inv` bf16 `[128×128]` scales) — and the
-     rtn rule re-encodes the bf16 PLE table at the same fixed scale, an
-     exact round-trip (nearest-code distance 2⁻³ vs the bf16 rounding
-     perturbation 2⁻⁹): the container gets the checkpoint's own PLE codes
-     plus the one-element scale plane (`blk.1.ple_ngram.scale`, view {1});
-  4. THREE LAWS the live runs proved (each cost a full verify+declare
-     cycle; all fixed in the two scripts, 2026-10-05/06):
-     - **Scale-name law**: `scale_name()` appends `_scale_inv` to the
-       TENSOR NAME verbatim — a stacked tensor named
-       `…experts.gate_up_proj` (no `.weight` suffix) has its scale plane
-       at `…experts.gate_up_proj_scale_inv`, NOT `…weight_scale_inv`.
-     - **File-shadow law**: the reader opens every index-named file in
-       SORTED order and emplaces EVERY tensor each file physically
-       holds, first-wins (`checkpoint.cpp add_shard`). The raw
-       model-000NN shards carry tensors NAMED exactly what the merged
-       PLE shard is named, so the merged file MUST sort first
-       (`00-ple.merged.safetensors`) or a raw fp8 shard wins the name
-       and the plan reads one shard as the codes.
-     - **Symlink-through law**: staging symlinks the raw index into the
-       stage; `open(path,"w")` writes THROUGH it and corrupts the raw
-       checkpoint (raw verify then fails 143/144). The stack script
-       writes index/config temp+`os.replace` — an atomic swap replaces
-       the symlink itself; the build script COPIES the index and
-       symlinks only the shards.
-  5. `rad-info` summary + size floor.
-- **Why the staging dir is mandatory** (the descriptor law, spec §2.4):
-  the plugin is selected by the exact `(arch_id, quant descriptor)` pair
-  with no fallback. A raw official checkpoint carries
-  `quantization_config.quant_method: "fp8"` → descriptor `"fp8"` → **no
-  shipped plugin claims `(qwen4exp, "fp8")`** — raw serve AND unmodified
-  convert both refuse. The shipped `qwen4exp_fp8` plugin claims
-  `(qwen4exp, "")` (recipe-built containers). `rad_convert.cpp` writes
-  `w.set_quant(meta.quant)` into the header, so the staged config's empty
-  quant method flows through as the container descriptor while **every
-  weight stays FP8 data**. Native precision is untouched; only the
-  selection key differs.
-- `--plan-only` first: the convert's declare phase is the engine's own,
-  so a plan that resolves every op will boot. The convert runs card-less
-  (declare = CPU-side layout queries); add `--device` only if a plugin
-  load ever demands it.
-
-## The shape (`scripts/frank-archon.env`, every value commented there)
-
-| Knob | Value | Why |
-|---|---|---|
-| Image | `stilldeadcode/radiance@sha256:4c3f4845…` | digest-pinned INDEX digest (RepoDigests also carry the per-arch manifest `1c582005…`; the image ID `69d05724…` is the config digest, NOT a manifest digest — pinning it sends `podman run` to the network); entrypoint = engine, tools (`rad-convert`/`rad-info`/`rad-tune`/`rad-kbench`) on PATH |
-| Container | `frank-archon` | identity = service name across runtimes; legacy list retires `frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe` |
-| `--tp 2 --tp-wire wht6` | both R9700s | fused WHT-6 lossy wire: +5.6–6.3 % prefill, ~−1.3 % 8-stream decode at TP2; `--tp-wire-min-kb` stays engine default (128 KiB) |
-| `--max-num-seqs 2` | first boot | Frank: "parallelism = 2 to start". Raise only after a real-traffic look at slot occupancy + the ledger's KV headroom line. The fused all-reduce row count is `seqs × (1 + spec) = 8` |
-| `--max-model-len 200000` | served window | native 262,144; the publisher's production value and what `models.yml` quotes. Longer prompt = refused, never truncated |
-| `--placement expert_tiered --host-pool-mib 40960` | pinned expert pool **per rank** | native FP8 experts are 112.8 GiB total (56.4 GiB/rank slice) vs ~29 GiB at the 4-bit publisher shape — the pool is the price of native precision. Resize from the boot ledger's coverage line, never arithmetic |
-| `--gpu-headroom-mib 96 --expert-vs-cache-ratio 0.82` | publisher values | 0.82 = measured 99 % expert-cache hit; 0.87 → 0 % (the cliff) |
-| `--kv-cache-dtype fp8` | **7.32 KiB/token** | 12 QSA full-attention layers, 2 KV heads × 256 dim, K+V; a 200K-token sequence is 1.43 GiB. KV is sized from the VRAM the ratio leaves |
-| `--num-speculative-tokens 3` | MTP head | the checkpoint carries `mtp.*` (3,101 tensors, `mtp_num_hidden_layers: 1`); the plugin's probe returns draft depth 3 — the publisher's operating point |
-| `--max-num-batched-tokens 2048` | prefill chunk | A/B-verified optimum (1024/2048/4096 ladder, see "Tuning levers") |
-| `--prefix-cache-host-mib 4096 --prefix-cache-dir /data/kvcache/frank-archon --prefix-cache-disk-mib 65536` | 3 tiers | host tier pinned (see the ledger), disk tier in `/home/dank/radiance-state` (mounted rw at `/data`; `/` is ext4, mapped-tier-eligible); a session snapshot moves with its blocks |
-| `--ngram-placement ram` | **PLE pinned to host RAM** | Frank's pin law. The PLE n-gram table is 51.2 GB (320,001,536 rows × 160 dims = 29 % of all parameters) served via io_uring reads; `ram` **refuses the boot** if memlock is too small |
-| `--override-chat-template /templates/template.jinja` | froggeric v22.5 | vendored `scripts/qwen38-froggeric-v22.5.jinja` (28,234 B, `qwen3.8-froggeric-v22.5`); rad-convert bakes the checkpoint's own template, this overrides it, and the engine derives the reasoning/tool-call reply format from it |
-
-**Pinned-RAM ledger** (root unit + `LimitMEMLOCK=infinity` exist for this):
-2 × 40 GiB expert pools + 47.7 GiB PLE + 4 GiB prefix host tier + ~2 GiB
-spare ≈ **134 GiB**. `ttm.pages_limit` (this kernel has NO `pages_max` —
-the two-knob story was stale) caps GPU-mappable pinned pages (default = half
-of RAM): the pre-cutover cap was **123 GiB** — BELOW the ledger — so the
-serve script's TTM preflight (`FRANK_ARCHON_TTM_MIN_MIB=140000`) refuses to
-start until it is raised. DONE 2026-10-05 pre-cutover: sysfs write of
-**52,428,800 pages = 200 GiB** (`/sys/module/ttm/parameters/pages_limit`;
-verified read-back; raising at runtime is harmless and allocates nothing).
-PERSISTENCE: the sysfs write does not survive reboot, and the kernel-cmdline
-`ttm.pages_limit=52428800` was never added — the unit now raises the knob at
-every boot instead (`scripts/frank-archon.service:44`,
-`ExecStartPre=+…/scripts/ttm-preflight.sh --raise`).
-
-**Per-rank VRAM** at TP2: expert slice 56.4 GiB total across VRAM + host
-pool; trunk ~3.75 GiB/rank; token embedding vocab-parallel (~1.18 GiB/rank
-— the bf16 table is 2.43 GiB, sharding hands the other half to KV); lm_head
-row-sharded; recurrent state 54 MiB/copy/card (2 copies with speculation);
-KV from the `0.82`-side remainder of claimed VRAM at 96 MiB headroom.
-
-## The engine contract (all verified in source, not assumed)
-
-- **`/health` == `/ping`**: 200 JSON (`model_id, in_flight, queue_depth`)
-  while serving, 503 only while stopping — **the listener binds AFTER the
-  full load**, so a bare `curl -fsS http://127.0.0.1:7770/health` gate is
-  honest (the deployer loops on it; the FreeToken-era `"status":"ok"` body
-  grep is gone). Boot budget 1800 s (publisher `start_period` 15 m);
-  measured 2026-10-06: health 200 ~2-3 min after start (~120 s warm) —
-  the big terms are the 125.26 GiB page-in (38.1 s at 3.5 GB/s) and the
-  host-pool pin (PLE 47.68 GiB in 43.9 s).
-- **Sampling** (oai.cpp): `temperature, top_p, top_k, min_p, typical_p,
-  presence_penalty, frequency_penalty, repetition_penalty, repeat_last_n,
-  chat_template_kwargs, reasoning_effort, enable_thinking,
-  preserve_thinking` accepted; **unknown params are refused (400)** —
-  models.yml's extraBody set is fully inside it; nothing unverified rides
-  the wire. `max_tokens` is clamped, not refused; `seed` absent = pick
-  one; `response_format` supported; **`logprobs` is refused** (harness
-  never sends it).
-- **Reasoning**: `--reasoning-format` defaults to `auto` → thinking in
-  `reasoning_content` (models.yml `reasoningContentField` matches). Tool
-  calls: the qwen3.8 dialect → `message.tool_calls` +
-  `finish_reason: tool_calls`. No parser flags anywhere — parsers are
-  auto.
-- **Vision**: the official checkpoint carries the 27-layer vision tower;
-  the engine runs the Qwen-VL processor (patching, pad-token expansion,
-  3-part rotary are core). `models.yml` keeps `input: [text, image]`;
-  `--mm-max-patches` available.
-- **`/metrics`**: vLLM-compatible names + `radiance:*` families — the
-  Dankclaw Prometheus job re-scrapes 192.168.0.235:7770 (plain LAN routing
-  reaches it; the netavark published-port quirk is same-host only).
-  `/stats`, `/sessions`, `/dashboard` for live internals; the repo ships
-  `deploy/grafana/radiance.json` if a dashboard is wanted.
-- **`n` > 1 requests**: `n` defaults to `max(8, max-num-seqs)`; stop
-  strings ≤ 64 × 4096 B. `model` field not enforced.
-
-## Deploy + verify
-
-```bash
-# one-time (already done 2026-10-05): payload fetch + sha256 verify + convert
-ssh dankserver 'bash /home/dank/frank-archon-build/build-frank-archon-rad.sh --plan-only'   # declare + plan, no bytes
-ssh dankserver 'bash /home/dank/frank-archon-build/build-frank-archon-rad.sh'               # full convert + rad-info
-./scripts/deploy.sh dankserver --service frank-archon
-```
-
-Post-cutover checklist (each is observable, not assumed):
-
-1. Unit: `is-active` active, `is-enabled` enabled, container answers
-   `/health` with 200 JSON.
-2. Boot ledger (engine stdout → `/home/dank/logs/frank_archon.log`):
-   P2P probe per rank pair, TP=2 shard placement, expert pool coverage
-   line ("backs N % of the worst case … M full-length sessions"), KV pool
-   bytes, PLE pinned-RAM line, drafter depth 3, prefix tiers.
-3. MTP live: a completion's `timings` object shows draft/accept counts;
-   `radiance:*spec*` metrics in `/metrics`.
-4. Froggeric template: a probe prompt renders the v22.5 shape
-   (tool-call + thinking turn both exercise it).
-5. Vision: a ≥256 px solid-color PNG answers the exact color through an
-   `image_url` content part.
-6. Prometheus target `frank-archon` green (not never-green).
-
-## Cutover measured (2026-10-06, all values observed live)
-
-- Container: 1,857,127,997,440 B = 177,109 MiB = 173.0 GiB. Plan-only
-  priced 172.76 GiB — the DATA planes; the written container adds a
-  metadata tail. Floor gates must be set from the real file (the first
-  deploy refused on 178000 > 177109; floor is 177000).
-- Boot: 125.26 GiB read in 38.1 s (3.5 GB/s), PLE pinned
-  "47.68 GiB in 43.9 s" (the 2 B scale plane with it), each rank's
-  mover holds 25.46 GiB static+experts (19.15 GiB resident experts,
-  79 %), 16,820 pinned pool slots, health 200 ~2-3 min after start
-  (~120 s warm). No 7-minute vLLM-style boot.
-- KV ledger: kv_attn 99,956 blocks = 2.48 GiB (399,824 tokens),
-  kv_qsa_bkey 317.24 MiB, kv_gdn_state 4 slots × 54 MiB, PLE conv
-  states < 1 MiB; linear checkpoints 56.37 MiB × 8 slots (3 of 8
-  card-backed at boot, 5 withdrawn under the 208 MiB floor).
-- MTP on real traffic (temp 0.7, 220-token ask): draft_n 225 /
-  draft_n_accepted 119 = 52.9 % acceptance, decode 23.8 t/s. ONE
-  sample — the shape's real number needs live-traffic measurement.
-- Contracts: `reasoning_content` field (matches models.yml), usage
-  metering separates reasoning_tokens; vision — a 512² solid-red PNG
-  through `image_url` answers `Red` (279 prompt tokens; NOTE the
-  32-token budget is eaten by thinking — give vision probes ≥256
-  max_tokens); native `tool_calls` with finish_reason tool_calls.
-- Boot defects fixed the same day: the SSD prefix tier opened only
-  after the serve script mkdir -p'd the full path (the engine's
-  create is NOT recursive: "cannot create /data/kvcache/..."); the
-  converter's scale-name law, the PLE file-shadow and the
-  symlink-through-raw-index bug are in the payload section above.
-- Prometheus: `/metrics` answers 200 with vLLM-compatible families
-  (verified from Dankclaw); the `frank-archon` job needs a
-  prometheus-service deploy to go green (config pushed, reload
-  deferred behind a live parallel writer's dirty tree).
-
-## Tuning levers (in order of likely use)
-
-- **`FRANK_ARCHON_MAX_SEQS`**: raise 2 → 3/4 only when the ledger's KV
-  headroom line and real slot occupancy agree; each slot buys a
-  recurrent-state copy (54 MiB/card) and shrinks the per-request KV share.
-- **`FRANK_ARCHON_HOST_POOL_MIB`**: the ledger's coverage line is the
-  oracle — it prints what the pool backs at the worst case and how many
-  full-length sessions fit. RAM budget: pools + PLE + prefix ≤ ~140 GiB of
-  the host's 246.
-- **`FRANK_ARCHON_EXPERT_VS_CACHE`**: 0.82 is the measured 99 %-hit
-  point; 0.87 zeroes it. Move only with a measured hit-rate reason.
-- **`FRANK_ARCHON_TP_WIRE`**: `wht6` vs `exact` — `exact` trades the
-  +5.6 % prefill for bit-exact reductions; the fused one-shot forms are
-  declared over the seqs × (1+spec) row bound.
-- **`FRANK_ARCHON_EXTRA_ARGS`**: the boot-ladder escape hatch, expanded
-  unquoted.
-- **`rad-tune -m <rad> --tp 2 --num-speculative-tokens 3`**: benchmarks
-  the tunable space at this model's shapes on this machine, writes
-  `$RADIANCE_HOME/tune/<machine>.tune` which the engine's selector reads at
-  boot. Declare at the shapes you serve (`--tp 2
-  --num-speculative-tokens 3`), or you measure a program the serve never
-  runs. `$RADIANCE_HOME` is `/opt/radiance/share/radiance` — in the IMAGE — so a
-  `--rm` scratch container's cache dies with it: the serve script
-  bind-mounts `$FRANK_ARCHON_STATE_DIR/tune` (`/home/dank/radiance-state/tune`)
-  at `/opt/radiance/share/radiance/tune`, so the cache (36 GEMM rows,
-  `gfx1201-x3-AMD_Radeon_Graphics.tune`, generated 2026-10-06 in a
-  stopped-service window) survives and is read at every production boot.
-  The two attention requests have no shape recipe (`attn_paged`,
-  `attn_paged_gate_quant` — "see tools/opshapes.h") and are not tuned.
-  `RADIANCE_TUNE` pins a kernel for an A/B — an instrument, not a setting.
-- **`FRANK_ARCHON_MAX_BATCHED_TOKENS` (MNBT)**: A/B'd 2026-10-06 in a
-  stopped-service window (real-text probes: 20,417-token cold prefill,
-  1,985-token warm prefill, 256-token decode). The prefill ladder is
-  NON-MONOTONIC with an interior optimum at 2048: 1024 → 2048 → 4096 =
-  231 → 460 → 448 t/s cold, 221 → 412 → 427 t/s warm. Decode is flat within
-  noise (24.0 / 25.1 / 24.3 t/s); draft acceptance 60.1 / 67.1 / 63.3 %.
-  **2048 wins** (peak prefill + peak acceptance) — it stays the production
-  value. 1024 halves prefill (the batch too small to saturate the card);
-  4096 is a hair off 2048 (the chunk past the kernel's sweet spot).
-- **Diagnostics**: `rad-info` (container: encodings, recipe, profile),
-  `--debug-graph` (what declared vs what issued), `rad-kbench` (kernel
-  correctness/speed vs libref), `/stats` + `/sessions` (live), and the
-  publisher's `scripts/` harnesses (`fnpf`, `fnconc`, `cstep2`, `ident`,
-  `mctools`) — note `mcserve/fnstop`-family scripts `pkill` the engine
-  binary by name; on this host the service is a systemd unit, so use
-  `sudo systemctl stop frank-archon` instead.
-
-## Rollback
-
-| Layer | Path |
-|---|---|
-| FreeToken (one card, native FP8 safetensors) | `git checkout frank-archon-freetoken-pr587-20261005 -- scripts/frank-archon.env scripts/frank-archon.service scripts/serve-frank-archon.sh scripts/docker/ tests/frank_archon_unit.bats` + redeploy; the image in storage is `localhost/frank-primary-freetoken:pr587-77a7201`, and the payload `/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8` must be re-fetched — it is gone from the host |
-| tcclaviger vLLM fork (TP=2, shape G) | git tag `frank-archon-vllm-shapeG-20261003` (trio + env lore; image still in dank's rootless storage) |
-| 4-bit StillDeadcode publisher shape | `qwen3.8-next-flash-fp8-iq4r-moe.rad` — **this became the live payload on 2026-10-15** ([[frank-archon-radiance-qwen38-iq4r]]), deployed with `--host-pool-mib 44928` (`scripts/frank-archon.env:184`), not the 12,288 MiB first sketched here |
-
-The FreeToken-era orcarouter payload has since been deleted from the host
-(`ls /home/dank/models/` on Dankserver, 2026-10-18) — there is nothing left to
-trash, and the FreeToken rollback needs a re-fetch.
-
-## Links
-
-- [[LLM/index]] — fleet index
-- [[frank-archon-freetoken-pr587]] — the retired FreeToken era
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — the retired tcclaviger vLLM era
-- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here
-- [[RDNA4 Container Serving]] — rootless-podman GPU laws on these hosts
-- `serverAppConfigs-docs/Architecture/Testing Policy.md` — the seam contract

serverAppConfigs-docs/LLM/frank-archon-radiance-swift15-w4dn8.md

@@ -1,625 +0,0 @@
-# frank-archon — Radiance (StillDeadcode), Swift1.5 Qwen3.8-Flash-Next W4Dn8, 2× R9700 TP=2
-
-**Status: RETIRED — superseded 2026-10-15 by the publisher iq4r payload.** · Lived 2026-10-08 → 10-15 (main `3ca64e1`, e2e 33/33). · **Host:** Dankserver (192.168.0.235) · **Port:** 7770 · **GPU:** 2× Radeon AI PRO R9700 (gfx1201), TP=2
-
-The FP8-native era (`qwen4exp-fp8-native.rad`, 173.0 GiB, no-recipe convert)
-is replaced by a **GPTQ-calibrated 4-bit container** of
-`ukisai/Swift1.5-Qwen3.8-Flash-Next` (BF16, 335.3 GiB, abliterated
-finetune) converted with the recipe
-`scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe` on engine
-**1.0.13** (`stilldeadcode/radiance@sha256:4c3f484564…` — same digest as
-the FP8-native era; 1.2.0 was used ONLY for the calibration serve).
-Same port, alias, container name, systemd unit, and every serving argument
-carried into the successor: [[frank-archon-radiance-qwen38-iq4r]]. The
-`.rad` itself was purged from disk 2026-10-17; the byte-exact payload
-lives on the Hub — see Rollback.
-
-## Why this swap
-
-- **Context economics at production memory budgets**: the FP8-native
-  experts (56.4 GiB/rank slice) forced a 40 GiB/rank pinned host pool and
-  left KV ~3.5 GiB. W4Dn8 experts cut the resident set to ~1/4, freeing
-  host and VRAM budget for KV and slots at equal-or-better measured
-  decode (gate: ≥ 30.8 t/s vs the 30.08 t/s control).
-- **Swift 1.5 base**: abliterated finetune of Flash-Next (Frank's
-  fleet-personality call; frank-dragoon serves the same family).
-- **GPTQ closes the precision gap**: RTN alone on BF16 weights lands
-  measurably worse; GPTQ with Hessians from a 10M-token corpus of this
-  model's own distribution recovers most of it, verified end-to-end by a
-  KLD gate against the BF16 container itself (KL ≤ 0.03, top-1 ≥ 95 %).
-
-## The payload chain
-
-1. **Fetch**: `scripts/download-frank-archon-swift15.sh` →
-   `/home/dank/models/Swift1.5-Qwen3.8-Flash-Next` (131 shards, 335.3
-   GiB, per-shard LFS sha256 verified, no token on the command line).
-   48 layers, 512 experts (top-10 routed), moe_inter 640, hidden 2560,
-   PLE 20M-vocab table 95.37 GiB, MTP head, vision tower.
-2. **BF16 container**: `run-swift15-bf16.sh convert` →
-   `rad-convert` no-recipe (pure copy) at `--max-ctx 4096` →
-   `qwen4exp-swift15-bf16.rad` (335.28 GiB). Serves on 1.0.13 as the
-   KLD reference engine (disk-tier placement; boot ~35 min).
-3. **KLD reference**: `run-swift15-bf16.sh kld` (1.0.13) →
-   `/home/dank/corpora/kld-ref` — `radiance-kld-1` format, `rows:
-   50235`, 197 source-labelled docs, top-16 log-probs (f16), corpus
-   fnv1a64 `a9d62b33c18b3d9c`. ~4 h at 32k prefill steps
-   (`--max-num-batched-tokens 32768` — a 65536 step OOMs an arena
-   tensor at `--max-model-len 16384`). **The wrapper's exit line is the
-   receipt: never scp this script while it runs** (bash lazy-read;
-   a mid-run swap killed a clean engine exit with rc=127, third fleet
-   occurrence of the script-swap law). `kld-then-calib.sh SKIP_KLD=1`
-   resumes from the `rows: 50235` receipt.
-4. **Calibration**: `run-swift15-bf16.sh calib` on **1.2.0** (the only
-   engine with `RADIANCE_CALIB_TOKENS`; grams are engine-invariant
-   bf16 activations — RADGRAM2 format byte-identical across versions)
-   + `calib-push.py` pushing `calib-corpus.jsonl` (10,014,932 tokens,
-   1,294 prompts, assembled from real HF sources, chat-templated,
-   source-labelled). The tap (in the `qwen4exp_fp8.so` architecture
-   plugin) writes per-expert down-proj Gram matrices
-   `gram.r<rank>.blk.L.ffn_down_exps.<e>.bin` (32-byte RADGRAM2 header +
-   640×640 f32; `rows` = routing count, `trace` = activation energy)
-   and stops the serve at the token threshold. TP=2: whole experts per
-   rank (256/rank); gate_up input is rank-sliced (plugin warning — the
-   recipe only casts the down projection).
-5. **Protected-expert derivation**: `derive-protected-experts.py
-   /home/dank/corpora/calib` → the 24 highest-energy experts (the law
-   is data-driven; Swift drew 24, not the fp8-era 10 — its routing is
-   spikier: `blk.47 e445` alone carries 32 % of its layer's down energy
-   at 70× routing share) promoted to whole-expert bf16.
-   Protection law: energy share ≥ 3 % AND ≥ 3× routing share; even count
-   per rank half; odd half gains the next most energetic.
-   `apply-protected-cast.sh` splices the derived names between the
-   `# PROTECTED-BEGIN/END` markers of the recipe (refuses double-derive).
-6. **Final convert**: `convert-swift15-w4dn8.sh` (1.0.13, host-backend
-   CPU-only — no GPU visible to `rad-convert`; GPTQ solve is CPU BLAS) —
-   gates: ≥48 gram files present, bf16 container already deleted
-   (disk), ≥146 GiB free, output ≥108,000 MiB, `rad-info` shows the
-   gptq receipt. No `--reuse`. Measured: 5 h 50 m (6.0–6.5 min/layer on
-   49 GPTQ layers, load-average 8), output **116,416 MiB**.
-   The plugin's `rad_arch_declare` enforces ONE ENCODING PER EXPERT
-   TABLE (`one table of experts takes one encoding, and only a whole
-   expert kept plain bf16 is served beside it`) — a split tier
-   (down=fp8, gate_up=w4) is unservable: the kernel serves a single
-   w4nl64a8h table; both expert tables ride GPTQ w4 (down calibrated,
-   gate_up uncalibrated), protected experts bf16 beside them.
-
-## The recipe tiers (scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe)
-
-| Tier | Encoding | Why |
-|---|---|---|
-| MoE experts, both `gate_up` and `down` (one w4 table/layer) | **GPTQ w4nl64a8h** — u4 codes, fwht128 rotation, fp8 scales/64 + f32 scale plane | the campaign tier: GPTQ on down-input Hessians from the 10M-token corpus (the tap rides the down projection; gate_up shares the table uncalibrated — one-encoding-per-table plugin law) |
-| 24 derived protected experts (whole) | **bf16** | energy-share law (step 5) |
-| shared expert + trunk attention/SSM linears | **int8**, scale per 128 cols, searched | trunk precision law |
-| hyper-cond + MTP head | **fp8_e4m3 row-groups** | hc+mtp fp8 law |
-| lm_head | **int8** | i8 lm_head law (E4M3 costs 0.6 top-1 alone) |
-| PLE n-gram table | fp8 at the checkpoint's fixed scale 1.9931793212890625e-04 | the fixed-scale law; absmax audit passed the 2⁻¹³..2⁻¹⁰ plateau |
-| embeddings, vision tower, QSA indexer | **bf16 passthrough** | untouched |
-
-## The shape (`scripts/frank-archon.env`)
-
-TP=2 + `wht6`, `--max-model-len 200000`, MNBT 2048 (the MTP resonance law —
-2560 collapses decode to 2.6–7.7 t/s), `--ngram-placement ram`, froggeric
-v22.5 template override. `MODEL_FILE` points at
-`/home/dank/models/rad/qwen4exp-swift15-w4dn8-i8-hc8m.rad` (**116,416 MiB
-measured**); the serve size floor `FRANK_ARCHON_RAD_MIN_MIB` moves
-177000 → **110000**. Concurrency and the KV cache moved on 2026-10-11; that
-change is documented in its own section below.
-
-## 2026-10-11 — unquantized KV at 2 × 200,000
-
-The ask: two concurrent 200,000-token conversations on exact (unquantized)
-attention state. The fp8 pool backed 3,543 MiB → **386,904 tokens = 1.93
-sessions of the served window**, so two full-height conversations overflow it
-and the scheduler preempts the overflow to the host/disk prefix tiers
-(re-prefilling on return). Measured, that overflow is survivable — the fp8
-shape passed the same 2 × 197k pair probe — so this is a **headroom** change,
-not the repair of a hard failure. The hard limit the engine states is per
-request, not per pair: a request longer than the pool's token count is
-*"accepted by the API and then refused as too large to ever fit — this is a
-hard limit, not a backing ratio"*, and at 200,000 ≤ 386,904 no single request
-ever hit it.
-
-### The dtype law
-
-`--kv-cache-dtype` takes **only `bf16 | fp8`**: one `kv_dtype` for the whole
-cache. llama.cpp's exact-K / quantized-V pair (what frank-zealot runs as
-`--cache-type-k f16 --cache-type-v q8_0`) has **no radiance equivalent** — the
-rung under bf16 is fp8 on both halves. Evidence: the flag table in the image
-itself (`--kv-cache-dtype takes bf16 or fp8, not '%s'`) and the absence of any
-`cache-type-k`/`-v` symbol in the binary.
-
-bf16 costs **1.89× fp8, not 2×**, and the reason is in the group table:
-
-| Group | fp8 page | bf16 page | cost per token, per rank |
-|---|---|---|---|
-| `kv_attn` — 13 full-attention layers, 4-token blocks | 26.00 KiB | 52.00 KiB | 6.5 → 13.0 KiB |
-| `kv_qsa_bkey` — 13 layers, the QSA indexer keys | 3.25 KiB | **3.25 KiB** | 0.8125 KiB, dtype-fixed |
-| `kv_gdn_state` 54 MiB × 2 copies per sequence (+ conv/tail/ple) | per slot | per slot | ≈ 110 MiB **per sequence** |
-| prefix checkpoints 8 × 55.99 MiB (inside the pool) | 448 MiB | 448 MiB | fixed |
-
-→ `pool_mib ≥ tokens × 13.8125 KiB + 110 × seqs + 448`. The formula reproduces
-both live boots: fp8 `(3543 − 330 − 448) → 387,240` vs the ledger's 386,904;
-bf16 `(6912 − 220 − 448) → 462,946` vs the ledger's **462,752**. It is encoded
-as a test seam (`the KV pool backs seqs x max-model-len at the declared cache
-dtype`) so a future resize cannot silently under-provision the pool.
-
-### Why the weights slab pays — and why that is not pure loss
-
-The two VRAM budgets are one budget, and the ceiling is the engine's own — not a
-hand-picked reserve. Card 32,624 MiB, and the table that prints at the pair
-24,000/5,120 says `3.42 GiB is neither budgeted nor accounted — the driver's
-context, the activation arena and --gpu-headroom-mib come out of it; anything
-past those is throughput on the floor`, with `card 0 settled at 2.01 GiB free
-against 209.19 MiB (the 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the
-static region)`. Claimable reserve above the ENGINE's floor:
-2,058 − 209 = 1,849 MiB ⇒ **`weights + kv ≤ 30,969 MiB`**. (The 29,120 bound
-written on 2026-10-11 was my own 1 GiB "keep free" reserve, measured off a card
-figure of 31,856 MiB; the engine never asked for that reserve and the seam now
-derives the number from its table.) Paying for a 6,912 MiB pool therefore cuts
-the resident slab 26,071 → 22,208 MiB (−14.8 %), and the evicted experts spill
-into the pinned host pool (40,960 → 44,928 MiB per rank; ledger: pinned pool
-slots 7,190 → 10,375). The pinned-RAM ledger rises to ≈147 GiB, so the TTM
-floor moved 140,000 → 150,000 MiB (the kernel cap is 204,800).
-
-Half of that loss is refunded by an engine behaviour worth knowing: **idle KV
-VRAM is the expert cache's spillover**. The fp8 pool lent 2.69 GiB to the expert
-slab; the bf16 pool lends **6.08 GiB** (`mover: 5146 slots can be lent out of
-the KV cache`). On a llama.cpp endpoint KV is inert storage; on radiance it is
-a competing cache, and every KV resize is also an expert-residency resize.
-
-### seqs 3 → 2
-
-`--max-num-seqs 3` was never a radiance requirement: it is the tcclaviger vLLM
-fork's torch.compile constraint (`ConstraintViolationError: Constraints
-violated (L['query_start_loc'].size()[0])`), retired with that runtime — see
-[[dankserver-qwen38-fp8-expert-offload-tuning]]. Radiance booted seqs 2 in the
-fp8-native era, and the 2026-10-06 decode sweep measured seqs 2 and 3 tied
-inside the ±3 % noise floor. Two slots × the window is the capacity contract;
-it also saves 110 MiB of per-slot recurrent state and drops the fused TP
-all-reduce to 2 × 3 = 6 rows, i.e. deeper inside the 25-row bound at which the
-wht6 wire is bit-exact.
-
-### Boot ledger (deployed 2026-10-11, commit `0bf26b0`)
-
-```text
-kv cache           6.75 GiB   349% of it -- backs 2 sequences all at 200000
-                                tokens at once, which is the worst case
-vram_weights      21.69 GiB   stated by --vram-weights-mib
-vram_kv            6.75 GiB   stated by --vram-kv-mib
-unclaimed          2.03 GiB
-kv_attn      full 13 layers block 4 tok page 52.00 KiB 115688 blocks = 5.74 GiB (462752 tokens)
-kv_qsa_bkey  full 13 layers block 4 tok page  3.25 KiB 115688 blocks = 367.17 MiB (462752 tokens)
-load: rank 0, mover took 21.69 GiB, weights pool 147.00 KiB free of 21.69 GiB
-mover: 5146 slots can be lent out of the KV cache (6.08 GiB)
-card 0 settled at 1.85 GiB free
-serving on http://:::7770
-```
-
-### Verification (live service, no scratch container)
-
-| Check | Result |
-|---|---|
-| `/health`, `/v1/models` | 200 `{"status":"ok","model":"frank-archon"}`; served id `frank-archon` |
-| 2 × ~197k concurrent, a unique marker buried at the head of each | **both answered their own marker exactly** — 196,919 + 196,918 = 393,837 tokens resident, above the fp8 pool's 386,904 and inside bf16's 462,752; walls 138.1 s / 66.2 s, `finish_reason: stop`, prefill 2,747 / 2,991 t/s |
-| decode at 51.5k depth, 512 tokens, greedy | 94.1 cold / 102.3 warm / 117.6 on a prefix-cache hit t/s; draft acceptance 0.7284 / 0.7525 (fp8 control at cutover: 0.7246) |
-| fp8 control arm, same probes (`scripts/probe-frank-archon-kv.py`) | table in the next section |
-
-Probe law re-learned, worth recording: radiance routes the thinking block to
-`reasoning_content`, so a chat request whose budget is spent inside `<think>`
-comes back with **empty `content` and `finish_reason: stop`** — a probe that
-forbids thinking (`chat_template_kwargs.enable_thinking=false`) and reads both
-fields is the only honest shape. The first pair probe scored 0/2 on exactly
-that artifact, not on a KV fault.
-
-### Measured cost — A/B on the live service
-
-Method (no scratch container, no mask — the unit file is a relink symlink and
-`mask` refuses it): write a variant env file on the host
-(`/home/dank/frank-archon-fp8control.env`), point a transient drop-in at it
-(`Environment=FRANK_ARCHON_ENV_FILE=…`; drop-ins are read AFTER the unit's own
-`Environment=` lines, so they win), `daemon-reload`, restart, probe, delete the
-drop-in, restart. Control = the deployed 2026-10-08 shape, verified from
-`podman inspect` argv (`--kv-cache-dtype fp8 --vram-weights-mib 26071
---vram-kv-mib 3543 --max-num-seqs 3`) and the ledger page size (26.00 KiB).
-Identical probes, same salts, loopback, greedy, 512 generated tokens.
-
-| Probe | fp8 control (26071/3543, seqs 3) | bf16 deployed (22208/6912, seqs 2) | Δ |
-|---|---|---|---|
-| prefill 51.5k, cold | 4,379 t/s | 3,126 t/s | **−29 %** |
-| prefill 51.5k, warm | 4,864 t/s | 3,333 t/s | **−31 %** |
-| decode 51.5k, cold | 124.1 t/s | 94.1 t/s | −24 % |
-| decode 51.5k, warm | 99.2 t/s | 102.3 t/s | +3 % |
-| decode on a prefix-cache hit | 127.7 t/s | 117.6 t/s | −8 % |
-| draft acceptance, mean of passes | 0.661 | 0.740 | **+12 %** |
-| 2 × 196,919 concurrent pair | both markers HIT, 110.5 s | both markers HIT, 138.1 s | +25 % wall |
-
-Read it as: **the price of exact KV on this engine is prefill, not decode.**
-Prefill is expert-bandwidth-bound and the resident slab is 14.8 % smaller, so
-time-to-first-token at the full window rises ~25 % (the pair: 110 s → 138 s);
-both prefill passes agree, far outside the ±2 % prefill noise floor. Decode is
-inside the ±3 % band on two of three passes (warm +3 %, hit −8 %); the cold-pass
-−24 % is the largest single number, the least reproducible, and it moves the
-same direction as prefill — the cold arm is paying expert streaming, not
-attention. Spec-decode acceptance IMPROVED ~8 points (0.661 → 0.740): exact
-attention state makes the target agree with the MTP drafter more often, which
-pays on every token of every conversation, and it is the strongest argument for
-the shape.
-
-Open call (Frank): keep exact KV at the prefill cost, or take the cheap fp8
-variant — `--vram-kv-mib 3,648` + `--max-num-seqs 2` fits 2 × 200,000 at fp8
-for a 105 MiB pool bump and ≈ no decode cost. The tool is committed:
-`scripts/probe-frank-archon-kv.py` (run it on Dankserver; it rebuilds its
-prompts from the repo checkout there).
-
-### Rollback
-
-Revert `scripts/frank-archon.env` to the fp8 pair (26071/3543, fp8, seqs 3,
-host pool 40960, prefix host 4096) and redeploy: no payload change, boot ~2 min.
-The KLD gate scored at the fp8-KV shape, so a bf16-KV production runs *away*
-from the scored configuration — bf16 can only reduce the attention-state
-error, and a re-score is a drained-window job if the gate agreement is ever
-needed in the record.
-
-## 2026-10-13 — the power failure, the ttm reset, and 4 × 200,000 fp8 KV
-
-### The outage: a refusal, not a crash
-
-Dankserver lost power on 2026-10-12 ~23:17. The box came back; frank-archon
-never did. The unit sat in `activating (auto-restart)`, exiting 1 about 50 ms
-after starting, with one journal line on repeat:
-
-```text
-ERROR: ttm pages cap is 125988 MiB, the pinned ledger needs >= 150000
-frank-archon: raise /sys/module/ttm/parameters/pages_limit …
-```
-
-**6,919 refusals in ~20 hours.** No container was created (`podman ps -a` showed
-the name with a zero start time), so there was no engine, no port, and no engine
-log — the signature of a preflight refusal, which every deployer/monitor view
-outside the journal renders as an unexplained crash loop.
-
-The cause is a class, not an incident. The pinned-RAM ledger (~147 GiB: 2 × 44 GiB
-expert pools + the 47.7 GiB PLE table + the 8 GiB prefix host tier) rides TTM's
-shared pages, capped by `/sys/module/ttm/parameters/pages_limit`. **That knob is
-runtime state, and its default is half of RAM** — so the manual raise of
-2026-10-05 (52,428,800 pages = 200 GiB) died with the power failure and the cap
-fell back to 32,253,039 pages = 125,988 MiB, 24 GiB below the floor the serve
-script enforces. The fp8-native era had planned the persistent half of this
-("add `ttm.pages_limit=52428800` to the kernel cmdline at the next scheduled
-reboot", [[frank-archon-radiance-qwen38-native-fp8]]); it was never applied, and
-a power failure is not scheduled.
-
-### The repair: the gate raises the knob itself
-
-`scripts/ttm-preflight.sh` (sourced by `serve-frank-archon.sh`) reads the cap as
-the MIN over the knobs the running kernel exposes, and when it is below
-`FRANK_ARCHON_TTM_MIN_MIB` it writes `FRANK_ARCHON_TTM_TARGET_MIB` (204,800 MiB =
-52,428,800 pages) into every writable knob, **re-reads**, and only then refuses.
-The unit is root and the knob is a 0644 root-owned module parameter, so the write
-is legal; a kernel that refuses the move produces the same named refusal as
-before, and a reboot is now a boot rather than a visit to the machine.
-
-| Knob | Power-failure default | After the gate |
-|---|---|---|
-| `/sys/module/ttm/parameters/pages_limit` | 32,253,039 pages = 125,988 MiB | 52,428,800 pages = 204,800 MiB |
-
-Seam-tested, not inspection-tested: *the ttm gate raises a reset pages cap and
-refuses when the write fails* and *the ttm cap counts only the knobs the kernel
-exposes* drive the gate against a temp knob directory — a below-floor writable
-knob ends at the target and the start passes; a read-only knob leaves the cap
-untouched and the start refuses by name; an absent `pages_max` is skipped, never
-read as zero (a zero reading would refuse a healthy host).
-
-### The shape: 4 slots × 200,000, fp8 KV (same day, superseded by the re-cuts below)
-
-Answers the open call of 2026-10-11 from the other side: Frank asked for **four**
-concurrent 200,000-token conversations with quantized KV.
-
-fp8 is not a preference at four slots, it is the only dtype that affords them.
-The pool must back `seqs × max-model-len` (the engine's hard limit), so the
-fourth slot is bought in KV, out of the one per-card budget
-(`weights + kv ≤ 29,120 MiB`, the bound as it was understood then):
-
-| dtype | pool for 4 × 200,000 | weights slab left |
-|---|---|---|
-| bf16 | 10,791 + 440 + 448 = 11,679 MiB | 17,441 MiB (−33 % vs the fp8-era 26,071) |
-| **fp8 (deployed)** | 5,713 + 440 + 448 = 6,601 MiB | **22,208 MiB — unchanged** |
-
-So `--vram-weights-mib 22208` / `--vram-kv-mib 6912` stay exactly where the
-2026-10-11 change put them (pair sum = the measured 29,120 ceiling), the
-44,928 MiB/rank pinned host pool is untouched, and the RAM/TTM ledger does not
-move. What the fp8 pool buys: `(6912 − 110×4 − 448) × 1024 / 7.3125` =
-**843,566 predicted tokens** = four full-height conversations with 5.4 % slack
-(the bf16 pool backed 462,752 = two; the 2026-10-08 fp8 pool backed 386,904 =
-1.93).
-
-Terms that move with the dtype and the slot count:
-
-- **Per-slot recurrent state** is dtype-independent and lives inside the pool:
-  +110 MiB per slot, 440 MiB for four (already in the arithmetic above).
-- **The fused TP all-reduce** is declared over `seqs × (1 + spec)` rows: 4 × 3 =
-  **12 rows**, still inside the 25-row bound at which the `wht6` wire is
-  bit-exact — so four slots cost nothing in wire exactness.
-- **Prefix-cache entries shrink** with the attention plane (110.50 KiB at bf16 →
-  58.50 KiB at fp8), so the 8 GiB host tier parks ~5.8 sessions of 200k where at
-  bf16 it parked ~2.9 — four live slots plus a parked set fit with no re-size.
-- **`models.yml` is untouched**: `contextWindow` quotes the per-request window
-  (200,000), which did not change; only the concurrency did.
-
-Expectation carried from the 2026-10-11 A/B, whose arms bracket this shape
-(same slab 22,208): prefill is set by the slab, not the dtype, so ~3.1k t/s
-cold / ~3.3k warm; decode sits in the 94–118 t/s band with draft acceptance
-~0.73–0.75. The 4,379 t/s fp8 prefill of the 2026-10-08 control belongs to the
-26,071 MiB slab and does not come back by quantizing KV.
-
-### Boot ledger (2026-10-13 19:21, four slots, fp8 — superseded same evening)
-
-The sizing law's prediction, checked against the engine's own arithmetic:
-predicted 843,566 tokens, ledger **843,276** (0.03 % block rounding).
-
-| Group | Page / state | Size | Tokens |
-|---|---|---|---|
-| `kv_attn` | 13 layers, 4-token blocks, **26.00 KiB** (fp8 page) | 5.23 GiB | **843,276** |
-| `kv_qsa_bkey` | 13 layers, 4-token blocks, **3.25 KiB** (dtype-fixed) | 669.10 MiB | **843,276** |
-| `kv_gdn_state` | 36 layers, 54.00 MiB, **8 slots** (4 × 2 copies) | 432.00 MiB | per-slot |
-| `kv_gdn_conv` / `kv_ple_*` | 36 + 1 + 1 layers, 4 slots | 7.91 MiB | per-slot |
-
-`kv cache 6.75 GiB … backs 4 sequences all at 200000 tokens at once, which is
-the worst case` — the capacity contract, in the engine's words. The rest of the
-boot is the fp8-era picture: `vram_weights` 21.69 GiB filled to 147 KiB (rank 0)
-/ 355 KiB (rank 1) of slack, `unclaimed` 2.03 GiB, cards settled at 1.99 /
-2.00 GiB free, `mover: 4967 slab slots (64 shadow), 10375 pinned pool slots,
-4.24 GiB static vram`, and the half-refund in plain sight: `4967 slots can be
-lent out of the KV cache (5.87 GiB)` (the fp8-era 3,543 MiB pool lent 2.69 GiB).
-
-Two ledger lines that confirm the derived consequences: the sampler prints
-**12 rows** (4 sequences × (1 + 2 speculative) — inside the 25-row bound where
-`wht6` is bit-exact), and the prefix/tier arithmetic runs on the fp8 entry size
-(58.50 KiB, half of the bf16 entry).
-
-### The same-day re-cuts: 4 → 3 slots, then the slab takes the card's slack
-
-Frank's call, twice in one evening. Both are one-env-pair changes, and both were
-read off the boot ledger rather than assumed.
-
-**4 → 3 slots (22208/6912 → 24000/5120).** Three slots need
-3 × 200,000 at fp8 = 4,285 + 330 + 448 = **5,063 MiB** of pool; the 6,912 pool
-was backing 858,969 tokens — 43 % more window than the contract asks for.
-Cutting it to 5,120 (predicts 608,027; ledger **607,740**, 0.05 %) hands
-1,792 MiB to the slab and drops the all-reduce to 9 rows.
-
-**24,000 → 25,600 (the card's settling slack).** The 24,000 boot printed
-`card 0 settled at 2.01 GiB free against 209.19 MiB (the 96.00 MiB
---gpu-headroom-mib and 113.19 MiB kept for the static region)` — 1,849 MiB above
-the ENGINE's own floor, with the budget table calling the surplus `throughput on
-the floor`. Claiming 1,600 MiB of it puts the pair at 30,720 and the ledger at
-`unclaimed 477.30 MiB` (cards ~336 MiB free, floor 209.19).
-
-| pair | slab slots (rank 0) | pinned pool slots | A-cold 43.5k | concurrent 197k |
-|---|---|---|---|---|
-| 22208/6912, seqs 4 | 14,753 | 10,375 | 3,397 t/s | 3,307 t/s (4-way) |
-| 24000/5120, seqs 3 | 16,235 | 8,893 | 3,290 t/s | 3,051 t/s (3-way) |
-| **25600/5120, seqs 3** | **17,548** | 7,580 | **3,584 t/s** | **3,589 t/s** (3-way) |
-| 26071/3543, seqs 3 (2026-10-11 control) | — | 7,190 | 4,379 t/s | — |
-
-The claim shipped with the 25,600 arm — that the slab is a THRESHOLD below
-~26 GiB so the extra 1.6 GiB buys ~0 — is **falsified by its own measurement**:
-+8.9 % single-stream, +17.6 % concurrent, both outside the ±3 % noise floor. Two
-slab points that land inside noise of *each other* (22,208 vs 24,000) are not a
-plateau; the pinned host pool is the slow path, so every MiB moved from RAM onto
-the card is throughput (the ledger shows the transfer directly: slab
-+1,313 slots, pinned pool −1,313).
-
-The cliff is real, it just sits above 25,600: 26,071 measured 4,379 t/s, +22 %
-for +471 MiB. At three slots that is out of reach (26,112 + 5,120 = 31,232 > the
-30,969 ceiling). At **two** slots the pool falls to 3,072 and the slab reaches
-**27,897** — the ceiling exactly, slab above the cliff. That arm is untested and
-it is a capacity decision, not a tuning one.
-
-Capacity contract, live-verified on the 3-slot shape: three concurrent prompts
-of **199,984 tokens** each (599,952 tokens) were accepted and completed on
-`/v1/completions`; a fourth full-height request is refused as "too large to ever
-fit" — radiance never truncates.
-
-### The decode ladder (2026-10-13): slots are paid in concurrency, not prefill
-
-Frank's challenge — "why are you not tracking decode?" — was right, and the
-answer is a number now. 512 generated tokens (the bench law), greedy, cold
-prompts, this shape (3 x 200,000, fp8, slab 25,600):
-
-```
-1-stream  43.5k: decode  99.0 t/s  (prefill 4,042, accept 0.952)
-1-stream   197k: decode  93.3 t/s  (prefill 4,826, accept 0.938)
-3-stream  43.5k: decode 113.9 / 43.3 / 10.9 t/s   wall 83.4s
-3-stream   197k: decode  88.9 / 16.3 / 64.6 t/s   wall 122.4s
-```
-
-Two laws came out of it. **Decode is nearly window-free** — 99.0 at 43.5k,
-93.3 at 197k, −6 % for 4.5x the depth — so a long conversation costs
-prefill, not tokens-per-second. **Concurrency is where the slot count is
-paid**: three simultaneous streams give 18.4 t/s aggregate at 43.5k and 12.5
-at 197k against 99 single-stream, and the split is a lottery
-(113.9/43.3/10.9), not a fair share — the fused-step schedule hands the step
-to whoever it reached first, so one stream finishes at 113 t/s while its
-sibling crawls at 10.9. That is what "3 slots" owes per token, and no prefill
-number shows it.
-
-Corpus caveat, stated at the claim: this probe's filler is a repetitive
-ladder, so MTP acceptance is 0.93–0.97 against the 0.740 of the 2026-10-11
-table below; these rows flatter decode relative to that table and are the
-baseline for *this* corpus, not a replacement for those rows.
-`probe-frank-archon-kv.py` now carries this leg as **D** (leg B generates 24
-tokens, which is why a day of shape work priced only prefill).
-
-### Recovery evidence
-
-```text
-Oct 13 19:21:08 dankserver ttm-preflight.sh[1623778]: frank-archon: ttm pages cap
-  is 125988 MiB, below the pinned-ledger floor 150000 — raising the knob to
-  204800 MiB (52428800 pages)
-Oct 13 19:21:08 dankserver ttm-preflight.sh[1623778]: frank-archon: ttm pages cap
-  204800 MiB clears the pinned-ledger floor 150000
-Oct 13 19:21:08 dankserver systemd[1]: Started frank-archon.service
-```
-
-`cat /sys/module/ttm/parameters/pages_limit` → 52,428,800 (204,800 MiB) after
-the boot, i.e. the root `ExecStartPre` hook did the write the service process
-(User=dank) cannot.
-
-### Capacity probe (2026-10-13, live service)
-
-`scripts/probe-frank-archon-kv.py` now reads the slot count, window and KV dtype
-from the deployed env on the host, so the probe cannot drift from the service:
-it printed `shape: 4 x 200,000 tokens, kv fp8` and fired **four concurrent chats
-of 196,960 prompt tokens each** (646,299 chars, 98.5 % of the window), each
-carrying a marker whose first word is unique.
-
-| Stream | prompt | wall | prefill | retrieval |
-|---|---|---|---|---|
-| B1 | 196,961 | 238.3 s | 1,423 t/s | exact, HIT |
-| B2 | 196,960 | 98.0 s | 2,043 t/s | exact, HIT |
-| B3 | 196,961 | 201.1 s | 993 t/s | exact, HIT |
-| B4 | 196,960 | 149.9 s | 1,327 t/s | exact, HIT |
-| **total** | **787,842** | **238.4 s** | **3,305 t/s aggregate** | 4/4 |
-
-All four answered their own marker; nothing was preempted, refused, or truncated
-(`finish=stop`, `cache_n=0`, so no prefix cache padded the result). Aggregate
-prefill equals the single-stream rate measured in the same probe (3,397 t/s at
-43.5k), i.e. the four streams share one prefill pipeline at the slab-limited
-ceiling and the engine never ran short of KV. Prefix cache intact: the same
-43.5k prompt repeated answered in 1.0 s with `cache_n=43008`.
-
-The per-stream `decode` column is NOT a decode rate here — 8 generated tokens
-make it first-token dominated (the bench law: generate ≥ 128 tokens to read a
-rate). This probe answers one question only: does the endpoint owe four
-full-height conversations at once, and answer each one exactly. It does.
-
-## Gates (measured at cutover)
-
-- **KLD gate: PASS (2026-10-08 04:12)** — `kld-gate-swift15.sh`,
-  candidate on 1.0.13 vs `kld-ref`: **mean KL 0.0146** (median 0.00126,
-  p90 0.0315, p99 0.194, max 4.12), **top-1 97.84 %**,
-  **ppl ratio 1.0126** (candidate 1.6913 vs reference 1.6702), 50,235
-  positions / 197 docs. Report: `/home/dank/radiance-state/kld-gate.json`
-  (+ `.rows` per position). Gate shape: disk-tier, pool 40960, MNBT 3072
-  (the w4 activation arena scales hard with MNBT: 32768 → 33.3 GiB arena,
-  refused), KV 2048, headroom 512, no speculation. The engine ran rank 0
-  fully on card 0 (25.88 GiB) with rank 1 host-side in this shape —
-  scoring is placement-invariant; scored at 28 positions/s.
-- **KLD gate re-PASS (2026-10-08 04:49) at the production-legal shape**:
-  MNBT 2048 (the w4 file-tier law: experts stream in place only at
-  MNBT ≤ 2048 — the recording's 32768 demands a widened form the disk
-  tier refuses), pool 40960, vram-kv 3072 fp8, tp 2, headroom 768.
-  **mean KL 0.0131, top-1 97.92 %**. The Δ vs the first PASS (0.0015) is
-  the float-order noise band of the chunk-size change — the verdict is
-  chunk-size invariant, ~2.3× inside the 0.03 bar on both shapes.
-- **Throughput gate: PASS (2026-10-08 05:40)** — decode at 44.5k depth
-  (streamed chat probe, temp 0, 512 tokens, salted): arm A (main serve
-  shape, EVC 0.82) **146–173 t/s**, engine timings **121.9 t/s**,
-  draft accept **0.7246** (350/483); arm C (deployed vram-pair shape
-  26071/3543) median **136.0 t/s**, warm TTFT 9.45 s. Control
-  (fp8-native 1.0.13): 30.08 / accept 0.7549. Prefill ~4.7k t/s warm.
-  The w4 expert table is resident — the fp8-native era's NVMe expert
-  streaming (the 3.5 GB/s wall) is gone; the MTP ×3 head rides a 45–60
-  t/s base decode. Conversation battery (short context): geomean
-  172 t/s (min 100 / max 208), mean ttft 0.09 s. The EVC ladder is moot:
-  the w4 resident set fits fully inside the 40 GiB pool at 0.82
-  (ledger: mover full fill). Depth-output audit: the 44.5k-depth
-  summary is a coherent ordered index of the concatenated corpus
-  (no repetition, no NaN-token garbage) — the depth-corruption class
-  of 2026-10-04 is clean.
-- Fleet probes: PASS, re-run on the deployed prod — vision (solid-red
-  512 px → "Red"), tool_calls (`{"expr":"23 * 47"}`), reasoning_effort
-  xhigh/medium/low/high all 200 (the Swift1.5 froggeric template accepts
-  the full set), reasoning_content present, response_format json_object
-  honoured. **Vision probe law:** budget ≥ 256 tokens and read
-  `reasoning_content` as fallback — at 64 tokens thinking exhausts the
-  budget and `content` comes back empty (the answer lands in the
-  reasoning channel).
-- **Deployed verification (2026-10-08)**: `deploy.sh dankserver
-  --service frank-archon` green incl. Playwright e2e 33/33; unit
-  `active`, `/health` 200, `/v1/models` id `frank-archon` (the harness
-  alias is the served-model-name, independent of the file — models.yml
-  untouched). Deployed shape: fp8 KV (KLD-gate shape), spec depth 2,
-  seqs 3, vram pair 26071/3543. Boot ledger: KV pool 3.46 GiB, expert
-  pools 57.84 GiB, cards settled 1.41 GiB free, `serving on :7770`.
-  Live decode probe (512 tokens, streamed, temp 0): ~190 t/s.
-
-### Wire rename + the bench drop-in trap (2026-10-09)
-
-`FRANK_ARCHON_SERVED_NAME` moved the served id `frank-primary` →
-**frank-archon** (commit `7c77326`, 2026-10-09); the service, unit, scripts and
-env file kept `frank-primary` until the 2026-10-10 rename (`55f87fc`) brought
-them to `frank-archon` too. Radiance is the only
-fleet engine whose `/v1/models` listing reflects the flag, and it echoes the
-**requested** model name in chat responses — so a chat probe that returns 200
-proves nothing about the served id. Always read `/v1/models`.
-
-The first deploy after the edit came up green and `/v1/models` still said
-`frank-primary`. Cause: a campaign leftover drop-in
-`/etc/systemd/system/frank-archon.service.d/bench.conf` (2026-10-08 04:09)
-overrode BOTH `Environment=FRANK_ARCHON_ENV_FILE` and `ExecStart` to the
-staging worktree `/home/dank/serverAppConfigs-swift15`. Every
-`deploy.sh dankserver --service frank-archon` since then restarted the
-staging tree: green health gate, old configuration, silently. Verified before
-removing: the staging `serve-frank-archon.sh` and `frank-archon.service`
-are byte-identical to the checkout, its `frank-archon.env` differed only by
-the alias line, and its HEAD (`8375567`) is an ancestor of `main`. The
-drop-in was staged in `/home/dank/.AGENTTRASH/20261009-frank-archon-bench-dropin/`
-(that dated dir is gone — the trash dir is empty on Dankserver as of 2026-10-18).
-After `daemon-reload` + restart, `/v1/models` reports `frank-archon` (boot
-healthy in 102 s, shape unchanged: fp8 KV, spec depth 2, seqs 3, vram pair
-26071/3543).
-
-**Deployer blind spot:** `deploy.sh` plans from repo paths and gates on the
-port. It cannot see host-side drop-ins, so unit drift is invisible to a green
-run. After an LLM deploy, verify the served configuration behaviourally
-(here: `podman inspect <name> --format {{json .Config.Cmd}}` and `/v1/models`).
-
-
-
-## Deploy + verify
-
-```bash
-sudo systemctl stop frank-archon          # drained window
-bash convert-swift15-w4dn8.sh              # if not already run
-./scripts/deploy.sh dankserver --service frank-archon
-```
-
-Post-cutover checklist: unit active/enabled; `/health` 200; boot ledger
-(expert pool coverage now w4-sized, KV pool should be larger than the
-FP8-native era's 399,824 tokens); MTP draft counts in `timings`; froggeric
-probe; ≥256 px `image_url` vision probe answers the color; tool-call
-probe; `reasoning_effort` fold; Prometheus target green; `tests/run.sh`
-+ Playwright e2e green.
-
-## Rollback
-
-| Layer | Path |
-|---|---|
-| Publisher iq4r (successor) | [[frank-archon-radiance-qwen38-iq4r]] — the LIVE payload since 2026-10-15 |
-| FP8-native radiance | rebuild-only — the `.rad` (173.0 GiB) was `rm`'d 2026-10-15: `scripts/download-frank-archon.sh` + `scripts/build-frank-archon-rad.sh` |
-| FreeToken PR #587 | git tag `frank-archon-freetoken-pr587-20261005` |
-| tcclaviger vLLM shape G | git tag `frank-archon-vllm-shapeG-20261003` |
-
-**This era's own payload** (`qwen4exp-swift15-w4dn8-i8-hc8m.rad`,
-116,416 MiB) was purged in the 2026-10-17 disk cleanup (trash-staged, then
-emptied with Frank's approval). The byte-exact container is published as
-`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`
-(`qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, Hub LFS sha256
-`09dd93064365009c508b59012adadab6a36791a50fa319f76b31bf359c01e9ff`) —
-re-download restores the exact payload; a full re-convert is no longer
-possible without re-fetching a BF16 source (see next paragraph).
-
-Campaign artifacts: the raw BF16 checkpoint was `rm`'d 2026-10-15 (Frank's
-one-time authorization), so re-convert needs a fresh BF16 source; `kld-ref`
-(24 GiB), grams (`/home/dank/corpora/calib`), corpora JSONL, and the logs
-`/home/dank/logs/swift15_*.log` remain. The BF16 `.rad` container was
-deleted before the final convert (disk gate).
-
-## Links
-
-- [[frank-archon-radiance-qwen38-native-fp8]] — the replaced FP8-native era
-- [[LLM/index]] — fleet index
-- [[swift15-campaign-worklog]] — campaign record
-- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here

serverAppConfigs-docs/LLM/frank-archon.md

@@ -0,0 +1,440 @@
+---
+type: reference
+title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
+description: "The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras table carrying every rollback path including the re-download doors for the purged payloads, the laws this endpoint taught (pinned-RAM/TTM preflight, the KV dtype law, the one per-card VRAM budget, descriptor selection), the wire contract, and the files that own it. It replaces the nine per-era runbooks."
+tags:
+  - llm
+  - frank-archon
+  - dankserver
+  - radiance
+  - gfx1201
+  - r9700
+  - tp2
+  - expert-offload
+timestamp: 2026-10-18
+---
+
+# frank-archon — radiance (StillDeadcode), 2× R9700 TP=2, Dankserver :7770
+
+The radiance engine (C++/HIP, `libr4d` kernels built for gfx1201, no Python in the serving
+path) driving `podman run` straight from the digest-pinned image under a **root** systemd
+unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI
+PRO R9700s. Capacity contract **3 conversations × 200,000 tokens at fp8 KV**; a payload swap
+is a `MODEL_FILE` line plus a re-download — see Eras.
+
+## Current shape
+
+Authoritative source: `scripts/frank-archon.env` + `scripts/serve-frank-archon.sh` (the env
+header carries the measured arithmetic beside each number).
+
+**Payload** — `/home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad`, 121,969,901,568 B
+= **116,322 MiB**, sha256 `0af5e962…` (the Hub LFS oid), from
+`StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe`: the engine author's own rad-convert of the
+**official (censored)** `Qwen/Qwen3.8-Flash-Next` bf16 checkpoint. Encodings (recipe
+`qwen4exp-w4nl64-i8-hc8m`): routed experts as 4-bit codes in a 16-level non-uniform codebook
+(128-pt Walsh–Hadamard rotation, GPTQ on a 10M-token calibration + 3 coordinate-descent
+sweeps, E4M3 scale per 64 weights); **10 whole bf16-protected experts per layer**; int8 W8A8
+trunk + lm_head; fp8 HC mixtures; PLE n-gram fp8 at the checkpoint's fixed scale (47.7 GiB);
+MTP head and the bf16 27-layer vision tower **in the container**. Card quality vs bf16: mean
+KL 0.0789 all-tokens / 0.0239 assistant turns, top-1 92.85 %. The abliterated family lives on
+[[frank-dragoon]]. Fetched by `scripts/download-frank-archon-iq4r.sh --fetch` (hfget resume +
+declared size + sha256) as the transient unit `iq4r-fetch` (log
+`/home/dank/logs/iq4r-fetch.log`, `…rc` sentinel); the staging dir
+`/home/dank/models/StillDeadcode/…` keeps card, LICENSE and recipe as provenance, and only
+`.rad` files are mounted.
+
+**Runtime** — `stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e`
+= pin **1.0.13** (Codeberg `v1.0.13`, commit `53282fa`): a 287 MiB image in dank's
+**rootless** storage, host needs nothing but the amdgpu driver. Pin the INDEX manifest digest,
+never the image ID (`69d05724…` is the config digest and sends `podman run` to the network).
+The serve script's gate is that digest ref itself — `podman image exists name@sha256:…` in one
+command (the 2026-09-30 no-pipe/grep-race lesson) — so a retagged look-alike or a newer
+`latest` cannot satisfy it. 1.2.0 is a rejected serve pin (Laws); the
+`--vram-weights-mib`/`--vram-kv-mib` pair is accepted by both pins, which is the rollback symmetry.
+
+**Trio** — `scripts/frank-archon.env` (every number, with its arithmetic in the header),
+`scripts/serve-frank-archon.sh`, `scripts/frank-archon.service`. Gates run in order: image
+digest present → `.rad` present and ≥ `RAD_MIN_MIB` (a truncated convert would otherwise buy a
+guaranteed long-timeout boot instead of this fast failure) → template present → `/dev/kfd` +
+`/dev/dri` → `ttm_ensure_cap` (the serve script re-verifies the cap after the unit's hook) →
+`podman rm -f` over `FRANK_ARCHON_LEGACY_CONTAINERS` (`frank-archon-qwen frank-archon-r9v
+qwen-fp8 glm-probe`) plus its own name, because systemd SIGKILLs conmon on a crash and leaves
+the name registered, and every one of those identities holds this GPU. Container contract
+(publisher-vetted, `docs/DOCKER.md`): `--network host`, `--init` (so the engine releases the
+cards on SIGTERM), `--security-opt seccomp=unconfined` (the default profile refuses io_uring —
+the PLE table's reads — and the NUMA calls the ROCm runtime makes for host memory), `--ulimit
+memlock=-1:-1`, `/models` ro + `/data` rw + the tune cache at `/opt/radiance/share/radiance/tune`,
+template as a single-file mount, then `exec podman wait`: the script's lifetime IS the
+container's, so a crash is a nonzero `ExecStart` and the restart policy is real.
+Unit: `After=network-online.target`, `WantedBy=multi-user.target`, `User=dank`,
+`LimitMEMLOCK=infinity`, `StartLimitIntervalSec=0`, `OOMScoreAdjust=-500` (the last process the
+OOM killer should pick), `Restart=always` **not** `on-failure` — a clean API-server exit after a
+worker fault reads as success to systemd (the 2026-10-01 vLLM lesson) — and
+`TimeoutStopSec=180` > the `ExecStop=podman stop -t 120` grace, because systemd's 90 s default
+cut the stop short and recorded `Failed with result timeout` (measured 2026-10-01 13:27:17). No
+drain contract is claimed: the OpenAI front end holds no queue, so an in-flight decode is lost
+on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_period`).
+
+| Knob (env name, `FRANK_ARCHON_` dropped) | Live value |
+|---|---|
+| TP / wire | `TP=2`, `TP_WIRE=wht6` — libr4d one-shot all-reduce, not RCCL; `min-kb` stays the engine default |
+| Slots / window | `MAX_SEQS=3`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) |
+| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds 3 slots (Laws) |
+| VRAM pair | `VRAM_WEIGHTS_MIB=25600` + `VRAM_KV_MIB=5120` = 30,720 of the 30,969 ceiling; `GPU_HEADROOM_MIB=96` |
+| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=44928` **per rank** |
+| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
+| Spec / batch | `SPEC_TOKENS=2` — the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); the publisher's card ships depth 3 as its peak and this fleet's decode ladder was measured at 2. `MAX_BATCHED_TOKENS=2048` |
+| Prefix cache | `PREFIX_HOST_MIB=8192` + `PREFIX_DISK_MIB=65536` under `STATE_DIR=/home/dank/radiance-state` (mounted rw at `/data`) |
+| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
+| TTM gate | `TTM_MIN_MIB=150000`, `TTM_TARGET_MIB=204800` |
+| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
+
+Boot ledger for this shape (2026-10-15, pin 1.0.13): pool 5.00 GiB backs **607,740 tokens**
+(`kv_attn` fp8 page 26.00 KiB, `kv_qsa_bkey` 3.25 KiB), `kv_gdn_state` 54 MiB × 6 slots, slab
+**17,548 slots** with 4.22 GiB of KV lent to it, sampler **9 rows**, template 28,234 B, vision
+live (≤ 4,194,304 px, 16,384 patches/pass).
+
+The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
+anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
+the P2P answer per pair) — read it after every resize, and unlike every predecessor there is no
+per-boot VRAM planner to second-guess. For comparison, the card's own suggested run line on 32
+GB cards is `--host-pool-mib 12288 --expert-vs-cache-ratio 0.82` with seqs 8 and spec 3; the
+shape above is the fleet's measured one. `FRANK_ARCHON_EXTRA_ARGS` is the boot-ladder escape
+hatch (unset → no extra argv; it is expanded unquoted on purpose, so keep multi-flag values as
+one string).
+
+**Harness contract** (`/home/blank/.omp/agent/models.yml`) — provider `llamacpp (dankserver)`
+→ `http://192.168.0.235:7770/v1`, `api: openai-completions`, `auth: none`, `id: frank-archon`
+(= `FRANK_ARCHON_SERVED_NAME`; the provider *label* is legacy and renaming it breaks
+`config.yml` references). `contextWindow: 200000` must equal `MAX_MODEL_LEN`;
+`maxTokens: 16384` (clamp `min(maxTokens, window − estimate − 4096)`; compaction fires at
+85 % → 170,000 est × 1.03 + 16,384 ≤ 200,000); `input: [text, image]`;
+`compat.reasoningContentField: reasoning_content`; `reasoningEffortMap` minimal/low→low,
+medium→medium, high/max→xhigh; `extraBody` temperature 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
+presence_penalty 0 / repetition_penalty 1.0 + `chat_template_kwargs {enable_thinking,
+preserve_thinking}`, every key inside the engine's known-parameter list (unknowns 400).
+`compactionModel` routes to `llamacpp (dankclaw-3090)/frank-zealot`.
+
+**Deploy / verify** — `./scripts/deploy.sh dankserver --service frank-archon`; launch it
+detached when the swap restarts the endpoint your own session runs on. Then verify
+**behaviourally**, not by the green gate: `podman inspect frank-archon --format '{{json
+.Config.Cmd}}'` and `/v1/models` — `deploy.sh` plans from repo paths and gates on the port, so
+host-side unit drift is invisible to it.
+
+## Eras
+
+One row per retired era on this port. The **Rollback** cell is the load-bearing one: anything
+marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `StillDeadcode`,
+`rad` and `tpl.jinja`.
+
+| Window | Payload | Runtime | Shape | Replaced because | Rollback |
+|---|---|---|---|---|---|
+| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
+| 2026-10-05 → 10-08 | `qwen4exp-fp8-native.rad`, 177,109 MiB (173.0 GiB) — native modelopt block-FP8, no requantisation | radiance 1.0.7 line, same digest | TP=2 wht6, 200,000 × **2** slots (3 later), slab 26,071 + fp8 KV pool 3,543, host pool 40,960/rank, `--expert-vs-cache-ratio 0.82`, spec 3, MNBT 2048; that pair measured 30.08 t/s decode on the 2026-10-06 43.6k greedy probe and 4,379 t/s A-cold prefill on 2026-10-11 | native FP8 experts (56.4 GiB/rank) ate a 40 GiB pool and left KV at 3,543 MiB = 386,904 tokens, **1.93 sessions** — context economics, not quality; and the host pool could not be trimmed to pay for it (36,864 refused the boot) | rebuild-only: `scripts/download-frank-archon.sh` (official `Qwen/Qwen3.8-Flash-Next-FP8`, 131 shards, 172.8 GB, ungated) + `scripts/build-frank-archon-rad.sh` (no-recipe convert under the descriptor law); raw **and** `.rad` were `rm`'d 2026-10-15 (Frank's call, to free the volume for the iq4r download) |
+| 2026-10-05 (same day) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` safetensors, 131 shards | FreeToken PR #587 head `77a7201` (`rocm714_torch214`), image `localhost/frank-primary-freetoken:pr587-77a7201` | ONE R9700; KV sized in tokens `299,968` (a multiple of 64 — the `qsa_sparse` backend pins page 64), 2 running requests, `--moe-cache-rate 0.10`; 17.1 t/s decode | TP=2 cannot boot on ROCm: the per-rank `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201, so the second card idled | git tag `frank-archon-freetoken-pr587-20261005` (trio + `scripts/docker/frank-primary-freetoken.Dockerfile`, which exists only there); image still in rootless storage; payload needs a re-fetch, and that repo 403s until the gate is accepted on the account |
+| 2026-09-29 → 10-05 (interrupted 09-30 → 10-01 by GLM) | same orcarouter FP8 safetensors | tcclaviger vLLM fork, `docker.io/tcclaviger/vllm@sha256:da402966…` | shape G: 3 × 200,704 fp8, KV 3,166,500,000 B/rank → 401,408 tokens, arena 17.77 GiB / 7,762 slots, util 0.982, MNBT 2048, no spec, no prefetch, prefix caching ON | radiance gave real TP=2 (no RCCL) and a 2-3 min boot, and the fork never fixed its long-context class | git tag `frank-archon-vllm-shapeG-20261003` (trio + the 362-line env lore); image still in dank's rootless storage; payload needs a re-fetch. The 2026-10-01 fp8+MTP4 rung is tag `frank-archon-fp8-mtp-20261001` |
+| 2026-09-30 → 10-01 | `Intel/GLM-5.3-Flash-W4A16-AutoRound` ~170 GiB (sym int4 g128, `auto_gptq`) | same tcclaviger digest; `R4dW4a16MoEMethod` → libr4d grouped W4A16 | TP=2 + `--enable-expert-offload --expert-precise-memory`, 2 × 131,072 bf16 KV, util 0.98, MNBT 2048, 2,383 slots + 73.64 GiB pinned per rank; 9.9-10.0 t/s decode / 918 t/s prefill | the 35 t/s target is unreachable as the fork ships: 44 % of the ~93 ms/token budget is pointwise glue in an **uncompiled** GLM trunk — the expert gather is 1.05 % of device time | payload (170 GiB) + drafter (2.2 GiB) went to `~/.AGENTTRASH/20261001-glm-purge/` and were emptied; the cutover was reversed by `d91379c`, restoring the `2339fae` trio; `scripts/patches/glm53-*.patch` survive in-tree but none applies byte-exact since the 2026-10-10 rename rewrote their context lines |
+| 2026-09-27 → 09-30 | unsloth `Qwen3.8-Flash-Next-GGUF` Q8_0, 175.30 GiB / 6 shards + blockfp8 MTP4 head + mmproj; PLE derived 54,400,261,120 B | R9V radiance-vLLM fork `drwolfen/radiance-vllm-qwen4exp`, docker bundle `2dac17a2`, behind the docker→podman shim | `MAX_MODEL_LEN=131072`, 80-slot tiered cache (the policy max), MTP4, PLE **pinned**, prefix caching ON from 09-28 | the tcclaviger fork took the port for the GLM W4A16 experiment; Q8_0 had halved expert residency and a 128K envelope | host state survives: `/home/dank/r9v-state/q8xl/setup.json`, the fork checkout `/home/dank/GitPackages/radiance-vllm-qwen4exp` (local-only commits `363d606`/`30d4a32`/`2de0357`) and `.podman-shim/docker`; payload re-fetch first |
+| 2026-09-27 (same day) | `Dyluhn/Qwen3.8-Flash-Next-Uncensored-R9V-IQ4_XS`, 92.39 GiB + PLE 26.82 GiB (`34fa36f8…`) | same R9V bundle + shim | `MAX_MODEL_LEN=196608`, offload budgets 146.25 / 78.75 GiB, PLE residency `ssd` mmap, tool parser `qwen3_coder`, thinking on the `reasoning` (vLLM) channel | same-day flip back to the reference quant with the MTP4 head, at Frank's direction | `/home/dank/r9v-state/uncensored/` (`setup.json`, `restore-config-20260927.env`) is still on disk; the IQ4_XS payload was deleted in the 2026-10-17 cleanup → re-fetch, point `R9V_STATE_FILE` back, drop the two q8-only keys and the template keys |
+| 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy |
+| 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet |
+
+Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-medium-radiance-mxfp4]]
+(vllm-radiance TP=2 MXFP4, 2026-09-20 → 09-26) is that era's page and is **not** superseded here.
+
+## Laws learned here
+
+### The radiance engine
+
+- **KV dtype law.** `--kv-cache-dtype` takes only `bf16|fp8` — ONE dtype for the whole cache,
+  so llama.cpp's exact-K / quantized-V pair (what [[frank-zealot]] runs as `--cache-type-k`/
+  `--cache-type-v`) is unservable here: the rung below bf16 is fp8 on both halves. Measured
+  per-rank cost out of the boot ledger's group table (fp8 boot 2026-10-08, bf16 boot
+  2026-10-11): `kv_attn` 13 layers / 4-token blocks pages **26.00 → 52.00 KiB**, while
+  `kv_qsa_bkey` stays **3.25 KiB at both** — the QSA block key is an index, not attention state,
+  and does not follow the flag. So 7.3125 KiB/token at fp8 vs 13.8125 at bf16: **1.89×, not
+  2×**, precisely because the bkey plane is dtype-fixed. Secondary saving: a prefix-cache entry
+  tracks the attention plane (110.50 KiB at bf16 → 58.50 at fp8, per the `kv tier transfer`
+  lines), so the same 8 GiB host tier parks ~5.8 sessions of 200k at fp8 where it parked ~2.9 at bf16.
+- **Pool sizing law.** `pool_mib ≥ tokens × 7.3125 KiB + 110 × seqs + 448` — the dtype-
+  independent terms are also in the pool: `kv_gdn_state` 54 MiB × 2 copies per sequence (+conv/
+  tail/ple ≈ 110 MiB a sequence) and the prefix checkpoint region 8 × 55.99 MiB. Deployed: 3 ×
+  200,000 at fp8 = 4,285 + 330 + 448 = **5,063 MiB**, pool 5,120, which predicts (5120 − 330 −
+  448) × 1024 / 7.3125 = **608,027 tokens** against a 600,000 contract (1.3 % slack) — every
+  MiB above 5,063 was handed to the expert slab instead. At bf16 the same three slots cost
+  8,093 + 778 = 8,871 and the slab falls to 20,249 (−9 %); four slots at bf16 cost 10,791 + 888
+  = 11,679 and the slab falls to 17,441 (−33 %); fp8 keeps 22,208 of slab at both slot counts.
+  **fp8 is not a preference, it is what funds the slot count.** Measured backings to sanity-
+  check the formula: bf16 pool 6,912 at seqs 2 backed 462,752 tokens (2026-10-11), the fp8-era
+  pool of 3,543 gave 386,904 (1.93 sessions), and the four-slot fp8 boot of 2026-10-13 19:21
+  backed 843,276. The ledger's `backs N sequences all at 200000 tokens at once` line is the
+  referee — a pair this tight is read off the boot, never assumed, and the concurrent probe is
+  its regression test. Fourth full-height conversation needs 6,601 MiB of pool: refusing it is
+  Frank's explicit call, and serving it is a `--vram-kv-mib` change, not a code change.
+- **The hard limit is per request.** The engine treats `kv group tokens >= max-model-len` as a
+  HARD limit — "one request longer than N tokens is accepted by the API and then refused as too
+  large to ever fit — this is a hard limit, not a backing ratio" — so the pool is sized for
+  `seqs × max-model-len`, not for one sequence. Radiance never truncates, at any size.
+- **One VRAM budget per card:** `weights + kv ≤ 30,969 MiB`, read off the engine's own budget
+  table (boot 2026-10-13 at pair 24000/5120): card 32,624 MiB, budgets claimed 29,120, and
+  "*3.42 GiB is neither budgeted nor accounted — the driver's context, the activation arena and
+  `--gpu-headroom-mib` come out of it; anything past those is throughput on the floor*", with
+  the card settling 2.01 GiB free against a 209.19 MiB floor (96 headroom + 113.19 kept for the
+  static region). Reserve above the engine's floor = 2,058 − 209 = 1,849, ceiling 29,120 + 1,849
+  = 30,969; the deployed pair lands 458 MiB above that floor. The 2026-10-11 bound of 29,120
+  ("31,856 − 1,710 outside both budgets − keep 1 GiB free") was a hand-picked reserve, not the
+  engine's wall. Boot with the explicit `--vram-weights-mib`/`--vram-kv-mib` pair, never
+  `--kv-cache-size-mib`; with the disk tier engaged the engine also forces ~9.6 GiB of weight
+  buffers + arena onto the weights side (~29.6 GiB/card effective).
+- **Idle KV VRAM is the expert cache's spillover — the price is paid twice.** `mover: 2275
+  slots can be lent out of the KV cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <=
+  256 tokens`; the lend grows with the pool (2.69 GiB from 3.46, 6.08 from 6.912, 4.22 today),
+  so a bigger KV pool steals weight residency from the slab AND from the lendable cache. Priors
+  on this card/model: stealing 2.14 GiB of weights to buy `MAX_MODEL_LEN 262144` cost 2.5 %
+  decode; the other direction (`--expert-vs-cache-ratio` 0.82 → 0.84, +2.5 GiB of weights) cost
+  8.9 %.
+- **A KV-pool cut is a runtime-headroom lever, never an arena lever.** The 6.1e9 → 3.15e9 B/rank pool cut
+  (2.75 GiB/rank) bought **zero** extra expert-cache slots. The arena moves with the reserve, one-for-one
+  (d(reserve) 0.57 GiB = d(arena) 0.57 GiB = **+248 slots**). The shape that pool cut replaced died with
+  `0 bytes free` on a real 995-token request — that is headroom, not slot count. Read `expert cache space … / N
+  slots` out of the boot ledger before trusting any pool arithmetic: the 2026-10-13 read was **14.87 GiB /
+  6,496 slots** against `GPU KV cache size: 398,658 tokens`, and it is the POOL that caps resident
+  conversations, not the arena.
+- **MNBT moves the compile RANGE, not the batch width.** The engine sets `num_reqs = min(max_num_batched_tokens,
+  max_num_reqs)`, so the CUDA-graph compile width is **SEQS + 1** for any `MNBT >= SEQS` — 3 at SEQS 2, 4 at
+  SEQS 3, exactly the constant printed in the engine's own message. MNBT only shifts the compiled range
+  ((1,1024) versus (1,2048)), which is why the identical capture failure appears at both rungs;
+  `--enforce-eager` bypasses the path and loses CUDA graphs with it.
+- **The prefix cache proves itself at depth, not in the ledger.** Two concurrent conversations of 196,919 tokens
+  each answered the marker buried at the head of **their own** prompt with **393,837 tokens resident** — above the
+  fp8 pool's 386,904, so the answer came from a cached prefix and not a re-prefill. Same boot (2026-10-13):
+  decode at 51.5k depth **94.1 cold / 102.3 warm / 117.6 prefix-hit t/s**, draft acceptance **0.7284 / 0.7525**.
+- **The host pool moves with the slab.** `vram_weights` and `host_pool_mib` serve the per-rank
+  resident set *together*: at the fp8-KV pair (26071/3543) the ledger showed the slab full to
+  259 KiB of slack, and cutting `vram-weights` by 3.8 GiB to pay for a bf16 pool spills ~3.8
+  GiB/rank into host RAM — under-sizing the pool there is a **boot failure** (36,864 refused the
+  boot in the fp8 era). The ledger's coverage line, not the arithmetic, arbitrates.
+- **Slab → prefill: the arms disagree, the cliff does not.** 2026-10-11 exact-KV A/B (slab
+  22,208 vs the fp8-era 26,071, −14.8 %): prefill −29 %, TTFT at the full window +25 %, decode
+  inside the noise band on 2 of 3 passes, draft acceptance UP 0.661 → 0.740. 2026-10-13
+  (`probe-frank-archon-kv.py`, env header): A-cold 43.5k 22,208 → 3,397, 24,000 → 3,290, 25,600
+  → **3,584** t/s, and the 3-way 197k arm's aggregate 3,051 → 3,589 (wall 193.7 → 164.5 s), so
+  +1,600 MiB of slab bought +8.9 % single-stream / +17.6 % concurrent — read then as a SLOPE,
+  the 22,208/24,000 pair having landed inside noise of each other and falsely looked like a
+  plateau. Re-measured at 3 slots after the 2026-10-15 swap the 25,600 arm returned 3,290: the
+  3,584 did **not** reproduce, so treat 22,208-25,600 as one band until someone re-runs the
+  ladder. What is not disputed is the cliff: 26,071 → 4,379 t/s (the 2026-10-11 control at seqs
+  3 / pool 3,543), +22 % for +471 MiB — unreachable at 3 slots (26,112 + 5,120 = 31,232 vs
+  30,969) and free at 2 (pool 3,072 + slab 27,897 = the ceiling exactly). Two points inside
+  noise of each other are not a plateau, and two points are not a curve.
+- **Decode is nearly window-free; slots are paid in concurrency.** 99.0 t/s at 43.5k vs 93.3 at
+  197k (−6 % for 4.5× depth), while three streams give 113.9 / 43.3 / 10.9 = 18.4 t/s aggregate
+  — a fused-step lottery, not a fair share. Read it from leg D of `probe-frank-archon-kv.py`
+  with ≥ 128 generated tokens; an 8-token probe measures first-token noise.
+- **MNBT 2048.** 2560 collapses decode to 2.6-7.7 t/s (MTP resonance); the w4 file tier streams
+  in place only at MNBT ≤ 2048 and the activation arena scales with the chunk (32768 → 33.3
+  GiB, refused).
+- **Slots cost KV and rows, never speed.** `--max-num-seqs` carries no radiance constraint: it
+  adds 110 MiB of recurrent state per slot and declares the fused TP all-reduce over `seqs ×
+  (1 + spec)` rows, and `wht6` is bit-exact up to 25 rows (single-sequence decode and any step
+  of ≤ 25 rows is exact; only busier steps are lossy) — 3 × 3 = 9 today. Measured 2026-10-06:
+  seqs 2 and seqs 3 are statistically tied (30.9-33.1 t/s, inside the ±3 % floor). The "seqs 2
+  cannot boot" law belonged to the retired tcclaviger fork's `torch.compile` shape
+  specialization, and radiance booted seqs 2 on its first fp8-native start.
+  **Same era, separate law: a dequantize-to-compute path bills host RAM twice.** On the
+  EMULATION backend an offload request of **118 GiB** landed as **~238 GiB** of real host RAM —
+  the dequantized copy rides on top of the pinned weights, and the doubling is invisible in the
+  weight count. Read the engine log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding
+  "doesn't fit"; the container-side procedure and the kernel gate are [[RDNA4 Container Serving]].
+- **One encoding per expert table** (`rad_arch_declare`): a whole expert kept bf16 is the only
+  thing legal beside the table, so a split tier (down=fp8, gate_up=w4) is unservable.
+- **Descriptor-selection law.** The plugin is picked by the exact `(arch_id, quant descriptor)`
+  pair with no fallback: a raw official FP8 checkpoint (`quant_method: "fp8"`) is refused by
+  serve *and* convert, because the shipped `qwen4exp_fp8` plugin claims `(qwen4exp, "")`. Stage
+  a `config.json` with `quantization_config` dropped — the descriptor changes, the data stays FP8.
+- **Three convert laws, each paid in a full verify+declare cycle.** Scale-name: `scale_name()`
+  appends `_scale_inv` to the *tensor* name, so stacked `…experts.gate_up_proj` keeps its scales
+  at `…gate_up_proj_scale_inv`. File-shadow: the reader opens index-named files sorted and
+  emplaces every tensor each file physically holds, first-wins — a merged PLE shard must sort
+  first (`00-ple.merged.safetensors`) or a raw shard wins the name. Symlink-through: staging
+  symlinks the raw index, so `open(path,"w")` writes through it and corrupts the checkpoint —
+  write temp + `os.replace`.
+- **Convert is card-less** (`rad-convert` declares on CPU, GPTQ solves on CPU BLAS) and
+  `--plan-only` goes first: a plan that resolves every op will boot. `rad-tune` must be
+  declared at the shapes you serve and its cache lives in the *image* (`$RADIANCE_HOME/tune`),
+  so it only survives on the `$FRANK_ARCHON_STATE_DIR/tune` bind-mount the serve script adds.
+- **Quant gates are scored at the production shape** (MNBT 2048, not the recording's 32768)
+  against the BF16 container: KL ≤ 0.03, top-1 ≥ 95 %, and the verdict is chunk-size invariant
+  (arm Δ ≈ the 0.0015 float-order band) and placement-invariant. Protected experts are
+  data-driven — energy share ≥ 3 % AND ≥ 3× routing share, even count per rank half — and the
+  PLE rides the checkpoint's own fixed scale (1.9931793212890625e-04, absmax inside the
+  2⁻¹³..2⁻¹⁰ plateau).
+
+### This host, and the deploy path
+
+- **The pinned-RAM ledger is why this is a root unit:** 2 × 44 GiB expert pools + the 47.7 GiB
+  PLE n-gram table (51B params, read from pinned host RAM over io_uring — Frank's pin law:
+  "*Native FP8. Experts from ram. Pin the PLE to ram.*") + the 8 GiB prefix host tier ≈ **147
+  GiB** charged against `RLIMIT_MEMLOCK`; dank's login session caps at 8 MiB and a user quadlet
+  is capped by the user manager, so only PID1 can grant `infinity` — and
+  `--ngram-placement ram` refuses the start when the limit is short.
+- **TTM is runtime state, and the gate that owns it raises before it refuses.** The pinned pages
+  ride `/sys/module/ttm/parameters/pages_limit` (+ `pages_max` on kernels that expose one; this
+  kernel exposes **none** — the cap is the MIN over the knobs present, and an absent knob is
+  skipped, never read as zero, because a zero would refuse a healthy cap). The kernel default is
+  **half of RAM**, so every raise dies with the reboot: the 2026-10-05 manual raise to
+  52,428,800 pages (200 GiB) was gone after the **2026-10-12 23:17 power failure**, the cap fell
+  to 125,988 MiB, and the preflight refused the start **6,919 times over ~20 h** while
+  `Restart=always` cheerfully re-ran a refusal. A manual sysfs write is therefore not an
+  acceptable repair procedure — `scripts/ttm-preflight.sh` writes `TTM_TARGET_MIB` (204,800 MiB
+  = 52,428,800 pages; pages are 4 KiB so pages = MiB × 256; the *recorded host ceiling*, because
+  writing the 150,000 floor would leave the ledger no slack) into every knob present, re-reads,
+  and only then refuses. Write failures are swallowed deliberately: the caller re-reads, and ttm
+  itself refuses a limit below the pages it has already pinned.
+- **Privilege of that knob, measured the hard way:** it is root-owned 0644 and the service runs
+  `User=dank`, where `-w` is false and a plain `echo` fails. The canonical privileged path is the
+  unit's `ExecStartPre=+…` (the `+` runs it as root even under `User=`); `sudo -n` is the
+  fallback for a human. The file is *sourced* by the serve script (which re-verifies the cap
+  before `podman run`) and *executed* by the unit. A refusal is invisible outside the journal —
+  no container is created, the unit sits in `activating (auto-restart)` on one repeating line —
+  and without the gate the load instead dies mid-pin with a bare allocator error.
+- **The prefix host tier needs real RAM:** 8 GiB of it, and with the service loaded the host
+  still has ~104 GiB available (246 GiB total) — that headroom is what the pinned ledger is
+  measured against.
+- **The deployer's blind spot is host-side unit drift.** A leftover
+  `frank-archon.service.d/bench.conf` overrode both `FRANK_ARCHON_ENV_FILE` and `ExecStart` to a
+  staging worktree and every `deploy.sh` since was green against the old configuration —
+  drop-ins are read *after* the unit's own `Environment=` lines, so they win.
+- **Never `systemctl disable` a unit on these hosts** — it removes the `/etc/systemd/system/`
+  entry, which here is the symlink into the checkout. `mask` is reversible but *fails on a
+  relinked unit*, so a bench window uses `stop` and the deploy restores the unit.
+- **The restore oracle for an experiment window is a conjunction:** `is-active` == active AND
+  `is-enabled` == enabled AND no `/run/systemd/system/<unit>` mask symlink AND the container is
+  owned by the unit and not `Stopping`. The vLLM/GLM-era engines ignored SIGTERM and answered
+  `/health` 200 for 20+ minutes while `Stopping`; a runtime mask survives `stop` and blocks
+  `start`, and a failed unit needs `reset-failed` first.
+- **`MemAvailable` is an input, not a backdrop:** the expert-offload planner reads it, so
+  anything booted beside the live service dies at plan time (`ExpertPlanError: host holds 5113
+  of 12096 swappable experts`) — and fleet law forbids GPU experiments beside production
+  anyway. Long remote work runs as a transient unit or `setsid`+disown: `nohup X &` inside `ssh`
+  races ssh teardown and can die before writing a byte.
+- **The ~40k-token corruption class was the RAM, not the engine.** Dropping DDR5 from 4800 to
+  4600 MT/s ended it, with zero `gdn_hip` faults either side; the prefix-caching / GDN re-seed
+  theory was confounded on both arms. Consumer non-ECC raises no MCE for a silent flip and a
+  60 s / 8 GiB pattern check covers 0.5 % of 246 GiB — silent RAM corruption outranks every
+  software theory.
+- **TP=2 here** is legal since the 2026-09-20 ProArt B850 swap (x8/x8 on CPU root ports,
+  `Routing+`/`ReqEn+` end to end) only because radiance's wire is `libr4d`'s own one-shot
+  all-reduce; RCCL needs atomics on the whole chain and fails `hipErrorIllegalState` without
+  them ([[Dual-R9700-Cooperation]]). `wht6` buys +5.6-6.3 % prefill for ~1.3 % of 8-stream
+  decode, `exact` is the bit-exact alternative.
+- `--prefix-cache-dir` creation is **not recursive** — the serve script `mkdir -p`s the path or
+  the disk tier silently never opens. The tier does survive a payload swap when arch and
+  tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; `cache_n` proves a hit).
+
+### Dead ends — do not re-try
+
+- **radiance 1.2.0 as the serve pin** (`sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa`,
+  commit `e95d397`), benched 2026-10-06 on this exact shape with a 43.6k greedy probe: decode
+  29.21 t/s and draft acceptance 0.7123 vs the 1.0.13 control's 30.08 / 0.7549, and the control
+  reproduced the recorded 0.755 exactly — greedy acceptance is deterministic, so the drop is a
+  real numerics change (the decode loss tracks the acceptance ratio, 0.943 vs 0.945) with the
+  1.2.0 MoE weight-rotation change `92b8aaf` the named suspect. The vram-pair translation itself
+  is exact (the 1.2.0 ledger took 25.46 GiB weights / 3.46 GiB KV per card, byte-identical to
+  1.0.13 at EVC 0.82), and 1.2.0 REMOVED `--expert-vs-cache-ratio` in favour of the pair, which
+  both pins accept. Re-test 1.2.0+ once the rotation change is untangled.
+- **FreeToken TP=2 on ROCm:** the `pynccl.cu` JIT cannot resolve `cuda_runtime.h` for gfx1201 —
+  an untested edge of that PR, not a board fault.
+- **tcclaviger forks:** `--max-num-seqs 2` (`ConstraintViolationError` on `query_start_loc`);
+  `--gpu-memory-utilization 1.00` unbootable, 0.982 is the law; expert prefetch is poison (36 %
+  precision — deleting it was +40-51 % decode); TunableOp-with-offload corrupts numerics
+  (repetition loops, early EOS) — autotuning that changes text is broken tuning; ngram-only
+  drafting dies at init (`PLE inputs were not prepared`); DCP needs TP > `num_kv_heads` (2 here,
+  so 4 GPUs); 6 × 262,144 fits no KV dtype (6 sequences leave 9.18 GiB of pool where fp8 needs
+  12.47). fp8 KV on that path also removes the only MLA decode gfx12 has (the aiter op is gated
+  on CDNA > 2, so the draft step demands `…mla_decode_fwd`, which cannot exist there) and its
+  KV reserve model is dtype-blind (identical 13,600 B/token plan line) — no capacity for the breakage.
+- **DeepSeek-V4.1-Flash on 32 GiB cards:** only `--ds4-prefill exact` is legal,
+  `--ds4-fused-decode` is refused, and at ~22 % resident experts each token pulls ~187 × 12 MiB
+  from NVMe — low GPU util with a pegged link is a *capacity verdict*. Two corrections survive:
+  expert TP **does** work on gfx1201+gfx1201, and an expert-placement JSON outside the mounted
+  `/models` tree gives a silent single-tier fallback.
+- **Which experts you pin does not matter; how many does.** An IBS census of the live server
+  (`scripts/frank-archon-expert-activity.py`) measured per-block read volume flat at 0.93-1.08×
+  of mean across every CPU-resident block — top-k routing fires all of them every token, so at
+  `-ot` granularity the lever is COUNT.
+- **Never mount a diagnostic hook on `PYTHONPATH` without a marker guard:** `sitecustomize` is
+  imported by every interpreter, a child inherits the hook and spawns again — a 19-level python
+  fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation),
+  never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions.
+
+## Contract (the wire)
+
+- **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions`,
+  `/v1/completions`, `/metrics`, `/stats`, `/sessions`, `/dashboard`. `/health` answers 200
+  JSON (model id, `in_flight`, `queue_depth`) while serving and **503 only while stopping**
+  (`core/server/admin.cpp:385`), and the listener binds *after* the full load (`main.cpp`:
+  `engine.start()` then `engine.serve()`) — during the load the port refuses connections, so a
+  bare `curl -fsS …/health` gate is honest and needs no body grep.
+- **Served id.** `FRANK_ARCHON_SERVED_NAME=frank-archon` (the wire rename of 2026-10-09 moved
+  only the alias; container, unit, scripts and env keep the service name). The env's contract
+  line is that **radiance validates the requested model name, so every client — harness, bench
+  scripts, probes — MUST send it** — while the era pages also record that a chat response
+  *echoes* the requested name, which is why `/v1/models` is the served-id oracle and a 200 chat
+  probe proves nothing about what is actually loaded.
+- **Reasoning** lands in `reasoning_content` (`--reasoning-format` defaults to `auto`; no
+  parser flags exist or are needed) and `usage` bills it separately (`reasoning_tokens`).
+  **Thinking consumes `max_tokens`**, so the classic false negative is a probe with empty
+  `content` and `finish_reason: length`/`stop` because the whole budget went inside the think
+  block: either set `chat_template_kwargs.enable_thinking=false` or budget a few hundred tokens
+  and read `reasoning_content`. Same trap, sharper, on the vision probe below.
+- **Sampler surface:** `temperature, top_p, top_k, min_p, typical_p, presence_penalty,
+  frequency_penalty, repetition_penalty, repeat_last_n, chat_template_kwargs, reasoning_effort,
+  enable_thinking, preserve_thinking` accepted; **unknown parameters are refused 400** (nothing
+  unverified can ride the wire, and an `extraBody` typo fails loudly); `max_tokens` is clamped
+  not refused; `response_format: json_object` honoured; **`logprobs` refused**; no `seed` = the
+  engine picks; `n` defaults to `max(8, max-num-seqs)`; stop strings ≤ 64 × 4096 B.
+- **Tools:** the qwen3.8 dialect from the chat template → `message.tool_calls` +
+  `finish_reason: tool_calls`. **`reasoning_effort`:** the froggeric template does not gate the
+  set (xhigh/high/medium/low all 200); the harness map is a safety net, not a requirement.
+- **Vision is native** (the checkpoint's 27-layer Qwen-VL tower through the engine's Qwen-VL
+  processor): an `image_url` content part, no legacy top-level `images` field, ≤ 4,194,304 px
+  and 16,384 patches per pass. Probe with a ≥ 256 px solid-colour PNG and a ≥ 256 token budget
+  — at 64 tokens you get empty `content` and the colour only inside `reasoning_content`.
+- **Capacity behaviour:** a prompt over `--max-model-len` is refused, never truncated; a request
+  longer than the KV pool's token count is accepted then refused as too large to fit; under the
+  3-slot contract a **fourth** full-height conversation is refused, not paged or squeezed.
+- **Observability:** `/metrics` carries vLLM-compatible names plus `radiance:*`, scraped by the
+  Dankclaw Prometheus job at `192.168.0.235:7770` over plain LAN routing (the netavark
+  published-port quirk is same-host only). The boot ledger — the referee for every shape number
+  above — is in the container log (`podman logs frank-archon`);
+  `/home/dank/logs/frank_archon.log` is where the serve script redirects `podman run`.
+
+## Files
+
+- Shape and law owner: `scripts/frank-archon.env` (measured arithmetic in its header),
+  `scripts/serve-frank-archon.sh` (gates image / `.rad` size / template / GPU nodes, retires
+  stale containers, `podman wait`), `scripts/frank-archon.service`, `scripts/ttm-preflight.sh`.
+- Payload doors: `scripts/download-frank-archon-iq4r.sh` (live), `scripts/download-frank-archon.sh`
+  + `scripts/build-frank-archon-rad.sh` + `scripts/stack-qwen38-official-fp8.py` (the fp8-native
+  rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert
+  needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template
+  `scripts/qwen38-froggeric-v22.5.jinja`.
+- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh`,
+  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh`,
+  `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py`,
+  `scripts/make-calib-corpus.py`, `scripts/derive-protected-experts.py`,
+  `scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`.
+- Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so
+  it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`,
+  `scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`.
+- Deploy and tests: `scripts/deploy.sh` / `scripts/deploy-remote.sh` (`deploy_llm_archon`),
+  `tests/frank_archon_unit.bats` (the seams gating the KV-pool arithmetic, the VRAM pair and the
+  ttm gate), `tests/changed_tests.bats`, `tests/deployment_scoping.bats`.
+- Retired-era leftovers in-tree: `scripts/patches/glm53-*.patch`,
+  `scripts/patches/gpu_worker-kv-cap.patch` (fork archaeology; none applies byte-exact today).
+
+Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
+[[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]],
+[[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]],
+[[frank-medium-radiance-mxfp4]], [[LLM/index]].

serverAppConfigs-docs/LLM/frank-dragoon-benchmark.md

@@ -1,128 +0,0 @@
----
-type: reference
-description: "This page records the benchmarks that chose the runtime for `frank-dragoon` on Dankclaw."
-tags:
-  - llm
-  - frank
-timestamp: 2026-09-03
----
-
-> **SUPERSEDED 2026-08-17.** The Frank redo retired this topology.
-> See [frank-redo-2026-08-17.md](frank-redo-2026-08-17.md) and
-> [deployment.md](deployment.md) for the active configuration.
-
-# frank-dragoon — Backend and Spec Benchmark
-
-This page records the benchmarks that chose the runtime for
-`frank-dragoon` on Dankclaw. Every number came from the live
-llama.cpp build on that machine (upstream, b10133+).
-
-## Summary
-
-- **Backend**: **Vulkan0** (2026-08-15).  At 262k ctx Vulkan decodes 48%
-  faster than ROCm (5.68 vs 3.83 t/s); ROCm only wins prefill 119 vs 106
-  t/s.  Generation is the interactive cost, so Vulkan0.
-- **Spec (2026-08-15)**: llama.cpp `--spec-default` preset (ngram-mod,
-  n_match 24 / n_min 48 / n_max 64, draftless).  Replaced DSpark, which
-  crashed the ROCm backend under two concurrent 16k prompts (see
-  [[qwen3.8-spec-matrix]]).
-- **Parallel**: 2 slots — passes the 2x32k parallel stream test.
-- **Context**: 2 slots x 262144.
-- **KV**: `q8_0` K / `q8_0` V on GPU.
-- **Batch**: 8192 / ubatch 256.
-
-> The A/B tables below are retained for history but predate the 2026-08-14
-> Qwen3.8-27B `UD-Q8_K_XL` switch.  The current production spec decision,
-> the DSpark parallel crash, and the high-context ROCm-vs-Vulkan bench are
-> recorded on [[qwen3.8-spec-matrix]].
-
-## 2026-08-14: Spec A/B (ROCm, production config, default sampling, 1.1k-token prompt)
-
-| Spec | Generation (t/s) | Parallel-safe |
-| ---- | ---------------- | -------------- |
-| DFlash (draft-dflash, n_max 8) | ~20.2 (from earlier 4k bench) | **NO** - premature EOS |
-| MTP (draft-mtp, n_max 3) | **15.3 - 19.5** | YES |
-| MTP + ngram-mod (n_max 3) | 12.0 - 13.6 | YES |
-| MTP (draft-mtp, n_max 2) | 12.4 - 14.3 | YES |
-| ngram-mod | 6.7 - 8.7 | YES |
-
-Verdict: `draft-mtp` at `n_max 3` is the winner. It nearly matches DFlash's speed, beats ngram-mod by ~2x, and does not exhibit the DFlash parallel bug. Layering ngram under MTP (12-13.6 t/s) does not pay off, the same result as layering ngram under DFlash.
-
-## Incident: DFlash Premature EOS Under Parallel Load (2026-08-14)
-
-Reproduced three times by the user. When a new task lands on a slot with high prompt-cache reuse (LCP similarity ~0.99, `f_keep` ~0.98) while another slot is generating, the generating slot is released ~1 second later with `finish_reason "stop"` and a truncated response. The client records it as a normal completion (no abort). Server log shows a `slot release` with `truncated = 0` and no `cancel task` line.
-
-Mechanism: DFlash's draft model uses one shared draft context (`ctx_dft`) across all parallel slots. When the new task's first DFlash draft pass runs after a large prompt-reuse operation, it corrupts the generating slot's draft state, and the target spuriously samples the EOG token. Known class of bug upstream (`server : do not default to multiple slots with speculative decoding`, PR #17017; DFlash multi-slot routing fixes in the beellama.cpp fork).
-
-Fix: switch away from DFlash. ngram-mod (draftless) and MTP (separate draft machinery) both pass the strict parallel test. MTP was chosen for speed.
-
-## Backend Comparison (llama-bench, raw target model, no draft)
-
-Both backends ran the same target model at 64k context in `llama-bench`
-(`-p 65536 -n 256`).
-
-| Device   | Prefill 64k (t/s) | Generation 64k (t/s) |
-| -------- | ----------------- | -------------------- |
-| ROCm0    | 220.28            | 6.99                 |
-| Vulkan0  | 72.34             | 7.00                 |
-
-ROCm prefills 3x faster. Generation is a tie in the raw benchmark.
-
-## Full-Pipeline Comparison (4k context, real served chain)
-
-These used `bench-pipeline-64k.sh` driving `llama-server` through the API at
-4k context, reading the server's own `print_timing` lines.
-
-| Spec                        | Backend | Prefill (t/s) | Generation (t/s) | Draft acceptance |
-| --------------------------- | ------- | ------------- | ---------------- | ---------------- |
-| `draft-dflash`              | ROCm    | 148.2         | **20.17**        | 0.50             |
-| `draft-dflash,ngram-mod`    | ROCm    | 149.9         | 16.93            | 0.42             |
-| `ngram-mod`                 | ROCm    | 154.0         | 8.68             | 0.57             |
-| `ngram-mod`                 | Vulkan  | 259.6         | 6.90             | 0.17             |
-
-### Interpretation
-
-- DFlash alone gives the fastest generation on ROCm (20.17 t/s).
-- Layering ngram-mod lowered ROCm generation (16.93 t/s), so it is not
-  layered in production. llama.cpp *can* layer DFlash + ngram
-  (`--spec-type draft-dflash,ngram-mod`), but it was slower here.
-- Vulkan only works without the DFlash draft. The DFlash `ctx_other`
-  setup aborts the Vulkan backend (`ggml-backend.cpp:898: pre-allocated
-  tensor ... cannot run the operation (NONE)`). Vulkan ngram generation is
-  also slowest (6.90 t/s) with poor acceptance (0.17).
-
-**Verdict: ROCm + `draft-dflash`.**
-
-## Memory Constraint at 602100 Context
-
-At `3 x 200700` context the KV cache alone is ~50 GB and the model is
-29.77 GB, leaving only ~16 GB of the ~96 GB ROCm allocation for compute
-graphs. The compute graph scales with **ubatch**, so:
-
-- batch 12288 / ubatch 4096 -> OOM (80.7 GB prefill buffer)
-- batch 6144 / ubatch 2048  -> OOM (40.9 GB prefill buffer)
-- batch 2048 / ubatch 2048  -> OOM (ubatch 2048 too large)
-- batch 2048 / ubatch 512   -> **fits and loads**
-
-The requested batch 12288 / ubatch 4096 is not feasible at this context
-size on this hardware.
-
-## Command to Reproduce
-
-Raw backend bench (ROCm shown; swap `-dev` for the other):
-
-```
-llama-bench -m <main>.gguf -p 65536 -n 256 -c 65536 -dev ROCm0 -r 1
-```
-
-Full-pipeline bench:
-
-```
-./scripts/bench-pipeline-64k.sh ROCm0 draft-dflash
-```
-
-## See Also
-
-- [[deployment]] — deploy and update process
-- [[DFlash DSpark Speculative Decoding]] — DFlash / DSpark / layering details
-- [[index]] — knowledge base index

serverAppConfigs-docs/LLM/frank-dragoon-ciru-orca.md

@@ -1,16 +0,0 @@
----
-type: reference
-title: "CIRU v4.4.1 Orca on Dankclaw (frank-dragoon)"
-description: "The retired frank-dragoon runbook (2026-09-19 to 2026-09-21, rollback target) — CIRU v4.4.1 (QSA window hotfix) serving jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca on the Dankclaw Strix Halo iGPU: the private host build, the runtime gate, the payload, the deployed config, MTP and PLE, the client contract, and the failure modes worth knowing."
-tags:
-  - llm
-  - ciru
-  - dankclaw
-  - frank-dragoon
-  - strix-halo
-  - gfx1151
-  - qwen4exp
-  - rocm
-  - mtp
-timestamp: 2026-09-19
----

serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-native.md

@@ -1,338 +0,0 @@
-# frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 NATIVE abliterated (Dankclaw, Strix Halo iGPU)
-
-**ACTIVE since 2026-10-17.** Rollback = revert the swap commit and redeploy
-(`scripts/frank-dragoon.env`, `scripts/serve-frank-dragoon.sh`,
-`scripts/frank-dragoon.service`); the Swift1.5 payload stays on disk at
-`/home/dank/models/halogen-flashnext-v2/` (its fetch script
-`scripts/download-frank-dragoon-flashnext.sh` stays the door back), and the
-27B-era payload beside that at `/home/dank/models/halogen-qwen3.8-27b/`.
-Era write-up: [[frank-dragoon-flash-server-v2-swift15]].
-
-## Why this payload
-
-Frank's ask: serve
-`https://huggingface.co/Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated`
-with the card's recommended settings, and **save memory**.
-
-This is the **native 2026-10-11 v2 conversion** of orcarouter's pinned BF16
-Qwen3.8-Flash-Next, abliterated — the base itself, converted straight into
-the v2 `.hgn` format. It is **not** the writer transplant that carried this
-filename before: the artifact measures **66,687,678,432 B** where the
-Swift1.5 r2 measured 66,687,678,464 B — 32 bytes apart, same filename
-family, different model. Byte counts and the Hub LFS sha256 are the only way
-to tell them apart, which is why the fetch script and both gates check them.
-
-What the swap buys, measured (table below): retrieval at depth, decode,
-prefill, and **7.4 GiB of RAM handed back**. What it costs: the Swift1.5
-finetune's compressed reasoning and its small perplexity edge.
-
-## The companion law (the reason this swap needed a referee)
-
-The release publishes **only the main file**. The n-gram lookup table
-(REQUIRED — the engine reads it beside the checkpoint), the vision tower,
-and the tokenizer are peonist's, and the card does **not** establish them as
-interchangeable with this native conversion. No source-matched pair is
-published anywhere, so the only available companions are the stock ones.
-
-They are **hardlinked** out of the verified Swift1.5 payload directory
-(`scripts/download-frank-dragoon-v2abl.sh`): a hardlink is a regular file, so
-it satisfies the serve gate's no-symlink law, and it costs no disk and no
-copy next to a 62 GB payload. The engine's own checkpoint tools are the
-referee, and they answered: **NIAH 18/18** (vs 17/18 for the payload those
-files were published with), needle retrieval exact at both depths, the red-PNG
-vision probe exact, perplexity in the normal band. A wrong PLE table does not
-whisper — see Verification.
-
-## The payload — one new file, three inherited ones
-
-`/home/dank/models/halogen-v2-abliterated/`, fetched by
-`scripts/download-frank-dragoon-v2abl.sh` (`-f` + exact byte counts + LFS
-sha256 + `--verify-only`; no token on the command line):
-
-|File|Bytes|sha256|Source|
-|---|---|---|---|
-|`qwen38-flash-next-v2-abliterated.hgn`|66,687,678,432|`b5bd925f…`|Quat3rnion `@main`, fetched|
-|`qwen38-flash-next-ngram.hgn`|51,200,246,144|`3450acada…`|peonist `@5cc17cea`, **hardlinked**, REQUIRED|
-|`qwen38-flash-next-vision.hgn`|897,916,416|`d62e0ae5…`|peonist `@e053f488`, hardlinked, vision ON|
-|`tokenizer/` (6 files)| — |`tokenizer.json` LFS|peonist `@e053f488`, hardlinked|
-
-The engine stays **halogen-flash-server 0.16.2**, digest-pinned (the card
-tested 0.15.0; 0.16.2 is newer in the same line and is what the v2 format was
-qualified on in this fleet):
-
-```
-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
-```
-
-## Shape — the card's settings, and the three deliberate deviations
-
-`HALOGEN_CTX=200704` (per-request ceiling — the card's 262,144 lowered the
-same evening, see below), `HALOGEN_KV_SLOTS=2`,
-**`HALOGEN_KV_POOL_POSITIONS=401408`** (one pool behind both slots, raised to
-2 × ctx on 2026-10-18 — see *Pool doubled for slot parity* below; the card
-recommends it equal to ctx, and that equality is where the memory is),
-`HALOGEN_SPEC_ADAPT=0`, `HALOGEN_MAX_TOK=8192`,
-`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` (coupled pair),
-sampler `1.0 / 0.95 / 20`, `HALOGEN_WEIGHTS_LOCK=1`,
-`HALOGEN_HOST_RESERVE_GIB=20`.
-
-Deviations from the card, each on purpose:
-
-- `HALOGEN_MODEL_ID=frank-dragoon`, not the card's `halogen-v2-abliterated` —
-  the fleet naming law (service = unit = served id).
-- `HALOGEN_SPEC_ADAPT=0` — the card is silent; this is a measured 0.16.2 law
-  (the adaptive default disables the drafter per request and halves decode).
-- the sampler values are peonist's engine recipe; Quat3rnion publishes none.
-
-**The pool is one budget, not a partition.** Boot ledger form: `listening on
-127.0.0.1:8730, 2 slots over ONE <pool>-position KV pool, each request up to
-<HALOGEN_CTX>`. Both numbers were lowered to **200,704** the same evening
-(the card ships 262,144; Frank took the RAM — nothing else in the fleet asks
-for the bigger window), so `models.yml` `contextWindow` moved to **155,648**
-= ceiling − 45k with it. With pool == ctx, one maximum-length request fills
-the pool: **two long conversations can no longer share the endpoint** (two of
-≤100k can). That is acceptable here because frank-dragoon carries the `slow`
-(medium-thinking) role and the `security-reviewer` subagent, not the
-interactive workhorse (that is frank-archon).
-Frank lifted exactly that concurrency limit on 2026-10-18 and the lift is
-**deployed** (section below): the pool is again `slots × ctx`, so two
-maximum-length conversations fit the endpoint at once.
-
-**Memory ledger** (121.2 GiB unified), from the boot ledgers. The 22:20 boot
-printed `pinned 62.11 GiB in 1 range(s) in 1.6 s (41.0 GB/s)` trunk,
-`0.84 GiB` vision tower, `reserving the KV pool, 262144 positions (about
-7.2 GiB)`, and the engine's own sum: `memory: 62.9 GiB of weights locked in
-RAM (0.8 of them the vision tower's), 7.2 GiB of KV pool, …, 75.8 GiB in
-all`. The cut boot printed the same shape at the new size: `reserving the KV
-pool, 200704 positions (about 5.5 GiB)` and `… 5.5 GiB of KV pool, 5.7 GiB of
-working memory, 74.1 GiB in all`.
-The n-gram table rides as page cache — `mapped 51.2 GB,
-NOT pinned (Pin::None)` — which is exactly why every pool cut is a *double*
-saving. FLAGS.md law: **RAM the pool takes is RAM that table loses**; the ~9
-GiB the two cuts together handed back is page cache for the 47.68 GiB table
-that long prompts read.
-
-`HALOGEN_FLASH_PIN_TRUNK=0` is **rejected by the v2 loader** — the trunk
-stays pinned. Disk-backing the weights is not a lever on this format.
-
-## Pool doubled for slot parity — committed AND deployed 2026-10-18
-
-`HALOGEN_KV_POOL_POSITIONS` **200,704 → 401,408** (commit `2b12716`), Frank's
-call: frank-dragoon's total in-flight context was **half** frank-zealot's —
-the llama.cpp endpoint runs one 401,408-position pool split into two 200,704
-conversations, while dragoon ran one 200,704-position pool behind its two
-slots, so one maximum-length request filled it and any second long
-conversation had to stay under ~100k (commit message;
-`scripts/frank-dragoon.env:26-27`, `:183-186`).
-
-- **The per-request ceiling does NOT move.** `HALOGEN_CTX` stays 200,704, the
-  native wall stays 262,144 (`HALOGEN_ROPE_YARN` law), and `models.yml`
-  `contextWindow` stays **155,648** — the pool is the shared budget behind the
-  slots, not the window. The serve seam enforces `HALOGEN_CTX ≤
-  HALOGEN_KV_POOL_POSITIONS ≤ HALOGEN_KV_SLOTS × HALOGEN_CTX`
-  (`scripts/serve-frank-dragoon.sh:85-95`); 401,408 is `slots × ctx`, the top
-  of the legal band, and exactly frank-zealot's pool number — parity, not
-  approximation.
-- Measured reservation **11.0 GiB** — the boot ledger prints `reserving the KV
-  pool, 401408 positions (about 11.0 GiB)`. The preflight KV-budget line bills
-  **11.3 GiB** (`~28 KiB/position incl. block scratch + ~115 MiB/slot`) plus
-  1.3 GiB of prompt cache. The measured points beside it: 5.5 GiB at 200,704,
-  7.2 GiB at 262,144, and 14.6 GiB at the 2026-10-06 shape's 524,288
-  (`scripts/frank-dragoon.env:49-52`, `:74-76`).
-- That RAM comes out of the n-gram table's page cache — FLAGS.md law: RAM the
-  pool takes is RAM that table loses.
-
-**Deployed 2026-10-18** (`./scripts/deploy.sh dankclaw --service
-frank-dragoon`, Playwright gate 33/33). The boot ledger confirms the shape:
-`reserving the KV pool, 401408 positions (about 11.0 GiB)`,
-`2 slots over ONE 401408-position KV pool, each request up to 200704 (it
-reserves prompt + max_tokens)`, and the sum
-`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB
-of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`. Prompt cache ON (12
-entries × 111 MiB at ctx 200704 = the 1.3 GiB in the KV-budget line), n-gram
-47.68 GiB still disk-backed (`Pin::None`), no clamp line. Host `MemAvailable`
-fell 26 → 22 GiB across the raise, which is the FLAGS.md collision being paid
-in page cache. **Warm reuse survived it — measured after the deploy.** One
-**101,987**-token wikitext prompt through `/v1/chat/completions` took **81.5 s**
-cold (1,251 t/s of prefill) and **0.3 s** on each of two repeats:
-`HALOGEN_CACHE_INPLACE=1` keeps the KV rows in the pool, so a warm turn prefills
-nothing and the doubling evicted nothing the cache needed. (The pre-raise
-136.7× figure was a `ttft` reading; this engine sends **no `Server-Timing`**
-header and has **no `/completion` path** — `/ping`, `/props`, `/tokenize` are
-404 too — so wall-clock around the POST plus `usage.prompt_tokens` is the only
-instrument available here.)
-
-## Measured A/B — same battery, both boots (2026-10-17)
-
-Behavior probes ran against the live service (tagged `swift15` / `v2abl`);
-the engine tools ran with the service **stopped**, per the fleet benchmarking
-law, under one identical knob set (`scripts/census-frank-dragoon.sh`).
-
-|Probe|Swift1.5 r2 (out)|native v2 (in)|
-|---|---|---|
-|`verify` on the file|—|OK|
-|NIAH 18 cases (4k/32k × 3 depths × 3 positions)|17/18 (94.4 %)|**18/18 (100 %)**|
-|wikitext-2 PPL (297,053 ids, 2 × 262,144 seqs)|**3.8772**|3.9679 (+2.3 %)|
-|KL of the new model against the old model's logits|—|mean 0.155 / median 0.057, top-1 85.1 %|
-|refusal census (5 hostile prompts)|0/5 refusing|0/5 refusing|
-|thinking tokens, mean of 6 questions (temp 1.0)|**784**|1,476 (×1.9)|
-|tool call, 1-turn / 2-turn|`{"city": "Seattle"}` / answer|`{"city": "Seattle"}` / answer|
-|vision (512 px solid red, `image_url` part)|Red|Red|
-|needle @21,568 tokens, planted 0.85 / 0.10|hit / hit|hit / hit|
-|decode, greedy, 512 tokens|46.8 t/s (spread 25.6–46.8)|50.0 t/s (steady 50.0/50.2)|
-|prefill, cold, 59,024 tokens|1,424.8 t/s|**1,604.3 t/s (+12.6 %)**|
-|prompt-cache ratio (same prompt, second pass)|136.7×|131.6×|
-|KV pool|14.6 GiB (524,288 positions)|**7.2 GiB (262,144)**|
-
-Read the KL honestly: it is a **model-vs-model** divergence, not the
-quant-vs-source comparison the tool was built for. Two different models on
-the same text *should* differ; median 0.057 with 85 % top-1 agreement is the
-signature of two sound models that disagree on content, and the worst rows
-are date/name completions. A n-gram table mismatched to the checkpoint
-corrupts the forward pass and shows up as elevated KL **everywhere** plus
-degraded retrieval — which NIAH (18/18) and the needle probes rule out.
-
-The decode line needs care: the baseline's own spread was 25.6–46.8 t/s
-because its samples were taken while the 62 GB fetch was still streaming into
-page cache, and the candidate's **first** sample after the battery's long
-prompts read 32.8 before settling at 50.0/50.2. Honest claim: **no decode
-regression, plausibly faster**; never quote either single number.
-
-## Verification — how to reproduce
-
-```
-# 1. payload: bytes + LFS sha256 + the hardlinked trio
-bash scripts/download-frank-dragoon-v2abl.sh --verify-only
-# 2. engine tools on a STOPPED service (fleet law: no GPU experiment beside
-#    a live endpoint; stop the unit first — mask fails on a relinked unit)
-bash scripts/census-frank-dragoon.sh verify    # no GPU, safe while serving
-sudo systemctl stop frank-dragoon
-bash scripts/census-frank-dragoon.sh ref       # reference dump, old payload
-bash scripts/census-frank-dragoon.sh cand      # PPL + KL/top-1 vs that dump
-bash scripts/census-frank-dragoon.sh niah-cur  # baseline retrieval
-bash scripts/census-frank-dragoon.sh niah-new  # candidate retrieval
-# 3. boot the candidate through a transient drop-in, then the battery
-bash scripts/trial-frank-dragoon.sh candidate
-python3 scripts/battery-frank-dragoon.py v2abl all
-bash scripts/trial-frank-dragoon.sh restore    # back onto the repo env
-```
-
-The battery needs two staged assets in `/tmp/halogen-eval/` (a 512 px solid
-red PNG as base64, `red512.b64`, and the wikitext-2 raw test split,
-`wikitext-2-raw-test.txt`). Laws it encodes:
-
-- **the engine bills wikitext at ~4.33 characters/token** — the harness
-  chars/4 estimate under-bills by ~8 %, and a prompt over `HALOGEN_CTX` is
-  refused with a 400 that names the exact overage (never truncated). Size
-  haystacks in characters against a **token** budget.
-- the wikitext-2 raw test split as staged joins rows with a **single**
-  newline (2,891 lines, zero blank lines), so a blank-line split returns one
-  1.29 MB "paragraph" and every budget silently becomes "everything".
-- the first request after a boot is a warmup artifact; the battery discards
-  it (`warmup` row) and never measures it.
-- `ppl`/`niah` **load the model** → stopped service. `verify` reads the file
-  back with no GPU and can run while serving.
-
-## Contract (unchanged from the Swift1.5 era)
-
-`reasoning_content` carries the thinking and counts against `max_tokens`
-(≥512-token budget on any vision probe or the endpoint looks blind while it
-is mid-thought); vision via OpenAI `image_url` content parts only, ≥256 px
-probes; `response_format` unsupported; the chat template rejects
-`reasoning_effort` outside `xhigh|medium|low` (the `models.yml` fold stays
-mandatory); unknown sampling params are a 400; the prompt cache is on by
-default, so a warm 100k follow-up costs seconds (measured after the pool raise:
-**81.5 s cold vs 0.3 s warm** at 101,987 prompt tokens). The HTTP surface is
-thin — `/health`, `/metrics`, `/v1/models` and POST `/v1/chat/completions`
-answer, while `/ping`, `/props`, `/completion` and `/tokenize` are 404 and no
-response carries `Server-Timing`: llama.cpp's timing instruments do not exist
-on this engine, so measure wall-clock plus `usage.prompt_tokens`. See
-[[frank-dragoon-flash-server-v2-swift15]] for the engine-flag detail that
-carries verbatim, and [[deployment]] for the deploy/health gate.
-
-## Production boot — the deploy ledger (2026-10-17 22:20, `fcd7630`)
-
-`./scripts/deploy.sh dankclaw --service frank-dragoon`: healthy after ~12 s,
-deploy marker `fcd7630`, e2e gate 33/33. The trial drop-in went to
-`~/.AGENTTRASH/20261017-v2abl-trial/` FIRST, so this boot is governed by the
-repo env — `systemctl show -p DropInPaths` is empty.
-
-Ledger lines that settle the shape (engine stdout, not the journal):
-
-|Line|Figure|
-|---|---|
-|preflight|`MemAvailable now 105.4 GiB` against the engine's own estimate of 62.1 weights + 6.0 working + 7.4 KV + 1.3 prompt cache|
-|weights|`62.9 GiB of weights locked in RAM (0.8 of them the vision tower's)`|
-|KV pool|`reserving the KV pool, 262144 positions (about 7.2 GiB)`; `the KV is ONE pool of 262144 positions (6656.0 MiB) carved into per-request regions`; slots `1 -> 2 POOLED, 111.0 MiB each of O(1) state`|
-|working memory|5.7 GiB at `HALOGEN_MAX_TOK 8192`|
-|total|`75.8 GiB in all` — ~25 GiB of the 121.2 GiB box left for co-tenants|
-|n-gram|`47.68 GiB from /models/qwen38-flash-next-ngram.hgn`, `checkpoint: mapped 51.2 GB, NOT pinned (Pin::None)` — disk-backed exactly as designed|
-|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, `HALOGEN_CACHE_INPLACE=1` (the KV rows are never copied)|
-|spec decode|`MTP depth-2 selectable (drafter 1, speculates while alone)`|
-|clamps|none — the requested pool is the delivered pool|
-
-Read the two KV figures together: the engine's preflight FORMULA bills 7.4 GiB
-(~28 KiB/position **including** block scratch and ~115 MiB/slot of O(1) state)
-while the RESERVATION is 7.2 GiB / 6,656 MiB. Quote the reservation for the
-RAM ledger and the formula when comparing pool sizes.
-
-Live probes on the deployed endpoint: `27*59` → `1593` with a warm thinking
-channel, tool call `{"city": "Seattle"}`, 512 px solid-red PNG → **Red** (320
-prompt tokens — the image reached the model), greedy 512-token decode 46.9 t/s
-on the FIRST request after boot, i.e. the warmup artifact; the trial's settled
-figure is 50.0. OpenWebUI's `frank-dragoon` row description re-keyed to the
-native payload (backup `webui.db.bak-20261017b`; no container restart forced —
-forcing it logs every family session out for one cosmetic string).
-
-## Ceiling lowered to 200,704 — the second production boot (`7c4dbeb`)
-
-The same evening, after the ledger above was read: `HALOGEN_CTX` and
-`HALOGEN_KV_POOL_POSITIONS` 262144 → **200704** (49 × 4096, the per-slot rung
-the llama.cpp endpoint frank-zealot already runs). The checkpoint's native
-window is 262,144 and the engine serves anything below it, so this is a RAM
-decision, not a capability limit. Nothing in the fleet asks for the bigger
-window; the harness moved WITH the ceiling (400-zone law), `models.yml`
-`contextWindow: 155648`.
-
-|Line|Figure|
-|---|---|
-|marker|`revision=7c4dbeb…`, `status=deployed`; unit `active`, `/health` 200|
-|KV pool|`reserving the KV pool, 200704 positions (about 5.5 GiB)`|
-|shape|`2 slots over ONE 200704-position KV pool, each request up to 200704 (it reserves prompt + max_tokens), max_tok 8192 … MTP depth-2 selectable (drafter 1…)`|
-|working|5.7 GiB (unchanged — it scales with `HALOGEN_MAX_TOK`, not the window)|
-|total|`62.9 GiB of weights locked … 5.5 GiB of KV pool, 5.7 GiB of working memory, 74.1 GiB in all`|
-|prompt cache|ON, resume-anywhere, 12 entries, chunk 32768, **111 MiB each at ctx 200704**|
-|clamps|none — requested pool == delivered pool|
-
-The refusal contract at the new ceiling, asked for explicitly (the 1.29 MB
-wikitext corpus through `/v1/chat/completions`):
-
-```
-HTTP 400 {"error":{"message":"the prompt does not fit: it is 297109 tokens and
-the context is 200704, 96405 tokens over. Shorten the prompt, or raise the
-context (HALOGEN_CTX, and HALOGEN_ROPE_YARN for more than the native 262144).",
-"type":"invalid_request_error"}}
-```
-
-Refused in 0.86 s — the preflight rejects before any prefill, so probing the
-ceiling costs the endpoint nothing. OpenWebUI's `frank-dragoon` row description
-re-keyed to "ONE 200,704-position KV pool at ctx 200,704" (still no container
-restart; `webui.db.bak-20261017b` holds the pre-cutover row).
-
-**What the cut cost:** with pool == ctx one maximum-length request fills the
-pool, so two 200k conversations never coexist (two of ≤100 k do), and the
-harness window is now 155,648 — the 180 k compaction target of the 262,144 era
-is unreachable because the window itself is smaller. (That pool == ctx
-concurrency limit IS superseded: the doubling deployed the next day — see
-*Pool doubled for slot parity* above. The window math stands, though: the pool
-never moves the per-request ceiling.)
-**What it bought:** 1.7
-GiB back as page cache for the 47.68 GiB n-gram table (9.1 GiB against the
-2026-10-06 shape's 524,288 pool), and a smaller prompt-cache entry.
-
-**Third production boot — the `401408` pool, deployed 2026-10-18.** Same ctx,
-same working set, same 111 MiB prompt-cache entry; the pool line reads
-`reserving the KV pool, 401408 positions (about 11.0 GiB)` and the total
-`79.6 GiB in all`. Ledger and the measured cost in
-*Pool doubled for slot parity* above; host `MemAvailable` after that boot:
-22 GiB.

serverAppConfigs-docs/LLM/frank-dragoon-flash-server-v2-swift15.md

@@ -1,197 +0,0 @@
-# frank-dragoon — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, Strix Halo iGPU)
-
-**RETIRED 2026-10-06 → 2026-10-17**, superseded by the native v2 conversion:
-[[frank-dragoon-flash-server-v2-native]]. Rollback target — the payload
-stays on disk at `/home/dank/models/halogen-flashnext-v2/`,
-`scripts/download-frank-dragoon-flashnext.sh` re-verifies it, and restoring
-the era is a revert of the 2026-10-17 swap commit. The 27B payload this era
-replaced stays at `/home/dank/models/halogen-qwen3.8-27b/` with
-`scripts/download-frank-dragoon-halogen.sh`. Everything below is the era as
-it shipped; the engine-flag detail still governs the current page.
-
-## Why this shape exists
-
-Frank's ask: replace the frank-dragoon model with
-`https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`.
-That payload is an **abliterated Swift1.5 finetune of peonist's
-Qwen3.8-Flash-Next v2 checkpoint in the v2 `.hgn` format** — a file that
-loads ONLY in the **halogen-flash-server** engine line, NOT in the halogen
-0.1.4 container that served the 27B dense checkpoint. The model move is
-therefore a **full engine migration**, and the accepted-product facts flip
-twice in the fleet's favour:
-
-- the base is **abliterated** (the 27B halogen endpoint was a finetune of
-  the censored `Qwen/Qwen3.8-27B`; refusals were an accepted regression there),
-- **vision is back** — the v2 vision tower loads beside the checkpoint and
-  the engine runs it (the 27B halogen engine silently ignored its encoder),
-- **MTP speculative decoding returns** with the head inside the checkpoint
-  itself (the 27B shipped a DFlash2 drafter instead; comparable purpose,
-  different file plumbing),
-- the card claims **53–57 t/s greedy decode, ~1,640 t/s prefill at 34k, ~72 %
-  fewer thinking tokens** than the censored 27B (905 mean), and **262,144
-  native context**. Publisher-card numbers — verify against live traffic
-  before quoting further (see Verification).
-
-The engine is **closed-source**, ships as `ghcr.io/peonist-ai/
-halogen-flash-server:0.16.2` (the version the model card's quickstart pins),
-digest-pinned:
-
-```
-ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
-```
-
-(3.6 GB image; RepoDigest captured at pull 2026-10-06. Card-tested fallback
-if a future pin misbehaves: 0.15.1.)
-
-## The payload — a triangle of files, all sha256-verified at fetch
-
-| File | Bytes | Source repo @ revision | sha256 (LFS) |
-|---|---|---|---|
-| `qwen38-flash-next-v2-swift15-abliterated.hgn` | 66,687,678,464 | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated @ main (`69a85db`) | `9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037` |
-| `qwen38-flash-next-ngram.hgn` (REQUIRED) | 51,200,246,144 | peonist-ai/halogen-qwen3.8-flash-next @ `5cc17cea` | `3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe` |
-| `qwen38-flash-next-vision.hgn` (enabled) | 897,916,416 | peonist-ai/halogen-qwen3.8-flash-next @ `e053f488` | `d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10` |
-| `tokenizer/` (6 files, flat dir) | 22.4 MB total | peonist @ `e053f488` | `tokenizer.json` is LFS `0997f410…` (byte-identical to the 27B-era copy); the rest are plain git blobs, size-checked |
-
-Payload notes worth keeping:
-
-- The deployed checkpoint is a **compacted r2** of the card's append-only
-  original (89,775,868,032 B → 66,687,678,464 B): the abliteration workflow
-  appends and the repo's `tooling/` compact drops dead padding; live tensors
-  are byte-identical, the LFS sha256 is the contract, and the 24 GiB saving
-  is why it fits.
-- **`HALOGEN_MTP_HEAD` stays UNSET**: the MTP head is INSIDE the v2
-  checkpoint (the separate `qwen38-flash-next-mtp.hgn` sidecar is for a
-  different assembly and is deliberately not fetched).
-- The **n-gram lookup table is REQUIRED**. The engine reads it beside the
-  checkpoint (documented fallback); the card's quickstart also names it
-  explicitly via `HALOGEN_NGRAM_TABLE`, and the serve script passes both —
-  the env var is belt-and-braces over the documented beside-the-checkpoint
-  default, and both mechanisms resolve to the same file.
-- The tokenizer is fetched fresh from peonist @ `e053f488` into
-  `/home/dank/models/halogen-flashnext-v2/tokenizer/` — same flat-directory,
-  no-symlink law as every other payload (an HF cache snapshot's symlinks
-  dangle inside the container mount; both the serve gate and the deployer
-  refuse them).
-
-Fetch: `scripts/download-frank-dragoon-flashnext.sh` (`--verify-only` to
-re-check; `-f` + exact byte counts + LFS sha256 + resolve-the-CDN-then-`-C -`
-resume law throughout; no token — the repos are public).
-
-## Shape — the knobs that are not free
-
-`scripts/frank-dragoon.env` carries the full rationale inline; the table:
-
-| Env | Value | Why |
-|---|---|---|
-| `HALOGEN_CTX` | 262144 | per-slot HARD ceiling (the checkpoint's native window; a longer prompt is a 400 naming the limit, never truncation). **RAISED from 200192 on 2026-10-07 — the 400-zone law:** the harness clamps `max_tokens = min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts the rendered prompt by ~3 % (measured twice: est 152,938 → server 157,388, request over room by 354; est 158,906 → server 165,301, over by 2,300), and compaction only fires at 85 % of the harness window — so with window == engine ceiling, every request in the est ~146k–184k zone was a guaranteed 400. Harness window 217,088 = ceiling − 45k absorbs the error: worst-case request ≈ 219k ≤ 262,144 (≈10 % under-count headroom) |
-| `HALOGEN_KV_SLOTS` | 2 | the 27B engine's "slots>1 disables the drafter" law does NOT carry over — on this engine batching and speculation coexist (the card's quickstart ships 2 slots WITH the head). 2 slots = concurrent capacity |
-| `HALOGEN_KV_POOL_POSITIONS` | 524288 | explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ 14.0 GiB KV |
-| `HALOGEN_HOST_RESERVE_GIB` | 20 | keeps the host honest (the w4b-era lesson) |
-| `HALOGEN_SPEC_ADAPT` | **0** | MANDATORY (card): the default adaptive policy turns the MTP head off for whole requests after low acceptance and **halves decode**. The head stays on; temp-1.0 traffic pays some acceptance for it — accepted trade |
-| `HALOGEN_MAX_TOK` / `HALOGEN_MAX_TOKENS_CAP` / `HALOGEN_QUEUE_TIMEOUT` | 8192 / 65536 / 7200 | 8192 is only the per-request DEFAULT (the harness sends at most its `maxTokens` 45,056); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s) |
-| `HALOGEN_TEMPERATURE` / `TOP_P` / `TOP_K` | 1.0 / 0.95 / 20 | the card's thinking recipe as SERVER-SIDE defaults — the engine HAS a sampler now, so an absent temperature is no longer greedy; models.yml extraBody matches it (consistent, not load-bearing) |
-| `HALOGEN_WEIGHTS_LOCK` | 1 | card's belt-and-braces; the engine pins weights by default on v2 (`HALOGEN_FLASH_PIN_TRUNK=0` is a w4b-only knob and is REFUSED by v2) |
-| `HALOGEN_MODEL_ID` / publish | **frank-dragoon** / 7770:8731 | served id renamed 2026-10-09 (service/unit/env keep `frank-dragoon`); engine port 8730 never published |
-| MTP depth | default 2 | FLAGS.md: 2 is best all-round, 3 for code — Frank's traffic is agent/prose |
-
-**Memory ledger** (121.2 GiB unified, the referee is the boot ledger):
-weights ~62 GiB resident + KV 14.0 + arena ~8 (`3.8 + 8.4 × 8192/16384`)
-+ vision ~2 ≈ **86 GiB engine-side**, + 20 host reserve = ~106 with
-~15 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram
-table rides as mmap'd page cache beside the checkpoint — the card's own
-quickstart arithmetic does not bill it as resident, but WATCH the ledger
-at boot and re-size if the engine prints a clamp. Prompt cache is ON by
-default (`PROMPT_CACHE=2`): a follow-up turn at 100k ctx costs ~2 s
-instead of ~88 s cold — the big-TTFT win the 27B engine needed a manual
-`HALOGEN_CACHE_MB` pin for is built in here.
-
-## Engine contract facts (FLAGS.md of the flash-server repo)
-
-- OpenAI chat **and** `/v1/responses` **and** `/v1/messages` **and**
-  `/v1/completions`; tool calls verified on the card; `response_format`
-  still unsupported (no constrained decoding).
-- Thinking → **`reasoning_content`** (same field as every engine this
-  endpoint has run); the v2 chat template rejects `reasoning_effort`
-  outside `xhigh|medium|low` — the models.yml fold (high→xhigh) stays
-  mandatory.
-- Vision: OpenAI `image_url` content parts only (the fleet's standard
-  contract; the legacy top-level `images` field is the frank-zealot trap).
-  ≥256 px probe images or the model answers "video".
-- `HALOGEN_MAX_TOKENS_CAP` over-cap is a 400 naming the limit — never a
-  truncation. `max_tokens` INCLUDES thinking.
-- All flags read once at startup; every flag is BITWISE except
-  `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`,
-  `HALOGEN_INDEXER_BUDGET`, `HALOGEN_COMPOSABLE_CONTEXT`. Unset
-  `HALOGEN_CHECKPOINT` defaults to `/models/qwen38-flash-next-v2.hgn`
-  with the lookup table beside it.
-- Prompt-lookup drafting (3,3 window) rides beside the MTP head by
-  default and is byte-identical to greedy verification where it fires.
-- Watchdog: `HALOGEN_ENGINE_WATCHDOG_S=180` with defer-logic for memory
-  stalls — on a shared host the defaults are the safe choice; the
-  restart policy stays real through the root unit's `Restart=always`.
-
-Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it); `amdgpu.noretry=-1`
-(auto) — for this IP auto resolves to retry-on, the same effective state
-the 27B engine ran on. Only revisit if the ledger shows SVM complaints.
-
-## Mechanics
-
-- **Root systemd unit** (`scripts/frank-dragoon.service`, `User=dank`,
-  `LimitMEMLOCK=infinity`, `Restart=always`, `ExecStop=podman stop -t 60
-  frank-dragoon-halogen`), driven by `scripts/serve-frank-dragoon.sh` — the
-  fleet's `podman run`-direct pattern. The script runs the container
-  FOREGROUND so the boot ledger (weights resident, pool fit, MTP head,
-  prompt cache, any clamp) lands in `/home/dank/logs/frank_dragoon.log`.
-- Serve gates mirror the deployer EXACTLY (2026-10-01 crash-loop law):
-  image digest in podman storage, byte-exact checkpoint + n-gram table +
-  vision tower, flat tokenizer with all six files as regular files,
-  GPU nodes present.
-  - **Host-RAM pin-gate preflight** (added 2026-10-06): the engine
-  refuses to pin when `MemAvailable` is below the checkpoint (62.11 GiB)
-  plus a **16 GiB floor** — pinned pages cannot be swapped, and under the
-  floor the n-gram lookup pages in from disk on every long prompt. A
-  RAM-short host crash-loops the unit (each boot dies in ~5 s at the pin
-  gate) until the RAM clears; the 2026-10-06 cutover hit this (8 boots,
-  MemAvailable 74.77 GiB vs the 78.11 GiB needed, self-resolved in ~2 min
-  when the dying 27B engine freed RAM). The serve script now checks
-  `MemAvailable` BEFORE `podman run` and WAITS up to `FRANK_DRAGOON_RAM_WAIT`
-  (default 600 s) for the RAM to clear, so a transient dip boots once
-  instead of crash-looping; if it never clears it refuses loudly. The 16
-  GiB floor is the engine's (not `HALOGEN_HOST_RESERVE_GIB=20` — a
-  separate, higher, overall-budget knob).
-- Engine stdout goes to the log, NOT the journal. The FIRST request after
-  a (re)start can sit silent ~45–60 s on this engine class (health
-  `engine_unresponsive`, 503s) and self-recovers — do not restart over it.
-
-## Verification checklist (run after any redeploy)
-
-1. Boot ledger greps in `/home/dank/logs/frank_dragoon.log`: resident
-   weights (~62 GiB), KV pool line, MTP/drafter line, prompt-cache line,
-   **no clamp named**.
-2. `curl http://192.168.0.143:7770/health` → 200.
-3. Text completion probe: reasoning lands in `reasoning_content`,
-   `timings` honest (generate ≥128 tokens; at 8 the inter_token number is
-   first-token noise).
-4. Vision probe: a 512×512 solid-red PNG through an `image_url` part →
-   "Red" (≥256 px, one word expected).
-5. Spec decode: `draft_n`/`draft_n_accepted` present in timings on
-   real-shaped traffic (SPEC_ADAPT=0 means the head is always engaged).
-
-## Client contract (harness)
-
-`~/.omp/agent/models.yml` — provider `llamacpp (dankclaw)` @
-`http://192.168.0.143:7770/v1`: `contextWindow: 217088` — deliberately
-BELOW the 262,144 engine ceiling: the 45k gap absorbs the harness's
-~3 % chars/4 est under-count (see the `HALOGEN_CTX` row above) —
-`maxTokens: 45056`, `input: [text, image]`,
-`reasoningContentField: "reasoning_content"`, the `high→xhigh` effort
-fold, `extraBody` temperature 1.0 / top_p 0.95. Compaction still rides
-frank-zealot (the `compactionModel` steering is unchanged; the input
-≈ 170k fits its 200,704 slot).
-
-## Rollback
-
-Revert the swap commit and redeploy — the 27B trio is intact on disk
-(payload at `~/models/halogen-qwen3.8-27b/`, fetch script, and the halogen
-0.1.4 image digest still in podman storage). The older ROCmFPX rollback
-door (`build-rocmfpx.sh` pin) is untouched.

serverAppConfigs-docs/LLM/frank-dragoon-gufo-qwen38-27b.md

@@ -1,350 +0,0 @@
----
-type: reference
-title: "gufo, Qwen3.8-27B UD-Q4_K_XL (DFlash2) on Dankclaw (frank-dragoon)"
-description: "The RETIRED frank-dragoon runbook (rollback target) — gufo (gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from source on Dankclaw and serving unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL (vanilla) with the z-lab DFlash2 Q4_K_M drafter on the gfx1151 iGPU, ONE session at the full native 262,144 window with unquantized f16 KV: the from-source build and its two host workarounds, the GGUF header evidence, why the fleet's Qwen-Sharp chat template CANNOT be used on this runtime, the client-contract delta, the measured prefill/decode trade, and the failure modes worth knowing."
-tags:
-  - llm
-  - gufo
-  - swift-1.5
-  - dankclaw
-  - frank-dragoon
-  - strix-halo
-  - gfx1151
-  - q8
-  - dflash2
-  - rocm
-timestamp: 2026-09-28
----
-
-# frank-dragoon — gufo, Qwen3.8-27B (UD-Q4_K_XL, DFlash2)
-
-> **RETIRED 2026-10-02 — the rollback target.** Superseded by
-> [[frank-dragoon-rocmfpx-q8agent-mtp]] (Q8_0_ROCMFPX_AGENT at 8.39 bpw with
-> MTP4, 17.0 t/s measured). This era was itself a return after one day of
-> [[frank-dragoon-vllm-mimo-v26-bf16]] (battery decode 6.19-9.01 t/s; every
-> ≥6-bit vLLM quant path dead on gfx1151). The shape
-> moved on the way back: payload is now **unsloth/Qwen3.8-27B-GGUF
-> UD-Q4_K_XL** (16.35 GiB, vanilla — the abliterated Swift Q8 was
-> reclaimed), and the session grid is **one session at the full native
-> 262,144 window** (`SESSIONS=1 CONTEXT_SIZE=262144`, f16 KV). The
-> Swift-1.5 Q8 history below is retained for the engine/template law,
-> which still governs.
-> **Originally ACTIVE 2026-09-28.** Replaces
-> [[frank-dragoon-unsloth-qwen38-27b-q8xl]] (stock llama.cpp, the unsloth
-> b11030 prebuilt), which had replaced
-> [[frank-dragoon-kairic-edge-qwen38-27b]]. Same host, same port (7770), same
-> alias (`frank-dragoon`).
->
-> **The payload also changed on 2026-09-28**, later the same day: the target
-> went from `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` to
-> `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`'s
-> `UD-Q8_K_XL` — an abliterated (uncensored) fine-tune of the same base. See
-> [Payload](#payload) and, for what that costs, [the template
-> law](#the-template-law-qwen-sharp-is-gone).
-
-## Identity
-
-- **Endpoint:** `frank-dragoon` at `192.168.0.143:7770` (Dankclaw).
-- **Runtime:** [gufo](https://github.com/gufo-org/gufo) — "the Strix Halo
-  inference engine", a vertical engine built for exactly this hardware
-  (Ryzen AI MAX+ 395 / Radeon 8060S, `gfx1151`, unified memory). Pinned at
-  **v0.1.1** (commit `b0f8673`), built from source on the host to
-  `/home/dank/GitPackages/gufo-v0.1.1/`.
-- **Service shape:** root systemd unit `scripts/frank-dragoon.service`
-  (`User=dank`, `LimitMEMLOCK=infinity`), launched by
-  `scripts/serve-frank-dragoon.sh` with tunables from
-  `scripts/frank-dragoon.env`.
-
-gufo ships **no prebuilt Linux binaries** (the v0.1.0/v0.1.1 release assets
-are empty) and no Nix on this host, so the runtime is a source build. It is
-also young: v0.1.0 landed 2026-09-28, the same day as this cutover.
-
-## Why this engine
-
-gufo's own measurement of the two engines on the same Qwen3.8-27B targets
-(Strix Halo, `628ed18e` vs llama.cpp `b11069`, greedy, DFlash2):
-
-| Q8_K_XL, DFlash2 | gufo | llama.cpp | delta |
-|---|---:|---:|---:|
-| prefill, depth 0 | 457.05 t/s | 333.85 t/s | **+36.9 %** |
-| prefill, depth 131,072 | 237.24 t/s | 131.80 t/s | **+80.0 %** |
-| decode, depth 0 (mixed) | 16.08 t/s | 15.00 t/s | +7.2 % |
-| decode, depth 32,768 (mixed) | 14.41 t/s | 14.44 t/s | −0.2 % |
-| decode, depth 65,536 (mixed) | 10.87 t/s | 12.62 t/s | **−13.9 %** |
-| decode, depth 131,072 (mixed) | 8.87 t/s | 11.62 t/s | **−23.7 %** |
-| cold load to HTTP-ready | 8.17 s | 36.47 s | **4.5× faster** |
-| peak memory @ 262,144 ctx | 50.58 GiB | 48.60 GiB | +4 % |
-
-Read that honestly: **prefill, load time and short-context decode improve;
-deep-context decode on the 8-bit target gets worse.** The crossover on Q8 is
-around 32k of cached depth. frank-dragoon runs one 262,144-token conversation,
-so a long agent session spends its life on the losing side of that line
-while a short turn wins. The Q4_K_XL target does not show the regression
-(gufo is +11.7 % to +17.0 % at every depth) — that is the honest reason to
-revisit the quantization separately from the runtime.
-
-## Build
-
-`scripts/build-gufo.sh` reproduces the runtime end to end. It clones the
-pinned tag, configures the `release` preset (HIP, `--offload-arch=gfx1151`),
-builds, installs, and finishes with `gufo diagnose`. Two host facts it
-encodes, both found the hard way on 2026-09-28:
-
-- **Ubuntu's `librocwmma-dev` must not be installed.** It ships a truncated
-  rocWMMA header set (the public headers without `rocwmma/internal/`) into
-  `/usr/include`, where it shadows AMD's complete copy under
-  `/opt/rocm/include`. The first build died on `'internal/accessors.hpp'
-  file not found` from `/usr/include/rocwmma/rocwmma.hpp`, and **a `-I`
-  cannot fix it**: in HIP mode the host compilation pass searches the system
-  directories first. The package has no reverse dependencies; the script
-  removes it.
-- **`--gcc-install-dir` must point at GCC 15.** Ubuntu 26.04 has
-  `/usr/include/c++/16`, and the ROCm clang picks the newest tree, whose
-  `<atomic_wait.h>` references `std::__detail::__notify_impl` at a symbol
-  version the installed `libstdc++.so.6` (GLIBCXX_3.4.35) does not export.
-  The link then fails with `undefined symbol`. Pinning the headers to the
-  GCC the runtime library came from fixes it.
-
-Verify an installed runtime with `bash scripts/build-gufo.sh --verify`, or
-`gufo diagnose` directly — it reports `[PASS]` plus the detected
-architecture, driver and queue budget.
-
-## Shape
-
-### Payload
-
-- **Model:** `ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF`,
-  file `UD-Q8_K_XL` — 27B dense, 31,457,990,272 bytes, fetched and
-  sha256-verified (`9076f99a…`, the Hub LFS oid) by
-  `scripts/download-frank-dragoon-swift.sh`. It is an **abliterated
-  (uncensored)** fine-tune of Qwen3.8-27B. The repository also publishes an
-  `mmproj-BF16.gguf`, which is **not** used: this endpoint is text-only.
-- **Licence:** the repo declares `other` — the **SWIFT OPEN LICENSE 1.0**,
-  which is *not* an OSI licence. That is a real constraint on redistribution,
-  noted here so nobody discovers it later. It is fine for a private fleet.
-- **Replaced:** `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL`, still on disk and
-  still the rollback target for the payload.
-
-The two targets are **structurally interchangeable for this engine**, which
-is why the swap needed no engine change. Header-diffed directly:
-
-| | unsloth | Swift-1.5 |
-|---|---|---|
-| `general.architecture` | `qwen35` | `qwen35` |
-| tensors (count, names, order) | 866 | 866, **identical** |
-| per-tensor `ggml_type` | Q8_0×453, F32×360, BF16×53 | **identical** |
-| per-tensor dims | — | **0 mismatches** |
-| MTP head | 4 tensors at `blk.64.nextn.*` | identical |
-| geometry | 65 blk, 5120 embd, 17408 FFN, 24/4 heads | identical |
-
-The two metadata differences that look alarming are both no-ops, and both
-were confirmed against gufo's source rather than assumed:
-
-- **`general.file_type` is 15 vs 7.** gufo's `gguf_reader.cpp` states it is
-  *"deliberately not consulted… The Unsloth UD-\*_K_XL artifacts this repo
-  targets carry a stale one — UD-Q8_K_XL records ftype 15 (Q4_K_M) … The
-  tensor table always is."* The file gufo already served lies about itself
-  here, so gufo reads the tensor table.
-- **Swift adds `qwen35.attention.recurrent_layers`.** gufo never reads that
-  key anywhere in `src/`; layer kinds come from `full_attention_interval`,
-  which both files carry as `4` and which agrees with it.
-
-### Serving shape
-
-- **Context:** one session, full 262,144 tokens (the model's native window),
-  `SESSIONS=1`. The fleet's single stream is unchanged.
-- **Spec method:** DFlash2, gufo's **adaptive** block policy
-  (`--draft-policy adaptive`, `--draft-tokens 7`). The drafter is the
-  **unchanged** z-lab `Qwen3.8-27B-DFlash2-Q4_K_M` (1.07 GiB) — it did not
-  move with the target, which is why `DRAFTER_DIR` is a separate tunable
-  from `MODEL_DIR` and why the deployer's stray-run sweep keys on the
-  drafter's filename rather than the target's.
-- **Sampling:** greedy (`--temperature 0`) server-side, as every prior
-  frank-dragoon served. The harness overrides per request via `extraBody`
-  (1.0 / 0.95 / top-k 20).
-- **Memory:** gufo's own table puts the 8-bit target at **50.58 GiB** peak
-  with context capacity 262,144, against the iGPU's ~114 GiB GTT aperture.
-
-### Measured at the 2026-09-28 cutover
-
-Live on Dankclaw. The **target** numbers are the Swift-1.5 load test
-(a standalone run under the production flags, service stopped first); the
-**engine** numbers are from the gufo cutover earlier the same day and are
-unchanged by the payload swap.
-
-- **9.2 s** `load_started` → `load_completed` for the Swift target, against a
-  20-minute `LLM_WAIT` budget. The unsloth target loaded in 13 s.
-- **66.5 GiB** of the 116.7 GiB iGPU device at context capacity 262,144;
-  host RSS 30.6 GiB. No loader warnings, no skipped sections.
-- Contract battery, all green on the Swift target: `/v1/models` →
-  `context_length 262144`, a chat completion carrying both `content` and
-  `reasoning_content`, a `tool_calls` response with
-  `finish_reason: tool_calls`, SSE streaming, and a **two-turn tool loop**
-  (tool call → tool result → follow-up) with prior reasoning preserved.
-  Arithmetic spot-check: "17*23" → `391`.
-- Behavioural spot-check that this is the abliterated model: asked for a
-  two-sentence lockpicking scene, it wrote one without hedging or refusing.
-- From the gufo cutover, on the unsloth target: a short-prompt live request
-  returned
-  `Server-Timing: ttft;dur=549.8, inter_token;dur=24.2` (≈41 tok/s).
-  Short-prompt decode is gufo's *winning* regime; see the depth table above
-  before quoting it as the endpoint's number.
-
-### Tuning: the served flags are already optimal (2026-09-28)
-
-Frank asked for iterative decode/prefill improvement at a 32k working point
-without touching the model or the quantization. **Result: no win exists at
-this engine's served settings, and nothing was changed.** Measured on real
-repo prose and shell sources at **32,762 prompt tokens**, salted per run so
-a warm prefix cache could not help, 3–5 repetitions each.
-
-| lever | range swept | outcome |
-|---|---|---|
-| `--prefill-chunk` | 512 → 8192 | **no effect** — 410.7–412.4 t/s prefill across a 16× change |
-| `--draft-tokens` | 5 → 10 | **flat** — 13.79–15.86 t/s decode, inside run-to-run noise |
-| `--draft-policy` | fixed vs adaptive | 35.90 vs 35.50 (bench) — no effect |
-| `--min-draft-tokens` | — | **rejected by the engine**: "DFlash2 requires … 1" |
-| MTP backend | — | **unsupported for Q8**: `token_embd.weight` is Q8_0 |
-| `--context` 262144 → 40960 | diagnostic | 13.93 vs 14.71 t/s — no gain, and it would break the declared client context |
-
-The confirmation round at 5 reps **reversed** the 3-rep leaders: the pair
-that looked best (`dt9 + chunk4096`) came last at 12.86 t/s, and the
-**deployed configuration came first at 14.71 t/s**. Every difference sits
-inside the noise band.
-
-What *does* move the number is speculative decoding itself: **6.65 t/s off →
-~15 t/s on, a 5.3× step** — and that is already switched on. Prefill is
-insensitive to everything tried, which is consistent with gufo's own
-`EXPERIMENTS.md`, where dozens of kernel-level alternatives are recorded as
-rejected for "no release throughput improvement".
-
-**Do not repeat this investigation without new information.** Two traps
-cost most of the time, and both are in AGENTS.md's benchmarking policy:
-`gufo bench` reports `acceptance=0.933 ± 0.000` on an identical trace hash
-and **inverts** the draft-width ranking; and a live A/B needs ≥128 generated
-tokens or `inter_token` is first-token noise.
-
-## The template law: Qwen-Sharp is gone
-
-`scripts/qwen-sharp-v22.4.0.jinja` is the fleet's chat template — it is
-wired into `frank-archon` as `R9V_CHAT_TEMPLATE_FILE` and it was
-`frank-dragoon`'s `--chat-template-file` under llama.cpp. **frank-dragoon can no
-longer use it**, and the reason is stronger than "gufo lacks the flag":
-
-> gufo has **no Jinja engine at all.** `src/models/qwen/chat_template.cpp`
-> is 587 lines of hand-written C++ that appends strings to build ChatML.
-> There is no `--chat-template-file`, no environment override, and no
-> template body anywhere in the binary to point at. Qwen-Sharp is 453 lines
-> of Jinja2 using `namespace()`, macros, `raise_exception` and image/video
-> branches — supplying it would mean **reimplementing it in C++**, not
-> handing gufo a file.
-
-Given the choice on 2026-09-28, the decision was to **ship on gufo's stock
-Jinja and accept the loss**. What that gives up, concretely:
-
-| Qwen-Sharp feature | On gufo |
-|---|---|
-| `terse` system-prompt injection (the headline behaviour) | **gone** |
-| in-band `<\|think_off\|>` / `<\|think_xhigh\|>` control | **gone** |
-| `tool_call_format` `json` \| `xml` switch | **gone** |
-| `max_tool_arg_chars` / `max_tool_response_chars` truncation | **gone** |
-| `add_vision_id` → `Picture N:` prefixes | **gone** |
-| effort instruction strings in the system prompt | kept (gufo has its own, same text) |
-| `preserve_thinking` | kept (`--preserve-thinking on`) |
-
-The `terse` injection is the one that matters most here, and it is worth
-being blunt about the combination: this endpoint now runs an **abliterated**
-model *without* the prompt that tells it to stop waffling. If verbosity
-shows up, the cheapest fix is to send the `terse` text as an ordinary
-**system message from the client** (`models.yml` / the harness) rather than
-through a template — that recovers the behaviour without forking the
-renderer. The full-fidelity options are reverting to llama.cpp or porting
-Qwen-Sharp into the C++ renderer; both are worse trades today, and the
-second is the same class of change that produced the 2026-09-21
-"73,600-token prompt" incident.
-
-## Client contract — what actually changed
-
-Everything the fleet depended on still holds: port 7770, alias `frank-dragoon`,
-`reasoning_content`, `tool_calls`, streaming SSE, and text-only (gufo
-auto-discovers a `mmproj` beside the model; this payload has none).
-
-The differences worth knowing:
-
-- **No custom chat template — see
-  [The template law](#the-template-law-qwen-sharp-is-gone) above**, which is
-  the most consequential change on this endpoint. Thinking defaults to on at
-  `xhigh` with prior reasoning preserved (`--think auto
-  --preserve-thinking on`, spelled out in the serve script).
-- **`reasoning_effort` accepts the full OpenAI set** (`none`, `minimal`,
-  `low`, `medium`, `high`, `xhigh`, `max`), all verified 200 against the
-  live endpoint. Qwen-Sharp rejected anything outside `{xhigh, medium, low}`
-  with an HTTP 500, which is why `models.yml` still carries a
-  `reasoningEffortMap` fold for this model. **The fold is kept on
-  purpose**: removing it would silently move every `high` role from `xhigh`
-  to `high` effort. Drop it as its own change.
-- **Every field the harness sends is accepted.** `top_k`, `min_p`,
-  `presence_penalty`, `repetition_penalty` and `preserve_thinking` are all
-  outside gufo's documented request subset, and all returned 200. They are
-  applied from `models.yml`'s `extraBody` unchanged.
-- **No KV-cache quantization flag.** gufo picks its own KV dtype; there is
-  no `--cache-type-k`. The old `CACHE_RAM=8192` tunable is gone with it.
-  Prompt reuse is on by default, and `--cache-disk` (restart-safe) is opt-in
-  and currently off.
-- **Health/readiness split:** `GET /health` is process liveness and is what
-  the deployer gates on; `GET /ready` is model/backend readiness.
-  `GET /metrics` is Prometheus-format.
-
-- **Long bodies:** `--max-request-bytes` defaults to 8 MiB.
-
-## Failure modes worth knowing
-
-- **A missing runtime is a deploy error, not a restart loop.** The handler
-  gates on `LLM_DRAGOON_RUNTIME` *before* it stops anything, and the serve
-  script exits with a `build-gufo.sh` pointer rather than exec'ing.
-- **Host memory is the ceiling, and it is shared.** frank-dragoon and ComfyUI
-  both hold gfx1151 compute queues. gufo reads
-  `/sys/class/kfd/kfd/proc/*/queues/*/type` and caps itself
-  (`GPU_MAX_HW_QUEUES`, 2 resident for `serve llm`) so a full device
-  time-slices instead of failing; four HIP processes is the device ceiling.
-- **Load is seconds, not minutes** (9.2 s measured cold on the Swift
-  target), so a redeploy no longer spends 20 minutes in `LLM_WAIT`.
-- **A payload is two files in two directories.** The target and the drafter
-  no longer live together, so "the payload is present" is not one check; the
-  deployer gates on both paths and the serve script refuses to start if
-  either is absent. A symlinked "one payload directory" would hide that.
-- **A v0.1.x runtime is a moving target.** Rebuilds are not automatic —
-  `RUNTIME_DIR` names a version, and the deployer will not silently
-  substitute another.
-
-## Rollback
-
-Two independent rollbacks, because two things changed on 2026-09-28:
-
-1. **Payload only** — repoint `MODEL_FILE` in `scripts/frank-dragoon.env` at
-   `/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/Qwen3.8-27B-UD-Q8_K_XL.gguf`
-   and redeploy. **The 29.3 GiB target was reclaimed on 2026-09-28**, so this
-   path now needs a re-fetch first: `scripts/download-frank-dragoon.sh`, then
-   the env edit. The DFlash2 drafter was never touched and still lives in
-   that directory.
-2. **Engine** — [[frank-dragoon-unsloth-qwen38-27b-q8xl]]: the unsloth b11030
-   prebuilt is still at `/home/dank/GitPackages/unsloth-llama-b11030/`.
-   Revert `RUNTIME_DIR` and the gufo-only tunables, restore the serve
-   script's llama.cpp command, and redeploy. This is also the only way to
-   get `qwen-sharp-v22.4.0.jinja` back.
-
-# Citations
-
-- Runtime: https://github.com/gufo-org/gufo (pinned `v0.1.1`, MIT)
-- Measured tables: `docs/models/qwen3.8-27b/BENCHMARKS.md` in that repo
-- API contract: `docs/SERVER.md`; flags: `docs/CLI.md`; the C++ renderer
-  that rules out a Jinja template:
-  `src/models/qwen/chat_template.cpp` (587 lines)
-- GGUF header evidence for the payload swap: both files' headers were parsed
-  and diffed directly (architecture, all 866 tensor names in order, per-tensor
-  `ggml_type` and dims) before either was loaded
-- Build: `scripts/build-gufo.sh`; serve: `scripts/serve-frank-dragoon.sh`;
-  tunables: `scripts/frank-dragoon.env`
-- Payload: `scripts/download-frank-dragoon-swift.sh` (target),
-  `scripts/download-frank-dragoon.sh` (DFlash2 drafter)
-- Predecessor: [[frank-dragoon-unsloth-qwen38-27b-q8xl]]

serverAppConfigs-docs/LLM/frank-dragoon-halogen27b-dflash2.md

@@ -1,200 +0,0 @@
-# frank-dragoon — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE 2026-10-05)
-
-frank-dragoon is the Strix Halo iGPU endpoint on Dankclaw (`:7770`). Since
-2026-10-05 it runs the **halogen** engine — `ghcr.io/peonist-ai/halogen` at a
-pinned digest, launched by `scripts/serve-frank-dragoon.sh` under a root
-systemd unit as a podman container — serving
-**`peonist-ai/halogen-qwen3.8-27b`** (`qwen3.8-27b-p1w4d-d2.hgn`,
-35,865,565,184 B ≈ 33.4 GiB, sha256 `274c3fc7…`, 1352 tensors) with the
-**DFlash2** drafter at **one slot x 200,700 ctx**.
-
-It replaced the ROCmFPX llama.cpp fork endpoint ([[frank-dragoon-rocmfpx-q6agent-uncensored-mtp]]),
-which remains the rollback target with its payload on disk.
-
-## Why swap
-
-|Metric|ROCmFPX Q6_0_ROCMFPX_AGENT + MTP (retired)|halogen qwen3.8-27b + DFlash2 (active)|
-|---|---|---|
-|Prose decode|27.6–28.1 t/s measured (4.8k depth, acceptance 0.868)|31.7 t/s mean across eight text workloads (publisher card, bf16 reference hardware)|
-|Copy-echo decode|31.4 t/s|publisher reports echo inside the same 31.7 mean|
-|Context|1 x 262,144|1 x 200,700|
-|Format|GGUF (fork-private tensor types 102/103)|`.hgn` — loads in NO other engine|
-
-The swap was accepted with eyes open on two product regressions, both
-confirmed by Frank before deploy:
-
-1. **Text-only.** halogen loads the language tower only; README "honest
-   limits": *"The vision encoder is ignored; image inputs are not supported."*
-   The mmproj-era vision role moves wholly to frank-zealot.
-2. **Censored base.** The checkpoint is a finetune of the censored
-   `Qwen/Qwen3.8-27B` base, built to make its DFlash2 head strong. The
-   ROCmFPX AGENT file was an *Uncensored* build; refusals can return.
-
-## Shape and why
-
-`scripts/frank-dragoon.env` carries every tunable with its evidence; the load-
-bearing numbers:
-
-- **`HALOGEN_KV_SLOTS=1` + `HALOGEN_DRAFTER=2`** — the engine offers batch OR
-  speculation, never both (FLAGS.md: "Batching and speculation are exclusive:
-  a step batches when more than one conversation is running, so the drafter
-  stays off with more than one slot"). Slots>1 silently loses DFlash2; the
-  sibling halogen engine measured 41 -> 14 t/s when slots went up. frank-dragoon
-  was a one-conversation endpoint before (PARALLEL=1), so the contract holds.
-- **`HALOGEN_SLOT_CTX=200700`** — the per-conversation HARD ceiling: a longer
-  prompt is a 400 naming the limit, never a truncation. The harness side is
-  wired to it: `models.yml` `contextWindow: 200700`, and OMP compacts at 70%
-  = 140,490, leaving 60,210 of the slot for the reply.
-- **`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200`** — coupled
-  pair (FLAGS.md table): a request at the cap must not outlive the timeout.
-  65536 > the harness's 57,344 `maxTokens` so an eager request never meets
-  the engine's 400.
-- **KV math:** 1 slot x 200,700 x 64 KiB/token (d2 KV = 2x fp8) = 12.2 GiB
-  pool, inside the ~65 GiB left after 33.4 GiB mmap'd weights + engine working
-  set + host baseline on the 121.2 GiB box. The engine sizes the pool itself
-  and **prints a boot ledger naming any clamp** — the post-deploy check reads
-  that ledger, not a formula.
-- **`HALOGEN_CACHE_MB=16384` (pinned 2026-10-06)** — auto is OFF on this
-  box: the floor is 2 x a full-context entry (36.9 GB at 262k; one entry
-  is ~13.7 GB at our 200,700-slot x ~70 KB/token) and the ledger had
-  35.7 GB free, so the engine refused and every deep agent request
-  re-prefilled its whole 79-81k context — 84.9-179.8 s measured
-  (serve_api lines, 2026-10-06) for ~500-650 new tokens per turn. The
-  README prices the cache at ~20x time-to-first-token on a warm turn;
-  entries evict LRU by bytes; the reserve (HALOGEN_CACHE_RESERVE_MB,
-  default 8192) is re-checked against live MemAvailable on every store.
-- **No server-side sampler:** an absent temperature is GREEDY. The card's
-  thinking recipe (temp 1.0 / top_p 0.95) lives in `models.yml` `extraBody`
-  — on the wire only. The template defaults `reasoning_effort` to **xhigh**
-  when a request sends none, and **thinking tokens consume the
-  `max_tokens` budget**: a budget exhausted mid-thought returns
-  `finish_reason: "length"` with EMPTY `content` and the partial reasoning
-  in `reasoning_content` — clients must read both fields.
-
-## Client contract deltas vs llama.cpp
-
-|Behavior|halogen|
-|---|---|
-|Prompt over slot|400 naming the limit (no truncation)|
-|`max_tokens` over cap|400 naming the limit (never silent truncation)|
-|full-length request outliving `HALOGEN_QUEUE_TIMEOUT`|503s for everyone queued|
-|`timings`|every response carries llama-server-shape `timings` (`draft_n`, `draft_n_accepted`) — the drafter check reads the RESPONSE, and it proves speculation fired|
-|`response_format`|NOT supported (no constrained decoding)|
-|thinking field|`reasoning_content` (same as before)|
-|`reasoning_effort`|same Qwen3.8 template; outside {xhigh,medium,low} rejected — models.yml fold stays; unset defaults to xhigh, `none` disables thinking|
-|tools|server-side tool calls advertised — round-trip MUST be re-probed after deploy|
-|image input|ignored/error — route to frank-zealot|
-
-## Deploy mechanics
-
-- **Image:** `ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea`
-  (published tag 0.1.4). The deployer gates the exact digest exists in podman
-  storage before dispatch. NOT the 2026-09 `halogen-flash-server` repo — that
-  was the Flash-Next w4b era, different image, retired.
-- **Payload:** `scripts/download-frank-dragoon-halogen.sh` (fetch law: `curl -f`,
-  exact byte count, LFS sha256 where the Hub publishes one, `--verify-only`,
-  token never in argv) into `/home/dank/models/halogen-qwen3.8-27b/`. The
-  tokenizer is a FLAT DIRECTORY (`tokenizer/`, 5 files — tokenizer.json,
-  vocab.json, merges.txt, chat_template.jinja, tokenizer_config.json),
-  mounted whole at `/tokenizer`; the engine's `HALOGEN_TOKENIZER` default is
-  that directory, so the container gets no tokenizer flag (model-card
-  docker-run parity). The deployer and serve script refuse a missing file or
-  ANY symlink: an HF cache snapshot symlinks each entry into `blobs/` and
-  those links dangle inside the container mount.
-- **Unit:** `scripts/frank-dragoon.service` (root, `User=dank`, relinked into
-  `/etc/systemd/system/`), `ExecStart` = serve script, `ExecStop` = `podman
-  stop -t 120 frank-dragoon-halogen`, script runs `podman run` FOREGROUND
-  (2026-10-06: the 2026-10-05 `-d` + `exec podman wait` shape captured
-  only the container ID — the engine ledger existed nowhere on disk and
-  the post-deploy grep below read a one-line hex file; foreground run
-  returns the container's exit code so `Restart=always` is real and the
-  ledger lands in the log). Engine stdout goes to
-  `/home/dank/logs/frank_dragoon.log` (the boot ledger lives there, not in
-  the journal).
-- **Cutover:** `deploy_llm_dragoon` in `scripts/deploy-remote.sh` sweeps the
-  legacy bare `llama-server --model /home/dank/models/kingjones-*` process and
-  the legacy `frank-dragoon` container name by NAME (never `pkill -f
-  llama-server` — that hits frank-zealot), retires the old quadlet if the 2026-09
-  halogen era's unit lingers, waits port 7770 free, relinks, health-gates.
-- **Bench law:** stop+mask the unit before any GPU experiment on this box
-  (AGENTS.md); `/health` answers 503 while the 33.4 GiB payload loads — that
-  is loading, not failure.
-
-## Verification (post-deploy checklist)
-
-1. `grep -iE 'clamp|kv|pool|drafter|dflash|cache' /home/dank/logs/frank_dragoon.log`
-   — ledger shows the pool at the asked shape, DFlash2 active, the
-   prompt cache line naming the pinned budget, NO clamp line. EXPECT the
-   WARNING "smaller than ONE full-context entry (18.4 GB at
-   kMaxCtx=262144)": it sizes against the NATIVE window, not our
-   200,700 slot — the deepest possible entry (~14 GB at ~70 KB/token)
-   fits the 16 GB pin, and the 262k entry it fears cannot exist (over
-   slot is a 400). Probe-proven 2026-10-06: the warm turn hit with
-   cache_n=16,384.
-2. `curl -s http://192.168.0.143:7770/health` -> 200.
-3. Real completion with `reasoning_effort` + temp/top_p from the wire recipe;
-   `reasoning_content` populated; decode rate from the timings.
-4. Spec decode fires: multi-token step behavior visible in rate; compare
-   against DRAFTER=0 arm if in doubt (never against gufo — bench law).
-5. Harness round-trip: an OMP call through `llamacpp (dankclaw)/frank-dragoon`
-   incl. a tool call (V1TOOLS).
-6. Prompt > 200,700 tokens returns the 400 naming the limit (contract test).
-
-## Tuning notes (2026-10-06)
-
-Measured on real agent traffic (serve_api lines) the day after cutover:
-
-- **Decode is depth-bound, and the KV path is the reason.** Shallow
-  prompts (< 1k tokens) decode 30-50 t/s at commit 4-6.3/round; 46k runs
-  5.5, 63-67k runs 4.1-6.5, 79-81k runs 3.3-3.8. The round-time slope is
-  ~8 us/token of context: the KV pool reads at ~8 GB/s effective while
-  the weights stream ~180-240 GB/s (the engine's README puts batch-1
-  weight streaming at the 240 GB/s hardware wall). Commit also falls
-  from 4-6 to 2.5-3 accepted/round as the conversation grows.
-- **No supported flag touches decode.** `docs/FLAGS.md`: HALOGEN_W4A4 /
-  W4A4_EXCL are prefill-only ("decode is unaffected either way"); the
-  undocumented kernel levers are declared unsupported and their defaults
-  are the measured winners. 0.1.4 is the latest release (CHANGELOG
-  head; kernels unchanged since 0.1.0). The publisher's tg bench never
-  measured depth — its tg rows run over the ten SHORT eval prompt
-  shapes — so the README's "decode barely degrades at depth" claim was
-  never exercised at 46-80k.
-- **A/B'd the one FA lever with public precedent (2026-10-06 stop-window
-  probe): rejected.** Issue #12's `HALOGEN_FA_TQ=256` (the 64 GB-carve
-  crash workaround) vs the default tile — same 80k filler prompt,
-  128-token essays, the runtime's own `timings`: decode 6.35 vs 6.34 t/s
-  cold and 5.74 vs 5.76 warm (±0.3 % = noise; draft acceptance identical,
-  53/74 and 58/69), while prefill LOST 9.5 % on this topology (447.5 ->
-  404.8 t/s). The depth collapse is not an FA tile artifact — it sits
-  deeper than the knob. The engine binary advertises 40+ undocumented
-  `HALOGEN_*` kernel levers (FA/DN/MATMUL families, readable via
-  `grep -aoE "HALOGEN_[A-Z0-9_]+"` on the binary); `FA_TQ` is the only
-  one with public evidence, and it moved nothing that matters here.
-- **Routing is the applied mitigation (2026-10-06):** the harness's
-  deep-conversation roles (`slow`, `advisor` in `~/.omp/agent/config.yml`)
-  moved to frank-archon (radiance on Dankserver: fp8 KV at
-  7.32 KiB/token = 8.7x less KV to read per round at depth, MTP depth 3,
-  200k window, prefix-cache host+disk tiers). frank-dragoon keeps OpenWebUI
-  and shallow bursts, where it runs 30-50 t/s. frank-archon's
-  depth-decode rate is designed-for but not yet measured on record.
-- **The lever that existed was the prompt cache, and it was OFF** (see
-  the shape bullet). The pin turns the 85-180 s re-prefill per deep
-  turn into a resume; decode itself is expected UNCHANGED by the pin —
-  the cache resumes prefill, it does not touch the round loop.
-  Probe-proven (two synthetic 17.3k turns, 2026-10-06): cold prefill
-  27.0 s (640 t/s, cache_n 0) -> warm turn cache_n 16,384, prompt_n 946
-  (the extension + one 2048-aligned chunk), prefill 1.7 s — 16x on the
-  pair, answer correct warm.
-- **Everything else stays put.** dflash2 over mtp (publisher 31.71 vs
-  26.88 mean; at depth both pay the same KV term and dflash2's higher
-  commit wins), KV_SLOTS=1 (batch XOR speculation), SLOT_CTX=200700
-  (Frank's window), W4A4 shipped (+9 % prefill, decode unharmed).
-
-## Rollback
-
-Revert the frank-dragoon swap commit and redeploy `dankclaw --service frank-dragoon`:
-the ROCmFPX payload (`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`),
-the fork build (`/home/dank/GitPackages/kingjones-ROCmFPX/build`), and its
-`scripts/build-rocmfpx.sh` pin (commit `36e9acd`, seam-tested) all remain on
-disk. The halogen payload can stay: it costs nothing while the unit points
-elsewhere.

serverAppConfigs-docs/LLM/frank-dragoon-ik-uncensored.md

@@ -1,149 +0,0 @@
----
-type: reference
-description: frank-dragoon on Dankclaw serving the mradermacher Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M GGUF via ik_llama.cpp on the RTX 3090 + Strix Halo hybrid — measured performance envelope, root-cause analysis, tuning levers, and traps
-tags: [llm, frank-dragoon, ik_llama, dankclaw, rtx3090, strix-halo, performance]
-timestamp: 2026-09-16T03:30:00-07:00
----
-
-# frank-dragoon — ik_llama.cpp, Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M
-
-**RETIRED 2026-09-16, after one day.** Replaced by [[halogen-flash-server]],
-which was the runtime before it as well — the fleet went ik → halogen → back.
-Its unit, env, serve script and the Qwen-Sharp template are archived in the repo
-under `scripts/.AGENTTRASH/*.ik-llama.cpp` and `*.ik-llama-qwen-sharp`, and the
-payload, build and chat template stay on disk. The measurements below are not a
-stale plan: they are the complete account of what this model costs on this
-hardware, and they are why the fleet went back. It served
-`mradermacher/Qwen3.8-Flash-Next-Uncensored` (111 GiB GGUF + mmproj-f16) at
-`192.168.0.143:7770` while it ran.
-
-## Shape
-
-- Runtime: ik_llama.cpp master `d85a1d7`, built at
-  `/home/dank/GitPackages/ik_llama.cpp/build` with `-DGGML_CUDA=ON
-  -DCMAKE_CUDA_ARCHITECTURES=86 -DGGML_AVX512=ON -DGGML_AVX512_VBMI=ON
-  -DGGML_AVX512_VNNI=ON -DGGML_AVX512_BF16=ON -DLLAMA_CURL=OFF`.
-- Unit: **root systemd unit** `/etc/systemd/system/frank-dragoon.service`
-  symlinked to `scripts/frank-dragoon.service` in the repo; enabled at boot;
-  `Restart=always`. Serve script `scripts/serve-frank-dragoon.sh` + env
-  `scripts/frank-dragoon.env`.
-- Topology: 3090 (PCIe x4 in the M.2 dock, link downgraded from x16) carries
-  dense/attention/KV (~11.5 GiB); ALL MoE expert tensors (~84 GiB) and the
-  ~33 GiB PLE n-gram table stay in LPDDR via CPU buffers.
-- Client contract: greedy default (`--temp 0.0`), reasoning on (deepseek
-  format, unlimited budget), vision via mmproj, slot-state persistence under
-  `scripts/slot-state`, 262144 ctx, 1 slot.
-
-## Measured performance envelope (2026-09-15/16 campaign, all single-instance verified)
-
-| config | decode tok/s | prefill tok/s | verdict |
-|---|---|---|---|
-| shipped (T16, `-ot exps=CPU`, NO_PINNED_WEIGHTS) | **3.8-4.0** | ~193 at 1.8k / 191 at 32k | shipped |
-| T8 / T4 / T2 / T1 | 4.18 / 4.20 / 4.26 / 3.97 | small-prompt only | decode flat across threads |
-| `-ncmoe 42` (6 layers' experts on GPU) | 4.54 | 32 (small prompt) | VRAM 22.0/24.6 GiB — fragile |
-| `-ncmoe 48 -thp` | 4.60 | 18 | **110 GiB anon host buffer — co-tenant killer, never ship** |
-| `--prefetch-experts` | 4.03 | — | no effect (page cache already resident) |
-| `--defer-experts --defer-ple -thp` | crashes | — | experts under `-ngl 999` still target CUDA0 → cudaMalloc OOM |
-| mainline llama.cpp `37b53fd` (`--n-cpu-moe 48`) | **1.01** | 34 | ruled out — CPU MoE kernels 4x slower than ik's iqk |
-
-- Large-prompt prefill ≈ 192 tok/s is the **CPU AVX512-VNNI compute ceiling**
-  for CPU-resident experts (expert weight reads amortize across the batch;
-  compute-bound). Halogen's "4x llama.cpp family" prefill claim is therefore
-  unreachable with experts on CPU — the physics caps hybrid MoE prefill here.
-- Decode is gated by a **serial per-token critical path**: ~100 CPU↔GPU
-  backend splits per token (`graph splits = 100` in the log); the 3090 sits at
-  **5% SM utilization** during decode; thread count does not matter (T1 ≈
-  T16 ≈ 4 tok/s). The GPU sections are launch/sync-bound, not compute-bound.
-
-## Root cause chain (perf evidence)
-
-1. `perf` at T16: ~53% cycles in `ggml_graph_compute_thread` + ~37% in
-   `ggml_compute_forward_mul_mat_id`, of which **~98% is the `pause`
-   instruction** (thread-pool barrier spin). Actual vec_dot math: ~4%.
-2. T1 = T16 decode (3.97 vs 3.8-4.0) ⇒ the barrier spin is threads waiting on
-   a serial critical path, not parallel overhead.
-3. GPU 95% idle during decode ⇒ the critical path is the GPU-section
-   launch/sync overhead per split (x4 dock link, eager kernel launches).
-4. ik's CUDA-graph capture would collapse those launches but **aborts** on
-   qwen4exp: the per-layer hyper-connection broadcasts (`ggml_repeat_4d`,
-   4 sites in `src/graphs/build_qwen4exp.cpp`) resolve to a REPEAT node whose
-   CUDA path does a capture-illegal operation
-   (`CUDA error: operation not permitted when stream is capturing` → abort).
-5. Memory placement is NOT the lever: mmap-off/anon/THP/deferred variants all
-   land at 3.8-4.6 tok/s.
-
-## Why the 30 tok/s halogen bar is out of reach (honest record)
-
-Halogen's engine held the whole 115.55 GiB w4b checkpoint resident in LPDDR
-(`HALOGEN_FLASH_PIN_TRUNK=1`) and served from one engine. This deployment must
-split 84 GiB of expert tensors across a 24 GB GPU + CPU, and every public
-runtime pays the split penalty above. Paths to >30 tok/s, in order of
-realism: (a) a runtime with capture-safe CUDA graphs + dedicated hc ops —
-mainline is building exactly this (ggml-org PR #28901 "qwen4exp: add hc ops");
-re-evaluate mainline when its hc-ops + graph handling mature; (b) experts on
-GPU — impossible, 84 GiB > 24 GiB; (c) a smaller model.
-
-## Upstream issues tracking the slow TG path
-
-ik_llama.cpp: #2374 (QSA opt POC), #2406 (single-threaded sigmoid), #2433
-(cache-miss full reprocess), #2460 (PLE history correctness). MTP/speculative
-decoding is unavailable for this GGUF — mradermacher stripped the NextN head
-(verified: no `blk.48.*`/nextn tensors in the file).
-
-## Traps (learned the hard way 2026-09-15/16)
-
-- **Pinned host buffers strand the host**: a default-policy start
-  page-locks ~108 GiB (`CUDA_Host buffer`), evicting co-tenants into swap and
-  starving sshd until a power cycle. `GGML_CUDA_NO_PINNED_WEIGHTS=1` + the
-  serve-script guard exist for this. `-ncmoe N` with default NO_PINNED
-  settings produces the same 108 GiB pinned buffer — never run it without the
-  guard.
-- **Any `-ot`/`-ncmoe` override disables mmap** for the whole model load
-  (`create_tensors` returns false → `ml.use_mmap = false`): the "mmap-backed
-  CPU buffer" comment in the serve script is wrong on the mmap word; the
-  buffer is anonymous. All placement variants perform alike anyway.
-- **`--defer-experts` does not keep experts off the GPU** under
-  `-ngl 999` — only PLE defers (32.78 GiB). With ngl 999 the load dies with
-  `cudaMalloc failed: out of memory` at ~79.7 GiB on device 0.
-- **The systemd unit self-heals through manual tests**: `pkill` the server and
-  the unit respawns it ~4 s later (Restart=always) — two instances, port
-  races, phantom measurements. For manual experiments:
-  `sudo systemctl stop frank-dragoon.service` first (it is a root unit; plain
-  `systemctl --user` lines in older docs are stale), and restore with
-  `sudo systemctl start frank-dragoon.service`.
-- Verify "HTTP server listening" in the CURRENT log and
-  `pgrep -xc llama-server` == 1 before trusting any measurement.
-- FA must stay ON — FA-off crashes the loader on this arch.
-- Do not run builds concurrently with decode measurements (`-j16` starves the
-  service and corrupts both results).
-
-## Rollback
-
-Stop and disable the root unit, trash it, restore the halogen quadlet from
-`~/.AGENTTRASH/20260915-ik-swap/halogen-quadlet` to
-`~/.config/containers/systemd/`, `systemctl --user daemon-reload`, start;
-the .hgn checkpoint and the kingjones STRIX llama.cpp payload remain on disk.
-
-## Hybrid campaign envelope (2026-09-16, 3090 + Strix Halo)
-
-Why this deployment could not be tuned into a production endpoint: decode
-3.8–4.3 tok/s flat across thread counts (T1 ≈ T16); prefill ~192 tok/s at
-1.8k–32k prompts = the **CPU AVX512-VNNI compute ceiling** for CPU experts.
-The decode root cause is structural: ~100 CPU↔GPU backend splits per token
-with the GPU at 5 % SM during decode — launch/sync-bound over the PCIe x4
-dock, not compute- or memory-bound. T16 profiling shows ~90 % of cycles in
-`pause` (ggml_barrier spin): threads wait on the serial critical path.
-Memory placement (mmap/anon/THP/deferred) does not matter — all variants
-landed 3.8–4.6.
-
-- CUDA-graph capture aborts: qwen4exp's hyper-connection broadcasts use
-  per-layer `ggml_repeat_4d`, whose CUDA path is capture-illegal; until ik
-  gains capture-safe repeat/hc ops (mainline PR #28901 direction), graphs
-  stay dead.
-- Mainline llama.cpp itself runs 1.0 tok/s on this model (CPU MoE kernels
-  4× slower than ik's iqk) — do not switch engines for speed.
-- MTP/speculative decoding is impossible for mradermacher i1 GGUFs: the
-  NextN head is stripped (no `blk.48` tensors).
-- Tuning lever ranked first: `-ncmoe 42` (6 layers' experts on the 3090) →
-  decode 4.54, small-prompt prefill 32 — but VRAM 22/24.6 GiB at 262k ctx,
-  an operator OOM-margin decision, and it needs `GGML_CUDA_NO_PINNED_WEIGHTS=1`.

serverAppConfigs-docs/LLM/frank-dragoon-kairic-edge-qwen38-27b.md

@@ -1,48 +0,0 @@
----
-type: reference
-title: "Kairic Edge Qwen3.8-27B IU4 (W4A4) on Dankclaw (frank-dragoon)"
-description: "The RETIRED frank-dragoon runbook (2026-09-21, same-day swap; replaced by [[frank-dragoon-unsloth-qwen38-27b-q8xl]]) — the Kairic Edge runtime (ciru-ai/ROCmFPX, branch kairic-edge-qwen38-27b-v1.2, commit 205a3e5, TheRock 7.15.0, Composable Kernel) serving jcbtc/Qwen3.8-27B-IU4-Kairic-Edge (27B dense, IU4 W4A4) on the Dankclaw Strix Halo iGPU: the source-built runtime, the full-window/one-stream shape, the MTP depth 4 + n-gram spec method, the three PromptForge sidecars, the compatibility mode, the payload, the client contract, and the failure modes worth knowing."
-tags:
-  - llm
-  - kairic-edge
-  - rocmfpx
-  - dankclaw
-  - frank-dragoon
-  - strix-halo
-  - gfx1151
-  - iu4
-  - w4a4
-  - rocm
-  - mtp
-  - promptforge
-timestamp: 2026-09-21
----
-
-# frank-dragoon — Kairic Edge Qwen3.8-27B IU4 (W4A4) (RETIRED)
-
-> **RETIRED 2026-09-21 (same-day swap).** This was the ACTIVE frank-dragoon
-> runbook on 2026-09-21 before it was replaced by
-> [[frank-dragoon-unsloth-qwen38-27b-q8xl]] later the same day. The payload
-> and runtime remain on disk as the rollback target.
-
-## What it served
-
-- **Runtime:** Kairic Edge (ciru-ai/ROCmFPX, branch
-  `kairic-edge-qwen38-27b-v1.2`, commit 205a3e5, TheRock 7.15.0,
-  Composable Kernel), root systemd unit, port 7770.
-- **Model:** `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` (a 27B dense model, IU4
-  W4A4) — main GGUF + FFN/GDN/GDN-Output `.pfs` sidecars.
-- **Shape:** one 262,144-token conversation (single stream), native MTP
-  depth 4 + n-gram drafting, compatibility mode.
-- **Memory:** the 114 GiB GTT aperture on the Strix Halo iGPU.
-
-## Rollback
-
-Payload and runtime intact on disk. See [[frank-dragoon-unsloth-qwen38-27b-q8xl]]
-for the current ACTIVE runbook and [[frank-dragoon-unsloth-qwen38-flash-next]]
-for the predecessor.
-
-# Citations
-
-- Kairic Edge runtime: https://github.com/ciru-ai/ROCmFPX
-- Model: https://huggingface.co/jcbtc/Qwen3.8-27B-IU4-Kairic-Edge

serverAppConfigs-docs/LLM/frank-dragoon-rocmfpx-q6agent-uncensored-mtp.md

@@ -1,177 +0,0 @@
----
-type: reference
-title: "frank-dragoon on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP"
-description: "The active frank-dragoon runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Uncensored Q6_0_ROCMFPX_AGENT GGUF (ftype 114, protected output heads), with the repo's own MTP-Q4_0 head drafting four tokens at one native 262,144 window, f16 KV, the BF16 projector for vision, and the measured rejection of the ngram-mod chain."
-tags:
-  - llm
-  - frank-dragoon
-  - dankclaw
-  - llama-cpp
-  - rocmfpx
-  - gfx1151
-  - gguf
-  - spec-decode
-  - mtp
-timestamp: 2026-10-03
----
-
-# frank-dragoon — ROCmFPX Uncensored Q6_0_ROCMFPX_AGENT + MTP (gfx1151)
-
-> **RETIRED 2026-10-03 → 2026-10-05 (rollback target; payload, fork build and
-> build script all on disk).** Replaced by [[frank-dragoon-halogen27b-dflash2]]
-> (DFlash2, publisher 31.7 t/s mean vs the 27.6–28.1 measured here). This page
-> remains the contract record for the rollback. Original banner follows.
-> ACTIVE since 2026-10-03. Cut over from
-> [[frank-dragoon-rocmfpx-q8agent-mtp]] (ALIGNED Q8_0_ROCMFPX_AGENT, 17.0 t/s)
-> on Frank's request: the *uncensored* variant of the same protected-heads
-> family, one quant rung down. The swap turned out to be a strict upgrade on
-> the axis that matters here: **decode 27.6–28.1 t/s at a 4,813-token prompt
-> against 17.0 at a 2,202-token prompt** — faster with a deeper context, not a
-> same-depth comparison, but the sign is not in question.
-
-## Identity
-
-| | |
-|---|---|
-| Host / device | Dankclaw (192.168.0.143), Radeon 8060S iGPU `gfx1151`, ~114 GiB GTT aperture |
-| Endpoint | `http://192.168.0.143:7770` — alias `frank-dragoon`, `/v1` OpenAI surface |
-| Runtime | **`kingjones30/ROCmFPX`** llama.cpp fork @ `36e9acd40e10a87cd3c3ef8ec734668757dc8520`, built by [`scripts/build-rocmfpx.sh`](../../scripts/build-rocmfpx.sh) — unchanged from the Q8 era; the fork already knew type 102 |
-| Unit | root systemd `frank-dragoon.service` ([`scripts/frank-dragoon.service`](../../scripts/frank-dragoon.service)), `User=dank`, `Restart=always` |
-| Serve script | [`scripts/serve-frank-dragoon.sh`](../../scripts/serve-frank-dragoon.sh) (owns the argv), tunables [`scripts/frank-dragoon.env`](../../scripts/frank-dragoon.env) |
-| Weights | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` — 24,856,607,008 B, sha256 `e966deb3…` |
-| Draft | `…/mtp-Qwen3.8-27B-Q4_0.gguf` — 1,680,271,648 B, sha256 `051a1764…` (byte-identical to the aligned repo's head; hardlinked, not re-downloaded) |
-| Projector | `…/mmproj-Qwen3.8-27B-BF16.gguf` — 931,145,888 B (hardlinked from the aligned payload dir; the same file the Q8 era served) |
-| Fetch | [`scripts/download-frank-dragoon-kingjones-uncensored.sh`](../../scripts/download-frank-dragoon-kingjones-uncensored.sh) — hardlinks the draft/projector from the aligned dir when present, otherwise fetches; `-f`, byte-count + LFS sha256 verified, `--verify-only`, token never on the command line |
-| Seam | `tests/frank_dragoon_unit.bats` (payload gates, deployer/serve parity, spec block, KV dtype) |
-
-## Why this file
-
-`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF` ships the same
-"protected heads" construction as the aligned repo, in the uncensored tuning.
-Parsed header of the deployed file:
-
-| Field | Value |
-|---|---|
-| ftype | **114** — the fork's 64-bit-block family, ≈7.44 bpw |
-| Tensor mix | 353 F32 + **339 type-102 (`GGML_TYPE_Q6_0_ROCMFPX`)** + **159 type-103 (`Q8_0_ROCMFPX`)** |
-| Protected heads | 64 output-head tensors kept at plain precision — the AGENT construction from the aligned repo, at the 6-bit rung |
-| Arch | qwen35, 64 layers, `full_attention_interval 4`, `context_length 262144` |
-| Embedded sampling recipe | temp 1.0 / top_p 0.95 / top_k 20 |
-
-The same lock-in law as the Q8 era applies: types 102 and 103 exist in no
-loader but this fork; stock llama.cpp and gufo refuse the file at load
-(`unknown tensor type`). The loader was not re-litigated — the file picks the
-runtime, and the pin in `scripts/build-rocmfpx.sh` (commit by SHA) stays.
-
-Rejected sibling: `Q6_0_ROCMFPX_LEAN` (6.45 bpw, no protected heads) — the
-protection is what keeps the output-head logits at full precision and the
-drafter accurate, so it stays deployed.
-
-## Shape and budget
-
-Unchanged from the Q8 era — the swap was payload-only:
-
-| Dial | Value | Why |
-|---|---|---|
-| `--ctx-size` | 262,144 | the native window, no rope scaling |
-| `--parallel` | 1 | one conversation, all memory to it |
-| KV dtype | **f16** (both K and V) | ~64 KiB/token; free inside a 114 GiB aperture |
-| `--fit off` | on | the auto-fitter would silently resize the window |
-| `--flash-attn on` | on | required for the long-window attention path |
-| `--direct-io`, `--ctx-checkpoints 16`, `--cache-ram 8192`, `--cache-idle-slots`, `--jinja`, `--reasoning on --reasoning-format deepseek --reasoning-budget -1` | | same as [[frank-dragoon-rocmfpx-q8agent-mtp]] |
-
-Spec decode: `--spec-type draft-mtp`, the repo's `mtp-Qwen3.8-27B-Q4_0.gguf`,
-`--spec-draft-n-max 4`, pinned `--spec-draft-device ROCm0`.
-
-Measured steady state: GTT **~61 GiB** under live load (host baseline 16.7),
-boot to `/health` **~15 s**.
-
-## Measured on this payload (2026-10-03, production service, `/completion` timings, temp 0, `cache_prompt: false`)
-
-| Traffic | Decode t/s | Prefill t/s | Draft acceptance |
-|---|---|---|---|
-| Prose, 4,813-token prompt, 512 tokens | **27.57 / 28.10** (two runs) | ~205–217 | **396/456 = 0.868** → mean acceptance length ≈ 4.4, i.e. drafting at nearly the full n-max 4 |
-| Schema echo (structured repetition) | **31.44** | — | 639/639 = 1.000 |
-| Block copy (text repeated verbatim) | ~28.0 | — | 0.848 |
-
-vs the Q8 era's 0.462 acceptance at a 2,202-token prompt: the MTP head drafts
-materially better on this file. The card's claimed 0.953-class acceptance is
-no longer science fiction on real prose — but keep reading it as a file
-property, not a throughput promise.
-
-## The ngram-mod chain — measured and rejected
-
-Frank's ask: layer `--spec-type draft-mtp,ngram-mod` on top of MTP for an
-extra gain. Measured in three bench windows (production service stopped per
-the benchmarking law; artifacts `/home/dank/fl-ngram-ab/` on Dankclaw),
-comparing `draft-mtp` alone vs the chain on the same traffic:
-
-| Traffic | MTP alone | MTP + ngram-mod | Delta |
-|---|---|---|---|
-| Prose | 27.57 t/s | 23.40 t/s | **−15 %** |
-| Block copy | 28.0 t/s | 26.1 t/s | **−7 %** (with ngram-mod demonstrably firing: 138 proposals/run) |
-
-Mechanism, straight from arm B's boot log:
-
-```
-recurrent state rollback is not compatible with 'ngram-mod' - disabling rollback support
-```
-
-On the qwen35 hybrid (Gated DeltaNet recurrent layers) adding the ngram layer
-disables the rollback path MTP's own verification relies on, so acceptance —
-and throughput — drop on *all* traffic, even where the ngram matcher fires.
-**Deployed decision: `draft-mtp` alone.** Chaining is not free on recurrent
-models; the fleet law in `AGENTS.md → llama.cpp serving` records it.
-
-## Bench-window hardening laws (learned building the A/B)
-
-1. **Health 200 is not proof of ownership.** After boot, verify the pid you
-   started owns the port (`ss -ltnp`); a zombie server from a prior window
-   answers `/health` too and silently serves your arm.
-2. **Kill scratch servers with SIGKILL after a grace period.** llama-server's
-   graceful SIGTERM waits on open keep-alive connections — a harness holding
-   the socket leaves the process (and its ~24 GiB of GTT) alive after the
-   script exits.
-3. **Prove the port is free and GTT dropped between arms**, or the second
-   arm measures a collision.
-
-## Client contract
-
-Unchanged from the Q8 era and re-verified on this payload 2026-10-03: model id
-`frank-dragoon`, native tools, thinking in `reasoning_content`,
-`reasoning_effort` ∈ {xhigh, medium, low} only (anything else HTTP 500),
-card-recipe sampling defaults server-side, vision through the BF16 projector
-(red PNG → **`Red`**, 74 prompt / 165 completion tokens). The traps list —
-`content: null` when reasoning eats the budget, the self-recovering MTP
-boundary log line, `enable_thinking:false` not `reasoning_effort:"none"` — is
-in [[frank-dragoon-rocmfpx-q8agent-mtp]] and still applies.
-
-Sweep key for scratch servers is now the payload name:
-`pkill -f 'Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT'` — never `pkill -f
-llama-server` (frank-zealot is also one).
-
-## Operations
-
-```bash
-./scripts/deploy.sh dankclaw --service frank-dragoon
-systemctl status frank-dragoon
-journalctl -u frank-dragoon -n 200 --no-pager
-scripts/download-frank-dragoon-kingjones-uncensored.sh --verify-only
-```
-
-## Rollback
-
-| Target | How |
-|---|---|
-| **ALIGNED Q8_0_ROCMFPX_AGENT** (previous shape, 17 t/s) | the payload dir `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` is untouched on disk; revert the swap commit (unit/env/serve trio) and redeploy, or point `FRANK_DRAGOON_MODEL_FILE` + `LLM_DRAGOON_MODEL_FILE` at the old paths. Verify with `scripts/download-frank-dragoon-kingjones.sh --verify-only` |
-| gufo Q4_K_XL era | git tag `frank-dragoon-gufo-q4kxl-20261001`, **but the gufo payloads (UD-Q4_K_XL 16.35 GiB, UD-Q8_K_XL + DFlash2 drafter) were removed from disk 2026-10-03 with Frank's approval** — this rollback now requires re-download per [[frank-dragoon-gufo-qwen38-27b]] |
-
-## When it misbehaves
-
-Same decision tree as [[frank-dragoon-rocmfpx-q8agent-mtp]]:
-`unknown tensor type` ⇒ off the pinned fork; SIGSEGV in
-`ggml_compute_forward_set_rows` ⇒ pre-`36e9acd` build; slow decode with no
-error ⇒ draft not engaged (`draft_n` in `/completion` timings) or an orphaned
-second copy eating GTT (`/sys/class/drm/card1/device/mem_info_gtt_used`);
-500 naming Jinja ⇒ `--jinja` lost; 500 naming the template ⇒ unmapped
-`reasoning_effort` in `models.yml`.

serverAppConfigs-docs/LLM/frank-dragoon-rocmfpx-q8agent-mtp.md

@@ -1,232 +0,0 @@
----
-type: reference
-title: "frank-dragoon on Dankclaw — ROCmFPX llama.cpp, Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (retired)"
-description: "The retired frank-dragoon runbook — the kingjones30/ROCmFPX llama.cpp fork (pinned commit, gfx1151) serving kingjones777's Q8_0_ROCMFPX_AGENT GGUF, a ftype-115 file no other loader can read, with the repo's own MTP-Q4_0 head, f16 KV, and the BF16 projector. Measured MTP-vs-none A/B, the client contract, and the rollback state as of 2026-10-03."
-tags:
-  - llm
-  - frank-dragoon
-  - dankclaw
-  - llama-cpp
-  - rocmfpx
-  - gfx1151
-  - gguf
-  - spec-decode
-  - mtp
-timestamp: 2026-10-02
----
-
-# frank-dragoon — ROCmFPX Q8_0_ROCMFPX_AGENT + MTP (gfx1151, retired)
-
-> **RETIRED 2026-10-03** — superseded by
-> [[frank-dragoon-rocmfpx-q6agent-uncensored-mtp]]: Frank swapped to the
-> uncensored Q6_0_ROCMFPX_AGENT file (protected heads, one rung down), which
-> measures **27.6–28.1 t/s** at a deeper prompt than this file's 17.0. This
-> runbook stays live as the rollback reference and the trap catalogue; the
-> payload it documents remains on disk at
-> `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`.
->
-> Cut over 2026-10-02 from [[frank-dragoon-gufo-qwen38-27b]] (Q4_K_XL, ~15 t/s)
-> on two requests from Frank: *highest quality that runs*, and *one full
-> context*. It delivered 8.39 bpw weights — two quant rungs above what gufo
-> could load — at 17.0 t/s, so quality was not bought with speed.
->
-> The capability that forced this whole shape is a GGUF tensor type:
-> **`GGML_TYPE_Q8_0_ROCMFPX` (103)**. It exists in exactly one loader on this
-> host, so the runtime was decided by the file, not by preference.
-
-## Identity
-
-| | |
-|---|---|
-| Host / device | Dankclaw (192.168.0.143), Radeon 8060S iGPU `gfx1151`, ~114 GiB GTT aperture |
-| Endpoint | `http://192.168.0.143:7770` — alias `frank-dragoon`, `/v1` OpenAI surface |
-| Runtime | **`kingjones30/ROCmFPX`** llama.cpp fork @ `36e9acd40e10a87cd3c3ef8ec734668757dc8520`, built by [`scripts/build-rocmfpx.sh`](../../scripts/build-rocmfpx.sh) → `/home/dank/GitPackages/kingjones-ROCmFPX/build/bin/llama-server` |
-| Unit | root systemd `frank-dragoon.service` ([`scripts/frank-dragoon.service`](../../scripts/frank-dragoon.service)), `User=dank`, `Restart=always` |
-| Serve script | [`scripts/serve-frank-dragoon.sh`](../../scripts/serve-frank-dragoon.sh) (owns the argv), tunables [`scripts/frank-dragoon.env`](../../scripts/frank-dragoon.env) |
-| Weights | `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` — 28,214,425,952 B, sha256 `f3ef2bdbf56e244f…` |
-| Draft | `…/mtp-Qwen3.8-27B-Q4_0.gguf` — 1,680,271,648 B, sha256 `051a1764cff8c4f3…` |
-| Projector | `…/mmproj-Qwen3.8-27B-BF16.gguf` — 931,145,888 B, sha256 `de2a4986…` |
-| Fetch | [`scripts/download-frank-dragoon-kingjones.sh`](../../scripts/download-frank-dragoon-kingjones.sh) (`-f`, byte-count + LFS sha256 verified, `--verify-only`, token never on the command line) |
-| Seam | `tests/frank_dragoon_unit.bats` (19 tests: payload gates, deployer/serve parity, spec block, KV dtype, sweep identity) |
-
-## Why this file, and why only this runtime
-
-`kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF` ships several rungs. Parsed
-headers of the two candidates:
-
-| File | ftype | Tensor mix | Output-head tensors at plain Q8_0 | Card MTP acceptance |
-|---|---|---|---|---|
-| `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` | **115** | 353 F32 + 187 Q8_0 + **311 type-103** | **187** | 0.953 |
-| `Qwen3.8-27B-Q8_0_ROCMFPX.gguf` | 111 | 2 Q8_0 + type-103 | 2 | 0.911 |
-
-Two consequences, both measured off the header rather than guessed:
-
-1. **Quality.** The AGENT routing variant keeps the output-head (and other
-   routing-sensitive) tensors at plain Q8_0 instead of folding them into the
-   ROCmFPX kernel layout — 187 vs 2 — so the logits the sampler reads are the
-   highest-precision tensors in the file. It is also the card's MTP-tuned
-   variant, so it is the *faster* drafter base at the same time.
-2. **The runtime is forced.** Type 103 is a private extension: stock llama.cpp
-   and gufo both fail at load (`unknown tensor type`). Verified on the host
-   before deploying — the bake-off was not a horse race, it was a lock-in
-   check.
-
-**The build identity matters as much as the fork.** The 2026-08-25 ROCmFPX
-builds read this file and then died in `ggml_compute_forward_set_rows` — the
-exact failure documented in [[kingjones-q8agent-rocmfpx]] when this payload was
-first attempted on Dankclaw and routed to Dankserver as
-[[frank-medium-rocmfpx]] instead. `scripts/build-rocmfpx.sh` therefore checks
-the commit out **by SHA** and refuses to build anything else; the pin is
-load-bearing, not tidiness.
-
-## Shape and budget
-
-One session, the model's full native window, unquantized KV:
-
-| Dial | Value | Why |
-|---|---|---|
-| `--ctx-size` | 262,144 | "one full context" — the native window, no rope scaling |
-| `--parallel` | 1 | one conversation, all memory to it |
-| KV dtype | **f16** (both K and V) | ~64 KiB/token → ~16 GiB; free inside a 114 GiB aperture, so there is no reason to pay a quant |
-| `--fit off` | on | the auto-fitter would silently resize the window we asked for |
-| `--flash-attn on` | on | required for the long-window attention path to fit |
-| `--direct-io` | on | bypasses page cache for the 26 GiB weight stream (mmap is disabled by it — deliberate) |
-| `--ctx-checkpoints 16` | 16 | slot state for checkpoint/rollback |
-| `--cache-ram 8192` + `--cache-idle-slots` | 8 GiB | prompt cache; verified a repeat pass over the same 6k prompt replays in **0.5 s** |
-| `--jinja` | on | without it `chat_template_kwargs` are silently ignored |
-| `--reasoning on --reasoning-format deepseek --reasoning-budget -1` | | thinking in its own channel, unbounded |
-
-Measured steady state (production, one model copy resident):
-`mem_info_gtt_used` **63.1 GiB** of the ~114 GiB aperture, `mem_info_vram_used`
-0.17 / 2.0 GiB (the iGPU's dedicated carve-out is a stub; everything real lives
-in GTT). llama-server RSS 2.1 GiB. Boot to `/health` 200: **16 s** — the fastest
-endpoint bring-up in the fleet, because there is no engine compile step.
-
-## Speculative decode — measured, not assumed
-
-```
---spec-type draft-mtp --model-draft <mtp-Qwen3.8-27B-Q4_0.gguf>
---spec-draft-n-max 4 --spec-draft-n-min 0 --spec-draft-p-min 0.0
---spec-draft-device ROCm0 --spec-draft-ngl 99
-```
-
-MTP in llama.cpp is a property of the file, not of the flags: the head is a
-trained tensor set (`nextn_predict_layers = 1` in this repo's head, arch
-`qwen35`, 65 blocks) and the draft context runs on the main model's weights, so
-there is no second 26 GiB copy in memory. See the fleet law in
-`AGENTS.md → llama.cpp serving`.
-
-**A/B, 2026-10-02** — same session, temp 0, 2,202-token prompt, 512 tokens,
-`cache_prompt: false`, `frank-dragoon.service` stopped for every scratch arm (the
-benchmarking law), baseline arm stripped of `--model-draft` **and** every
-`--spec-*` flag:
-
-| Arm | Decode t/s (2 runs) | Prefill t/s | Draft acceptance |
-|---|---|---|---|
-| **MTP, n-max 4 (deployed)** | **17.00 / 17.14** | 350.6 / 324.9 | 0.462 (mean acceptance length ≈ 1.86) |
-| no draft | 7.60 / 7.66 | 381.8 / 381.0 | — |
-
-**+123 % decode for an 8–15 % prefill tax.** Notes that matter before someone
-re-litigates this:
-
-- Acceptance here (0.462) is nowhere near the card's 0.953. The card measures
-  its own short, repetitive text; this is a 2.2k-token prose prompt. Treat the
-  card's number as a file property, not a throughput promise.
-- `n-max 4` is where the gain was measured; 2 and 8 are **unmeasured**. The
-  gufo-era law applies in spirit — never rank spec settings on a
-  self-predictable corpus.
-- End-to-end on the chat path is slower than the `/completion` number because
-  thinking tokens are included: a 2,253-token chat prompt generating 512
-  tokens took 43.8 s wall (11.7 t/s end-to-end) — that is the cost of an
-  unbounded reasoning budget, not a regression in decoding.
-
-## Client contract (live-verified 2026-10-02)
-
-| Behavior | Observation |
-|---|---|
-| Model id | `/v1/models` → `frank-dragoon` |
-| Tools | native: `get_weather` → `{"city": "Seattle"}` in `tool_calls` |
-| Thinking | in **`reasoning_content`** (llama.cpp field — *not* vLLM's `reasoning`); `models.yml` sets `reasoningContentField` accordingly |
-| `reasoning_effort` | accepts **only** `xhigh`, `medium`, `low`; anything else is HTTP 500 (the template raises, it does not clamp). `models.yml` folds `high`/`max` → `xhigh` |
-| Sampling | server defaults = the card's thinking recipe (temp 1.0, top_p 0.95, top_k 20, min_p 0, presence 0), mirrored in the harness `extraBody` |
-| Vision | the BF16 projector loads (`loaded multimodal model`); a red square answers **`Red`** (`usage: 75 prompt / 71 cached`) |
-| Prompt cache | second pass over the same 6k prompt: 0.5 s wall |
-
-Three traps, all hit for real:
-
-1. **`content: null` when reasoning eats the budget.** With
-   `--reasoning-parser`-style separation, a request whose `max_tokens` is
-   consumed entirely by thinking returns `message.content = None`. A bench
-   client that indexes `content` crashes *after* measuring and loses the
-   numbers — fall back to `reasoning_content`, or size the budget (200 was
-   enough for the vision probe).
-2. **`MTP boundary missing for seq_id=… ; resyncing after a non-token step`**
-   appears in the log under draft resets. Self-recovering; not an outage signal.
-3. **Never `pkill -f llama-server` on Dankclaw** — frank-zealot is also a
-   `llama-server`. The sweep key here is the payload name:
-   `pkill -f 'Qwen3.8-27B-Q8_0_ROCMFPX_AGENT'`.
-
-## Operations
-
-```bash
-# deploy (planner selects this endpoint only via its own unit/env/serve trio)
-./scripts/deploy.sh dankclaw --service frank-dragoon
-
-# status / logs / cause of a crash loop
-systemctl status frank-dragoon
-journalctl -u frank-dragoon -n 200 --no-pager
-
-# rebuild the pinned fork (only when deliberately moving the pin)
-scripts/build-rocmfpx.sh
-
-# re-verify the payload without re-downloading
-scripts/download-frank-dragoon-kingjones.sh --verify-only
-```
-
-`serve-frank-dragoon.sh` refuses to start — before touching the port — unless the
-fork binary, the weights, the MTP head and (unless `FRANK_DRAGOON_MMPROJ=""`) the
-projector all exist; `scripts/deploy-remote.sh` mirrors the same predicate so a
-broken payload never stops the running unit. Caller environment **overrides**
-`frank-dragoon.env`, and defaults use `:=` so an explicitly empty value surfaces
-as the error it reads as.
-
-## Rollback (as of retirement, 2026-10-03)
-
-**Primary rollback: the ALIGNED Q8 file itself is the previous shape's
-payload and remains on disk** at
-`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` — revert the
-2026-10-03 swap commit (unit/env/serve trio) or repoint
-`FRANK_DRAGOON_MODEL_FILE`/`LLM_DRAGOON_MODEL_FILE`, verify with
-`scripts/download-frank-dragoon-kingjones.sh --verify-only`, redeploy.
-
-git tag **`frank-dragoon-gufo-q4kxl-20261001`** (gufo v0.1.1, Q4_K_XL +
-DFlash2) is now a **re-download** rollback: the gufo payloads
-`/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL/` (16.35 GiB) and
-`/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL/` (incl. the DFlash2 drafter) were
-removed from disk on 2026-10-03 with Frank's explicit approval to free space.
-`scripts/build-gufo.sh` remains on disk; re-fetch per
-[[frank-dragoon-gufo-qwen38-27b]].
-
-The one-day vLLM MiMo era's payloads (MiMo-V2.6 BF16 18 GiB,
-Qwen3.5-9B MTP source 19 GiB, both DFlash draft dirs) were staged to
-`/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02 —
-recoverable until the trash is purged, and never part of a rollback plan.
-
-## When it misbehaves
-
-- **`unknown tensor type` / ftype 115 refused** — something moved off the
-  ROCmFPX binary. Check `readlink -f $(systemctl show frank-dragoon -p ExecStart)`
-  and the fork commit; this file cannot run on stock llama.cpp or gufo.
-- **SIGSEGV in `ggml_compute_forward_set_rows` during load** — a fork build
-  older than the `36e9acd` pin; rebuild with `scripts/build-rocmfpx.sh`.
-  History: [[kingjones-q8agent-rocmfpx]].
-- **Slow decode with no log error** — check whether the draft is engaged
-  (`/metrics`, `draft_n` / `draft_n_accepted`; a zero draft count means the head
-  did not load), and check GTT pressure with
-  `cat /sys/class/drm/card1/device/mem_info_gtt_used` — a second resident copy
-  of this model (an orphaned scratch server) drives the whole host into
-  eviction and halves the rate silently.
-- **`500` naming Jinja** — a `chat_template_kwargs` field reached a server
-  without `--jinja`.
-- **HTTP 500 naming the template on the first request of a role** — an
-  unmapped `reasoning_effort`; fix the fold in `models.yml`, not the model.

serverAppConfigs-docs/LLM/frank-dragoon-unsloth-qwen38-27b-q8xl.md

@@ -1,117 +0,0 @@
----
-type: reference
-title: "stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2) on Dankclaw (frank-dragoon)"
-description: "The RETIRED frank-dragoon runbook (2026-09-21 → 2026-09-28; replaced by [[frank-dragoon-gufo-qwen38-27b]], and the rollback target) — the stock llama.cpp prebuilt (unsloth b11030, ROCm gfx1151) serving unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (27B dense, 31.46 GiB) with the z-lab DFlash2 Q4_K_M drafter (depth 4) and q8_0 KV on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the full-window/one-stream shape, the DFlash2 spec method, the Qwen-Sharp template, the payload, the client contract, and the failure modes worth knowing."
-tags:
-  - llm
-  - llama.cpp
-  - unsloth
-  - dankclaw
-  - frank-dragoon
-  - strix-halo
-  - gfx1151
-  - q8
-  - dflash2
-  - rocm
-  - q8-kv
-timestamp: 2026-09-21
----
-
-# frank-dragoon — stock llama.cpp Qwen3.8-27B UD-Q8_K_XL (DFlash2)
-
-> **RETIRED 2026-09-28**, replaced by [[frank-dragoon-gufo-qwen38-27b]] (the
-> gufo engine, same weights, same port and alias); the prebuilt is intact on
-> disk as the rollback target. ACTIVE from 2026-09-21, when it replaced
-> [[frank-dragoon-kairic-edge-qwen38-27b]] (Kairic Edge IU4 W4A4) and
-> [[frank-dragoon-unsloth-qwen38-flash-next]] (the Flash-Next MoE it
-> preceded).
-
-## Identity
-
-- **Endpoint:** `frank-dragoon` at `192.168.0.143:7770` (Dankclaw).
-- **Runtime:** stock llama.cpp — the unsloth `b11030` prebuilt
-  (`/home/dank/GitPackages/unsloth-llama-b11030/llama-server`), ROCm gfx1151.
-  A mainline build, not a fork: it supports the `qwen3_5` architecture and the
-  DFlash2 drafter (`llama_model_qwen35`, `llama_model_dflash`,
-  `common_speculative_impl_draft_dflash`).
-- **Service shape:** root systemd unit `scripts/frank-dragoon.service`
-  (`User=dank`), launched by `scripts/serve-frank-dragoon.sh` with tunables from
-  `scripts/frank-dragoon.env`.
-
-## Shape
-
-- **Model:** `unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL` — a 27B **dense** model,
-  31.46 GiB GGUF (sha256-verified by `scripts/download-frank-dragoon.sh`).
-- **Context:** one stream, full 262,144 tokens (the model's native window),
-  `PARALLEL_SLOTS=1`.
-- **Placement:** all layers on the iGPU (`--n-gpu-layers 999`). The 27B dense
-  weights (31.46 GiB) + the DFlash2 drafter (1.14 GiB) + q8_0 KV (~8.6 GiB) +
-  compute buffers fit the iGPU's GTT aperture (~114 GiB).
-
-## Spec method
-
-DFlash2 (a block-diffusion drafter) at **depth 4**. The drafter is the z-lab
-`Qwen3.8-27B-DFlash2-Q4_K_M` (1.14 GiB), loaded via `--model-draft` with
-`--spec-type draft-dflash` and `--spec-draft-device ROCm0`. It has a sliding
-window of 2048, so its draft KV is small (~40 MiB). Greedy verification
-(`--temp 0 --top-p 1.0 --top-k 0 --min-p 0.0`) is required.
-
-## KV
-
-**q8_0** (`--cache-type-k q8_0 --cache-type-v q8_0`), ~8.6 GiB at 262k. The
-unsloth b11030 build does **not** support fp8 KV (allowed: f32/f16/bf16/q8_0/
-q4_0/q4_1/iq4_nl/q5_0/q5_1); q8_0 is the highest-precision 8-bit option.
-
-## Template
-
-The Qwen-Sharp v22.4.0 template is pinned via `--chat-template-file`
-(`dankserver/frank-medium/qwen-sharp-v22.4.0.jinja`) with `--jinja` and
-`--chat-template-kwargs '{"preserve_thinking":true}'`. Without
-`--chat-template-file` the binary falls back to the built-in `peg-native`
-template, which produces a ~73,600-token prompt for a ~250-token request.
-`preserve_thinking` keeps prior thinking blocks across turns for multi-turn
-agents. The template gates `reasoning_effort` to `{xhigh, medium, low}` — the
-harness folds its levels onto that set in `models.yml` (`reasoningEffortMap`).
-
-## Payload
-
-`scripts/download-frank-dragoon.sh` fetches the main GGUF and the DFlash2
-drafter, each verified against its declared byte count + sha256
-(`--verify-only` re-checks). The serve script refuses to start if the binary,
-the main GGUF, or the drafter is missing.
-
-## Client contract
-
-- **Alias:** `frank-dragoon` (the `id` at `/v1/models`).
-- **Reasoning:** extracted to `reasoning_content` (`--reasoning auto`,
-  `--reasoning-format deepseek`, unlimited budget).
-- **Tools:** supported (`--jinja` is pinned).
-- **Vision:** none — text-only (no mmproj), matching the prior frank-dragoon.
-- **Sampling:** greedy by default; a client that sends a temperature gets its
-  own, and the harness applies per-model overrides from `models.yml`.
-
-## Failure modes worth knowing
-
-- **Model load is slow (minutes cold).** Health returns 503 while loading; do
-  not kill the process. `deploy.sh` waits up to 20 minutes (`LLM_WAIT`).
-- **`--ngl` (double-dash) is invalid in b11030** — use `--n-gpu-layers`
-  (the serve script uses the long form).
-- **The 8-bit UD-Q8_K_XL decodes ~17 t/s** vs the prior 4-bit Kairic Edge IU4
-  at ~22 t/s — the ~23 % larger weight size (31.46 vs 26 GiB) explains the
-  gap; it is not a regression.
-- **DFlash2 draft acceptance is text-variety-dependent** (0.85 on a trivial
-  one-word answer, 0.519 on a varied explanatory prompt) — do not read a
-  single-request acceptance as the steady state.
-
-## Rollback
-
-Payload and runtime for [[frank-dragoon-kairic-edge-qwen38-27b]] (Kairic Edge
-IU4 W4A4) remain intact on disk. See [[frank-dragoon-unsloth-qwen38-flash-next]]
-for the Flash-Next predecessor it replaced on 2026-09-21.
-
-# Citations
-
-- unsloth prebuilts: https://huggingface.co/unsloth
-- Model: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
-- DFlash2 drafter: https://huggingface.co/z-lab
-- Serve script: `scripts/serve-frank-dragoon.sh`; tunables: `scripts/frank-dragoon.env`

serverAppConfigs-docs/LLM/frank-dragoon-unsloth-qwen38-flash-next.md

@@ -1,47 +0,0 @@
----
-type: reference
-title: "unsloth llama.cpp Qwen3.8-Flash-Next UD-IQ4_XS on Dankclaw (frank-dragoon)"
-description: "The RETIRED frank-dragoon runbook (2026-09-21, same-day swap; replaced by [[frank-dragoon-unsloth-qwen38-27b-q8xl]]) — the unsloth llama.cpp prebuilt (b11030-mix-5ff778e, ROCm gfx1151) serving unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS on the Dankclaw Strix Halo iGPU: the prebuilt runtime, the 200k/one-stream shape, the explicit -md MTP wiring, the payload, the client contract, and the failure modes worth knowing."
-tags:
-  - llm
-  - unsloth
-  - dankclaw
-  - frank-dragoon
-  - strix-halo
-  - gfx1151
-  - qwen4exp
-  - rocm
-  - mtp
-timestamp: 2026-09-21
----
-
-# frank-dragoon — unsloth llama.cpp Qwen3.8-Flash-Next UD-IQ4_XS (RETIRED)
-
-> **RETIRED 2026-09-21 (same-day swap).** This was the ACTIVE frank-dragoon
-> runbook on 2026-09-21 before it was replaced by
-> [[frank-dragoon-unsloth-qwen38-27b-q8xl]] (27B dense UD-Q8_K_XL) later the
-> same day. The payload and runtime remain on disk as the rollback target.
-
-## What it served
-
-- **Runtime:** unsloth llama.cpp prebuilt `b11030-mix-5ff778e` (ROCm
-  gfx1151), root systemd unit, port 7770.
-- **Model:** `unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS` (a 125B MoE / 6B
-  active), 87.3 GiB shipped as three split parts. The split parts ARE the
-  payload — concatenating them trips `invalid split file name` (part 1's
-  header advertises `file_count=3`).
-- **Shape:** one 200,000-token conversation (single stream), MTP depth 2 via
-  the explicit `-md` shared-Q8_0 head. Draft acceptance verified live
-  0.83–0.92.
-- **Memory:** the 114 GiB GTT aperture on the Strix Halo iGPU.
-
-## Rollback
-
-Payload and runtime intact on disk. See [[frank-dragoon-unsloth-qwen38-27b-q8xl]]
-for the current ACTIVE runbook and [[frank-dragoon-ciru-orca]] for the
-predecessor.
-
-# Citations
-
-- unsloth prebuilts: https://huggingface.co/unsloth
-- Model: https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF

serverAppConfigs-docs/LLM/frank-dragoon-vllm-mimo-v26-bf16.md

@@ -1,186 +0,0 @@
----
-type: reference
-title: "frank-dragoon on Dankclaw — vLLM TheRock, MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (RETIRED)"
-description: "The RETIRED frank-dragoon runbook (2026-10-01, one day) — the digest-pinned kyuz0 TheRock gfx1151 vLLM image serving XiaomiMiMo MiMo-V2.6-Distill-Qwen-9B in full BF16 on the Strix Halo iGPU. Kept for the measured record: the roof math, the cross-model MTP drafter, and the autoresearch finding that spec decode is NET-NEGATIVE on real chat+reasoning traffic here (6.19 t/s with MTP-2 vs 9.01 t/s without). Rolled back to [[frank-dragoon-gufo-qwen38-27b]] by Frank the same night."
-tags:
-  - llm
-  - frank-dragoon
-  - dankclaw
-  - vllm
-  - strix-halo
-  - mimo
-  - mtp
-  - spec-decode
-  - bf16
-timestamp: 2026-10-01
----
-
-# frank-dragoon — vLLM MiMo-V2.6-Distill-Qwen-9B BF16 (cross-model MTP)
-
-> **RETIRED 2026-10-01 night, one day active.** Frank's call after the
-> autoresearch loop: battery decode 6.19 t/s (MTP-2) / 9.01 t/s (no draft)
-> against gufo's ~15 t/s on the 27B — and every ≥6-bit quant path in vLLM
-> 0.30 is dead on gfx1151 (quark int8 rejected; all FP8 kernels CDNA-only;
-> torchao config unwired). Back to [[frank-dragoon-gufo-qwen38-27b]]
-> (Q4_K_XL, 1 x 262144). The measured content below stands as the record.
-
-## Identity
-
-- **Endpoint:** `frank-dragoon` at `192.168.0.143:7770` (Dankclaw).
-- **Runtime:** vLLM 0.30.0 in the **digest-pinned** `docker.io/kyuz0/vllm-therock-gfx1151`
-  image (image ID `2d24a8c5a3fd`, digest `sha256:98c1f3ac9833…` — ROCm 10.0.0 /
-  torch 2.11.0 / TheRock, Triton Gated DeltaNet kernels). The official vLLM
-  images are CUDA-only; this fork is the fleet's gfx1151 path. Plain
-  `podman run` driven by `scripts/serve-frank-dragoon.sh` under a root unit
-  (`scripts/frank-dragoon.service`, `User=dank`, `Restart=always`,
-  `ExecStop=podman stop -t 120`, `TimeoutStopSec=180` > the stop grace).
-- **Model:** `XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B` in **full BF16**
-  (17.53 GiB, 4 shards) — the same checkpoint frank-zealot serves as an INT8
-  W8A16 AutoRound conversion on the 3090, here unquantized.
-- **Draft:** `Qwen/Qwen3.5-9B` (18 GiB on disk; only the 15 `mtp.*` tensors
-  are loaded into the draft pass).
-
-## Why this shape (the roof math)
-
-Decode speed for a dense model on Strix Halo is bandwidth-bound:
-BF16 weights x one pass per token against the LPDDR5x aperture. Measured
-2026-10-01 on this host: **13.1 t/s is ~94% of the roof** — the payload, not
-a misconfiguration. The same INT8 payload reaches 83.9 t/s on the 3090
-because it moves ~12 GiB through ~936 GB/s. The only way past a bandwidth
-wall is generating more than one token per weight pass: **speculative decode
-is the performance feature of this endpoint**, not a garnish.
-
-The memory side is the opposite of a squeeze: KV grows ONLY on the 8 full
-attention layers (32 KiB/token bf16: 8 x 2 x 4 heads x 256 dim x 2 B);
-the 24 Gated DeltaNet layers hold a FIXED ~25 MiB state per running
-sequence. `--max-model-len 262144` (the native window) with
-`--max-num-seqs 4` at `--gpu-memory-utilization 0.45` boots clean — the
-four-session contract the gufo shape ran, with the pool shared by the
-scheduler instead of carved per session.
-
-## Spec method: cross-model MTP — measured, not assumed
-
-| arm | prefill | decode | acceptance |
-|---|---|---|---|
-| BF16 base | 17,876 t/s | 13.1 t/s | — |
-| BF16 + DFlash 8 tok | 7,976 t/s | 17.4 t/s | pos0 59% pos1 26% (token rate 14.8%) |
-| **BF16 + MTP 2 tok** | 7,380 t/s | **19.8 t/s** | pos0 71% pos1 44% (token rate 57.4%) |
-
-One session, temperature 0, 28,647-token prefill / 512-token decode, same
-client for all three arms (2026-10-01 bench window, frank-dragoon stopped per
-Benchmarking Policy). **MTP wins (+51% vs +33%) and needs only 2 draft
-tokens** — acceptance past position 1 is 44% and falling; DFlash's own curve
-decays 59/26/14/7, so 8 proposals are mostly wasted verification.
-`num_speculative_tokens` beyond 2 is UNMEASURED; do not raise it without a
-fresh A/B. Both spec arms pay ~55–59% of prefill (the draft passes share
-the compute queues) — this endpoint's thinking-heavy traffic spends its wall
-clock in decode, which is the regime where that trade wins.
-
-**Why a Qwen head accepts on a MiMo target:** MiMo-V2.6 is a distillation of
-Qwen3.5-9B that preserves the hidden-state geometry — identical
-4096-hidden/248320-vocab/32-layer/interval-4 structure, and the 71%/44%
-per-position acceptance is the direct measurement that the distillation
-preserved more than geometry. MiMo itself ships **ZERO `mtp.*` tensors**
-(verified in both the BF16 base and the INT8 conversion) — which is exactly
-why frank-zealot's SGLang NEXTN draft is a 0.00-acceptance ghost: SGLang
-reads the head from the TARGET checkpoint, vLLM reads the draft from a
-separate path. That difference is the whole trick:
-
-```
---speculative-config '{"method":"mtp","model":"/models/Qwen3.5-9B","num_speculative_tokens":2}'
-```
-
-The alternative drafter (`z-lab/Qwen3.5-9B-DFlash`, 6 layers, target_layer_ids
-[1,5,…,29]) stays fetched on disk; flip `FRANK_DRAGOON_SPEC=dflash` and repoint
-`FRANK_DRAGOON_SPEC_DIR` to use it. Its card's "vLLM PR #40898 unmerged" claim
-is FALSE — `vllm/v1/spec_decode/dflash.py` ships in 0.30.
-
-## The numerics trap (greedy text is NOT preserved by spec decode)
-
-Measured: temperature-0 completions DIFFER across arms (sha `1318aeb6` base,
-`0208b45d` dflash, `20c41c38` mtp). Batched verification changes the bf16
-reduction order, argmax near-ties diverge, and the divergence cascades.
-vLLM spec decode is distribution-preserving, not bitwise-greedy-preserving.
-The gufo-era "greedy == autoregressive text" law does not carry over; do not
-chase text drift between spec and no-spec arms as a regression.
-
-## Payload
-
-- Target: 4 safetensors shards (17.53 GiB) + config/tokenizer under
-  `/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-BF16/`, each shard
-  byte-count + LFS sha256-verified by
-  `old/download-frank-dragoon-mimo.sh` (`--verify-only` re-checks; no token on
-  the curl command line).
-- MTP source: `old/download-frank-dragoon-qwen35.sh` →
-  `/home/dank/models/Qwen3.5-9B/` (4 shards behind
-  `model.safetensors.index.json`; the shards carry the odd HF asset names
-  `model.safetensors-0000N-of-00004.safetensors` — the index resolves them,
-  the engine boots from exactly this tree).
-- DFlash drafter: `old/download-frank-dragoon-dflash.sh` →
-  `/home/dank/models/Qwen3.5-9B-DFlash/`.
-- **Payload status 2026-10-02:** all three trees moved to
-  `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` (59 GiB staged with
-  the other retired frank payloads) and the fetch trio moved to `old/` — the
-  endpoint is two runtimes past this one, and re-fetch is cheaper than disk.
-- The serve script gates the image by REPO DIGEST (rootless podman resolves
-  tags from the network; the pin exists to boot exactly these bytes
-  offline), then config + index + exact shard count, then the spec draft
-  (either weight layout: single `model.safetensors` or a sharded index),
-  then `/dev/kfd` + whole `/dev/dri` (never individual renderD nodes —
-  Dankserver 2026-09-29 lesson).
-
-## Client contract
-
-- **Alias:** `frank-dragoon` (the `id` at `/v1/models`).
-- **Reasoning:** `--reasoning-parser qwen3`; vLLM emits the thinking text in
-  **`reasoning`**, NOT llama.cpp/gufo's `reasoning_content` — the harness
-  `models.yml` compat block for this provider sets
-  `reasoningContentField: "reasoning"`. Miss that and thinking silently
-  vanishes from transcripts.
-- **Tools:** `--enable-auto-tool-choice --tool-call-parser qwen3_coder`.
-- **Vision:** the checkpoint carries the `qwen3_5_vision` tower but the
-  serve script passes `--language-model-only` — the endpoint stays
-  text-only by decision; dropping the flag brings vision back.
-- **Sampling:** request-side; the harness sends the Qwen3.5-family recipe
-  (temp 0.7, top_p 0.8) via `compat.extraBody`, and
-  `chat_template_kwargs.{enable_thinking,preserve_thinking}` — the
-  `chat_template_kwargs` form is what vLLM reads; a top-level
-  `preserve_thinking` is ignored there.
-- **Env hygiene:** every tunable is `FRANK_DRAGOON_`-prefixed because HIP
-  reads bare `TEMP`/`HOST`/`GPU_MAX_HEAP_SIZE` and the serve script exports
-  the env file with `set -a`.
-
-## Ops
-
-- Boot ~200–240 s warm; health 503 while loading (do not kill it).
-- The serve script sweeps stale container identities before starting:
-  `frank-dragoon-mimo` itself and the bench-window container `vllm-mimo`.
-- Container death → `podman wait` returns non-zero from ExecStart →
-  `Restart=always` self-heals (a clean exit-0 drain also restarts, the
-  frank-archon lesson).
-- Bench windows: `systemctl stop frank-dragoon` is enough (mask fails on the
-  relinked unit; the deploy restores).
-- First-deploy facts (2026-10-01, deploy #3): healthy after **230 s**;
-  boot log `GPU KV cache size: 819,332 tokens, Maximum concurrency for
-  262,144 tokens per request: 3.13x` — four sessions share the pool
-  (the scheduler, not a carve). Live battery green: alias `frank-dragoon`,
-  reasoning split into `reasoning`, tool_calls with correct args, and
-  `/metrics` acceptance pos0 69% pos1 44% (54/86 draft tokens accepted),
-  matching the bench window.
-
-## Rollback
-
-Git tag **`frank-dragoon-gufo-q4kxl-20261001`** — gufo v0.1.1 (built from
-source, `scripts/build-gufo.sh` stays on disk) + the Q4_K_XL Qwen3.8-27B
-payload + DFlash2 drafter, all still on the host. Restoring the tag's
-unit/env/serve trio and redeploying returns the 27B endpoint.
-
-# Citations
-
-- Model: https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B
-- Draft source: https://huggingface.co/Qwen/Qwen3.5-9B
-- DFlash drafter: https://huggingface.co/z-lab/Qwen3.5-9B-DFlash
-- Engine image: https://hub.docker.com/r/kyuz0/vllm-therock-gfx1151
-- Serve script: `scripts/serve-frank-dragoon.sh`; tunables: `scripts/frank-dragoon.env`
-- Predecessor: [[frank-dragoon-gufo-qwen38-27b]]; sibling on the same arch:
-  [[frank-zealot-sglang-mimo-v26]]

serverAppConfigs-docs/LLM/frank-dragoon.md

@@ -0,0 +1,236 @@
+---
+type: reference
+title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
+description: "The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env, every retired era with its rollback path, the durable laws this endpoint paid for, the wire contract, and the files that own it."
+tags:
+  - llm
+  - frank-dragoon
+  - dankclaw
+  - strix-halo
+  - gfx1151
+  - halogen
+timestamp: 2026-10-18
+---
+
+# frank-dragoon — halogen-flash-server 0.16.2, native v2 abliterated (Dankclaw iGPU)
+
+Strix Halo (gfx1151) endpoint `http://192.168.0.143:7770`, served id `frank-dragoon`: a podman container
+under a root `User=dank` unit. Live shape since 2026-10-17, pool raised to 401,408 on 2026-10-18
+(`2b12716`). This page replaces every per-era dragoon runbook; the Eras table is the rollback record.
+
+## Current shape
+
+`scripts/frank-dragoon.env` is authoritative and `scripts/serve-frank-dragoon.sh` passes exactly those
+flags and nothing else. Payload `/home/dank/models/halogen-v2-abliterated/`, verified by
+`scripts/download-frank-dragoon-v2abl.sh --verify-only`:
+
+| File | Bytes | sha256 | Source |
+|---|---|---|---|
+| `qwen38-flash-next-v2-abliterated.hgn` | 66,687,678,432 | `b5bd925f…` | Quat3rnion `@main` — the NATIVE 2026-10-11 v2 conversion of orcarouter's BF16 Qwen3.8-Flash-Next, abliterated |
+| `qwen38-flash-next-ngram.hgn` | 51,200,246,144 | `3450acada…` | peonist `@5cc17cea`, **REQUIRED**, hardlinked |
+| `qwen38-flash-next-vision.hgn` | 897,916,416 | `d62e0ae5…` | peonist `@e053f488`, hardlinked — vision ON |
+| `tokenizer/` (6 flat files) | ~22 MB | `tokenizer.json` LFS | peonist `@e053f488`, hardlinked |
+
+Image `ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a`
+(0.16.2, closed source), container `frank-dragoon-halogen`, published `7770:8731`; the engine's
+unauthenticated token port 8730 stays on container loopback. Engine stdout →
+`/home/dank/logs/frank_dragoon.log`, **not** the journal.
+
+- Shape: `HALOGEN_CTX=200704` hard per-request ceiling · `HALOGEN_KV_SLOTS=2` ·
+  `HALOGEN_KV_POOL_POSITIONS=401408` = slots × ctx, the top of the band `tests/frank_dragoon_unit.bats`
+  enforces (`ctx ≤ pool ≤ slots × ctx`) · KV ~28 KiB/position → **11.0 GiB reserved** · no KV-dtype knob
+  exists here, the pool is the only KV lever · native window 262,144, `HALOGEN_ROPE_YARN` beyond it.
+- Knobs: `HALOGEN_MAX_TOK=8192` sizes the prefill arena (5.7 GiB) and is only the default for requests
+  omitting `max_tokens` · `HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled
+  pair · MTP depth 2 with the head INSIDE the checkpoint (**`HALOGEN_MTP_HEAD` stays unset**) and
+  `HALOGEN_SPEC_ADAPT=0` · sampler 1.0 / 0.95 / 20 server-side · `HALOGEN_WEIGHTS_LOCK=1` ·
+  `HALOGEN_HOST_RESERVE_GIB=20` · prompt cache ON by image default (`PROMPT_CACHE=2`, no flag
+  passed): 12 entries, which the boot ledger prices at 111 MiB each at this window = 1.3 GiB
+  (the env header's "~85 MiB here" estimate did not survive that ledger line).
+- Deployed boot ledger: `62.9 GiB of weights locked in RAM (0.8 of them the vision tower's), 11.0 GiB
+  of KV pool, 5.7 GiB of working memory, 79.6 GiB in all`; n-gram `mapped 51.2 GB, NOT pinned
+  (Pin::None)`; no clamp line. Host `MemAvailable` read 22 GiB right after the 2026-10-18 raise —
+  ~20 GiB above the 20 GiB reserve, against 10–15 GiB of baseline co-tenancy (frank-zealot, ComfyUI,
+  Caddy/observability).
+- Harness (`~/.omp/agent/models.yml`, provider `llamacpp (dankclaw)`): `contextWindow: 155648`
+  (= ceiling − 45k per the 400-zone law, never higher), `maxTokens: 45056`, `input: [text, image]`,
+  `reasoningContentField: "reasoning_content"`, `high→xhigh` fold, `extraBody` 1.0/0.95. Compaction
+  rides [[frank-zealot]].
+- Deploy `./scripts/deploy.sh dankclaw --service frank-dragoon`; gate = boot-ledger greps (weights,
+  pool, MTP, prompt cache, **no clamp**) + `/health` 200 + a tool call + the 512 px red-PNG probe.
+  Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it) and `amdgpu.noretry=0` — the env says check it at
+  deploy; boot line per [[dankclaw-iommu-translation-off]].
+
+## Eras
+
+Rollback = revert that era's swap commit (or repoint the payload vars) and redeploy
+`dankclaw --service frank-dragoon`. Newest first; payloads staged under `~/.AGENTTRASH/` are **not** a
+rollback plan — recoverable only until the trash is purged.
+
+| Window | Payload | Runtime | Shape | Replaced because | Rollback |
+|---|---|---|---|---|---|
+| 2026-10-06 → 10-17 | Swift1.5 abliterated v2 `.hgn` r2 66,687,678,464 B `9c880b8b…` + same trio | flash-server 0.16.2, same digest | 2 slots, ctx 262,144, pool 524,288 (14.6 GiB), MTP always-on | 2026-10-17 A/B at that shape: NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls + red-PNG probe identical, needle hit at both depths of a 21.5k haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s, PPL 3.968 vs 3.877 (finetune's edge gone: +2.3 %), KL vs the swift15 logits mean 0.155 / median 0.057 with top-1 agreement 85.1 % — two sound models disagreeing on content, not a corrupt forward pass, thinking 1,476 vs 784 mean, 7.4 GiB RAM back | `/home/dank/models/halogen-flashnext-v2/` + `scripts/download-frank-dragoon-flashnext.sh` |
+| 2026-10-05 → 10-06 | peonist `halogen-qwen3.8-27b` `qwen3.8-27b-p1w4d-d2.hgn` 35,865,565,184 B `274c3fc7…` | halogen 0.1.4 `sha256:dc0a39a0…` | 1 slot × 200,700, DFlash2 drafter, `HALOGEN_CACHE_MB=16384`, text-only, censored base | flash-server v2 restored vision, abliteration and MTP | `/home/dank/models/halogen-qwen3.8-27b/` + `scripts/download-frank-dragoon-halogen.sh`; 0.1.4 digest still in podman storage |
+| 2026-10-03 → 10-05 | kingjones777 Uncensored `Q6_0_ROCMFPX_AGENT.gguf` 24,856,607,008 B `e966deb3…` + `mtp-Q4_0` `051a1764…` + BF16 mmproj | `kingjones30/ROCmFPX` @ `36e9acd` (`scripts/build-rocmfpx.sh`) | 1 × 262,144 f16 KV, `draft-mtp` n-max 4, vision on | halogen card claimed 31.7 t/s vs 27.6–28.1 measured | `/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/` + `/home/dank/GitPackages/kingjones-ROCmFPX/build` + `download-frank-dragoon-kingjones-uncensored.sh` |
+| 2026-10-02 → 10-03 | aligned `Q8_0_ROCMFPX_AGENT.gguf` 28,214,425,952 B `f3ef2bdb…` | same fork pin | 1 × 262,144 f16 KV, `draft-mtp` 4 (MTP +123 % decode) | the uncensored file was faster *and* uncensored | `/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` + `download-frank-dragoon-kingjones.sh` |
+| 2026-10-01 (one day) | MiMo-V2.6-Distill-Qwen-9B BF16 17.53 GiB + cross-model MTP from `Qwen3.5-9B` | vLLM 0.30 TheRock `kyuz0/vllm-therock-gfx1151` `sha256:98c1f3ac…` | 4 seqs × 262,144 on a scheduler-shared pool, 2 draft tokens | 6.19 t/s with MTP / 9.01 without vs gufo ~15, and no ≥6-bit vLLM quant path boots on gfx1151 | staged `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` until purged; fetch trio in `old/` |
+| 2026-09-28 → 10-01, again 10-01 → 10-02 | Swift-1.5 Uncensored `UD-Q8_K_XL` 31,457,990,272 B `9076f99a…`, then `UD-Q4_K_XL`; z-lab DFlash2 Q4_K_M | gufo v0.1.1 (`b0f8673`), source build | 1 × 262,144, `--draft-policy adaptive --draft-tokens 7`, no KV knob, no Jinja | ROCmFPX gave 8.39 bpw + MTP at 17 t/s | payloads **deleted 2026-10-03** → re-download `download-frank-dragoon-swift.sh` (+ `download-frank-dragoon.sh` for the drafter); tag `frank-dragoon-gufo-q4kxl-20261001`; `scripts/build-gufo.sh` |
+| 2026-09-21 → 09-28 | unsloth `Qwen3.8-27B-GGUF:UD-Q8_K_XL` 31.46 GiB + DFlash2 Q4_K_M 1.14 GiB | unsloth llama.cpp `b11030` prebuilt | 1 × 262,144, q8_0 KV, Qwen-Sharp v22.4.0 template | gufo: +37 % prefill shallow, +80 % at 131k, 4.5× faster load | `/home/dank/GitPackages/unsloth-llama-b11030/` on disk; payload deleted 2026-10-03 → `download-frank-dragoon-unsloth.sh` |
+| 2026-09-21 (same day) | unsloth `Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS` 87.3 GiB in 3 split parts | same prebuilt | 1 × 200,000, MTP depth 2 via explicit `-md` shared Q8_0 head | swapped to the 27B dense hours later | payload + prebuilt on disk at retirement; the parts ARE the payload (concatenating → `invalid split file name`) |
+| 2026-09-21 (same day) | `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` + 3 `.pfs` = 26.05 GiB | ciru-ai/ROCmFPX `kairic-edge-qwen38-27b-v1.2` @ `205a3e5` | 1 × 262,144, native MTP 4 + n-gram | same-day swap to the 27B dense | payload + runtime intact on disk at retirement |
+| 2026-09-19 → 09-21 | `jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca` | CIRU v4.4.1 (QSA window hotfix) | never written down — that runbook is a frontmatter stub | the stock llama.cpp prebuilt won | `scripts/download-frank-dragoon-orca.sh` |
+| 2026-09-11 → 09-15, again 09-16 → 09-19 | `peonist-ai/halogen-qwen3.8-flash-next` w4b 115.55 GiB + in-place-versioned `overlay.hgn` | halogen-flash-server 0.5.8, rootless Quadlet | 2 × 200,192 over a 400,384 pool, `MAX_TOK 24576`, `HOST_RESERVE 20`, vision OFF | displaced by CIRU Orca; the shape starved co-tenants | re-tag 0.5.8 + first-start `HALOGEN_DOWNLOAD` re-fetch into `/home/dank/models/halogen`; the Quadlet left the repo 2026-10-05, so the unit must be rewritten |
+| 2026-09-15 → 09-16 (one day) | `mradermacher/Qwen3.8-Flash-Next-Uncensored` i1-Q4_K_M 111 GiB + f16 mmproj | ik_llama.cpp `d85a1d7`, 3090 + CPU hybrid | 1 × 262,144, `-ot exps=CPU`, `GGML_CUDA_NO_PINNED_WEIGHTS=1` | 3.8–4.0 t/s: ~100 CPU↔GPU graph splits per token, GPU at 5 % SM | `/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`, `/home/dank/GitPackages/ik_llama.cpp/build`, trio `scripts/.AGENTTRASH/frank-dragoon.{env,service}.ik-llama.cpp` + `serve-frank-dragoon.sh.ik-llama.cpp` |
+| 2026-08-30 → 09-11 | kingjones STRIX `Q4_0_ROCMFP4-STRIX.gguf` 113.47 GiB (51.2 B PLE via mmap) | ROCmFPX @ `36e9acd` | 2 × 200,704 over one 401,408 f16 window, spec off, f16 mmproj | halogen-flash-server ≈ 4× prefill | payload dir + `/home/dank/GitPackages/kingjones-ROCmFPX/build`; trio `.AGENTTRASH/frank-dragoon.{env,service}.llama.cpp` + `serve-frank-dragoon.sh.llama.cpp` |
+| 2026-08-25 → 08-29 (restored 08-29 → 08-30 after the Q8_0 AGENT file crashed before it ever served) | `kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP`: `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` 23.15 GiB (private ftype 114) + a **SEPARATE** MTP head `mtp-Qwen3.8-27B-Q4_0.gguf` 1.56 GiB via `--model-draft` + `mmproj-Qwen3.8-27B-Q8_0.gguf` 0.59 GiB | CIRU ROCmFPX llama.cpp (`build-promptforge`) | 1 slot at `-c 196608`, `-b 8192 -ub 8192 -ctxcp 32`, `-fa on --no-mmap`, f16 K+V, `draft-mtp` n-max 4 with the head on ROCm1 at f16 KV, QwenSharp template (`enable_thinking`, `preserve_thinking`) behind the `peg-native` + `deepseek` chat/reasoning pairing the Pi agent expected | the Q8_0 AGENT experiment, then the STRIX Q4_0 tier | `/home/dank/models/kingjones-Qwen3.8-27B-STRIX-MTP/` (six card files + projector), runtime `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server` |
+| 2026-08-29 → 08-30 (one day) | agentionai FAST-imatrix **`v2/` joined-PLE** file 87.06 GiB | charlie12345/ROCmFPX `build-qwen38` + `scripts/patches/qwen4exp-qsa-f32-mask.patch` | 2 × 200,704 / 401,408 f16 KV, no MTP head loads | kingjones STRIX on its own pinned fork | `/home/dank/GitPackages/ROCmFPX/build-qwen38` + the agentionai payload, both on disk |
+| 2026-08-18 → 08-25 | caf-eemin `Q4_0_ROCMI4.gguf` 13.50 GiB `ad51ecbe…` with embedded NextN/MTP | charlie12345/ROCmFPX, ROCmI4 W4A4 | one window, MTP `n4 / p0.60` | the 27B STRIX tier took the slot | re-fetch into `/home/dank/models/eemin-Qwen3.8-27B-ROCMI4-MTP/` |
+| through 2026-08-24 | CIRU ActiveFPX PromptForge v3: GGUF 15.44 GiB + 3 `.pfs` + mmproj ≈ 36.7 GiB | CIRU ROCmFPX `build-promptforge` | 1 × 262,144, native MTP depth 4, ngram off | eemin ROCmI4 | `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/` + `/home/dank/ciru-qwen38/ROCmFPX/build-promptforge/bin/llama-server`; scripts `git revert 4c43839 7aee1df 40d50fe` or checkout `132e147` |
+| 2026-08-22 (same day) | `jcbtc/Qwen3.8-27B-IU4-Kairic-Edge` 15.48 GiB + 3 `.pfs` | ciru-ai/ROCmFPX tag `kairic-edge-qwen38-27b-v1` @ `e97b324` | one window, IU4 W4A4 lane, ubatch 512 | prefill 1.5–2.3× faster but prose decode 15–26 % slower → back to v3 the same day | `/home/dank/models/Kairic-Qwen3.8-27B-IU4-Edge/` + `/home/dank/ciru-qwen38/ROCmFPX-kairic/build-kairic/bin/llama-server` |
+| 2026-08-21 (same day) | unsloth `UD-Q8_K_XL` 31.4 GiB + z-lab DFlash2 Q4_K_M 2.0 GiB | upstream llama.cpp PR #27342, `GGML_VULKAN=ON` | Vulkan0 — HIP has no `TOP_K`, so DFlash could not run there | slower than the CIRU runtime it replaced | both files in `/home/dank/models/`; the PR build must be rebuilt |
+
+Port note: this slot settled on **:7770** on 2026-08-22 (52625 before it); the 2026-08-17 fleet redo
+created it — earlier topologies in [[frank-redo-2026-08-17]].
+
+## Laws learned here
+
+- **KV arithmetic:** ~28 KiB/position incl. block scratch plus ~115 MiB/slot of O(1) state; reservations
+  measured 5.5 GiB @200,704, 7.2 @262,144, 11.0 @401,408, 14.6 @524,288. Quote the `reserving the KV
+  pool` line, not the preflight formula.
+- **One pool, not a partition:** the pool decides how many conversations stay resident; only
+  `HALOGEN_CTX` sets the window, and over-window is a 400 naming the limit, never a truncation. The
+  2026-10-18 doubling to 401,408 was bought as pool parity with [[frank-zealot]] so each slot can
+  hold a maximum-length request at once — it did not lengthen the window by one token.
+- **Budget:** device budget = 121.2 GiB (unrounded `MemTotal` 121.2159) − pinned weights −
+  `HOST_RESERVE_GIB`; need = KV + arena + vision, arena = 3.80 + 8.40 × (`MAX_TOK`/16384) GiB — it
+  predicts every recorded point to 0.1 GiB. `HOST_RESERVE_GIB` is a term in that inequality, **not an
+  allocation**: it cannot return RAM, only take the pool away (ceiling 16 at the 524,288 shape, ≥18
+  clamped it). Free RAM by cutting the pool or `MAX_TOK`.
+- **`HALOGEN_KV_POOL_FIT` prints its sizing block only when it clamped** — silence is not headroom and a
+  green `/health` is not the pool you asked for.
+- **RAM the pool takes is RAM the 47.68 GiB n-gram table loses** (page cache, `Pin::None`). The real
+  failure is file-cache starvation, not RAM pressure — long prompts crawl with the disk busy and the
+  process uninterruptible, and the fix is a *smaller* pool.
+- Pinned weights count as reclaimable file cache, so `free`, `MemAvailable` and Grafana overstate free
+  RAM by ~63 GiB; believe the engine's `… in all` line.
+- **Pin gate:** the engine refuses to pin below checkpoint + 16 GiB of `MemAvailable` (pinned pages
+  cannot swap, and under the floor the n-gram table pages in from disk every long prompt), so a
+  RAM-short host crash-loops the unit every ~5 s per boot — 2026-10-06: 8 boots at `MemAvailable`
+  74.77 GiB against the 78.11 GiB needed, self-resolved in ~2 min when the dying 27B engine freed RAM.
+  The serve script waits `FRANK_DRAGOON_RAM_WAIT` (600 s) and refuses loudly if the RAM never clears.
+- `HALOGEN_FLASH_PIN_TRUNK=0` is refused by the v2 loader (and cost several times decode on the w4b
+  build): disk-backing the weights is not a lever here.
+- **MTP is always on:** `HALOGEN_SPEC_ADAPT=0` is mandatory — the adaptive default disables the head for
+  whole requests and **halves decode**. Depth 2 all-round, 3 for code.
+- **Batch XOR speculation was engine-version-specific:** halogen 0.1.4 lost the drafter above one slot
+  (41 → 14 t/s); flash-server runs two slots *with* the head. On the ROCmFPX/llama.cpp line
+  `--spec-draft-n-max` defaults to **16** and must be pinned to 4 — the measured knee on the dense
+  family, with the draft head offloaded (`--spec-draft-ngl 999`) and ngram never layered under it.
+- **Prompt cache is free here and was manual on the last engine:** flash-server ships it on and its
+  auto-sizing fits this box's ledger (no pin) — a 100k follow-up re-prefills in ~2 s instead of ~88 s
+  cold (env, 2026-10-17), and the 2026-10-18 A/B measured 81.5 s cold against 0.3 s warm at 101,987
+  prompt tokens. halogen 0.1.4's auto-sizing refused here and re-prefilled every 79–81k turn in
+  85–180 s until `HALOGEN_CACHE_MB=16384` was pinned. What the pool takes is what the table loses, so
+  re-measure the warm ratio after any pool change.
+- **400-zone law:** the harness window MUST sit ~45k under the engine ceiling (`contextWindow:
+  155648` = 200,704 − 45,056) because the harness clamps `max_tokens` to `contextWindow − est − 4096`,
+  its chars/4 `est` under-counts code/agent traffic by ~3 %, and compaction only fires at 85 %. The
+  3 % is measured, not theory: 2026-10-07 at the old 200,192 ceiling, est 152,938 vs server 157,388 —
+  over by 354, a 400. At the live shape compaction fires at 132,301 est ≈ 136k measured (the 180k
+  target of the 262,144 era is unreachable — the window itself is 155,648), and the worst-case request
+  is 155,648 − 4,096 + 3 % ≈ 156.2k ≤ 200,704: ~9.5× the error of margin, robust to a ~25 %
+  under-count. Compaction rides [[frank-zealot]] at ≈ threshold − keepRecent(20k) ≈ 112k input.
+- **Admission reserves prompt + `max_tokens`:** an un-fittable request waits in arrival order, then
+  503s `engine_busy` for the whole queue after `QUEUE_TIMEOUT` — and the cap/timeout pair moves
+  together: 65536 tokens at ~10 t/s serial needs >6550 s, which is why 7200 is pinned (env, 2026-10-17).
+- **Depth is the cost on this silicon:** ~8 µs per token of context per round (KV reads ~8 GB/s against
+  180–240 GB/s of weight streaming) — 30–50 t/s shallow, 3.3–3.8 at 79–81k, so deep-conversation roles
+  belong on the fp8-KV endpoint [[frank-archon]]. BF16 dense decode tops out at 13.1 t/s (≈94 % of the
+  roof): only speculative decode moves it.
+- **The file picks the runtime, never the reverse:** v2 `.hgn` only in flash-server, the 27B `.hgn` only
+  in halogen, ROCmFPX types 101/102/103/105/108 in no stock loader; fork builds pin by SHA —
+  pre-`36e9acd` ROCmFPX loads the AGENT files then SIGSEGVs in `ggml_compute_forward_set_rows`.
+- **Same filename ≠ same model:** native v2 is 66,687,678,432 B, the Swift1.5 r2 is 66,687,678,464 B.
+  Byte count + LFS sha256 is the only discriminator, and upstream versions sidecars in place, so equal
+  sizes prove nothing.
+- **Companion law:** the n-gram table is REQUIRED beside the checkpoint; if the publisher ships only the
+  main file, hardlink the engine vendor's stock table/tower/tokenizer (a hardlink is a regular file and
+  passes the no-symlink gate) and referee with the engine's own tools — a mismatched table raises KL
+  *everywhere* and degrades retrieval, which is what NIAH + needle probes rule out.
+- **Payload gates:** flat tokenizer directory, every entry a regular file (HF cache symlinks dangle
+  inside the mount), and the serve gate must mirror the deployer exactly or deploy goes green while the
+  unit crash-loops.
+- Serve-script lore (this file has been bitten by all three): capture `podman images -q` into a
+  variable *before* grepping it — under `pipefail`, grep's early exit killed the pipeline and the
+  deploy refused with "no match for the pinned digest" on a box that had the image (2026-09-30); never
+  put a comment inside the `podman run` backslash chain — a mid-chain `#` terminates the continuation
+  and runc gets `--name run` (2026-10-05, exit-127 crash loop); pass only `--ulimit memlock`, since
+  Dankclaw's nofile hard limit is 524,288 and rootless runc cannot raise it (exit 126), and clear the
+  legacy `frank-dragoon` container name before starting.
+- **Bench discipline (this endpoint wrote the fleet law):** stop the service for any GPU probe
+  (`census-frank-dragoon.sh verify` is the only GPU-free one); `gufo bench` cannot rank speculative
+  settings and inverts them; ≥128 generated tokens or the per-token figure is first-token noise; card
+  acceptance/t-s numbers are file properties, not promises — the live card's 53–57 t/s greedy decode
+  and ~1,640 t/s prefill at 34k measured 50.0 and 1,604 here (2026-10-17), and the 27B card's 31.7
+  t/s measured 27.6–28.1; health 200 does not prove your pid owns the port; there are no scratch ports
+  on this host — A/B through `scripts/trial-frank-dragoon.sh` drop-ins; never `pkill -f llama-server`
+  (frank-zealot is one), sweep by payload name.
+- **Dead ends — do not re-try:** ngram under MTP or DFlash (slower at every depth; on recurrent
+  Gated-DeltaNet layers it also disables MTP's rollback path, −7…−15 %); DFlash across parallel slots
+  (shared draft ctx spuriously emits EOG); `-ot`/`-ncmoe` on a hybrid split (disables mmap, page-locks
+  ~108 GiB and stranded the host until a power cycle); `--no-mmap` on any PLE-table file (silent cgroup
+  OOM-kill); forcing PLE to CPU (23 → 13.4 t/s); `HALOGEN_FA_TQ=256` (no decode win, −9.5 % prefill);
+  big ubatches as a decode knob (prefill-only, ub 512 fastest); Flash-Next MTP heads on the qwen4exp
+  loader (only `blk.48`, never the full per-layer norm set); mradermacher i1 GGUFs (NextN stripped);
+  vLLM quant paths on gfx1151; a custom Jinja template on gufo (no Jinja engine — send the `terse` text
+  as a client system message instead).
+- Build lore if a fork is rebuilt: purge Ubuntu `librocwmma-dev` (shadows AMD's headers, `-I` cannot beat
+  it), pin `--gcc-install-dir` to GCC 15, `-DLLAMA_BUILD_WEBUI=OFF`; four HIP processes is the device
+  ceiling and ComfyUI is a co-tenant; `amdgpu: SVM mapping failed` spam is a symptom, not a cause.
+
+## Contract (wire)
+
+- Live: `/v1/models`, `/health`, `/metrics`, POST `/v1/chat/completions`; `404`: `/ping`, `/props`,
+  `/completion`, `/tokenize`. **No `Server-Timing` header and no `/completion` timing object** — measure
+  wall-clock plus `usage.prompt_tokens`.
+- Thinking lands in `reasoning_content` and **counts against `max_tokens`**: a budget that dies
+  mid-thought returns `finish_reason: "length"` with empty `content`, so clients read both fields; size
+  vision probes at ≥512 tokens or the model looks blind. Budget for the base reasoning long: 1,476
+  thinking tokens mean on the 2026-10-17 battery, against 784 for the retired Swift1.5 finetune (×1.9)
+  and 905 for the censored 27B.
+- `reasoning_effort` accepts only `xhigh|medium|low` (anything else is a template error — the
+  `models.yml` fold stays) and there is **no `"none"`**: disable thinking with `enable_thinking:false`,
+  which is also what the llama.cpp-era QwenSharp template wanted (`reasoning_effort:"none"` threw
+  there too). `preserve_reasoning` is a 400 on this engine; the llama.cpp-era key was
+  `chat_template_kwargs.preserve_thinking`, which decided whether `<think>` blocks stayed in recorded
+  assistant messages. Sampler defaults 1.0/0.95/20 are server-side, so an absent temperature is NOT
+  greedy (halogen 0.1.4 and w4b decoded greedily); unknown sampling fields are a 400, out-of-range
+  values are refused not clamped, and `response_format` is unsupported.
+- Over-ceiling prompt → 400 naming the exact overage, rejected by preflight in <1 s, so probing the
+  ceiling is free; over-cap `max_tokens` → 400 naming the cap; `max_tokens`, `max_completion_tokens` and
+  `max_output_tokens` all work and must agree.
+- Vision: OpenAI `image_url` parts only (top-level `images` is the frank-zealot dialect), ≥256 px or the
+  model answers "video", ~2,040 tokens per image, `http(s)` image URLs refused on purpose.
+- The first request after a boot is a warmup artifact and may sit silent 45–60 s with 503
+  `engine_unresponsive` then self-recover — do not restart over it; `/health` 503 during load is loading.
+  Flags read once at startup, all bitwise except `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`,
+  `HALOGEN_INDEXER_BUDGET`, `HALOGEN_COMPOSABLE_CONTEXT`.
+- Deploy waits for :7770 to be free: a rootless container binds it at CREATE and exits 126, and a
+  leftover Quadlet beside the root unit guarantees the race.
+
+## Files
+
+- `scripts/frank-dragoon.env` · `scripts/serve-frank-dragoon.sh` (gates, RAM pin-gate wait, foreground
+  `podman run`, ledger to `/home/dank/logs/frank_dragoon.log`) · `scripts/frank-dragoon.service`
+  (`User=dank`, `LimitMEMLOCK=infinity`, `Restart=always`, `OOMScoreAdjust=-500`, `ExecStop=podman stop
+  -t 60`) → `deploy_llm_dragoon` in `scripts/deploy-remote.sh`.
+- Fetchers: `download-frank-dragoon-v2abl.sh` (live); era rollbacks `-flashnext`, `-halogen`,
+  `-kingjones`, `-kingjones-uncensored`, `-swift`, `-unsloth`, `-q4`, `-orca`,
+  `download-frank-dragoon.sh`; runtime pins `build-rocmfpx.sh`, `build-gufo.sh`.
+- Verification: `tests/frank_dragoon_unit.bats` (payload gates, deployer/serve parity, spec block, KV
+  band), `tests/llm_units.bats`, `tests/llm_secrets.bats`, `scripts/census-frank-dragoon.sh`,
+  `scripts/battery-frank-dragoon.py`, `scripts/stage-frank-dragoon-eval.sh`,
+  `scripts/trial-frank-dragoon.sh`.
+- Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
+  [[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
+  [[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].

serverAppConfigs-docs/LLM/frank-medium-batch-ubatch.md

@@ -25,6 +25,12 @@ what each buys. The answer is that `--ubatch-size` has an **optimum** on this
 stack, that the logical batch does not matter at all, and that the optimum is
 much cheaper than what was deployed — which then moved the layer split.
 
+The deployment this sweep tuned (`frank-medium`, 2x R9700, llama.cpp layer
+split) is retired; its shape and boot ledger are
+[[frank-medium-llamacpp-dflash2]]'s, and the endpoint lineage continues on
+[[frank-archon]]. What this page owns is the sweep, the harness traps and the
+coupling law between batch and split.
+
 ## The shape of the measurement
 
 A cell is a **64k prefill measured with 64k of KV already resident**, plus a
@@ -104,7 +110,7 @@ kill — no requests):
 |---|---|---|---|---|
 | 36,29 — deployed before | 1536 | 4.71 GiB | 4.91 GiB | 0.20 GiB |
 | 36,29 | **512** | 6.01 GiB | 7.32 GiB | 1.31 GiB |
-| **35,30 — deployed now** | **512** | **6.44 GiB** | **6.90 GiB** | **0.46 GiB** |
+| **35,30 — deployed after** | **512** | **6.44 GiB** | **6.90 GiB** | **0.46 GiB** |
 
 Two things follow.
 
@@ -153,7 +159,7 @@ other (measured 2.41 GiB vs 1.30 GiB going 1536 → 512). Balance by
 measurement only: one `-ts` step is one layer, worth ~0.43 GiB on a DeltaNet
 boundary and ~1.43 GiB on an attention one, so the reachable minimum gap is
 about one DeltaNet layer, never zero. Reference: 36,29 @1536 → 4.71/4.91;
-36,29 @512 → 6.01/7.32; **35,30 @512 → 6.44/6.90 (deployed)**.
+36,29 @512 → 6.01/7.32; **35,30 @512 → 6.44/6.90 (the shape that deployed)**.
 
 Re-fit both ubatch coefficients in `tests/frank_medium_unit.bats` at the new
 deployed point and re-anchor the per-card constants; check neighbouring
@@ -211,7 +217,7 @@ re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating.
 
 - [[frank-medium-llamacpp-dflash2]] — the deployment this tuned, and its split
   history
-- [[frank-zealot-benchmark]] — the same sweep on a CUDA 3090, where a 16k prompt
+- [[frank-zealot]] — the same sweep on a CUDA 3090, where a 16k prompt
   at depth 0 made prefill climb all the way to ubatch 2048. Depth is the
   difference: with no KV to attend over, per-pass overhead dominates and a
   bigger micro-batch wins; at 64k of resident KV it does not.

serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md

@@ -724,7 +724,7 @@ fleet.
   2026-09-15, whose `--split-mode layer` measurements this service reuses
 - [[Dual-R9700-Cooperation]] — why the two cards cannot take part in a
   collective, and every software escape that is closed
-- [[frank-zealot-qwen38-gsq-rco]] — the fleet's other llama.cpp endpoint (RTX
+- [[frank-zealot]] — the fleet's other llama.cpp endpoint (RTX
   3090, MTP, the image path)
 - [[Dankserver System Specs]] — hardware inventory
 - [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed

serverAppConfigs-docs/LLM/frank-medium-model-swap.md

@@ -1,7 +1,7 @@
 ---
 type: runbook
 title: frank-medium Model Swap
-description: How to swap the checkpoint behind the frank-medium endpoint (Dankserver :7770, vllm-radiance TP=2) — the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
+description: RETIRED-era procedure (frank-medium is a retired endpoint; `:7770` on Dankserver is [[frank-archon]] today and the live swap law is [[Model Swap and Rollback]]) — how a checkpoint swap behind frank-medium worked: the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
 tags:
   - frank-medium
   - dankserver
@@ -14,12 +14,13 @@ timestamp: 2026-09-20
 
 # frank-medium Model Swap
 
-Swapping the checkpoint behind **frank-medium** (Dankserver `:7770`,
-vllm-radiance TP=2) is a four-step process, and the toolchain below drives all
-of it. The design goal (2026-09-20, after the MXFP4 → FP8 swap) is that a swap
-is **one command**: generate a manifest, download it verified, point the
-quadlet at it, deploy. The entrypoint is quantization-aware, so the only repo
-change a swap makes is the quadlet's four env vars.
+**Historical.** frank-medium is a retired endpoint — `:7770` on Dankserver is
+[[frank-archon]] today, and the live procedure is [[Model Swap and Rollback]].
+This page records how a swap worked in that era: a four-step process whose design
+goal (2026-09-20, after the MXFP4 → FP8 swap) was that a swap be **one command** —
+generate a manifest, download it verified, point the quadlet at it, deploy. The
+entrypoint was quantization-aware, so the only repo change a swap made was the
+quadlet's four env vars.
 
 ## Why this exists
 

serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md

@@ -1,7 +1,7 @@
 ---
 type: configuration
 title: frank-medium vllm-radiance MXFP6 Runbook
-description: ACTIVE frank-medium runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
+description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
 tags:
   - frank-medium
   - dankserver
@@ -19,7 +19,8 @@ timestamp: 2026-09-21
 
 # frank-medium vllm-radiance MXFP6 Runbook
 
-The ACTIVE frank-medium deployment: the `ggz14/vllm-radiance-mxfp4:0.13.0-e060895`
+**RETIRED era (2026-09-21 → 10-10; the endpoint is [[frank-archon]] today and this
+shape is one row in its Eras table.)** The deployment was the `ggz14/vllm-radiance-mxfp4:0.13.0-e060895`
 image (built from PR #51's head, tagged locally as
 `localhost/radiance-vllm:0.13.0-e060895`) plus the `radiance-vllm-mxfp4`
 patches at e060895, serving `hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6`

serverAppConfigs-docs/LLM/frank-medium-rocmfpx.md

@@ -25,7 +25,7 @@ timestamp: 2026-09-15
 when the RTX 3090 left the box and two **AMD Radeon AI PRO R9700** cards
 (Navi 48, RDNA4, `gfx1201`, 32,624 MiB each, PCIe 5.0 x16) took its place.
 It replaced the retired FreeToken/CUDA `frank-small`
-([[freetoken-frank-small]]), which cannot run on this hardware at all.
+([[frank-zealot]]), which cannot run on this hardware at all.
 
 Stack: **charlie12345/ROCmFPX llama.cpp** (`main`, `c49ebdb`, 2026-08-22)
 serving **kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF** with the
@@ -213,7 +213,7 @@ depend on any of it.
 
 ## See Also
 
-- [[freetoken-frank-small]] — the retired predecessor (FreeToken/CUDA on the 3090)
+- [[frank-zealot]] — the retired predecessor (FreeToken/CUDA on the 3090)
 - [[kingjones-q8agent-rocmfpx]] — the gfx1151 investigation of this checkpoint family
 - [[Llama Cpp Runtimes]] — where the ROCmFPX fork sits among the runtimes
 - [[Dankserver System Specs]] — current hardware

serverAppConfigs-docs/LLM/frank-npu.md

@@ -25,7 +25,7 @@ timestamp: 2026-09-11
 > file and serve script are staged in `serverAppConfigs/.AGENTTRASH/`
 > (suffixed `.flm-npu`), and the host unit is still masked
 > (`systemctl is-enabled frank-npu` → `masked`, read 2026-10-18). Port 7771
-> belongs to [[frank-zealot-llama-prism-bonsai2-pq20]] today.
+> belongs to [[frank-zealot]] today.
 
 The third inference endpoint on Dankclaw: a FastFlowLM (FLM) v1.0.1 server
 (`/usr/bin/flm`, dpkg `fastflowlm`) on the **XDNA2 NPU** (50 TOPS), serving the
@@ -161,4 +161,4 @@ retirement; the routes are mapped in the table above.
 
 - [FastFlowLM](https://github.com/FastFlowLM/FastFlowLM) — FLM runtime and NPU model packs
 - [[dankclaw-iommu-translation-off]] — the decision that retired this endpoint
-- [[halogen-flash-server]] — the surviving image-input path
+- [[frank-dragoon]] — the surviving image-input path

serverAppConfigs-docs/LLM/frank-redo-2026-08-17.md

@@ -14,14 +14,10 @@ The pipeline-parallel split and the FastFlowLM NPU endpoint were retired.
 The old scripts moved to `old/scripts/` in this repository.
 
 > **HISTORY — the topology below is two fleet generations dead.** Today the
-> fleet is three endpoints: [[frank-archon-radiance-qwen38-iq4r]]
-> (`192.168.0.235:7770`, radiance 1.0.13, 2×R9700 TP=2),
-> [[frank-dragoon-flash-server-v2-native]] (`192.168.0.143:7770`, halogen
-> flash-server 0.16.2 on the iGPU) and
-> [[frank-zealot-llama-prism-bonsai2-pq20]] (`192.168.0.143:7771`, llama.cpp
-> PrismML fork build 10754 on the RTX 3090 — the card that sat in Dankserver
-> until 2026-09-15). No vLLM and no ROCm endpoint serves anything; there is no
-> pipeline parallelism and no NPU endpoint.
+> fleet is three endpoints, each owned by its page: [[frank-archon]]
+> (`192.168.0.235:7770`), [[frank-dragoon]] (`192.168.0.143:7770`) and
+> [[frank-zealot]] (`192.168.0.143:7771`). No vLLM and no ROCm endpoint serves
+> anything; there is no pipeline parallelism and no NPU endpoint.
 
 ## New Topology
 

serverAppConfigs-docs/LLM/frank-zealot-benchmark.md

@@ -1,193 +0,0 @@
----
-timestamp: 2026-09-03
-type: reference
-title: frank-zealot Benchmark — ncmoe and Batch Tuning
-description: llama-bench sweep of MoE CPU offload (ncmoe) and batch/ubatch on the RTX 3090 for Qwen3.6-35B-A3B Q6_K_P.
-tags:
-  - benchmark
-  - llamacpp
-  - ncmoe
-  - ubatch
-  - dankserver
----
-
-# frank-zealot Benchmark — ncmoe and Batch/Ubatch Tuning
-
-> **HISTORY (runs 2026-08-13 and 2026-08-25, Dankserver's RTX 3090).** That
-> card was pulled from Dankserver on 2026-09-15 (`lspci | grep -ci nvidia` → 0
-> there on 2026-10-18 — [[Dankserver System Specs]]) and today drives
-> frank-zealot on **Dankclaw**: [[frank-zealot-llama-prism-bonsai2-pq20]]
-> (`Ternary-Bonsai-2-27B-PQ2_0.gguf`, 2 × 200,704 q8_0 KV, MTP off, no
-> `--n-cpu-moe` — the model fits the card outright). What survived the payloads:
-> decode is ubatch-invariant, CUDA prefill climbs monotonically to ubatch 2048,
-> and the logical `--batch-size` is neutral — the live env still ships
-> 4096/2048 (`scripts/frank-zealot.env:44-45`).
-
-This page records a llama-bench sweep on Dankserver (RTX 3090, 24 GiB) for
-the HauhauCS Qwen3.6-35B-A3B Q6_K_P model. The sweep decided the final
-`--n-cpu-moe` and `--ubatch-size` values for the llama.cpp rung then served on
-that card (the `frank-zealot` name dates from 2026-10-06/10-09).
-
-Run date: 2026-08-13. Build: llama.cpp `555881ebc` (10121). CUDA, 7 threads,
-`--cache-type-k q8_0 --cache-type-v f16`, `--fit-target 0` (KV auto-sized to
-fill VRAM). Prompt 16384 tokens, generation 512 tokens, 2 repetitions.
-
-## Headline Findings
-
-### Generation does not depend on batch or ubatch
-
-For every `ncmoe` value, generation throughput stayed flat across all
-`ubatch` values from 256 to 4096. The delta between the lowest and highest
-was under one token per second. Generation depends only on how many MoE
-expert layers stay on the GPU as `ncmoe` gets smaller.
-
-| `ncmoe` | Expert layers on GPU | Generation (t/s) |
-| ------- | -------------------- | ---------------- |
-| 40      | 0                    | 40.9             |
-| 38      | 2                    | 42.6             |
-| 36      | 4                    | 44.4             |
-| 35      | 5                    | 45.5             |
-| 34      | 6                    | 46.4             |
-| 32      | 8                    | 48.7             |
-| 30      | 10                   | 51.1             |
-
-Each pair of expert layers moved to the GPU adds roughly 1.1-2.0 t/s of
-generation. Moving from `ncmoe 35` to `ncmoe 32` adds about 3.2 t/s, or 7
-percent.
-
-### Prefill and graph memory scale with ubatch
-
-Prefill throughput climbs with `ubatch` because a prompt splits into
-`prompt / ubatch` micro-batches. For `ncmoe 35` at 16384 prompt tokens:
-
-| `ubatch` | Prefill (t/s) | Peak graph VRAM (MiB) |
-| -------- | ------------- | --------------------- |
-| 256      | 245           | 6838                  |
-| 512      | 416           | 6852                  |
-| 768      | 556           | 7074                  |
-| 1024     | 710           | 7330                  |
-| 1536     | 895           | 7846                  |
-| 2048     | 1188          | 8574                  |
-
-The logical `--batch-size` does not matter. At a fixed `ubatch`, `batch`
-values of 2048, 4096, and 8192 gave identical prefill speed and peak VRAM.
-
-## Peak VRAM by ncmoe and ubatch
-
-Peak VRAM (llama-bench, single context) rises both when `ncmoe` drops
-(more experts on GPU) and when `ubatch` rises (bigger compute graph):
-
-| `ncmoe` | ub 512 | ub 1024 | ub 1536 | ub 2048 |
-| ------- | ------ | ------- | ------- | ------- |
-| 38      | 4590   | 5068    | 5584    | 6086    |
-| 36      | 6098   | 6576    | 7092    | 7594    |
-| 35      | 6852   | 7330    | 7846    | 8574    |
-| 34      | 7606   | 8084    | 8600    | 9102    |
-| 32      | 9114   | 9592    | 10108   | 10610   |
-| 30      | 10622  | 11100   | 11616   | 12118   |
-
-## VRAM Budget Model
-
-The live service holds a fixed 700416-token KV pool (f16/f16 since
-2026-08-14; refitted from 800000) plus the mmproj vision tower plus CUDA
-overhead. The model below fits llama-bench peak VRAM into a 24 GiB
-budget:
-
-```
-service VRAM = llama-bench peak + KV pool (13.68 GiB f16) + overhead (2.2 GiB)
-```
-
-This model reproduces the historical OOMs. Configurations whose sum
-exceeds 24 GiB fail. For example, `ncmoe 32 - ubatch 2048` sums to about
-24.4 GiB and fails, which matches the recorded OOMs at `ncmoe 32` and
-`33`. The f16/f16 KV pool (2026-08-14) costs 20,480 bytes per cell; at
-800768 cells it is 15,640 MiB and does not fit the 24 GiB card, so the
-service runs 700416 cells (13,680 MiB).
-
-## Candidate Configurations
-
-| Configuration      | Gen (t/s) | Prefill (t/s) | Headroom (MiB) | Fit |
-| ------------------ | --------- | ------------- | -------------- | --- |
-| ncmoe 35, ub 2048  | 45.5      | 1188          | 2035           | OK  |
-| ncmoe 34, ub 2048  | 46.4      | 1221          | 1507           | OK  |
-| ncmoe 34, ub 1536  | 46.5      | 914           | 2009           | OK  |
-| ncmoe 32, ub 1024  | 48.7      | 766           | 1017           | OK  |
-| ncmoe 32, ub 768   | 48.8      | 600           | 1273           | OK  |
-| ncmoe 30, ub 1536  | 51.2      | 1009          | under          | OOM |
-
-## Chosen State
-
-> **Live drift (2026-08-22 audit):** the deployed `frank-zealot` now runs `--n-cpu-moe 38`, `--batch-size 4096`, `--ubatch-size 2048` (ncmoe raised from 36 for stability; batch lowered). The values below are the original benchmark recommendation.
-
-Prefill speed is the priority, so the state keeps the top prefill tier and
-moves one expert layer to CPU for stability:
-
-```
---n-cpu-moe 36     # 4 expert layers on GPU
---batch-size   8192
---ubatch-size  3072
-```
-
-This gives prefill about 1427 t/s at 16384 prompt tokens, generation about
-44.4 t/s, and about 2.0 GiB of headroom at the 700416-token f16/f16 pool
-(2026-08-14 refit from 800000; the dual-f16 pool costs 13.68 GiB and
-removes the context-proportional flash-attn F16-conversion workspace that
-caused the cuBLAS crash class).
-
-`ncmoe 36` came from a live crash rather than this sweep. At `ncmoe 35` with
-`ubatch 3072`, a big prefill burst failed a cuBLAS call under memory
-pressure (a `cudaErrorInvalidValue` in `ggml_cuda_mul_mat_cublas`), which
-killed the service. Raising `ncmoe` one step frees about 0.75 GiB of peak
-VRAM (the grid shows `ncmoe 36 - ubatch 3072` peak 8606 MiB versus 9360 MiB
-at `ncmoe 35`) for about 2 percent less prefill and 2 percent less decode.
-
-If a crash still occurs under load, set `GGML_CUDA_FORCE_MMQ=1` to bypass
-the cuBLAS path for quantized matmuls, or drop `ubatch`, or shrink the
-context pool.
-
-If decode speed matters more than prefill, swap to `ncmoe 32 - ubatch 1024`
-(generation about 48.7 t/s, prefill about 766 t/s, about 1.0 GiB headroom).
-
-## Ornith Q6_K — 8K `ubatch` Sweep Without Speculation
-
-Run date: 2026-08-25. This controlled sweep used the active Ornith Q6_K model
-and hybrid expert placement, but explicitly disabled speculative decoding with
-`--spec-type none`. It used one slot, an 8192-token context, a fixed logical
-`--batch-size 4096`, prompt caching disabled, and one unique approximately
-8K-token prompt per point. The production service was stopped during the
-sweep and automatically restarted afterward.
-
-| `ubatch` | Prompt tokens/s | Generation tokens/s | CUDA compute reservation |
-|---:|---:|---:|---:|
-| 1024 | 1944 | 53.91 | 248 MiB |
-| 1280 | 2030 | 53.87 | 248 MiB |
-| 1536 | 2190 | 53.66 | 248 MiB |
-| 1792 | 2203 | 53.69 | 248 MiB |
-| 2048 | 2423 | 53.80 | 248 MiB |
-
-Prefill improves through `ubatch=2048`, while generation is effectively flat.
-The 8K prompt sweep therefore favors `ubatch=2048` as the balanced point; a
-larger value was not tested in this corrected no-spec run.
-
-The same workload with `draft-mtp` enabled produced approximately 26.8–35.2
-generation tokens/s at these `ubatch` values, with variable draft acceptance.
-The no-spec path produced approximately 53.7–53.9 generation tokens/s. This
-means MTP was a net throughput loss for this workload and configuration,
-despite accepting draft tokens. The production service remains on MTP until a
-separate decision is made; acceptance count alone is not sufficient to select
-speculation.
-
-## Limitations
-
-llama-bench measures raw decode and prefill. It does not run speculative
-decoding, parallel slots, or the mmproj vision tower. The live service adds
-those on top. The VRAM model uses a fixed 2.2 GiB overhead estimate; the
-real overhead varies a little with load. The Ornith sweep above used the
-OpenAI-compatible server API rather than `llama-bench`, and its throughput
-includes the server's timing instrumentation.
-
-## See Also
-
-- [[Dankserver System Specs]]
-- [[deployment]]
-- [[DFlash DSpark Speculative Decoding]]

serverAppConfigs-docs/LLM/frank-zealot-gemma4-12b.md

@@ -1,134 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp, Gemma4-12B QAT Q4_K_M"
-description: "The RETIRED frank-zealot runbook (2026-09-17 → 10-01) — mainline llama.cpp on Dankclaw's RTX 3090 serving HauhauCS Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M with a separate MTP assistant sidecar chained behind ngram-mod, q8_0 KV, and --fit sizing the full 262k window: identity, payload, KV arithmetic, the spec-decode chain, deployed flags, the client contract, and what to check when it misbehaves."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama.cpp
-  - rtx3090
-  - gemma
-  - mtp
-  - vision
-timestamp: 2026-09-17
----
-
-# frank-zealot — llama.cpp Gemma4-12B QAT Q4_K_M (MTP + ngram-mod)
-
-> **RETIRED 2026-09-17 → 2026-10-01** (AGENTS.md Runbook Index). It replaced
-> [[frank-zealot-qwen38-gsq-rco]] (ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S) and
-> [[frank-zealot-vllm-gemma4]] (the same-day vLLM experiment).
-> Same port (7771), same alias (`frank-small` — this endpoint only carried the
-> `frank-zealot` name from 2026-10-06/10-09), same client contract surface.
-> :7771 today runs [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML
-> fork build 10754, `Ternary-Bonsai-2-27B-PQ2_0.gguf`, 2 × 200,704 q8_0 KV, MTP
-> off); the audio input this page documents left the fleet with this payload.
-
-## Identity
-
-- **Endpoint:** `frank-small` at `192.168.0.143:7771` (Dankclaw) —
-  `git show e23df8b:scripts/frank-small.env` → `ALIAS=frank-small`.
-- **Runtime:** mainline llama.cpp master `37b53fd`, CUDA build for sm_86 with
-  `-DGGML_CUDA_FA_ALL_QUANTS=ON`
-  (`/home/dank/GitPackages/llama.cpp-mainline/build-cuda/bin/llama-server`).
-  The FA_ALL_QUANTS flag is required: without it, q8_0 KV + flash attention
-  collapses prefill ~12x on CUDA. Host check 2026-10-18: that binary reports
-  `0.4.1-dev (build 1, commit 37b53fd)` and its `CMakeCache.txt` reads
-  `GGML_CUDA_FA_ALL_QUANTS:BOOL=ON`.
-- **Service shape:** root systemd unit `scripts/frank-zealot.service`
-  (`User=dank`), launched by `scripts/serve-frank-zealot.sh` with tunables from
-  `scripts/frank-zealot.env`.
-
-## Shape
-
-- **Model:** `HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced` at
-  Q4_K_M (6.87 GiB). Architecture `gemma4`: 48 layers, MHA with 16 kv heads
-  x 512 head_dim, rope freq_base 1e6, final logit softcapping 30, per-layer
-  token embeddings (PLE) inside the file, and a sliding-window attention
-  pattern (`attention.sliding_window = 1024`) — most layers cache only a
-  1024-position window, which is why the KV pool is a fraction of the
-  full-ctx x all-layers number.
-- **Context:** four FULL 262,144-token slots (`CTX_SIZE=1048576`,
-  `PARALLEL=4`). The explicit `CTX_SIZE` bypasses `--fit` on purpose: `--fit`
-  maximises the TOTAL pool, which would shrink each slot below the full window
-  once parallel > 1. The 4-slot boot idles at 22,346 MiB; each extra
-  full-context slot costs 3,450 MiB, and a fifth projects to ~25.8 GiB — past
-  the card. 4 is the measured max.
-- **Placement:** the whole model in VRAM (`--gpu-layers 999`); the host CPU
-  only tokenizes and samples (its CPU belongs to the halogen engine serving
-  frank-dragoon).
-
-## Spec method
-
-The MTP head ships as a **separate** file (`mtp-gemma-4-12B-it.gguf`, arch
-`gemma4-assistant`, 4 nextn layers, 0.24 GiB), loaded via `--model-draft`
-with `--spec-type draft-mtp`. The same `--spec-type` value chains the
-draftless **ngram-mod** matcher, which attaches unconditionally; two
-context-backed drafters cannot share one process, so `draft-mtp,ngram-mod` is
-the only legal shape that carries both. Draft depth 3 + poll 50 (measured:
-depth 4 + poll 100 decoded WORSE at long context — the longer verification
-round costs more attention re-read than the extra accepted token earns).
-
-## KV
-
-**q8_0** (`--cache-type-k q8_0 --cache-type-v q8_0`), with flash attention
-(`--flash-attn on`, required for quantized KV). CPU-side cache capped at 2048
-MiB (`CACHE_RAM=2048`) to give back ~6 GiB of LPDDR — Gemma4 runs fully on the
-3090, so the CPU cache is not the hot path, and it is not a DeltaNet model so
-the >12.8 GiB floor does not apply.
-
-## Vision + audio
-
-`--mmproj` loads the BF16 projector
-(`mmproj-Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced-BF16.gguf`, 11 tensors,
-52M params), which also carries an audio encoder. The fleet's image path
-lives here. Audio works: `input_audio` content parts transcribe verbatim
-(verified 2026-09-17 with a 16k mono WAV). Fleet-wide audio ended with this
-payload: no fleet endpoint takes audio input today.
-
-## Payload
-
-The main GGUF, the BF16 mmproj, and the MTP sidecar lived under
-`/home/dank/models/Gemma4-12B-QAT`; they were staged to
-`.AGENTTRASH/20261002-retired-llm-payloads/Gemma4-12B-QAT/` on 2026-10-02 and
-that stage dir is gone as of 2026-10-18, so the payload needs a re-fetch. The serve script refuses to start if any
-is missing (a missing file is an unfinished download, and the unit's restart
-policy would otherwise turn it into a crash loop that reads like a runtime
-fault).
-
-## Client contract
-
-- **Alias:** `frank-small` (the `id` at `/v1/models`).
-- **Reasoning:** extracted to `reasoning_content` (`--reasoning-format
-  deepseek`, unlimited budget).
-- **Tools:** supported (`--jinja` is pinned even though this build defaults
-  it on, so a future build that flips the default cannot silently take tools
-  away).
-- **Vision + audio:** both supported in this era; audio is gone fleet-wide
-  today (`input: [text, image]` — `~/.omp/agent/models.yml:214`).
-- **Sampling:** greedy by default (fleet contract); a client that sends a
-  temperature gets its own, and the harness applies per-model overrides from
-  `models.yml`.
-
-## Measured (max length)
-
-At 254k-token prefill / 253k-ctx decode, 4 slots, q8_0 KV, ubatch 1024:
-pp ~965 t/s, tg 257–302 t/s, idle 23,300 MiB of 24,576. Larger ubatches
-(2048) OOM the MTP draft context's pp compute buffer and must not be tried.
-
-## Rollback
-
-Payload and runtime for [[frank-zealot-qwen38-gsq-rco]] (ISTA-DASLab
-Qwen3.8-27B-GSQ-RCO IQ3_S) are **not on disk any more** (the 2026-10-18 census
-of `/home/dank/models/` has no GSQ-RCO directory and the era fetch script is
-not in the tree) — that rollback re-downloads from ISTA-DASLab.
-[[frank-zealot-vllm-gemma4]] is the same-day vLLM experiment; its weights are
-gone too, though its derived image is still in rootless storage
-(`localhost/vllm/vllm-openai:frank-small`, id `8dad46d79c46`).
-
-# Citations
-
-- Model + mmproj + MTP sidecar: https://huggingface.co/HauhauCS
-- llama.cpp: https://github.com/ggml-org/llama.cpp
-- Serve script: `scripts/serve-frank-zealot.sh`; tunables: `scripts/frank-zealot.env`

serverAppConfigs-docs/LLM/frank-zealot-llama-gemma4-q4km-200704.md

@@ -1,116 +0,0 @@
-# frank-zealot — llama.cpp CUDA, Gemma4-26B-A4B Q4_K_M, 1×200,704 (Dankclaw)
-
-**RETIRED 2026-10-07** — same day it went live (AGENTS.md Runbook Index).
-Superseded by
-[[frank-zealot-llama-qwen35-9b-q6k-2x200704]] (Qwen3.5-9B Q6_K at
-2 × 200,704 with exact KV); :7771 today runs
-[[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork build 10754,
-`Ternary-Bonsai-2-27B-PQ2_0.gguf`, 2 × 200,704 q8_0 KV, MTP off). The payload
-stays at `/home/dank/models/Gemma4-26B-A4B/` as the rollback (all three files
-present in the 2026-10-18 host census). This page keeps the
-Gemma4 KV-billing law (the `attention_k_eq_v` pair-billing measurement) and
-the drafter-cost boots, both of which outlive the model.
-
-## Payload
-
-|File|Path|Bytes|
-|---|---|---|
-|Main (Q4_K_M)|`/home/dank/models/Gemma4-26B-A4B/Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf`|16,796,015,520 on disk; the engine reports 16,780,192,888|
-|Vision (mmproj BF16)|`/home/dank/models/Gemma4-26B-A4B/mmproj-Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-BF16.gguf`|1,194,827,776 B on disk (2026-10-18) = 1.11 GiB|
-|MTP assistant (on disk, NOT loaded)|`/home/dank/models/Gemma4-26B-A4B/mtp-gemma-4-26B-A4B-it.gguf`|251,937,728 B on disk (2026-10-18) = 0.23 GiB|
-
-Repo: `HauhauCS/Gemma4-26B-A4B-QAT-Uncensored-HauhauCS-Balanced-MTP`
-(base `google/gemma-4-26B-A4B-it`). Downloaded 2026-10-07 by plain
-`curl -fL` (no token needed, ungated).
-
-## Shape
-
-```
---ctx-size 200704 --parallel 1        # ONE conversation, Frank's 200k cap
---flash-attn on
---cache-type-k f16 --cache-type-v q8_0   # exact K, FP8-class V (q8_0)
---batch-size 4096 --ubatch-size 1024
---n-gpu-layers 99
---log-verbosity 4                     # debug: KV layout, buffers, graph capture
---spec-type none                      # MTP OFF (see below)
---mmproj <BF16 projector>
---jinja --reasoning auto --reasoning-format deepseek
-```
-
-Measured boot at **q8_0 K + q8_0 V, ubatch 2048, MTP off** (2026-10-07
-morning): **21,890 MiB used / 2,235 MiB free** (`nvidia-smi`). With MTP ON
-(the assistant loaded) the same shape was **23,806 / 319** — the drafter's
-context costs ~1.9 GiB because its KV is built against the target's full
-window ([[Llama Cpp Serving Sizing]]).
-
-**Deployed shape (2026-10-07 evening, Frank): K = f16, ubatch 1024, verbosity
-4. Measured boot: 21,868 MiB used / 2,257 MiB free** — i.e. the f16-K raise
-(+2,161 MiB of KV, see the accounting below) was paid for by the ubatch cut
-(2048 → 1024 returns ~616 MiB of compute buffer) plus slot-geometry variance.
-Exact K came essentially free. The verbosity-4 ledger (journal, `llama_kv_cache`
-lines): model 16,002.91 MiB (CUDA0) + 577.50 MiB (CPU-mapped), **KV 3,001.25
-MiB (5 full layers) + 306.25 MiB (25 SWA rings)**, compute buffers 926.39 +
-150.63 MiB, output buffer 1.00 MiB.
-
-**Measured decode: 139.9 tok/s** (`/completion` timings, temp 0, 256 tokens,
-deployed shape) — the ~4 B-active MoE is at home on the 3090. The model
-thinks: a 256-token budget on a short prompt can be consumed entirely by
-`reasoning_content` with empty `content` and `finish_reason: length`, so
-harness requests must budget for thinking (same law as the uncensored Swift
-payload). Vision probe on this build: solid-red 512 px PNG via the OpenAI
-`image_url` part → `Red`, `prompt_tokens` 151 (the image reached the model).
-
-Decode is expected ubatch-invariant (Swift-era 3090 law; the 139.9 t/s above
-is at ubatch 1024 — one rung measured, not a ladder).
-
-## Architecture facts
-
-Gemma4 MoE hybrid: 30 layers, 128 experts (8 active), n_embd 2816, 262,144
-native window, n_vocab 262,144. Attention is hybrid per
-`gemma4.attention.sliding_window_pattern` (read from the file at load): **25
-sliding-window layers (window 1024, `n_head_kv = 8`) + 5 full-attention layers
-(indices 5, 11, 17, 23, 29, `n_head_kv = 2`)**, head_dim 256 for K and V in
-both classes. The KV pool is TWO caches: the full-context cache scales with
-the 5 layers at the full 200,704 window; the SWA cache is a fixed 1024-token
-ring per sliding layer.
-
-**The rate law (measured, exact):** bytes/token = `n_kv_heads × 256 × 6.125 B`
-per layer at f16 K + q8_0 V — `attention_k_eq_v` does **NOT** shrink the
-allocation: `Vcur = Kcur` is a compute-graph mapping, the cache reserves K and
-V slots and, with `k_eq_v`, reserves the pair **twice** (6.125 = 2 × (2 B f16 +
-1.0625 B q8_0)). Verified against the boot: 5 × 2 × 256 × 6.125 = 15,680
-B/token × 200,704 = **3,001.25 MiB** ✓ and 25 × 1024 × 8 × 256 × 6.125 =
-**306.25 MiB** ✓. At q8_0 K + q8_0 V the rate is 2.125 B/elem: the same shape
-costs 1,041 + 106 = **1,147 MiB**, so the f16-K raise is **+2,161 MiB**. The
-`--cache-type-v` flag is honoured, not inert.
-
-The llama.cpp master build at commit `5fc4f3c` supports the arch
-(`src/models/gemma4.cpp` and `LLM_ARCH_GEMMA4 = "gemma4"` — both present in
-`/home/dank/GitPackages/llama-master` as read on 2026-10-18, `src/llama-arch.cpp:59`). Note: `grep -c`
-arch names on the packed binary returns 0 for EVERY arch (including the
-`qwen3_5` it serves) — check the source tree, not the binary.
-
-## MTP law
-
-MTP for Gemma4 is a SEPARATE file (`mtp-gemma-4-26B-A4B-it.gguf`, arch
-`gemma4-assistant`), loaded via `--spec-draft-model` beside
-`--spec-type draft-mtp` — unlike Swift, whose MTP head ships inside the main
-GGUF. The serve script gained `SPEC_DRAFT_FILE` for this. 2026-10-07 (Frank):
-**MTP disabled** — `SPEC_TYPE=none` strips both spec flags (flags move as a
-set). The assistant payload stays on disk; setting `SPEC_TYPE=draft-mtp` +
-`SPEC_DRAFT_FILE` re-enables it at the cost of ~1.9 GiB (boots at 319 MiB
-free).
-
-## Verification
-
-- `curl http://192.168.0.143:7771/health` → `{"status":"ok"}`
-- `/v1/models` reports `n_ctx: 200704`, `ftype: Q4_K - Medium`
-- Vision probe: solid-color PNG ≥ 256 px via OpenAI `image_url` part,
-  expect the exact color word.
-
-## Rollback
-
-Revert to the Swift trio: `MODEL_FILE=/home/dank/models/Swift15-UncIQ3XXS/…`,
-`CTX_SIZE=229376`, `PARALLEL=2`, in-file `SPEC_TYPE=draft-mtp` — the Swift
-payload remains on disk. See
-[[frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688]].

serverAppConfigs-docs/LLM/frank-zealot-llama-mimo-q5km.md

@@ -1,182 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q5_K_M GGUF (retired)"
-description: "The retired 2026-10-02/03 frank-zealot runbook — the Q5_K_M rung at two wide unquantized-KV slots on the RTX 3090, superseded 2026-10-03 by the Q6_K file. Kept as the rollback reference: it carries the measured boot-pair table that prices every pool shape on this card, the slot-window contract, and the misbehaves tree."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama-cpp
-  - rtx3090
-  - mimo
-  - gguf
-  - kv-cache
-timestamp: 2026-10-03
----
-
-# frank-zealot — llama.cpp MiMo-V2.6 Q5_K_M (RTX 3090, retired)
-
-> **RETIRED 2026-10-02 → 2026-10-03** — superseded by
-> [[frank-zealot-llama-mimo-q6k-2x200704]]: the ~3.5 GiB this shape left free
-> bought the **Q6_K quant rung** (same runtime, same 2 × 200,704 pool, same
-> f16 KV), which measures 2,849 MiB free and 94.44 tok/s — 10 % slower
-> decode is the measured price of the higher rung on this bandwidth-bound
-> card. This file remains the rollback reference and the owner of the
-> measured boot-pair table. :7771 today runs
-> the PrismML 27B era of [[frank-zealot-llama-prism-bonsai2-pq20]] (build 10754,
-> 2 × 200,704 q8_0 KV, MTP off) — retired 2026-10-17 → 10-18 as a rollback
-> target; the active row today is build **10777** on
-> `Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf` at 3 × 262,144 q8_0 KV, MTP off by
-> measurement (AGENTS.md Runbook Index). The Q5_K_M
-> payload is still on disk (`stat -c %s` 2026-10-18 → 6876124704).
-
-> Its own history: one quant step down from [[frank-zealot-llama-mimo-q6k]]
-> (2026-10-01), resized twice on 2026-10-02 (three contexts with vision, then
-> the 2–3 GiB-band pool), and on 2026-10-03 morning reshaped to **two
-> conversations of 200,704 tokens with the KV VALUES unquantized** —
-> *"2 streams with 200700 tokens context or more… if we have extra space on
-> the 3090, unquantize the kv."* That shape measured 20,530 used / 4,046
-> apparent free (the true `nvidia-smi` free was 3,571 — see *Shape*).
-
-## Identity
-
-|Field|Value|
-|---|---|
-|Endpoint|`192.168.0.143:7771`, served alias `frank-small` in this era (`git show e5893dc:scripts/frank-small.env` → `ALIAS=frank-small`; `frank-zealot` only from 2026-10-06/10-09)|
-|Runtime|llama.cpp **master** (qwen35 model support), CUDA build `sm_86`, at `/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server` — unchanged across the quant swap|
-|Payload|`bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF` **Q5_K_M**, 6,876,124,704 B (6.40 GiB), sha256 `04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f` + the repo's `mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf`, 918,166,048 B, sha256 `ff348f3180a63188aa7285db85f550fe38acb61dd013c599eb8bad08d2cc2576` — both at `/home/dank/models/MiMo-Q5KM/`, fetched and byte- and sha256-verified by `scripts/download-frank-zealot.sh`|
-|Unit|root `frank-zealot.service` (repo `scripts/frank-zealot.service`, `User=dank`, relinked), env `scripts/frank-zealot.env`, serve `scripts/serve-frank-zealot.sh`|
-|Repo pins|`tests/frank_zealot_unit.bats` (stub-argv seam + the VRAM arithmetic), `deploy_llm_zealot` gates in `scripts/deploy-remote.sh`|
-
-## Shape and the KV contract
-
-`--ctx-size 401408 --parallel 2` — llama.cpp divides the pool across slots,
-so each conversation gets `n_ctx_slot = 200704` (boot log 2026-10-03:
-`n_slots = 2, n_ctx_slot = 200704, kv_unified = 'false'`). `CTX_SIZE` MUST
-stay divisible by `PARALLEL`; the bats seam enforces it.
-
-The KV contract is Frank's, as of 2026-10-03: **both K and V unquantized
-(f16)** — `--cache-type-k f16 --cache-type-v f16`. The asymmetric contract it
-replaced (f16 K + `q8_0` V) had existed to make three slots fit; with two
-slots the exact-value budget opens, and 4 KiB/token × 8 layers × 2 of extra
-V traffic is the measured ~2 % decode tax. The serve script still passes both
-flags explicitly — the asymmetric-era law that a quantized V requires
-`--flash-attn on` stays pinned by the seam.
-
-**Pool sizes on this stack come from boots, not formulas.** The compute
-buffers, CUDA graphs and vision tower all bill separately, so every shape
-change is priced from measured rows:
-
-|Shape|KV dtype (K+V)|Projector|`nvidia-smi` used|Non-KV (weights + buffers + projector)|
-|---|---|---|---|---|
-|2 × 262,144 (pool 524,288)|f16 + q8_0|none|20,190 MiB|**7,390 MiB**|
-|3 × 196,608 (pool 589,824)|f16 + q8_0|f16|22,972 MiB|**8,859 MiB**|
-|3 × 180,224 (pool 540,672)|f16 + q8_0|f16|21,684 MiB|~8,419 MiB|
-|**2 × 200,704 (pool 401,408)**|**f16 + f16**|f16|**20,530 MiB**|**~7,986 MiB**|
-
-The 32 KiB/token full-attention KV price (8 layers × 4 KV heads × 256 dim ×
-2 B × 2 halves) prices the deployed pool at 401,408 × 32 KiB = **12,544 MiB
-of KV**; the card is 24,576 MiB, so the shape leaves **4,046 MiB free**
-(measured steady-state — the KV pool is allocated whole at boot, so this is
-the ceiling). The headroom above the 2–3 GiB floor is deliberate slack, not
-an invitation: another 32,768 tokens of pool (1 GiB) would trade the last
-safety margin for window the harness compacts before reaching anyway.
-
-**Why 2 × 262,144 with f16 V does not fit either.** The native window pair is
-524,288 tokens = 16,384 MiB of KV plus ~7,986 non-KV = 24.4 GiB: under the
-floor by a hair. 200,704 is the largest 4,096-aligned slice that keeps the
-measured band.
-
-## Measured on this shape (2026-10-03, 2 × 200,704 + vision, f16 KV)
-
-|Check|Result|
-|---|---|
-|Boot to healthy|~5 s, `n_slots = 2`, `n_ctx_slot = 200704`, `loaded multimodal model` logged|
-|VRAM|**20,530 MiB used / 4,046 MiB free** of 24,576 (`nvidia-smi`, steady state)|
-|Decode, temp 0, no thinking|**104.84 tok/s** (128-token completion at ~500-token depth; the 3 × 180,224 q8_0-V shape measured 106.8 — ~2 % is the honest price of exact V)|
-|Prefill|1,650 t/s on the same shallow 506-token probe (rates are depth- and prompt-dependent; not comparable across shapes)|
-|Batch geometry|`--batch-size 2048 --ubatch-size 1024` — ubatch 2048 failed the scratch boot; 1024 boots clean|
-|Vision|projector loads at boot (`loaded multimodal model`); red-PNG probe passed on 2026-10-02. Caveat discovered on 2026-10-03: probes must use a >= 256 px image — a 10x10 px PNG undersupplies image tokens and the model hallucinates ("it's a video")|
-|Tools / thinking|unchanged by the reslice: native `tool_calls`, thinking in `reasoning_content`|
-
-## What the slot geometry costs
-
-The per-slot window is the whole price, in both directions:
-
-- **A prompt over 200,704 tokens is a hard 400**, so `~/.omp/agent/models.yml`
-  sets `contextWindow: 200704` for this endpoint — the harness compacts before
-  it reaches the ceiling instead of discovering it.
-- **frank-dragoon's tallest pre-compaction prompt now fits here.** Compaction
-  for frank-dragoon sessions runs on this endpoint (`compactionModel` in
-  `models.yml`, still true 2026-10-18 at `~/.omp/agent/models.yml:107`); frank-dragoon compacts at 70 % of 262,144 = 183,501 tokens,
-  which did **not** fit the old 180,224 slot and fits 200,704 with 17,163
-  tokens to spare (2026-10-18: frank-dragoon's `contextWindow` is 155,648 —
-  `~/.omp/agent/models.yml:131` — so its compaction input is smaller still).
-  The 2026-10-02 window trim had restored the fit for
-  frank-archon's smaller 140,492-token trigger; the 2026-10-03 reslice widens
-  the fit to frank-dragoon itself. Nothing routes compaction automatically —
-  the point is the ceiling no longer forbids it.
-- Two simultaneous generations are the slot ceiling; a third request queues
-  behind a released slot (the mechanism measured on the 3-slot shape).
-
-## Client contract
-
-- Thinking arrives in **`reasoning_content`** (`--reasoning-format deepseek`),
-  the field `models.yml` reads for this endpoint.
-- **Thinking is off unless the request asks**: the MiMo GGUF template defaults
-  `enable_thinking` false, so `models.yml` ships
-  `chat_template_kwargs: {enable_thinking: true, preserve_thinking: true}` in
-  `extraBody`. A raw client that omits it gets a direct answer — measured, and
-  not a fault. Trivial questions produce no thinking even with the flag on.
-- Tools: `--jinja` + server-side parsing, verified 2026-10-02.
-- **Vision is on**, through the projector file (`--mmproj`). Use the
-  OpenAI-standard `image_url` content part — that path answers correctly
-  ("Red." on a solid-red PNG). The legacy top-level `images` field is **not**
-  read by `/v1/chat/completions`: the same request sent that way answered
-  "Brown.", i.e. it generated from text alone. `/completion` takes `images`
-  but refuses a request without `messages` in this build, so chat is the only
-  vision path.
-- Setting `MMPROJ_FILE=` (explicitly empty) in `scripts/frank-zealot.env` drops
-  `--mmproj` and returns ~1.4 GiB of the card to KV — the documented text-only
-  opt-out. `MMPROJ_FILE` pointing at a file that does not exist is a boot
-  refusal, never a silent fallback to blindness.
-
-## No spec decode
-
-The GGUF has no MTP tensors (the same fact that made SGLang's NEXTN a 0.00
-ghost), and the cross-model DFlash draft
-(`/home/dank/models/DFlash-Qwen35/qwen3.5-9b-dflash-Q8_0.gguf` — no longer on
-disk: the 2026-10-18 census of `/home/dank/models/` has no `DFlash-Qwen35/`) is an
-optimization candidate, not a default — acceptance has to be measured on this
-traffic first. The bats seam fails if a draft flag ships unannounced.
-
-## Rollback
-
-|Target|How|
-|---|---|
-|Previous shape: 3 × 180,224 + q8_0 V (2026-10-02)|`scripts/frank-zealot.env` `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (measured 21,684 MiB used, 2,441 free) and `contextWindow: 180224` in `models.yml` — choose it when three simultaneous conversations matter more than per-slot window or exact V|
-|Q6_K, same runtime|`scripts/frank-zealot.env` `MODEL_FILE=/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` + `MMPROJ_FILE` in that dir — the deployed rung **2026-10-03 → 10-05** ([[frank-zealot-llama-mimo-q6k-2x200704]]): at 2 × 200,704 it fits with 2,849 MiB free, no pool cut needed (the older "+1.4 GiB, drop 49,152 tokens" estimate held only for the 3-slot shapes)|
-|Two 262,144 windows|`CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=q8_0` (measured 20,190 MiB used, 4,386 free **with q8_0 V** — with f16 V the same shape is 16,384 MiB of KV + ~7,986 non-KV and does not fit) and `contextWindow: 262144` in `models.yml` — choose it when a single conversation needs the native window|
-|Vision off|`MMPROJ_FILE=` in the env file; the deployer gate then has to be satisfied too, so point `LLM_ZEALOT_MMPROJ` at any real projector file or drop the gate in the same commit|
-|SGLang + INT8|[[frank-zealot-sglang-mimo-v26]] — venv and old unit/env/serve trio still in place; the INT8 payload was staged to `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` on 2026-10-02 and **that stage dir no longer exists** (`ls /home/dank/.AGENTTRASH/`, 2026-10-18), so re-fetch with `old/download-frank-zealot-sglang-int8.sh`|
-
-## When it misbehaves
-
-- Port 7771 dark: `journalctl -u frank-zealot -n 50`; the serve script exits 1
-  naming the missing file if the binary or GGUF moved.
-- `No CUDA GPUs are available`: the 3090 vanished from the bus or the UVM node
-  is missing — `nvidia-modprobe -u -c=0` (the serve script runs it).
-- Requests queue behind each other: two concurrent long generations are the
-  slot ceiling by contract (2026-10-03 shape); a third waits for a slot to
-  free (the mechanism measured on the 3-slot shape: 1.7 s for three, 3.0 s for
-  the fourth).
-- `400 … exceeds the available context size (200704 tokens)`: the conversation
-  outgrew ONE slot. This is not pool exhaustion — the pool is 401,408 tokens
-  and the per-slot window is the binding limit. Shorten the prompt, compact,
-  or roll back to the 262,144 shape above.
-- Vision answers a color it never saw, or claims the image "is a video":
-  either the client sent `images` instead of an `image_url` content part, or
-  the probe image is too small to supply real image tokens (use >= 256 px;
-  see *Client contract*).
-- Slow decode after a CTX or quant change: check `nvidia-smi` — pushing KV out
-  of VRAM into unified memory halves the rate silently.

serverAppConfigs-docs/LLM/frank-zealot-llama-mimo-q6k-2x200704.md

@@ -1,163 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp, MiMo-V2.6 Q6_K, 2 x 200704, f16 KV (RETIRED)"
-description: "The RETIRED frank-zealot runbook (2026-10-03 → 10-05 evening, rollback target) — llama.cpp master (CUDA sm_86) serving bartowski's MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF on the RTX 3090: two conversations of 200,704 tokens with fully unquantized f16 KV, vision through the bundled projector, and the boot-pair arithmetic that prices every pool change. Superseded by the Swift-1.5-27B IQ3_S-mtp swap."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama-cpp
-  - gguf
-  - rtx3090
-timestamp: 2026-10-03
----
-
-# frank-zealot — llama.cpp MiMo Q6_K, 2 × 200,704, f16 KV
-
-> **RETIRED 2026-10-05 (evening).** Superseded by
-> [[frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688]] — the Swift-1.5-27B
-> RCO-IQ3_XXS-mtp swap (uncensored, MTP drafted). :7771 today runs
-> [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork build 10754,
-> `Ternary-Bonsai-2-27B-PQ2_0.gguf`, 2 × 200,704 q8_0 KV, MTP off). Two facts for the trail:
-> committed earlier the same day (77decb5..380aafa) was **never deployed** —
-> THIS MiMo shape was the live payload when Swift cut over; (2) the KV
-> contract Frank set here (f16 K + f16 V, "no further quantization on this
-> endpoint") was superseded the same evening by the Swift swap's explicit
-> q8_0/q8_0 order. This page stays as the rollback and the measured
-> boot-pair record. Payload on disk (`stat -c %s`, 2026-10-18: 7793710624):
-> `/home/dank/models/MiMo-Q6K/`.
-
-## Identity
-
-| | |
-|---|---|
-| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
-| Endpoint | `http://192.168.0.143:7771` — alias `frank-small` on the wire in this era (`git show e5893dc:scripts/frank-small.env` → `ALIAS=frank-small`; the id became `frank-secondary` 2026-10-06 and `frank-zealot` 2026-10-09), `/v1` OpenAI surface |
-| Runtime | llama.cpp master, CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (f16 KV needs no flash-attn option; the disk build's `CMakeCache.txt` reads `GGML_CUDA_FA_ALL_QUANTS:BOOL=OFF` and that tree has no CUDA-side guard for the option — `ggml/src/ggml-cuda/fattn.cu:438-449`) |
-| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
-| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
-| Weights | `/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` — bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K**, 7,793,710,624 B, sha256 `ef96d05a…` (qwen3_5 hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, native 262,144, no MTP tensors) |
-| Projector | `…/MiMo-Q6K/mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf` — 918,166,048 B, sha256 `ff348f…` (hardlink of the Q5KM dir's copy, link-count-2 verified; the dir is self-contained) |
-| Fetch | [`scripts/download-frank-zealot.sh`](../../scripts/download-frank-zealot.sh) — `-f`, byte count + LFS sha256 verified, `--verify-only`; both files verified against the pinned sha on the host before this deploy |
-| Seam | `tests/frank_zealot_unit.bats` (argv assertions against a stub engine: payload path, slot geometry, KV pair, batch pair, projector opt-out) |
-
-## Shape and KV contract
-
-`--ctx-size 401408 --parallel 2` — llama.cpp divides the pool across slots,
-so each conversation gets `n_ctx_slot = 200704`. `CTX_SIZE` MUST stay
-divisible by `PARALLEL`; the seam enforces it. Both K and V are unquantized
-(f16) — the flags stay explicit because `--flash-attn` must not be lost if V
-ever goes back to `q8_0` (the serve script pins it, the seam pins the pin).
-
-**The Q6_K rung is a weights-side spend only.** File delta +917,585,920 B
-(+875 MiB) against Q5_K_M, same KV, same slot geometry, same batch pair.
-Priced from the measured Q5 boot at this shape (non-KV 7,986 MiB): the boot
-landed **21,276 MiB used / 2,849 MiB true free** — non-KV measured 8,732 MiB,
-inside the 2–3 GiB floor, so the pool stayed at 401,408 and the
-12,288-token step-down ladder was never climbed. Quote `nvidia-smi`'s `free`
-field, never `total − used`: the driver holds ~450 MiB of reserve the
-subtraction hides (that error once overstated this card's slack by a
-half-GiB).
-
-**Quant ceiling, evaluated and closed 2026-10-03 (Frank: stay here).**
-`Q8_0` is the only rung above this one on the Hub (9,545,979,424 B = 9,104
-MiB, +1,671 over Q6_K): at the live pool it would leave 1,178 MiB free —
-under the floor — and the calibrated bandwidth law (it predicted this file's
-94.4 t/s from Q5_K_M's 104.84 within 1 %) puts decode at ~72 t/s, a further
-−24 % for the near-lossless rung. Fitting it was priced two ways (trim the
-pool to 2 × 182,272 — which also breaks frank-dragoon's 183,501-token
-compaction-prompt fit by 1,236 tokens — or revert V to `q8_0`), and both
-were declined. Re-open only on evidence the Q6_K residual matters.
-
-**Pool prices on this card come from boots, never formulas** — the measured
-boot-pair table lives in [[frank-zealot-llama-mimo-q5km]] (*Shape and KV
-contract*) and still prices every rollback shape.
-
-## Compute-buffer geometry (2026-10-05)
-
-`BATCH_SIZE=4096 UBATCH_SIZE=2048` — both doubled from 2048/1024, and the trade
-measured rather than assumed, because the predecessor shape recorded **"ubatch
-2048 failed the scratch boot"** and this card has less free space than it did.
-
-**That failure was a different shape, and it did not reproduce here.** The
-service itself is the bench (one card holds one server, so no scratch port):
-the old rung was re-booted through a transient `FRANK_ZEALOT_ENV_FILE` drop-in
-and reproduced the 2026-10-03 boot **byte-identically** — 21,276 MiB used /
-2,849 MiB free — which makes the ubatch the only variable in the comparison.
-
-|Probe|ubatch 1024|ubatch 2048|Δ|
-|---|---|---|---|
-|Prefill, 8,928 tokens|3,972 / 4,077 t/s|**4,207 / 4,212 t/s**|**+4.6 %**|
-|Decode, 128 tokens at that depth|91.87 / 93.01 t/s|93.81 / 94.27 t/s|noise|
-|Prefill, 279 tokens|2,675.7 t/s|2,658–2,811 t/s|noise|
-|VRAM used / free|21,276 / 2,849 MiB|**21,846 / 2,279 MiB**|**−616 MiB**|
-
-Three readings:
-
-- **The gain only exists above the old ubatch.** A 279-token prompt is shorter
-  than either micro-batch, so it cannot see the knob — and it did not move.
-  That is the control proving the 8.9k delta is geometry, not drift.
-- **Decode is flat**, as every prior sweep on this card found. Ubatch is a
-  prefill knob.
-- **The price is 616 MiB out of a 2.8 GiB margin.** 2,279 MiB free is still
-  inside the 2–3 GiB band, at its bottom edge. `4096/1536` is the middle rung,
-  and the pool's 12,288-token (384 MiB) ladder in
-  [[frank-zealot-llama-mimo-q5km]] is the other way to buy room back.
-
-**Why 2048 is right HERE while 512 was right on the R9700 sibling: the curve is
-backend-specific.** The 2× R9700 layer split peaks at ubatch 512 and is 10 %
-off that peak by 2048 ([[frank-medium-batch-ubatch]]); on this CUDA card the
-curve climbs monotonically to 2048 (710 → 1,188 t/s on the 35B MoE sweep,
-1,944 → 2,423 t/s on the Ornith 8k sweep, both with decode flat). Never carry a
-ubatch verdict across backends.
-
-The logical `--batch-size` is VRAM- and throughput-neutral — two sweeps and this
-A/B agree — so 4096 costs nothing and buys nothing by itself; it is here because
-that is the pair the curve was measured at.
-
-## Measured on this shape (2026-10-03, Q6_K, 2 × 200,704, f16 KV)
-
-|Check|Result|
-|---|---|
-|Boot to healthy|model + projector resident in ~2.3 s (load log `0.02.27`), `n_slots = 2`, `n_ctx_slot = 200704`, `loaded multimodal model`, health 200; deploy e2e gate 33/33|
-|VRAM|**21,276 MiB used / 2,849 MiB free** of 24,576 (`nvidia-smi` `free`, steady state) — inside the 2–3 GiB floor with exact V AND the higher rung. Superseded 2026-10-05 by the batch doubling: **21,846 / 2,279** — see *Compute-buffer geometry*|
-|Decode, temp 0, no thinking|**94.44 tok/s** on the identical 506-token probe that measured 104.84 on Q5_K_M — **−10 %**. Decode on this card is weight-bandwidth-bound, and Q6_K streams ~10 % more bytes per token; the quant rung buys quality with decode rate, measurably|
-|Prefill|2,229.9 t/s (same shallow probe; rates are depth-dependent, not comparable across probes)|
-|Vision|512×512 solid-red PNG → **`Red`** (277 prompt tokens — the image really reached the tower). See the probe-size trap in *Client contract*|
-|Tools / thinking|native `tool_calls`, thinking in `reasoning_content`, thinking OFF unless `enable_thinking: true` (MiMo template default; `models.yml` ships it in `extraBody`)|
-
-## Client contract
-
-llama.cpp surface, live-verified on this model family: model id `frank-zealot`,
-native tools via `--jinja` (`tool_calls` in the response), thinking in
-**`reasoning_content`**, vision via `image_url` content parts. Two vision
-traps, both measured here:
-
-1. A top-level `images` field on `/v1/chat/completions` is silently ignored
-   and the model invents a color.
-2. **The probe image must be big enough to supply real image tokens.** A
-   10×10 px solid-red PNG came back "The image is a video, so I can't see its
-   content" — the engine itself warns at boot that Qwen-VL wants ≥ 1024 image
-   tokens, and a postage-stamp image lands one patch below usable signal.
-   512×512 answers `Red` in 19 tokens. Always probe vision with a ≥ 256 px
-   image; a wrong-color answer from a tiny image is the probe's fault, not
-   the endpoint's.
-
-`~/.omp/agent/models.yml` sets `contextWindow: 200704` — the PER-SLOT window,
-so the harness compacts before llama.cpp answers a longer prompt with a 400.
-
-## Rollback
-
-|Target|How|
-|---|---|
-|Q5_K_M at the same shape|`scripts/frank-zealot.env` `MODEL_FILE=/home/dank/models/MiMo-Q5KM/MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf` (+ `MMPROJ_FILE` in that dir) and the deployer default — buys back 875 MiB (3,571 free measured) **and the 10 % decode rate** (104.84 tok/s measured). Payload sha-verified on disk|
-|3 × 180,224 + q8_0 V|`CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (measured 21,684 used / 2,441 free **with Q5_K_M** — Q6_K does NOT fit that shape inside the floor; pair them explicitly)|
-|SGLang + INT8|[[frank-zealot-sglang-mimo-v26]] — the 2026-10-02 staging dir `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/` is **gone** (`ls /home/dank/.AGENTTRASH/` on 2026-10-18 lists only the 20261005/06/07/10/17 batches), so this arm needs `old/download-frank-zealot-sglang-int8.sh`; the `sglang-cu128` venv is still on disk|
-
-## When it misbehaves
-
-Same decision tree as [[frank-zealot-llama-mimo-q5km]] (*When it misbehaves*):
-`No CUDA GPUs are available` ⇒ `nvidia-modprobe -u -c=0`; a 400 naming
-200,704 ⇒ the conversation outgrew one slot, not the pool; slow decode after
-a shape change ⇒ check `nvidia-smi` for KV pushed into unified memory; queue
-behind two long generations ⇒ the slot ceiling is two, by contract.

serverAppConfigs-docs/LLM/frank-zealot-llama-mimo-q6k.md

@@ -1,117 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp, MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF (retired)"
-description: "The retired 2026-10-01 frank-zealot runbook — the first llama.cpp + GGUF shape on the RTX 3090: two native-window sessions, the asymmetric KV contract (f16 K + q8_0 V) that motivated the move off SGLang, the VRAM fit at Q6_K, and what to check when it misbehaves. Superseded the same night by the Q5_K_M payload."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama-cpp
-  - rtx3090
-  - mimo
-  - gguf
-  - kv-cache
-timestamp: 2026-10-01
----
-
-# frank-zealot — llama.cpp MiMo-V2.6 Q6_K (RTX 3090)
-
-> **RETIRED 2026-10-01 → 10-02** (one night): superseded by
-> [[frank-zealot-llama-mimo-q5km]], the same runtime and shape one quant rung
-> down. The Q6_K file stays at `/home/dank/models/MiMo-Q6K/` as the rollback
-> target (on disk 2026-10-18, `stat -c %s` → 7793710624). :7771 today runs
-> [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork build 10754,
-> `Ternary-Bonsai-2-27B-PQ2_0.gguf`, 2 × 200,704, q8_0 K + q8_0 V, MTP off).
->
-> Original order (Frank, 2026-10-01 night): move this endpoint to the GGUF
-> path for **asymmetric KV** — unquantized K + q8_0 V — which only llama.cpp
-> exposes per-tensor. The per-tensor lever stayed; this pairing did not — the
-> live endpoint bills q8_0 K + q8_0 V (`scripts/frank-zealot.env:55-56`).
-
-## Identity
-
-|Field|Value|
-|---|---|
-|Endpoint|`192.168.0.143:7771`, served alias `frank-small` in this era (`ALIAS` reached `frank-secondary` 2026-10-06 and `frank-zealot` 2026-10-09, commit `7c77326`)|
-|Runtime|llama.cpp **master** (qwen35 model support), CUDA build `sm_86`, at `/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`|
-|Payload|`bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF` **Q6_K**, 7,793,710,624 B, sha256 `ef96d05a2ddf2cbb450d1af1ac3860ec769d3575bafa70692ee5609bda3fad7d`, at `/home/dank/models/MiMo-Q6K/`|
-|Unit|root `frank-zealot.service` (repo `scripts/frank-zealot.service`, `User=dank`, relinked), env `scripts/frank-zealot.env`, serve `scripts/serve-frank-zealot.sh`|
-|Repo pins|`tests/frank_zealot_unit.bats` (stub-argv seam), `deploy_llm_zealot` gates in `scripts/deploy-remote.sh`|
-
-## Shape and the KV contract
-
-`--ctx-size 524288 --parallel 2` — llama.cpp divides the pool across slots,
-so each session gets `n_ctx_slot = 262144`, the model's full native window
-(boot log confirms `n_slots = 2, n_ctx_slot = 262144, kv_unified = 'false'`).
-
-Frank's KV contract: **K unquantized (f16), V `q8_0`** —
-`--cache-type-k f16 --cache-type-v q8_0`. Two hard dependencies ride on it:
-
-- **`--flash-attn on` is mandatory**: quantized V exists only on the FA
-  path. Without the flag the engine falls back silently and the contract
-  becomes a lie. The serve script pins the trio; the bats seam pins the
-  script.
-- The VRAM budget: attention KV lives in the 8 full-attention layers only
-  (4 KV heads x 256 dim). K f16 ≈ 16 KiB/token, V q8_0 ≈ 8.5 KiB/token
-  (34 B per 32-element block) ⇒ ~24.5 KiB/token over the 524,288-token pool
-  ≈ **12.6 GiB**. Weights 7.8 + KV 12.6 + compute buffers/graphs ≈
-  **20.9 GiB measured** against 24.5 GiB — the "room for checkpoints" the
-  order asked for. If graph capture OOMs, halve `CTX_SIZE`; never quantize
-  K to fix it.
-
-## Client contract
-
-- Thinking arrives in **`reasoning_content`** (`--reasoning-format
-  deepseek`) — the same field SGLang's qwen3 parser used, so `models.yml`
-  `reasoningContentField` did NOT move. Live-verified 2026-10-01 22:57.
-- **Thinking is off unless the request asks**: the MiMo GGUF template
-  defaults `enable_thinking` false; `models.yml` now ships
-  `chat_template_kwargs: {enable_thinking: true, preserve_thinking: true}`
-  in `extraBody`. A raw client that omits it gets a no-thinking answer
-  (measured — this is not a fault).
-- Tools: `--jinja` + server-side tool parsing, verified
-  `tool_calls` round-trip 2026-10-01 23:0x.
-- **Text-only in this era.** The GGUF carries no vision projector; the
-  in-checkpoint tower existed only in the SGLang safetensors, so `models.yml`
-  carried `input: [text]`. Vision returned with the next payloads: the endpoint
-  is `input: [text, image]` today (`~/.omp/agent/models.yml:214`).
-
-## No spec decode (yet)
-
-The GGUF has no MTP tensors (same fact that killed SGLang NEXTN), and the
-cross-model DFlash draft (`/home/dank/models/DFlash-Qwen35/qwen3.5-9b-dflash-Q8_0.gguf`,
-Q8_0 — that path is no longer on disk; the 2026-10-18 census of
-`/home/dank/models/` has no `DFlash-Qwen35/`) is an optimization-loop candidate, not a default — `--spec-type
-draft-dflash` needs acceptance measured on this traffic first. The bats
-seam fails if a draft flag ships unannounced.
-
-## Build
-
-No ccache, CUDA 12.4 system nvcc with `--gcc-install-dir`-free gcc-13 host
-compiler (nvcc 12.4 rejects gcc ≥ 14):
-
-```
-cmake -B build-cuda -S . -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \
-  -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13 \
-  -DLLAMA_CURL=OFF -DGGML_CCACHE=OFF
-cmake --build build-cuda -j 16 --target llama-server
-```
-
-The SGLang venv (`/home/dank/GitPackages/sglang-cu128`) is still on disk; the
-INT8 AutoRound payload (`/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound`)
-is not — it was staged to `/home/dank/.AGENTTRASH/20261002-retired-llm-payloads/`
-on 2026-10-02 and that stage dir is gone as of 2026-10-18, so this rollback
-re-fetches with `old/download-frank-zealot-sglang-int8.sh`: repoint the unit's serve script and env from
-git history (`git show frank-dragoon-gufo-q4kxl-20261001` era is BEFORE this;
-use the parent of the 2026-10-01 night swap commit).
-
-## When it misbehaves
-
-- Port 7771 dark: `journalctl -u frank-zealot -n 50`; the serve script
-  exits 1 naming the missing file if the binary or GGUF moved.
-- `No CUDA GPUs are available`: the 3090 vanished from the bus or the UVM
-  node is missing — `nvidia-modprobe -u -c=0` (the serve script runs it).
-- Requests hang at first token while `/health` is 200: both slots busy —
-  two concurrent long generations are the ceiling by contract.
-- Slow decode after a CTX change: check `nvidia-smi` — swapping KV out of
-  VRAM into unified memory halves the rate silently.

serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai2-pq20.md

@@ -1,226 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10754), Bonsai-2-27B PQ2_0, 2 x 200704, q8_0 KV (RETIRED, rollback target)"
-description: "The frank-zealot runbook of 2026-10-17 → 2026-10-18 (RETIRED — rollback target) — the PrismML fork build 10754 of llama.cpp serving prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 on the RTX 3090: two conversations of 200,704 tokens with q8_0 K + q8_0 V (f16 KV does not fit this card at this window), vision through the publisher's Q8_0 Qwen3-VL projector, MTP impossible (no tensors in the file), and the measured four-arm VRAM census that priced the shape. Superseded by the 9B MTP export at 3 × 262,144."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama-cpp
-  - gguf
-  - rtx3090
-timestamp: 2026-10-17
----
-
-# frank-zealot — llama.cpp (PrismML) Bonsai-2-27B PQ2_0, 2 × 200,704, q8_0 KV
-
-**RETIRED 2026-10-17 → 2026-10-18** (AGENTS.md Runbook Index), superseded by
-[[frank-zealot-llama-prism-bonsai9-mtp]] — prism-ml/Ternary-Bonsai-2-**9B**
-PQ2_0 **with the MTP head**, THREE conversations at the native 262,144 window
-on fork build 10777. This era stays the **rollback target**: the payload
-(`/home/dank/models/Bonsai2-PQ2_0/`) and the 10754 fork tree
-(`~/GitPackages/llama-prism/`) are both still on disk. It owns the laws below —
-the 34,816 B/token KV figure over 16 full-attention layers, the 25,088 MiB f16
-OOM wall, and the two-boot VRAM pricing that every later shape on this card
-reused.
-
-## 2026-10-18 bench window — production unit stopped, candidate arm on 7771
-
-As of 2026-10-18 frank-zealot sits in a deliberate, dated **bench window**
-(fleet Benchmarking Policy: stop the production unit, bench candidate boots
-against the service itself). The journal shows the stop at 03:38:31 and the
-known llama.cpp shutdown shape — SIGTERM ignored past `TimeoutStopSec`,
-`State 'stop-sigterm' timed out. Killing.` at 03:40:01 — after which the unit
-reports `failed` (`is-enabled` → enabled); self-healing is merely parked
-while the window runs. What answers 7771 is a **hand-booted candidate arm**
-(`llama-prism-b10777/llama-server` on the 9B MTP file
-`Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf`, alias
-`frank-zealot`), not this runbook's payload: candidate-arm flags are not a
-state — the pid and shape moved between two censuses ten minutes apart (one
-argv read `--ctx-size 786432 --parallel 3 --spec-type draft-mtp`; a later
-`/props` read `n_ctx 262144`, `speculative.types=none`); quote neither, and
-expect `/v1/models` to report the candidate until the window closes. The
-window's remedy is the redeploy: `./scripts/deploy.sh dankclaw --service
-frank-zealot`. The production trio below is unchanged — the last PRODUCTION
-boot under the unit (03:14–03:38 the same morning) printed the contract
-exactly: `n_ctx = 401408`, `n_ctx_seq = 200704`, `CUDA0 KV buffer size =
-13328.00 MiB`, 2 slots (journal).
-
-**Window CLOSED 2026-10-18:** that candidate became production. The redeploy
-shipped the 9B MTP export at `3 × 262,144` on fork build 10777 — see
-[[frank-zealot-llama-prism-bonsai9-mtp]]; this payload is the rollback target.
-
-## Identity
-
-| | |
-|---|---|
-| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB (24,123 MiB visible to the engine) |
-| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS`), `/v1` OpenAI surface |
-| Runtime | **PrismML fork build** `/home/dank/GitPackages/llama-prism/llama-server` — vendor release, `--version` = `0.2.0-dev (build 10754, commit 2459f68b5)`, sha256 `12ae5baba033d8cd…`. NOT a preference: see *Why the fork* |
-| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
-| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
-| Weights | `/home/dank/models/Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-PQ2_0.gguf` — 7,206,168,928 B, sha256 `3907dc1658db1f78a9826bf8d5bcb8dc65db0d466388937af57f2294fae62ec1` |
-| Projector | `Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf` — 629,246,976 B (full sha256 pinned in the fetch script), Qwen3-VL tower; **873.10 MiB worst-case VRAM, loaded AFTER the engine's fit check** |
-| Fetch | [`scripts/download-frank-zealot-bonsai2.sh`](../../scripts/download-frank-zealot-bonsai2.sh) — `-f`, byte-count + sha256 verified, idempotent, `--verify-only`, no token on the command line |
-| Seam | `tests/frank_zealot_unit.bats` (argv assertions against a stub engine) |
-
-## Why the fork — the format is the lock
-
-Read from the file's own metadata, not the model card:
-
-|Key|Value|
-|---|---|
-|`general.architecture`|`qwen35` (hybrid: Gated DeltaNet + softmax attention)|
-|`qwen35.block_count`|64, `full_attention_interval` 4 ⇒ **16 full-attention + 48 DeltaNet layers**|
-|Attention|24 heads / **4 KV heads** × key 256 / value 256; `rope.dimension_sections` [11,11,10,0], `freq_base` 1e7, partial rope `dimension_count` 64|
-|Widths|embedding 5,120, feed-forward 17,408|
-|`qwen35.ssm.*`|conv_kernel 4, state_size 128, group_count 16, **time_step_rank 48**, **inner_size 6,144**|
-|Window / vocab|`context_length` 262,144; vocab 248,320 (gpt2 BPE, pre `qwen35`), eos 248046|
-|`general.file_type`|**141 = PQ2_0 — 2.13 bpw (group 128)**|
-|Tensor histogram|851 tensors: **`pq2_0` ×402** (custom GGML type **142**), f32 ×353, bf16 ×96|
-|`prism.hadamard.*`|transform `normalized-sylvester-walsh-hadamard`, block_size 1024, axis `input-last-dimension`, sign_mode explicit, **401 named weights**, sign widths [5120, 6144, 17408], 28,672 sign values, inverse applied on `token_embd.weight`, `gdn_v_grouped` true|
-|Sampler metadata|`general.sampling.temp` 1.0 / `top_p` 0.95 / `top_k` 20|
-|Chat template|8,952 chars; reads `enable_thinking`, **`reasoning_effort` ∈ {`xhigh` (default), `medium`, `low` — anything else is `raise_exception`**, `preserve_thinking`, `tools`/`tool_call`, image + video content parts|
-
-Stock llama.cpp cannot load this file: type 142 is not in its enum (`gguf-py`'s
-stock `GGMLQuantizationType` raises `ValueError: 142 is not a valid…` on the
-same read), and even a loader that parsed it would ignore the
-`prism.hadamard.*` dequant transform the weights depend on. Same class of lock
-as the ROCmFPX custom-type era. The vendor build 10754 already carries the
-September upstream fixes (AVX-512 load segfault, and a `GGML_CUDA_FA_ALL_QUANTS`
--capable CUDA backend — proven by q8_0 V serving under flash attention at the
-correct 34,816 B/token, below).
-
-## VRAM census (2026-10-17, four arms, production stopped)
-
-The engine's own `common_memory_breakdown_print` ledger + settled `nvidia-smi`,
-all arms `--n-gpu-layers 99 --flash-attn on --batch-size 4096 --ubatch-size 2048`,
-mmproj attached:
-
-|Shape|KV|projected `model + context + compute`|settled used / free|verdict|
-|---|---|---|---|---|
-|**2 × 200,704 (pool 401,408)**|**q8_0 / q8_0**|21,439 = 6,539 + 13,627 + 1,272|**22,664 / 1,461**|**DEPLOYED**|
-|2 × 200,704|f16 / f16|33,198 = 6,539 + 25,387 + 1,272|—|`cudaMalloc failed: out of memory` on the **25,088.00 MiB** KV request|
-|1 × 262,144|q8_0 / q8_0|16,905 = 6,539 + 8,853 + 1,512|18,130 / 5,995|fits, but one slot serializes compaction|
-|2 × 262,144|q8_0 / q8_0|≈25,758 (arithmetic, same terms)|—|does not fit (f16 analogue measured 41,118)|
-|2 × 200,704|q4_0 / q4_0|15,167 = 6,539 + 7,355 + 1,272|16,392 / 7,733|the roomy fallback rung|
-
-**KV rate law (confirmed to the MiB):** 16 full layers × 4 KV heads × 256 dim =
-16,384 elements/token/cache; q8_0 bills 17,408 B per cache ⇒ **34,816 B/token**,
-so the 401,408 pool = **13,328.00 MiB** — the ledger printed exactly that
-(`CUDA0 KV buffer size = 13328.00 MiB`). f16 doubles it to 25,088 MiB, which is
-the wall. The 48 DeltaNet layers bill a fixed recurrent state per **slot**:
-context 13,627 − KV 13,328 = **299 MiB for two slots (~150 MiB each)** — pool
-arithmetic alone under-prices the shape by that much.
-
-**The projector is outside the fit check.** `[mtmd] estimated worst-case memory
-usage of mmproj is 873.10 MiB` is printed before fitting but not billed in it:
-settled used (22,664) − projection (21,439) ≈ 1.2 GiB is that projector plus the
-CUDA context. This is the difference between "2.6 GiB free on paper" and the
-real **1.4 GiB** — the same late-allocation failure class that killed the
-DSpark drafter boot in the llama.cpp eras. Any resize must re-read the settled
-`nvidia-smi`, never the projection.
-
-## Measured rates (greedy, `Server-Timing`-free `/completion` `timings`)
-
-|Probe|Result|
-|---|---|
-|Short, warm, single stream (128 tok)|**76.6 t/s** decode (the 9B Q6_K predecessor ran 94.4 — the ternary 27B is ~19 % slower, weight bandwidth + Hadamard dequant)|
-|First request after boot|2.6 t/s prefill / 37.7 t/s decode — **warmup artifact; never report the first request** (measured twice on this payload)|
-|Prefill at ~35k depth|**1,258.4 t/s** (27.8 s)|
-|Prefill at 166,027-token depth|**691.7 t/s** (240.0 s), zero errors, slot intact|
-|Two concurrent streams, 256 tok each|**54.7 + 54.7 = 109.4 t/s aggregate** — `--parallel 2` really batches|
-|Long-context retrieval (25,960-token prompt, needle at position 180 of 300 passages)|exact: *"Frank keeps the fleet in git, only through it."*|
-
-Production boot (the 19:44 cutover deploy, same binary/flags/file): **75.8 t/s**
-single-stream decode at a short prompt — within 1 % of the trial arm, so the
-ladder above carries to the shipped shape.
-
-## Behavior contract (probes, not assumptions)
-
-- **Reasoning** lands in `reasoning_content` (`--reasoning-format deepseek`) ✓.
-- **Tool calls**: a `get_weather` request returns `finish_reason: tool_calls`
-  with `arguments: {"city":"Seattle"}` — valid JSON, no loop, no HTTP 500. The
-  fork's `KNOWN_ISSUES.md` warning about malformed/looping tool calls did not
-  reproduce on build 10754 at this shape.
-- **Vision**: a 512 px solid-red PNG answers **"Red"** — but ONLY with a
-  thinking budget ≥ ~512 tokens. At `max_tokens: 64` the request returns
-  `finish_reason: "length"` with EMPTY `content` and half a thought in
-  `reasoning_content`: thinking tokens count against the budget, so a
-  short-budget vision probe reads as "the model is blind". Images do reach the
-  model (a 512 px image bills ~280 prompt tokens: 322 total vs 42 for the text
-  alone). Keep the ≥256 px probe rule from the earlier eras.
-- **Log noise law**: `/completion` (text context) at `VERBOSITY=4` prints
-  `failed to parse reasoning content (response_format json_schema is currently
-  only supported with parse_reasoning=true), content will be stored in
-  'content' field` (`probe_request.cpp:789`) whenever the model closes a raw
-  prompt with ``. It is **cosmetic** — the full chain of thought
-  is in `content`. The OpenAI `/v1/chat/completions` path parses it into
-  `reasoning_content` correctly, which is the path the harness and every
-  consumer uses. Do not diagnose this as a fault, and do not drop verbosity
-  below 4 — the KV/buffer ledger in the journal is what VRAM forensics read.
-- **Long raw-prompt completions can come back empty**: at ~26k+ tokens a bare
-  `/completion` request generated 0–1 tokens with no error (the model emits an
-  end-of-turn marker immediately, and a text context has no role frame to keep
-  it talking). Chat traffic is unaffected; nothing in the harness posts raw
-  `/completion` at depth.
-
-## Ops notes
-
-- **Trial-window law (cost one boot window on 2026-10-17):** a scratch env file
-  must use the serve script's OWN variable names — `CACHE_TYPE_K`/`CACHE_TYPE_V`,
-  `BIND_ADDR`, `NGL`. An invented `KV_CACHE_TYPE` is silently swallowed by the
-  script's `: "${CACHE_TYPE_K:=f16}"` default and the arm boots **f16**, which
-  for this payload is an OOM that looks like a model problem.
-- **`/metrics` needs `--metrics` (fixed 2026-10-18):** the fork answers **501 Not
-  Implemented** without the flag, and Dankclaw's Prometheus scrapes `/metrics`, so
-  `up{job="frank-zealot"}` has been 0 for the whole retained metric history — the
-  oldest sample (2026-10-11) is already 0 — while the endpoint itself answered
-  `/health` 200 the entire time. `METRICS=1` in `scripts/frank-zealot.env` now
-  ships the flag and the seam asserts it reaches the argv. halogen serves
-  `/metrics` natively, which is why that target was always green.
-- **Base alignment:** Bonsai-2 is the BASE (censored) Qwen3.8-27B lineage, so a
-  refusal is possible on a compaction or subagent request. The CRACK re-export
-  (`dealignai/Bonsai-2-27B-Ternary-CRACK-GGUF`, already on disk at
-  `/home/dank/models/Bonsai2-CRACK/Bonsai-2-27B-PQ2_0-CRACK.gguf`, sha256
-  `5b24ea3e…`) is a **one-line `MODEL_FILE` swap**: the header census proves
-  byte-identical geometry — 851 tensors, the same `{142:402, f32:353, bf16:96}`
-  histogram, 64 blocks, vocab 248,320, the same 8,952-char template. The VRAM
-  ledger does not move. If a refusal ever surfaces, that is the fix; there is
-  also an MTP-bearing CRACK re-export upstream
-  (`ManniX-ITA/…-CRACK-MTP-GGUF`, 7,657,489,920 B, +451 MB head) if decode ever
-  needs the drafter — untested here.
-- **Compaction hub unchanged:** all three endpoints still route
-  `compactionModel` here; 200,704 per slot is the fleet's only window that fits
-  a worst-case compaction input, and `--parallel 2` keeps compaction off the
-  critical path of a live generation.
-- **Harness side** (`~/.omp/agent/models.yml`): `contextWindow: 200704` and
-  `maxTokens: 16384` unchanged; `extraBody` now quotes the payload's own
-  1.0 / 0.95 / 20 with `preserve_thinking: true`. The `reasoningEffortMap` fold
-  became **load-bearing** (the template raises on any level outside
-  `xhigh|medium|low`) — an unmapped level is a hard 500 on the role's first
-  request, naming Jinja rather than the model.
-- **Rollback:** revert the swap commit. The Qwen3.5-9B Q6_K payload, its mmproj
-  and the master CUDA build all remain on disk
-  (`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`,
-  `~/GitPackages/llama-master/build-cuda/`), and the retired era keeps its own
-  page: [[frank-zealot-llama-qwen35-9b-q6k-2x200704]].
-- **Deployed 2026-10-17 19:44** (`deploy.sh dankclaw --service frank-zealot
-  --service homepage`): boot ledger 22,664 used / 1,461 free, engine fit
-  projected 21,439 and left 2,119 ≥ 1,872 MiB, KV 13,328.00 MiB at
-  `n_ctx_seq = 200,704`, `n_slots = 2`, projector loaded; live probes green
-  (`/v1/models` serves `frank-zealot`, exact `pong` + `reasoning_content`,
-  `tool_calls` `{"city":"Seattle"}`, 512 px PNG → **Red**). e2e gate 33/33.
-- **Picker row is live state:** the OpenWebUI `model` row's
-  `meta.description` named the 9B, so it was re-keyed in the database too
-  (backup `webui.db.bak-20261017a`). The row was written WITHOUT restarting
-  `open-webui` — the cached list refreshes on its next restart, and forcing one
-  would have logged every family session out for a cosmetic string. See
-  [[OpenWebUI]] for the picker trio.
-
-## Related
-
-- [[frank-zealot-llama-qwen35-9b-q6k-2x200704]] — the replaced era (rollback target)
-- [[frank-dragoon-flash-server-v2-swift15]] — the abliterated sibling on the iGPU
-- [[frank-archon-radiance-qwen38-iq4r]] — the censored-base flagship this payload shares base alignment with
-- [[deployment]] — deploy flow and incidents

serverAppConfigs-docs/LLM/frank-zealot-llama-prism-bonsai9-mtp.md

@@ -1,286 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Bonsai-2-9B PQ2_0 with the MTP head, 3 × 262144, q8_0 KV (ACTIVE)"
-description: "The ACTIVE frank-zealot runbook (since 2026-10-18) — the PrismML fork build 10777 of llama.cpp serving prism-ml/Ternary-Bonsai-2-9B-gguf at PQ2_0 on the RTX 3090: THREE conversations of 262,144 tokens (every slot at the model's trained window) over q8_0 K + q8_0 V, vision through the publisher's Q8_0 Qwen3-VL projector, and the five-arm boot census that put speculative decoding OFF by measurement even though the checkpoint carries the MTP head."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama-cpp
-  - gguf
-  - rtx3090
-  - mtp
-  - spec-decode
-timestamp: 2026-10-18
----
-
-# frank-zealot — llama.cpp (PrismML) Bonsai-2-9B PQ2_0 + MTP head, 3 × 262,144, q8_0 KV
-
-Live since 2026-10-18, replacing the
-[[frank-zealot-llama-prism-bonsai2-pq20]] era (Ternary-Bonsai-2-**27B** PQ2_0 at
-2 × 200,704). The trade in one sentence: the 9B bills **half** the KV per token
-of the 27B, and that is exactly what funds **three** conversations at the
-model's **native 262,144** window where the 27B afforded only two shortened
-ones (Frank's ask: *"if you can possibly fit a third context, do it"*). Every
-axis improved — decode, prefill, three-stream aggregate, and free VRAM.
-
-## Payload identity
-
-| Item | Value |
-|---|---|
-| Runtime | `/home/dank/GitPackages/llama-prism-b10777/llama-server` — PrismML **fork** build, `--version` → `0.2.0-dev (build 10777, commit e3de71522)` |
-| Checkpoint | `/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf` — 2,675,410,720 B, Hub-LFS sha256 `c79fcdfaf6b7bf47089b50dadcfd11f9ce7d3e96adf3bbefde7912e5ef69e22b` |
-| Vision | `/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf` — 624,229,664 B, Hub-LFS sha256 `e0cb6f6988cea323ef798cd77d019bbb819af75582f034eaab0e7232314953db` |
-| Fetch | `scripts/download-frank-zealot-bonsai9.sh [DEST]` (`--verify-only`; `-f`, byte-count and LFS sha256 per file) |
-| Alias | `frank-zealot` (`ALIAS`); the SERVICE/unit/scripts keep the StarCraft name, only the wire id is the model id |
-| Port | 7771, root systemd unit `frank-zealot.service` (relinked), `TimeoutStopSec=180` |
-
-### The runtime lock has two tumblers now
-
-1. **The tensor type.** `general.file_type 141` with **202 tensors of the fork's
-   PQ2_0 type** (of 442 total; the rest are f32 norms/scalars and 8 q8_0 in the
-   MTP block) plus `prism.hadamard.*` metadata
-   (`normalized-sylvester-walsh-hadamard`, `block_size 4096`,
-   `axis input-last-dimension`, `sign_mode explicit` over 201 weight names,
-   inverse on `token_embd.weight`, `gdn_v_grouped true`). Stock llama.cpp
-   refuses the custom type at load; a loader that ignored the Hadamard metadata
-   would dequant **wrong**, silently. Same lock class as the ROCmFPX era.
-2. **The build floor.** The fork's card gates **MTP beside a quantized V cache**
-   on `prism-b10776-d00e252` or newer, so the prebuilt moved **10754 → 10777**
-   for this swap. The 10754 tree stays on disk (`~/GitPackages/llama-prism/`) as
-   the 27B rollback runtime. `tests/frank_zealot_unit.bats` pins both paths.
-
-## Header census (read from the GGUF, not the card)
-
-`general.architecture qwen35`, `general.name Ternary-Bonsai-2-9B`,
-`general.size_label 9B`, `general.quantization_version 2`,
-`general.file_type 141`:
-
-| Key | Value | Note |
-|---|---|---|
-| `qwen35.block_count` | **33** | 32 trunk blocks + `blk.32` = the MTP block |
-| `qwen35.full_attention_interval` | 4 | ⇒ **8 full-attention + 24 Gated DeltaNet** trunk layers |
-| `qwen35.context_length` | 262,144 | native; the deployed slot equals it |
-| `qwen35.embedding_length` / `feed_forward_length` | 4,096 / 12,288 | |
-| `attention.head_count` / `head_count_kv` | 16 / **4** | |
-| `attention.key_length` / `value_length` | 256 / 256 | |
-| `ssm.conv_kernel` / `state_size` / `group_count` | 4 / 128 / 16 | the 27B: 4 / 128 / 16 |
-| `ssm.time_step_rank` / `inner_size` | **32 / 4,096** | the 27B: 48 / 6,144 |
-| `rope.freq_base` / `dimension_count` / `dimension_sections` | 1e7 / 64 / `[11,11,10,0]` | |
-| `qwen35.nextn_predict_layers` | **1** | the MTP head is IN the file |
-| `general.sampling.*` | temp 1.0 / top_p 0.95 / top_k 20 / min_p 0.0 / repeat 1.0 | the card recipe; llama.cpp reads it itself |
-
-**KV accounting.** Only the 8 full-attention layers scale with context:
-8 × 4 KV heads × 256 dim = **8,192 elements/token per cache**, so
-**q8_0 K + q8_0 V bills 17,408 B/token** — the 786,432-position pool is
-**13,056.00 MiB**, ledger-exact. That is exactly HALF the 27B's 34,816 B/token.
-**f16 KV would bill 26,112 MiB against a 24,123 MiB card** and dies at
-`cudaMalloc`. The 24 DeltaNet layers bill a fixed recurrent state per SLOT
-instead: the PRODUCTION ledger prints `RS buffer size = 150.75 MiB (3 cells,
-32 layers, 3 seqs)` ≈ 50 MiB a cell, while the SPEC-ARMED arms printed
-452.25 MiB — MTP depth 2 bills three seqs per slot, so the drafter triples
-even the recurrent state. Pool arithmetic alone under-prices the shape.
-
-## The shape (`scripts/frank-zealot.env`)
-
-`--ctx-size 786432 --parallel 3` (⇒ `n_ctx_slot = 262,144`), `--flash-attn on`,
-`--cache-type-k q8_0 --cache-type-v q8_0`, `--batch-size 4096`,
-`--ubatch-size 2048`, `--n-gpu-layers 99`, `--jinja`, `--reasoning auto`,
-`--reasoning-format deepseek`, `--metrics`, `--mmproj …`, `--log-verbosity 4`,
-`--presence-penalty 1.0`,
-**`SPEC_TYPE=none`** (which strips `--spec-type` AND every `--spec-draft-*`).
-
-Per-slot window is the **contract** the harness quotes: `models.yml` carries
-`contextWindow: 262144`. llama.cpp refuses a prompt longer than the SLOT with a
-400 naming the limit — it never truncates — and since this swap the slot window
-equals the native window, so no YaRN and no cap is in play.
-
-## The MTP verdict: the head is in the file and it is still OFF
-
-The drafter is **good**: acceptance 0.798 / 0.617 per position, mean accepted
-length 2.41, and the card recommends depth 2. It loses **at this shape**
-because the draft context bills its own **f16** KV over the WHOLE pool —
-`blk.32` carries real attention (4 KV heads × 256 × 2 caches × 2 B =
-**4,096 B/position**) × 786,432 positions = **3,072.00 MiB**, 4,830 MiB with its
-compute buffers — and `--ubatch-size 2048` refuses to give that up.
-
-Booted arms, same card, same traffic, 2026-10-18 (`/completion` `timings`,
-greedy 512 tokens; never the first request after a boot):
-
-| slots | spec | ubatch | used / free | decode | 3-stream | prefill @106k |
-|---|---|---|---|---|---|---|
-| **3 × 262,144** | **off** | **2048** | **17,876 / 6,249** | **167.2** | **280.2** | **2,877** |
-| 3 × 262,144 | on | 2048 | 22,706 / 1,419 | 109.2 | — | — |
-| 3 × 262,144 | on | 1024 | 21,400 / 2,725 | 213.5 | 250.8 | 2,395 |
-| 2 × 262,144 | on | 2048 | 17,178 / 6,947 | 213.2 | — | — |
-| 1 × 262,144 | on | 2048 | 11,652 / 12,473 | 214.2 | — | — |
-
-**MTP is worth +28 % single-stream decode whenever the card has room** (rows
-4–5 prove the drafter itself is a win; row 2 proves the squeeze is what halves
-it). The arm that runs MTP at three windows is `SPEC_TYPE=draft-mtp` +
-`UBATCH_SIZE=1024` (row 3) and pays **prefill −17 %** and **2,725 MiB of
-headroom** on a card whose mmproj worst-case lands AFTER the engine's own fit
-check. This endpoint is the fleet's **compaction hub** — prefill-bound ~170k
-inputs plus 2–3 concurrent subagents — so the shipped arm takes the +17 %
-prefill and +12 % three-stream aggregate over single-stream speed, and keeps
-6.2 GiB free. Frank confirmed this arm on 2026-10-18.
-
-## Boot ledger (production boot 2026-10-18 13:19, under the unit, verbosity 4)
-
-```
-CUDA0 model buffer size  =  2036.71 MiB    # + 257.66 MiB CPU_Mapped (mmap)
-CUDA0 KV buffer size     = 13056.00 MiB    # q8_0 K + q8_0 V, 786,432 positions
-CUDA0 RS buffer size     =   150.75 MiB    # 3 cells, 32 layers, 3 seqs
-CUDA0 compute buffer     =  1409.02 MiB    # + 1088.87 MiB CUDA_Host, 2.84 output
-mmproj worst-case (est.) =   868.32 MiB    # 248.10 CUDA + 24.93 CPU, AFTER fit
-nvidia-smi settled: 17,922 MiB used / 6,203 MiB free of 24,123
-n_slots = 3, n_ctx_slot = 262144, kv_unified = false
-```
-
-No clamp line, no `cudaMalloc` error. The engine's own projection (21,439-class
-numbers from the 27B era) excludes the projector — **read settled `nvidia-smi`,
-never the projection**.
-
-## Measured (production boot, `/completion` `timings`, greedy, `ignore_eos`)
-
-- **168.7 t/s** decode — three runs of 512 tokens: 168.4 / 168.7 / 168.7
-  (the trial arm printed 167.2; same shape, same noise band).
-- **~300 t/s aggregate** across three concurrent 512-token streams
-  (102.8 t/s per stream, round 1 and 2 both) — the trial arm's 280.2.
-- **2,877 t/s prefill at 106,418 tokens** — the trial arm; the production boot
-  has not re-run that depth.
-- The 27B record it replaces: 76.6 decode, 109.4 aggregate across two streams,
-  691.7 prefill at 166k. Beaten on every axis.
-- The **first request after a boot is a warmup artifact** (37.7 t/s class) —
-  never report it. And never time a wall-clock aggregate on a generation that
-  stops on its own: a 24-token countdown measured 72 t/s across three streams
-  while the same shape with `ignore_eos` gave 300.
-
-## Contract traps
-
-- **`--metrics` is mandatory.** Without it llama.cpp answers **501** on
-  `/metrics`, so the Prometheus scrape reads DOWN while `/health` answers 200 —
-  that is how this endpoint sat red across four payload generations
-  (`up{job="frank-zealot"}` oldest retained sample, 2026-10-11, is already 0).
-- **Q8_0 mmproj, not BF16.** The publisher's card: the BF16 tower "can crash
-  llama-server when several image requests run at once; Q8_0 is not affected" —
-  and `--parallel 3` is precisely that shape.
-- **Only the `image_url` content part reaches the model.** The legacy top-level
-  `images` field bumps `usage.prompt_tokens` and the model still invents an
-  answer. Probe with a solid color and expect the exact word.
-- **≥256 px probe images** or the model answers "video".
-- **Thinking tokens count against `max_tokens`.** A vision probe at 64 tokens
-  returns EMPTY `content` + `finish_reason: "length"` — the endpoint looks blind
-  while it is only mid-thought. Budget ≥512.
-- **`presence_penalty` has no GGUF metadata key**, so the one recipe value that
-  must reach the argv is this one. The serve script uses
-  `${PRESENCE_PENALTY=1.0}` — **not `:=`** — because `""` is the documented
-  opt-out and `:=` would silently re-add the flag (caught by the seam, fixed
-  2026-10-18).
-- **Spec flags move as a SET.** A bare `--model-draft`/`--spec-type` arm without
-  the whole set has segfaulted at load on this fork family in the past;
-  `SPEC_TYPE=none` strips all of them.
-- **Base alignment is the censored base lineage, and the CRACK swap is NOT
-  one line here.** `/home/dank/models/Bonsai2-CRACK/` matches the **27B**
-  geometry, not this 9B file; a dealigned 9B would have to be fetched.
-- **A raw `/completion` request** logs `failed to parse reasoning content
-  (… parse_reasoning=true)` when the model closes a bare prompt with `` —
-  cosmetic (`probe_request.cpp:789`); `/v1/chat/completions` parses
-  `reasoning_content` correctly. Do not lower `VERBOSITY`: the KV/buffer ledger
-  is what VRAM forensics read.
-- **A failed stop leaves a `failed` unit, and an orphan keeps answering.** On
-  2026-10-18 the trial stop hit `State 'stop-sigterm' timed out. Killing.` →
-  `Active: failed (Result: timeout)`, while the hand-booted arm kept serving
-  7771 — a live port with no unit behind it. `TimeoutStopSec=180` is now on the
-  unit (llama.cpp needs the drain time to release slots and the card); the
-  hand-booted arm must be killed by PID, never `pkill -f llama-server`.
-- **A leftover arm drop-in hijacks the production boot, and no health check
-  sees it.** The sanctioned bench method is a transient
-  `Environment=FRANK_ZEALOT_ENV_FILE=/tmp/<arm>.env` drop-in, and an arm env
-  `source`s the repo env and then overrides it. On 2026-10-18 a stale
-  `frank-zealot.service.d/ptq1.conf` — written 03:29 for the 27B **PTQ1_0** arm
-  Frank had ordered at 02:36 and measured from 03:14 (deploy ledger below) —
-  was still installed, so the cutover deploy restarted
-  the unit and booted THAT: `--model …/Bonsai2-PTQ1_0/Ternary-Bonsai-2-27B-PTQ1_0.gguf
-  --ctx-size 401408 --parallel 3 --batch-size 3072 --ubatch-size 1536`, three
-  slots of 133,888, 21,188 MiB used. `/health` was 200, `/v1/models` said
-  `frank-zealot` (the alias is a flag, not a fact), and the Playwright gate
-  passed 33/33 — none of those read the payload. **After deploying an endpoint
-  that was recently benched: `systemctl show -p DropInPaths`, then
-  `tr '\0' ' ' </proc/$pid/cmdline`, then the ledger — in that order.** Retire
-  the drop-in to `~/.AGENTTRASH/<date>-<topic>/`, `daemon-reload`, restart.
-
-## Compaction hub
-
-`models.yml` routes `compactionModel` for `frank-archon`, `frank-dragoon` and
-`frank-zealot` here — 262,144 is the widest window in the fleet and the only one
-that comfortably fits a worst-case compaction input (~170k tokens; the clamp is
-`min(maxTokens 16384, contextWindow − est − 4096)`, compaction fires at 85 % of
-the window ≈ 222,822 est, worst case ≈ 245,891 ≤ 262,144). With `--parallel 3`
-a compaction request no longer serializes behind a live generation (the Gemma4
-era's `--parallel 1` law).
-
-## Deploy ledger (2026-10-18)
-
-- `./scripts/deploy.sh dankclaw --service frank-zealot` at 13:16 PDT; the e2e
-  gate ran 33/33. The first boot was the hijacked arm above; the drop-in went
-  to `~/.AGENTTRASH/20261018-zealot-ptq1-arm/` and the 13:19 restart is the
-  ledger recorded here.
-- Live probes on the shipped boot: chat `27*59` → **1593**; `where_am_i` tool
-  call fired with schema-correct empty arguments; 512 px solid-red PNG →
-  **Red** at 278 prompt tokens with `process_mtmd` chunk encoding in the
-  journal; `/props` reports `n_ctx 262144` and `presence_penalty 1.0`;
-  `/metrics` answers 200 with 15 `llamacpp:*` families and
-  `up{job="frank-zealot"}` is **1** in Prometheus for the first time in its
-  retained history.
-- OpenWebUI picker rows re-keyed in the live DB (`Connection.backup()` snapshot
-  `webui.db.bak-20261018` taken first; `frank-dragoon`'s `context.length` also
-  corrected 262,144 → 200,704 — see [[OpenWebUI]]).
-- **What the bench window left behind is not garbage — the PTQ1_0 arm was
-  ordered, measured, and then scratched by Frank himself.** The instruction at
-  **02:36** was: run frank-zealot on `Ternary-Bonsai-2-27B-PTQ1_0.gguf`, the
-  **Q8** mmproj, batch/ubatch **3072/1536**, and as much context as fits in
-  **two** contexts. The unit booted it at **03:14** (journal:
-  `type ptq1_0: 402 tensors`, `file type = PTQ1_0 - 1.75 bpw ternary`, mmproj
-  loaded, 2 × 200,704) and served 24 minutes of traffic. Its own
-  `print_timing` lines measure **~48 t/s** single-stream decode
-  (48.0 / 47.9 / 47.6 / 49.0 t/s on 512 / 229 / 512 / 451-token generations),
-  25–27 t/s per stream with two streams live, and prefill **1,295 t/s at
-  30,637 → 943 at 97,856 → 801 t/s at 141,365** tokens. It bought the
-  1,201 MiB the fetch script promised and paid for it on decode, exactly as the
-  publisher's Ampere row warned (the PQ2_0 record for the same weights is
-  76.6 t/s). At **03:33** Frank scratched the whole 27B plan and ordered the
-  9B PQ2_0 + MTP shape deployed here; the arm process ended at 03:38.
-- The actual failure was **documentation, not measurement.** An approved
-  payload that survives only as a systemd drop-in plus a journal window is,
-  a day later, indistinguishable from a rogue arm — which is why the 13:16 boot
-  read as a hijack. Payload decisions belong in this runbook the moment they
-  are made. The evidence is still perishable: `/tmp/zealot-arm-current.env` and
-  the 9B `/tmp/arm-*.jsonl` results die on reboot, and only the journal
-  (`journalctl -u frank-zealot --since "2026-10-18 03:10"`) holds the PTQ1_0
-  numbers. `/home/dank/models/Bonsai2-PTQ1_0/` (5,946,648,928 B + hardlinked
-  mmproj) stays on disk as the 27B rollback.
-
-## Fallback rungs
-
-1. `UBATCH_SIZE=1024` — frees **1,306 MiB** measured on this shape (prefill
-   −17 %).
-2. `CACHE_TYPE_K/V=q4_0` — frees ~6.5 GiB at three windows (the 27B era measured
-   the q4_0 rung at 16,392 used / 7,733 free at 2 × 200,704).
-3. Cut to 2 slots — but that breaks the third concurrent subagent, so it is a
-   last resort, not a knob.
-
-## Rollback
-
-Revert the swap commit. On disk: the **27B payload**
-(`/home/dank/models/Bonsai2-PQ2_0/`) + the **10754** fork tree, the Qwen3.5-9B
-Q6_K payload (`/home/dank/models/Qwen35-9B-HauhauCS-Q6K/`) + master CUDA build,
-the Gemma4, MiMo, Swift and Ornith payloads. Then
-`./scripts/deploy.sh dankclaw --service frank-zealot`.
-
-## Related
-
-- [[frank-zealot-llama-prism-bonsai2-pq20]] — the retired 27B era (rollback target)
-- [[Llama Cpp Serving Sizing]] — the two-boot VRAM pricing method
-- [[frank-dragoon-flash-server-v2-native]] — the other 2026-10-18 cutover
-- [[deployment]] — deploy flow and incidents

serverAppConfigs-docs/LLM/frank-zealot-llama-qwen35-9b-q6k-2x200704.md

@@ -1,167 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp, Qwen3.5-9B Q6_K, 2 x 200704, f16 KV (RETIRED, rollback target)"
-description: "The frank-zealot runbook of 2026-10-07 → 2026-10-17 (RETIRED — rollback target) — llama.cpp master (CUDA sm_86) serving HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive at Q6_K on the RTX 3090: two conversations of 200,704 tokens with exact f16 KV, vision through the publisher's BF16 projector, MTP impossible (no tensors in the file), and the measured geometry that priced the shape."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama-cpp
-  - gguf
-  - rtx3090
-timestamp: 2026-10-07
----
-
-# frank-zealot — llama.cpp Qwen3.5-9B Q6_K, 2 × 200,704, f16 KV
-
-**RETIRED 2026-10-07 → 2026-10-17** (AGENTS.md Runbook Index), superseded by
-[[frank-zealot-llama-prism-bonsai2-pq20]] (27B PQ2_0, 2 × 200,704 q8_0 KV, build
-10754), itself retired 2026-10-17 → 10-18 and now the rollback target: :7771
-today is llama.cpp PrismML build **10777** on
-`Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf`, 3 × 262,144 q8_0 KV, MTP off by
-measurement (AGENTS.md Runbook Index active row; `scripts/frank-zealot.env`).
-Transient 2026-10-18: :7771 is inside a bench window — the unit is `failed`
-since 03:14 and a hand-booted process answers the port; restore with
-`./scripts/deploy.sh dankclaw --service frank-zealot`.
-This era is the rollback target: the payload is still on disk
-(`stat -c %s /home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf`
-→ 7359259008, read 2026-10-18) with its master CUDA build. Same census: the
-10754 rollback payload `Bonsai2-PQ2_0/Ternary-Bonsai-2-27B-PQ2_0.gguf` is
-present at 7206168928 with its Q8_0 mmproj at 629246976, and
-`Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf` at 2675410720 —
-nothing this era or its successors need has been purged. It replaced the
-[[frank-zealot-llama-gemma4-q4km-200704]] era (Q4_K_M on ONE 200,704
-slot). The swap is a capacity trade: the 9B dense-hybrid rung costs ~7 GiB of
-weights where the 26B MoE cost ~16.6 GiB, and the freed room buys the second
-conversation slot and unquantized KV back.
-
-## Identity
-
-| | |
-|---|---|
-| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
-| Endpoint | `http://192.168.0.143:7771` — served model id **`frank-zealot`** (`ALIAS` in the env; the SERVICE keeps the `frank-zealot` name), `/v1` OpenAI surface |
-| Runtime | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` — `--version` on the disk build reads `0.5.0-dev (build 1, commit 5fc4f3c)` (built 2026-10-01); its `CMakeCache.txt` says `GGML_CUDA_FA_ALL_QUANTS:BOOL=OFF` and that tree carries no CUDA-side guard for the option (`ggml/src/ggml-cuda/fattn.cu:438-449` dispatches Q4/Q5 KV unconditionally), so quantized KV never required it; `qwen35` is registered in the build's own `src/llama-arch.cpp` and `src/models/qwen35.cpp` exists at that commit |
-| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
-| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
-| Weights | `/home/dank/models/Qwen35-9B-HauhauCS-Q6K/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-Q6_K.gguf` — HauhauCS **Q6_K**, 7,359,259,008 B, sha256 `c0ba7beb68fd3fe47891bd549486d38dcf62d00817296ea314ad37017f5a4986` |
-| Projector | `mmproj-Qwen3.5-9B-Uncensored-HauhauCS-Aggressive-BF16.gguf`, 921,704,448 B, sha256 `05f662501f8bd45607b079723a3e238a4e888fd085a10a53f4057a0e250f6934` — the model is natively multimodal and the Q6_K file carries no vision tensors, so this file IS the vision capability |
-| Fetch | [`scripts/download-frank-zealot.sh`](../../scripts/download-frank-zealot.sh) — `-f`, byte-count + sha256 verified (the hashes are computed locally: the repo is Xet-backed, so the Hub publishes file OIDs, not file sha256), idempotent, `--verify-only` |
-| Seam | `tests/frank_zealot_unit.bats` (argv assertions against a stub engine) |
-
-## Payload geometry (read from the GGUF header, not the model card)
-
-Range-read of the file's own metadata + tensor-info section:
-
-|Key|Value|
-|---|---|
-|`general.architecture`|`qwen35` (dense hybrid: Gated DeltaNet + softmax attention)|
-|`qwen35.block_count`|32, `full_attention_interval` 4 ⇒ **8 full-attention layers (3, 7, 11, …, 31) + 24 DeltaNet layers**|
-|attention|16 heads, **4 KV heads**, `key_length` = `value_length` = 256|
-|`qwen35.ssm.*`|`conv_kernel` 4, `state_size` 128, `group_count` 16, `time_step_rank` 32, `inner_size` 4096|
-|context / vocab|262,144 native / 248,320|
-|tensor types|Q6_K × 250 + F32 × 177 — standard types only, so the stock master build loads it (the ROCmFPX-style custom-quant lock-in does not apply)|
-|MTP|**no `nextn`/`mtp` tensors exist** — in-file MTP is impossible; the card's "MTP support" line describes the base architecture, not this quantized export|
-|chat template|supports `enable_thinking`, `tools`/`tool_call`, image/video content parts; it has **no `reasoning_effort` variable** (unused Jinja variables are ignored, so the harness's effort map is inert here, not an error source)|
-
-## Shape
-
-`--ctx-size 401408 --parallel 2` — llama.cpp divides the pool across slots,
-so each conversation gets `n_ctx_slot = 200,704` (Frank's cap; the harness
-`contextWindow` quotes the per-slot number). `--flash-attn on`,
-**`--cache-type-k f16 --cache-type-v f16`** (exact KV), batch 4096 /
-**ubatch 2048**, `--log-verbosity 4`, `--jinja --reasoning auto
---reasoning-format deepseek`, `--mmproj`, **speculative decoding OFF**
-(`SPEC_TYPE=none` strips both spec flags — the file has no MTP tensors).
-
-**KV rate law.** The pool scales with the 8 full-attention layers: 8 × 4 KV
-heads × 256 dim = 8,192 elements/token per cache, so f16 K + f16 V bills
-32,768 B/token = **32 KiB/token** → 12.25 GiB at the 401,408 pool. The 24
-DeltaNet layers bill a fixed recurrent state **per slot**, not per token —
-that is what makes two 200k slots affordable where the 26B MoE fit only one.
-
-**Priced from boots, not formulas.** MiMo-V2.6-Distill-Qwen-9B (identical
-qwen35 geometry, 7,793,710,624 B, same mmproj class) at 2 × 200,704 f16 K +
-f16 V measured **21,276 used / 2,849 free** at ubatch 1024 and **21,846 /
-2,279** at ubatch 2048 — see [[frank-zealot-llama-mimo-q6k-2x200704]].
-This file is 415 MiB lighter in weights and its projector is 3 MiB heavier,
-so the projection is ~21,430 used / ~2,700 free.
-
-Ubatch 2048 is the measured CUDA optimum for this card and arch (+4.6 %
-prefill at 8.9k depth, decode flat, cost 616 MiB); `1024` is the documented
-fallback rung if a future shape lands under the 2 GiB floor.
-
-## Measured boot (2026-10-07, deploy `ec6184d`)
-
-`nvidia-smi`: **21,436 MiB used / 2,689 MiB free** — the MiMo-priced
-projection (~21,430 / ~2,700) was within 11 MiB. Engine ledger
-(`--log-verbosity 4`, `journalctl -u frank-zealot`):
-
-|Line|Value|
-|---|---|
-|`llama_context`|`n_ctx = 401408`, `n_ctx_seq = 200704`, `kv_unified = false`|
-|`llama_kv_cache`|`CUDA0 KV buffer size = 12544.00 MiB` — `size = 12544.00 MiB (200704 cells, 8 layers, 2/2 seqs), K (f16): 6272.00 MiB, V (f16): 6272.00 MiB`|
-|`sched_reserve`|`CUDA0 compute buffer size = 1136.06 MiB`, `CUDA_Host compute buffer size = 848.07 MiB`|
-|`clip_model_loader` (mmproj)|`model size: 878.99 MiB` + `reserve_compute_meta: CUDA0 248.10 MiB, CPU 24.93 MiB`|
-|`srv load_model`|`n_slots = 2, n_ctx_slot = 200704`|
-|`print_info`|`n_ctx_train = 262144`, `n_head_kv = 4`|
-
-The `32 KiB/token` law is confirmed exactly: 401,408 × 32 KiB = 12,544 MiB,
-which is what the engine printed. The `n_ctx_seq (200704) < n_ctx_train
-(262144)` line is informational (a context **below** the trained window is
-safe; the Gemma4 era's warning was the opposite case, a drafter running 53 %
-**past** its window). Weights load in ~6 s from page cache; `/health` answers
-`{"status":"ok"}` and `/v1/models` serves the alias `frank-zealot`.
-
-**Wire rename (2026-10-09):** `ALIAS` moved `frank-secondary` → `frank-zealot`
-(commit `7c77326`; the service-file rename to the `frank-zealot.*` names
-followed 2026-10-10, commit `55f87fc`),
-so the served id matches the harness model id and the family-facing name.
-Consumers re-keyed in the same change: OpenWebUI (`model` row, `access_grant`
-row, `openai.api_configs` index-2 `model_ids`) and `models.yml` (already
-renamed 2026-10-08). llama.cpp does NOT validate the requested model name —
-it answers and echoes the served alias — so a stale client keeps working;
-the picker and any id-keyed grant is what breaks. See [[OpenWebUI]].
-
-The deploy's Playwright gate passed (33 tests) and the fleet e2e set is
-unchanged.
-
-## Client contract
-
-llama.cpp surface: model id `frank-zealot`, native tools via `--jinja`,
-thinking in **`reasoning_content`**, vision via OpenAI `image_url` content
-parts (probe with a ≥ 256 px image, or the Qwen-VL tower answers "video").
-The publisher's recipe ships in `models.yml` `extraBody`: thinking mode
-`temperature 0.6 / top_p 0.95 / top_k 20 / min_p 0` (non-thinking mode is
-0.7/0.8/20/0). The card's advice "maintain at least 128K context to preserve
-thinking" is satisfied by the 200,704 slot.
-
-**Compaction hub**: `models.yml` routes `compactionModel` for all three
-endpoints here — still true 2026-10-18 (`~/.omp/agent/models.yml:25,107,185` all name `llamacpp (dankclaw-3090)/frank-zealot`). With `--parallel 2` a compaction request no longer serializes
-behind a live generation (the Gemma4 era's `--parallel 1` law), and the
-200,704 slot still fits the worst-case compaction input (~133k tokens).
-
-## Rollback
-
-|Target|How|
-|---|---|
-|Gemma4-26B-A4B Q4_K_M, 1 × 200,704, f16 K + q8_0 V|revert the swap commit — payload stays at `/home/dank/models/Gemma4-26B-A4B/`; see [[frank-zealot-llama-gemma4-q4km-200704]]|
-|MiMo-V2.6 Q6_K, 2 × 200,704, f16 KV|`MODEL_FILE=/home/dank/models/MiMo-Q6K/MiMo-V2.6-Distill-Qwen-9B-Q6_K.gguf` + its mmproj — measured 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048|
-|Swift-1.5-27B IQ3_XXS 2 × 114,688 + MTP|payload on disk at `/home/dank/models/Swift15-UncIQ3XXS/`|
-
-## When it misbehaves
-
-Same decision tree as the predecessors: `No CUDA GPUs are available` ⇒
-`nvidia-modprobe -u -c=0`; a 400 naming 200,704 ⇒ the conversation outgrew
-one slot, not the pool; queue behind two long generations ⇒ the slot ceiling
-is two, by contract; a boot that OOMs ⇒ read the verbosity-4 ledger in
-`journalctl -u frank-zealot` (KV MiB, compute buffers) before touching
-flags — the engine prints the exact figures the shape needs.
-
-**`status=203/EXEC` boot loop**: the rewrite of `serve-frank-zealot.sh`
-shipped without its `#!/usr/bin/env bash` line, and systemd failed the
-`ExecStart` before the script ran — 10 s restarts, `Mem peak: 1.9M`, and
-`nvidia-smi` flat at 1 MiB used (the GPU is never touched). Journal signature:
-`Failed at step EXEC spawning /home/dank/…/serve-frank-zealot.sh`. A
-shebang is not decoration on this host: every LLM serve script is a direct
-`ExecStart`, so `head -1` is part of the deploy check (commit `ec6184d`).

serverAppConfigs-docs/LLM/frank-zealot-llama-swift15-unc-iq3xxs-mtp-114688.md

@@ -1,353 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 x 114688, draft-mtp, q8_0 KV"
-description: "The frank-zealot runbook of 2026-10-05 → 2026-10-07 (RETIRED — rollback target; live from 2026-10-05 evening, renamed from frank-small 2026-10-06, per-slot lowered to 112k and ubatch raised to 1024 the same evening) — llama.cpp master (CUDA sm_86) serving ajgazin's Swift-1.5-Qwen3.8-27B Uncensored RCO-IQ3_XXS (MTP head in the file, drafted) on the RTX 3090: TWO 114,688-token conversations at Frank's 112k cap, q8_0 K + q8_0 V (llama.cpp's FP8 class), vision through the publisher's own BF16 projector, and the full two-slot + drafter arithmetic."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama-cpp
-  - gguf
-  - rtx3090
-  - qwen35
-  - mtp
-  - vision
-timestamp: 2026-10-06
----
-
-# frank-zealot — llama.cpp Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2 × 114,688, draft-mtp, q8_0 KV
-
-> **RETIRED 2026-10-06 → 2026-10-07** (AGENTS.md Runbook Index; cut over
-> 2026-10-05 ~23:26, reshaped three times on 2026-10-06, renamed from
-> frank-small the same night). Superseded by
-> [[frank-zealot-llama-qwen35-9b-q6k-2x200704]]; the production shape on :7771
-> today is llama.cpp PrismML build **10777** on
-> `Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf`, 3 × 262,144 q8_0 KV, MTP off by
-> measurement (AGENTS.md Runbook Index active row); the 27B PQ2_0 / 2 × 200,704
-> shape of [[frank-zealot-llama-prism-bonsai2-pq20]] is that era's rollback
-> target. Transient 2026-10-18 bench window on :7771 — the unit is `failed`
-> since 03:14 and a hand-booted process answers the port; restore with
-> `./scripts/deploy.sh dankclaw --service frank-zealot`).
-> **Rollback target**: the payload is still on disk —
-> `stat -c %s` on 2026-10-18 gives 10442827392 for
-> `/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf`
-> and 931145888 for its `mmproj-BF16.gguf`. A 27B replaces
-> the 9B on the same 24 GiB card: KV at FP8 class, vision on. The
-> morning order (drop MTP, ubatch 512, TWO 128k contexts) bought the
-> second slot — 1,397 MiB free. The evening order swapped to the
-> UNCELERSED Swift-1.5 at RCO-IQ3_XXS and put MTP back ON — landing
-> **489 MiB free**: the MTP draft ctx costs **~2.4 GiB, not the ~0.4
-> GiB assumed** (the drafter's KV is built against the target's full
-> context, not the 3-token proposal depth), paid for by the lighter
-> quant (−1.56 GiB). The late order dropped the per-slot window
-> 131,072 → **114,688** (Frank's 112k cap) and climbed the prefill ladder
-> ubatch 512 → 1024 → **2048** — landing **1,005 MiB free** (1,751 at
-> the 1024 rung; the 2048 rung cost 746 MiB, ~130 over the 616 MiB
-> measured prediction). This supersedes BOTH earlier 2026-10-05
-> payloads: the live MiMo-V2.6 Q6_K shape
-> ([[frank-zealot-llama-mimo-q6k-2x200704]], now retired) and the
-> Ornith-1.5-9B Q8_0 swap that was committed (77decb5..380aafa) but
-> never deployed.
-
-## Identity
-
-| | |
-|---|---|
-| Host / device | Dankclaw (192.168.0.143), RTX 3090, 24,576 MiB |
-| Endpoint | `http://192.168.0.143:7771` — alias `frank-secondary` on the wire until the 2026-10-09 rename to `frank-zealot` (commit `7c77326`), `/v1` OpenAI surface |
-| Runtime | llama.cpp master (0.5.0-dev, commit 5fc4f3c), CUDA sm_86 build `…/llama-master/build-cuda/bin/llama-server` (quantized V needs `--flash-attn on`; it does NOT need `GGML_CUDA_FA_ALL_QUANTS` — see the KV section) |
-| Unit | root systemd `frank-zealot.service` ([`scripts/frank-zealot.service`](../../scripts/frank-zealot.service)), `User=dank`, `Restart=on-failure`, `LimitMEMLOCK=infinity` |
-| Serve script | [`scripts/serve-frank-zealot.sh`](../../scripts/serve-frank-zealot.sh), tunables [`scripts/frank-zealot.env`](../../scripts/frank-zealot.env) |
-| Weights | `/home/dank/models/Swift15-UncIQ3XXS/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf` — ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF **RCO-IQ3_XXS -mtp**, 10,442,827,392 B, sha256 `e13d3e74…` (qwen35 hybrid: 64 real layers, 16 full-attention + 48 Gated DeltaNet, native 262,144; **MTP head in the file**: blk.64.nextn.*, `nextn_predict_layers=1`, and DRAFTED with it) |
-| Projector | `…/Swift15-UncIQ3XXS/mmproj-BF16.gguf` — PUBLISHER-SHIPPED in the same repo, 931,145,888 B, sha256 `19acd7fc…` (27-layer ViT → 5120) |
-| Fetch | [`scripts/download-frank-zealot.sh`](../../scripts/download-frank-zealot.sh) — `-f`, byte count + sha256 verified per file (Xet repo: the Hub APIs expose no LFS oid, so the shas were computed locally after the first verified fetch; byte counts match the Hub's declared sizes) |
-| Seam | `tests/frank_zealot_unit.bats` (argv assertions against a stub engine: payload path, slot geometry, KV pair, spec pair (both flags present), batch pair, projector opt-out) |
-
-## Why this payload (2026-10-06 evening)
-
-Frank asked for the UNCELERSED Swift-1.5 with MTP back on. The RCO-IQ3_XXS
-is 1.56 GiB lighter than the IQ3_S-mtp it replaced — and that lighter
-weight is exactly what funds the MTP draft context (~2.4 GiB measured)
-PLUS the second 131,072 slot, while keeping the q8_0 KV contract and the
-two-slot shape Frank ordered that morning.
-
-**Uncensored, intentionally:** the publisher reports 23/100 refusals vs
-Swift 1.5's 98/100 (KL 0.0884 against the censored base). The refusal
-escape hatch is the same as every payload here: revert and redeploy.
-
-## Shape and KV contract
-
-`--ctx-size 229376 --parallel 2` — TWO conversations at Frank's 112k
-cap (2026-10-06 late: cut from the 128k the morning/evening shapes ran
-at, to fund ubatch 1024). `CTX_SIZE` stays divisible by `PARALLEL`; the
-seam enforces it, and the per-slot number (`229376 / 2 = 114,688`) is
-the product contract — models.yml follows at 114,688 (a longer prompt
-is a 400, never a truncation).
-
-**KV at FP8 class: `q8_0` K + `q8_0` V (Frank, 2026-10-05 evening).**
-llama.cpp has no true fp8 KV dtype; q8_0 is the 8-bit class (1.0625 B/elem
-⇒ 8.5 effective bits) and the only FP8-class lever this runtime moves.
-The order supersedes the same-day f16/f16 rule, which was written for the
-Ornith 9B fit — at 64 KiB/token f16 the single 131,072 slot alone would
-cost 8.0 GiB of KV. At q8_0 the rate is **34 KiB/token** (16 attn layers ×
-4 KV heads × 256 dim × 1.0625 B, K and again V). `--flash-attn on` is
-pinned: quantized V requires it. The era note claiming the build carried
-`GGML_CUDA_FA_ALL_QUANTS=ON` does not describe the binary on disk: its
-`CMakeCache.txt` reads `GGML_CUDA_FA_ALL_QUANTS:BOOL=OFF` (binary built
-2026-10-01 22:35) and the option is vestigial in that tree — `ggml/src/ggml-cuda/fattn.cu:438-449`
-dispatches the Q4/Q5 KV cases unconditionally.
-
-**Two slots + drafter, by arithmetic and by order (2026-10-06 morning /
-evening, the 2 × 131,072 era).** The morning order bought the second slot
-by spending the drafter and ubatch headroom (1,397 MiB free). The evening
-swap put the drafter BACK and changed the arithmetic: the draft ctx is
-built against the target's FULL context (262,144 — the draft window, not
-the 3-token proposal depth), so it costs **~2.4 GiB measured** (23,636 −
-22,728 + 1,561), not the ~0.4 GiB the one-slot era assumed. The lighter
-quant pays for it: **23,636 used / 489 MiB free** (nvidia-smi `free`
-field) — the tightest fit in frank-zealot's history, the 2–3 GiB floor
-knowingly violated, and the 512×512 vision transient (34 MiB) still fits
-and recovers.
-
-**The late reshape (2 × 114,688, ubatch 2048) is the current fit.** The
-per-slot cut frees 2 × 16,384 tokens × 34 KiB ≈ 1.05 GiB of KV; the
-ubatch rungs 512 → 1024 → 2048 spend it back on compute buffers (a
-prefill knob — decode never moves with it; the rungs cost ~0.3 and
-~0.75 GiB measured). Measured boot at 2048: **23,120 used / 1,005 MiB
-free**. The drafter's own KV shrank with its target (draft window
-229,376). Compute buffers allocate at graph capture during BOOT, so a
-miss is loud and fast — the failure mode is a crash loop, not silent
-corruption.
-
-## Speculative decoding — ON since 2026-10-06 evening (draft-mtp, depth 3)
-
-**Frank's evening order: MTP back on.** The MTP head ships IN the file —
-the `-mtp` GGUF carries `blk.64.nextn.*` (`qwen35.nextn_predict_layers=1`)
-— and `--spec-type draft-mtp` loads it against the main model's own
-weights (no `--model-draft`, no second draft model — two context-backed
-drafters cannot share a process). `SPEC_DRAFT_N_MAX=3` is the proposal
-depth. **ngram-mod stays skipped, not swapped in:** on a qwen35 hybrid,
-ngram speculation logs *"recurrent state rollback is not compatible with
-'ngram-mod' — disabling rollback support"* (fleet lesson from
-frank-dragoon's chain measurement, 2026-10-03). A rejected draft on Gated
-DeltaNet layers cannot be rewound by KV truncation — without the rollback
-checkpoints the DeltaNet state carries phantom transitions and output
-degrades silently. The flags move as a set: the serve script emits BOTH
-`--spec-type` and `--spec-draft-n-max` together or neither
-(`SPEC_TYPE=none` strips both).
-
-## Vision contract
-
-`--mmproj` is the entire vision path (the GGUF carries no vision tensors).
-Probes: a solid-color PNG **≥ 256 px** answers its exact color — a tiny
-probe undersupplies image tokens and the model invents ("video"); the
-legacy top-level `images` field is IGNORED, only the OpenAI `image_url`
-content part reaches the model. **New on this payload: the uncensored
-model THINKS more than the censored Swift 1.5** — a 64-token budget was
-eaten entirely by `reasoning_content` (`finish_reason: length`, empty
-`content`); 128 tokens clears it. Token budgets for vision probes must
-leave room for thinking.
-
-## Measured: the 1 × 131,072 MTP boot (2026-10-05, historical)
-
-- **Boot ledger**: `creating MTP draft context against the target model
-  '…/Swift-1.5-Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf'`, `loaded multimodal
-  model, '…/Swift15-IQ3S/mmproj-Qwen3.8-27B-F16.gguf'`, `n_slots = 1,
-  n_ctx_slot = 131072` — every expected line, no q8→f16 fallback.
-- **VRAM: 19,404 used / 4,721 MiB free** (nvidia-smi `free` field). The
-  4.5 GiB prediction held; f16 KV would have left ~700 MiB, so this free
-  number doubles as the q8_0 proof.
-- **Decode + MTP acceptance** (draft counts from the live `timings`):
-  temp 0.8 → 66.1 tok/s, 182/349 accepted (52.1 %); temp 0 → 66.9 tok/s,
-  166/318; **deployed recipe (temp 1.0/top_p 0.95/top_k 20) → 58.9 tok/s,
-  159/355 (44.8 %)** — temperature buys fluency, costs acceptance and
-  rate. For scale: MiMo Q6_K measured 94.4 tok/s on the same card — this
-  27B at IQ3_S runs ~2/3 the rate with ~3× the capability class.
-- **Vision**: 512×512 solid-red PNG → `Red`, `finish_reason: stop`.
-- **Template gate**: `reasoning_effort: xhigh` → 200 with
-  `reasoning_content` populated; `high` → HTTP 500 (Jinja raise) — the
-  models.yml fold is mandatory, exactly as the MiMo eras found.
-- Streaming completion: 200. No first-request silence window (the
-  halogen-era 45–60 s stall has no analogue here).
-
-## Measured: the 2 × 131,072 no-spec boot (2026-10-06 morning, historical)
-
-- **Boot ledger**: `loaded multimodal model, '…/Swift15-IQ3S/
-  mmproj-Qwen3.8-27B-F16.gguf'` and NO draft-context line (speculation
-  off). `/slots`: **2 slots, n_ctx 131,072 each** (pool 262,144).
-- **VRAM: 22,728 used / 1,397 MiB free** (nvidia-smi `free` field) — the
-  predicted 0.3–1.3 GiB window held; the old 2-3 GiB floor is knowingly
-  violated by Frank's order. After a 512×512 image probe: 1,363 MiB —
-  the vision transients fit.
-- **Decode without the drafter**: 48.0 tok/s (temp 0) / 47.8 (deployed
-  recipe temp 1.0). `draft_n` absent from `timings` (no drafter). The MTP
-  era's 58.9–66.9 tok/s is the honest price of the second context:
-  −18–28 %.
-- **Vision**: 512×512 solid-red PNG → `Red` again on this shape.
-- Deploy: commit 812d9eb, e2e gate 33/33 green on the live fleet.
-
-## Measured: the 2 × 131,072 draft-mtp boot (2026-10-06 evening, historical)
-
-- **Boot ledger**: `creating MTP draft context against the target model
-  '…/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-RCO-IQ3_XXS.gguf'`,
-  `loaded multimodal model, '…/Swift15-UncIQ3XXS/mmproj-BF16.gguf'`,
-  `n_slots = 2, n_ctx_slot = 131072` — every expected line, no q8→f16
-  fallback.
-- **VRAM: 23,636 used / 489 MiB free** (nvidia-smi `free` field) — the
-  draft ctx measured ~2.4 GiB, ~6× the ~0.4 GiB the one-slot era
-  assumed; the lighter quant (−1.56 GiB) plus the morning's no-spec base
-  still fund both slots. After a 512×512 image probe: 489 MiB — the vision
-  transient (34 MiB) fits and recovers.
-- **Decode + MTP acceptance** (draft counts from the live `timings`,
-  512-token essay): temp 0 → **72.9 tok/s**, 333/534 accepted (62.4 %);
-  deployed recipe (temp 1.0/top_p 0.95/top_k 20) → **63.3 tok/s**,
-  305/612 (49.8 %). e2e traffic in the boot window read 0.52–0.67
-  acceptance. Against the no-spec shape (48.0 tok/s): **+31 % at the
-  recipe temp** — the drafter earned its 2.4 GiB.
-- **Vision**: 512×512 solid-red PNG → `Red` at a 128-token budget (64
-  tokens: all thinking, empty content). `prompt_tokens: 321` confirms the
-  image part reached the model.
-- **Prompt cache**: `cached_tokens: 317` of 321 on the second probe — the
-  slot prefix cache is live.
-- Deploy: commit da36fa3, e2e gate 33/33 green on the live fleet.
-
-## Measured: the 2 × 114,688 / ubatch-1024 boot (2026-10-06 late, historical)
-
-- **argv** (from `/proc/<MainPID>/cmdline`): `--ctx-size 229376
-  --parallel 2` (per-slot 114,688), `--batch-size 4096 --ubatch-size
-  1024`, `--spec-type draft-mtp`. Deploys 87721ab (the rename — old
-  `frank-small.service` stopped + disabled first, by hand: `disable` on
-  a relinked unit removes only its symlink) then f2575a1 (the shape);
-  health gate + MainPID check green, e2e 32 passed / 1 flaky-retry.
-- **VRAM: 22,374 used / 1,751 MiB free** (nvidia-smi `free` field) —
-  the per-slot cut (≈1.05 GiB of KV) paid for the ubatch-1024 buffers
-  and netted back ~1.25 GiB against the evening's 489 MiB; back above
-  the morning's 1,397 MiB.
-- **Decode + MTP acceptance** (live `timings`): recipe (temp 1.0) →
-  **58.2 t/s** (430/1005, 42.8 %); temp 0 → **66.9 t/s** (475/873,
-  54.4 %). The 2 × 131,072 boot measured 63.3 / 72.9 — −8–10 %, but
-  acceptance is text-dependent and these prompts differ from that
-  measurement's, so the comparison is CONFOUNDED, not a shape verdict;
-  a clean A/B needs stop + mask (bench policy).
-- **Smoke**: `READY` completion + `reasoning_content` populated; the
-  harness reached :7771 under the new alias (models.yml provider/id
-  `frank-zealot`, config.yml modelRoles rewritten; per-slot
-  `contextWindow: 114688`).
-
-## Measured: ubatch 2048 (2026-10-06 midnight, live)
-
-- **argv**: `--ctx-size 229376 --parallel 2 --batch-size 4096
-  --ubatch-size 2048 --spec-type draft-mtp` (commit d8da7a9, Frank's
-  "set ubatch to 2k"; deploy green, e2e gate 33/33).
-- **VRAM: 23,120 used / 1,005 MiB free at boot** — the 1024 → 2048 rung
-  cost **746 MiB**, ~130 over the 616 MiB the rung measured on
-  2026-10-05 (rung cost is shape-dependent, not a constant). The
-  512×512 solid-red vision probe (`Red`) then took ~240 MiB **and kept
-  it**: settled **763 MiB free** — on this boot the vision encoder's
-  buffers stay resident after the request (the evening boot saw its
-  34 MiB transient return).
-- **Decode (ubatch-invariant knob, same prompt as the 1024 rung)**:
-  recipe → 52.0 t/s (388/1133, 34.2 %) vs 58.2 t/s / 42.8 % at 1024 —
-  acceptance variance at temp 1.0, not a ubatch effect.
-- **Prefill (THE ubatch knob)**: **1,209 t/s @ 3,760 tokens** — first
-  recorded number for this shape; future rungs compare against this
-  (no same-prompt 1024 baseline exists; the law predicts ≈ +4.6 %
-  1024 → 2048).
-- **Headroom left ~0.7–1.0 GiB.** ubatch 4096 (= the logical batch, no
-  micro-splitting) doubles the buffer growth again — not affordable;
-  **2048 is the ceiling for this shape.**
-
-## Client contract
-
-- Reasoning in `reasoning_content` (`--reasoning-format deepseek`);
-  thinking tokens COUNT against `max_tokens`. **This payload thinks more
-  than the censored Swift 1.5** (uncensored finetune) — short `max_tokens`
-  budgets can be eaten entirely by thinking (`finish_reason: length`,
-  empty content); the 32,768 `maxTokens` in models.yml is comfortably
-  above.
-- The embedded chat template gates `reasoning_effort` to
-  `{xhigh, medium, low}` — anything else is a raise/HTTP 500, so
-  `models.yml` `compat.reasoningEffortMap` folds the harness levels onto
-  that set (mandatory, same as every Qwen3.8-template endpoint).
-- Sampling rides the wire (`extraBody`): the publisher recipe temp 1.0 /
-  top_p 0.95 / top_k 20 / min_p 0 (evaluated at ctx 131,072).
-- The RCO-IQ3_XXS tier is a **per-tensor allocation mix** (IQ3_S 97 /
-  BF16 96 / IQ3_XXS 83 / IQ2_S 72 / IQ4_XS 33 / IQ2_XS 32 / IQ2_XXS 30 /
-  Q2_K 28 / Q4_K 19 / Q6_K 8 / IQ1_S 4 / IQ1_M 4) — all stock quant types;
-  stock llama.cpp loads it. Publisher KLD vs Swift BF16: mean 0.002631,
-  90th 0.042136, max 34.49 — RCO is the publisher's "more experimental"
-  family.
-
-## Rollback
-
-**Predecessor shape (same 2 × 114,688, ubatch 1024) is ONE revert
-away** — revert d8da7a9 and redeploy. The 2 × 131,072 / ubatch-512
-era is one revert further (f2575a1). The 2 × 131,072 no-spec
-IQ3_S-mtp shape is one revert further — revert da36fa3 next: the env
-trio then ships `SPEC_TYPE=none`, `CTX_SIZE=262144`, `PARALLEL=2`,
-`UBATCH_SIZE=512` against `/home/dank/models/Swift15-IQ3S/` (the
-IQ3_S-mtp payload, still on disk, measured 48.0 tok/s no-spec). The
-1 × 131,072 MTP shape is two reverts further back (812d9eb's parent):
-the same IQ3_S-mtp file with `SPEC_TYPE=draft-mtp`, `UBATCH_SIZE=1024`,
-`CTX_SIZE=131072`, `PARALLEL=1` (measured 58.9–66.9 tok/s with the head
-drafting).
-
-Three payloads on disk, all fetchable by the same script's git-history
-recipes:
-
-1. **Swift-1.5 IQ3_S-mtp** — the payload live until this cutover:
-   `/home/dank/models/Swift15-IQ3S/`; the IQ3_XXS swap above.
-2. **MiMo-V2.6 Q6_K 2×200,704 f16 KV** — the payload that was LIVE until
-   the 2026-10-05 cutover: `/home/dank/models/MiMo-Q6K/`; revert
-   [[frank-zealot-llama-mimo-q6k-2x200704]] (env trio at commit e5893dc or
-   380aafa's parent), one `MODEL_FILE` + shape edit.
-3. **Ornith-1.5-9B Q8_0** — the committed-but-never-deployed swap:
-   `/home/dank/models/Ornith-Q8/`; revert to 380aafa's tree (ONE native
-   262,144 slot, f16/f16 KV).
-
-## Trail
-
-- The Ornith-1.5-9B Q8_0 swap (77decb5 → 380aafa, earlier 2026-10-05) was
-  committed and pushed but its deploy never completed (blocked at the
-  whole-tree docs gate); the live service was still MiMo Q6_K when Swift
-  cut over. Both Ornith payloads were verified on disk before Swift.
-- Swift header facts (block_count 65, interval 4, KV geometry, nextn) were
-  read from the Hub file by HTTP range + struct parse — no download needed
-  — before the payload was fetched.
-- Cutover 2026-10-05 ~23:26: the radiance-cutover session (parallel)
-  label-committed both sets at Frank's direction — 1919ad7 (frank-archon
-  radiance) then baf33d0 (this swap: trio, seam, runbooks, fleet docs).
-  Deploy ran `deploy.sh dankclaw --service frank-zealot`; the service
-  dispatch and health gate went green; the first e2e run failed ONE spec
-  (12-crawl, `net::ERR_ABORTED` on `www.franksdank.cloud` — a transient,
-  unrelated to frank-zealot), green on rerun.
-- The unit's `Description=` (MiMo era) was stale at cutover time and was
-  updated in-repo afterward — the relinked unit picks it up at the next
-  deploy/restart, not retroactively.
-- 2026-10-06 (Frank): drop MTP, skip ngram, ubatch 512, TWO 128k contexts
-  (812d9eb). The tight fit landed: 1,397 MiB free, vision `Red`, decode
-  48.0 t/s no-spec. Model of the trade: the drafter's ~0.4 GiB + ubatch
-  ~0.3 GiB bought the second 4,352 MiB KV slot.
-- 2026-10-06 (Frank, evening): swap to the UNCELERSED Swift-1.5 at
-  RCO-IQ3_XXS, MTP back on (da36fa3). The lighter quant (−1.56 GiB) funds
-  the draft ctx (~2.4 GiB measured — the drafter's KV scales with the
-  target's full 262,144 context, not the 3-token proposal depth) plus the
-  second 131,072 slot: 489 MiB free, vision `Red` at a 128-token budget,
-  decode 63.3 t/s recipe / 72.9 temp 0 (48.0 no-spec).
-- 2026-10-06 (late, Frank): the endpoint renamed frank-small →
-  **frank-secondary** (87721ab — "rename frank-small → frank-secondary", scripts, unit, seam, runbooks,
-  identifiers; historical references to the OLD FreeToken/CUDA
-  frank-small, and to the Dankserver service that became frank-medium,
-  stay frank-small) and the per-slot window cut 131,072 → 114,688 to
-  fund ubatch 512 → 1024 (f2575a1). Live unit transition done by hand
-  before the deploy: `systemctl stop frank-small` + `systemctl disable
-  frank-small`, then `deploy.sh dankclaw --service frank-zealot`.
-  1,751 MiB free; decode 58.2 t/s recipe / 66.9 temp 0; models.yml +
-  config.yml retargeted.
-- 2026-10-06 (midnight, Frank): "set ubatch to 2k" → `UBATCH_SIZE=2048`
-  (d8da7a9). Cost 746 MiB: 23,120 used / 1,005 free at boot, settling
-  **763 MiB free after a vision probe** (encoder buffers stay resident
-  on this boot). Decode 52.0 t/s recipe on the shared prompt (58.2 at
-  the 1024 rung — acceptance noise, decode is ubatch-invariant);
-  prefill 1,209 t/s @ 3,760 tokens (first recorded baseline). 2048 is
-  the ceiling for this shape; 4096 would not fit.

serverAppConfigs-docs/LLM/frank-zealot-qwen38-gsq-rco.md

@@ -1,22 +0,0 @@
----
-type: reference
-title: "frank-zealot on Dankclaw — llama.cpp, Qwen3.8-27B GSQ-RCO IQ3_S"
-description: "The RETIRED frank-zealot runbook (2026-09-16 → 09-17) — mainline llama.cpp on Dankclaw's RTX 3090 serving ISTA-DASLab Qwen3.8-27B-GSQ-RCO at IQ3_S with the model's own in-file MTP head and the BF16 vision projector: identity, KV arithmetic, deployed flags, the MTP contract, the client contract, and what to check when it misbehaves."
-tags:
-  - llm
-  - frank-zealot
-  - dankclaw
-  - llama.cpp
-  - rtx3090
-  - qwen35
-  - mtp
-  - vision
-timestamp: 2026-09-16
----
-
-**RETIRED 2026-09-16 → 2026-09-17** (AGENTS.md Runbook Index); superseded by
-[[frank-zealot-gemma4-12b]], and :7771 today runs
-[[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork build 10754,
-`Ternary-Bonsai-2-27B-PQ2_0.gguf`). The payload is **not** a rollback target any
-more: the 2026-10-18 census of `/home/dank/models/` has no GSQ-RCO directory, so
-restoring this shape means re-downloading from ISTA-DASLab.