frank-primary: digest-pin FreeToken base image, symlink libhiprtc dev name

- FROM pinned to sha256:a223aee1... (the pulled rocm/pytorch rocm7.14
  pytorch 2.11.0 manifest); a re-pushed tag cannot change the stack.
- The pip-ROCm torch ships libhiprtc.so.7 only; the PR's _cpu_moe links
  -lhiprtc and ld needs the unversioned symlink in the already-linked
  _rocm_sdk_core/lib (first build died: cannot find -lhiprtc).

.gitignore

@@ -37,3 +37,4 @@ __pycache__/
 # Editor/OS noise
 *.swp
 .DS_Store
+third_party/

AGENTS.md

@@ -9,11 +9,11 @@ services run from `/home/dank/serverAppConfigs` on each host (the 2026-08-22
 merge absorbed the former `vllmScripts` repo, history preserved).
 
 - **Dankserver** (192.168.0.235) — media/library stack (RomM, Copyparty,
-  Calibre-Web, Jellyfin, Plex) plus the `frank-medium` LLM service.
+  Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service.
 - **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
   via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
-  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (halogen-flash-server on
-  the gfx1151 iGPU) and `frank-small` (llama.cpp on the RTX 3090). Caddy on
+  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (halogen container on
+  the gfx1151 iGPU) and `frank-small` (llama.cpp CUDA on the RTX 3090). Caddy on
   Dankclaw reverse-proxies both hosts.
 - **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
   (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) as quadlets inside
@@ -24,9 +24,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
-| Dankserver | `frank-medium` | mainline llama.cpp master fb27a525 (root systemd unit, gfx1201 HIP build) | ajgazin Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP UD-Q8_K_XL + z-lab Qwen3.8-27B-DFlash2 Q8_0 drafter, 2x Radeon AI PRO R9700 in ONE layer-split process (35:30), 262144 ctx over 2 slots, batch 6144x512, text only | 7770 |
-| Dankclaw | `frank-large` | halogen-flash-server 0.5.8 (rootless Quadlet) | peonist-ai Qwen3.8-Flash-Next w4b `.hgn` on the gfx1151 iGPU (Strix Halo LPDDR), 200192 ctx per conversation over one 400384-position KV pool / 2 slots, text only | 7770 |
-| Dankclaw | `frank-small` | mainline llama.cpp master 37b53fd (root systemd unit) | ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`-mtp` + BF16 mmproj) on the RTX 3090, 131072 ctx / 1 slot, MTP speculative decoding, the fleet's vision endpoint | 7771 |
+| Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **fp8 KV** (`--kv-cache-dtype fp8`), **no speculative decode**, MNBT 2048 (arena-free: the registry prices overhead flat up to its nbt 4096 anchor), MoE gather 32×64, **`--expert-precise-memory`** (fitted 5.25 GiB overhead, not the 7.60 GiB default), **`VLLM_CUDAGRAPH_MEM_BASE_MB=0` / `_PER_GRAPH_MB=12`** (the plan's CUDA-graph term priced at the measured capture — 0.31 GiB then a NEGATIVE −0.06 GiB delta, so the 256 MiB analytic base was fiction → graph term 0.07 GiB), **`GPU_MAX_HW_QUEUES=2`**, **mounted cache dir** at `/root/.cache` (the image ships no `/root/.cache` at all; unmounted, every boot re-pays `torch.compile took 181.20 s`), **3 slots × 200704 ctx** at KV_BYTES 3,166,500,000/rank (~400,700 KV tokens at the engine's 7,891.8 B/token here) — the **pool, not the slot count, caps resident conversations** (measured live: `GPU KV cache size: 401,408 tokens, Maximum concurrency for 200,704 tokens per request: 2.00×`; the 3.15e9 pool below it measured 398,658/1.99×), and shorter requests use the spare blocks. **`--max-num-seqs` must stay 3: SEQS=2 cannot boot this model** — torch.fx `ConstraintViolationError` on `query_start_loc`, measured at BOTH MNBT 1024 and 2048. **util stays 0.982 BY LAW**: at `le=1` the worker init refuses `budget == total` with `ValueError: Free memory on device cuda:N (31.29/31.86 GiB)` on both ranks before a single weight loads — measured crash loop 2026-10-03. Arena at shape G: **17.77 GiB / 7,762 slots measured**, KV 401,408 tokens / 2.00× concurrency, 0.89 GiB of real free VRAM per card held through a live 4,096-token generation (the ~3.5 GiB of driver-reported free VRAM at shape E was estimator slack — the registry's 9190 B/token KV reserve vs the real 7,892, and the 5.25 GiB overhead vs far less — NOT allocatable), and the client contract wired: `--reasoning-parser …
+| Dankclaw | `frank-large` | **halogen container** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | `peonist-ai/halogen-qwen3.8-27b` **`qwen3.8-27b-p1w4d-d2.hgn`** (35,865,565,184 B; the `.hgn` format loads in NO other engine — llama.cpp, vLLM and transformers all refuse it), **DFlash2 spec decode (`HALOGEN_DRAFTER=2`)**, **1 slot × 200,700 ctx** — `HALOGEN_KV_SLOTS=1` is load-bearing: batching XOR speculation, slots>1 silently disables the drafter (measured: 8 slots at serial ~6 t/s/stream), d2 KV (2× fp8, 64 KiB/token → 12.2 GiB pool, engine sizes it and prints a boot ledger naming any clamp), **TEXT-ONLY** (the engine ignores the vision encoder — vision is frank-small's job), **censored base** (the ROCmFPX AGENT rollback is the refusal escape hatch), **no server-side sampler: absent temperature is GREEDY**, so the card's thinking recipe (temp 1.0 / top_p 0.95) rides the wire in `models.yml`. Publisher card 31.7 t/s mean across eight text workloads vs the retired ROCmFPX endpoint's measured 27.6–28.1. Prompt over the slot AND `max_tokens` over `HALOGEN_MAX_TOKENS_CAP` are both 400s naming the limit, never truncations. Serve passes NO `--ulimit nofile` (rootless runc cannot raise Dankclaw's 524288 hard limit — see Key Lessons). Cutover 2026-10-05|7770|
+| Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** (9B dense qwen3_5 hybrid, 7.26 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **2 × 200,704** (pool 401,408 — NOT the native window) with **fully unquantized KV (f16 K + f16 V)** under flash attention, **batch 4096 / ubatch 2048** (doubled 2026-10-05: measured +4.6 % prefill at 8,928 tokens for 616 MiB), 21,846 used / **2,279 free** of 24,576 MiB, 94.4 tok/s decode. Resized 2026-10-03 (3 × 180,224 + q8_0 V → 2 × 200,704 + f16 V → + Q6_K rung: the freed slots paid for exact values AND the higher quant; the rung costs a measured −10 % decode — weight-bandwidth-bound); rollback `MODEL_FILE=…/MiMo-Q5KM/…Q5_K_M.gguf` (buys the 10 % back) or `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (Q5 only) | 7771 |
 
 Retired endpoints (`frank-npu`, and every prior runtime) are in
 the [runbook index](#runbook-index). The full swap history lives in
@@ -61,18 +61,27 @@ All edits flow through git. Never edit files on a remote machine directly.
 ./scripts/deploy.sh dankclaw
 ./scripts/deploy.sh dankserver
 ./scripts/deploy.sh dankclaw --service searxng
-./scripts/deploy.sh dankserver --service frank-medium
+./scripts/deploy.sh dankserver --service frank-primary
 ./scripts/deploy.sh dankclaw --dry-run
 ```
 
 The planner maps changed paths to services; the remote runner restarts only
 the selected service and records its revision outside the checkout.
 Documentation-only changes are a no-op. LLM services are selected only by
-their own files — frank-large by `dankclaw/frank-large/quadlet/`, frank-small
-and frank-medium by their unit/env/serve trio in `scripts/` — so unrelated
+their own files — every endpoint (`frank-large`/`frank-small`/`frank-primary`)
+by its unit/env/serve trio in `scripts/` — so
+unrelated
 container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
 runs on a host without passwordless sudo; the value never enters the repo.
 
+**`deploy.sh <target>` with no `--service` is a fleet-wide DRAIN, not a sync.**
+It also walks every per-service revision marker under the deploy-state dir and
+restarts each service whose marker is behind HEAD *and* whose paths changed
+since that marker — so a backlog accumulated by service-scoped deploys fires
+all at once, and one failing health gate aborts the run with every later
+service left pending. Name the service unless a drain is what you want
+(`./scripts/deploy.sh dankserver --service frank-primary`).
+
 ### Deployment completeness (MANDATORY)
 
 A deploy is **not complete** when `deploy.sh` exits green. It requires:
@@ -81,7 +90,14 @@ A deploy is **not complete** when `deploy.sh` exits green. It requires:
 2. the Playwright e2e suite (`tests/e2e/run.sh`) — runs **automatically** as
    the `[3/3]` gate at the end of `deploy.sh` against the LIVE fleet; a
    behavior-affecting deploy that fails it exits `DEPLOY INCOMPLETE`. Bypass
-   only with `--skip-e2e` in an emergency.
+   only with `--skip-e2e` in an emergency. The gate runs the conditional
+   `frankbook` Playwright project (the `*arr`/qBittorrent walk,
+   `tests/e2e/specs/19-frankbook.spec.ts`) ONLY when a frankbook blast-radius
+   path changed since origin/main — that spec, `frankbook-pro/**`,
+   `shared/homepage/services.json`, or the `dankclaw/Caddyfile` serving the
+   arr vhosts — so a sleeping FrankBook-Pro never fails an unrelated deploy.
+   The default project set excludes it everywhere else (`E2E_INCLUDE_FRANKBOOK=1`
+   opts in; see `serverAppConfigs-docs/Architecture/E2E Suite.md`).
 
 ```mermaid
 graph LR
@@ -122,14 +138,18 @@ This rule exists because reboots found gaps twice (2026-08-26: Caddy as a
 bare compose container never returned; 2026-09-11: five observability
 services had no units). The observability services now have units under
 `dankclaw/observability/quadlet/` and their compose definitions were removed —
-one owner per container name. The stack is **configured as of 2026-09-16**: a
-Grafana datasource and the `llm-fleet` dashboard are provisioned from
-`shared/grafana/provisioning/`, and `shared/prometheus/rules/llm-fleet.yml`
-loads 5 alert rules + 1 recording rule. Nothing is *delivered* — there is no
-Alertmanager, so a firing alert reaches nobody (an operator decision, not an
-oversight). Five Prometheus targets read down for structural reasons
-(HTML-scraped "probes", Gitea metrics off); the remedy is a blackbox exporter.
-See `serverAppConfigs-docs/Infrastructure/Observability.md`.
+one owner per container name. The stack is **configured and delivering as of
+2026-09-17**: a Grafana datasource and the `llm-fleet` dashboard are
+provisioned from `shared/grafana/provisioning/`, `shared/prometheus/rules/
+llm-fleet.yml` loads 5 alert rules + 1 recording rule, and firing rules are
+delivered: Prometheus → Alertmanager (`dankclaw/observability/quadlet/
+alertmanager.container`, loopback :9093) → host-side Signal relay
+(`scripts/signal-relay.py`, user unit, 192.168.0.143:18093 — the host LAN
+IP, the one bind a container can reach as `host.containers.internal`;
+iptables admits loopback, the host and the podman bridge subnet only) → signal-cli account 318311.
+Prometheus targets read down for structural reasons (HTML-scraped "probes",
+Gitea metrics off); the remedy is a blackbox exporter. See
+`serverAppConfigs-docs/Infrastructure/Observability.md`.
 
 ## Testing
 
@@ -203,14 +223,27 @@ archive.
 
 **STOP the production service BEFORE benchmarking** — these servers serve
 real users, and a bench beside a live service starves the GPU and corrupts
-results. Use the runtime's own harness: `llama-bench` (llama.cpp), `vllm
-bench` (vLLM), and halogen-flash-server's own `bench`/`sweep` modes for
-frank-large, which is the active runtime there again. Rolling your own bench
+results. Use the runtime's own harness: `llama-bench` for the llama.cpp
+endpoints; a live llama.cpp server also reports honest per-request numbers in the `/completion` `timings` object (prefill/decode rates, `draft_n`, `draft_n_accepted`), which is how frank-large and frank-small were measured. Rolling your own bench
 requires asking first.
 Scratch ports are not used on LLM hosts — a
 test process cannot share the single GPU anyway; validate against the service
 itself.
 
+(A gufo-era law; frank-large ran ROCmFPX llama.cpp 2026-10-02 → 10-05 and is the halogen container since, but the rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode settings, and will invert the ranking.**
+Measured 2026-09-28 on frank-large at depth 32768: its corpus
+reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
+draft widths, i.e. the same highly-predictable text every run, and 93 % of
+every proposal accepted. It reported 29–37 t/s decode where real traffic
+gives ~15 t/s, and it made `--draft-tokens 16` look 6 % faster than 7 when
+real text made it 14 % *slower*. Use it for shape, not for A/B. For an A/B,
+replay **real** text through the live server and read the `Server-Timing`
+header — but note the unit rides in the **name** half (`ttft;dur=402.485`),
+so splitting on `=` alone keys the dict `ttft;dur` and every lookup silently
+returns nothing. Generate **at least 128 tokens** for the measurement: at 8
+tokens, `inter_token` is first-token noise and the run-to-run spread swamps
+every real difference.
+
 This covers **any GPU experiment, not just benchmarks** — including a bare
 2-rank RCCL/`torch.distributed` probe. On 2026-09-16 one such probe was run
 against the live service "because it only needs 200 MB of VRAM"; the host went
@@ -223,16 +256,17 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
 merely wedged kernel usually still answers ARP.
 
 1. Stop and mask (self-healing would restart it):
-   - Dankclaw `frank-large` (rootless Quadlet again since 2026-09-16):
-     `systemctl --user stop frank-large`, then move its
-     `~/.config/containers/systemd/frank-large.container` aside and
-     `systemctl --user daemon-reload` — the quadlet generator re-creates that
-     unit from the file on every reload, so the source is the lever, not `mask`.
+   - Dankclaw `frank-large` (root unit, halogen podman on the iGPU):
+     `sudo systemctl stop frank-large && sudo systemctl mask frank-large`
    - Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
      `sudo systemctl stop frank-small && sudo systemctl mask frank-small`
-   - Dankserver `frank-medium` (system unit):
-     `sudo systemctl stop frank-medium && sudo systemctl mask frank-medium`
+   - Dankserver `frank-primary` (root unit, tcclaviger vLLM on the 2x R9700):
+     `sudo systemctl stop frank-primary && sudo systemctl mask frank-primary`
 2. Bench.
+   NOTE: `systemctl mask` FAILS on a relinked unit (the unit file in
+   `/etc/systemd/system/` is itself a symlink into the checkout — mask
+   refuses to replace it). Stop is enough for the bench window; the deploy
+   restores the unit, so re-deploy after benching instead of unmasking.
 3. Unmask and start; verify `curl http://192.168.0.<host>:7770/health`.
 
 **CRITICAL:** never `systemctl disable` a system unit on these hosts — it
@@ -244,9 +278,8 @@ deletes the unit file from `/etc/systemd/system/`, not just the symlink.
 The harness reaches the fleet through `.omp/agent/models.yml` in the
 operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
 where the harness runs; there is no copy on the servers. Each provider is one
-HOST:PORT; each model `id` is the served alias: `frank-medium` →
+HOST:PORT; each model `id` is the served alias: `frank-primary` →
 `192.168.0.235:7770`, `frank-large` → `192.168.0.143:7770`, `frank-small` →
-`192.168.0.143:7771`. Per-role routing lives beside it in `config.yml` under
 `modelRoles`.
 
 Resolution rules: the default model resolves as a `defaultProvider` +
@@ -259,102 +292,274 @@ When renaming a service, retiring an endpoint, or moving a model: update the
 provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
 `models.yml`, then fix every `modelRoles` entry that named the old endpoint —
 a role pointing at a provider the file does not define fails at call time. As
-of 2026-09-16 the image path is `frank-small` (`input: [text, image]`), and
-`frank-large` is text-only because its halogen deployment loads no vision
-tower. There is **no audio** input anywhere in the fleet: that capability left
-with the FLM NPU endpoint on 2026-09-11.
+of 2026-10-05 **two of the three endpoints accept images** (`input: [text,
+image]`) — `frank-large` went TEXT-ONLY with the halogen swap (the engine
+ignores the vision encoder; the BF16-mmproj era ended 2026-10-05) — while
+`frank-primary` takes them through a BF16 mmproj loaded explicitly (this build never
+autoloads projectors), and `frank-small` through
+`mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf` passed as `--mmproj` — the MiMo
+GGUF itself carries no vision tensors. `frank-small`'s `contextWindow` is
+**200704** (resliced from 180224 on 2026-10-03), the per-slot window, not the
+model's native 262144: the harness must compact before llama.cpp answers a
+longer prompt with a 400. Note also
+that
+the reasoning channel differs by runtime: llama.cpp AND SGLang (qwen3
+parser, live-verified) emit `reasoning_content`, vLLM emits `reasoning` —
+models.yml sets `reasoningContentField` per endpoint to match.
+There is **no audio**
+input anywhere in the fleet: that capability left with the FLM NPU
+endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
+audio encoder, and left again with the Gemma4 retirement on 2026-10-01
+(the MiMo tower is vision-only).
 
-### frank-large — halogen-flash-server (Dankclaw, ACTIVE)
+### frank-large — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE)
 
-halogen-flash-server 0.5.8 as a **rootless Quadlet again since 2026-09-16**
-(`dankclaw/frank-large/quadlet/frank-large.container`), serving
-`peonist-ai/halogen-qwen3.8-flash-next` (115.55 GiB w4b `.hgn` checkpoint + a
-quality overlay that re-fetched to 2.40 GiB on 2026-09-16) on port 7770 from
-the gfx1151 iGPU. Two conversations of 200,192 tokens resident (pool 400,384,
-2 slots), greedy default, reasoning extracted to `reasoning_content`, and **no
-vision tower** — that moved to frank-small. It replaced ik_llama.cpp, which had
-replaced it for two days.
+The **halogen engine** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`,
+digest-pinned, published tag 0.1.4) as a podman container driven by
+`scripts/serve-frank-large.sh` under a **root systemd unit**
+(`scripts/frank-large.service`, `User=dank`, `Restart=always`, `ExecStop=podman
+stop -t 120 frank-large-halogen`, the script `exec podman wait` so the restart
+policy is real) on port 7770 from the Strix Halo iGPU. Engine stdout goes to
+`/home/dank/logs/frank_large.log` — **the boot ledger (KV pool, drafter state,
+any clamp) lives there, not in the journal**. Serving
+`peonist-ai/halogen-qwen3.8-27b`: `qwen3.8-27b-p1w4d-d2.hgn` (35,865,565,184 B
+≈ 33.4 GiB, sha256 `274c3fc7…`, 1352 tensors; p1 weights / w4 activations /
+d2 KV) + the flat `tokenizer/` directory (5 files). Cutover 2026-10-05 from the ROCmFPX llama.cpp fork
+([[frank-large-rocmfpx-q6agent-uncensored-mtp]], measured 27.6–28.1 t/s prose),
+chosen for the publisher's 31.7 t/s DFlash2 mean.
 
-The engine **fetches its own weights on first start** (`HALOGEN_DOWNLOAD`, into
-the `/home/dank/models/halogen` bind mount): clearing that directory and
-restarting the unit is the supported way to pick up upstream updates, and it
-needs the directory to exist **empty** — a missing directory fails the
-container with `Error: statfs /home/dank/models/halogen: no such file or
-directory` and the unit loops on `Restart=on-failure` until `mkdir -p` brings
-it back. The sidecars are versioned in place, so a stale copy is invisible from
-the file name (both overlays were ~95 MB larger upstream than the copies this
-host had been serving).
+**The format is the lock.** `.hgn` loads in nothing else — the runtime is
+chosen by the FILE, as ROCmFPX's tensor types were before it. This is NOT the
+2026-09 `halogen-flash-server` image (different repo, Flash-Next w4b era); the
+halogen-server repo restarted versioning at 0.1.x.
 
-Measured boot ledger (2026-09-16, `HALOGEN_VERBOSE=1` ships in the unit):
-device budget `121 − 67.7 − 20 = 33.5 GiB` against a need of `~27.2 GiB`, so
-the configured pool was **not clamped**; 68.0 GiB of weights locked + 11.0 GiB
-of KV pool + 16.5 GiB of working memory = 95.4 GiB in all, leaving 14.7 GiB
-(475 contiguous 2 MiB blocks) for everything else: frank-small on the 3090 and
-~28 co-tenant containers. The engine printed its co-tenant stall NOTE and
-logged 4,350 compaction stalls (285 failed) while reserving the pool, so read
-that as "tight and fragmented", not "out of memory".
+**Two product regressions, both accepted by Frank before deploy:** the
+endpoint is **text-only** (README honest limits: the vision encoder is
+ignored — frank-small owns images now) and the checkpoint is a finetune of the
+**censored** `Qwen/Qwen3.8-27B` base (refusals can return; the rollback is one
+commit-revert away with the AGENT payload still on disk).
 
-Runbook: `serverAppConfigs-docs/LLM/halogen-flash-server.md`.
-Rollback: [[frank-large-ik-uncensored]].
+**Engine laws that shape the config** (all in `scripts/frank-large.env` with
+evidence): `HALOGEN_KV_SLOTS=1` — batch XOR speculation, slots>1 turns the
+drafter off silently. `HALOGEN_SLOT_CTX=200700` — the per-conversation HARD
+ceiling (400 naming the limit, never truncation); `models.yml`
+`contextWindow: 200700` must match or the harness discovers the wall as a 400.
+`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled
+pair; the harness `maxTokens: 57344` stays under the cap (over-cap is a 400
+naming the limit — README, never a 413). Thinking tokens COUNT against the
+budget: exhausted mid-thought is `finish_reason: "length"` with empty
+`content` and the reasoning in `reasoning_content`.
+`HALOGEN_DRAFTER=2` = DFlash2. **No server-side sampler**: an absent
+temperature is greedy, so the thinking recipe exists only in `models.yml`
+`extraBody`. `response_format` is unsupported (no constrained decoding).
+Thinking lands in **`reasoning_content`** (harness field unchanged); the same
+Qwen3.8 template rejects `reasoning_effort` outside `xhigh|medium|low`, so the
+`models.yml` fold stays mandatory.
+
+Fetch: `scripts/download-frank-large-halogen.sh` into
+`/home/dank/models/halogen-qwen3.8-27b/` (`-f` + byte count + LFS sha256 +
+`--verify-only`, no token on the command line; resume goes against the
+RESOLVED CDN URL because curl drops `Range` across HF's 302). The deployer
+and serve script both refuse a missing tokenizer file or **any symlink** —
+an HF cache snapshot symlinks each entry into `blobs/` and those links
+dangle inside the container mount.
+
+Runbook: `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md`.
+Rollback: revert the swap commit and redeploy — the ROCmFPX payload
+(`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`), the fork build
+and `scripts/build-rocmfpx.sh` (pin `36e9acd`) all remain on disk, seam-tested.
+The older gufo tag `frank-large-gufo-q4kxl-20261001` stays a **re-download**
+rollback (payloads removed 2026-10-03; `scripts/build-gufo.sh` stays).
 
 ### frank-small — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
 
-Mainline llama.cpp master `37b53fd` as a **root systemd unit** (repo
-`scripts/frank-small.service`, `relink_unit`ed into `/etc/systemd/system/`),
-serving `ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF` at IQ3_S — the `-mtp` build, so
-the model's own Multi-Token Prediction head drives speculative decoding — with
-the BF16 mmproj, on port 7771, 131,072 ctx / 1 slot, f16 KV, greedy default.
-Deployed flags are in `scripts/frank-small.env`; the serve script refuses to
-start if the weights, the projector or the binary are missing.
+**llama.cpp master** (CUDA sm_86 build
+`/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server`) as a
+**root systemd unit** (`scripts/frank-small.service`, `relink_unit`ed into
+`/etc/systemd/system/`), serving
+`bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF` at **Q6_K** (7,793,710,624 B,
+sha256 `ef96d05a…`) **plus `mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf`**
+(918,166,048 B, sha256 `ff348f…`), both `/home/dank/models/MiMo-Q6K/` (the
+projector is a hardlink of the Q5KM copy — the dir is self-contained), on
+port 7771. The rung went Q5_K_M → Q6_K on 2026-10-03 the same morning as the
+reslice, funded by the freed slots. The runtime arrived 2026-10-01 night for
+one reason:
+**asymmetric KV** — `--cache-type-k f16` + `--cache-type-v q8_0` is the fleet's
+only per-tensor KV lever, and SGLang's `--kv-cache-dtype` moves K and V
+together. On 2026-10-03 Frank spent the headroom that lever bought: the reslice
+from three slots to two made room for **fully unquantized V**
+(`CACHE_TYPE_V=f16`); both flags stay explicit in the serve script. Vision came
+2026-10-02 via `--mmproj`: the GGUF carries no `v.*` tensors, so the projector
+file is the entire vision path (and ~1.4 GiB of the card).
 
-Verified at deploy (2026-09-16): `/v1/models` reports `frank-small`; a chat
-completion returns content plus `reasoning_content`; a tool call returns
-`tool_calls`; the slot timing lines show `draft acceptance = 0.66–0.94, mean
-len ≈ 3` (MTP engaged); an image request is answered from the projector while
-the halogen side refuses images with a named reason; 22,362 MiB of the card's
-24,576 MiB is in use; `n_slots = 1, n_ctx_slot = 131072`.
+**Architecture: qwen3_5 hybrid** — 32 layers, 8 full-attention + 24 Gated
+DeltaNet, native 262,144 window. KV exists only in the 8 attention layers
+(4 KV heads × 256 dim × 2 B ≈ 2 KiB/token/layer, ~16 KiB/token for f16 K+V);
+the DeltaNet state is a fixed per-slot buffer, so slot count — not context —
+drives that term.
 
-### frank-medium — llama.cpp layer split, DFlash2 + ngram-mod (Dankserver, ACTIVE)
+**The shape (2026-10-03):** `--ctx-size 401408 --parallel 2` = **TWO
+conversations of 200,704 tokens each** (llama.cpp divides the pool:
+`n_ctx_slot = CTX_SIZE / PARALLEL`), `--flash-attn on` (the seam pins it),
+**f16 K + f16 V**, all layers on the GPU, `--mmproj` for vision, and since
+2026-10-05 **batch 4096 / ubatch 2048** (doubled; the boot-pair A/B is in the
+runbook). Measured trade at 8,928 prompt tokens: prefill 4,024 → **4,209 t/s,
++4.6 %**, decode flat (91.9–93.0 vs 93.8–94.3 — noise), and a 279-token prompt,
+which sits below either ubatch, did not move at all. The old "ubatch 2048
+failed the scratch boot" note belongs to the Q5_K_M predecessor and does NOT
+generalize: this shape boots 2048 clean. Measured VRAM **21,846 used / 2,279
+free** of 24,576 MiB against **21,276 / 2,849** for the 1024 arm, reproduced
+byte-identically the same hour — the doubling costs **616 MiB** and lands at
+the BOTTOM of the 2–3 GiB floor Frank asked for (`4096/1536` is the middle rung
+if that margin is ever wanted; non-KV 9,302 MiB, up from 8,732 by exactly the
+compute buffers; f16 K+V is 32 KiB/token across the 8 attention layers;
+quote `nvidia-smi`'s `free` field, never `total − used` — the driver holds
+~450 MiB of reserve the subtraction hides). The 2026-10-02 3 × 180,224 +
+q8_0-V shape stays one env edit away (`CTX_SIZE=540672 PARALLEL=3
+CACHE_TYPE_V=q8_0`, and it must pair with Q5_K_M — Q6_K breaks the floor
+there). Three native windows need 18.4 GiB of KV alone and were never on the
+table.
 
-Mainline llama.cpp at **`fb27a525`** (a gfx1201 HIP build, `Root=`-less root
-systemd unit `scripts/frank-medium.service`) serving
-`ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF` at **UD-Q8_K_XL**
-(29.29 GiB) with the **z-lab DFlash2** `Q8_0` drafter chained to the
-draftless **ngram-mod** matcher, in **one process across both R9700s**,
-`--split-mode layer` at a **35:30 tensor split** (the balance point for the
-deployed batch: the drafter rides ROCm1, so it takes fewer target layers),
-262144 ctx over **2 slots** of 131072, batch 6144x512, text only.  DSpark was
-put up in the drafter's place the same day and reverted: it declares no sliding
-window, so its draft KV spanned the whole context (5.0 GiB) and cost ROCm1 four
-layers of the split.
+Verified at deploy (2026-10-03): boot logs `n_slots = 2, n_ctx_slot = 200704`
+and `loaded multimodal model`; decode **94.44 tok/s** at temp 0 on the probe
+that measured 104.84 on Q5_K_M — **the Q6_K rung costs a measured −10 %**:
+decode on this card is weight-bandwidth-bound and Q6_K streams ~10 % more
+bytes/token; health 200, e2e 33/33; native `tool_calls`; thinking in
+`reasoning_content`; a **512×512** solid-red PNG answers `Red` through an
+`image_url` content part (two traps: the legacy top-level `images` field is
+ignored, and a TINY probe image undersupplies image tokens — a 10×10 PNG made
+the model claim "it's a video"; probe with ≥256 px). A prompt over the slot
+is a **400 `exceed_context_size_error`**, which is why `models.yml` sets
+`contextWindow: 200704`.
 
-**Layer split is why both cards now serve ONE model.** Tensor parallelism is
-impossible (see the PCIe lesson below), and so is vLLM's pipeline parallelism —
-its stage handoff is an RCCL `send`/`recv` of the hidden state, the exact
-primitive that cannot be dispatched on card #2. llama.cpp's layer split needs
-no collective at all: layers are distributed in order and the boundary
-activation is a per-backend copy, once per forward pass. Measured on this host
-(the predecessor runtime): 1991 t/s prefill layer-split vs 272 tensor.
+Runbook: `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q6k-2x200704.md`
+(the boot-pair table that prices pool changes lives in the retired
+[[frank-small-llama-mimo-q5km]]).
+Rollback: `MODEL_FILE=/home/dank/models/MiMo-Q5KM/MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf`
+buys back 875 MiB and the −10 % decode rate (3,571 free, 104.84 tok/s, both
+measured); the older shape `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0`
+pairs with Q5 only; SGLang rollback in
+[[frank-small-sglang-mimo-v26]] (venv on disk at
+`/home/dank/GitPackages/sglang-cu128/`, INT8 payload staged to
+`~dank/.AGENTTRASH/20261002-retired-llm-payloads/`).
 
-This is the box's **only** LLM runtime: the vllm-radiance replica pair and its
-routing proxy (`frank-medium-a/-b/-proxy`) were retired the same day they went
-live, with their units moved to `~/.AGENTTRASH/20260916-vllm-radiance-retire/`.
+### frank-primary — tcclaviger vLLM, Qwen3.8-Flash-Next FP8 (Dankserver, ACTIVE)
 
-Files: `scripts/build-llama-rdna4.sh` (builds the runtime, gfx1201 only, private
-ROCm 7.14 toolchain bundled beside the binary), `scripts/download-frank-medium-gguf.sh`
-(resumable, byte-verified payload), `scripts/frank-medium.env`,
-`scripts/serve-frank-medium.sh`, `scripts/frank-medium.service`. The deploy
-handler refuses to start a unit whose runtime or payload is missing, retires
-the vLLM quadlets, waits for 7770 to be released (the retired proxy held it) and
-gates on `/health`.
+The **tcclaviger RDNA4 vLLM fork** (`docker.io/tcclaviger/vllm@sha256:da4029666…`,
+digest-pinned) serving **`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8`**
+(~172.8 GiB, 131 shards) on **both** Radeon AI PRO R9700s (gfx1201), **TP=2**,
+**fp8 KV**, **no speculative decode**, **expert prefetch OFF**, and since
+2026-10-03 the **shape-G** geometry (shape E + MNBT 2048 + the CUDA-graph plan
+term at the measured floor): `--max-model-len 200704` × 3 slots with
+`--kv-cache-memory-bytes 3,166,500,000`/rank (~400,700 KV tokens — the engine
+prices this pool at 7,891.8 B/token here), util **0.982** (the law — see the
+unbootable-1.0 lesson below), `--expert-precise-memory`,
+`VLLM_CUDAGRAPH_MEM_BASE_MB=0` / `_PER_GRAPH_MB=12`,
+`GPU_MAX_HW_QUEUES=2` and a mounted `/root/.cache`. `models.yml` carries
+`contextWindow: 200704` for this endpoint — the served window, not the model's
+native 262144, so the harness compacts before vLLM answers a longer prompt
+with a 400.
 
-The ref is pinned by two features and must not float: **DFlash2**
-(ggml-org/llama.cpp#27342) and **qwen35 MTP**. The ROCmFPX fork the endpoint
-used earlier that day has neither.
+**`--max-num-seqs 2` is impossible on this model.** Measured 2026-10-02 by
+single-variable boot probe against the then-deployed shape: SEQS 2 fails at
+BOTH MNBT 1024 and 2048 with `torch.fx.experimental.symbolic_shapes.Constraint
+ViolationError: Constraints violated (L['query_start_loc'].size()[0]) …
+specialized it to be a constant (3)`; SEQS 3 passes at both. `model_runner.py`
+sizes that input buffer `max_num_reqs + 1` and `_dummy_run` sets
+`num_reqs = min(max_num_batched_tokens, max_num_reqs)`, so the width is
+`SEQS + 1` for any `MNBT ≥ SEQS` — MNBT only moves the compile RANGE
+((1,1024) vs (1,2048)). `--enforce-eager` bypasses the path but drops CUDA
+graphs. The same failure class killed the 2026-10-01 bf16 2×262144 shape.
 
-Runbook: `serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md`.
-Retired runtimes: [[frank-medium-radiance-mxfp4]] (vLLM replicas, same day),
-[[frank-medium-rocmfpx]] (llama.cpp fork, earlier that day).
+**The expert arena has THREE reachable levers: `max_model_len`, the overhead
+term, and the CUDA-graph estimate.**
+The fork's planner sizes expert cache as `budget − non-expert − KV reserve −
+graphs − overhead`, and the KV reserve is registry-priced at 9190 B/token ×
+`max_model_len`: 262144 → 2.48 GiB reserve → arena 14.30 GiB / **6,248 slots**;
+200704 → 1.91 GiB → 14.87 GiB / **6,496 slots**. Δreserve 0.57 GiB = Δarena
+0.57 GiB. The 6.1e9 → 3.15e9 pool cut bought **zero** arena; it buys runtime
+headroom (the shape it replaced died with `0 bytes free` on a real 995-token
+request). The second lever is `--expert-precise-memory`, **ON since
+2026-10-02**: it swaps the registry's default overhead (7.60 GiB = fitted +
+compile/graph-capture headroom) for the fitted figure, and the 2.35 GiB
+difference lands in the arena — measured plan lines at 200704 × 3: overhead
+5.25 GiB, expert cache **17.22 GiB / 7,522 slots** (+1,026 / +16 %), swap region
+13.7 GiB / 5,986 slots, boot healthy in 420 s. The pool stayed at 3.15e9
+deliberately: because the pool is not an arena lever, cutting it would cost
+concurrency (a 2.2e9 probe reports 277,686 tokens / **1.38×**) and buy no slots.
+The arena (2.344 MiB per slot) is the decode-critical region — cutting it for KV
+headroom collapses the offload hit rate.
+
+**Free VRAM on this endpoint is NOT arena headroom (2026-10-03 audit).**
+`rocm-smi` reads ~3.5 GiB free per card after hours of real traffic, but the
+planner charges `budget = total × util` against terms that overstate reality:
+the KV reserve is registry-priced at 9190 B/token (`1.91 GiB` at 200704) while
+the real pool bills 2.93 GiB at 7,892 B/token, and the precise overhead (5.25
+GiB) is far above the idle-measured figure. So the visible free space is the
+estimator gap, not allocatable arena. `gpu_memory_utilization` is a real arena
+lever (each +0.01 = +0.32 GiB) but **0.982 is the law, and 1.00 is UNBOOTABLE
+on this fork — measured 2026-10-03**: the worker init demands
+`free > budget`, and at `le=1` budget == device total, so both ranks raise
+`ValueError: Free memory on device cuda:N (31.29/31.86 GiB ...)` before a
+single weight loads, twice (2-restart crash loop, zero payload progress). The
+CLI field accepting `le=1` is the trap; Frank's 0.982 is where it always
+ships. What IS reachable from flags: `VLLM_CUDAGRAPH_MEM_BASE_MB` /
+`_PER_GRAPH_MB`, read at call time by `estimate_cudagraph_memory()` — the
+analytic default (256 MiB + 52 MiB/graph) charged 0.55 GiB where the engine's
+own post-capture line reported 0.31 GiB, and a second boot (with the plan
+already charging 0.32 GiB) reported **-0.06 GiB**, so the 256 MiB base pool is
+pure over-charge: shape G charges `0 base + 12 MiB/graph` = 0.07 GiB, which
+still prices per graph for a future 18-graph spec-decode shape (0.21 GiB).
+Verify every deploy by comparing the `Estimated CUDA graph memory`
+line with `Graph capturing finished … took X GiB`; if capture exceeds the
+estimate, raise `_PER_GRAPH_MB` back toward 52. MNBT 1024 → 2048 costs ZERO
+arena: the registry overhead is `base + 0.22 GiB × max(0,(nbt−4096)/1024)`
+anchored at nbt 4096, so every rung ≤4096 charges the same 5.25 GiB — and the
+fitted figure was fitted AT 4096.
+
+**A container's compile cache does NOT survive `podman rm -f`, and on this
+endpoint that is 181 s per boot.** Measured 2026-10-02: the image ships **no
+`/root/.cache` at all** (fresh container: `No such file or directory`); after a
+boot it holds `/root/.cache/vllm` at **1,154,714,333 B in 4,665 files** plus
+`/root/.cache/comgr` at 229 MB, all in the ephemeral layer, and the engine
+prints `torch.compile took 181.20 s in total` (173.19 s of it one graph,
+`Compiling a graph for compile range (1, 1024)`). Every deploy, restart and
+reboot re-pays it. The serve script now mounts
+`FRANK_PRIMARY_CACHE_DIR` at `/root/.cache` — the parent, so the vLLM AOT /
+torch_compile tree (`VLLM_CACHE_ROOT`, whose default is
+`/root/.cache/vllm`) and the comgr HIP-kernel JIT cache both persist, and
+nothing in the image is hidden because the image has no such directory. The
+compile RANGE is part of the cache key, so an MNBT change costs one
+recompilation, not a permanent tax. Same class of fix as the recipe's
+`/cache/{triton,vllm,inductor}` volumes. **Proven live 2026-10-02 on
+consecutive boots:** cold `Compiling a graph for compile range (1, 1024) takes
+179.24 s` / `torch.compile took 188.26 s`; warm `Directly load the compiled
+graph(s) for compile range (1, 1024) from the cache, took 2.143 s` /
+`torch.compile took 3.97 s in total` / `init engine (profile, create kv cache,
+warmup model) took 41.13 s (compilation: 3.98 s)`. Time to `/health` is
+dominated by the 172.8 GiB payload load, so the visible win is the ~184 s of
+inductor/AOT build, not a proportional boot cut.
+
+Measured 2026-09-29 tuning campaign (fp8 + MTP4 era, since retired): decode
+**37.4-40.6 tok/s, acceptance ~54 %**; the 256k ctx arm measured 34.1-36.5;
+2-way concurrency sums BELOW parity (25.49 vs 26.8 single) — slots are a
+capacity feature, not a throughput one. V-only q8 KV is NOT possible via
+config: stock `--kv-cache-dtype` is K+V together; a V-only quant needs a fork
+patch.
+
+Unit and self-healing: root systemd unit (`scripts/frank-primary.service`,
+`User=dank`, `LimitMEMLOCK=infinity`), `ExecStop=podman stop -t 120
+frank-primary-qwen`. The serve script drives `podman run` directly, gates
+the digest-pinned image, the payload's `config.json` and its exact
+131-shard count and the GPU device nodes, clears stale container
+identities, then blocks on `podman wait`. `deploy_llm_primary` gates the
+same, waits out :7770, relinks/enables/restarts the unit and health-gates
+with the MainPID check. Boot cost ~7 min warm (the 172.8 GiB payload load; the
+~184 s inductor/AOT build inside it is what the mounted cache now skips — see
+the compile-cache lesson below); health returns 503 while loading.
+
+Runbook: `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md`.
+Retired GLM experiment: `serverAppConfigs-docs/LLM/frank-primary-glm53-flash-w4a16.md`.
 
 ## Key Lessons Learned
 Durable operational lessons from the fleet's history. Full detail in the
@@ -376,29 +581,40 @@ linked runbooks.
   and the rest behind the chipset (usually not) — that asymmetry is what
   blocked frank-medium's TP=2. A one-card collective proves nothing: NCCL
   short-circuits at `world_size=1`.
-- **The fix is physical — and cheap on this board.** Two paths put both cards
-  on CPU root ports, which restores the atomics routing and therefore TP=2: an
-  M.2→PCIe x4 riser into a *CPU-attached* M.2 slot (on dankserver's TUF
-  GAMING B650-E WIFI that is M.2_2, which is free — per ASUS's spec page;
-  `PCIEX16_2`, where card #2 sits today, is chipset and x1-mode), or a board
-  whose CPU x16 bifurcates to x8/x8. Bandwidth is not the constraint: a 27B
-  TP=2 allreduce at batch 4 is ≈10 MB per token-step, ≈1.4 ms at Gen4 x4.
-  RCCL's P2P `send`/`recv` fails on such a card too, so pipeline parallelism is
-  not an escape either. Full evidence matrix, the open rebuild hypothesis, and
-  the test plan: [[Dual-R9700-Cooperation]].
+- **That wall was physical, and it was fixed on 2026-09-20 by a board swap.**
+  Both paths that put two cards on CPU root ports were: an M.2→PCIe x4 riser
+  into a *CPU-attached* M.2 slot, or a board whose CPU lanes bifurcate to
+  x8/x8. dankserver took the second: **ASUS ProArt B850-Creator WiFi NEO**,
+  `PCIEX16_1` + `PCIEX16_2` at x8/x8 Gen5 on root ports `00:01.1` / `00:01.3`,
+  every bridge now `Routing+` and **both endpoints `ReqEn+`** — the exact
+  precondition the old test plan named. Bandwidth was never the constraint (a
+  27B TP=2 allreduce at batch 4 is ≈10 MB per token-step, ≈1.4 ms at Gen4 x4
+  and under 0.4 ms at Gen5 x8).
+  **Proven on the same day.** vLLM's R4D one-shot allreduce runs TP=2 here and
+  frank-medium serves on it; llama.cpp's RCCL path was built too
+  (`GGML_HIP_RCCL=ON`) and its tensor mode measured — real collective, slower
+  shape. Note the collective choice matters: TheRock's RCCL builds lack gfx1201
+  device code (`invalid device function`), which is why R4D is the path vLLM
+  uses at all. Evidence matrix and the measured three-arm verdict:
+  [[Dual-R9700-Cooperation]].
 - **When a GPU collective dies, read the runtime, not the framework.** The
   message that identified the cause came from `AMD_LOG_LEVEL=3`
   (`Pcie atomics not enabled, hostcall not supported` → `AQL dispatch
   failed!`), not from vLLM or RCCL. `AMD_LOG_LEVEL` costs one env var and
   prints the failing HIP call with its return code.
-- **The one software path that lets both cards serve ONE model: llama.cpp's
-  layer split.** vLLM cannot do it here in any of its parallelisms — TP needs
-  an allreduce, PP needs the same p2p primitive, and both die on the card that
-  cannot dispatch an RCCL kernel. `--split-mode layer` in llama.cpp needs no
+- **The software path that puts both cards on ONE model without a collective:
+  llama.cpp's layer split.** On the old board vLLM could not do it in any of its
+  parallelisms — TP needs an allreduce, PP needs the same p2p primitive, and both
+  died on the card that could not dispatch an RCCL kernel. `--split-mode
+  layer` in llama.cpp needs no
   collective: layers are distributed in order and only the boundary activation
   crosses PCIe, once per forward pass. Measured here: 1991 t/s prefill /
   18.8 tok/s decode layer-split, against 272 / 26.3 tensor (butterfly
-  fallback). That is what `frank-medium` runs — see
+  fallback).
+  This was what `frank-medium` ran until 2026-09-20, when the board swap made
+  collectives legal and radiance TP=2 beat it on every axis (95.8 vs 45.7 t/s
+  decode, 0.21 vs 0.72 s TTFT, 259 vs 72 t/s burst-4) — see
+  [[frank-medium-radiance-mxfp4]]; the llama shape is the rollback target
   [[frank-medium-llamacpp-dflash2]].
 - **Layer split is also the cross-vendor answer — and `-ot` is what cannot be
   combined with it.** On Dankclaw (2026-09-16) a CUDA 3090 + Vulkan iGPU layer
@@ -437,9 +653,10 @@ linked runbooks.
 
 **ROCm / GPU**
 
-- **Layer split is the deployed shape on both multi-device llama.cpp services,
-  for two different reasons.** On Dankserver it is the *only* way both R9700s
-  serve one model (no collective can be dispatched on card #2). On a
+- **Layer split was the deployed shape on both multi-device llama.cpp services,
+  for two different reasons.** On Dankserver it was the *only* way both R9700s
+  served one model (no collective could be dispatched on card #2) until the
+  2026-09-20 board swap; frank-medium now runs radiance TP=2 instead. On a
   RCCL-less fork build it is also the fast one: tensor parallel falls back to
   a butterfly allreduce and collapses prefill ~7x (measured 271.8 vs 1991 t/s).
   Pin the spec-decode drafter to one device — an unpinned draft under tensor
@@ -472,9 +689,13 @@ linked runbooks.
 - **QSA flash-attn assert:** `GGML_ASSERT(a-type == GGML_TYPE_F32)` on
   qwen4exp needs `scripts/patches/qwen4exp-qsa-f32-mask.patch` — but only on
   the agentionai FAST layer set; the kingjones STRIX set works unpatched.
-- **GPU budget on Dankclaw is the rocm-smi figure** (64 GiB, SVM-backed).
-  `free`/`dmidecode` show the CPU-side carve-out — intentional, do not
-  change.
+- **GPU budget on Dankclaw is the GTT aperture (~114 GiB), not the
+  rocm-smi "dedicated" figure.** The Radeon 8060S (gfx1151) reports
+  116,732 MiB (`llama-server --list-devices`;
+  `gtt_total=122402963456` in `/sys/class/drm/card1/device/`). The older
+  "64 GiB, SVM-backed" lesson was already stale — the CIRU Orca predecessor
+  held 92.6 GiB of weights on the iGPU. `free`/`dmidecode` show the
+  CPU-side carve-out — intentional, do not change.
 
 **Load / crash diagnosis**
 
@@ -504,15 +725,56 @@ linked runbooks.
   the KV pool is worse than the OOM it prevents.
 - Host swap state is invisible to `deploy.sh` (no repo file) — check
   `swapon --show` before reasoning about headroom.
+- **Silent RAM corruption outranks every software theory.** frank-primary's
+  ~40k-token output corruption (2026-10-04/05, repetition/garbage in long
+  agent conversations) was marginal DDR5 at 4800 MT/s on four DIMMs, fixed
+  by 4600. Consumer non-ECC memory raises **no MCE** for a silent flip, and
+  a short in-band pattern check (60 s, 8 GiB of 246 GiB) is not a stability
+  proof — combined, those two instruments said "ruled out" while the RAM
+  was the cause. "4800 is the JEDEC floor on 5600-rated DIMMs" is a rating,
+  not a margin guarantee. When the fault is data-shaped (NaN states,
+  garbage tokens) and a software revert doesn't move it, price in the RAM
+  before the runtime — and remember a revert tested under the same marginal
+  hardware is itself confounded. See the runbook's 2026-10-05 section.
 
 **Runtime-specific guards**
 
+- **Rootless podman cannot RAISE rlimits, and `--ulimit nofile=1048576`
+  dies at container create, not at runtime.** runc reports
+  `error setting rlimit type 7: operation not permitted` (type 7 =
+  RLIMIT_NOFILE) when the requested hard limit exceeds the host process's
+  hard limit — on Dankclaw that ceiling is 524288. Inherited-equal limits
+  are fine, so `memlock=-1:-1` works iff the unit grants
+  `LimitMEMLOCK=infinity` (systemd is privileged; the value flows through
+  podman to runc without any raise). Upstream halogen's own run lines pass
+  NO `--ulimit` at all — copy vendor run lines before adding flags
+  (measured 2026-10-05 on the frank-large cutover).
 - **Kairic Edge** (if ever re-deployed): the pinned CK commit needs
   `third_party/composable_kernel/include/ck/config.h` generated manually
   (untracked; `git clean` removes it), and its greedy-argmax fastpath rejects
   non-greedy requests and reasoning budgets. Re-test chat before production.
 - **FastMTP sidecar**: build must be commit `4df29be4` + the HauhauCS patch;
   stock binaries fail the draft load with a tensor-shape mismatch.
+- **A GGUF's custom tensor type picks the runtime — read the histogram before
+  promising a file to a loader.** `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` is
+  ftype 115 with 311 tensors of type 103 (`GGML_TYPE_Q8_0_ROCMFPX`); stock
+  llama.cpp and gufo refuse it at load, so ROCmFPX was not a preference but
+  the only door. Counting tensor types out of the GGUF KV section takes one
+  python read and settles a runtime argument in minutes. The build identity
+  matters as much as the fork: the 2026-08-25 ROCmFPX builds SIGSEGV in
+  `ggml_compute_forward_set_rows` on this same file, so
+  `scripts/build-rocmfpx.sh` pins commit `36e9acd` by SHA.
+- **Speculative-decode flags move as a SET.** Handing the ROCmFPX fork
+  `--model-draft` with no `--spec-type` (the naive way to build a "no MTP"
+  bench arm) segfaulted at load, core dumped, no diagnostic (2026-10-02). A
+  baseline arm must strip `--model-draft` AND every `--spec-draft-*` AND
+  `--spec-type`, or it measures a crash instead of the absence of drafting.
+- **An orphaned scratch server taxes the whole host.** A bench child left
+  alive by a client-side exception kept ~26 GiB of weights resident on a
+  121 GiB box, pushed `available` to 3 GiB, and every later rate was suspect.
+  Record the child pid and kill it in a `finally`, then prove `free` came back
+  before believing a number. Kill by pid — never `pkill -f llama-server`,
+  which also hits frank-small.
 
 **llama.cpp serving**
 
@@ -522,19 +784,26 @@ linked runbooks.
   the MAIN model's weights (`shares_model`), so there is no second copy of a
   12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
   tensors at all, and pointing `--model-draft` at the file would load it twice.
-- **What can be chained, and what cannot.** `--spec-type a,b` builds every
-  enabled implementation and chains drafting through them, in a fixed internal
-  order (ngram types first, then the drafters — CLI order is irrelevant). The
-  gate is `ctx_dft`: **draftless** types (the ngram family) are added
-  unconditionally, while `draft-mtp`, `draft-dflash`, `draft-dspark` and
-  `draft-eagle3` all require a draft CONTEXT. So a chain may hold one
-  context-backed drafter plus any number of ngram types — `draft-dflash,ngram-mod`
-  is legal and measured (2.6x the drafter alone on prompt-echo traffic, byte-
-  identical to it on prose at the fleet's 48/64 window) — but
-  `draft-mtp,draft-dflash` dies at load: *"context type MTP requested but model
-  doesn't contain MTP layers"*, because MTP builds its context against whatever
-  model the draft context holds. Two context-backed types cannot share one
-  process, and MTP's head only loads when MTP **is** the draft model.
+- **What can be chained, what cannot, and what a chain costs on a hybrid.**
+  `--spec-type a,b` builds every enabled implementation and chains drafting
+  through them, in a fixed internal order (ngram types first, then the
+  drafters — CLI order is irrelevant). The gate is `ctx_dft`: **draftless**
+  types (the ngram family) are added unconditionally, while `draft-mtp`,
+  `draft-dflash`, `draft-dspark` and `draft-eagle3` all require a draft
+  CONTEXT. So a chain may hold one context-backed drafter plus any number of
+  ngram types — `draft-dflash,ngram-mod` is legal and measured (2.6x the
+  drafter alone on prompt-echo traffic, byte-identical to it on prose at the
+  fleet's 48/64 window) — but `draft-mtp,draft-dflash` dies at load: *"context
+  type MTP requested but model doesn't contain MTP layers"*, because MTP
+  builds its context against whatever model the draft context holds. Two
+  context-backed types cannot share one process, and MTP's head only loads
+  when MTP **is** the draft model. **A LEGAL chain can still lose on a
+  recurrent model:** `draft-mtp,ngram-mod` on frank-large's qwen35 hybrid
+  measured −15 % prose / −7 % block-copy (2026-10-03) — adding the ngram layer
+  logs `recurrent state rollback is not compatible with 'ngram-mod' -
+  disabling rollback support`, and MTP verification depends on that rollback,
+  so acceptance drops on ALL traffic even where the matcher fires. Measure
+  chains per architecture, never by analogy.
 - **A drafter's KV scales with the TARGET's context, and its window policy
   decides how much that costs.** The draft context is built with
   `cparams.n_ctx = llama_n_ctx(ctx_tgt)` (`common/speculative.cpp`) — there is
@@ -551,7 +820,8 @@ linked runbooks.
   trip moved the balance point FOUR layers (~4.3 GiB of headroom, 40:25 ->
   36:29), which is the practical size of the effect. See
   [[frank-medium-llamacpp-dflash2]].
-- **`--ubatch-size` has an OPTIMUM on this stack, not a plateau — and its
+- **`--ubatch-size` has an OPTIMUM on the 2x R9700 layer split, not a plateau —
+  and the curve is BACKEND-SPECIFIC, so never carry it across (next bullet). Its
   compute buffers land about TWICE as heavy on the card that holds the output
   layers and the drafter.** Measured on frank-medium 2026-09-16 at the deployed
   shape (262144 over two slots, a 64k prefill measured at 64k depth,
@@ -569,6 +839,19 @@ linked runbooks.
   its 2020 MiB pp buffer here under the DSpark drafter, when ROCm1 had 0.08 GiB
   free. That was a CONTIGUITY failure, not a total-bytes one, and no totals
   model — the sweep's or the seam's — can see it.)
+- **On the CUDA 3090 the same knob has NO interior optimum up to 2048, so
+  frank-small ships 4096/2048.** Measured 2026-10-05 by boot-pair A/B on the
+  live service: at 8,928 prompt tokens ubatch 1024 gives 3,972 / 4,077 t/s and
+  ubatch 2048 gives 4,207 / 4,212 — **+4.6 %** — for **616 MiB** (free 2,849 →
+  2,279). Decode did not move (91.9–93.0 vs 93.8–94.3, i.e. noise), and a
+  279-token prompt, which is below either ubatch, came back identical — that is
+  the mechanical proof the knob was the only variable, since a prompt shorter
+  than the ubatch cannot see it. The logical `--batch-size` was again free. And
+  the predecessor's "ubatch 2048 failed the scratch boot" was a shape
+  observation, not a law: the same file boots 2048 clean at 2 × 200,704. Bench
+  it by restarting the SERVICE through a transient `FRANK_SMALL_ENV_FILE`
+  drop-in, never a scratch port — one card holds one server. See
+  [[frank-small-llama-mimo-q6k-2x200704]].
 - **One `-ts` step is one LAYER, and a hybrid stack's layers are not the same
   size.** qwen35 is 16 x (3 Gated DeltaNet + 1 Gated Attention), so a step is
   worth ~0.43 GiB on a DeltaNet boundary and ~1.43 GiB on a full-attention one
@@ -585,6 +868,41 @@ linked runbooks.
   host-RAM hash table) and its window decides which traffic it can see: too
   short and it proposes on ordinary prose fragments, which costs verification
   without acceptance.
+- **`--parallel` divides the pool, so MORE conversations cost WINDOW, and the
+  per-slot number is the contract — not `ctx-size`.** `n_ctx_slot =
+  CTX_SIZE / PARALLEL`, and llama.cpp refuses a prompt longer than the SLOT
+  (`400 exceed_context_size_error: request (184001 tokens) exceeds the
+  available context size (180224 tokens)`) even when the pool has room. So
+  frank-small's 2 → 3 reslice moved each conversation from the model's native
+  262,144 down to 180,224, and `models.yml` `contextWindow` had to follow or
+  the harness would discover the ceiling as a 400 on a long conversation. Side
+  effect worth knowing: the shorter slot made decode ~5 % FASTER (106.8 vs
+  100.4 tok/s) — less attention to scan and one fewer DeltaNet state resident.
+  The 2026-10-03 reslice went the other way (3 → 2 slots, 180,224 → 200,704,
+  pool 401,408) to fund unquantized f16 V, and `contextWindow` followed again;
+  decode landed 104.8 tok/s — the ~2 % off 106.8 is the exact-V tax. The same
+  day the freed space also bought the Q6_K rung (now 94.4 tok/s — the rung
+  itself is a further measured −10 %, weight-bandwidth, not slot geometry).
+  All moves measured; both slot directions cost or buy window, never pool.
+  See [[frank-small-llama-mimo-q6k-2x200704]].
+- **Price a VRAM budget from two boots, never from a formula — and an mmproj
+  costs ~1.5x its file size.** The frank-small 3090 fit was solved from two
+  measurements: 2 × 262,144 with no projector = 20,190 MiB used (KV 12,800 ⇒
+  **non-KV 7,390**), 3 × 196,608 with the f16 projector = 22,972 used (KV
+  14,113 ⇒ **non-KV 8,859**). The difference is the vision tower: a 918 MB file
+  that bills ~1.4 GiB, because the encoder's activation and compute buffers
+  ride with it. Subtracting the two rows is exact; extrapolating weights+KV+
+  "a bit for buffers" is what leaves 1.1 GiB free when you meant 2.7. Then
+  solve `KV ≤ card − non-KV − headroom` and round to a multiple of
+  `PARALLEL × 4096`. Recorded in `scripts/frank-small.env` so the next resize
+  starts from measurement.
+- **`images` and `image_url` are NOT interchangeable: only the content part
+  reaches the model.** `--mmproj` loads, `usage.prompt_tokens` rises, and the
+  model still invents an answer if the client used the legacy top-level
+  `images` field on `/v1/chat/completions` — frank-small answered "Brown." for
+  a solid-red PNG that way and "Red." through the OpenAI `image_url` part. The
+  failure is silent because a hallucinated color is a plausible answer. Probe
+  vision with a solid color and expect an exact word.
 - **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
   `500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
   then served frank-large: every tool-using agent on that endpoint was broken
@@ -593,7 +911,9 @@ linked runbooks.
   returns `tool_calls`. Check tool support before routing a role at an endpoint.
 - **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
   option and the same ~12x prefill collapse described in the ROCm note above.
-  f16 KV needs nothing, which is why frank-small runs f16.
+  f16 KV needs nothing; frank-small ran f16 for that reason until
+  2026-09-17, when `build-cuda` was rebuilt with the option ON so the
+  endpoint could take q8_0 KV.
 - **llama-bench separates a LIST with `/` and SWEEP values with `,`.** Both
   `-dev` and `-ts` follow this: `-dev CUDA0,Vulkan1` is two *single-device*
   configurations, never one two-device run, and `-ts 0.85,0.15` is two sweep
@@ -683,9 +1003,9 @@ Active:
 
 | Service | Runbook |
 |---------|---------|
-| frank-large (halogen-flash-server, Dankclaw) | `serverAppConfigs-docs/LLM/halogen-flash-server.md` |
-| frank-small (llama.cpp Qwen3.8-27B GSQ-RCO, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-qwen38-gsq-rco.md` |
-| frank-medium (llama.cpp + DFlash2, Dankserver) | `serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md` |
+| frank-large (halogen container, Qwen3.8-27B .hgn + DFlash2, 1×200,700, text-only, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md` |
+| frank-small (llama.cpp CUDA, MiMo-V2.6-Distill-Qwen-9B Q6_K, 2×200,704 f16 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q6k-2x200704.md` |
+| frank-primary (tcclaviger vLLM, Qwen3.8-Flash-Next FP8, Dankserver) | `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
 
@@ -693,8 +1013,28 @@ Retired (rollback targets and history):
 
 | Runtime | Host | Window | Runbook |
 |---------|------|--------|---------|
-| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
+| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
+| ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
+| gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
+| vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
+| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q6_K (`frank-small`) | Dankclaw | 2026-10-01 → 10-02 (the rung returned ACTIVE 2026-10-03 at the 2×200,704 shape — `frank-small-llama-mimo-q6k-2x200704.md`) | `frank-small-llama-mimo-q6k.md` |
+| llama.cpp master MiMo-V2.6-Distill-Qwen-9B Q5_K_M 2×200,704 f16 KV (`frank-small`) | Dankclaw | 2026-10-02 → 10-03 (rollback target — one `MODEL_FILE` line, buys back 875 MiB + the −10 % rung tax; owns the boot-pair table) | `frank-small-llama-mimo-q5km.md` |
+| SGLang MiMo-V2.6 INT8-W8A16-AutoRound (`frank-small`) | Dankclaw | 2026-10-01 (same-day swap to the GGUF path) | `frank-small-sglang-mimo-v26.md` |
+|tcclaviger vLLM GLM-5.3-Flash W4A16 int4, +/- DFlash2 drafter (`frank-primary`)|Dankserver|2026-09-30 → 10-01|`frank-primary-glm53-flash-w4a16.md`|
+| llama.cpp b11030 Qwen3.8-27B UD-Q8_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-21 → 09-28 | `frank-large-unsloth-qwen38-27b-q8xl.md` |
+| R9V radiance-vLLM Qwen3.8-Flash-Next-Uncensored IQ4_XS (`frank-primary`) | Dankserver | 2026-09-27 (same-day swap to Q8_0) | `frank-primary-r9v-radiance-qwen38-uncensored.md` |
+| llama.cpp b11030 Qwen3.8-Flash-Next Q8_0 + shared MTP (`frank-primary`) | Dankserver | 2026-09-26 → 09-27 | `frank-primary-llamacpp-qwen38-flash-next-q8.md` |
 | ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-large`) | Dankclaw | 2026-09-15 → 09-16 | `frank-large-ik-uncensored.md` |
+| vllm-radiance TP=2 MXFP4 + DFlash2 (`frank-medium`) | Dankserver | 2026-09-20 → 09-26 | `frank-medium-radiance-mxfp4.md` |
+| Lucebox DS4.1 DeepSeek-V4.1-Flash ROCmFP23 (`frank-primary`) | Dankserver | 2026-09-26 (same-day swap) | `frank-primary-lucebox-ds4.md` |
+| kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
+| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-large`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
+| CIRU v4.4.1 Orca Qwen3.8-Flash (`frank-large`) | Dankclaw | 2026-09-19 → 09-21 | `frank-large-ciru-orca.md` |
+| unsloth llama.cpp b11030 Qwen3.8-Flash-Next UD-IQ4_XS (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-unsloth-qwen38-flash-next.md` |
+| Kairic Edge Qwen3.8-27B IU4 W4A4 (`frank-large`) | Dankclaw | 2026-09-21 (same-day swap) | `frank-large-kairic-edge-qwen38-27b.md` |
+| ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-small`) | Dankclaw | 2026-09-16 → 09-17 | `frank-small-qwen38-gsq-rco.md` |
+| vLLM Gemma4-12B w4a16 (`frank-small`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-small-vllm-gemma4.md` |
+| llama.cpp Gemma4-12B QAT Q4_K_M (`frank-small`) | Dankclaw | 2026-09-17 → 10-01 | `frank-small-gemma4-12b.md` |
 | agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
 | eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
 | Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |
@@ -704,6 +1044,7 @@ Retired (rollback targets and history):
 | FreeToken (official + Huihui FP8, `frank-small`) | Dankserver | 2026-09-02 → 09-15 | `freetoken-frank-small.md` |
 | FastMTP patched llama.cpp | Dankserver | retired 2026-08-18 | `qwen3.8-spec-matrix.md` |
 | ik_llama.cpp Ornith CRACK | Dankserver | retired 2026-09-02 | `ikllama-ornith-dankserver.md` |
+| llama.cpp layer split + DFlash2 (`frank-medium`) | Dankserver | 2026-09-16 → 09-20 | `frank-medium-llamacpp-dflash2.md` |
 
 Retired runbooks are under `serverAppConfigs-docs/LLM/`. Retired scripts live
 in `old/` and `.AGENTTRASH/`.

README.md

@@ -13,7 +13,7 @@ one deploy script.
 | Host | User | IP | Podman stack | LLM service | LLM port |
 |------|------|----|--------------|-------------|----------|
 | dankserver | dank | 192.168.0.235 | media and library | frank-medium (ROCmFPX llama.cpp, Qwen3.8-27B AGENT) | 7770 |
-| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (CIRU ROCmFPX, Qwen3.8-27B) | 7770 |
+| dankclaw | dank | 192.168.0.143 | AI, search, development, observability | frank-large (gufo, Qwen3.8-27B UD-Q8_K_XL) | 7770 |
 
 ## Active Services
 

autoresearch.sh

@@ -0,0 +1,117 @@
+#!/usr/bin/env bash
+# autoresearch.sh — canonical benchmark entrypoint: speed of frank-primary.
+#
+# Runs on the operator box and drives one bench window on DANKSERVER:
+#   1. fingerprint the candidate trio (env + serve script + unit)
+#   2. memo fast-path: an identical fingerprint that already produced a FULLY
+#      successful window (bench + verified production restore) returns the
+#      cached metrics without touching the fleet
+#   3. stage the candidate trio plus the measure client into
+#      /home/dank/.autoresearch-frank-primary/ (ephemeral; the repo checkout
+#      on the host stays the production source of truth)
+#   4. run the window (scripts/autoresearch/frank-primary-window.sh):
+#      stop + mask the production unit, boot the staged config AS THE SERVICE
+#      (same serve script, same container, same port 7770), wait for health,
+#      run the in-container usage-counter measure, then restore the production
+#      unit and verify its health — AGENTS.md Benchmarking Policy
+#   5. print METRIC lines (parse by run_experiment)
+#
+# Primary metric: decode_tps (HIGHER is better) — median across the three
+# prompt depths (115/1056/4096) of per-depth medians of
+# completion_tokens / (wall - ttft), 3 runs each after a warm pass,
+# temperature 0, the campaign's deterministic word-seed workload.
+#
+# Exit 0 with METRIC lines on success; non-zero on any failure (the window
+# script's EXIT trap restores production on every path).
+set -euo pipefail
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+HOST="${FRANK_PRIMARY_AR_HOST:-dankserver}"
+STAGE="/home/dank/.autoresearch-frank-primary"
+
+# FRANK_PRIMARY_AR_ENV_FILE stages a DIFFERENT env than the repo's — that is how
+# the arms driver A/Bs engine knobs without mutating the checkout. The
+# fingerprint covers whatever file is staged, so the memo cannot cross arms.
+ENV_FILE="${FRANK_PRIMARY_AR_ENV_FILE:-$SCRIPT_DIR/scripts/frank-primary.env}"
+SERVE="$SCRIPT_DIR/scripts/serve-frank-primary.sh"
+UNIT="$SCRIPT_DIR/scripts/frank-primary.service"
+MEASURE="$SCRIPT_DIR/scripts/autoresearch/frank-primary-measure.py"
+WINDOW="$SCRIPT_DIR/scripts/autoresearch/frank-primary-window.sh"
+
+for f in "$ENV_FILE" "$SERVE" "$UNIT" "$MEASURE" "$WINDOW"; do
+    [ -f "$f" ] || { echo "[autoresearch] missing harness file: $f" >&2; exit 1; }
+done
+command -v python3 >/dev/null 2>&1 || { echo "[autoresearch] python3 required on operator box" >&2; exit 1; }
+
+# Fingerprint of everything that can change the engine's argv or environment.
+HASH="$(cat "$ENV_FILE" "$SERVE" "$UNIT" | sha256sum | cut -d' ' -f1)"
+echo "[autoresearch] candidate fingerprint $HASH" >&2
+
+ssh_run() { ssh -o BatchMode=yes -o ConnectTimeout=15 "$HOST" "$@"; }
+
+emit_metrics() { # $1 = run.json
+    python3 -c '
+import json, sys
+run = json.loads(sys.argv[1])
+m = run["measure"]
+print("METRIC decode_tps=%s" % m["decode_tps"])
+for k in ("decode_tps_115", "decode_tps_1056", "decode_tps_4096"):
+    print("METRIC %s=%s" % (k, m[k]))
+print("METRIC ttft_ms_4096=%s" % m["ttft_ms_4096"])
+print("METRIC kv_tokens=%s" % (run.get("kv_tokens") or 0))
+print("METRIC boot_seconds=%s" % (run.get("boot_seconds") or -1))
+print("METRIC completion_tokens_4096=%s" % m.get("completion_tokens_4096", 0))
+# Any extra sampling temperature the measure swept (AR_TEMPS), e.g. the 0.7
+# arm — that is where speculative-decode acceptance shows up, since a greedy
+# draft matches a greedy target exactly and is rejected more under real
+# sampling. Suffixed keys only (_t0.7), so the historical lines are untouched.
+for k in sorted(m):
+    if "_t" in k and (k.startswith("decode_tps") or k.startswith("ttft_ms")):
+        print("METRIC %s=%s" % (k, m[k]))
+' "$1"
+}
+
+# --- memo fast-path -------------------------------------------------------
+cached_hash="$(ssh_run "cat '$STAGE/last/hash' 2>/dev/null" 2>/dev/null || true)"
+if [ -n "$cached_hash" ] && [ "$cached_hash" = "$HASH" ]; then
+    cached_run="$(ssh_run "cat '$STAGE/last/run.json' 2>/dev/null" || true)"
+    if [ -n "$cached_run" ]; then
+        echo "[autoresearch] memo hit — cached metrics for $HASH, fleet untouched" >&2
+        emit_metrics "$cached_run"
+        exit 0
+    fi
+fi
+
+# --- stage the candidate ----------------------------------------------------
+echo "[autoresearch] staging candidate on $HOST:$STAGE" >&2
+ssh_run "mkdir -p '$STAGE'"
+scp -q "$ENV_FILE" "$HOST:$STAGE/frank-primary.env"
+scp -q "$SERVE" "$HOST:$STAGE/serve.sh"
+scp -q "$MEASURE" "$HOST:$STAGE/measure.py"
+scp -q "$WINDOW" "$HOST:$STAGE/window.sh"
+ssh_run "printf '%s\n' '$HASH' > '$STAGE/hash'"
+
+# --- the window -------------------------------------------------------------
+# Budgets: bench boot up to 20 min (warm boots measured ~7 min), measure up
+# to 30 min, restore boot up to 20 min — call run_experiment with
+# timeout_seconds >= 3600.
+echo "[autoresearch] opening bench window (stop+mask -> boot -> measure -> restore)" >&2
+set +e
+ssh_run "bash '$STAGE/window.sh'"
+rc=$?
+set -e
+if [ "$rc" -ne 0 ]; then
+    echo "[autoresearch] window failed (rc=$rc); host-side evidence:" >&2
+    ssh_run "tail -60 '$STAGE/fail.log' 2>/dev/null || tail -40 '$STAGE/serve.out' 2>/dev/null || tail -40 '$STAGE/measure.out' 2>/dev/null || true" >&2 || true
+    exit 1
+fi
+
+run_json="$(ssh_run "cat '$STAGE/run.json'")" || {
+    echo "[autoresearch] could not read $STAGE/run.json back" >&2; exit 1; }
+echo "[autoresearch] window result:" >&2
+printf '%s\n' "$run_json" >&2
+emit_metrics "$run_json"
+
+echo "[autoresearch] engine evidence (plan / compile path / spec counters):" >&2
+ssh_run "cat '$STAGE/evidence.log' 2>/dev/null" >&2 || true
+exit 0

dankclaw/.env.example

@@ -4,6 +4,11 @@
 # SearXNG
 SEARXNG_SECRET=changeme
 SEARXNG_BASE_URL=http://search.franksdank.cloud/
+# Kagi search API key (https://help.kagi.com/kagi/api/). Powers the paid
+# `kagi` engine in shared/searxng/settings.yml; deploy-remote.sh bakes it into
+# the live settings.yml via scripts/searxng-render-settings.py. The quadlet
+# does NOT pass it into the container environment.
+KAGI_API_KEY=changeme
 
 # Gluetun VPN (see https://github.com/qdm12/gluetun-wiki)
 # Single-peer WireGuard model. Fill from your provider's WireGuard config.

dankclaw/Caddyfile

@@ -309,6 +309,10 @@ http://:8017 {
 	@home host www.franksdank.cloud
 	handle @home {
 		root * /srv/homepage
+		# Deploys overwrite /srv/homepage in place; without revalidation a
+		# browser keeps serving the pre-deploy CSS/JS/HTML. no-cache forces an
+		# ETag round-trip on every load (a cheap 304 when unchanged).
+		header Cache-Control "no-cache"
 		# RBAC (2026-09-01): /admin requires membership in the homepage-admins
 		# authentik group. In proxy-provider mode the outpost injects
 		# X-Authentik-Groups on its upstream request to this listener and strips
@@ -323,6 +327,16 @@ http://:8017 {
 			}
 			respond "Forbidden: homepage admins only" 403
 		}
+		# User docs page (static). Anchors (#models etc.) are client-side,
+		# so both /docs and /docs/ map onto the single docs.html.
+		handle /docs {
+			try_files /docs.html
+			file_server
+		}
+		handle /docs/ {
+			try_files /docs.html
+			file_server
+		}
 		@data path /services.json /status.json /app.js /style.css /deployed.js
 		handle @data {
 			file_server

dankclaw/Containerfile.comfyui

@@ -21,4 +21,20 @@ RUN mkdir -p /opt/ComfyUI/models /opt/ComfyUI/input /opt/ComfyUI/output /opt/Com
 RUN grep -viE '^(torch|torchvision|torchaudio)([^a-z]|$)' requirements.txt > /tmp/req-notorch.txt \
     && pip install --no-cache-dir -r /tmp/req-notorch.txt
 
+# The city96/ComfyUI-GGUF custom node (loaded from the host's custom_nodes
+# volume) needs the gguf library to decode GGUF-quantized diffusion models.
+# sentencepiece and protobuf are its optional tokenizer deps. Baking them into
+# the image keeps the dependency across container recreations — the node itself
+# lives in the host volume, which persists, but the Python env does not.
+RUN pip install --no-cache-dir "gguf>=0.13.0" sentencepiece protobuf
+
+# Bake the qwen_image21 patch script into the image. It is run by the
+# startup wrapper before ComfyUI starts, so the patch survives container
+COPY dankclaw/apply_qwen21_patch.py /opt/ComfyUI/apply_qwen21_patch.py
+COPY dankclaw/comfyui-start.sh /opt/ComfyUI/comfyui-start.sh
+RUN chmod +x /opt/ComfyUI/comfyui-start.sh
+
+CMD ["/opt/ComfyUI/comfyui-start.sh"]
 EXPOSE 8188
+# v2: VAE 4D-to-5D latent expansion for video VAEs (qwen_image21)
+# v3: fix in_channels to use shape[0] (latent input dim) not shape[1] (hidden dim)

dankclaw/apply_qwen21_patch.py

@@ -0,0 +1,52 @@
+import os
+
+# Apply qwen_image21 detection patch
+f = "/opt/ComfyUI/comfy/model_detection.py"
+with open(f) as fh:
+    code = fh.read()
+if "qwen_image21" not in code:
+    qwen21_check = (
+        "    if '{}txt_in.text_norm.weight'.format(key_prefix) in state_dict_keys:  # Qwen Image 2.1\n"
+        "        dit_config = {}\n"
+        '        dit_config["image_model"] = "qwen_image21"\n'
+        "        dit_config[\"in_channels\"] = state_dict['{}img_in.weight'.format(key_prefix)].shape[0]\n"
+        "        dit_config[\"num_layers\"] = count_blocks(state_dict_keys, '{}transformer_blocks.'.format(key_prefix) + '{}.')\n"
+        "        return dit_config\n"
+    )
+    anchor = "    if '{}txt_norm.weight'.format(key_prefix) in state_dict_keys:  # Qwen Image"
+    code = code.replace(anchor, qwen21_check + "\n" + anchor)
+    with open(f, "w") as fh:
+        fh.write(code)
+    print("Applied qwen_image21 detection patch")
+
+# Add qwen_image21 to GGUF node arch list
+loader = "/opt/ComfyUI/custom_nodes/ComfyUI-GGUF/loader.py"
+if os.path.exists(loader):
+    with open(loader) as fh:
+        lcode = fh.read()
+    if '"qwen_image21"' not in lcode:
+        old = '"lumina2", "qwen_image"}'
+        new = '"lumina2", "qwen_image", "qwen_image21"}'
+        if old in lcode:
+            lcode = lcode.replace(old, new)
+            with open(loader, "w") as fh:
+                fh.write(lcode)
+            print("Applied qwen_image21 arch list patch")
+
+# Patch VAE decode to expand 4D latents to 5D for video VAEs (latent_dim=3)
+sd = "/opt/ComfyUI/comfy/sd.py"
+with open(sd) as fh:
+    sdc = fh.read()
+if "self.latent_dim == 3 and samples_in.ndim == 4" not in sdc:
+    anchor = "        if self.latent_dim == 2 and samples_in.ndim == 5:\n            samples_in = samples_in[:, :, 0]"
+    replacement = (
+        "        if self.latent_dim == 2 and samples_in.ndim == 5:\n"
+        "            samples_in = samples_in[:, :, 0]\n"
+        "        elif self.latent_dim == 3 and samples_in.ndim == 4:\n"
+        "            samples_in = samples_in.unsqueeze(2)"
+    )
+    if anchor in sdc:
+        sdc = sdc.replace(anchor, replacement)
+        with open(sd, "w") as fh:
+            fh.write(sdc)
+        print("Applied VAE 4D-to-5D latent expansion patch")

dankclaw/authentik/quadlet/authentik-outpost-ldap.container

@@ -20,10 +20,10 @@ Wants=authentik-server.service
 [Container]
 ContainerName=authentik-outpost-ldap
 Network=dankclaw_default
-# Image=ghcr.io/goauthentik/ldap:2026.8.0 (amd64 digest; podman 5.7.0 refuses
+# Image=ghcr.io/goauthentik/ldap:2026.8.3 (amd64 digest; podman 5.7.0 refuses
 # the tag's nested manifest list, so pin the single-platform digest).
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env
-Image=ghcr.io/goauthentik/ldap@sha256:fa33bc8f522113830e1662f4c034352f9eb08e0a7d3da2ddadad9a4b33b4cbf4
+Image=ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9
 Environment=AUTHENTIK_HOST=https://auth.franksdank.cloud
 Environment=AUTHENTIK_INSECURE=false
 Environment=AUTHENTIK_TOKEN=${AUTHENTIK_LDAP_OUTPOST_TOKEN}

dankclaw/authentik/quadlet/authentik-outpost.container

@@ -20,7 +20,7 @@ Wants=authentik-server.service
 
 [Container]
 ContainerName=authentik-outpost
-Image=ghcr.io/goauthentik/server:2026.8.0
+Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=worker
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env

dankclaw/authentik/quadlet/authentik-server.container

@@ -19,7 +19,7 @@ Wants=authentik-postgres.service
 
 [Container]
 ContainerName=authentik-server
-Image=ghcr.io/goauthentik/server:2026.8.0
+Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=server
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env

dankclaw/authentik/quadlet/authentik-worker.container

@@ -16,7 +16,7 @@ Wants=authentik-server.service
 
 [Container]
 ContainerName=authentik-worker
-Image=ghcr.io/goauthentik/server:2026.8.0
+Image=ghcr.io/goauthentik/server:2026.8.3
 Exec=worker
 Network=dankclaw_default
 EnvironmentFile=/home/dank/serverAppConfigs/dankclaw/.env

dankclaw/comfyui-start.sh

@@ -0,0 +1,4 @@
+#!/bin/bash
+# Apply qwen_image21 patches before starting ComfyUI
+python3 /opt/ComfyUI/apply_qwen21_patch.py 2>/dev/null
+exec python main.py --listen 0.0.0.0 --port 8188

dankclaw/comfyui/quadlet/comfyui.container

@@ -21,11 +21,13 @@
 # user-unit spawn — which carries no supplementary groups — can still open them.
 # Both halves are asserted by tests/comfyui_rocm.bats.
 #
-# EXPOSURE. Loopback only. ComfyUI has no authentication and executes arbitrary
-# node graphs, and it has no Caddy vhost, so the repository's rule applies:
-# an unguarded service is internal-only. The homepage status card probes
-# 127.0.0.1:8188, which this satisfies. To reach the UI from another machine,
-# guard it behind the auth chain and give it a vhost — do not widen the bind.
+# EXPOSURE. LAN-reachable, no auth. ComfyUI has no authentication and executes
+# arbitrary node graphs. It is internal-only (no public vhost, no guard chain);
+# it is reached on the LAN via the Pi-hole DNS record
+# (comfyui.franksdank.cloud -> 192.168.0.143, in dankclaw/pihole/addn-hosts),
+# which resolves to this host's IP — so the bind is widened from loopback to
+# 0.0.0.0:8188. The homepage admin card links to that URL and probes
+# 127.0.0.1:8188.
 #
 # DATA. Models, input, output and custom_nodes live under
 # /home/dank/comfyui-data (host state, not in the repo).
@@ -39,7 +41,7 @@ ContainerName=comfyui
 Image=localhost/comfyui-rocm:gfx1151
 Pull=never
 Network=dankclaw_default
-PublishPort=127.0.0.1:8188:8188
+PublishPort=0.0.0.0:8188:8188
 AddDevice=/dev/kfd
 AddDevice=/dev/dri
 GroupAdd=keep-groups
@@ -48,7 +50,7 @@ Volume=/home/dank/comfyui-data/input:/opt/ComfyUI/input
 Volume=/home/dank/comfyui-data/output:/opt/ComfyUI/output
 Volume=/home/dank/comfyui-data/custom_nodes:/opt/ComfyUI/custom_nodes
 Volume=/home/dank/serverAppConfigs/dankclaw/comfyui/comfyui-healthcheck.py:/usr/local/bin/comfyui-healthcheck.py:ro
-Exec=python main.py --listen 0.0.0.0 --port 8188
+Exec=/opt/ComfyUI/comfyui-start.sh
 # The probe lives in a file because the podman 5.x v1 generator splits
 # HealthCmd on spaces into an argv array; an inline `python3 -c "..."` becomes
 # a puzzle of quoting that fails silently.

dankclaw/frank-large/quadlet/frank-large.container

@@ -1,219 +0,0 @@
-# Quadlet unit: frank-large (rootless, boot-persistent).
-#
-# Install path on dankclaw: ~/.config/containers/systemd/frank-large.container
-# Unit name: frank-large.service (container name: frank-large)
-#
-# The large inference service. The halogen-flash-server runtime (since
-# 2026-09-11, restored 2026-09-16 after a two-day ik_llama.cpp hybrid
-# interlude) serving peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next,
-# w4b .hgn checkpoint + quality overlay sidecar). It replaced the llama.cpp /
-# kingjones30-ROCmFPX STRIX deployment because the engine is ~4x faster on
-# prefill (the dominant cost of any real prompt) on this exact SKU. The RTX
-# 3090 it briefly shared with frank-large is now frank-small's card.
-#
-# SHAPE. The image runs both halves in one container: the engine on container
-# loopback (:8730, UNAUTHENTICATED — never publish it) and the OpenAI
-# front-end on :8731, published here as host 7770 so the fleet's clients,
-# services.json probe and docs keep the port they have always used. Same for
-# HALOGEN_MODEL_ID=frank-large: it is the id at /v1/models, in /health and in
-# every response, so no client had to be renamed. Halogen never rejects a
-# request for naming a different model, it just serves it.
-#
-# DEVICES. gfx1151 through the amdgpu/KFD stack: /dev/kfd plus the DRM render
-# node. Both are 0666 on this host, so the rootless user namespace can open
-# them (id dank is also in render/video). seccomp=unconfined and --ipc=host are
-# the image's documented requirements; memlock must be unlimited.
-#
-# DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the HIP
-# runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
-#
-# MEMORY. Weights are pinned (HALOGEN_FLASH_PIN_TRUNK default 1) — that is the
-# configuration the published decode numbers were measured with, and `free`,
-# MemAvailable and every monitor built on them will overstate free memory by
-# the size of the model (~68 GiB) because the kernel counts the locked weights
-# as reclaimable page cache. Halogen prints the corrected figure at startup.
-# The pinned trunk means this container holds most of the host; a co-tenant
-# that then takes the remainder can stall both for minutes at full CPU with no
-# disk activity and no output. That is documented behaviour, not a crash, and
-# the escape hatch is HALOGEN_FLASH_PIN_TRUNK=0 (at several times the decode
-# cost). Sizing is chosen for a SHARED 121 GiB host, and HALOGEN_KV_POOL_FIT
-# stays on as the safety net: if the machine refuses the configured pool the
-# engine lowers it itself and prints what it settled on. Read that ledger on
-# every restart — it is the ground truth for sizing, not this comment.
-#
-# POOL RETUNED 2026-09-16: 400,384 positions -- exactly two 200,192-token
-# conversations -- beside a per-conversation ceiling of 200,192 and two slots.
-# It replaces the 401,408 the 2026-09-13 trim left behind, which held one full
-# native-262k conversation beside a ~139k one, or three at 131k.
-#
-# MEASURED AT 401,408 (2026-09-13 deploy ledger): 68.0 weights + 11.0 KV +
-# 21.0 working = 100.0 GiB held, 9.1 GiB / 1396 contiguous 2 MiB blocks left,
-# engine need printed ~31.5 against the modelled ~33.4, and NO stall-warning
-# NOTE. Fit-check budget here: 121.2 - 67.7 - 20 = 33.5. That boot is the
-# baseline the 09-16 retune is measured against: the vision tower (1.97 GiB)
-# and roughly 4.2 GiB of arena come back to the host, which is what pays for
-# the co-tenant below.
-#
-# CO-TENANT since 2026-09-16: this host also serves frank-small on the RTX
-# 3090 (llama.cpp, ~12.2 GiB of weights that load through the page cache and
-# then live in VRAM, plus a CUDA context). It is the first real co-tenant this
-# service has had, which is why the arena was halved and the reserve raised.
-#
-# BOOT NOTE: on the first request after a (re)start the engine can go silent
-# for ~45-60 s ("has not answered PING") while it first-touches the freshly
-# reserved pool; /health reads engine_unresponsive during it and recovers by
-# itself. The quadlet health check's HealthStartPeriod=300s covers it; do not
-# restart the unit over it.
-#
-# WEIGHTS. HALOGEN_DOWNLOAD fetches the whole weights repo on FIRST START ONLY
-# (the entrypoint checks whether the checkpoint is absent) into the bind mount
-# below, which must therefore be read-WRITE. huggingface_hub resumes partial
-# files natively, and the quadlet runs the container with --replace --rm, so a
-# restart mid-fetch continues rather than starting over. Roughly 127 GB: the
-# 115.55 GiB checkpoint, the 2.31 GiB quality overlay, the 0.84 GiB vision
-# tower and the tokenizer. The repo is public and ungated, so no token is
-# needed; later starts open no outbound connections at all.
-#
-# VISION. Deliberately NOT set since 2026-09-16. The tower is a 1.97 GiB
-# charge against the same device budget as the pool and the arena, and the
-# fleet's image path moved to frank-small (llama.cpp on the RTX 3090, loaded
-# with --mmproj mmproj-Qwen3.8-27B-BF16). This server refuses image content
-# parts, and models.yml declares frank-large text-only to match.
-#
-# CLIENT CONTRACT (see serverAppConfigs-docs/LLM/halogen-flash-server.md).
-# There are no server-side sampler defaults: a request that sends no
-# temperature decodes GREEDY. The per-request budget defaults to 8,192 tokens
-# and REASONING COUNTS AGAINST IT, so a budget that runs out mid-thought
-# returns finish_reason "length" with empty content and the partial reasoning
-# in reasoning_content. The cap is 65,536; above it a request is a 400, never a
-# silent truncation.
-#
-# IMAGE. The unit runs the localhost/-aliased tag: this host's podman quadlet
-# path cannot resolve registry-prefixed refs from local storage at create
-# time. deploy-remote.sh pulls the ghcr.io ref and re-tags it before the
-# restart. Pull=never keeps boots off the registry.
-[Unit]
-Description=frank-large — halogen-flash-server, Qwen3.8-Flash-Next (w4b) on gfx1151
-After=network-online.target
-
-[Container]
-ContainerName=frank-large
-Image=localhost/peonist-ai/halogen-flash-server:0.5.8
-# Published port stays 7770: Hermes, open-webui, the harness and the homepage
-# probe all point here. 8730 (the engine) is deliberately not published.
-PublishPort=7770:8731
-AddDevice=/dev/kfd
-AddDevice=/dev/dri
-# Read-write: the first start downloads into it. Contains only weights.
-Volume=/home/dank/models/halogen:/models
-# Fetch the weights on first start only (absent checkpoint gates it).
-Environment=HALOGEN_DOWNLOAD=peonist-ai/halogen-qwen3.8-flash-next
-# Fleet identity: the id at /v1/models, /health and every response body.
-Environment=HALOGEN_MODEL_ID=frank-large
-# Per-conversation ceiling: exactly half the pool, so two conversations of the
-# full window are always resident. 200,192 rather than a round 200,000 is
-# pool/2. The model's native window is 262,144; a conversation past this
-# ceiling is refused, never silently truncated. Clients declare 200192 (see
-# the frank-large contextWindow in models.yml).
-Environment=HALOGEN_CTX=200192
-# The KV pool: 400,384 positions = 2 x 200,192 (the 2026-09-16 retune, down
-# from 401,408). ~28 KiB per position including block scratch, so ~11.0 GiB
-# held -- the 401,408 ledger entry measured exactly 11.0. The two slots still
-# OVERSUBSCRIBE the pool for any real request, because admission reserves
-# prompt + max_tokens: two full-window conversations each holding an output
-# budget are pool-bound, and the second waits in arrival order for the first
-# to release its region. Read the `kv pool:` lines on every boot;
-# HALOGEN_KV_POOL_FIT lowers the pool itself if the budget refuses the
-# configured one, and a green health check does not mean the pool you asked
-# for is the pool you got.
-Environment=HALOGEN_KV_POOL_POSITIONS=400384
-# Concurrency cap: two, one per half of the pool. Each slot is 111.0 MiB of
-# O(1) state, so this hands ~111 MiB back against the three set on 2026-09-12
-# for the fleet size of the time.
-Environment=HALOGEN_KV_SLOTS=2
-# Sizing is budget arithmetic, all of it. ONE budget covers the KV pool, the
-# per-call prefill arena and the vision tower together:
-#
-#   budget = 121.2 host - 67.7 resident weights - reserve
-#   need   = KV (~28 KiB/position) + arena + vision
-#   arena  = 3.80 GiB + 8.40 GiB x (HALOGEN_MAX_TOK / 16384)
-#
-# The arena term is linear in max_tok, and it reproduces five live restarts
-# AND upstream's three published "on the device" figures to within 0.1 GiB —
-# note those published figures are pool+arena totals, not pool alone, which is
-# the misreading that first made a doubling look affordable here. Corners this
-# box offered at a reserve of 24, at the old 262,144-position pool:
-#
-#   16384 -> need 28.6 vs 29.5  FITS, 0.9 spare   (doubling, 8-9% prefill cost)
-#   32768 -> need 37.0 vs 29.5  POOL CLAMPED to 262144, deploy still green
-#   65536 -> need 46.5 vs 29.5  OOM, will not boot; unit still reads ACTIVE
-#
-# LOWERED 32768 -> 24576 on 2026-09-16 for the frank-small co-tenant on this
-# same host. The arena's variable term goes 8.40 x 2 -> 8.40 x 1.5, handing
-# ~4.2 GiB back to the host at roughly a 4-5 % prefill cost -- the same lever
-# at 16,384 costs 8-9 %. The vendor's own notes name 24,576 as the largest
-# arena that allocated with headroom to spare, so this stays inside the range
-# it has tested.
-#
-# The arena is a PER-CALL PREFILL CHUNK, not a prompt ceiling: the
-# 200,192-token conversation is prefilled in arena-sized pieces, so a smaller
-# arena means more pieces and a slower prefill, not a smaller conversation.
-#
-# THE POOL, ARENA AND RESERVE ARE A MATCHED SET, NOT independent tunables --
-# the fit check's printed ledger on each boot is the arbiter, not these
-# comments.
-Environment=HALOGEN_MAX_TOK=24576
-# RAM the fit check refuses to count on, watched over by the engine's own
-# n-gram lookup table (47.7 GiB, read through the page cache and never held)
-# and, since 2026-09-16, by frank-small on the same host.
-#
-# RAISED 15 -> 20 with the 2026-09-16 retune. This knob feeds the fit-check
-# budget and is NOT an allocation: raising it cannot hand memory back to the
-# host, because the actual footprint is weights + KV + arena + vision, none of
-# which it touches — it can only take the pool away. What it does is keep the
-# fit check honest at the smaller arena. With the arena at 24,576 and no
-# vision tower, the engine's printed need falls to roughly 25 GiB against a
-# budget of 121.2 - 67.7 - 20 = 33.5, leaving ~8 GiB of margin, so the pool is
-# not clamped.
-#
-# The 2026-09-13 profile for comparison: 103.5 GiB held and "7.1 GiB left for
-# everything else" (310 contiguous 2 MiB blocks) at the doubled 524,288 pool,
-# which is why the pool came down; at 401,408 the same ledger read 9.1 GiB /
-# 1396 blocks. The 09-16 retune gives back the vision tower (1.97) and ~4.2 of
-# arena on top of that for the co-tenant, so the expected figure at this boot
-# is ~15 GiB left. Read the ledger; do not trust this arithmetic.
-#
-# To free host memory, change the POOL or the ARENA, and the reserve then
-# becomes free to rise: HALOGEN_KV_POOL_POSITIONS=262144 hands back 3.6 GiB,
-# HALOGEN_MAX_TOK=16384 hands back another 4.2. Both are one-line edits the
-# fit check confirms on the next boot.
-Environment=HALOGEN_HOST_RESERVE_GIB=20
-# Verbose engine logging: the vendor's first-line evidence for load,
-# allocation and admission questions. Shipping it on means an incident report
-# already has the full startup account instead of needing a restart-into-debug.
-# HALOGEN_DMALLOC_LOG is deliberately NOT set — it logs every allocation
-# over 64 MB, which is steady-state noise here; raise it only for a live
-# OOM hunt.
-Environment=HALOGEN_VERBOSE=1
-# Required by the image: the engine maps the checkpoint and registers it with
-# the GPU in place, which needs host IPC and an unconfined seccomp profile.
-PodmanArgs=--ipc=host --security-opt seccomp=unconfined
-Ulimit=memlock=-1:-1
-# The image ships its own HEALTHCHECK; this pins it. /health asks the ENGINE
-# for a PONG before answering, so 200 means "ready to serve", not "process
-# alive". During the first-start download it stays unhealthy by design.
-HealthCmd=halogen-healthcheck api
-HealthInterval=30s
-HealthTimeout=10s
-HealthRetries=3
-HealthStartPeriod=300s
-Pull=never
-
-[Service]
-# The first start reads ~68 GiB off disk and pins it before serving.
-TimeoutStartSec=900
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=default.target

dankclaw/observability/quadlet/alertmanager.container

@@ -0,0 +1,54 @@
+# Quadlet unit: alertmanager (rootless, boot-persistent).
+#
+# Install path on dankclaw: ~/.config/containers/systemd/alertmanager.container
+# Unit name: alertmanager.service (container name: alertmanager)
+#
+# The delivery half of the LLM alert rules. Until 2026-09-17 the rules in
+# shared/prometheus/rules/llm-fleet.yml evaluated and reached nobody — a
+# firing alert was visible in Prometheus and on the Grafana dashboard and
+# nowhere else, and an undelivered alert looks identical to a quiet one.
+# This unit closes that gap: Prometheus pushes to alertmanager:9093 over the
+# dankclaw_default network, and the single receiver webhooks the host-side
+# signal-relay user unit, which hands the text to signal-cli under Frank's
+# existing Signal account.
+#
+# PUBLISHING. 127.0.0.1 only, like every other observability unit: the
+# unauthenticated Alertmanager API (lifecycle endpoints are always on in
+# this binary — there is no flag to gate them) has no business on a host
+# interface, and no off-host consumer needs it. Caddy gains no vhost for
+# it; the homepage timer probes loopback.
+#
+# CONFIG. /opt/alertmanager/config/alertmanager.yml is installed from
+# shared/prometheus/alertmanager.yml by deploy-remote.sh on every deploy, so
+# the repo copy is the source of truth. STATE lives in /opt/alertmanager/data.
+#
+# Rootless: container root maps to the invoking user (dank) in this user
+# namespace and dank owns /opt/alertmanager/data. The image's default user
+# (nobody) maps to a subuid that cannot write it — same User=0:0 as prometheus.
+
+[Unit]
+Description=Alertmanager alert delivery (Dankclaw)
+After=network-online.target
+Wants=network-online.target
+
+[Container]
+ContainerName=alertmanager
+Image=docker.io/prom/alertmanager:latest
+Network=dankclaw_default
+User=0:0
+Exec=--config.file=/etc/alertmanager/alertmanager.yml --storage.path=/alertmanager
+PublishPort=127.0.0.1:9093:9093
+Volume=/opt/alertmanager/config:/etc/alertmanager
+Volume=/opt/alertmanager/data:/alertmanager
+HealthCmd=wget --spider -q http://localhost:9093/-/healthy
+HealthInterval=30s
+HealthTimeout=10s
+HealthRetries=3
+HealthStartPeriod=30s
+
+[Service]
+Restart=always
+RestartSec=10
+
+[Install]
+WantedBy=default.target

dankclaw/observability/quadlet/prometheus.container

@@ -27,10 +27,11 @@
 #
 # RULES. Five LLM alert rules and one recording rule load from
 # /etc/prometheus/rules/ (installed from shared/prometheus/rules/ by the
-# deployer). There is still **no Alertmanager**, so nothing is delivered: a
-# firing alert is visible in Prometheus and on the Grafana dashboard's ALERTS
-# panel and reaches nobody. Delivery is an operator decision, not an oversight;
-# the fragility is that an undelivered alert looks identical to a quiet one.
+# deployer). Delivery went live 2026-09-17: the alerting block in
+# prometheus.yml points at alertmanager:9093, which webhooks every firing
+# group to the host-side Signal relay (192.168.0.143:18093, reached from
+# the container as host.containers.internal), which sends through signal-cli account 318311.
+# The Grafana dashboard's ALERTS panel remains the second surface.
 #
 # The config volume is also why rules belong in git: editing them in the UI is
 # lost on the next deploy, since the deployer overwrites the whole directory.

dankclaw/open-webui/quadlet/open-webui.container

@@ -32,6 +32,52 @@ PublishPort=3005:8080
 Environment=OPENAI_API_BASE_URL=http://192.168.0.143:7770
 Environment=ANONYMIZED_TELEMETRY=False
 Environment=DO_NOT_TRACK=True
+
+# SSO via authentik (OIDC authorization-code flow, 2026-09-24). The OAuth
+# client (application slug "open-webui") lives in authentik; the client
+# SECRET stays off-repo in the git-ignored host env file (mode 600). The
+# container reaches the IdP over the public URL: Caddy publishes 0.0.0.0:443
+# and Pi-hole resolves auth.franksdank.cloud to the host LAN IP, so the
+# server-side code exchange works from this bridge. Roles come from the
+# "Open WebUI roles" scope mapping (entitlements "Open WebUI Users" /
+# "Open WebUI Admins"); OAUTH_MERGE_ACCOUNTS_BY_EMAIL folds SSO logins into
+# the existing local account (frank8465@protonmail.com). The OAuth vars are
+# read directly from the environment at runtime (the per-key config DB holds
+# only Admin-Panel overrides, so nothing is seeded on the live data dir).
+# NOTE: the OAUTH_SCOPES value MUST stay quoted — systemd splits unquoted
+# Environment= values on whitespace (unquoted, only "openid" survived).
+EnvironmentFile=/home/dank/Containers/open-webui/env
+Environment=OAUTH_CLIENT_ID=KmtzKc0bjWjaIFeM0puYudI1CLXR2p0jnPCSf4N0
+Environment=OAUTH_PROVIDER_NAME=authentik
+Environment=OPENID_PROVIDER_URL=https://auth.franksdank.cloud/application/o/open-webui/.well-known/openid-configuration
+Environment=OPENID_REDIRECT_URI=https://ai.franksdank.cloud/oauth/oidc/callback
+Environment=WEBUI_URL=https://ai.franksdank.cloud
+Environment=ENABLE_OAUTH_SIGNUP=true
+Environment=OAUTH_MERGE_ACCOUNTS_BY_EMAIL=true
+Environment="OAUTH_SCOPES=openid email profile roles"
+Environment=ENABLE_OAUTH_ROLE_MANAGEMENT=true
+# SSO-only (2026-09-24): the OIDC round-trip is verified, so the local
+# username/password path is retired. ENABLE_LOGIN_FORM hides the form;
+# ENABLE_PASSWORD_AUTH refuses password logins entirely, so the only way in
+# is the authentik OIDC flow (which folds into the local account by email).
+# The per-key config DB override (ui.enable_login_form) is flipped to false
+# in the same change — the DB wins over env, so both must agree.
+Environment=ENABLE_LOGIN_FORM=false
+Environment=ENABLE_PASSWORD_AUTH=false
+
+# Web search -> SearXNG, reached over the host-published port (NOT the
+# search.franksdank.cloud vhost): the Anubis->Authentik guard chain is a
+# browser mechanism and a plain HTTP client cannot solve it. The internal
+# port bypasses Caddy entirely. These env vars SEED the per-key config DB on
+# first run (fill-missing only); once the keys exist in /app/backend/data the
+# DB is authoritative, so this block documents the intent and makes a fresh
+# data dir come up wired. SearXNG's limiter exempts this source (the bridge
+# egress 10.89.x is in pass_ip), so the 4-req/IP/hour JSON cap never applies.
+Environment=ENABLE_WEB_SEARCH=True
+Environment=WEB_SEARCH_ENGINE=searxng
+Environment=SEARXNG_QUERY_URL=http://192.168.0.143:8081/search
+Environment=SEARXNG_LANGUAGE=all
+Environment=WEB_SEARCH_RESULT_COUNT=6
 Volume=/home/dank/Containers/open-webui/data:/app/backend/data
 
 # App liveness (/health) AND network liveness (a non-loopback interface must

dankserver/.AGENTTRASH/20260916-vllm-radiance/frank-medium-healthcheck.py

@@ -1,21 +0,0 @@
-#!/usr/bin/env python3
-"""frank-medium container healthcheck: the port's /health must answer 200.
-
-Runs INSIDE the container. The vLLM /health endpoint only answers once the
-engine is up (weights loaded, cudagraphs captured), so a healthy container is
-a serving container, not merely a live process. PORT selects the endpoint so
-the single-GPU replicas (:7771/:7772) and the round-robin proxy (:7770) can
-share this probe. The HealthCmd in the quadlet splits on spaces into an argv
-array (podman 5.x v1 generator), which is why this lives in a file rather
-than an inline python one-liner.
-"""
-import os
-import sys
-import urllib.request
-
-port = os.environ.get("PORT", "7770")
-try:
-    with urllib.request.urlopen(f"http://127.0.0.1:{port}/health", timeout=5) as r:
-        sys.exit(0 if r.status == 200 else 1)
-except Exception:
-    sys.exit(1)

dankserver/.AGENTTRASH/20260916-vllm-radiance/frank-medium.container

@@ -1,147 +0,0 @@
-# Quadlet unit: frank-medium (rootless, boot-persistent).
-#
-# Install path on dankserver: ~/.config/containers/systemd/frank-medium.container
-# Unit name: frank-medium.service (container name: frank-medium)
-#
-# The dankserver LLM endpoint (:7770, alias frank-medium), since 2026-09-15
-# running the vllm-radiance MXFP4 stack (stilldeadcode/vllm-radiance:0.9.3
-# image + the ggz14/radiance-vllm-mxfp4 patches) serving
-# just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq on the 2x Radeon AI PRO
-# R9700 (gfx1201) cards. It replaced the charlie12345/ROCmFPX llama.cpp
-# deployment, which served the same alias from the Q8_0_ROCMFPX_AGENT GGUF;
-# the MXFP4 stack is tensor-parallel (TP=2, R4D one-shot all-reduce) with the
-# DFlash2-FP8 block-diffusion drafter (SPEC=7) — the measured production
-# configuration of the upstream repo, on our exact hardware signature
-# (2x7551-32624, see the KV pin below).
-#
-# SHAPE. The launcher the upstream repo ships binds the server with
-# --network=host (vLLM takes --host 0.0.0.0 --port 7770 directly), which is
-# what this unit does too — no PublishPort. The fleet contract is unchanged:
-# :7770, served model name frank-medium (SERVED_NAMES), OpenAI /v1.
-#
-# NOT INSTALLED ON THIS HOST. This is the target shape: one tensor-parallel
-# process across both R9700s. It cannot launch a collective here — RCCL's
-# kernels need PCIe atomics and the second card sits behind chipset bridges
-# advertising `AtomicOpsCap: Routing-` — so the active deployment is the
-# two-replica trio in this directory (frank-medium-a/b/proxy) and
-# deploy-remote.sh deliberately does not install this unit. Keep it for a
-# host whose second card is on an atomic-capable root port; the full evidence
-# is in serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md.
-#
-# DEVICES. /dev/kfd + /dev/dri through the amdgpu/KFD stack; HIP/ROCR device
-# indices 0,1 (the two R9700s). The 2 GiB Raphael iGPU enumerates as index 2
-# and is excluded both by the index pin and by the upstream MIN_GPU_MIB=8192
-# logic this unit replaces. --privileged --ipc=host --security-opt
-# seccomp=unconfined --cap-add SYS_PTRACE is the upstream launcher's exact
-# podman posture.
-#
-# MOUNTS. /models (rw checkpoints), the HF cache (tokenizer files),
-# /cache (Triton/inductor/aiter/vLLM compile cache — first start compiles for
-# several minutes, later starts reuse), /patches (the pinned radiance clone —
-# its patches are applied at EVERY container start by the entrypoint), /r4d
-# (the patched libr4d build; the image's shipped r4d.so NaNs the
-# gated-delta-net and must never be used), and the entrypoint + healthcheck
-# scripts from this checkout (mounted read-only straight out of
-# /home/dank/serverAppConfigs — deploy.sh git-pulls the checkout before
-# dispatching).
-#
-# IMAGE. The unit runs the localhost/-aliased tag: this host's podman quadlet
-# path cannot resolve registry-prefixed refs from local storage at create
-# time (same as frank-large on dankclaw). deploy-remote.sh pulls the
-# docker.io ref and re-tags it before the restart. Pull=never keeps boots off
-# the registry.
-#
-# KEEP IN SYNC: the radiance clone is pinned (97353293...); if it is updated,
-# re-diff dankserver/frank-medium/radiance-entrypoint.sh against the new
-# serve-mxfp4.sh container command and bump the pin deliberately.
-#
-# DO NOT name an environment variable TEMP, HOST or GPU_MAX_HEAP_SIZE: the
-# HIP runtime reads bare TEMP for its JIT cache path (2026-08-13 incident).
-[Unit]
-Description=frank-medium — vllm-radiance MXFP4 Qwen3.8-27B-Uncensored on 2x R9700 (gfx1201)
-After=network-online.target
-
-[Container]
-ContainerName=frank-medium
-Image=localhost/radiance-vllm:0.9.3
-# vLLM binds 0.0.0.0:7770 on the host network directly; no PublishPort.
-AddDevice=/dev/kfd
-AddDevice=/dev/dri
-GroupAdd=keep-groups
-# The upstream launcher's exact podman posture (rootless; podman-only flags
-# like --replace are not needed under systemd management).
-PodmanArgs=--privileged --ipc=host --security-opt seccomp=unconfined --cap-add SYS_PTRACE --network=host
-# The two R9700s. The Raphael iGPU (index 2, 2 GiB) stays out: shared system
-# memory is not a compute target.
-Environment=ROCR_VISIBLE_DEVICES=0,1
-Environment=HIP_VISIBLE_DEVICES=0,1
-# The uncensored MXFP4 checkpoint ships its fp8 MTP head already, so no
-# fp8_mtp.py rewrite step exists or is needed (unlike amd/Qwen3.8-27B-Quark-AWQ-MXFP4).
-Environment=SNAP=/models/Qwen3.8-27B-Uncensored-MXFP4-awq
-Environment=DRAFTER=/models/Qwen3.8-27B-DFlash2-FP8
-Environment=CHAT_TEMPLATE=/patches/qwen-fixed-v22.3.jinja
-# Fleet identity: the id at /v1/models and in every response body. Every
-# client (Hermes, the harness, services.json) points at frank-medium.
-Environment=SERVED_NAMES=frank-medium
-Environment=PORT=7770
-# Fixed 2-card tensor parallel: the tested configuration of the upstream
-# repo; the R4D one-shot all-reduce kernel serves it natively.
-Environment=TP=2
-# Measured production batch shape (upstream README): MAXSEQS 8 peaks
-# aggregate throughput at 573 t/s; MAXLEN is the model's native context.
-Environment=MAXSEQS=8
-Environment=CHUNK=8192
-Environment=MAXLEN=262144
-Environment=GPU_UTIL=0.98
-# Speculative decoding: the DFlash2 drafter at depth 7 is the configuration
-# every published number (224 t/s single-stream decode, 4749-4810 t/s
-# prefill) was measured with.
-Environment=SPEC_METHOD=dflash
-Environment=SPEC=7
-# KV cache pin, bytes. From the upstream repo's shipped kv-profiles.tsv row
-#   sig=2x7551-32624 maxseqs=8 chunk=8192 maxlen=262144 spec=dflash
-# which is EXACTLY this host (2x R9700 DID 0x7551, 32624 MiB each). Pinned
-# explicitly because this unit does not run the launcher's lookup; valid
-# only at this batch shape (change MAXSEQS/CHUNK/SPEC_METHOD => recalibrate
-# with calibrate-kv.sh or set KV_MEM=0 to let vLLM profile). Worth
-# 943,581 KV tokens vs ~892,799 profiled (~5.7%).
-Environment=KV_MEM=18563072000
-# Checkpoints (rw: nothing downloads at runtime, but vLLM mmaps the shards;
-# keep parity with the launcher's rw mount).
-Volume=/home/dank/models:/models
-# Tokenizer files ride the HF cache; HF_HUB_OFFLINE=1 is set in the
-# entrypoint.
-Volume=/home/dank/.cache/huggingface:/root/.cache/huggingface
-# Compile cache. Keyed on model + stack version + every graph-changing knob;
-# never share it across configurations (upstream rule).
-Volume=/home/dank/.radiance-cache-w4a8-093-gdnm-nqft-fp8s-gnq:/cache
-# The pinned radiance clone: patches applied at every container start.
-Volume=/home/dank/GitPackages/radiance-vllm-mxfp4:/patches:z
-# The patched libr4d (pin b9e42ab + repo extras, key b9e42ab-rx6). The
-# image's shipped r4d.so predates the GDN overflow fix and NaNs this model.
-Volume=/home/dank/.cache/radiance-libr4d/b9e42ab-rx6:/r4d:z
-# Entrypoint + healthcheck from this checkout, mounted read-only.
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/radiance-entrypoint.sh:/usr/local/bin/frank-medium-entrypoint.sh:ro
-Volume=/home/dank/serverAppConfigs/dankserver/frank-medium/frank-medium-healthcheck.py:/usr/local/bin/frank-medium-healthcheck.py:ro
-# The entrypoint must be the container command; the image's own entrypoint
-# (which the patched script execs at the end) must not run unpatched.
-PodmanArgs=--entrypoint /usr/local/bin/frank-medium-entrypoint.sh
-# 200 only once the engine serves; during first-start compilation the
-# container is legitimately unhealthy. HealthCmd is split on spaces into an
-# argv array by the podman 5.x v1 generator, hence the file-based probe.
-HealthCmd=python3 /usr/local/bin/frank-medium-healthcheck.py
-HealthInterval=60s
-HealthTimeout=30s
-HealthRetries=3
-HealthStartPeriod=1800s
-Pull=never
-
-[Service]
-# First start compiles Triton/inductor kernels (several minutes) before the
-# engine loads ~9.5 GiB of weights per card.
-TimeoutStartSec=1800
-Restart=on-failure
-RestartSec=10
-
-[Install]
-WantedBy=default.target

dankserver/.AGENTTRASH/20260916-vllm-radiance/radiance-entrypoint.sh

@@ -1,229 +0,0 @@
-#!/usr/bin/env bash
-# frank-medium container entrypoint: vllm-radiance MXFP4 on 2x Radeon AI PRO
-# R9700 (gfx1201), serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq.
-#
-# This is a fork of the patch-and-serve block inside radiance-vllm-mxfp4's
-# serve-mxfp4.sh (clone at /home/dank/GitPackages/radiance-vllm-mxfp4, pinned
-# commit 9735329348ae1c9319f01cbe5f7004ec5f4dba63). The published image alone
-# is NOT sufficient for this model: the shipped libr4d predates the
-# gated-delta-net overflow fix and NaNs the output, and the MXFP4 gates need
-# relaxing, so the repo's patches are applied at every container start from
-# the /patches mount. It runs INSIDE the container (executed as the image
-# command by the frank-medium quadlet); the launcher's host-side GPU/TP/KV
-# detection is not needed here because this fleet is fixed 2x gfx1201 and the
-# KV pin is pinned explicitly in the quadlet.
-#
-# KEEP IN SYNC: if the radiance clone is updated, re-diff this file against
-# the new serve-mxfp4.sh container command and against the DRY_RUN output.
-#
-# Env contract: every knob below is ${VAR:-default}; the quadlet sets the
-# fleet-specific ones (SNAP, PORT, SERVED_NAMES, TP, KV_MEM, ...).
-set -euo pipefail
-
-# ---------------------------------------------------------------- serving shape
-: "${SNAP:=/models/Qwen3.8-27B-Uncensored-MXFP4-awq}"
-: "${DRAFTER:=/models/Qwen3.8-27B-DFlash2-FP8}"
-: "${CHAT_TEMPLATE:=/patches/qwen-fixed-v22.3.jinja}"
-: "${PORT:=7770}"
-: "${SERVED_NAMES:=frank-medium}"
-: "${MAXLEN:=262144}"
-: "${MAXSEQS:=8}"
-# KV_MEM=0 is passed THROUGH as --kv-cache-memory 0, which the image's
-# patched KV sizing reads as "profile the shape" (the launcher's rule).
-# Blanking the variable instead omits the flag and makes vLLM size the cache
-# from --gpu-memory-utilization alone: measured on the single-GPU replicas
-# that overshoots and the allocation dies with torch.OutOfMemoryError right
-# after the weights load (2.16 GiB requested, 116 MiB free). Do not "fix"
-# this into an empty expansion.
-: "${KV_MEM:=18563072000}"   # shipped kv-profiles.tsv row 2x7551-32624 @ 8/8192/262144/dflash
-: "${SPEC_METHOD:=dflash}"
-if [ "$SPEC_METHOD" = dflash ]; then : "${SPEC:=7}"; else : "${SPEC:=4}"; fi
-: "${MIN_M:=0}"              # 0 = the W4A8 kernel serves every M (MIN_M=1 still routes M=1 to aiter)
-: "${R4D_ATTN:=1}"
-: "${ASYNC:=0}"
-
-if [ "$R4D_ATTN" = "1" ]; then ATTN=R4D; else ATTN=ROCM_AITER_UNIFIED_ATTN; fi
-if [ "$ASYNC" = 1 ]; then ASYNC_FLAG="--async-scheduling"; UNPAD=false; else
-  ASYNC_FLAG="--no-async-scheduling"; UNPAD=true; fi
-
-# Drafting config. dflash: the DFlash2-FP8 block-diffusion drafter (its
-# attention backend must support full cuda graphs -> TRITON_ATTN). The
-# disable_padded_drafter_batch unpad is the single-stream lever (~+50%).
-if [ "$SPEC_METHOD" = dflash ]; then
-  if [ ! -f "$DRAFTER/config.json" ]; then
-    echo "frank-medium: no dflash drafter at $DRAFTER" >&2
-    exit 1
-  fi
-  DRAFT_SAMPLE=${DRAFT_SAMPLE:-greedy}
-  SPEC_CFG="{\"method\":\"dflash\",\"model\":\"$DRAFTER\",\"num_speculative_tokens\":$SPEC,\"attention_backend\":\"TRITON_ATTN\",\"disable_padded_drafter_batch\":$UNPAD,\"draft_sample_method\":\"$DRAFT_SAMPLE\"}"
-  RADIANCE_DRAFT_RERANK=${RADIANCE_DRAFT_RERANK:-80}
-  RADIANCE_VERIFY_HEAD=${RADIANCE_VERIFY_HEAD:-1}
-else
-  SPEC_CFG="{\"method\":\"mtp\",\"num_speculative_tokens\":$SPEC,\"attention_backend\":\"$ATTN\",\"disable_padded_drafter_batch\":$UNPAD}"
-  RADIANCE_DRAFT_RERANK=${RADIANCE_DRAFT_RERANK:-32}
-  RADIANCE_VERIFY_HEAD=${RADIANCE_VERIFY_HEAD:-0}
-fi
-
-# The AR size gate compares raw bf16 bytes: CHUNK x hidden(5120) x 2. Derived
-# so changing CHUNK cannot silently drop prefill back onto RCCL.
-export RADIANCE_AR_MAX_KB=$(( (CHUNK * 5120 * 2) / 1024 + 4096 ))
-
-# ---------------------------------------------------------------- runtime env
-export HF_HUB_OFFLINE=1
-export VLLM_LOGGING_LEVEL=${VLLM_LOGGING_LEVEL:-INFO}
-export VLLM_NO_USAGE_STATS=${VLLM_NO_USAGE_STATS:-1}
-export VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION=1
-export VLLM_ROCM_USE_AITER_MHA=0 VLLM_ROCM_USE_AITER_MLA=0 VLLM_ROCM_USE_AITER_MOE=0
-export VLLM_ROCM_USE_AITER_LINEAR=0 VLLM_ROCM_USE_AITER_FP8BMM=0
-export VLLM_ROCM_USE_AITER_FP4BMM=0 VLLM_ROCM_USE_AITER_RMSNORM=0
-export NCCL_PROTO=Simple
-export RADIANCE_USE_R4D=${RADIANCE_USE_R4D:-1} RADIANCE_USE_R4D_AR=${RADIANCE_USE_R4D_AR:-1}
-export RADIANCE_DFLASH_SELECTOR_TOPK="${RADIANCE_DFLASH_SELECTOR_TOPK:-}"
-export RADIANCE_R4D_REPORT=1
-export RADIANCE_PRESHUFFLE=${RADIANCE_PRESHUFFLE:-1}
-export RADIANCE_MXFP4_SANITIZE="${RADIANCE_MXFP4_SANITIZE:-0}"
-export RADIANCE_MXFP4=1 RADIANCE_MXFP4_W4A8=1 RADIANCE_MXFP4_W4A8_MIN_M="$MIN_M"
-export RADIANCE_FAST_DRAFT=${RADIANCE_FAST_DRAFT:-1}
-export RADIANCE_DRAFT_TAU=${RADIANCE_DRAFT_TAU:-0.20}
-export RADIANCE_DRAFT_RERANK
-export RADIANCE_VERIFY_HEAD
-export RADIANCE_VERIFY_HEAD_MAX_M=${RADIANCE_VERIFY_HEAD_MAX_M:-32}
-export RADIANCE_MXFP4_DEBUG=${RADIANCE_MXFP4_DEBUG:-0}
-export RADIANCE_MXFP4_PUREQUANT=${RADIANCE_MXFP4_PUREQUANT:-0}
-export RADIANCE_MXFP4_SYNC=${RADIANCE_MXFP4_SYNC:-0}
-export RADIANCE_MXFP4_CLONE=${RADIANCE_MXFP4_CLONE:-0}
-export RADIANCE_MXFP4_CHECKX=${RADIANCE_MXFP4_CHECKX:-0}
-export RADIANCE_MXFP4_PADOUT=${RADIANCE_MXFP4_PADOUT:-0}
-export RADIANCE_MXFP4_TN4_MIN_M=${RADIANCE_MXFP4_TN4_MIN_M:-2048}
-# The decode-kernel band must cover MAXSEQS x (SPEC+1) rows: 64 covers 8x8.
-if [ "${MAXSEQS}" -gt 8 ]; then
-  export RADIANCE_MXFP4_DECODE_MAX_M=${RADIANCE_MXFP4_DECODE_MAX_M:-128}
-fi
-export RADIANCE_MXFP4_DECODE_NT=${RADIANCE_MXFP4_DECODE_NT:-1}
-export RADIANCE_MXFP4_A_TILED_MIN_M=${RADIANCE_MXFP4_A_TILED_MIN_M:-513}
-export RADIANCE_MXFP4_WPERM=${RADIANCE_MXFP4_WPERM:-1}
-export RADIANCE_TP_PAD=${RADIANCE_TP_PAD:-0}   # TP=2 here; 3 is the only non-zero value upstream supports
-export RADIANCE_TP_PAD_DRAFTER=${RADIANCE_TP_PAD_DRAFTER:-1}
-export RADIANCE_TP_PAD_STRICT=${RADIANCE_TP_PAD_STRICT:-1}
-export RADIANCE_GDN_MERGE_INPROJ=${RADIANCE_GDN_MERGE_INPROJ:-1}
-export RADIANCE_GDN_NORM_QUANT=${RADIANCE_GDN_NORM_QUANT:-1}
-export RADIANCE_GDN_STRIDED_GATES=${RADIANCE_GDN_STRIDED_GATES:-0}
-export RADIANCE_GDN_EMPTY_OUT=${RADIANCE_GDN_EMPTY_OUT:-0}
-export R4D_ATTN_FP8=${R4D_ATTN_FP8:-3}
-export RADIANCE_DFLASH_CALIB="${RADIANCE_DFLASH_CALIB:-}"
-export RADIANCE_DFLASH_CALIB_TOKENS="${RADIANCE_DFLASH_CALIB_TOKENS:-200000}"
-export RADIANCE_MXFP4_SHADOW="${RADIANCE_MXFP4_SHADOW:-}"
-export RADIANCE_MXFP4_KERNEL_N="${RADIANCE_MXFP4_KERNEL_N:-}"
-export RADIANCE_MXFP4_KERNEL_NK="${RADIANCE_MXFP4_KERNEL_NK:-}"
-export RADIANCE_MXFP4_CHECKALL="${RADIANCE_MXFP4_CHECKALL:-}"
-export RADIANCE_MXFP4_DECODE_KS="${RADIANCE_MXFP4_DECODE_KS:-}"
-export RADIANCE_MXFP4_DECODE_BK="${RADIANCE_MXFP4_DECODE_BK:-}"
-export RADIANCE_MXFP4_PERBLOCK_NK="${RADIANCE_MXFP4_PERBLOCK_NK:-}"
-export RADIANCE_MXFP4_REFLINEAR="${RADIANCE_MXFP4_REFLINEAR:-0}"
-export PYTORCH_CUDA_ALLOC_CONF="${PYTORCH_CUDA_ALLOC_CONF:-}"
-export RADIANCE_MXFP4_EPIFAST="${RADIANCE_MXFP4_EPIFAST:-1}"
-export RADIANCE_AR_OVERLAP=${RADIANCE_AR_OVERLAP:-0}
-export RADIANCE_GDN_FUSED_UPDATE=${RADIANCE_GDN_FUSED_UPDATE:-1}
-export RADIANCE_DYNAMIC_WIDTH=${RADIANCE_DYNAMIC_WIDTH:-1}
-export RADIANCE_DYNW_ALPHA=${RADIANCE_DYNW_ALPHA:-0.35}
-export RADIANCE_DYNW_MARGIN=${RADIANCE_DYNW_MARGIN:-2}
-export RADIANCE_DYNW_MIN=${RADIANCE_DYNW_MIN:-2}
-export RADIANCE_DYNW_MIN_BATCH=${RADIANCE_DYNW_MIN_BATCH:-3}
-export RADIANCE_AR_QNB=${RADIANCE_AR_QNB:-96}
-export RADIANCE_AR_QNT=${RADIANCE_AR_QNT:-1024}
-export MXFP4_CUMODE=${MXFP4_CUMODE:-0}
-export RADIANCE_STEP_TRACE=${RADIANCE_STEP_TRACE:-0}
-export RADIANCE_AR_OVERLAP_MIN_M=${RADIANCE_AR_OVERLAP_MIN_M:-2048}
-export RADIANCE_AR_OVERLAP_SLICES=${RADIANCE_AR_OVERLAP_SLICES:-4}
-export RADIANCE_MXFP4_EPIFAST=${RADIANCE_MXFP4_EPIFAST:-1}
-export RADIANCE_MXFP4_R4D_DECODE_MAX_M=${RADIANCE_MXFP4_R4D_DECODE_MAX_M:-0}
-export RADIANCE_TOPK_TRITON_MIN_ROWS=${RADIANCE_TOPK_TRITON_MIN_ROWS:-1}
-export RADIANCE_SKINNY_GEMM=${RADIANCE_SKINNY_GEMM:-1}
-export RADIANCE_MXFP4_HOIST_QUANT=${RADIANCE_MXFP4_HOIST_QUANT:-1}
-export RADIANCE_MXFP4_TRACED_QUANT=${RADIANCE_MXFP4_TRACED_QUANT:-1}
-export RADIANCE_FP8_STREAM=${RADIANCE_FP8_STREAM:-1}
-export RADIANCE_RMS_QUANT_FUSION=${RADIANCE_RMS_QUANT_FUSION:-1}
-export RADIANCE_NVFP4_MXFP4=${RADIANCE_NVFP4_MXFP4:-0}
-export RADIANCE_NVFP4_EXP=${RADIANCE_NVFP4_EXP:-mse}
-export RADIANCE_NVFP4_FP8_LAYERS=${RADIANCE_NVFP4_FP8_LAYERS:-mxfp4}
-export RADIANCE_NVFP4_BF16_LAYERS=${RADIANCE_NVFP4_BF16_LAYERS:-in_proj_ba}
-export RADIANCE_NVFP4_LMHEAD=${RADIANCE_NVFP4_LMHEAD:-bf16}
-export RADIANCE_GDN_PATHS=${RADIANCE_GDN_PATHS:-both}
-export RADIANCE_GDN_NANTRACE=${RADIANCE_GDN_NANTRACE:-0}
-export RADIANCE_MXFP4_CHECK_MAX_M=${RADIANCE_MXFP4_CHECK_MAX_M:-128}
-export RADIANCE_MXFP4_MHIST=${RADIANCE_MXFP4_MHIST:-0}
-export VLLM_CACHE_ROOT=/cache/vllm
-export TORCHINDUCTOR_CACHE_DIR=/cache/inductor
-export TRITON_CACHE_DIR=/cache/triton
-export AITER_ROOT_DIR=/cache/aiter
-export TRITON_CACHE_AUTOTUNING=1
-# The patched libr4d built by setup-mxfp4.sh, bind-mounted at /r4d.
-R4D_SO=${R4D_SO:-/r4d}
-
-# The norm+quant fusion turns on vLLM's passes (one --compilation-config
-# instance; argparse keeps the last, so entries accumulate into one flag).
-CC_ITEMS=""
-if [ "${RADIANCE_NORMQUANT_FUSION:-1}" != "0" ]; then
-  CC_ITEMS="\"pass_config\":{\"fuse_norm_quant\":true,\"fuse_act_quant\":true}"
-fi
-EXTRA=${EXTRA:-}
-if [ -n "$CC_ITEMS" ]; then
-  EXTRA="$EXTRA --compilation-config {$CC_ITEMS}"
-fi
-
-# ---------------------------------------------------------------- patch block
-# Byte-for-byte the launcher's container-start block. The patches come from
-# the /patches mount (the pinned radiance clone). SP must match the image's
-# venv layout; the image tag is pinned in the quadlet.
-SP=/opt/vllm/lib/python3.12/site-packages
-cd /patches
-python3 patch_quark_mxfp4.py
-python3 patch_nvfp4_mxfp4.py
-python3 patch_tp3_pad.py
-python3 patch_ar_maxbytes.py
-python3 patch_topk_triton_rows.py
-python3 patch_dflash_calib.py
-python3 patch_dflash_mxfp4_kv.py
-python3 patch_rmsquant_fusion.py
-python3 patch_verify_head.py
-python3 patch_kv_group_size.py
-python3 patch_topk_composite.py
-python3 patch_gdn_shared_build.py
-python3 patch_dflash_selector_topk.py
-python3 patch_gdn_merge_inproj.py
-python3 patch_dynwidth.py
-python3 patch_async_dynwidth.py
-python3 patch_step_trace.py
-python3 patch_ar_geometry.py
-python3 patch_ar_3rank.py
-python3 patch_gdn_glue.py
-# Non-fatal: fixes content=null on thinking-off requests; not required to serve.
-python3 patch_qwen3_thinkoff.py \
-  || echo "[radiance] WARNING: thinkoff patch did not apply; thinking-off requests will return empty content"
-cp mxfp4-configs/*.json "$SP"/aiter/ops/triton/configs/gemm/
-cp radiance_preamble.py /opt/radiance_preamble.py
-cp radiance_nvfp4.py radiance_mxfp4.py radiance_gdn.py radiance_rmsquant.py radiance_drafthead.py \
-   radiance_verifyhead.py radiance_gdnmerge.py radiance_aroverlap.py radiance_topk.py \
-   radiance_arnq.py radiance_tp3pad.py "$SP"/
-hipcc -O3 -w -std=c++17 -fPIC -shared --offload-arch=gfx1201 $([ "${MXFP4_CUMODE:-0}" = 1 ] && echo -mcumode) \
-  $(python3 -m pybind11 --includes) radiance_mxfp4_fp8.hip -o "$SP"/radiance_mxfp4_fp8.so
-if [ -n "${R4D_SO:-}" ] && [ -f /r4d/r4d.so ]; then
-  cp /r4d/r4d.so "$SP"/r4d.so
-  echo "[radiance] using patched r4d.so from $R4D_SO"
-fi
-# Leave /patches before exec: a stale radiance_mxfp4_fp8.so there would shadow
-# the one just compiled into site-packages (cwd precedes it on sys.path).
-cd /
-exec /opt/radiance_entrypoint.sh "$SNAP" --served-model-name $SERVED_NAMES \
-  --host 0.0.0.0 --port "$PORT" \
-  --kv-cache-dtype fp8 --tensor-parallel-size "$TP" \
-  --gpu-memory-utilization "$GPU_UTIL" \
-  ${KV_MEM:+--kv-cache-memory "$KV_MEM"} \
-  --max-model-len "$MAXLEN" --max-num-seqs "$MAXSEQS" --max-num-batched-tokens "$CHUNK" \
-  --attention-backend "$ATTN" \
-  --speculative-config "$SPEC_CFG" \
-  $ASYNC_FLAG $EXTRA \
-  --enable-prefix-caching --mamba-cache-mode align --enable-auto-tool-choice \
-  --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
-  --override-generation-config '{"temperature":0.7,"top_p":0.95,"top_k":20}' \
-  --chat-template "$CHAT_TEMPLATE"

dankserver/calibre-server/quadlet/calibre-server.container

@@ -0,0 +1,63 @@
+# Quadlet unit: Calibre Content Server (rootless, boot-persistent).
+#
+# Install path on dankserver:
+# ~/.config/containers/systemd/calibre-server.container
+# Unit name: calibre-server.service (podman 5.x drops the container- prefix)
+#
+# The image is the CURRENT linuxserver/calibre (9.15.0), which is a desktop
+# image (Selkies web-streamed GUI on 8080/8181) with the headless
+# calibre-server available inside. The compose-era snippet in
+# serverAppConfigs-docs/Dankserver/Calibre.md predates the desktop revamp;
+# the dedicated headless images (ta264/docker-calibre, agentgreen) are
+# dormant (calibre 6.10 / 2017), so this unit runs calibre-server as the
+# image's s6-overlay main service via Exec=; the desktop also runs but its
+# ports are internal-only (see the port map below).
+#
+# Port map: Network=pasta is port-FORWARD mode on this host (the container
+# keeps its own netns; only PublishPort'd ports reach the host — verified
+# 2026-09-21: 8080/8090/8182 inside the container are not visible on the
+# host, only the 8084 forward). So the ONLY exposed surface is:
+#   8090 -> host 8084  calibre-server (the content server; 8080 inside the
+#                      container is the image's own nginx, 8083 on the host
+#                      is calibre-web)
+# The desktop GUI (nginx 8080/8181 + WS 8082, inside the container) runs but
+# is internal-only: no PublishPort. Exposing it is one PublishPort line away
+# if the browser GUI is ever wanted (the basic-auth credential is already in
+# the env file).
+# Auth: calibre-server 9.x keeps its users in a SQLite userdb (the pw column
+# is PLAINTEXT, so the file is mode 600). The single admin account is
+# bootstrapped once by the deploy handler (deploy_calibre_server) via
+# `calibre-server --manage-users -- add`, password piped on stdin with
+# printf (a trailing newline from echo lands in the stored password and
+# breaks every login). The desktop's basic auth (CUSTOM_USER/CUSTOM_PASSWORD)
+# reuses the same credential pair. Both live in the machine-local env file;
+# this unit holds no secrets.
+#
+# --auth-mode=basic is required: on a non-SSL port the server defaults to
+# Digest auth, which rejects plain Basic clients with 400 "Unsupported
+# authentication method".
+
+[Unit]
+Description=Calibre content server container
+After=network.target
+
+[Container]
+ContainerName=calibre-server
+Image=lscr.io/linuxserver/calibre:9.15.0
+Network=pasta
+EnvironmentFile=/home/dank/Containers/calibre-server/env
+Environment=PUID=1000
+Environment=PGID=1000
+Environment=TZ=America/Vancouver
+Volume=/home/dank/Containers/calibre-web/books:/books
+Volume=/home/dank/Containers/calibre-server/userdb.sqlite:/userdb.sqlite
+Exec=calibre-server --port 8090 --enable-auth --auth-mode=basic --userdb /userdb.sqlite /books
+PublishPort=8084:8090
+
+[Service]
+TimeoutStartSec=300
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=default.target

dankserver/calibre-web/quadlet/calibre-web.container

@@ -16,7 +16,7 @@ After=network.target
 
 [Container]
 ContainerName=calibre-web
-Image=lscr.io/linuxserver/calibre-web:0.6.27-ls397
+Image=lscr.io/linuxserver/calibre-web:0.6.27-ls402
 Network=pasta
 Environment=DOCKER_MODS=linuxserver/mods:universal-calibre
 Environment=LANG=en_US.UTF-8

dankserver/compose.yaml

@@ -127,7 +127,7 @@ services:
   # Calibre-Web — e-book web frontend
   # ═══════════════════════════════════════════════
   calibre-web:
-    image: lscr.io/linuxserver/calibre-web:0.6.27-ls397
+    image: lscr.io/linuxserver/calibre-web:0.6.27-ls402
     container_name: calibre-web
     restart: unless-stopped
     environment:

dankserver/plex/quadlet/plex.container

@@ -4,7 +4,7 @@ After=network-online.target
 Wants=network-online.target
 
 [Container]
-Image=lscr.io/linuxserver/plex:1.43.3.10896-cb3ebc72d-ls322
+Image=lscr.io/linuxserver/plex:1.43.4.10903-e5521bd8c-ls325
 ContainerName=plex
 Network=dankserver_appnet:alias=plex
 Environment=PUID=0

dankserver/tautulli/quadlet/tautulli.container

@@ -4,7 +4,7 @@ After=network-online.target
 Wants=network-online.target
 
 [Container]
-Image=lscr.io/linuxserver/tautulli:v2.18.1-ls242
+Image=lscr.io/linuxserver/tautulli:v2.18.1-ls245
 ContainerName=tautulli
 Network=dankserver_appnet:alias=tautulli
 Environment=PUID=0

old/download-frank-large-dflash.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+# Download the z-lab Qwen3.5-9B-DFlash drafter for frank-large's DFlash
+# speculative-decode arm (vLLM --speculative-config method=dflash). Public
+# repo — no auth header on the command line.
+#
+# The drafter is a 6-layer qwen3 draft whose dflash_config pins
+# target_layer_ids [1,5,9,13,17,21,25,29] and block_size 16; its hidden_size
+# 4096 / vocab_size 248320 match the MiMo-V2.6-Distill-Qwen-9B target exactly
+# (both preserve the Qwen3.5-9B geometry), which is what lets vLLM load it as
+# the draft for that target.
+#
+# Usage:
+#   scripts/download-frank-large-dflash.sh [DEST_DIR] [--verify-only]
+set -euo pipefail
+
+REPO="z-lab/Qwen3.5-9B-DFlash"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/Qwen3.5-9B-DFlash}"
+
+# name bytes sha256(- = non-LFS, size-only check)
+FILES=(
+  "model.safetensors 2583816465 0a42274b32554f48de1faa0d42824e9c2ceda649c30ae0a731cddf410dd698c7"
+  "config.json 1222 -"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"

old/download-frank-large-mimo.sh

@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+# Download the BF16 MiMo-V2.6-Distill-Qwen-9B payload for frank-large
+# (vLLM/kyuz0 gfx1151, Strix Halo iGPU). Public repo — no auth header on the
+# command line (a Bearer token in argv is world-readable via /proc).
+#
+# Per-file FRESH downloads, no resume: the house rule (see
+# download-frank-small.sh); -f/--fail-with-body plus size+sha256 against the
+# Hub LFS oid is what makes an access error fail instead of "succeeding".
+#
+# Usage:
+#   scripts/download-frank-large-mimo.sh [DEST_DIR] [--verify-only]
+#
+# The verified set below is exactly what the 2026-10-01 bench window served:
+# 17.53 GiB of BF16 weights (4 shards, 18,819,720,848 B total with the rest).
+set -euo pipefail
+
+REPO="XiaomiMiMo/MiMo-V2.6-Distill-Qwen-9B"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-BF16}"
+
+# name bytes sha256(- = non-LFS, size-only check)
+FILES=(
+  "model-00001-of-00004.safetensors 5276436216 aab052180118aee34abc3029b54eaa49096aac606b97d703866b420dceb703c3"
+  "model-00002-of-00004.safetensors 5033171280 7a0486565f06d25ac4628e9dba470dc3f604353471d240d5a0bf7128f64df396"
+  "model-00003-of-00004.safetensors 5234499504 6c73207563d1879bfd6c143a028cc70be68edff56280458c71a43df4240300f4"
+  "model-00004-of-00004.safetensors 3275613848 1379a7cf8c0b8555a39ab65a47e830e0eb45e776b46045734da3afd73c09eea2"
+  "model.safetensors.index.json 69217 -"
+  "config.json 2784 -"
+  "generation_config.json 126 -"
+  "chat_template.jinja 3916 -"
+  "tokenizer.json 19989325 06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523"
+  "tokenizer_config.json 1165 -"
+  "vocab.json 6722759 -"
+  "merges.txt 3353259 -"
+  "preprocessor_config.json 443 -"
+  "processor_config.json 1191 -"
+  "video_preprocessor_config.json 385 -"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"

old/download-frank-large-qwen35.sh

@@ -0,0 +1,85 @@
+#!/usr/bin/env bash
+# Download the source Qwen3.5-9B checkpoint — the ONLY place this model
+# family's MTP head (15 mtp.* tensors) exists. frank-large runs it as the
+# vLLM cross-model MTP draft (MiMo-V2.6-Distill-Qwen-9B itself ships zero
+# mtp tensors; that is why SGLang's NEXTN on frank-small draws 0.00
+# acceptance). Public repo — no auth header on the command line.
+#
+# NOTE the odd upstream shard names (model.safetensors-0000N-of-00004...).
+#
+# Usage:
+#   scripts/download-frank-large-qwen35.sh [DEST_DIR] [--verify-only]
+set -euo pipefail
+
+REPO="Qwen/Qwen3.5-9B"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/Qwen3.5-9B}"
+
+# name bytes sha256(- = non-LFS, size-only check)
+FILES=(
+  "model.safetensors-00001-of-00004.safetensors 5276436216 db6f444b43d318c92f360a13a25561a6a65b10c0631b8ed305a426dbaa6c380e"
+  "model.safetensors-00002-of-00004.safetensors 5335161512 31c7d7e2dd5d207840b31cc59083c8f4c4718959149e0358c0364052bb9a0330"
+  "model.safetensors-00003-of-00004.safetensors 5368717440 7ec36ba3a4176a44c3c0876ad80c56a2f70c84bf008d82e9501df642f17dadec"
+  "model.safetensors-00004-of-00004.safetensors 3325995712 b62b0c4cd7e44edee103ee8f4fe225f246d5e768e07bfd5f25b63a8aa1fdd0c6"
+  "model.safetensors.index.json 79657 -"
+  "config.json 3126 -"
+  "chat_template.jinja 7756 -"
+  "tokenizer.json 12807982 5f9e4d4901a92b997e463c1f46055088b6cca5ca61a6522d1b9f64c4bb81cb42"
+  "tokenizer_config.json 16710 -"
+  "vocab.json 6722759 -"
+  "merges.txt 3353259 -"
+  "preprocessor_config.json 390 -"
+  "video_preprocessor_config.json 385 -"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"

old/download-frank-primary-fp8.sh

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+# download-frank-primary-fp8.sh — fetch/verify the Qwen3.8-Flash-Next-FP8
+# payload (131 safetensors shards, 172.78 GiB total, plus aux files) for the
+# vLLM frank-primary deployment on dankserver.
+#
+#   bash scripts/download-frank-primary-fp8.sh [dest_dir]
+#
+# The manifest (scripts/manifests/qwen3.8-flash-next-fp8.manifest) carries
+# per-file sizes + LFS sha256s from the Hub; hf-fetch.sh is idempotent — an
+# interrupted re-run fetches only what is missing or corrupt, and a complete
+# on-disk payload is re-verified (full sha256 pass) with zero downloads.
+# The repo is public; no HF_TOKEN required.
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+dest="${1:-/home/dank/models/Qwen3.8-Flash-Next-FP8}"
+
+exec bash "$REPO_ROOT/scripts/hf-fetch.sh" "$dest" \
+    "$REPO_ROOT/scripts/manifests/qwen3.8-flash-next-fp8.manifest" \
+    Qwen/Qwen3.8-Flash-Next-FP8

old/download-frank-small-sglang-int8.sh

@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Download MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound for frank-small
+# (SGLang, RTX 3090). Public repo — no auth header on the command line.
+#
+# Per-file FRESH downloads, no resume: `curl -C -` once produced a 213 MB
+# oversized file on a 31 GiB payload; a fresh -f fetch plus size+sha256 is
+# the house rule.
+#
+# Usage:
+#   scripts/download-frank-small.sh [DEST_DIR] [--verify-only]
+#
+# LFS shards are verified against the Hub LFS sha256; non-LFS text files
+# against their declared byte size. Idempotent: complete files are skipped.
+set -euo pipefail
+
+REPO="letechlead/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/MiMo-V2.6-Distill-Qwen-9B-INT8-W8A16-AutoRound}"
+
+# name bytes sha256(- = non-LFS, size-only check)
+FILES=(
+  "model-00001-of-00008.safetensors 2946296312 ddb6117c04cb6feaf7c265e5d75efadf21949037d73253a8e4521d40e164366d"
+  "model-00002-of-00008.safetensors 1402134792 2dc78d55d1d33d7b3b21dd6f8b4b817b62c2fb1376e6448b322428798c10e257"
+  "model-00003-of-00008.safetensors 1427365904 5c797d14e345309d67c78306b9de4aa01a21dbd92dd98e40b0ad69b2900e9203"
+  "model-00004-of-00008.safetensors 1420041856 7402f852e2c7a24ebf4694b98a220e71dfb2c93ba3776272cf2acfc65acd8126"
+  "model-00005-of-00008.safetensors 1410655648 991cbf2eda2ff91fc3ffbb15591d7fcac340fb10d9c33bd3da20010e2f3f4090"
+  "model-00006-of-00008.safetensors 1427627856 efc42448054cfb5e3061e154ba4539204088ce95176d689a95b58de885ef7eeb"
+  "model-00007-of-00008.safetensors 2035442408 75a101fc71ae6f2e1f5c4816f711d62cf1d7a0e02ef6e5182361b8c11e72b626"
+  "model-00008-of-00008.safetensors 822153664 0fb90cb50e2d336fbad58bd4c01880eeddbf537c1dca1cf0ae291195d5efbb4b"
+  "tokenizer.json 19989325 06b9509352d2af50381ab2247e083b80d32d5c0aba91c272ca9ff729b6a0e523"
+  "model.safetensors.index.json 106278 -"
+  "config.json 4608 -"
+  "generation_config.json 161 -"
+  "quantization_config.json 1654 -"
+  "chat_template.jinja 3916 -"
+  "tokenizer_config.json 1168 -"
+  "preprocessor_config.json 443 -"
+  "processor_config.json 1191 -"
+  "README.md 8448 -"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if [[ "$sha" == "-" ]] || sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  # no -C resume; -f + --fail-with-body so an HTTP error is not a success
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if [[ "$sha" != "-" ]] && ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"

old/qwen3.8-flash-next-fp8.manifest

@@ -0,0 +1,144 @@
+.gitattributes 1635 
+LICENSE 3235 
+README.md 65400 
+chat_template.jinja 8952 
+config.json 72514 
+generation_config.json 202 
+merges.txt 3353259 
+model-00001-of-00131.safetensors 1040155912 774f0ceeadb40d165f2b3ff397d5f3840e6ca8fcb8f3d39d8acb4fea9e52c941
+model-00002-of-00131.safetensors 1678209208 6841fe21fa8a8a7a693c585efe65cd2732889095b696da88bda0cb287366910b
+model-00003-of-00131.safetensors 993901136 974a2a2ab551f8f1405a4955ab32a8721c68c73dd85b382491d9f0e6a34ee752
+model-00004-of-00131.safetensors 1678209208 591f488ab5cd5f0bd4fe28266099523761b9b9339137800734c34ffd84595538
+model-00005-of-00131.safetensors 1717267690 c50bf465a4a0129f1a1196c0e75d881048238f7107b356fc734007eac0d3b123
+model-00006-of-00131.safetensors 1600008328 57f103367fe36c9dc355c648d521d026caabb4ce5ba51038c78bd935f6593736
+model-00007-of-00131.safetensors 1600008328 cde2a12854770c706a74a5c7b85ffc140e3d4dba93dd4bee1cd7aa07cc348c90
+model-00008-of-00131.safetensors 1600008328 98ae323b5a61abdcf58a0dc9db7eb1a5a748cebc1ef1932804a1411500c09d84
+model-00009-of-00131.safetensors 1600008328 70f4b66bca5c4c0b441cf4284d4ab311670e182bfc5ec51c7294a4054baeb20f
+model-00010-of-00131.safetensors 1600008328 6d5dfa1dfa9a6c482f100f6e75c1f5e8ac02391c62e844e53f4a53996baf2206
+model-00011-of-00131.safetensors 1600008328 de085adb563e530e7ad157220cd1a219be189a1db0fcc3f2507886a2d78bd4e1
+model-00012-of-00131.safetensors 1600008328 97dac7c366ba19858d78c01364054136449ccb3cf60768587461cbf0bb582c90
+model-00013-of-00131.safetensors 1600008328 467be10b473d25e0581f8fb1bc986916c8b9301272f72e7a05762484ffc080a9
+model-00014-of-00131.safetensors 1600008328 38def6e9bbe6f242c2f92858c3e9a2b356d74d0180e323677158016dbee32ae7
+model-00015-of-00131.safetensors 1600008328 90c6c665f1f819bc6350714652c8fb77064fec57832ef51b878a0cdf9f602236
+model-00016-of-00131.safetensors 1600008328 88185c2530641b37dcad80d37b505c5257035e87ad29fa86640aae4814903e4f
+model-00017-of-00131.safetensors 1600008328 42e3b96359116d7471a74cb7de08f6a4e9e173830d562f1f74ff09d1d253ad38
+model-00018-of-00131.safetensors 1600008328 040adc7270efca6498c6f6a07dd7da14925d20217fe6de67223d6856fb4f2ad7
+model-00019-of-00131.safetensors 1600008328 b8a831992deb1a694439e37b7136ff2e181457a8683bff65f1f4b99772cabe58
+model-00020-of-00131.safetensors 1600008328 9426112fbd49ef4f5e9dd38dc2f93b046602e6d92cff732d33b96413ff1b40dc
+model-00021-of-00131.safetensors 1600008328 fce8426665f749ef38c4fad33b612f892b728c0f74fe60cab0e43ece785b4e77
+model-00022-of-00131.safetensors 1600008328 a20dd415f280774321538e6b9e78fba9f8e218a9090eae52ea7172b85b9c5a6d
+model-00023-of-00131.safetensors 1600008328 4100591839471a8721d9cedb251554dcef5ebc1dfd96ea7dce8912dd1e9921b2
+model-00024-of-00131.safetensors 1600008328 bbd6d7a4b54b6a6032e15b7c2fc9d6526377f89992d61da8837ad059f3210b45
+model-00025-of-00131.safetensors 1600008328 f9fa7b04de155dd2be74b5693d5b36e5463dabf956b9be36f67d4aae6b8dd304
+model-00026-of-00131.safetensors 1600008328 193c6b9da2c6b69c34d8a14b65234871db272761e712c2e56bac3f13c2d0cbf7
+model-00027-of-00131.safetensors 1600008328 d7efa372552c07ee8783e27c4b8ebd12fc75333f92553861bec8c7c0155a03fe
+model-00028-of-00131.safetensors 1600008328 5fc13d084d62c06b928dfb388a1b36f5c27cd25bd3cbb68a0b21e1c4cfc68e6e
+model-00029-of-00131.safetensors 1600008328 a11f5a6360b349159575871db5c553c40c37be68382b82551a2c2372f15f2811
+model-00030-of-00131.safetensors 1600008336 cac780a675f87fc699435a5207cbecb30a887271b5514b60b7e45d9bebe0e1ec
+model-00031-of-00131.safetensors 1600008336 d390a872eb3928b9d7c0a720ced792cc111613812a163d9fd6c97fd35d633897
+model-00032-of-00131.safetensors 1600008336 3b36fe4e7215a12cdc6c7499aafde51cfbce51266785184b99d3838a6b37b2c8
+model-00033-of-00131.safetensors 1600008336 3fb0504f406c0c71ee40e9e158492973c5c35337889c81b974892b407620c64c
+model-00034-of-00131.safetensors 1600008336 6d989a812a8d8d9330c8c4d21d7be98a0114705ca3bd7834d43bb268bad6be08
+model-00035-of-00131.safetensors 1600008336 8e56ba0a714198bd07a2d8fe9d91fbf5c72601c1b98edf09b289a2538445cb75
+model-00036-of-00131.safetensors 1600008336 c23fd6f6cfa9acfa961ba26e7c9c0dd7b05782c5bbf8f55e2526119b5c516fd4
+model-00037-of-00131.safetensors 942343240 b824a11bf460bd78c068e3d301e914a0bec2f243a6eea4679be7200d179248e7
+model-00038-of-00131.safetensors 1678211256 716eda81a763ffdefd1ac5306b4c80eef123c4dee24ed3a565a22438497581cb
+model-00039-of-00131.safetensors 877983696 b91e3bfbcd765ce704db85b78f69055da59456ae733df635e3973ff2f966a148
+model-00040-of-00131.safetensors 1678211256 3311ed7821b68bddb1542f06ca349feec72e68288113fe4899b08a3b8156343f
+model-00041-of-00131.safetensors 1103350296 1d79949b61730a0fa1f6cd1af149fa50fedc051566cdea48283de8c7a32d972c
+model-00042-of-00131.safetensors 1678211256 3f61684338e203cb2ccff6e451eda5705e10043d386b7ed55d4732d23eb5c77b
+model-00043-of-00131.safetensors 1000455912 bab92cfefbb811ee63c4e32595024117c7df3537f5359606eb1de23fe4f115b2
+model-00044-of-00131.safetensors 1678211256 1d05f138848edca4138c3d1c6be4a432a0817eb29fa65f12d8611ecc7685c493
+model-00045-of-00131.safetensors 993902176 4610384e28f7a91b8bf2a1527178c82cbe7d50ecd9a33b61ee878fa8e8bced85
+model-00046-of-00131.safetensors 1678211256 5ee9b0e45810300cbd4fd6f2acfe85080e7032a9ce5d77038b86f357b6700c23
+model-00047-of-00131.safetensors 878065752 efa6c0cc0e527486dc5955b00fa75665151d99997b722d8e56bd32da575e414c
+model-00048-of-00131.safetensors 1678211256 57b148a266c55a896f171b5c98b350092bf394c1fb836658c58c589a0a317ccf
+model-00049-of-00131.safetensors 1096817160 801dd3b475b43413ade810f4e8d3ace69d6ef5ee4bac5cb98d15e369343a5569
+model-00050-of-00131.safetensors 1678211256 b4d00aca34ba7a9f8427983fd0829408fb5e15b2bb7268199e4010bf0c460a04
+model-00051-of-00131.safetensors 993902176 f430125e9ac92c2de15bbb74bcd3c91bc5374eef389a1be1bc4a592f3a082f0c
+model-00052-of-00131.safetensors 1678211256 06c3d64136b097919b7e5a1715a70a60b8c2302fc4e80a33235607bb99d00f35
+model-00053-of-00131.safetensors 1000455920 147d38f786512f1a8f570a00100b46a3faab1bd649b3445ca0a50c00aa2e99a3
+model-00054-of-00131.safetensors 1678211256 31c08218039a246c333bfe0b068b8027fd92dddd542e581997af546eb104dd4d
+model-00055-of-00131.safetensors 877983696 971cd69c8d955bef6d557ebab3f170807a599517f7c69e9d8a6ba375214137f4
+model-00056-of-00131.safetensors 1678211256 a6a975ac346aa3bd1176cb816d8bbd31a4d7542aa3de9306ca5899bea44bd159
+model-00057-of-00131.safetensors 1103350288 d6acae1622abfa1e0253ae548bc1bdd4bc38cd9b096082acd455b6be592f4a54
+model-00058-of-00131.safetensors 1678209208 f02b6bd8f5cbd742e8bac3ba9537aa14a64c5a8f3fc75495227a14e1a5cec4ec
+model-00059-of-00131.safetensors 877982872 f3e2c94b8157a021b27e1efdb659333a8d1db330da5cd72ab9012fa0e5351506
+model-00060-of-00131.safetensors 1794100080 908cce7f5fe8dc34f4cca1fc8a676569ab6ce08bad536bd0091c4d75e7b65a05
+model-00061-of-00131.safetensors 1678211256 718532c88f4b7a4f402a50e4f1b6efc3cf71484c9e5a083ee30e96d5094a1bb3
+model-00062-of-00131.safetensors 1832994688 3c21b895a5a4f0dc03b669c730f8d8d9ce9a29cbc7628a25894a16ffda6f0a7a
+model-00063-of-00131.safetensors 1678211256 91d215e2135b2f5db5bc5a9541e5f6bf761205d67c6e7b8955ac73c815198f58
+model-00064-of-00131.safetensors 993902176 03a11cad2c283fb9752c6ac01c8f1d8b3127dcda963c5a8225b78f567239d872
+model-00065-of-00131.safetensors 1678211256 0f2f96331120779ce1ebd3f957882092d3c733aa9a6073ca87fc86c6eb4e9465
+model-00066-of-00131.safetensors 880605240 5dbe501e4e0cb9e3f37cc43da46673b4157fdffba09c065ca0b12d75092a79f2
+model-00067-of-00131.safetensors 1678211256 3ece325d2ca53eddf7b1782d5bbc4c376bc75c6d3da67813e110fd0af67cd09b
+model-00068-of-00131.safetensors 1096796552 0c5ef706afd6b3058af5bf8fdc1e0938cb6e29d0a526c771893c9dcb8e702463
+model-00069-of-00131.safetensors 1678211256 84999b24e3c55d2eae3ca187219a2e1a1872436ae9bafc91cc3e301180f36cfd
+model-00070-of-00131.safetensors 993907416 eac7940317ee401859caf38089b2b924274ab81a3e5fbd87c42d89923361aad8
+model-00071-of-00131.safetensors 1678211256 5ac60e474ccdab21257245630ed8240d75bc7115b3adb27ca65e39eb9ea8f20e
+model-00072-of-00131.safetensors 1000456024 b50a05c4c426fb00635d513a733a290da1007a89b8785f1ec02aad8e4d93d955
+model-00073-of-00131.safetensors 1678211256 9e0594d389debdfb26d22259725831ff34280edeef4144e683d70205050919d9
+model-00074-of-00131.safetensors 877983696 1b1a0294f8df2a431885783185cdb185106228c9567e67224fe2c3c6a2467f5a
+model-00075-of-00131.safetensors 1678211256 7dba189804ef7a8f9d926b27e1ddbe73767f1213cb7bb3cc582ab15fffec1792
+model-00076-of-00131.safetensors 1100073480 d95ebacb144b4220f2df60cce85a2c86e87cb8291b836d1bd04f9902fcbb9297
+model-00077-of-00131.safetensors 1678211256 26046977983e025defdf0f84cf5fc5f4ddfeabbd27e7abc43ff2b413265ab062
+model-00078-of-00131.safetensors 993984232 6fd46a95777392265b2afa66bf71d488856dafb7b76ff0337a0fe4ac857e961c
+model-00079-of-00131.safetensors 1678211256 77398002634ce852917c59cd642bec7dd3903381239da68912bf69743631e54e
+model-00080-of-00131.safetensors 877983688 f44aad95d36c443995b965aeef844c8e3a30ed172647dc57fcc0c5476c18d024
+model-00081-of-00131.safetensors 1678209208 b295a48088573654d1b4754e861ec329956504cb6687e9e2a8eb652b35eaf988
+model-00082-of-00131.safetensors 1096816112 83e0e171116c22eb8fdddfa3c387bebb9017eb0586410fd704ce70e7b3d23d68
+model-00083-of-00131.safetensors 1678211256 7e1021cde06c970e4b1d9eb3d02f66f29786c59a0da8a3f8879de1a4e3fbf6c0
+model-00084-of-00131.safetensors 877983696 8938fd3c7f88d61fd5b5306eb0ab253d3e42e7f64baf3c3e494d315c8ad5ab55
+model-00085-of-00131.safetensors 1678211256 cd5875a29d28d52578ae699fdd8891f605a0953cd64516bd88dc45dab50c383a
+model-00086-of-00131.safetensors 1103350296 885e570a662aaf266fd9af9562c65116998df5710b2a27d1bf6aa0c8708b3b74
+model-00087-of-00131.safetensors 1678211256 1472604826f0a0c2bbf7cd0f4a0c5b29f3012e4b3a4424d024d1ec17a3c9d0ea
+model-00088-of-00131.safetensors 1000455920 e7273f5435571e761429330ece81732e8fb3f8b5f0087bf80db693630cf702a1
+model-00089-of-00131.safetensors 1678211256 eb349562d23e330bbbd0a5f5ba90ac25ed0baa660dff230538a0fcaaf4c8b875
+model-00090-of-00131.safetensors 997179104 532915eebd4e231a240e185ba838a95a00ee14277e28fb04ad21508da5ff50aa
+model-00091-of-00131.safetensors 1678211256 68d8f46ffcd0ff46f4e73c848525cc6fd6af15eebbb9311ce9b4eb14638f5660
+model-00092-of-00131.safetensors 877983696 91abc087cd244ae62b8469576e209c8e25bc7726bc4fc020b687378059f5b5bd
+model-00093-of-00131.safetensors 1678211256 dacd0f2e5215fbf18fb7c6803f61963333e6243f404de83fcd5dfd38f43fe0d1
+model-00094-of-00131.safetensors 1096796928 cac515a778e89fb37e9f38fde6c8fdfa01ebf9d8944a2d6ff35c51ecf4bc875f
+model-00095-of-00131.safetensors 1678211256 a35975c2d21f18d58f21cb59e4eaf34a4bce11440e5112948e7fa6923658f8c4
+model-00096-of-00131.safetensors 993902176 03fd77f9d35024bba9e894c811300ef8aaf5b5ff99ebc2c89c9d9f87685adfe8
+model-00097-of-00131.safetensors 1678211256 f7b81ae09d97b8e2bb96a1916ede51da92209ec8481a31c56fb446acd691b8a8
+model-00098-of-00131.safetensors 993902176 70f208f47d99c25d5413cd32e0553cce22524cd3863c7797e2cfa37c2d526319
+model-00099-of-00131.safetensors 1678211256 de5c33dd717495657496067ea715442356fd9f7b64384d1a4c10c9f8fe6f9b7d
+model-00100-of-00131.safetensors 877984072 43752e81a5c2b376073d5d0d7b70da8d08c49bdbbfcfeb695ea947a2913e4bc8
+model-00101-of-00131.safetensors 1678211256 c2e978f7238036063914fe909214246730640be77eca215e3cf66d65724de9f1
+model-00102-of-00131.safetensors 1096797160 639efed3ea400a8dddeae25db120096898a514697dfc4cbbc0cd15f621c18421
+model-00103-of-00131.safetensors 1678209208 f8da1b61296e012dc268b4f0d84c127ef0967df488fe65b037f74c8f893decd1
+model-00104-of-00131.safetensors 993901144 adff0f2aee7b7b701e39247cfdfa135c0cab8dfa4f8c7662352c0233adb78829
+model-00105-of-00131.safetensors 1678211256 6643d71382f00f7ffe16b05879e6ff0f6e56986dee8ec079a84aae82912c6987
+model-00106-of-00131.safetensors 1025359600 680495ea0a23e73844e91670437b7b2a5ac5d46f9582923fb7fa91fe640821df
+model-00107-of-00131.safetensors 1678211256 886528d121b77433c1d66654294c13b4f61a004f1e6d1abf2d445fb0a99f7234
+model-00108-of-00131.safetensors 1062060032 a966c28a6267d43052fe87cb19e2e5730bad6154b12eb806b9ce49e007bfec96
+model-00109-of-00131.safetensors 1678211256 f6074252e492ee5b0933f78cc48f3b90694a8096befb7666cc379c3d47093756
+model-00110-of-00131.safetensors 877983696 e3e45e96bc13b52b748a0e2f397a624ad49958396468667d73ed4d725c8b5c1c
+model-00111-of-00131.safetensors 1678211256 d1d354e691eab0b77607faf7eb4adf00a75ee7f270c1cbcbad06aefc19aeac94
+model-00112-of-00131.safetensors 1096797168 e61a9a6ef0e538afca09d97a3ccbf3e7e2c647f65cc8eeebebb18621d7779b02
+model-00113-of-00131.safetensors 1678211256 a14b94db5326bfaab1d8029296b8938f6daf91ba2dc07a4950a38afafa84f776
+model-00114-of-00131.safetensors 993907400 ce4aeac52776038ec0c885915bb270dc65ee07d70d5f71827d5fba1045e2b43b
+model-00115-of-00131.safetensors 1678211256 8185d8822428d2125876a9aa05aea11c8165713bc99e04ace29921eef0e1b683
+model-00116-of-00131.safetensors 993902176 65954dccfa3be20611728311d97679e7f5d78ad519e52312524aa7cf666a48e5
+model-00117-of-00131.safetensors 1678211256 12c8027363b1a7bb782282293314fa57b07be27d17670861858c0aed1e28a3ac
+model-00118-of-00131.safetensors 878004272 2d06ec9c1726f42bfc9ce0bbb47129917d8ab373c88eed4e758fb6940c92ad4a
+model-00119-of-00131.safetensors 1678211256 36008b48c4480085bfd1a81439d70d1029cfaf06cfdd037cec19b491a40659ec
+model-00120-of-00131.safetensors 1109903856 49e4f90d92f60f6489bfe6d3e5250d8fe879c5995ae72ce67379cc7187fa4b0a
+model-00121-of-00131.safetensors 1678209208 ae8e07cceeef2f568df035a7bc0c555656acb566ba5936f61a8a019df23361ad
+model-00122-of-00131.safetensors 993901136 b3b21d33b925e31d475a509757cf7459aa4d3e73cb493706f40461387541e137
+model-00123-of-00131.safetensors 1678209208 f32fec30685afadd76f09bbb1dda680445ad765a97ff1febfc3122b7f73d7ec3
+model-00124-of-00131.safetensors 891089968 e857f36ddd0db135f858c2c87c9fcf39c758965c626b280e47cb0c1e6c1423f8
+model-00125-of-00131.safetensors 1678209208 b9de3774218d3cd7946cdbf913dedfc82591c18212d9c3fc9b9484e7dc68e9fe
+model-00126-of-00131.safetensors 1096795496 29b39319522a00b1ef4ac909c4fb306059999f0a48fd07ca73cea687647631ee
+model-00127-of-00131.safetensors 1678209208 bf48ccb3b4ba75c288d9db2b94817c6a3993e9f84a5b9410d9f24edcec12e177
+model-00128-of-00131.safetensors 993901136 7feaa51c906a1a735e5fb52fd5499b59c473d442d77b44cbd610c9b6a3a9e0ba
+model-00129-of-00131.safetensors 1678209208 a5374c8d86aea48f21a4be2ad6cda37fbfbb01eff333d1179939a2e55d8db816
+model-00130-of-00131.safetensors 2136177320 204b122d02425f61115db9ab896648d95f6dadd6a716c65daf6aec297392ed56
+model-00131-of-00131.safetensors 1271398496 ba6fc69ebb6a9da1938c2e041b8ef625f4550b641812da75845bc7dd7c361d80
+model.safetensors.index.json 17410140 0419e2c2dfbb925257d7409405433a793cf7ff7d96f3eba882a815ec6d9fe7a6
+preprocessor_config.json 390 
+tokenizer.json 12809320 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3
+tokenizer_config.json 17928 
+video_preprocessor_config.json 385 
+vocab.json 6722759 

prose_prompt.txt

@@ -0,0 +1,86 @@
+# Subsurface Aquifer Monitoring Network - System Design Document (Revision 7)
+
+This document describes the deployment of a subsurface aquifer monitoring network for the Cascadia sedimentary basin. It is written for the operations crew and the regional water authority. Every sensor node, relay, aggregation service, calibration procedure, and failure mode below carries a stable alphanumeric identifier that appears in every table and in the field notes.
+
+## Sensor nodes
+Node SN-000-08B9 sits in a discharge lens at 359.0 m below datum and reports a salinity of 2.38 ppt with the fractured interval reading 5.40 degC at dawn and the wellhead porous 1.72 m3/d while the electrode pair returns 3157 uS/cm.
+Node SN-001-0A65 sits in a percolate lens at 25.9 m below datum and steps down to a salinity of 2.13 ppt with the diapir interval reading 8.55 degC at dawn and the wellhead silty 0.67 m3/d while the electrode pair returns 979 uS/cm.
+Node SN-002-0571 sits in a diapir lens at 287.8 m below datum and peaks at a salinity of 7.63 ppt with the karst interval reading 4.26 degC at dawn and the wellhead spooned 11.50 m3/d while the electrode pair returns 2987 uS/cm.
+Node SN-003-0CE9 sits in a confining lens at 169.5 m below datum and logs a salinity of 0.96 ppt with the alluvial interval reading 5.03 degC at dawn and the wellhead salinized 12.08 m3/d while the electrode pair returns 2607 uS/cm.
+Node SN-004-0E2A sits in a freshened lens at 70.4 m below datum and settles near a salinity of 8.68 ppt with the spooned interval reading 4.88 degC at dawn and the wellhead siphon 4.38 m3/d while the electrode pair returns 2707 uS/cm.
+Node SN-005-0DDD sits in a percolate lens at 44.6 m below datum and reports a salinity of 6.25 ppt with the alluvial interval reading 10.85 degC at dawn and the wellhead brackish 13.99 m3/d while the electrode pair returns 3618 uS/cm.
+Node SN-006-0EFC sits in a till lens at 309.5 m below datum and reports a salinity of 3.49 ppt with the lacustrine interval reading 7.38 degC at dawn and the wellhead spooned 3.21 m3/d while the electrode pair returns 1253 uS/cm.
+Node SN-007-0B02 sits in a recharge lens at 262.0 m below datum and steps down to a salinity of 2.38 ppt with the discharge interval reading 8.18 degC at dawn and the wellhead perched 4.05 m3/d while the electrode pair returns 3900 uS/cm.
+Node SN-008-0448 sits in a karst lens at 406.5 m below datum and reports a salinity of 7.32 ppt with the upconing interval reading 6.17 degC at dawn and the wellhead fluvial 0.75 m3/d while the electrode pair returns 1448 uS/cm.
+Node SN-009-043C sits in a alluvial lens at 159.3 m below datum and flags a salinity of 6.20 ppt with the percolate interval reading 7.60 degC at dawn and the wellhead brackish 13.01 m3/d while the electrode pair returns 2025 uS/cm.
+Node SN-010-0FF5 sits in a karst lens at 264.2 m below datum and flags a salinity of 7.05 ppt with the freshened interval reading 7.88 degC at dawn and the wellhead siphon 8.41 m3/d while the electrode pair returns 1635 uS/cm.
+Node SN-011-098A sits in a drain lens at 365.7 m below datum and reports a salinity of 8.11 ppt with the wedge interval reading 5.51 degC at dawn and the wellhead glacial 2.52 m3/d while the electrode pair returns 2916 uS/cm.
+Node SN-012-0AE8 sits in a till lens at 259.6 m below datum and steps down to a salinity of 9.13 ppt with the recharge interval reading 11.60 degC at dawn and the wellhead spooned 0.46 m3/d while the electrode pair returns 3077 uS/cm.
+Node SN-013-0B83 sits in a salinized lens at 171.2 m below datum and peaks at a salinity of 2.83 ppt with the lacustrine interval reading 5.56 degC at dawn and the wellhead discharge 0.35 m3/d while the electrode pair returns 3083 uS/cm.
+Node SN-014-032E sits in a drain lens at 249.6 m below datum and flags a salinity of 1.09 ppt with the salinized interval reading 9.58 degC at dawn and the wellhead perched 12.00 m3/d while the electrode pair returns 2221 uS/cm.
+Node SN-015-004F sits in a aeolian lens at 264.4 m below datum and reports a salinity of 7.34 ppt with the salinized interval reading 8.76 degC at dawn and the wellhead perched 0.31 m3/d while the electrode pair returns 1483 uS/cm.
+Node SN-016-07C6 sits in a aeolian lens at 155.9 m below datum and reports a salinity of 0.64 ppt with the mineralized interval reading 11.75 degC at dawn and the wellhead diluted 13.46 m3/d while the electrode pair returns 524 uS/cm.
+Node SN-017-06C5 sits in a salinized lens at 72.1 m below datum and averages a salinity of 4.52 ppt with the siphon interval reading 8.93 degC at dawn and the wellhead confining 3.98 m3/d while the electrode pair returns 3687 uS/cm.
+Node SN-018-0E64 sits in a siphon lens at 334.9 m below datum and steps down to a salinity of 6.73 ppt with the salinized interval reading 7.63 degC at dawn and the wellhead freshened 9.64 m3/d while the electrode pair returns 1681 uS/cm.
+Node SN-019-0122 sits in a spooned lens at 117.2 m below datum and logs a salinity of 3.24 ppt with the fractured interval reading 9.26 degC at dawn and the wellhead karst 3.50 m3/d while the electrode pair returns 1065 uS/cm.
+Node SN-020-021D sits in a karst lens at 414.3 m below datum and recovers to a salinity of 0.76 ppt with the glacial interval reading 6.25 degC at dawn and the wellhead silty 9.60 m3/d while the electrode pair returns 1041 uS/cm.
+Node SN-021-06C6 sits in a percolate lens at 379.2 m below datum and drifts to a salinity of 7.61 ppt with the wedge interval reading 5.84 degC at dawn and the wellhead karst 1.65 m3/d while the electrode pair returns 1913 uS/cm.
+Node SN-022-0DC9 sits in a spooned lens at 327.1 m below datum and reports a salinity of 9.25 ppt with the freshened interval reading 5.05 degC at dawn and the wellhead silty 5.90 m3/d while the electrode pair returns 1544 uS/cm.
+Node SN-023-0717 sits in a karst lens at 222.0 m below datum and steps down to a salinity of 4.02 ppt with the alluvial interval reading 6.60 degC at dawn and the wellhead siphon 3.77 m3/d while the electrode pair returns 3892 uS/cm.
+Node SN-024-02A9 sits in a fractured lens at 479.7 m below datum and drifts to a salinity of 7.88 ppt with the silty interval reading 12.53 degC at dawn and the wellhead discharge 13.18 m3/d while the electrode pair returns 3592 uS/cm.
+Node SN-025-0C8F sits in a wedge lens at 434.3 m below datum and holds a salinity of 1.63 ppt with the alluvial interval reading 4.22 degC at dawn and the wellhead till 5.73 m3/d while the electrode pair returns 3905 uS/cm.
+Node SN-026-04BF sits in a colluvial lens at 460.0 m below datum and averages a salinity of 9.36 ppt with the diapir interval reading 8.98 degC at dawn and the wellhead brackish 10.29 m3/d while the electrode pair returns 802 uS/cm.
+Node SN-027-0DA4 sits in a silty lens at 265.7 m below datum and steps down to a salinity of 7.06 ppt with the percolate interval reading 4.69 degC at dawn and the wellhead freshened 8.42 m3/d while the electrode pair returns 2201 uS/cm.
+Node SN-028-01EB sits in a perched lens at 49.5 m below datum and recovers to a salinity of 1.83 ppt with the silty interval reading 9.32 degC at dawn and the wellhead drain 9.69 m3/d while the electrode pair returns 1126 uS/cm.
+Node SN-029-042C sits in a karst lens at 30.6 m below datum and averages a salinity of 0.86 ppt with the percolate interval reading 9.85 degC at dawn and the wellhead recharge 8.16 m3/d while the electrode pair returns 1452 uS/cm.
+
+## Relays
+Relay RL-00-02181 reburses the upconing trunk from the brackish margin and peaks at a round-trip time of 131.8 ms with a frame loss of 1.04 percent across the night window.
+Relay RL-01-00262 reburses the spooned trunk from the fluvial margin and holds a round-trip time of 372.2 ms with a frame loss of 3.65 percent across the night window.
+Relay RL-02-03691 reburses the recharge trunk from the percolate margin and steps down to a round-trip time of 398.5 ms with a frame loss of 0.29 percent across the night window.
+Relay RL-03-0E117 reburses the lacustrine trunk from the confining margin and averages a round-trip time of 373.9 ms with a frame loss of 3.44 percent across the night window.
+Relay RL-04-0B41B reburses the aeolian trunk from the colluvial margin and peaks at a round-trip time of 70.7 ms with a frame loss of 3.25 percent across the night window.
+Relay RL-05-0D12D reburses the recharge trunk from the diluted margin and steps down to a round-trip time of 264.7 ms with a frame loss of 0.03 percent across the night window.
+Relay RL-06-043B4 reburses the colluvial trunk from the upconing margin and reports a round-trip time of 49.5 ms with a frame loss of 3.42 percent across the night window.
+Relay RL-07-09AD6 reburses the fractured trunk from the freshened margin and averages a round-trip time of 225.3 ms with a frame loss of 1.06 percent across the night window.
+Relay RL-08-04394 reburses the brackish trunk from the fluvial margin and steps down to a round-trip time of 88.6 ms with a frame loss of 2.47 percent across the night window.
+Relay RL-09-017A0 reburses the wedge trunk from the lacustrine margin and drifts to a round-trip time of 363.0 ms with a frame loss of 3.30 percent across the night window.
+Relay RL-10-0217D reburses the mineralized trunk from the lacustrine margin and peaks at a round-trip time of 26.2 ms with a frame loss of 1.30 percent across the night window.
+Relay RL-11-08F97 reburses the perched trunk from the freshened margin and logs a round-trip time of 181.8 ms with a frame loss of 2.75 percent across the night window.
+Relay RL-12-02628 reburses the fractured trunk from the glacial margin and steps down to a round-trip time of 378.5 ms with a frame loss of 2.69 percent across the night window.
+Relay RL-13-06E06 reburses the silty trunk from the mineralized margin and flags a round-trip time of 153.4 ms with a frame loss of 2.15 percent across the night window.
+
+## Aggregation services
+Aggregator AG-00-3BB logs a queue depth of 5.2 messages and a histogram bucket of 6.82 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-01-0FF reports a queue depth of 86.4 messages and a histogram bucket of 4.10 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-02-37F drifts to a queue depth of 34.7 messages and a histogram bucket of 10.25 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-03-3B3 averages a queue depth of 93.5 messages and a histogram bucket of 13.59 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-04-33E flags a queue depth of 83.2 messages and a histogram bucket of 17.58 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-05-2F2 settles near a queue depth of 84.8 messages and a histogram bucket of 0.84 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-06-2AD averages a queue depth of 75.4 messages and a histogram bucket of 7.65 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-07-06E drifts to a queue depth of 26.5 messages and a histogram bucket of 14.26 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-08-0E3 averages a queue depth of 83.9 messages and a histogram bucket of 15.51 seconds before the dedupe pass settles the stream for the next rotation.
+Aggregator AG-09-138 averages a queue depth of 78.6 messages and a histogram bucket of 8.50 seconds before the dedupe pass settles the stream for the next rotation.
+
+## Calibration procedures
+Procedure CP-00-047 steps the probe through a two-point rinse, then applies a trim of 0.2084 at the 66 milli-bar mark and records a residual of 0.02052 before the next field seal is crimped.
+Procedure CP-01-0A4 steps the probe through a two-point rinse, then applies a trim of 0.7793 at the 96 milli-bar mark and records a residual of 0.01468 before the next field seal is crimped.
+Procedure CP-02-054 steps the probe through a two-point rinse, then applies a trim of 0.4634 at the 68 milli-bar mark and records a residual of 0.04234 before the next field seal is crimped.
+Procedure CP-03-02D steps the probe through a two-point rinse, then applies a trim of 0.8459 at the 12 milli-bar mark and records a residual of 0.04855 before the next field seal is crimped.
+Procedure CP-04-098 steps the probe through a two-point rinse, then applies a trim of 0.5267 at the 97 milli-bar mark and records a residual of 0.00287 before the next field seal is crimped.
+Procedure CP-05-09B steps the probe through a two-point rinse, then applies a trim of 0.3968 at the 72 milli-bar mark and records a residual of 0.01637 before the next field seal is crimped.
+Procedure CP-06-030 steps the probe through a two-point rinse, then applies a trim of 0.8860 at the 12 milli-bar mark and records a residual of 0.04508 before the next field seal is crimped.
+Procedure CP-07-0EC steps the probe through a two-point rinse, then applies a trim of 0.2367 at the 70 milli-bar mark and records a residual of 0.00108 before the next field seal is crimped.
+Procedure CP-08-0BC steps the probe through a two-point rinse, then applies a trim of 0.7278 at the 68 milli-bar mark and records a residual of 0.04707 before the next field seal is crimped.
+Procedure CP-09-0AA steps the probe through a two-point rinse, then applies a trim of 0.6069 at the 37 milli-bar mark and records a residual of 0.00443 before the next field seal is crimped.
+
+## Failure modes
+Mode FM-00-27 declares a fluvial excursion when two independent probes disagree beyond the band for three rotations; the fleet reports a mean time between failures of 36902 hours for this class of excursion.
+Mode FM-01-13 declares a drain excursion when two independent probes disagree beyond the band for three rotations; the fleet peaks at a mean time between failures of 29042 hours for this class of excursion.
+Mode FM-02-14 declares a till excursion when two independent probes disagree beyond the band for three rotations; the fleet peaks at a mean time between failures of 26976 hours for this class of excursion.
+Mode FM-03-0C declares a diapir excursion when two independent probes disagree beyond the band for three rotations; the fleet drifts to a mean time between failures of 22578 hours for this class of excursion.
+Mode FM-04-39 declares a perched excursion when two independent probes disagree beyond the band for three rotations; the fleet recovers to a mean time between failures of 27383 hours for this class of excursion.
+Mode FM-05-19 declares a siphon excursion when two independent probes disagree beyond the band for three rotations; the fleet logs a mean time between failures of 23152 hours for this class of excursion.
+
+## End of document
+Summarize the network's dominant risk in one paragraph, then list the three node identifiers whose readings most limit the model this month.

scripts/.AGENTTRASH/20260920-llama-layersplit/frank-medium.env.llamacpp-layersplit

@@ -1,7 +1,7 @@
-# Tunables for frank-medium (ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP
-# UD-Q8_K_XL + z-lab/Qwen3.8-27B-DFlash2-GGUF Q8_0 drafter), mainline llama.cpp
-# on Dankserver, 2x AMD Radeon AI PRO R9700 (gfx1201).  Since 2026-09-16 this
-# is the SOLE LLM runtime on the box: the vllm-radiance replica pair and its
+# Tunables for frank-medium (orcarouter/Qwen3.8-27B-Uncensored-GGUF Q8_0 +
+# z-lab/Qwen3.8-27B-DFlash2-GGUF Q4_K_M drafter), mainline llama.cpp on
+# Dankserver, 2x AMD Radeon AI PRO R9700 (gfx1201).  Since 2026-09-16 this is
+# the SOLE LLM runtime on the box: the vllm-radiance replica pair and its
 # routing proxy are retired.
 #
 # Hardware: two R9700 (Navi 48, RDNA4, 32,624 MiB each), no XGMI, different
@@ -31,17 +31,17 @@
 # The ref is pinned by TWO features: DFlash2 (ggml-org/llama.cpp#27342 -- the
 # drafter is detected from its GGUF) and qwen35 MTP.  Do not float it.
 #
-# Model: ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF @ UD-Q8_K_XL,
-# 29.29 GiB, arch qwen35, Unsloth Dynamic 3.0 (per-tensor types from Unsloth's
-# own quant of the same size, on Unsloth's importance matrix).  Abliteration is
-# orcarouter's single refusal direction projected out of 131 tensors; the vision
-# tower loads from mmproj-BF16.gguf (see NO_MMPROJ).  The file also
-# carries the model's MTP head (blk.64.nextn.*), so a file-level switch to
-# --spec-type draft-mtp is available without a second copy in VRAM.
+# Model: orcarouter/Qwen3.8-27B-Uncensored-GGUF @ Q8_0, 27.0 GiB, arch qwen35
+# (verified from the GGUF header 2026-09-19: 262144 trained window, 24 heads /
+# 4 KV heads, nextn_predict_layers=1).  The file ALSO carries the model's MTP
+# head (blk.64.nextn.*), so --spec-type draft-mtp is available without a second
+# copy in VRAM -- one SPEC_TYPE change, the drafter below stays on disk.
 #
-# Drafter: z-lab/Qwen3.8-27B-DFlash2-GGUF @ Q8_0, 1.92 GiB: 5 layers, hidden
+# Drafter: z-lab/Qwen3.8-27B-DFlash2-GGUF @ Q4_K_M, 1.06 GiB (operator call
+# 2026-09-19: the Q4 rung -- the drafter's job is throughput, not output
+# quality, and the target verifies every drafted token).  5 layers, hidden
 # 5120, a block-diffusion proposal (block_size 8) under a 2048-token sliding
-# window.  Decoding stays lossless -- the target verifies every drafted token.
+# window (dflash.attention.sliding_window=2048, verified from the header).
 # The window is what makes it cheap HERE: its KV is bounded at 2048 positions
 # (~40 MiB) instead of spanning the context (see SIZING).
 #
@@ -52,10 +52,14 @@
 # in the attention layers only, plus ~0.16 GiB per sequence of f32 DeltaNet
 # state at any depth.
 #
-# SIZING, measured (rocm-smi, 2026-09-16).  Weights split across the two cards;
-# the drafter AND its KV ride the second.  The loader's compute buffers are part
-# of what is seen below -- they are allocated at load, not per request, and they
-# scale with --ubatch-size:
+# SIZING, the deployed shape (32:33 / ctx 327680 / 6144x512 / two slots /
+# DFlash2 Q4_K_M), measured 2026-09-19 after the model swap (rocm-smi):
+#
+#   card 0 (x16)  26.9 GiB used, 5.7 GiB free
+#   card 1 (x1)   27.8 GiB used, 4.8 GiB free
+#
+# The predecessor's measured table (rocm-smi, 2026-09-16, UD-Q8_K_XL target)
+# stays as the calibration history:
 #
 #   CTX     split  batch  slots  drafter  GPU 0 free  GPU 1 free   note (ubatch 1536)
 #   262144   1,1   2048   1      DFlash2   8.64 GiB    4.43 GiB   day's start
@@ -71,7 +75,7 @@
 # batch block below):
 #
 #   262144  36,29  6144   512    2      DFlash2   6.01 GiB    7.32 GiB   ubatch only
-#   262144  35,30  6144   512    2      DFlash2   6.44 GiB    6.90 GiB   <- deployed
+#   262144  35,30  6144   512    2      DFlash2   6.44 GiB    6.90 GiB
 #
 # ONE -ts STEP IS ONE LAYER, AND LAYERS ARE NOT THE SAME SIZE.  This is a hybrid
 # stack -- 16 x (3 Gated DeltaNet + 1 Gated Attention) blocks -- so moving a
@@ -80,36 +84,27 @@
 # were measured, which is why the balance is found by measurement rather than by
 # interpolating between splits.
 #
-# 262144 is the window the model ships (n_ctx_train) and it fits with room, so
-# the fleet gets the whole thing; the arithmetic that used to cap this endpoint
-# at 114688 was the vLLM predecessor's, not this runtime's.  If a future change
-# adds VRAM (a longer drafter, quantized KV, more slots) re-read the numbers
-# rather than assuming, and prefer CTX_SIZE over --gpu-layers.
+# WHY THE SPLIT IS 32:33 (operator call, 2026-09-19).  The operator asked for
+# the tensors BALANCED across the two cards, which for 65 layers is 32:33 --
+# the most even split that exists.  The predecessor's 35:30 skew existed to lean
+# work onto the x16 card (ROCm0), because ROCm1's Gen4 x1 chipset path made it
+# the 98 % utilization bottleneck; the measured trade of the balanced point is
+# that the x1 card runs hotter while the x16 card idles a little more, and the
+# operator accepted that for symmetry.  Do not re-skew without asking.
 #
-# WHY THE SPLIT IS 35:30.  Three effects decide it, and all three are measured:
-#   * THE DRAFTER AND ITS OUTPUT LAYERS RIDE ROCm1.  It carries the drafter's
-#     weights, the draft graph and the last target layers, so it takes fewer
-#     than half the layer count.
-#   * THE UBATCH BUFFERS COST ROCm1 MORE THAN ROCm0, and shrinking them pays
-#     ROCm1 back more: 1536 -> 512 freed 1.30 GiB on ROCm0 but 2.41 GiB on
-#     ROCm1 at this shape.  Read the other way, that is exactly why the smaller
-#     batch moved the balance toward ROCm1 -- and cost ROCm0 one layer.
-#   * THE LAYER GRANULARITY IS THE FLOOR ON HOW WELL ANY SPLIT CAN BALANCE.  One
-#     -ts step is one layer -- ~0.43 GiB DeltaNet, ~1.43 GiB attention -- so the
-#     reachable minimum gap is comparable to one step rather than to zero.  35:30
-#     sits at 0.46 GiB, i.e. one DeltaNet layer's worth, against 1.31 GiB at
-#     36:29.
-# It also leaves the most room: 6.44 / 6.90 GiB free clears the operator's 1.5
-# GiB-per-card floor by ~5 GiB, so no headroom has to be bought out of CTX_SIZE
-# and the full window stays.
+# THE TWO CARDS' PATHS ARE NOT SYMMETRIC (measured, 2026-09-17): ROCm0
+# (03:00.0) hangs off CPU root port 00:01.1 at Gen4 x16 end to end (~31.5 GB/s);
+# ROCm1 (0b:00.0) rides the B650 chipset through bridges that each negotiate
+# Gen4 x1 (~1.97 GB/s).  The layer-split boundary itself is not the cost
+# (~5.2 MB of activation per 512-token ubatch rides the wire); the x1 card is
+# COMPUTE-bound, not PCIe-bound.
 #
 # A FAULT TO WATCH.  A 39:26 split, at 1x batch / 1 slot, died on its eighth
 # request: the amdgpu driver logged SDMA0 UTCL2 page faults and the engine
 # surfaced `ROCm error: an illegal memory access was encountered` -> ggml_abort,
 # i.e. a fault in the cross-card boundary copy, this layer split's hot path.  It
-# has not reproduced (36:29 served this file's verification battery and the e2e
-# gate, and 35:30 is the deployed shape), but the signature is worth recognising
-# -- it is NOT an OOM, and no amount of headroom prevents it.
+# has not reproduced since, but the signature is worth recognising -- it is NOT
+# an OOM, and no amount of headroom prevents it.
 #
 # WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP reads
 # those bare vars; 2026-08-13 incident).  llama.cpp takes --host/--port/--temp
@@ -119,15 +114,15 @@
 LLAMA_BIN=/home/dank/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714/bin/llama-server
 
 # Payload, fetched with scripts/download-frank-medium-gguf.sh.
-MODEL_DIR=/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
-MODEL_FILE=Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
-# The BF16 vision projector, same repo as the target.  llama-server autoloads
-# any mmproj*.gguf beside the model; the serve script refuses to start without
-# it when vision is enabled, so a missing file is a deploy error, not a
-# text-only server that pretends to be multimodal.
-MM_PROJ_FILE=mmproj-BF16.gguf
+MODEL_DIR=/home/dank/models/Qwen3.8-27B-Uncensored-GGUF
+MODEL_FILE=Qwen3.8-27B-Uncensored-Q8_0.gguf
+# The f16 vision projector, same repo as the target.  llama-server loads it via
+# the explicit --mmproj flag (no autoload in this build); the serve script
+# refuses to start without it when vision is enabled, so a missing file is a
+# deploy error, not a text-only server that pretends to be multimodal.
+MM_PROJ_FILE=mmproj-Qwen3.8-27B-Uncensored-f16.gguf
 DRAFT_MODEL_DIR=/home/dank/models/Qwen3.8-27B-DFlash2-GGUF
-DRAFT_MODEL_FILE=Qwen3.8-27B-DFlash2-Q8_0.gguf
+DRAFT_MODEL_FILE=Qwen3.8-27B-DFlash2-Q4_K_M.gguf
 
 # Fleet identity.  7770 is the fleet's dankserver endpoint; the alias is the id
 # at /v1/models and what every client asks for.
@@ -141,17 +136,11 @@ HIP_VISIBLE_DEVICES=0,1
 DEVICES=ROCm0,ROCm1
 SPLIT_MODE=layer
 
-# 36:29 -- one layer over the 35:30 balance, moved to buy headroom on ROCm1,
-# the card that carries the drafter, its KV, the output layers and the heavier
-# compute buffers, now that the context grew to 327680 (see below).  Fit at
-# the deployed point (36:29, ctx 327680, batch 6144 x ubatch 512, two slots,
-# DFlash2 + ngram-mod, mmproj on ROCm0): ROCm0 29.19 / ROCm1 26.04 GiB of
-# 31.86, i.e. 2.67 / 5.82 GiB free and a 3.15 GiB gap BY CONSTRUCTION -- the
-# balance point at this context is 34:31 (0.11 GiB gap) and the operator chose
-# headroom on the drafter card over balance.  See SIZING above for the
-# measurement table; one -ts step is one LAYER, worth ~0.43 GiB (DeltaNet) or
-# ~1.43 GiB (attention), so do not move it without re-measuring.
-TENSOR_SPLIT=36,29
+# 32:33 -- balanced (operator call 2026-09-19): 65 layers split as evenly as
+# they can be, ROCm1 (the last-layers card, which also carries the drafter and
+# the output layers) holding the extra one.  See the WHY THE SPLIT IS 32:33
+# block above for the trade against the predecessor's 35:30 x16-lean.
+TENSOR_SPLIT=32,33
 
 NGL=999
 # Auto-fit has no business resizing a context we set explicitly.
@@ -160,10 +149,9 @@ FLASH_ATTN=on
 
 # 327680 positions over TWO slots, so each conversation gets 163840 --
 # --parallel DIVIDES --ctx-size, so more slots would shrink every
-# conversation rather than grow the cache.  The checkpoint's trained window is
-# 262144 and the GGUF carries NO rope scaling (freq_base 1e7, no YaRN keys),
-# so positions beyond 262144 extrapolate on the base rope: it runs, and 1.25x
-# is a modest ask of it, but the extra 65k is untrained territory.
+# conversation rather than grow the cache.  163840 is inside the checkpoint's
+# trained 262144 window, so the whole pool is trained territory (no rope
+# extrapolation, unlike the predecessor's 393216).
 CTX_SIZE=327680
 PARALLEL=2
 
@@ -211,11 +199,32 @@ THREAD_BATCH=8
 # f16 K/V: an agentic checkpoint is the wrong place to spend KV precision.
 CACHE_TYPE_K=f16
 CACHE_TYPE_V=f16
-# Context checkpoints: 0 is the build's default -- no per-slot KV snapshots, so
-# a re-sent or rewound conversation re-prefills instead of rewinding.  Stated
+# Context checkpoints: 0 -- no per-slot KV snapshots at all.  The build's
+# default is 32 (this binary's --help), so the 0 below is a deliberate
+# deviation, not an inherited default: a checkpoint is a VRAM->host-RAM copy
+# of part of a slot's KV, taken so a partially-matching prompt can be restored
+# instead of re-prefilled.  This endpoint trades that away -- its reuse comes
+# from the slot prompt and from the RAM-side prompt cache below.  Stated
 # explicitly so the tunable stays visible rather than merely absent.
 CTX_CHECKPOINTS=0
-CACHE_RAM=8192
+# PROMPT CACHE, HOST RAM -- this is the one cache tunable that is NOT VRAM.
+# --cache-ram caps the total bytes of saved prompt states
+# (server_prompt_cache, tools/server/server-task.cpp): update() evicts
+# oldest-first past the cap, and alloc() REFUSES outright any single state
+# bigger than the whole cap.  A state is the serialized KV of a whole sequence
+# plus its DeltaNet state plus the draft context's own KV, so at this shape it
+# costs ~68 KiB per token --
+#   16 full-attention layers x 4 kv heads x 256 dim x 2 B x 2 (K,V) = 64 KiB
+#   plus the draft context's window-bounded KV (a ~40 MiB constant for DFlash2)
+# -- which makes the cap a TOKEN budget in disguise: 16384 MiB holds ~240k
+# tokens, i.e. a FULL 163840-token slot (~10.6 GiB) with ~5 GiB of the budget
+# spare for a second entry.
+#
+# The cost is host RAM, allocated on demand and never pinned (plain heap
+# vectors), so no VRAM number above moves.  dankserver has 62 GiB beside the
+# media-stack co-tenants, so the cache is bounded rather than unlimited (-1):
+# the seam test refuses a budget over half the host's RAM.
+CACHE_RAM=16384
 
 # Sampling defaults for requests that send none -- the values the vLLM
 # predecessor served on this endpoint (`--override-generation-config`), so a
@@ -238,35 +247,33 @@ REASONING_BUDGET=-1
 CHAT_TEMPLATE_FILE=
 CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true}'
 
-# Speculative decoding: the DFlash2 block-diffusion drafter PLUS the draftless
-# n-gram matcher.  DFlash2's GGUF carries block_size 8, so at most 7 draft
-# tokens per verification step (llama.cpp asserts n_max <= block_size - 1).
-# The drafter rides ROCm1, the card that ends up holding the target's last
-# layers.
-SPEC_TYPE=draft-dflash,ngram-mod
+# Speculative decoding: the DFlash2 sidecar (operator call 2026-09-19).  The
+# target file also carries its own MTP head (nextn_predict_layers=1), so
+# draft-mtp is one SPEC_TYPE change away -- the drafter file stays on disk.
+# n_max 7 is the measured DFlash2 winner (see the table below); the drafter
+# rides ROCm1, the card that holds the last layers.
+SPEC_TYPE=draft-dflash
 SPEC_DRAFT_N_MAX=7
 SPEC_DRAFT_NGL=999
 SPEC_DRAFT_DEVICE=ROCm1
 SPEC_DRAFT_TYPE_K=f16
 SPEC_DRAFT_TYPE_V=f16
 
-# MTP (the model's own head) vs the drafter -- MEASURED 2026-09-16 (temperature
-# 0, 1k prompt / 512 out; the autoregressive baseline and both MTP depths were
-# run against the deployed configuration in one session, same binary, same
-# prompts):
+# DFlash2 vs the model's own MTP head -- MEASURED 2026-09-16 on the predecessor
+# payload (temperature 0, 1k prompt / 512 out; the autoregressive baseline and
+# both MTP depths were run in one session, same binary, same prompts):
 #
 #   none            17.7 tok/s                     card1 26.07 GB
 #   draft-mtp  n=3  32.4 tok/s  58 % accepted       card1 29.35 GB
 #   draft-mtp  n=7  26.7 tok/s  33 % accepted       card1 29.65 GB
 #   draft-dflash 7  43.5 tok/s  41 % accepted       card1 29.42 GB
 #
-# All four produced IDENTICAL greedy text, so verification is lossless whichever
-# proposer runs.  DFlash2 is worth 2.46x over autoregressive and 34 % over the
-# best MTP; deeper MTP drafting costs more than it returns (n=7 accepts 33 % of
-# 147 proposals, n=3 accepts 58 % of 72).  The MTP head is not free either:
-# selecting it loads blk.64.nextn.* and adds ~3.3 GB to the card that holds it,
-# against 2.06 GB for the drafter's file.  Do not switch this without
-# re-measuring.
+# All four produced IDENTICAL greedy text, so verification is lossless
+# whichever proposer runs.  DFlash2 is the faster proposer (2.46x over
+# autoregressive, 34 % over the best MTP).  The MTP head adds ~3.3 GB to the
+# card that holds it, against 1.06 GB for the Q4_K_M drafter's file -- at the
+# 327680 pool the drafter's window-bounded KV (~40 MiB) leaves headroom on
+# both cards, so the speed wins.
 #
 # THE TWO CANNOT BE CHAINED. `--spec-type draft-mtp,draft-dflash` dies at load:
 #   context type MTP requested but model doesn't contain MTP layers
@@ -278,16 +285,17 @@ SPEC_DRAFT_TYPE_V=f16
 # tuning question. See serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md.
 
 # ngram-mod: the draftless n-gram matcher (CPU hash table in host RAM, no VRAM,
-# no model file, no draft context). It joins the chain unconditionally --
+# no model file, no draft context).  It joins the chain unconditionally --
 # unlike MTP, which needs a draft context and therefore cannot share one with
-# DFlash2. Verified on the loaded server: with ngram-mod as the ONLY spec type,
-# a prose prompt produces ZERO drafts (plain autoregressive) while a
-# prompt-echo prompt accepts 100 % of them, so its drafts are unmistakable.
+# DFlash2.  NOT CURRENTLY ACTIVE: SPEC_TYPE is draft-dflash alone, and the serve
+# script only passes --spec-ngram-mod-* when the type contains "ngram".  The
+# window below is retained because the DFlash2 + ngram chain was measured free
+# on prose and 2.6x on prompt-echo -- one SPEC_TYPE change brings it back.
 #
 # WINDOW MEASURED 2026-09-16 (chain at each window, decode tok/s):
 #
 #   n_min/n_max    prose 1k/512   prompt-echo 1.5k/400
-#   48/64          43.9           200.6     <- deployed
+#   48/64          43.9           200.6
 #   16/32          36.0           161.0
 #    8/16          36.2           113.8
 #
@@ -304,9 +312,7 @@ SPEC_NGRAM_N_MATCH=24
 SPEC_NGRAM_N_MIN=48
 SPEC_NGRAM_N_MAX=64
 
-# Vision enabled: llama-server autoloads mmproj-BF16.gguf from MODEL_DIR
-# (888 MiB, lands on ROCm0 -- the fit above charges it there).  frank-small
-# (Dankclaw, RTX 3090) remains the fleet's dedicated image endpoint; this
-# makes frank-medium multimodal too, which is why the seam flips from
-# "text only" to "vision loads with the model".
+# Vision enabled: llama-server loads mmproj-Qwen3.8-27B-Uncensored-f16.gguf
+# (931 MiB, lands on ROCm0 -- the fit charges it there) via the explicit
+# --mmproj flag.  This keeps frank-medium multimodal across the model swap.
 NO_MMPROJ=0

scripts/.AGENTTRASH/20260920-llama-layersplit/serve-frank-medium.sh.llamacpp-layersplit

@@ -161,8 +161,13 @@ TEMPLATE_ARGS+=(--chat-template-kwargs "$CHAT_TEMPLATE_KWARGS")
 PLACEMENT_ARGS=(--device "$DEVICES" --split-mode "$SPLIT_MODE" --tensor-split "$TENSOR_SPLIT")
 
 MMPROJ_ARGS=()
-[[ "$NO_MMPROJ" == "1" ]] && MMPROJ_ARGS=(--no-mmproj)
-
+if [[ "$NO_MMPROJ" == "1" ]]; then
+    MMPROJ_ARGS=(--no-mmproj)
+else
+    # No autoload in this build: a server without the flag starts text-only
+    # and answers image requests with a 500, so name the projector.
+    MMPROJ_ARGS=(--mmproj "$MM_PROJ")
+fi
 exec "$LLAMA_BIN" \
     --model "$MODEL" \
     --alias "$ALIAS" \

scripts/autoresearch-bench.py

@@ -0,0 +1,141 @@
+#!/usr/bin/env python3
+"""autoresearch-bench.py — deterministic greedy battery for frank-large.
+
+Runs ON Dankclaw (no LAN jitter) against the candidate container on :7770.
+Fixed embedded corpus, temperature 0, streaming so decode speed is measured
+between real token arrivals (not request totals). Median of REPS samples per
+metric is the reported value; medians are far more stable than means under
+GPU thermal drift.
+
+Metrics emitted (stdout, one per line):
+  METRIC decode_tokens_per_sec=<float>   primary, higher better
+  METRIC prefill_tokens_per_sec=<float>  secondary
+  METRIC ttft_ms=<float>                 secondary, lower better
+Exit non-zero if the endpoint is unhealthy or the served alias is wrong.
+"""
+import argparse
+import json
+import statistics
+import sys
+import time
+import urllib.request
+
+# Fixed corpus: deterministic, ~28k tokens after tokenization (same scale as
+# the 2026-10-01 bench window). Built by repeating a fixed paragraph block a
+# fixed number of times - NO random, NO clock, NO network.
+_BLOCK = (
+    "The rendering pipeline walks the scene graph once per frame, resolving "
+    "material bindings into descriptor sets before any draw call is recorded. "
+    "Because the allocator recycles command buffers only after their fence "
+    "signals, a submission that outlives its frame must pin the pool entry; "
+    "otherwise the next frame overwrites recorded commands in flight. The "
+    "validation layer flags exactly this class of lifetime bug, but only for "
+    "buffers it can see - GPU-side timestamps and queue-family ownership "
+    "transfers hide in the driver's own bookkeeping, which is why release "
+    "builds must lean on explicit barriers and why a frame-time regression "
+    "often shows up as corruption one frame later than its cause.\n"
+)
+LONG_CTX = _BLOCK * 220  # ~28k-token prefill, matching the deployed workload
+_LONG = LONG_CTX + "\nNow, in one focused paragraph (80-150 words), explain " \
+    "which lifetime bugs this pipeline can still suffer and what explicit " \
+    "mitigations the text implies."
+_SHORT = "Design a rate limiter for a homelab reverse proxy that must not " \
+    "lock out the LAN. Think it through, then give the algorithm."
+
+REPS = 3
+MAX_TOKENS = 256
+REQ_TIMEOUT = 900
+
+
+def stream_completion(url: str, model: str, prompt: str):
+    """Greedy streaming completion.
+
+    Returns (ttft_s, decode_tps, prefill_tps, prompt_tokens, n_tokens).
+    ttft is RELATIVE to request send: time.perf_counter()'s epoch is host
+    boot, and returning the raw stamp read as an 11-day first token.
+    """
+    body = json.dumps({
+        "model": model,
+        "messages": [{"role": "user", "content": prompt}],
+        "max_tokens": MAX_TOKENS,
+        "temperature": 0,
+        "stream": True,
+        "stream_options": {"include_usage": True},
+    }).encode()
+    req = urllib.request.Request(
+        url.rstrip("/") + "/v1/chat/completions", data=body,
+        headers={"Content-Type": "application/json"})
+    t0 = time.perf_counter()
+    first = None
+    stamps = []
+    usage = {}
+    with urllib.request.urlopen(req, timeout=REQ_TIMEOUT) as resp:
+        for raw in resp:
+            line = raw.decode(errors="replace").strip()
+            if not line.startswith("data:"):
+                continue
+            payload = line[5:].strip()
+            if payload == "[DONE]":
+                break
+            chunk = json.loads(payload)
+            if chunk.get("usage"):
+                usage = chunk["usage"]
+            for ch in chunk.get("choices", []):
+                d = ch.get("delta") or {}
+                # reasoning AND content deltas both arrive token by token;
+                # count either so greedy text length drives the rate.
+                # (llama.cpp names the thinking field reasoning_content.)
+                if d.get("content") or d.get("reasoning") or d.get("reasoning_content"):
+                    now = time.perf_counter()
+                    if first is None:
+                        first = now
+                    stamps.append(now)
+    if first is None or len(stamps) < 2:
+        # A zero-argmax degenerate answer would poison the median; raise.
+        raise RuntimeError(f"stream produced {len(stamps)} tokens")
+    ttft_s = first - t0
+    decode_s = stamps[-1] - stamps[0]
+    decode_tps = (len(stamps) - 1) / decode_s if decode_s > 0 else 0.0
+    prompt_tokens = usage.get("prompt_tokens") or 0
+    prefill_tps = prompt_tokens / ttft_s if prompt_tokens else 0.0
+    return ttft_s, decode_tps, prefill_tps, prompt_tokens, len(stamps)
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--url", default="http://127.0.0.1:7770")
+    ap.add_argument("--model", default="frank-large")
+    args = ap.parse_args()
+
+    with urllib.request.urlopen(args.url.rstrip("/") + "/v1/models",
+                                timeout=10) as r:
+        ids = [m["id"] for m in json.load(r)["data"]]
+    if args.model not in ids:
+        print(f"FATAL: served ids {ids}, expected {args.model}",
+              file=sys.stderr)
+        sys.exit(2)
+
+    # Warmup (compile/graph warm + allocator settle), NOT measured.
+    stream_completion(args.url, args.model, _SHORT)
+
+    decodes, prefills, ttfts = [], [], []
+    for rep in range(REPS):
+        for name, prompt in (("long", _LONG), ("short", _SHORT)):
+            ttft, dt, pt, ptk, ntok = stream_completion(
+                args.url, args.model, prompt)
+            decodes.append(dt)
+            ttfts.append(ttft * 1000.0)
+            if name == "long":
+                prefills.append(pt)
+            print(f"  rep{rep} {name:5s} ttft={ttft*1000:8.1f}ms "
+                  f"prefill={pt:8.1f} t/s decode={dt:6.2f} t/s "
+                  f"({ntok} tok, prompt {ptk})", file=sys.stderr)
+
+    med = lambda xs: statistics.median(xs)
+    print(f"METRIC decode_tokens_per_sec={med(decodes):.2f}")
+    print(f"METRIC prefill_tokens_per_sec={med(prefills):.1f}")
+    print(f"METRIC ttft_ms={med(ttfts):.1f}")
+
+
+if __name__ == "__main__":
+    main()

scripts/autoresearch/frank-primary-arms.sh

@@ -0,0 +1,176 @@
+#!/usr/bin/env bash
+# frank-primary-arms.sh — run a SEQUENCE of frank-primary bench arms and keep
+# every arm's evidence. Runs on the operator box; each arm is one full window
+# of ./autoresearch.sh (stop+mask -> boot the arm as the service -> measure ->
+# restore production and verify), so the fleet is only ever down for one arm at
+# a time and the production unit is back up between arms.
+#
+# An arm is the repo's own frank-primary.env plus an OVERLAY of engine knobs
+# appended after it (later assignment wins when the serve script sources it),
+# staged through FRANK_PRIMARY_AR_ENV_FILE so the checkout is never mutated and
+# each arm gets its own fingerprint (the memo can never answer one arm with
+# another arm's numbers).
+#
+# Usage:  scripts/autoresearch/frank-primary-arms.sh [arm ...]
+#         scripts/autoresearch/frank-primary-arms.sh --list
+# Output, under /tmp/frank-primary-arms-<UTC stamp>/ (override with ARMS_DIR):
+#
+#   <arm>.out        full autoresearch output (metrics + engine evidence)
+#   <arm>.env        the exact staged env
+#   <arm>.rc         window exit code
+#   summary.tsv      arm, rc, decode_tps, decode_tps_t0.7, kv_tokens, boot_s
+#
+# Exit 0 when every arm produced metrics; 1 if any arm failed (a failed boot is
+# a RESULT — the window still restored production, and the rc says which arm).
+set -uo pipefail
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
+REPO="$(cd -- "$SCRIPT_DIR/../.." && pwd)"
+BASE_ENV="$REPO/scripts/frank-primary.env"
+ARMS_DIR="${ARMS_DIR:-/tmp/frank-primary-arms-$(date -u +%Y%m%dT%H%M%SZ)}"
+
+# --- arms ------------------------------------------------------------------
+# baseline: the shipped shape E, measured with THIS harness so the comparison is
+# matched (the ~29.6 t/s figure in the docs is an older shape, and no shape-E
+# decode number exists yet).
+OVER_baseline=()
+# B: MTP3 with the vendor's probabilistic draft sampling, a capture ladder whose
+# step is num_speculative_tokens + 1 = 4, and the sampling override. Precise
+# memory stays ON — this arm is simultaneously the MTP test and the test of
+# whether MTP's drafter workspace and the fitted 5.25 GiB overhead can coexist.
+SPEC3='{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}'
+LADDER3='{"cudagraph_capture_sizes":[4,8,12],"max_cudagraph_capture_size":12,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}'
+# The value must arrive in the staged env file SINGLE-QUOTED: sourcing an env
+# file performs quote removal, so an unquoted {"temperature":0.7} would reach
+# the engine as {temperature:0.7}.
+GEN07='{"temperature":0.7}'
+OVER_B=(
+    "FRANK_PRIMARY_SPEC='$SPEC3'"
+    "FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
+    'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
+)
+# B2: identical to B with the default (7.60 GiB) overhead, i.e. precise memory
+# OFF. It is the fallback if B dies at graph capture, and it prices the arena
+# that precise memory buys (7,522 slots -> ~6,496).
+OVER_B2=(
+    "FRANK_PRIMARY_SPEC='$SPEC3'"
+    "FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
+    'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
+    'FRANK_PRIMARY_PRECISE_MEMORY='
+)
+# C: the vendor's concurrency shape — SEQS 16 with the [4..32] ladder and util
+# 0.95. This is also the direct test of the theory that pinning the capture
+# ladder unlocks SEQS > 3 (the SEQS=2 failure was query_start_loc specialised to
+# the constant 3, which is NOT in the default ladder; 4 is).
+LADDER16='{"cudagraph_capture_sizes":[4,8,12,16,20,24,28,32],"max_cudagraph_capture_size":32,"inductor_compile_config":{"combo_kernels":false,"benchmark_combo_kernel":false}}'
+OVER_C=(
+    "FRANK_PRIMARY_SPEC='$SPEC3'"
+    "FRANK_PRIMARY_COMPCONFIG='$LADDER16'"
+    'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
+    'FRANK_PRIMARY_SEQS=16'
+    'FRANK_PRIMARY_MNBT=4096'
+    'FRANK_PRIMARY_UTIL=0.95'
+)
+
+# D/E: the CONTEXT arms. Measured 2026-10-02 (arms run 20261002T233855Z): the
+# KV pool is a hard byte request divided by a PER-TOKEN PRICE, and MTP raises
+# that price because the drafter keeps its own KV per position:
+#   no spec: 3.15e9 / 398,658 tokens =  7,901 B/token  (1.99x of 200,704)
+#   MTP3:    3.15e9 / 313,002 tokens = 10,064 B/token  (1.56x)
+# So arm C lost a fifth of the context to the drafter, not to util. Frank's
+# contract is >=401,400 tokens over >=2 seqs = 2.00x of 200,704, which at
+# 10,064 B/token costs 4.04e9 -> ask 4.05e9 (+0.83 GiB/rank).
+# D is arm B with EXACTLY ONE variable changed (the KV bytes), so its result
+# says whether the 4k-depth collapse (B 23.09 vs baseline 33.99) was KV or the
+# batch shape.
+KV401K='FRANK_PRIMARY_KV_BYTES=4050000000'
+OVER_D=(
+    "FRANK_PRIMARY_SPEC='$SPEC3'"
+    "FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
+    'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
+    "$KV401K"
+)
+# E: D with C's max_num_batched_tokens (4096 — C is the only arm that did NOT
+# collapse at 4k depth: 33.15 where B gave 23.09) and util 0.99, the value
+# Frank asked about. 0.99 is affordable on paper — production at util 0.98
+# measures 30.37/34.21 GiB per card, 3.58 GiB physically unused — but it leaves
+# only 0.32 GiB above the allowance while the CUDA-graph cost is an ANALYTIC
+# estimate (C's was 1.47 GiB over 24 graphs), so it is measured, not assumed.
+OVER_E=(
+    "FRANK_PRIMARY_SPEC='$SPEC3'"
+    "FRANK_PRIMARY_COMPCONFIG='$LADDER3'"
+    'FRANK_PRIMARY_GEN_CONFIG='"'$GEN07'"
+    "$KV401K"
+    'FRANK_PRIMARY_MNBT=4096'
+    'FRANK_PRIMARY_UTIL=0.99'
+)
+
+arm_overlay() {
+    case "$1" in
+        baseline) ;;                       # the shipped env, no overlay
+        B)  printf '%s\n' "${OVER_B[@]}" ;;
+        B2) printf '%s\n' "${OVER_B2[@]}" ;;
+        C)  printf '%s\n' "${OVER_C[@]}" ;;
+        D)  printf '%s\n' "${OVER_D[@]}" ;;
+        E)  printf '%s\n' "${OVER_E[@]}" ;;
+        *) return 1 ;;
+    esac
+}
+if [ "${1:-}" = "--list" ]; then
+    printf 'baseline B B2 C D E\n'
+    exit 0
+fi
+
+[ -f "$BASE_ENV" ] || { echo "[arms] missing $BASE_ENV" >&2; exit 1; }
+if [ "$#" -gt 0 ]; then ARMS=("$@"); else ARMS=(baseline B B2 C); fi
+mkdir -p "$ARMS_DIR"
+echo "[arms] dir $ARMS_DIR  arms: ${ARMS[*]}" >&2
+
+: >"$ARMS_DIR/summary.tsv"
+printf 'arm\trc\tdecode_tps\tdecode_tps_t0.7\tkv_tokens\tboot_s\n' >>"$ARMS_DIR/summary.tsv"
+overall=0
+
+for arm in "${ARMS[@]}"; do
+    if ! arm_overlay "$arm" >"$ARMS_DIR/$arm.overlay"; then
+        echo "[arms] unknown arm: $arm" >&2
+        overall=1
+        continue
+    fi
+    { cat "$BASE_ENV"; echo; echo "# --- arm $arm overlay ($(date -u +%FT%TZ)) ---"
+      arm_overlay "$arm"; } >"$ARMS_DIR/$arm.env"
+    # Gate the staged env BEFORE burning a ~20-minute window: every knob that
+    # holds JSON must still be one valid JSON object after the file is SOURCED
+    # (that is the step that silently eats unquoted braces).
+    if ! bash -c 'set -a; . "$1" >/dev/null; set +a
+        for k in FRANK_PRIMARY_SPEC FRANK_PRIMARY_COMPCONFIG FRANK_PRIMARY_GEN_CONFIG; do
+            v="${!k:-}"
+            [ -n "$v" ] || continue
+            case "$v" in
+                "{"*) python3 -c "import json,sys; json.loads(sys.argv[1])" "$v" \
+                        || { echo "[arms] $k is not valid JSON: $v" >&2; exit 1; } ;;
+                *) echo "[arms] $k is not a JSON object: $v" >&2; exit 1 ;;
+            esac
+        done' _ "$ARMS_DIR/$arm.env"; then
+        echo "[arms] $arm: staged env rejected, window not opened" >&2
+        echo 99 >"$ARMS_DIR/$arm.rc"
+        overall=1
+        continue
+    fi
+
+    echo "[arms] === $arm: opening window $(date -u +%H:%M:%S)Z ===" >&2
+    FRANK_PRIMARY_AR_ENV_FILE="$ARMS_DIR/$arm.env" \
+        "$REPO/autoresearch.sh" >"$ARMS_DIR/$arm.out" 2>&1
+    rc=$?
+    echo "$rc" >"$ARMS_DIR/$arm.rc"
+    echo "[arms] === $arm: rc=$rc $(date -u +%H:%M:%S)Z ===" >&2
+    grep -E '^METRIC ' "$ARMS_DIR/$arm.out" | sed 's/^METRIC //' >"$ARMS_DIR/$arm.metrics" || true
+    metric() { local v; v="$(sed -n "s/^$1=//p" "$ARMS_DIR/$arm.metrics" | tail -1)"; printf '%s' "${v:-$2}"; }
+    printf '%s\t%s\t%s\t%s\t%s\t%s\n' "$arm" "$rc" \
+        "$(metric decode_tps FAIL)" "$(metric decode_tps_t0.7 FAIL)" \
+        "$(metric kv_tokens ?)" "$(metric boot_seconds ?)" >>"$ARMS_DIR/summary.tsv"
+    [ "$rc" -ne 0 ] && overall=1
+done
+
+echo "[arms] summary:" >&2
+cat "$ARMS_DIR/summary.tsv" >&2
+exit "$overall"

scripts/autoresearch/frank-primary-measure.py

@@ -0,0 +1,216 @@
+#!/usr/bin/env python3
+"""frank-primary decode measurement — runs INSIDE the engine container.
+
+The method is the fleet's established one (the 2026-09-29 / 2026-10-01
+tuning campaigns, /home/dank/mtp_measure.py): SSE chunk counting is invalid
+under speculative decoding because rejected drafts change the accounting, so
+decode is measured from the server's own usage counters:
+
+    decode_tok_s = completion_tokens / (wall - ttft)
+
+with ttft = wall time to the FIRST content-bearing chunk (thinking arrives in
+`reasoning`/`delta.reasoning` on this stack and is excluded from the decode
+window — and is deterministic per prompt at temperature 0, so the exclusion
+cancels in any A/B on the same workload).
+
+Workload (fixed, deterministic, offline):
+  * prompts built from a large fixed-seed shuffled word list so tokens rarely
+    repeat (no n-gram/prefix predictability of the text itself), sized with
+    the server's own /v1/messages/count_tokens counter
+  * depths 115 / 1056 / 4096 target prompt tokens
+  * one 64-token warm pass per depth (warms the expert arena), then
+    max_tokens=256, 3 runs, median — repeated for every temperature in AR_TEMPS
+    (default 0 and 0.7). The prompt workload is deterministic (fixed seed, no
+    network, no time-of-day dependence); at temperature > 0 the generated text
+    is not, but the measured window is bounded by max_tokens, so the rate stays
+    comparable and the non-determinism is exactly the sampling that a
+    speculative-decode arm must survive.
+
+Env:
+  AR_BASE             endpoint (default http://127.0.0.1:7770)
+  AR_MODEL            served name (default frank-primary)
+  AR_PROMPT_CEILING   hard prompt ceiling in tokens (default 100000)
+  AR_TEMPS            comma-separated sampling temperatures (default "0,0.7")
+
+Output: progress on stderr, ONE json object on stdout (the last line).
+Exit codes: 0 ok, 2 endpoint unhealthy / no usable completion, 3 bad usage.
+"""
+import json
+import os
+import random
+import statistics
+import sys
+import time
+import urllib.error
+import urllib.request
+
+BASE = os.environ.get("AR_BASE", "http://127.0.0.1:7770")
+MODEL = os.environ.get("AR_MODEL", "frank-primary")
+PROMPT_CEILING = int(os.environ.get("AR_PROMPT_CEILING", "100000"))
+DEPTHS = (115, 1056, 4096)
+WARM_TOKENS = 64
+MAX_TOKENS = 256
+RUNS = 3
+# Temperature sweep. The unsuffixed keys are temperature 0 — byte-comparable to
+# every historical measurement — and AR_TEMPS adds more. The second one matters
+# for speculative decode: draft ACCEPTANCE is sampling-dependent (a greedy draft
+# matches a greedy target exactly and is rejected more under a non-greedy
+# target), so an MTP arm judged only at temp 0 would be judged on half the
+# contract. Production traffic is non-zero temperature (the payload's
+# generation_config says 1.0; an override to 0.7 is under test).
+TEMPS = tuple(float(x) for x in os.environ.get("AR_TEMPS", "0,0.7").split(","))
+
+# 43-word x 12 = 516-word vocabulary, seed 1234 — the campaign's exact list
+# and seed, so prompts are byte-identical to the historical measurements.
+_VOCAB = ("alpha bravo charlie delta echo foxtrot golf hotel india juliet kilo lima "
+          "mike november oscar papa quebec romeo sierra tango uniform victor whiskey "
+          "xray yankee zulu cardinal ordinal topology kernel scheduler resident "
+          "unified aperture expert arena slab offload prefetch gemm triton quant "
+          "tensor shard checkpoint manifest embed router cache page table window "
+          "gradient sparse dense moe attention head dim block stream decode prefill "
+          "bandwidth latency throughput residency eviction tier swap pin memlock").split()
+_R = random.Random(1234)
+_ORDER = list(range(len(_VOCAB)))
+_R.shuffle(_ORDER)
+
+
+def log(msg):
+    print("[measure] %s" % msg, file=sys.stderr, flush=True)
+
+
+def post_json(path, body, timeout):
+    req = urllib.request.Request(
+        BASE + path, data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"})
+    with urllib.request.urlopen(req, timeout=timeout) as r:
+        return json.load(r)
+
+
+def count_tokens(text):
+    """Token count without a prefill, from the server's own counter."""
+    body = {"model": MODEL,
+            "messages": [{"role": "user", "content": text}]}
+    r = post_json("/v1/messages/count_tokens", body, timeout=300)
+    n = r.get("input_tokens") or r.get("count") or r.get("token_count")
+    if not n:
+        raise RuntimeError("count_tokens returned no count: %r" % (r,))
+    return int(n)
+
+
+def words(n):
+    return " ".join(_VOCAB[_ORDER[i % len(_VOCAB)]] for i in range(n))
+
+
+def calibrate():
+    """Measure words->tokens from the server's own counter."""
+    return 2000.0 / count_tokens(words(2000))
+
+
+def prompt_for(target_tokens, wpt):
+    n = max(1, int(target_tokens * wpt))
+    text = words(n)
+    pt = count_tokens(text)
+    while pt > PROMPT_CEILING:
+        n = max(1, int(n * (PROMPT_CEILING / pt) * 0.98))
+        text = words(n)
+        pt = count_tokens(text)
+    return text, pt
+
+
+def measured(prompt, max_tokens, temperature=0):
+    """Return (ttft_s, decode_tok_s, completion_tokens) via usage counters."""
+    body = {"model": MODEL,
+            "messages": [{"role": "user", "content": prompt}],
+            "max_tokens": max_tokens, "temperature": temperature,
+            "stream": True, "stream_options": {"include_usage": True}}
+    req = urllib.request.Request(
+        BASE + "/v1/chat/completions", data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"})
+    t0 = time.time()
+    ttft = None
+    usage = {}
+    with urllib.request.urlopen(req, timeout=1800) as r:
+        for raw in r:
+            line = raw.decode("utf-8", "replace").strip()
+            if not line.startswith("data:"):
+                continue
+            s = line[5:].strip()
+            if s == "[DONE]":
+                break
+            try:
+                ev = json.loads(s)
+            except json.JSONDecodeError:
+                continue
+            if ev.get("usage"):
+                usage = ev["usage"]
+            for ch in ev.get("choices", []):
+                if (ch.get("delta") or {}).get("content"):
+                    if ttft is None:
+                        ttft = time.time() - t0
+    wall = time.time() - t0
+    ct = int(usage.get("completion_tokens") or 0)
+    if ct <= 0:
+        # No usage on the stream: the endpoint is unhealthy or the model
+        # returned nothing (e.g. HTTP 200 with an error payload).
+        raise RuntimeError("no completion_tokens in usage: %r" % (usage,))
+    dec_window = max(wall - (ttft or 0.0), 1e-9)
+    return (ttft or wall), (ct / dec_window), ct
+
+
+def fmt_temp(t):
+    return ("%.1f" % t).rstrip("0").rstrip(".") if t else "0"
+
+
+def main():
+    log("base=%s model=%s ceiling=%d temps=%s"
+        % (BASE, MODEL, PROMPT_CEILING, ",".join(fmt_temp(t) for t in TEMPS)))
+    try:
+        wpt = calibrate()
+    except Exception as e:
+        log("endpoint unhealthy during calibration: %s" % e)
+        return 2
+    log("words/token: %.3f" % wpt)
+
+    result = {"words_per_token": round(wpt, 5)}
+    for tgt in DEPTHS:
+        try:
+            p, pt = prompt_for(tgt, wpt)
+            measured(p, WARM_TOKENS)  # warm the arena and the prefix cache
+        except Exception as e:
+            log("warm pass failed at depth %d: %s" % (tgt, e))
+            return 2
+        for temp in TEMPS:
+            suffix = "" if temp == 0 else "_t%s" % fmt_temp(temp)
+            rates, tts, cts = [], [], []
+            for i in range(RUNS):
+                try:
+                    tt, r, ct = measured(p, MAX_TOKENS, temp)
+                except Exception as e:
+                    log("run %d/%d failed at depth %d temp %s: %s"
+                        % (i + 1, RUNS, tgt, fmt_temp(temp), e))
+                    continue
+                rates.append(r)
+                tts.append(tt)
+                cts.append(ct)
+            if len(rates) < 2:
+                log("only %d usable run(s) at depth %d temp %s — refusing to median"
+                    % (len(rates), tgt, fmt_temp(temp)))
+                return 2
+            result["decode_tps_%d%s" % (tgt, suffix)] = round(statistics.median(rates), 2)
+            result["ttft_ms_%d%s" % (tgt, suffix)] = round(1000.0 * statistics.median(tts), 1)
+            result["completion_tokens_%d%s" % (tgt, suffix)] = int(statistics.median(cts))
+            result["prompt_tokens_%d%s" % (tgt, suffix)] = pt
+            log("depth %5d tok temp %-4s decode %7.2f t/s  ttft %.3f s  (runs=%d)"
+                % (pt, fmt_temp(temp), result["decode_tps_%d%s" % (tgt, suffix)],
+                   result["ttft_ms_%d%s" % (tgt, suffix)] / 1000.0, len(rates)))
+
+    for temp in TEMPS:
+        suffix = "" if temp == 0 else "_t%s" % fmt_temp(temp)
+        meds = [result["decode_tps_%d%s" % (t, suffix)] for t in DEPTHS]
+        result["decode_tps%s" % suffix] = round(statistics.median(meds), 2)
+    print(json.dumps(result))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/autoresearch/frank-primary-window.sh

@@ -0,0 +1,218 @@
+#!/usr/bin/env bash
+# frank-primary-window.sh — one full bench window for frank-primary, run ON
+# DANKSERVER as the deploy user (invoked over ssh by ./autoresearch.sh).
+#
+# Discipline (AGENTS.md Benchmarking Policy): the production unit is
+# STOPPED + MASKED first, the candidate config boots AS THE SERVICE itself
+# (same serve script, same container name, same port 7770 — no scratch
+# ports, no co-tenant GPU), the in-container measure runs, and the window
+# ALWAYS ends by restoring the production unit and verifying its health.
+# The EXIT trap makes the restore unconditional: a boot failure, a measure
+# failure, or a dead ssh session all end with production back up.
+#
+# Staging layout (created by the orchestrator, ephemeral — never a config
+# source; the repo checkout is):
+#   /home/dank/.autoresearch-frank-primary/
+#     serve.sh             staged candidate serve script
+#     frank-primary.env    staged candidate env (the candidate knobs)
+#     measure.py           the in-container measure client
+#     hash                 candidate fingerprint
+#     run.json             result (metrics + boot + kv + fingerprint)
+#     serve.out/measure.out/fail.log
+#     last/                memo: run.json + hash of the last FULLY
+#                          successful window (bench AND verified restore)
+#
+# This script logs to stderr and exits 0 only when the measure produced a
+# usable result (the restore happens in the trap regardless of the bench
+# outcome; a failed restore forces exit 1).
+set -uo pipefail
+
+STAGE="${FRANK_PRIMARY_AR_STAGE:-/home/dank/.autoresearch-frank-primary}"
+UNIT="${FRANK_PRIMARY_AR_UNIT:-frank-primary}"
+CTR="${FRANK_PRIMARY_AR_CTR:-frank-primary-qwen}"
+HEALTH_URL="http://127.0.0.1:7770/health"
+MODELS_URL="http://127.0.0.1:7770/v1/models"
+BOOT_TIMEOUT="${FRANK_PRIMARY_AR_BOOT_TIMEOUT:-1200}"
+RESTORE_TIMEOUT="${FRANK_PRIMARY_AR_RESTORE_TIMEOUT:-1200}"
+MEASURE_TIMEOUT="${FRANK_PRIMARY_AR_MEASURE_TIMEOUT:-1800}"
+SERVED="$(sed -n 's/^FRANK_PRIMARY_SERVED_NAME=//p' "$STAGE/frank-primary.env" 2>/dev/null | head -1)"
+SERVED="${SERVED:-frank-primary}"
+
+RESTORED=0
+
+log() { printf '[window %s] %s\n' "$(date '+%H:%M:%S')" "$*" >&2; }
+
+container_state() { podman inspect -f '{{.State.Status}}' "$CTR" 2>/dev/null || echo absent; }
+
+wait_port_free() {
+    local i
+    for i in $(seq 1 90); do
+        [ "$(container_state)" != "running" ] && return 0
+        sleep 2
+    done
+    return 1
+}
+
+wait_health() { # $1 = timeout seconds
+    local t0 code
+    t0=$(date +%s)
+    while :; do
+        code="$(curl -s -o /dev/null -w '%{http_code}' --max-time 10 "$HEALTH_URL" 2>/dev/null || echo 000)"
+        [ "$code" = "200" ] && return 0
+        [ "$(( $(date +%s) - t0 ))" -ge "$1" ] && return 1
+        sleep 2
+    done
+}
+
+capture_bench_failure() {
+    log "capturing bench-boot failure evidence"
+    {
+        echo "=== podman logs (tail 400) ==="
+        podman logs --tail 400 "$CTR" 2>&1 || true
+        echo "=== staged serve.out (tail 60) ==="
+        tail -60 "$STAGE/serve.out" 2>/dev/null || true
+        echo "=== engine log (tail 60) ==="
+        tail -60 /home/dank/logs/frank_primary.log 2>/dev/null || true
+    } >"$STAGE/fail.log" 2>&1 || true
+    grep -E 'OutOfMemory|Tried to allocate|Engine core initialization failed|Error|error' \
+        "$STAGE/fail.log" 2>/dev/null | tail -25 >&2 || true
+}
+
+restore() {
+    [ "$RESTORED" = 1 ] && return 0
+    RESTORED=1
+    log "restoring production unit $UNIT"
+    if [ -f "$STAGE/serve.pid" ]; then
+        kill "$(cat "$STAGE/serve.pid" 2>/dev/null)" 2>/dev/null || true
+    fi
+    podman stop -t 120 "$CTR" >/dev/null 2>&1 || true
+    wait_port_free || log "WARNING: bench container still draining after 180s"
+    sudo systemctl unmask "$UNIT" 2>/dev/null || true
+    if ! sudo systemctl start "$UNIT"; then
+        log "ERROR: systemctl start $UNIT failed — operator intervention required"
+        return 1
+    fi
+    if wait_health "$RESTORE_TIMEOUT"; then
+        if curl -s --max-time 10 "$MODELS_URL" 2>/dev/null | grep -q "$SERVED"; then
+            log "production restored and verified ($SERVED answering on :7770)"
+            # Memo promotion lives HERE: a cached result is only ever the
+            # product of a fully successful window (bench AND verified
+            # restore), never of a bare bench success.
+            cp "$STAGE/run.json" "$STAGE/last/" 2>/dev/null || true
+            cp "$STAGE/hash" "$STAGE/last/" 2>/dev/null || true
+        else
+            log "ERROR: health 200 but /v1/models does not list $SERVED"
+            return 1
+        fi
+    else
+        log "ERROR: production not healthy within ${RESTORE_TIMEOUT}s — unit left to systemd (Restart=always)"
+        return 1
+    fi
+    return 0
+}
+trap 'restore; rc=$?; [ "$rc" -ne 0 ] && exit 1' EXIT
+
+main() {
+    local f
+    for f in serve.sh frank-primary.env measure.py hash; do
+        [ -f "$STAGE/$f" ] || { log "staged file missing: $STAGE/$f"; exit 1; }
+    done
+    mkdir -p "$STAGE/last"
+    rm -f "$STAGE/run.json" "$STAGE/fail.log" "$STAGE/measure.out" "$STAGE/evidence.log"
+
+    local t_start t0 i kvline kv_tokens kv_conc measure_json boot_seconds
+    t_start="$(date +%s)"
+
+    log "stopping and masking production unit $UNIT"
+    sudo systemctl stop "$UNIT" 2>/dev/null || true
+    # Explicit stop is not auto-restarted even with Restart=always; the mask
+    # is belt-and-braces against a stray start during the window.
+    sudo systemctl mask "$UNIT" 2>/dev/null || \
+        log "WARNING: mask failed — relying on explicit-stop semantics"
+    wait_port_free || log "note: production container still draining"
+
+    t0="$(date +%s)"
+    log "starting bench instance from staged config (fingerprint $(tr -d '[:space:]' <"$STAGE/hash"))"
+    FRANK_PRIMARY_ENV_FILE="$STAGE/frank-primary.env" \
+        setsid nohup bash "$STAGE/serve.sh" >"$STAGE/serve.out" 2>&1 </dev/null &
+    echo "$!" >"$STAGE/serve.pid"
+
+    for i in $(seq 1 45); do
+        [ "$(container_state)" = "running" ] && break
+        sleep 2
+    done
+    if [ "$(container_state)" != "running" ]; then
+        log "bench container never reached running state"
+        tail -40 "$STAGE/serve.out" >&2 || true
+        exit 1
+    fi
+
+    if wait_health "$BOOT_TIMEOUT"; then
+        boot_seconds=$(( $(date +%s) - t0 ))
+        log "bench healthy after ${boot_seconds}s"
+    else
+        if [ "$(container_state)" = "running" ]; then
+            log "bench boot timed out after ${BOOT_TIMEOUT}s (still loading)"
+        else
+            log "bench container died during boot"
+            capture_bench_failure
+        fi
+        exit 1
+    fi
+
+    log "running measure in container (timeout ${MEASURE_TIMEOUT}s)"
+    if ! podman cp "$STAGE/measure.py" "$CTR":/tmp/ar-measure.py; then
+        log "podman cp of measure client failed"
+        exit 1
+    fi
+    if ! timeout "$MEASURE_TIMEOUT" podman exec "$CTR" python3 /tmp/ar-measure.py \
+            >"$STAGE/measure.out" 2>&1; then
+        log "measure failed or timed out"
+        tail -40 "$STAGE/measure.out" >&2 || true
+        exit 1
+    fi
+
+    # Capacity facts for the run (context-size experiments change these).
+    kvline="$(podman logs "$CTR" 2>&1 | grep -E 'GPU KV cache size' | tail -1 || true)"
+    kv_tokens="$(printf '%s' "$kvline" | sed -E 's/.*GPU KV cache size: ([0-9,]+) tokens.*/\1/' | tr -d ',' || true)"
+    kv_conc="$(printf '%s' "$kvline" | sed -E 's/.*per request: ([0-9.]+x).*/\1/' || true)"
+
+    # Human-facing evidence, in a sidecar file because it is multi-line prose,
+    # not metrics: the expert-plan lines (the ONLY place the arena / runtime
+    # overhead / KV-reserve numbers exist — they are never in /metrics), the
+    # compile path (cold build vs cache hit), the graph-capture sizes, the
+    # speculative-decode counters, and any OOM signature. Without this an arm
+    # that loses decode speed is unexplainable afterwards.
+    {
+        echo "=== engine plan / capacity / spec evidence ==="
+        podman logs "$CTR" 2>&1 | grep -E \
+'KV cache size|Maximum concurrency|layers split|expert cache|runtime overhead|non-default args|r4d all-reduce|Using \[|cudagraph|CUDA graph|Capturing|graph for compile range|torch.compile took|Directly load the compiled|speculative|Speculative|acceptance|Acceptance|offload|precise|OutOfMemory|Tried to allocate|Engine core initialization failed' \
+            | tail -150 || true
+        echo "=== /metrics counters (spec decode, caches, success) ==="
+        curl -s --max-time 15 http://127.0.0.1:7770/metrics 2>/dev/null \
+            | grep -E '^vllm:(spec_decode|gpu_cache|prefix_cache|request_success|generation_tokens|iteration)' \
+            | head -80 || true
+    } >"$STAGE/evidence.log" 2>&1 || true
+    log "engine evidence -> $STAGE/evidence.log ($(wc -l <"$STAGE/evidence.log") lines)"
+
+    measure_json="$(grep -E '^\{' "$STAGE/measure.out" | tail -1 || true)"
+    if [ -z "$measure_json" ]; then
+        log "measure produced no JSON result line"
+        tail -40 "$STAGE/measure.out" >&2 || true
+        exit 1
+    fi
+
+    # The measure line is already JSON, so the run object can be assembled
+    # without any host-side interpreter.
+    printf '{"hash":"%s","boot_seconds":%s,"kv_tokens":%s,"kv_concurrency":"%s","total_seconds":%s,"measure":%s}\n' \
+        "$(tr -d '[:space:]' <"$STAGE/hash")" \
+        "$boot_seconds" "${kv_tokens:-0}" "${kv_conc:-unknown}" \
+        "$(( $(date +%s) - t_start ))" "$measure_json" >"$STAGE/run.json"
+
+    log "bench window complete in $(( $(date +%s) - t_start ))s"
+    # Memo only after a FULL success; the restore below still has to verify.
+    exit 0
+}
+
+main "$@"
+exit "$?"

scripts/bench-conversation.py

@@ -0,0 +1,269 @@
+#!/usr/bin/env python3
+"""bench-conversation.py — what an endpoint actually feels like, in tokens/s.
+
+The other harnesses here answer engine questions (which batch, which proposer,
+which cache). This one answers the user question: give the same ten jobs to an
+OpenAI-compatible endpoint and report the rate it answers them at. Every job is
+a real conversation turn — system prompt plus one user turn — because that is
+the shape the fleet serves, and it is measured by STREAMING so prefill and
+decode are separated:
+
+  ttft          seconds to the first content delta  -> prefill + queue
+  decode tok/s  completion_tokens / (total - ttft)  -> the rate you read at
+
+Both arms of a comparison must run the SAME prompt bytes, so a per-run salt
+rides in the system prompt: it changes the prompt itself, which is llama.cpp's
+prefix-cache key and vLLM's APC key, so no arm silently inherits a warm cache
+from the one before it. Run one arm, restart the engine, run the next with a
+new salt.
+
+Answers are generated at temperature 0 so arms serving the SAME model can be
+compared for identity as well as speed; different runtimes serving different
+quants legitimately answer differently, which is why the first line of every
+answer is printed.
+
+Usage:
+    bench-conversation.py <base-url> [--model NAME] [--salt TAG]
+                          [--max-tokens N] [--burst N] [--out file.json]
+
+    bench-conversation.py http://127.0.0.1:7770 --model frank-medium --salt L1
+    bench-conversation.py http://127.0.0.1:7770 --model frank-medium --salt R2 --burst 4
+
+One engine at a time: on a single GPU pair there is no room for two copies of a
+27B model, and the benchmarking policy requires the production service to be
+stopped while anything measures.
+"""
+
+import argparse
+import json
+import statistics
+import sys
+import threading
+import time
+import urllib.error
+import urllib.request
+
+# Ten jobs that cover what this fleet is actually asked to do. Ordered roughly
+# by prefill size so a runtime that is good at short prompts and one that is
+# good at long ones cannot hide behind an average.
+SYSTEM = (
+    "You are a staff engineer on a homelab fleet. Be concise and concrete. "
+    "Salt: {salt}"
+)
+
+BATTERY = [
+    ("greet", 128, "Say hello in one sentence."),
+    ("shell", 192, "One bash command: find every file under /home modified in "
+     "the last 2 days, sorted by size, largest first."),
+    ("math", 256, "A train covers 180 km in 1h40m, then 240 km in 2h20m. "
+     "What is the average speed in km/h? Show the arithmetic."),
+    ("json", 384, "Emit exactly 12 JSON records, one per line, no prose, each "
+     'with keys "host", "service", "port" and "up" for hosts dankclaw, '
+     "dankserver and dankware running llama-server, caddy and grafana."),
+    ("review", 512, "Review this function for bugs and name the failure mode:\n"
+     "def rotate(cfg, n):\n    for k in cfg:\n        cfg[k] = cfg[k][n:] + cfg[k][:n]\n"
+     "    return cfg\n"),
+    ("explain", 448, "Explain in four sentences why PCIe atomic operation routing "
+     "matters for GPU collectives, and what fails when a bridge reports "
+     "AtomicOpsCap: Routing-."),
+    ("translate", 256, "Translate to German, then to Japanese, one line each: "
+     "The load balancer refused the connection because the port was still held."),
+    ("plan", 640, "Give a 6-step runbook for swapping the model behind a live "
+     "LLM endpoint on a host that serves real users, with a rollback at each step."),
+    ("debug", 512, "A systemd user service restarts every 10 seconds and its log "
+     "ends with 'Error starting server failed to bind udp listener on 10.89.1.1:53: "
+     "Cannot assign requested address'. What is the root cause and the repair?"),
+    ("longdoc", 512, None),  # filled from LONG_FILES at runtime
+]
+
+# The long-context job reads real repo files so prefill is large and identical
+# across arms (~30k characters).
+LONG_FILES = [
+    "/home/dank/serverAppConfigs/AGENTS.md",
+    "/home/dank/serverAppConfigs/serverAppConfigs-docs/LLM/"
+    "frank-medium-llamacpp-dflash2.md",
+]
+LONG_CHARS = 30_000
+
+
+def long_prompt() -> str:
+    parts = []
+    for path in LONG_FILES:
+        try:
+            with open(path, encoding="utf-8", errors="replace") as handle:
+                parts.append(handle.read())
+        except OSError:
+            continue
+    text = "\n\n".join(parts)[:LONG_CHARS]
+    return ("Using only this document, answer: which deployment shapes for "
+            "two GPUs are ruled out by missing PCIe atomics, and what is the "
+            "one software path that still lets both cards serve one model? "
+            "Cite the measured numbers.\n\n---\n" + text)
+
+
+def stream_chat(base: str, model: str, messages, max_tokens: int, timeout: int):
+    """One streamed chat completion -> (ttft, total, usage, delta_count, text).
+
+    usage is the engine's own {prompt_tokens, completion_tokens}, asked for with
+    stream_options.include_usage.  There is deliberately NO cheap non-streamed
+    pre-probe to learn the prompt length: that probe would populate the prefix
+    cache and turn the measured run's prefill into a cache hit, which is exactly
+    the difference between runtimes this harness is trying to see.
+
+    ttft is None when the engine produced no content delta.
+    """
+    body = {
+        "model": model,
+        "messages": messages,
+        "max_tokens": max_tokens,
+        "temperature": 0,
+        "stream": True,
+        "stream_options": {"include_usage": True},
+    }
+    request = urllib.request.Request(
+        base + "/v1/chat/completions",
+        data=json.dumps(body).encode(),
+        headers={"Content-Type": "application/json"},
+    )
+    start = time.perf_counter()
+    ttft = None
+    deltas = 0
+    usage = {}
+    text = []
+    with urllib.request.urlopen(request, timeout=timeout) as response:
+        for raw in response:
+            line = raw.decode("utf-8", "replace").strip()
+            if not line.startswith("data:"):
+                continue
+            payload = line[5:].strip()
+            if payload == "[DONE]":
+                break
+            try:
+                chunk = json.loads(payload)
+            except json.JSONDecodeError:
+                continue
+            if chunk.get("usage"):
+                usage.update(chunk["usage"])
+            for choice in chunk.get("choices", []):
+                piece = choice.get("delta") or {}
+                # llama.cpp streams thinking as reasoning_content; vLLM's
+                # reasoning parser streams it as reasoning.  Either one is a
+                # generated token, so TTFT must be the first token of ANY kind —
+                # counting only `content` made a thinking-heavy arm look like it
+                # answered instantly (measured 2026-09-20: radiance's geomean
+                # read 994 tok/s with 6 of 10 jobs recorded as "(empty)").
+                content = next((piece[k] for k in ("content", "reasoning",
+                               "reasoning_content") if piece.get(k)), "")
+                if content:
+                    if ttft is None:
+                        ttft = time.perf_counter() - start
+                    deltas += 1
+                    text.append(content)
+    total = time.perf_counter() - start
+    return ttft, total, usage, deltas, "".join(text)
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("base")
+    ap.add_argument("--model", default="frank-medium")
+    ap.add_argument("--salt", default=f"C{int(time.time())}")
+    ap.add_argument("--max-tokens", type=int, default=None)
+    ap.add_argument("--burst", type=int, default=0,
+                    help="also run N concurrent copies of the plan job")
+    ap.add_argument("--timeout", type=int, default=1800)
+    ap.add_argument("--out", default=None)
+    args = ap.parse_args()
+
+    base = args.base.rstrip("/")
+    rows = []
+    print(f"# battery salt={args.salt} model={args.model} base={base}\n")
+    print(f"{'job':<10} {'ptok':>7} {'ctok':>6} {'ttft_s':>8} {'dec_tps':>8} "
+          f"{'total_s':>8}  answer")
+
+    for name, cap, prompt in BATTERY:
+        if prompt is None:
+            prompt = long_prompt()
+        messages = [
+            {"role": "system", "content": SYSTEM.format(salt=args.salt)},
+            {"role": "user", "content": prompt},
+        ]
+        limit = args.max_tokens or cap
+        try:
+            ttft, total, usage, deltas, text = stream_chat(
+                base, args.model, messages, limit, args.timeout)
+        except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError) as exc:
+            print(f"{name:<10} FAILED {exc}")
+            rows.append({"job": name, "error": str(exc)})
+            continue
+        ptok = usage.get("prompt_tokens")
+        ctok = usage.get("completion_tokens") or deltas
+        decode = None
+        if ttft is not None and ctok and total > ttft:
+            decode = (ctok - 1) / (total - ttft)
+        prefill = (ptok / ttft) if (ptok and ttft) else None
+        first = text.strip().splitlines()[0][:48] if text.strip() else "(empty)"
+        print(f"{name:<10} {ptok or 0:>7} {ctok:>6} "
+              f"{(ttft or 0):>8.2f} {(decode or 0):>8.1f} {total:>8.2f}  {first}")
+        rows.append({"job": name, "prompt_tokens": ptok, "completion_tokens": ctok,
+                     "ttft_s": ttft, "decode_tps": decode, "prefill_tps": prefill,
+                     "total_s": total, "answer_head": first})
+
+    good = [r for r in rows if r.get("decode_tps")]
+    if good:
+        rates = [r["decode_tps"] for r in good]
+        ttfts = [r["ttft_s"] for r in good]
+        pre = [r["prefill_tps"] for r in good if r.get("prefill_tps")]
+        print(f"\ngeomean decode  {statistics.geometric_mean(rates):8.1f} tok/s"
+              f"   (min {min(rates):.1f} / max {max(rates):.1f})")
+        print(f"mean ttft       {statistics.mean(ttfts):8.2f} s")
+        if pre:
+            print(f"geomean prefill {statistics.geometric_mean(pre):8.1f} tok/s")
+
+    if args.burst > 1:
+        prompt = next(p for n, _, p in BATTERY if n == "plan")
+        messages = [
+            {"role": "system", "content": SYSTEM.format(salt=args.salt + "B")},
+            {"role": "user", "content": prompt},
+        ]
+        results = []
+        lock = threading.Lock()
+
+        def worker():
+            try:
+                ttft, total, usage, deltas, _ = stream_chat(
+                    base, args.model, messages, 640, args.timeout)
+                ctok = usage.get("completion_tokens") or deltas
+                with lock:
+                    results.append((ttft, total, ctok))
+            except Exception as exc:  # noqa: BLE001 - report, do not abort the burst
+                with lock:
+                    results.append((None, 0.0, str(exc)))
+
+        threads = [threading.Thread(target=worker) for _ in range(args.burst)]
+        t0 = time.perf_counter()
+        for thread in threads:
+            thread.start()
+        for thread in threads:
+            thread.join()
+        wall = time.perf_counter() - t0
+        ok = [r for r in results if r[0] is not None]
+        if ok:
+            tokens = sum(r[2] for r in ok if isinstance(r[2], int))
+            print(f"burst {args.burst}x plan: wall {wall:.1f}s  aggregate "
+                  f"{tokens / wall:.1f} tok/s  "
+                  f"per-stream {statistics.mean([r[2] / r[1] for r in ok]):.1f} tok/s"
+                  f"  mean ttft {statistics.mean([r[0] for r in ok]):.2f}s")
+        else:
+            print(f"burst {args.burst}x plan: all failed: {results[0][2]}")
+
+    if args.out:
+        with open(args.out, "w", encoding="utf-8") as handle:
+            json.dump({"salt": args.salt, "model": args.model, "base": base,
+                       "rows": rows, "burst": args.burst}, handle, indent=1)
+        print(f"wrote {args.out}")
+    return 0 if good else 1
+
+
+if __name__ == "__main__":
+    sys.exit(main())

scripts/build-gufo.sh

@@ -0,0 +1,96 @@
+#!/usr/bin/env bash
+# Build the gufo runtime that serves frank-large on Dankclaw.
+#
+#   bash scripts/build-gufo.sh              # build + install the pinned tag
+#   bash scripts/build-gufo.sh --verify     # only check the installed tree
+#
+# gufo (gufo-org/gufo) is the Strix Halo inference engine: it targets
+# gfx1151 specifically and has no prebuilt Linux release binaries, so the
+# runtime is built from the pinned git tag and installed to
+# /home/dank/GitPackages/gufo-<tag>/. Everything below is what Dankclaw
+# needs on top of AMD's Linux instructions.
+#
+# The host already carries the ROCm 7.2.4 stack (/opt/rocm -> /etc/
+# alternatives/rocm) with the HIP dev packages, so this script installs
+# only the ordinary system libraries gufo links against.
+#
+# Two host facts this build depends on, both discovered on 2026-09-28:
+#
+#   1. Ubuntu's librocwmma-dev ships a TRUNCATED rocWMMA header set (the
+#      public headers without rocwmma/internal/) into /usr/include, where
+#      it shadows AMD's complete copy under /opt/rocm/include. The first
+#      build died on "'internal/accessors.hpp' file not found" from
+#      /usr/include/rocwmma/rocwmma.hpp. A -I cannot fix it: in HIP mode
+#      the host compilation pass searches the system directories first.
+#      The package has no reverse dependencies, so it is removed.
+#   2. The ROCm clang picks the NEWEST /usr/include/c++ tree, which on
+#      Ubuntu 26.04 is GCC 16's. Its <atomic_wait.h> emits a symbol
+#      reference the installed libstdc++ (GLIBCXX_3.4.35, GCC 15) does not
+#      export, and the link fails with "undefined symbol:
+#      std::__detail::__notify_impl". --gcc-install-dir pins the headers to
+#      the same GCC the runtime library came from.
+set -euo pipefail
+
+GUFO_TAG="${GUFO_TAG:-v0.1.1}"
+GUFO_REPO="${GUFO_REPO:-https://github.com/gufo-org/gufo}"
+# The commit the tag resolved to when this runtime was qualified. A tag that
+# moves is an unreproducible runtime, so the pin is checked, not assumed.
+GUFO_COMMIT="${GUFO_COMMIT:-b0f8673470a3dcc2b5e9c9546c48a40b82b5ff31}"
+SRC_DIR="${GUFO_SRC_DIR:-/home/dank/GitPackages/gufo-src}"
+PREFIX="${GUFO_PREFIX:-/home/dank/GitPackages/gufo-${GUFO_TAG}}"
+JOBS="${JOBS:-$(nproc)}"
+# The GCC toolchain that owns the installed libstdc++ on this host.
+GCC_DIR="${GCC_DIR:-/usr/lib/gcc/x86_64-linux-gnu/15}"
+
+# check_pin — refuse a source tree that is not the qualified commit.
+check_pin() {
+    local commit
+    commit="$(git -C "$SRC_DIR" rev-parse HEAD 2>/dev/null || true)"
+    if [[ -n "$commit" && "$commit" != "$GUFO_COMMIT" ]]; then
+        echo "GUFO_COMMIT mismatch: $SRC_DIR is at $commit, pinned $GUFO_COMMIT" >&2
+        exit 1
+    fi
+}
+
+if [[ "${1:-}" == "--verify" ]]; then
+    # The pin first: a moved tag is the more urgent thing to hear about, and
+    # it is the check that must not be masked by a missing install.
+    check_pin
+    [[ -x "$PREFIX/bin/gufo" ]] || { echo "no gufo at $PREFIX/bin/gufo" >&2; exit 1; }
+    "$PREFIX/bin/gufo" diagnose | tail -n 5
+    exit 0
+fi
+
+echo "== system libraries gufo links against =="
+sudo -n apt-get install -y -qq --no-install-recommends \
+    build-essential cmake ninja-build pkg-config \
+    libicu-dev libcurl4-openssl-dev libssl-dev libpng-dev libjpeg-dev ffmpeg
+
+echo "== dropping Ubuntu's truncated rocWMMA headers =="
+if dpkg -s librocwmma-dev >/dev/null 2>&1; then
+    sudo -n apt-get remove -y -qq librocwmma-dev
+fi
+
+echo "== source: $GUFO_REPO @ $GUFO_TAG =="
+if [[ ! -d "$SRC_DIR/.git" ]]; then
+    mkdir -p "$(dirname "$SRC_DIR")"
+    git clone --depth 1 --branch "$GUFO_TAG" "$GUFO_REPO" "$SRC_DIR"
+fi
+git -C "$SRC_DIR" fetch --depth 1 origin "refs/tags/$GUFO_TAG"
+git -C "$SRC_DIR" checkout -q --detach "refs/tags/$GUFO_TAG"
+echo "   $(git -C "$SRC_DIR" rev-parse HEAD)"
+check_pin
+
+echo "== configure + build (release, HIP gfx1151) =="
+cd "$SRC_DIR"
+toolchain="--gcc-install-dir=$GCC_DIR"
+cmake --preset release \
+    -DCMAKE_INSTALL_PREFIX="$PREFIX" \
+    -DCMAKE_HIP_FLAGS="$toolchain" \
+    -DCMAKE_EXE_LINKER_FLAGS="$toolchain"
+cmake --build --preset release --parallel "$JOBS"
+cmake --install build/release
+
+echo "== verify =="
+"$PREFIX/bin/gufo" diagnose
+echo "frank-large runtime ready: $PREFIX/bin/gufo ($GUFO_TAG @ $commit)"

scripts/build-kairic-edge-gfx1151.sh

@@ -0,0 +1,96 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+readonly repo_root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd)"
+readonly ck_root="${KAIRIC_CK_ROOT:-$repo_root/third_party/composable_kernel}"
+readonly ck_commit=fdf4bb7fcc984811cef48ce817d89aac064b984a
+readonly ck_patch="$repo_root/patches/composable-kernel-gfx1151-iu4.patch"
+readonly build_dir="${BUILD_DIR:-$repo_root/build-kairic}"
+readonly rocm="${ROCM_PATH:-/opt/rocm}"
+readonly jobs="${JOBS:-$(nproc)}"
+
+for tool in cmake git ninja; do
+  command -v "$tool" >/dev/null || {
+    echo "missing required tool: $tool" >&2
+    exit 1
+  }
+done
+
+[[ -f "$ck_patch" ]] || {
+  echo "missing Composable Kernel patch: $ck_patch" >&2
+  exit 1
+}
+
+if [[ ! -d "$ck_root/.git" ]]; then
+  mkdir -p "$(dirname -- "$ck_root")"
+  git clone https://github.com/ROCm/composable_kernel.git "$ck_root"
+fi
+
+git -C "$ck_root" checkout "$ck_commit"
+if git -C "$ck_root" diff --quiet; then
+  git -C "$ck_root" apply --check "$ck_patch"
+  git -C "$ck_root" apply "$ck_patch"
+elif git -C "$ck_root" apply --reverse --check "$ck_patch"; then
+  echo "Composable Kernel release patch is already applied"
+else
+  echo "Composable Kernel checkout does not match the pinned patch" >&2
+  exit 1
+fi
+
+if [[ -x "$rocm/bin/amdclang++" ]]; then
+  hip_compiler="$rocm/bin/amdclang++"
+elif [[ -x "$rocm/llvm/bin/clang++" ]]; then
+  hip_compiler="$rocm/llvm/bin/clang++"
+else
+  echo "cannot find a HIP compiler below ROCM_PATH=$rocm" >&2
+  exit 1
+fi
+
+if [[ -n "${CC:-}" ]]; then
+  c_compiler="$(command -v "$CC")"
+elif [[ -x /usr/bin/gcc ]]; then
+  c_compiler=/usr/bin/gcc
+else
+  c_compiler="$(command -v gcc)"
+fi
+if [[ -n "${CXX:-}" ]]; then
+  cxx_compiler="$(command -v "$CXX")"
+elif [[ -x /usr/bin/g++ ]]; then
+  cxx_compiler=/usr/bin/g++
+else
+  cxx_compiler="$(command -v g++)"
+fi
+
+export ROCM_PATH="$rocm"
+export PATH="$rocm/bin:$rocm/lib/llvm/bin:$rocm/llvm/bin:$PATH"
+export LD_LIBRARY_PATH="$rocm/lib:$rocm/lib/rocm_sysdeps/lib:$rocm/lib/llvm/lib:$rocm/llvm/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"
+readonly hip_flags="-DGGML_ROCMFPX_RDNA35_MMID_MAX_BATCH=5 -DGGML_ROCMFPX_MOE_MMVQ_ROWS_PER_BLOCK=4${KAIRIC_HIP_TOOLCHAIN_FLAGS:+ $KAIRIC_HIP_TOOLCHAIN_FLAGS}"
+
+cmake -S "$repo_root" -B "$build_dir" -G Ninja \
+  -DCMAKE_BUILD_TYPE=Release \
+  -DCMAKE_C_COMPILER="$c_compiler" \
+  -DCMAKE_CXX_COMPILER="$cxx_compiler" \
+  -DCMAKE_HIP_COMPILER="$hip_compiler" \
+  -DCMAKE_HIP_FLAGS="$hip_flags" \
+  -DCMAKE_PREFIX_PATH="$rocm" \
+  -DBUILD_SHARED_LIBS=ON \
+  -DGGML_CPU=ON -DGGML_OPENMP=ON -DGGML_HIP=ON -DGGML_CUDA=OFF \
+  -DGGML_VULKAN=OFF -DGGML_HIP_FORCE_MMQ=ON -DGGML_HIP_GRAPHS=ON \
+  -DGGML_HIP_MMQ_MFMA=ON -DGGML_HIP_NO_VMM=ON \
+  -DGGML_HIP_ROCWMMA_FATTN=OFF -DGGML_NATIVE=ON \
+  -DAMDGPU_TARGETS=gfx1151 -DGPU_BUILD_TARGETS=gfx1151 \
+  -DPROMPTFORGE_CK_ROOT="$ck_root" \
+  -DLLAMA_BUILD_WEBUI=OFF \
+  -DGGML_BUILD_TESTS=OFF -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_SERVER=ON
+
+cmake --build "$build_dir" --target llama-server -j"$jobs"
+
+"$build_dir/bin/llama-server" --version
+"$build_dir/bin/llama-server" --help | grep -A1 -- '--kairic-edge'
+ldd "$build_dir/bin/llama-server" 2>&1 | tee "$build_dir/llama-server.ldd.txt"
+if grep -Eiq 'not found|version .* required by .* not found' "$build_dir/llama-server.ldd.txt"; then
+  echo "runtime dependency verification failed" >&2
+  exit 1
+fi
+
+echo "Kairic Edge runtime built at $build_dir/bin/llama-server"

scripts/build-llama-rdna4.sh

@@ -38,6 +38,7 @@ JOBS="${JOBS:-$(nproc)}"
 ROCM_VERSION="${ROCM_VERSION:-7.14.0a20260624}"
 ROCM_LOCAL_PATH="${ROCM_LOCAL_PATH:-${XDG_DATA_HOME:-$HOME/.local/share}/rocm-${ROCM_VERSION}}"
 HIP_ARCH="${HIP_ARCH:-gfx1201}"
+GGML_HIP_RCCL="${GGML_HIP_RCCL:-OFF}"
 
 # ── Source ────────────────────────────────────────────────────────────
 if [[ ! -d "$LLAMA_SRC/.git" ]]; then
@@ -70,11 +71,13 @@ export ROCM_PATH="$ROCM_LOCAL_PATH"
 export HIP_PATH="$ROCM_LOCAL_PATH"
 
 # ── Configure ─────────────────────────────────────────────────────────
-# GGML_HIP_RCCL stays OFF: this host's second card cannot dispatch a RCCL
-# kernel at all (its PCIe path has no atomic routing), and layer-split serving
-# needs no collectives.  With RCCL off, llama.cpp logs "internal AllReduce init
-# failed ... falling back to meta-backend butterfly" and only --split-mode
-# tensor pays for it.
+# GGML_HIP_RCCL defaults OFF: layer-split serving needs no collectives, and
+# until 2026-09-20 this host's second card could not dispatch a RCCL kernel at
+# all (its PCIe path had no atomic routing).  With RCCL off, llama.cpp logs
+# "internal AllReduce init failed ... falling back to meta-backend butterfly"
+# and only --split-mode row/tensor pays for it.  Set GGML_HIP_RCCL=ON (with a
+# distinct BUILD_DIR) for the collective-backed runtime; the atomics
+# precondition now holds on both cards.
 cmake -S "$LLAMA_SRC" -B "$BUILD_DIR" \
     -G Ninja \
     -DCMAKE_BUILD_TYPE=Release \
@@ -83,7 +86,7 @@ cmake -S "$LLAMA_SRC" -B "$BUILD_DIR" \
     -DCMAKE_CXX_FLAGS="-I${ROCM_LOCAL_PATH}/include" \
     -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
     -DGGML_HIP=ON \
-    -DGGML_HIP_RCCL=OFF \
+    -DGGML_HIP_RCCL="$GGML_HIP_RCCL" \
     -DGGML_CUDA=OFF \
     -DGGML_VULKAN=OFF \
     -DGGML_NATIVE=OFF \
@@ -108,7 +111,7 @@ if command -v patchelf >/dev/null; then
     mkdir -p "$LIB_DIR"
     for pat in libamdhip64 librocblas libhipblas libhipblaslt librocsolver \
                libroctx64 libamd_comgr libhsa-runtime64 librocm_kpack \
-               librocroller librocprofiler-register; do
+               librocroller librocprofiler-register librocm-core; do
         cp -v "$ROCM_LOCAL_PATH/lib/$pat".so* "$LIB_DIR/" 2>/dev/null || true
     done
     cp -v "$ROCM_LOCAL_PATH/lib/llvm/lib"/libLLVM.so* \
@@ -116,6 +119,13 @@ if command -v patchelf >/dev/null; then
           "$ROCM_LOCAL_PATH/lib/llvm/lib"/libomp.so* \
           "$ROCM_LOCAL_PATH/lib/llvm/lib"/libomptarget.so* "$LIB_DIR/" 2>/dev/null || true
     cp -v "$ROCM_LOCAL_PATH/lib/rocm_sysdeps/lib"/librocm_sysdeps_*.so* "$LIB_DIR/" 2>/dev/null || true
+    # A RCCL build links librccl.so.1 (imported shared even with
+    # BUILD_SHARED_LIBS=OFF), and the binaries resolve libraries through
+    # $ORIGIN/../lib — so the collective library must ride beside the rest or
+    # llama-server fails to exec at all.
+    if [[ "$GGML_HIP_RCCL" == "ON" ]]; then
+        cp -v "$ROCM_LOCAL_PATH"/lib/librccl.so* "$LIB_DIR/" 2>/dev/null || true
+    fi
     # rocBLAS and hipBLASLt look for their device code at a FIXED depth under
     # the binary's lib dir: $ORIGIN/../lib/rocblas/library and
     # $ORIGIN/../lib/hipblaslt/library/<arch>. `cp -r SRC DST/` with a missing

scripts/build-rocmfpx.sh

@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+# Build the ROCmFPX runtime that serves frank-large on Dankclaw.
+#
+#   bash scripts/build-rocmfpx.sh              # clone/fetch + build the pin
+#   bash scripts/build-rocmfpx.sh --verify     # only check the installed tree
+#
+# Runtime: kingjones30/ROCmFPX — charlie12345/ROCmFPX plus the model-card
+# patches, i.e. the fork that owns the ROCmFPx tensor formats. frank-large's
+# payload is ftype 115 (`Q8_0_ROCMFPX_AGENT`): 311 tensors of
+# GGML_TYPE_Q8_0_ROCMFPX (103), a type that exists ONLY in this fork, so
+# stock llama.cpp, gufo and every other loader on the host reject the file.
+# This is the same tree that served the Qwen3.8-Flash-Next ROCmFP4 STRIX
+# endpoint 2026-08-30 → 09-11 (see the kingjones runbook), rebuilt here so a
+# fresh host can be brought up from the repo alone.
+#
+# Why this commit and not a newer one: 36e9acd (2026-08-28) is what was
+# qualified against this payload on 2026-10-02 — it loads ftype 115, survives
+# warmup, and drafts with the MTP head at acceptance 0.65. The five legacy
+# builds of 2026-08-25 crashed this same file in ggml_compute_forward_set_rows
+# (see kingjones-q8agent-rocmfpx.md), so the pin is load-bearing: do not bump
+# it without re-running that crash test.
+#
+# Host notes:
+#   * -DLLAMA_BUILD_WEBUI=OFF is REQUIRED on this box. The tree provisions its
+#     embedded UI assets via npm or the HF llama-ui bucket; npm is absent here
+#     and the bucket download half-failed, aborting the link at 98%. The API
+#     server is unaffected.
+#   * The build links against the host ROCm (/opt/rocm); no bundled toolchain
+#     like the gfx1201 R9700 build needed.
+set -euo pipefail
+
+ROCMFPX_REPO="${ROCMFPX_REPO:-https://github.com/kingjones30/ROCmFPX.git}"
+# The commit frank-large was qualified with (measured 2026-10-02).
+ROCMFPX_COMMIT="${ROCMFPX_COMMIT:-36e9acd40e10a87cd3c3ef8ec734668757dc8520}"
+SRC_DIR="${ROCMFPX_SRC_DIR:-/home/dank/GitPackages/kingjones-ROCmFPX}"
+BUILD_DIR="${ROCMFPX_BUILD_DIR:-$SRC_DIR/build}"
+GPU_TARGET="${ROCMFPX_GPU_TARGET:-gfx1151}"
+JOBS="${JOBS:-$(nproc)}"
+
+# check_pin — refuse a source tree that is not the qualified commit. A moved
+# fork means different kernels under the same path, and on this model line a
+# wrong build is a segfault at first request, not a clean error.
+check_pin() {
+    local commit
+    commit="$(git -C "$SRC_DIR" rev-parse HEAD 2>/dev/null || true)"
+    if [[ -n "$commit" && "$commit" != "$ROCMFPX_COMMIT" ]]; then
+        echo "ROCMFPX_COMMIT mismatch: $SRC_DIR is at $commit, pinned $ROCMFPX_COMMIT" >&2
+        exit 1
+    fi
+}
+
+if [[ "${1:-}" == "--verify" ]]; then
+    check_pin
+    [[ -x "$BUILD_DIR/bin/llama-server" ]] || {
+        echo "no ROCmFPX llama-server at $BUILD_DIR/bin/llama-server" >&2
+        exit 1
+    }
+    # The format the endpoint exists for: the fork must carry the ROCmFPx
+    # tensor type, or the payload cannot load at all.
+    "$BUILD_DIR/bin/llama-server" --help 2>&1 | grep -q -- '--spec-type' || {
+        echo "$BUILD_DIR/bin/llama-server has no --spec-type: MTP drafting is impossible" >&2
+        exit 1
+    }
+    "$BUILD_DIR/bin/llama-server" --version | head -1
+    exit 0
+fi
+
+echo "== source: $ROCMFPX_REPO @ $ROCMFPX_COMMIT =="
+if [[ ! -d "$SRC_DIR/.git" ]]; then
+    mkdir -p "$(dirname "$SRC_DIR")"
+    git clone "$ROCMFPX_REPO" "$SRC_DIR"
+fi
+git -C "$SRC_DIR" fetch --quiet origin "$ROCMFPX_COMMIT" || true
+git -C "$SRC_DIR" checkout -q --detach "$ROCMFPX_COMMIT"
+check_pin
+echo "   $(git -C "$SRC_DIR" rev-parse HEAD)"
+
+echo "== configure + build (release, HIP $GPU_TARGET) =="
+cmake -S "$SRC_DIR" -B "$BUILD_DIR" \
+    -DGGML_HIP=ON \
+    -DGPU_TARGETS="$GPU_TARGET" \
+    -DGGML_NATIVE=ON \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DLLAMA_BUILD_WEBUI=OFF
+cmake --build "$BUILD_DIR" --target llama-server llama-quantize -j "$JOBS"
+
+echo "== verify =="
+"$BUILD_DIR/bin/llama-server" --version | head -1
+echo "frank-large runtime ready: $BUILD_DIR/bin/llama-server ($ROCMFPX_COMMIT)"

scripts/dankserver-route-primary.sh

@@ -0,0 +1,58 @@
+#!/usr/bin/env bash
+# Pin Dankserver's fleet NIC (eno1, 192.168.0.235) as the primary default route.
+#
+# The 2026-09-20 board swap (TUF GAMING B650-E -> ProArt B850-Creator WiFi NEO)
+# brought up a second port, eno2, which took DHCP and — with NetworkManager's
+# default per-connection metric of 100 — came up with metric 101 while eno1's
+# static gateway got 102. Outbound traffic therefore started leaving by eno2
+# even though every fleet peer, Caddy upstream and monitoring target addresses
+# this host as .235. Keep eno2 (it is a real second link) but make it secondary:
+# route metrics decide the default route, so eno1 wins while eno2 stays a
+# failover.
+#
+# Run on Dankserver.  Idempotent; NetworkManager owns these connections (the
+# netplan files under /etc/netplan are its generated shim, so edit via nmcli).
+#
+#   bash scripts/dankserver-route-primary.sh [--check]
+set -euo pipefail
+
+PRIMARY='Wired connection 1'      # eno1, static 192.168.0.235, fleet address
+SECONDARY='Wired connection 2'    # eno2, DHCP 192.168.0.247
+PRIMARY_IF='eno1'
+SECONDARY_IF='eno2'
+PRIMARY_METRIC=100
+SECONDARY_METRIC=200
+
+show() {
+    printf 'routes:\n'; ip route show default
+    printf 'wanted: %s metric %s, %s metric %s\n' \
+        "$PRIMARY" "$PRIMARY_METRIC" "$SECONDARY" "$SECONDARY_METRIC"
+}
+
+if [[ "${1:-}" == '--check' ]]; then
+    for conn in "$PRIMARY" "$SECONDARY"; do
+        printf '%s: ipv4=%s ipv6=%s\n' "$conn" \
+            "$(nmcli -g ipv4.route-metric connection show "$conn")" \
+            "$(nmcli -g ipv6.route-metric connection show "$conn")"
+    done
+    show
+    exit 0
+fi
+
+nmcli connection modify "$PRIMARY" \
+    ipv4.route-metric "$PRIMARY_METRIC" ipv6.route-metric "$PRIMARY_METRIC"
+nmcli connection modify "$SECONDARY" \
+    ipv4.route-metric "$SECONDARY_METRIC" ipv6.route-metric "$SECONDARY_METRIC"
+
+# `nmcli device reapply <ifname>` pushes the edited profile without tearing the
+# link down.  `nmcli connection up` would re-run DHCP and briefly drop the route
+# this session rides on, and `connection reapply` is not a verb (NM 1.54).
+nmcli device reapply "$PRIMARY_IF"
+nmcli device reapply "$SECONDARY_IF"
+
+show
+if [[ "$(ip route show default | head -1)" != *"dev $PRIMARY_IF"* ]]; then
+    printf 'ERROR: eno1 is still not the preferred default route\n' >&2
+    exit 1
+fi
+printf 'eno1 is the preferred default route; eno2 remains up as failover.\n'

scripts/dankstatus.sh

@@ -0,0 +1,71 @@
+#!/usr/bin/env bash
+# Single source of truth for Dankserver GLM-5.3 state.
+#
+# This exists because the ad-hoc status commands kept producing WRONG readings.
+# The specific trap: /proc/meminfo separates key from value with a TAB, so the
+# common idiom
+#   grep MemAvailable /proc/meminfo | tr -s " " | cut -d" " -f2
+# yields an empty field (tab is not a space) and awk then prints 0.0 -- which
+# reads as "host out of memory" when the host actually has 240 GiB free.
+# Everything below parses on a colon split, which is tab-agnostic, and the
+# memory figure is cross-checked before it is shown.
+set -uo pipefail
+
+MB=1048576
+GLM_CTR="${GLM_CTR:-glm-w4a16}"
+
+mem_field() { awk -v k="$1:" '$1==k {printf "%.1f", $2/'"$MB"'; exit}' /proc/meminfo; }
+section() { printf '\n== %s ==\n' "$1"; }
+
+section "time / host"
+printf '  %s   %s\n' "$(date +%H:%M:%S)" \
+  "$(awk -F: '/model name/{print $2; exit}' /proc/cpuinfo | sed 's/^ *//')"
+
+section "memory (GiB)"
+TOT=$(mem_field MemTotal); AVAIL=$(mem_field MemAvailable)
+printf '  MemTotal     %8s\n' "$TOT"
+printf '  MemAvailable %8s\n' "$AVAIL"
+printf '  Cached       %8s\n' "$(mem_field Cached)"
+awk -v t="$TOT" -v a="$AVAIL" 'BEGIN{
+  if (t+0 <= 0 || a+0 <= 0) { print "  *** PARSE BUG: zero value ***"; exit }
+  if (a+0 > t+0)         { print "  *** PARSE BUG: available > total ***"; exit }
+  printf "  check        ok (%.0f%% free)\n", 100*a/t
+}'
+
+section "swap"
+swapon --show=NAME,SIZE,USED --noheadings 2>/dev/null \
+  | awk '{printf "  %-12s %6s used of %s\n",$1,$3,$2}' || echo "  none"
+
+section "gpu"
+# CSV layout: device,VRAM Total Memory (B),VRAM Total Used Memory (B)
+rocm-smi --showmeminfo vram --csv 2>/dev/null | awk -F, '
+  NR>1 && $1 ~ /^card[0-9]+$/ {
+    printf "  %-8s %7.2f GiB used of %.2f\n", $1, $3/1073741824, $2/1073741824
+  }'
+printf '  compute util: '
+rocm-smi --showuse --csv 2>/dev/null | awk -F, 'NR>1 && $1 ~ /^card[0-9]+$/ {printf "%s=%s%% ", $1, $2}'
+echo
+
+section "disk"
+df -h /home/dank 2>/dev/null | awk 'NR==2{printf "  %s  %s used  %s free  (%s)\n",$1,$3,$4,$5}'
+
+section "llm containers"
+if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qE 'glm|frank'; then
+  podman ps -a --format '{{.Names}}|{{.Status}}' 2>/dev/null \
+    | awk -F'|' '$1 ~ /glm|frank/ {printf "  %-18s %s\n",$1,$2}'
+else
+  echo "  none"
+fi
+
+section "frank-primary unit"
+printf '  active:  %s\n' "$(systemctl is-active frank-primary 2>/dev/null | head -1)"
+printf '  enabled: %s\n' "$(systemctl is-enabled frank-primary 2>/dev/null | head -1)"
+
+section "glm serve tail"
+if podman ps -a --format '{{.Names}}' 2>/dev/null | grep -qx "$GLM_CTR"; then
+  podman logs "$GLM_CTR" 2>&1 | grep -viE 'multiproc_executor\.py:943' | tail -5 | sed 's/^/  /'
+else
+  echo "  (container $GLM_CTR not present)"
+fi
+
+section "done"

scripts/deploy-plan.py

@@ -38,6 +38,7 @@ SERVICE_ORDER = (
     "loki",
     "alloy",
     "node-exporter",
+    "alertmanager",
     "authentik-postgres",
     "authentik-server",
     "authentik-worker",
@@ -51,6 +52,7 @@ SERVICE_ORDER = (
     "copyparty",
     "jellyfin",
     "calibre-web",
+    "calibre-server",
     "plex",
     "tautulli",
     "prowlarr",
@@ -61,7 +63,7 @@ SERVICE_ORDER = (
     "readarr",
     "frank-large",
     "frank-small",
-    "frank-medium",
+    "frank-primary",
 )
 
 DANKCLAW_SERVICES = {
@@ -83,6 +85,7 @@ DANKCLAW_SERVICES = {
     "loki",
     "alloy",
     "node-exporter",
+    "alertmanager",
     "authentik-postgres",
     "authentik-server",
     "authentik-worker",
@@ -101,6 +104,7 @@ DANKSERVER_SERVICES = {
     "copyparty",
     "jellyfin",
     "calibre-web",
+    "calibre-server",
     "plex",
     "tautulli",
     "prowlarr",
@@ -109,7 +113,7 @@ DANKSERVER_SERVICES = {
     "sonarr",
     "lidarr",
     "readarr",
-    "frank-medium",
+    "frank-primary",
     "node-exporter",
 }
 
@@ -246,6 +250,10 @@ def services_for_path(path: str, target: str) -> set[str]:
             _add(services, "searxng")
         elif group == "prometheus":
             _add(services, "prometheus")
+            if len(parts) >= 3 and parts[2] == "alertmanager.yml":
+                # The Alertmanager config ships from this directory: a change
+                # to it must redeploy the service that installs it.
+                _add(services, "alertmanager")
         elif group == "loki":
             _add(services, "loki")
         elif group == "alloy":
@@ -308,11 +316,8 @@ def services_for_path(path: str, target: str) -> set[str]:
         if len(parts) >= 2 and parts[1] == "open-webui":
             _add(services, "open-webui")
             return services
-        if len(parts) >= 2 and parts[1] == "frank-large":
-            # The service is a rootless Quadlet container since 2026-09-11;
-            # its unit and fleet settings live in one .container file.
-            _add(services, "frank-large")
-            return services
+        # frank-large left the quadlet shape on 2026-09-19 (CIRU v4.4.1 host
+        # build, root unit): its files live in scripts/ and map there.
         if len(parts) >= 2 and parts[1] == "observability":
             # The five observability containers became Quadlet units on
             # 2026-09-11. A unit change selects exactly its own service;
@@ -320,7 +325,7 @@ def services_for_path(path: str, target: str) -> set[str]:
             # consumer (prometheus before grafana, loki before alloy).
             if len(parts) >= 4 and parts[2] == "quadlet":
                 name = parts[3].removesuffix(".container")
-                if name in {"prometheus", "grafana", "loki", "alloy", "node-exporter"}:
+                if name in {"prometheus", "grafana", "loki", "alloy", "node-exporter", "alertmanager"}:
                     _add(services, name)
             return services
         if len(parts) >= 2 and parts[1] == "gitea":
@@ -350,6 +355,7 @@ def services_for_path(path: str, target: str) -> set[str]:
                 "romm": {"romm", "romm-mariadb", "romm-redis"},
                 "jellyfin": {"jellyfin"},
                 "calibre-web": {"calibre-web"},
+                "calibre-server": {"calibre-server"},
                 "copyparty": {"copyparty"},
                 "plex": {"plex"},
                 "tautulli": {"tautulli"},
@@ -365,13 +371,11 @@ def services_for_path(path: str, target: str) -> set[str]:
                 if name == "node-exporter":
                     _add(services, "node-exporter")
             return services
-        if len(parts) >= 2 and parts[1] == "frank-medium":
-            # frank-medium is a ROOT llama.cpp unit since 2026-09-16: its
-            # runtime is a native gfx1201 binary, not a container, so there is
-            # nothing under dankserver/frank-medium/ to plan any more. The
-            # retired vllm-radiance quadlets live in .AGENTTRASH/, which is
-            # archive material and plans nothing.
-            return services
+        # frank-primary's quadlet-era directory left the tree on 2026-09-26
+        # (the Lucebox DS4.1 unit was retired for the llama.cpp root unit —
+        # the trio in scripts/ now maps it). Nothing lives under
+        # dankserver/frank-primary/ except dated trash, and trash plans
+        # nothing (the retire rule above the service blocks).
         return services
 
     if parts[0] == "frankbook-pro":
@@ -391,18 +395,37 @@ def services_for_path(path: str, target: str) -> set[str]:
         name = p.name
         if name in {"homepage-status.py", "homepage-status.service", "homepage-status.timer"}:
             _add(services, "homepage")
+        if name in {"signal-relay.py", "signal-relay.service"}:
+            # The Signal relay is installed by the alertmanager deploy, so a
+            # change to it redeploying alertmanager is the only path that
+            # reaches it on the host.
+            _add(services, "alertmanager")
         if name in {"frank-small.env", "frank-small.service", "serve-frank-small.sh"}:
             # llama.cpp frank-small (2026-09-16) keeps its unit, env and serve
             # files here; each one selects frank-small only. The ik_llama.cpp
             # trio that lived here for two days is in scripts/.AGENTTRASH/,
             # which is archive material and plans nothing.
             _add(services, "frank-small")
-        if name in {"frank-medium.env", "frank-medium.service", "serve-frank-medium.sh"}:
-            # llama.cpp frank-medium (2026-09-16) keeps its unit, env and serve
-            # files here the same way. The files that BUILD and FETCH its
-            # runtime and payload are host-side one-offs: they change no
-            # deployed state, so they select nothing.
-            _add(services, "frank-medium")
+        if name in {"frank-large.env", "frank-large.service", "serve-frank-large.sh"}:
+            # CIRU v4.4.1 frank-large (2026-09-19) keeps its unit, env and
+            # serve files here the same way: a root unit on a host-built
+            # runtime. The download script fetches the payload, which is
+            # host-side state, so it selects nothing.
+            _add(services, "frank-large")
+        if name == "qwen-sharp-v22.4.0.jinja":
+            # The Qwen-Sharp chat template frank-large loads via
+            # --chat-template-file. It lived under dankserver/frank-medium/ until
+            # the 2026-09-26 cutover retired that directory; editing it changes
+            # what the live frank-large endpoint renders, so it plans
+            # frank-large.
+            _add(services, "frank-large")
+        if name in {"frank-primary.env", "frank-primary.service", "serve-frank-primary.sh"}:
+            # llama.cpp frank-primary (2026-09-26, replacing the Lucebox
+            # DS4.1 quadlet) keeps its unit, env and serve files here the
+            # same way: a root unit on the unsloth b11030-mix prebuilt
+            # serving Qwen3.8-Flash-Next Q8_0. The download script fetches
+            # the payload, which is host-side state, so it selects nothing.
+            _add(services, "frank-primary")
         # Deployment/test/benchmark scripts and other repo tooling do not
         # restart a runtime service.
         return services
@@ -412,7 +435,6 @@ def services_for_path(path: str, target: str) -> set[str]:
         return services
     return services
 
-
 def changed_paths(repo: str, base: str, head: str) -> list[str]:
     result = subprocess.run(
         ["git", "-C", repo, "diff", "--name-only", "--diff-filter=ACDMRTUXB", base, head],

scripts/deploy-remote.sh

@@ -86,16 +86,16 @@ service_is_allowed() {
         dankclaw:open-webui|dankclaw:gitea-postgres|dankclaw:gitea-server|\
         dankclaw:comfyui|dankclaw:hermes|\
         dankclaw:hermes-dashboard|dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|\
-        dankclaw:loki|dankclaw:alloy|dankclaw:node-exporter|\
+        dankclaw:loki|dankclaw:alloy|dankclaw:node-exporter|dankclaw:alertmanager|\
         dankclaw:authentik-postgres|dankclaw:authentik-server|\
         dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|\
         dankclaw:caddy|dankclaw:homepage|dankclaw:seerr|dankclaw:marreta|\
         dankclaw:frank-large|dankclaw:frank-small|\
         dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
-        dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:plex|dankserver:tautulli|\
+        dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
         dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
         dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
-        dankserver:frank-medium|dankserver:node-exporter|\
+        dankserver:frank-primary|dankserver:node-exporter|\
         frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
         frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
         frankbook-pro:readarr)
@@ -177,6 +177,17 @@ add_pending_services() {
         if [[ "$status" != deployed ]]; then
             # A pending service was intentionally skipped or its handler
             # failed. Retry it even when the checkout has not advanced.
+            # But a pending entry for a service RETIRED from this target can
+            # never be deployable: retrying it fails the whole run before
+            # the e2e gate (the frank-default-sightsound lockout, found
+            # 2026-09-17). Drop such entries into the host trash instead.
+            if ! service_is_allowed "$service"; then
+                local trash="$HOME/.AGENTTRASH/$(date +%Y%m%d)-retired-deploy-state"
+                mkdir -p "$trash"
+                mv -- "$state_file" "$trash/$service"
+                echo "WARN: $service is retired on $TARGET; dropped its stale $status state to $trash" >&2
+                continue
+            fi
             SERVICES+=("$service")
             continue
         fi
@@ -198,6 +209,11 @@ copy_searxng_config() {
     run_privileged mkdir -p /opt/searxng/config
     run_privileged cp shared/searxng/settings.yml /opt/searxng/config/settings.yml
     run_privileged cp shared/searxng/limiter.toml /opt/searxng/config/limiter.toml
+    # Bake ${KAGI_API_KEY}/${SEARXNG_SECRET}/${SEARXNG_BASE_URL} into the live
+    # copy from the host-side /opt/searxng/.env (SearXNG does not expand env
+    # vars in settings.yml). Fails the deploy if a placeholder is unresolvable.
+    run_privileged python3 scripts/searxng-render-settings.py \
+        /opt/searxng/.env /opt/searxng/config/settings.yml
 }
 
 copy_prometheus_config() {
@@ -210,6 +226,53 @@ copy_prometheus_config() {
         /opt/prometheus/config/rules/llm-fleet.yml
 }
 
+# Alertmanager config ships beside the Prometheus config: the two halves of
+# the alerting path change together, and a receiver with no route (or a
+# route with no receiver) fails silently.
+copy_alertmanager_config() {
+    run_privileged mkdir -p /opt/alertmanager/config /opt/alertmanager/data
+    run_privileged cp shared/prometheus/alertmanager.yml /opt/alertmanager/config/alertmanager.yml
+}
+
+# The Signal relay is Alertmanager's only webhook consumer and the only
+# thing on this host that talks to signal-cli. It is a plain systemd USER
+# unit (not a quadlet): it runs python3 from the checkout, so a deploy
+# updates it without a copy step, and — unlike generated quadlet units,
+# which this host's podman 5.x v1 generator rejects — a hand-written user
+# unit can be enabled, and Linger=yes for dank starts it at boot.
+install_signal_relay() {
+    local unit_dir="$HOME/.config/systemd/user"
+    mkdir -p "$unit_dir"
+    cp scripts/signal-relay.service "$unit_dir/signal-relay.service"
+    systemctl --user daemon-reload
+    systemctl --user enable signal-relay.service
+    systemctl --user restart signal-relay.service
+    # The relay binds the host's LAN IP (the one address a bridge container
+    # can reach as host.containers.internal), so keep the unauthenticated
+    # endpoint off the rest of the LAN: admit loopback, the host itself
+    # (its own health probe) and the podman bridge subnet only. Idempotent:
+    # -C before -I.
+    run_privileged iptables -C INPUT -p tcp --dport 18093 -s 127.0.0.1 -j ACCEPT 2>/dev/null \
+        || run_privileged iptables -I INPUT -p tcp --dport 18093 -s 127.0.0.1 -j ACCEPT
+    run_privileged iptables -C INPUT -p tcp --dport 18093 -s 192.168.0.143 -j ACCEPT 2>/dev/null \
+        || run_privileged iptables -I INPUT -p tcp --dport 18093 -s 192.168.0.143 -j ACCEPT
+    run_privileged iptables -C INPUT -p tcp --dport 18093 -s 10.89.0.0/16 -j ACCEPT 2>/dev/null \
+        || run_privileged iptables -I INPUT -p tcp --dport 18093 -s 10.89.0.0/16 -j ACCEPT
+    run_privileged iptables -C INPUT -p tcp --dport 18093 -j DROP 2>/dev/null \
+        || run_privileged iptables -I INPUT -p tcp --dport 18093 -j DROP
+    # The interpreter needs a moment to bind after restart; a one-shot curl
+    # raced the socket on 2026-09-17 and failed a healthy deploy.
+    local i
+    for i in $(seq 1 15); do
+        if curl -fsS -m 5 http://192.168.0.143:18093/health >/dev/null; then
+            return 0
+        fi
+        sleep 1
+    done
+    echo "signal-relay is not answering on 192.168.0.143:18093" >&2
+    return 1
+}
+
 copy_loki_config() {
     run_privileged mkdir -p /opt/loki/config
     run_privileged cp shared/loki/loki-config.yaml /opt/loki/config/loki-config.yaml
@@ -255,7 +318,7 @@ deploy_homepage() {
     fi
     run_privileged mkdir -p /srv/homepage
     run_privileged cp shared/homepage/index.html shared/homepage/admin.html shared/homepage/app.js \
-        shared/homepage/style.css shared/homepage/services.json /srv/homepage/
+        shared/homepage/style.css shared/homepage/services.json shared/homepage/docs.html /srv/homepage/
     if [[ -d shared/homepage/fonts ]]; then
         run_privileged cp -r shared/homepage/fonts /srv/homepage/
     fi
@@ -426,13 +489,17 @@ seed_grafana_env() {
 deploy_observability() {
     local service="$1"
     # Observability units live under <target>/observability/quadlet: Dankclaw
-    # has five, Dankserver one (node-exporter, since 2026-09-16). The config
+    # has six, Dankserver one (node-exporter, since 2026-09-16). The config
     # copies below are Dankclaw-owned state and run only there.
     local unit_dir="$TARGET/observability/quadlet"
     case "$service" in
         prometheus)
             copy_prometheus_config
             ;;
+        alertmanager)
+            copy_alertmanager_config
+            install_signal_relay
+            ;;
         loki)
             copy_loki_config
             build_compose_image loki
@@ -456,6 +523,7 @@ deploy_observability() {
     systemctl --user restart "$service.service"
     case "$service" in
         prometheus) wait_for_health prometheus http://127.0.0.1:9090/-/healthy 60 ;;
+        alertmanager) wait_for_health alertmanager http://127.0.0.1:9093/-/healthy 60 ;;
         grafana) wait_for_health grafana http://127.0.0.1:3002/api/health 90 ;;
         loki) wait_for_health loki http://127.0.0.1:3100/ready 60 ;;
         alloy) wait_for_health alloy http://127.0.0.1:12345/-/healthy 60 ;;
@@ -787,6 +855,50 @@ deploy_calibre_web() {
     systemctl --user restart calibre-web.service
 }
 
+# The content server's single admin account and the desktop's basic-auth
+# credential (same pair) are bootstrapped exactly once: the env file is the
+# plaintext store (mode 600), the userdb's pw column is plaintext too, so
+# its file is mode 600 as well. The password is piped on stdin with printf -
+# a trailing newline from echo lands in the stored password and breaks every
+# login (verified 2026-09-21).
+deploy_calibre_server() {
+    local data_root=/home/dank/Containers/calibre-server
+    mkdir -p "$data_root"
+    if [[ ! -f $data_root/env ]]; then
+        local pw
+        pw=$(openssl rand -base64 18 | tr -d '/+=' | head -c 20)
+        printf 'CUSTOM_USER=calibre\nCUSTOM_PASSWORD=%s\n' "$pw" > "$data_root/env"
+        chmod 600 "$data_root/env"
+        echo "calibre-server: generated credential (shown once): calibre / $pw"
+    fi
+    [[ -f $data_root/env ]] || {
+        echo "FAIL: $data_root/env does not exist; seed it before the first calibre-server deploy" >&2
+        return 1
+    }
+    if [[ ! -f $data_root/userdb.sqlite ]]; then
+        local pw
+        pw=$(grep '^CUSTOM_PASSWORD=' "$data_root/env" | cut -d= -f2)
+        touch "$data_root/userdb.sqlite"
+        printf '%s' "$pw" | podman run --rm -i --entrypoint /usr/bin/calibre-server \
+            -v "$data_root/userdb.sqlite:/userdb.sqlite" \
+            lscr.io/linuxserver/calibre:9.15.0 \
+            --userdb /userdb.sqlite --manage-users -- add calibre
+        chown 1000:1000 "$data_root/userdb.sqlite"
+        chmod 600 "$data_root/userdb.sqlite"
+    fi
+    [[ -f $data_root/userdb.sqlite && -s $data_root/userdb.sqlite ]] || {
+        echo "FAIL: $data_root/userdb.sqlite missing or empty; the --manage-users bootstrap failed" >&2
+        return 1
+    }
+    cp "dankserver/calibre-server/quadlet/calibre-server.container" "$quadlet_unit_dir/"
+    systemctl --user daemon-reload
+    if ! systemctl --user is-active -q calibre-server.service; then
+        podman stop calibre-server 2>/dev/null || true
+        podman rm calibre-server 2>/dev/null || true
+    fi
+    systemctl --user restart calibre-server.service
+}
+
 deploy_copyparty() {
     # The admin account is created from ${COPAY_ACCOUNT}:${COPAY_PASSWORD} expanded
     # at ExecStart time; a missing env file must fail the unit, not boot copyparty
@@ -823,15 +935,24 @@ deploy_plex() {
     fi
     systemctl --user restart plex.service
     # Readiness gate: /identity answers MediaContainer XML once PMS is up.
+    #
+    # HTTPS, NOT HTTP. PMS drops plain HTTP off-loopback: the probe's
+    # connection arrives on the container's eth0, so an http:// request is
+    # accepted and then answered with nothing (curl 52, Empty reply) against a
+    # perfectly healthy server. Measured 2026-10-02 — the gate burned its full
+    # 600s and failed a plex whose https://127.0.0.1:32400/identity answered
+    # 200 in 44 ms the whole time. The same law is why the Caddyfile dials the
+    # plex upstream over TLS (tests/caddyfile.bats pins it).
+    # -k: PMS presents its own self-signed cert on 32400.
+    #
     # Reality check from the 2026-09 cutover: the FIRST container boot of a
-    # years-old library ran ~10 min (PMS pre-binds 32400 during boot and
-    # empty-replies until the HTTP layer is ready), so the gate allows 600s.
+    # years-old library ran ~10 min, so the gate still allows 600s.
     # Fail the deploy (pending state preserved) rather than mark it deployed blind.
     for ((i = 1; i <= 120; i++)); do
-        curl -fsS --max-time 3 http://127.0.0.1:32400/identity >/dev/null && return 0
+        curl -fsSk --max-time 3 https://127.0.0.1:32400/identity >/dev/null && return 0
         sleep 5
     done
-    echo "FAIL: plex did not answer /identity within 600s" >&2
+    echo "FAIL: plex did not answer /identity over HTTPS within 600s" >&2
     return 1
 }
 
@@ -965,41 +1086,43 @@ deploy_anubis() {
     wait_container_healthy anubis 60 || return 1
 }
 
-# frank-large runs as a rootless Quadlet unit again since 2026-09-16: the
-# halogen-flash-server runtime is back (it replaced ik_llama.cpp, which had
-# replaced it), and the RTX 3090 it shared during that interlude now belongs
-# to frank-small.
+# frank-large is the halogen CONTAINER (ghcr.io/peonist-ai/halogen,
+# digest-pinned through scripts/frank-large.env) serving
+# peonist-ai/halogen-qwen3.8-27b — qwen3.8-27b-p1w4d-d2.hgn, 33.4 GiB,
+# DFlash2 spec decode, 1 slot x 200700, text-only — on the Strix Halo
+# iGPU. It stays a ROOT unit because the serve script drives `podman run`
+# directly (the frank-primary pattern): relink_unit points
+# /etc/systemd/system/frank-large.service at the repo file, boot
+# persistence is that unit's [Install], and LimitMEMLOCK=infinity is
+# fleet parity. 2026-10-05 cutover from the ROCmFPX llama.cpp shape —
+# rollback = revert the swap commit's trio; that payload, its draft head
+# and the fork build all stay on disk
+# (~/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP,
+# ~/GitPackages/kingjones-ROCmFPX).
 #
-# TWO MANAGERS, ONE UNIT NAME. The ik incarnation ran as a ROOT unit whose
-# file is a symlink into this checkout; the halogen unit is a USER (quadlet)
-# unit. Both must never exist at once — they would race for port 7770 — so the
-# root unit is retired first: stop, disable, move the /etc symlink into the
-# home trash (never `rm`), reload the root manager. This host's `systemctl
-# disable` has been observed to delete the unit file itself (2026-08-26
-# incident), so the move tolerates a file that is already gone.
-#
-# FIRST START fetches ~127 GB of weights, and /health cannot answer while it
-# does. wait_for_llm_large_ready recognises that (checkpoint absent) and waits
-# for the file instead of burning the health budget on a failure that is not
-# one.
-LLM_LARGE_CHECKPOINT="${LLM_LARGE_CHECKPOINT:-/home/dank/models/halogen/qwen38-flash-next-w4b.hgn}"
+# The gates mirror serve-frank-large.sh's predicates EXACTLY (the
+# 2026-10-01 crash-loop lesson): image digest in dank's podman storage,
+# the .hgn checkpoint at the exact byte count, the tokenizer directory
+# holding every named file as a regular (not symlinked) file — a green
+# deploy against a missing artifact would
+# leave a restart-looping unit and a dark endpoint. The port gate is the
+# same as every cutover since 2026-09-16: nothing starts until 7770 is
+# released.
+LLM_LARGE_IMAGE="${LLM_LARGE_IMAGE:-ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea}"
+LLM_LARGE_CKPT="${LLM_LARGE_CKPT:-/home/dank/models/halogen-qwen3.8-27b/qwen3.8-27b-p1w4d-d2.hgn}"
+LLM_LARGE_CKPT_BYTES="${LLM_LARGE_CKPT_BYTES:-35865565184}"
+LLM_LARGE_TOK_DIR="${LLM_LARGE_TOK_DIR:-/home/dank/models/halogen-qwen3.8-27b/tokenizer}"
+LLM_LARGE_TOK_FILES="${LLM_LARGE_TOK_FILES:-tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json}"
 
-retire_root_llm_large() {
-    local trash="$HOME/.AGENTTRASH/20260916-halogen-restore/host-units"
-    run_privileged systemctl stop frank-large.service 2>/dev/null || true
-    run_privileged systemctl disable frank-large.service 2>/dev/null || true
-    if [[ -e /etc/systemd/system/frank-large.service || -L /etc/systemd/system/frank-large.service ]]; then
+retire_halogen_large_quadlet() {
+    local trash="$HOME/.AGENTTRASH/20260919-orca-cutover/quadlet"
+    systemctl --user stop frank-large.service 2>/dev/null || true
+    if [[ -e "$quadlet_unit_dir/frank-large.container" ]]; then
         mkdir -p "$trash"
-        run_privileged mv -- /etc/systemd/system/frank-large.service \
-            "$trash/frank-large.service"
-        echo "retired root unit frank-large.service -> $trash"
-    else
-        # This host's `systemctl disable` is allowed to delete the unit file
-        # outright (2026-08-26 incident), so there is often nothing left to
-        # move. Say so: previously no message meant no evidence either way.
-        echo "root unit frank-large.service was already removed by disable"
+        mv -- "$quadlet_unit_dir/frank-large.container" "$trash/frank-large.container"
+        echo "retired quadlet frank-large.container -> $trash"
     fi
-    run_privileged systemctl daemon-reload
+    systemctl --user daemon-reload
 }
 
 # Wait until a TCP port has no socket on it at all -- listening, accepted, or
@@ -1022,166 +1145,210 @@ wait_port_free() {
 }
 
 deploy_llm_large() {
-    retire_root_llm_large
-    # The predecessor can also be THIS quadlet -- a redeploy of a running
-    # frank-large -- so stop the user unit before waiting on the port. On a
-    # first cutover it is not running and this is a no-op.
-    systemctl --user stop frank-large.service 2>/dev/null || true
-    # Pull once, then run the localhost/-aliased tag: this host's quadlet path
-    # cannot resolve registry-prefixed refs from local storage at create time
-    # (same workaround as anubis/hermes), and Pull=never keeps boots off the
-    # registry.
-    local image="ghcr.io/peonist-ai/halogen-flash-server:0.5.8"
-    local alias="localhost/peonist-ai/halogen-flash-server:0.5.8"
-    podman pull "$image"
-    podman tag "$image" "$alias"
-    # The weights volume must exist, and the container needs it read-write:
-    # that is where a first start downloads into.
-    mkdir -p /home/dank/models/halogen
-    # Nothing may start until the predecessor has released the port. On
-    # 2026-09-16 this sequence failed exactly here: the ik server logged
-    # "Deactivated successfully", pasta tried to bind 7770 a second later and
-    # reported "Address already in use", and the unit exited 126. Restart=
-    # on-failure recovered it minutes later, but a deploy that starts a doomed
-    # container has not deployed anything -- so gate on the port, not on hope.
+    # Refuse rather than restart-loop; every gate below has a twin in
+    # serve-frank-large.sh and they must agree (2026-10-01 lesson).
+    # Capture podman output before grepping: `podman images | grep -q`
+    # races under pipefail (SIGPIPE 141 becomes "no match", 2026-09-30).
+    if ! grep -qF "${LLM_LARGE_IMAGE#*@}" <<<"$(podman images --format '{{.RepoDigests}}' 2>/dev/null)"; then
+        echo "ERROR: halogen image absent: $LLM_LARGE_IMAGE" >&2
+        echo "frank-large: pull it: podman pull ${LLM_LARGE_IMAGE%@*}" >&2
+        return 1
+    fi
+    if [[ ! -f "$LLM_LARGE_CKPT" ]]; then
+        echo "ERROR: halogen checkpoint absent at $LLM_LARGE_CKPT" >&2
+        echo "frank-large: fetch it: bash scripts/download-frank-large-halogen.sh" >&2
+        return 1
+    fi
+    if [[ "$(stat -c %s "$LLM_LARGE_CKPT")" != "$LLM_LARGE_CKPT_BYTES" ]]; then
+        echo "ERROR: halogen checkpoint is not $LLM_LARGE_CKPT_BYTES bytes (partial download?)" >&2
+        echo "frank-large: re-check: bash scripts/download-frank-large-halogen.sh --verify-only" >&2
+        return 1
+    fi
+    if [[ -L "$LLM_LARGE_TOK_DIR" || ! -d "$LLM_LARGE_TOK_DIR" ]]; then
+        echo "ERROR: tokenizer dir missing or a symlink at $LLM_LARGE_TOK_DIR" >&2
+        echo "frank-large: fetch it: bash scripts/download-frank-large-halogen.sh" >&2
+        return 1
+    fi
+    for tf in $LLM_LARGE_TOK_FILES; do
+        if [[ ! -f "$LLM_LARGE_TOK_DIR/$tf" || -L "$LLM_LARGE_TOK_DIR/$tf" ]]; then
+            echo "ERROR: tokenizer file missing or a symlink: $LLM_LARGE_TOK_DIR/$tf" >&2
+            echo "frank-large: fetch it: bash scripts/download-frank-large-halogen.sh" >&2
+            return 1
+        fi
+    done
+
+    retire_halogen_large_quadlet
+    # The predecessor on a re-deploy is THIS unit: stop it (ExecStop stops
+    # the container) before waiting on the port. The stray sweep is keyed
+    # on the FULL container name, which appears in the argv of a manual
+    # `podman run --name frank-large-halogen` / `podman wait` but NOT in
+    # this deployer's own command line (`bash deploy-remote.sh
+    # frank-large`) — so it cannot shoot itself in the foot (the fleet
+    # pkill law), and it can never match frank-small's llama-server.
+    run_privileged systemctl stop frank-large.service 2>/dev/null || true
+    pkill -f 'frank-large-halogen' 2>/dev/null || true
+    podman rm -f frank-large frank-large-halogen >/dev/null 2>&1 || true
+    # Nothing may start until the predecessor has released the port
+    # (2026-09-16: "Deactivated successfully" was not "port free").
     wait_port_free 7770 || return 1
-    install_quadlet frank-large dankclaw/frank-large/quadlet frank-large
-    # No enable: generated units cannot be enabled on this host (v1 quadlet
-    # generator); boot persistence comes from [Install] in the unit file.
-    systemctl --user restart frank-large.service
-    wait_for_llm_large_ready
+    relink_unit frank-large
+    run_privileged systemctl daemon-reload
+    run_privileged systemctl enable frank-large.service
+    run_privileged systemctl restart frank-large.service
     wait_for_health frank-large http://127.0.0.1:7770/health 1200
 }
 
-# Bootstrap-aware gate: if the checkpoint is absent the container is fetching
-# it; wait for the file (the fetch resumes, and the unit restarts it on
-# failure) rather than gating on a /health that cannot answer yet. Reports
-# success either way, so a first deploy does not look like a failure while the
-# engine does exactly what it was told to.
-wait_for_llm_large_ready() {
-    [[ -f "$LLM_LARGE_CHECKPOINT" ]] && return 0
-    echo "frank-large: checkpoint absent at $LLM_LARGE_CHECKPOINT"
-    echo "frank-large: first start — the container is downloading ~127 GB from"
-    echo "frank-large: peonist-ai/halogen-qwen3.8-flash-next (resumes if interrupted)."
-    local i
-    for ((i = 1; i <= 1200; i++)); do
-        if [[ -f "$LLM_LARGE_CHECKPOINT" ]]; then
-            echo "frank-large: weights landed after approximately $((i * 2))s; gating on /health"
-            return 0
-        fi
-        sleep 2
-    done
-    echo "WARN: frank-large download still in progress after 40m — NOT a failure." >&2
-    echo "WARN: the unit keeps fetching; re-run this deploy (or re-check /health) later." >&2
-    return 0
-}
-
-# deploy_llm_small — frank-small is mainline llama.cpp on the RTX 3090, as a
-# ROOT unit: the runtime is a native CUDA binary, so there is no container
-# image to install and no generated unit to avoid. relink_unit points
-# /etc/systemd/system/frank-small.service at the repo file and boot persistence
-# is that unit's [Install] section.
+# deploy_llm_small — frank-small is llama.cpp (master build, CUDA sm_86) on
+# the RTX 3090 as a ROOT unit: the runtime is a native binary tree under
+# ~/GitPackages/llama-master/build-cuda, so there is no container image and
+# no generated unit to avoid. relink_unit points
+# /etc/systemd/system/frank-small.service at the repo file and boot
+# persistence is that unit's [Install] section.
 #
 # It is a SEPARATE endpoint on port 7771; frank-large keeps 7770. The 3090 is
-# not shared: frank-large's halogen engine runs on the gfx1151 iGPU.
-LLM_SMALL_MODEL="${LLM_SMALL_MODEL:-/home/dank/models/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf}"
+# not shared: frank-large's engine runs on the gfx1151 iGPU.
+LLM_SMALL_MODEL_FILE="${LLM_SMALL_MODEL_FILE:-/home/dank/models/Ornith-Q8/Ornith-1.5-9B-Q8_0.gguf}"
+LLM_SMALL_RUNTIME="${LLM_SMALL_RUNTIME:-/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
+# The projector IS the vision capability here: the Ornith GGUF ships no
+# `v.*` tensors, so a deploy that starts without this file silently ships a
+# text-only endpoint. Checked unconditionally, like frank-large's.
+LLM_SMALL_MMPROJ="${LLM_SMALL_MMPROJ:-/home/dank/models/Ornith-Q8/mmproj-Ornith-1.5-9B-BF16.gguf}"
 
 deploy_llm_small() {
-    if [[ ! -f "$LLM_SMALL_MODEL" ]]; then
-        echo "ERROR: model absent at $LLM_SMALL_MODEL" >&2
-        echo "frank-small: fetch ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF (the -mtp build plus the mmproj) first;" >&2
-        echo "frank-small: refusing to start a unit that cannot load." >&2
+    # Refuse rather than restart-loop: a unit whose runtime or payload is
+    # missing never becomes healthy, and a green deploy would leave the
+    # endpoint dark with nothing in the log to say why.
+    if [[ ! -x "$LLM_SMALL_RUNTIME" ]]; then
+        echo "ERROR: llama.cpp runtime absent at $LLM_SMALL_RUNTIME" >&2
+        echo "frank-small: build it (cmake -B build-cuda -DGGML_CUDA=ON" >&2
+        echo "frank-small: -DCMAKE_CUDA_ARCHITECTURES=86 -DCMAKE_CUDA_HOST_COMPILER=/usr/bin/gcc-13" >&2
+        echo "frank-small: && cmake --build build-cuda --target llama-server) first." >&2
         return 1
     fi
+    for f in "$LLM_SMALL_MODEL_FILE" "$LLM_SMALL_MMPROJ"; do
+        if [[ ! -f "$f" ]]; then
+            echo "ERROR: frank-small payload file absent at $f" >&2
+            echo "frank-small: run scripts/download-frank-small.sh first;" >&2
+            echo "frank-small: refusing to start a unit that cannot load." >&2
+            return 1
+        fi
+    done
     relink_unit frank-small
     run_privileged systemctl daemon-reload
     run_privileged systemctl enable frank-small.service
     run_privileged systemctl restart frank-small.service
-    wait_for_health frank-small http://127.0.0.1:7771/health 900
+    wait_for_health frank-small http://127.0.0.1:7771/health 900 || return 1
+    # False-green hardening (frank-large's lesson): the port answering
+    # proves nothing about WHO answers — the unit must be active with a
+    # live MainPID, or a stray SGLang holdover is masquerading as deployed.
+    local main_pid
+    main_pid="$(run_privileged systemctl show -p MainPID --value frank-small.service)"
+    if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
+        echo "FAIL: :7771 answered but frank-small.service has no MainPID — a stray process holds the port" >&2
+        return 1
+    fi
+    echo "frank-small: health answered by unit MainPID $main_pid"
 }
 
-# deploy_llm_medium — frank-medium is mainline llama.cpp on the two R9700s as a
-# ROOT unit since the 2026-09-16 cutover: the runtime is a native gfx1201 HIP
-# binary, so there is no image to install and no generated unit to dodge. It
-# replaces the vllm-radiance replica pair + routing proxy, which held this
-# endpoint for a day and could never use both cards for one model (TP=2 needs
-# an RCCL collective, and the second card cannot dispatch one).
+# deploy_llm_primary — frank-primary is the tcclaviger RDNA4 vLLM fork
+# (digest-pinned image, gfx1201) serving the orcarouter
+# Qwen3.8-Flash-Next-Uncensored-FP8 payload (131 safetensors shards) with
+# MTP4 spec decode and q8/q8 KV on the two R9700s as a ROOT unit whose serve
+# script drives `podman run` directly (the image entrypoint execs
+# `vllm serve`). 2026-09-29 cutover from the R9V radiance fork, whose
+# launcher/shim/setup-state machinery is retired (state dirs stay on disk as
+# rollback; pre-cutover tag frank-primary-r9v-q8-20260929). Same port and
+# alias as the endpoints it replaced; the harness models.yml id is unchanged.
 #
-# Layer split in ONE process is what the successor does instead: layers are
-# distributed across the cards and only the activation at the boundary crosses
-# PCIe, once per forward pass. That is the "pipeline parallelism" the endpoint
-# wanted, with no collective anywhere in it.
-LLM_MEDIUM_BIN="${LLM_MEDIUM_BIN:-/home/dank/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714/bin/llama-server}"
-LLM_MEDIUM_MODEL="${LLM_MEDIUM_MODEL:-/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf}"
-LLM_MEDIUM_DRAFT="${LLM_MEDIUM_DRAFT:-/home/dank/models/Qwen3.8-27B-DFlash2-GGUF/Qwen3.8-27B-DFlash2-Q8_0.gguf}"
-LLM_MEDIUM_MMPROJ="${LLM_MEDIUM_MMPROJ:-/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF/mmproj-BF16.gguf}"
+# Gates: the pinned runtime image must be loadable from dank's rootless
+# storage by its OCI manifest digest (the env file's FRANK_PRIMARY_IMAGE),
+# the payload dir must carry config.json and the expected shard count
+# (FRANK_PRIMARY_MODEL_SHARDS), and the unit env must be complete. Per-file
+# sha256 verification was the fetch script's job at download time, never
+# re-hashed here.
+LLM_PRIMARY_CONTAINER=frank-primary-qwen
+LLM_PRIMARY_LEGACY_CONTAINERS="frank-primary-r9v qwen-fp8 glm-probe"
 
-# The three vLLM user units are retired HERE, not left to rot: the proxy
-# published 7770 on the host network, so the llama-server below cannot bind
-# until they are stopped and the port is released. Never `rm` — the unit files
-# move into the home trash; the images stay in local storage.
-retire_vllm_medium_quadlets() {
-    local trash="$HOME/.AGENTTRASH/20260916-vllm-radiance-retire" unit
-    systemctl --user stop frank-medium-a.service frank-medium-b.service \
-        frank-medium-proxy.service 2>/dev/null || true
-    for unit in frank-medium-a frank-medium-b frank-medium-proxy frank-medium; do
-        if [[ -e "$quadlet_unit_dir/$unit.container" ]]; then
-            mkdir -p "$trash"
-            mv -- "$quadlet_unit_dir/$unit.container" "$trash/$unit.container"
-            echo "retired quadlet $unit.container -> $trash"
-        fi
-    done
-    systemctl --user daemon-reload
-}
-
-deploy_llm_medium() {
-    # Refuse rather than restart-loop. A unit whose runtime or payload is
+deploy_llm_primary() {
+    # Refuse rather than restart-loop: a unit whose image or payload is
     # missing never becomes healthy, and a green deploy would leave the
-    # endpoint dark with nothing in the deploy log to say why.
-    local missing=0
-    if [[ ! -x "$LLM_MEDIUM_BIN" ]]; then
-        echo "ERROR: llama-server absent at $LLM_MEDIUM_BIN" >&2
-        echo "frank-medium: build it ON the host (gfx1201): bash scripts/build-llama-rdna4.sh" >&2
-        missing=1
+    # endpoint dark with nothing in the log to say why.
+    local env_file="/home/dank/serverAppConfigs/scripts/frank-primary.env"
+    local image model_dir shards count
+    if [[ ! -f "$env_file" ]]; then
+        echo "ERROR: frank-primary env file absent: $env_file" >&2
+        return 1
     fi
-    if [[ ! -f "$LLM_MEDIUM_MODEL" ]]; then
-        echo "ERROR: target model absent at $LLM_MEDIUM_MODEL" >&2
-        echo "frank-medium: fetch it: bash scripts/download-frank-medium-gguf.sh" >&2
-        missing=1
+    image="$(sed -n 's/^FRANK_PRIMARY_IMAGE=//p' "$env_file" | tail -1)"
+    model_dir="$(sed -n 's/^FRANK_PRIMARY_MODEL_DIR=//p' "$env_file" | tail -1)"
+    shards="$(sed -n 's/^FRANK_PRIMARY_MODEL_SHARDS=//p' "$env_file" | tail -1)"
+    if [[ -z "$image" || -z "$model_dir" || -z "$shards" ]]; then
+        echo "ERROR: frank-primary.env missing FRANK_PRIMARY_IMAGE/MODEL_DIR/MODEL_SHARDS" >&2
+        return 1
     fi
-    if [[ ! -f "$LLM_MEDIUM_DRAFT" ]]; then
-        echo "ERROR: DFlash2 drafter absent at $LLM_MEDIUM_DRAFT" >&2
-        echo "frank-medium: fetch it: bash scripts/download-frank-medium-gguf.sh" >&2
-        missing=1
+    # Capture podman's output BEFORE grepping: under set -o pipefail, the
+    # classic `podman images | grep -q` races (grep exits on match, podman
+    # dies SIGPIPE 141 writing the rest, pipefail calls it no-match — the
+    # 2026-09-30 deploy refusal).
+    local digests
+    digests="$(podman images --format '{{.RepoDigests}}')"
+    if ! grep -qF "${image#*@}" <<<"$digests"; then
+        echo "ERROR: pinned runtime image absent from rootless storage: $image" >&2
+        echo "frank-primary: fetch it (podman pull $image)" >&2
+        return 1
     fi
-    if [[ ! -f "$LLM_MEDIUM_MMPROJ" ]]; then
-        echo "ERROR: vision projector absent at $LLM_MEDIUM_MMPROJ" >&2
-        echo "frank-medium: fetch it: bash scripts/download-frank-medium-gguf.sh" >&2
-        missing=1
+    if [[ ! -f "$model_dir/config.json" ]]; then
+        echo "ERROR: payload config missing: $model_dir/config.json" >&2
+        echo "frank-primary: re-run scripts/download-frank-primary.sh" >&2
+        return 1
+    fi
+    if [[ ! -f "$model_dir/model.safetensors.index.json" ]]; then
+        echo "ERROR: weight-map index missing: $model_dir/model.safetensors.index.json" >&2
+        echo "frank-primary: re-run scripts/download-frank-primary.sh" >&2
+        return 1
+    fi
+    count="$(find "$model_dir" -maxdepth 1 -name 'model-*-of-*.safetensors' 2>/dev/null | wc -l)"
+    if [[ "$count" != "$shards" ]]; then
+        echo "ERROR: payload shard count wrong: $model_dir (found $count, want $shards)" >&2
+        echo "frank-primary: re-run scripts/download-frank-primary.sh" >&2
+        return 1
     fi
-    [[ "$missing" -eq 0 ]] || return 1
 
-    retire_vllm_medium_quadlets
-    # The thing holding 7770 is now THIS service: on a re-deploy the previous
-    # llama-server is still bound to it, so waiting for the port without
-    # stopping the unit deadlocks the deploy (measured 2026-09-16: "FAIL: port
-    # 7770 is still held after 90s", with the old server happily serving a
-    # scrape). Stop the unit first, then sweep a stray manual run of the same
-    # binary -- that is how a benchmark harness leaves the port busy. `pkill -x
-    # llama-server` matches the process name exactly and can never match the
-    # deploying shell; never `pkill -f` here.
-    run_privileged systemctl stop frank-medium.service 2>/dev/null || true
-    pkill -x llama-server 2>/dev/null || true
-    sleep 2
-    # pasta released the retired proxy's socket at container teardown, and a
-    # stopped engine lingers in TIME_WAIT just past that. Same gate as
-    # frank-large's cutover (2026-09-16 addressed-in-use failure).
+    # A stale container with the same name makes `podman run --name` fail at
+    # start; remove any leftover from a failed previous attempt.
+    # A stopped unit can leave its container running: systemd's cgroup kill
+    # races conmon's --rm cleanup, and the orphan holds the port and GPUs.
+    # Stop the unit, then remove the containers BEFORE waiting on the port.
+    # The legacy identities (the retired R9V container, the ad hoc
+    # experiment container qwen-fp8, the retired glm-probe) hold the same
+    # GPUs and die here too.
+    run_privileged systemctl stop frank-primary.service 2>/dev/null || true
+    local ctr
+    for ctr in $LLM_PRIMARY_LEGACY_CONTAINERS "$LLM_PRIMARY_CONTAINER"; do
+        podman rm -f "$ctr" >/dev/null 2>&1 || true
+    done
+    # Nothing starts until the port is released: the lucebox cutover
+    # (2026-09-26) and the frank-large cutover (2026-09-16) both failed
+    # exactly here when the next engine bound 7770 a second too early.
     wait_port_free 7770 || return 1
-    relink_unit frank-medium
+    relink_unit frank-primary
     run_privileged systemctl daemon-reload
-    run_privileged systemctl enable frank-medium.service
-    run_privileged systemctl restart frank-medium.service
-    wait_for_health frank-medium http://127.0.0.1:7770/health 900
+    run_privileged systemctl enable frank-primary.service
+    run_privileged systemctl restart frank-primary.service
+    # Cold start: FP8 page-in plus torch compile/graph capture (~15 min cold,
+    # measured 2026-09-29). Parity with every predecessor on this port.
+    wait_for_health frank-primary http://127.0.0.1:7770/health 1800 || return 1
+    # False-green hardening (the 2026-09-26 lesson): a bare curl against the
+    # port cannot tell WHO answers — a stray probe holding :7770 satisfies
+    # it while the unit is stopped. The deploy is only green if the unit
+    # itself is active with a live MainPID behind the health answer.
+    local main_pid
+    main_pid="$(run_privileged systemctl show -p MainPID --value frank-primary.service)"
+    if [[ -z "$main_pid" || "$main_pid" == "0" ]]; then
+        echo "FAIL: :7770 answered but frank-primary.service has no MainPID — a stray process holds the port" >&2
+        return 1
+    fi
+    echo "frank-primary: health answered by unit MainPID $main_pid"
 }
 
 wait_for_health() {
@@ -1370,15 +1537,16 @@ deploy_service() {
         marreta) deploy_marreta ;;
         frank-large) deploy_llm_large ;;
         frank-small) deploy_llm_small ;;
-        frank-medium) deploy_llm_medium ;;
+        frank-primary) deploy_llm_primary ;;
         hermes|hermes-dashboard|hermes-webui) deploy_hermes ;;
         romm|romm-mariadb|romm-redis|mariadb|redis) deploy_romm_stack "$service" ;;
         jellyfin) deploy_jellyfin ;;
         calibre-web) deploy_calibre_web ;;
+        calibre-server) deploy_calibre_server ;;
         copyparty) deploy_copyparty ;;
         plex) deploy_plex ;;
         tautulli) deploy_tautulli ;;
-        prometheus|grafana|loki|alloy|node-exporter) deploy_observability "$service" ;;
+        prometheus|grafana|loki|alloy|node-exporter|alertmanager) deploy_observability "$service" ;;
         comfyui) deploy_comfyui ;;
         *) deploy_compose_service "$service" ;;
     esac

scripts/deploy.sh

@@ -35,7 +35,7 @@ Options:
 Examples:
   ./scripts/deploy.sh dankclaw
   ./scripts/deploy.sh dankclaw --service searxng
-  ./scripts/deploy.sh dankserver --service frank-medium
+  ./scripts/deploy.sh dankserver --service frank-primary
   ./scripts/deploy.sh dankclaw --dry-run
 EOF
 }
@@ -138,14 +138,15 @@ valid_service() {
         dankclaw:gitea-server|dankclaw:homepage|\
         dankclaw:comfyui|dankclaw:hermes|dankclaw:hermes-dashboard|\
         dankclaw:hermes-webui|dankclaw:prometheus|dankclaw:grafana|dankclaw:loki|dankclaw:alloy|\
+        dankclaw:alertmanager|\
         dankclaw:node-exporter|dankclaw:authentik-postgres|dankclaw:authentik-server|\
         dankclaw:authentik-worker|dankclaw:authentik-outpost|dankclaw:authentik-outpost-ldap|dankclaw:anubis|dankclaw:seerr|\
         dankclaw:frank-large|dankclaw:frank-small|\
         dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
-        dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:plex|dankserver:tautulli|\
+        dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
         dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
         dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|\
-        dankserver:frank-medium|\
+        dankserver:frank-primary|\
         frankbook-pro:gluetun|frankbook-pro:qbittorrent|frankbook-pro:prowlarr|\
         frankbook-pro:radarr|frankbook-pro:sonarr|frankbook-pro:lidarr|\
         frankbook-pro:readarr)
@@ -195,6 +196,21 @@ else
     printf '    - %s\n' "${PLAN[@]}"
 fi
 
+# --- frankbook e2e trigger --------------------------------------------------
+# The *arr/qBittorrent e2e units walk links that terminate on FrankBook-Pro
+# (192.168.0.112), which sleeps when idle. The [3/3] e2e gate therefore runs
+# the frankbook Playwright project ONLY when something in its blast radius
+# changed since origin/main: the frankbook spec itself, the frankbook-pro
+# stack, the homepage registry, or the Caddyfile serving the arr vhosts.
+# Every other deploy skips it (tests/e2e/run.sh excludes the project by
+# default). See tests/e2e/specs/19-frankbook.spec.ts.
+FB_E2E_TRIGGER=0
+while IFS= read -r fb_path; do
+    case "$fb_path" in
+        tests/e2e/specs/19-frankbook.spec.ts|frankbook-pro/*|shared/homepage/services.json|dankclaw/Caddyfile) FB_E2E_TRIGGER=1 ;;
+    esac
+done < <(git diff --name-only "$BASE_REVISION" "$HEAD_REVISION")
+
 REMOTE="${REMOTE_USER}@${REMOTE_HOST}"
 REMOTE_ARGS=("$TARGET" "$HEAD_REVISION" "$REMOTE_DIR" "$REMOTE_SHELL")
 if [[ "${#EXPLICIT_SERVICES[@]}" -eq 0 ]]; then
@@ -208,6 +224,11 @@ if [[ "$DRY_RUN" -eq 1 ]]; then
     printf '[dry-run] Runner arguments:'
     printf ' %q' "${REMOTE_ARGS[@]}"
     printf '\n'
+    if [[ "$FB_E2E_TRIGGER" -eq 1 ]]; then
+        echo "[dry-run] e2e gate: frankbook project INCLUDED (frankbook trigger paths changed)."
+    else
+        echo "[dry-run] e2e gate: frankbook project EXCLUDED (no frankbook trigger paths)."
+    fi
     exit 0
 fi
 
@@ -249,7 +270,12 @@ if [[ "$SKIP_E2E" -eq 1 ]]; then
     echo "[3/3] e2e gate SKIPPED (--skip-e2e): live fleet NOT verified." >&2
 else
     echo "[3/3] Running Playwright e2e gate against the live fleet..."
-    if bash "$REPO_ROOT/tests/e2e/run.sh"; then
+    E2E_ENV=()
+    if [[ "$FB_E2E_TRIGGER" -eq 1 ]]; then
+        echo "      frankbook trigger paths changed: including the frankbook e2e project."
+        E2E_ENV=(E2E_INCLUDE_FRANKBOOK=1)
+    fi
+    if env "${E2E_ENV[@]}" bash "$REPO_ROOT/tests/e2e/run.sh"; then
         echo "e2e gate PASSED."
     else
         echo "=== DEPLOY INCOMPLETE: e2e gate FAILED on $TARGET ===" >&2

scripts/docker/frank-primary-freetoken.Dockerfile

@@ -0,0 +1,62 @@
+# frank-primary-freetoken.Dockerfile — FreeToken (PR #587 ROCm) serving image
+# for the frank-primary endpoint (dankserver :7770, 1x R9700 gfx1201).
+#
+# Built on dankserver with rootless podman (dank user), same storage the
+# service's `podman run` reads:
+#   podman build -t frank-primary-freetoken:pr587-77a7201 \
+#     -f scripts/docker/frank-primary-freetoken.Dockerfile scripts/docker/
+#
+# Base: the PR's own docs/install_amd.md names this exact image as the
+# known-good ROCm 7.14 stack for gfx1201 (torch 2.11.0+rocm7.14, triton-rocm
+# 3.7 — the original pin; FreeToken main+PR587 accepts torch>=2.11,<2.16 and
+# triton>=3.6,<3.9). There is no rocm7.14+torch2.14 image on Docker Hub (the
+# 2.14 images are rocm10.1.0 — an untested pair for this PR), so the PR doc's
+# verbatim recommendation is the deployed stack.
+#
+# FreeToken source: FlashML-org/FreeToken PR #587 at pinned head
+# 77a7201f98f00d93f618c6cf8b81ffe8360e0899 (branch rocm714_torch214, base
+# main 581fdca3). The PR is ROCm-gated end to end; the NVIDIA/CUDA path is
+# byte-identical to main.
+#
+# Build law from docs/install_amd.md:
+#   - `pip install --no-build-isolation --no-deps -e .` — the image's ROCm
+#     torch is preserved and used to compile the C++/HIP extensions.
+#   - NEVER a plain `pip install -e .` on ROCm: the dependency set is
+#     authored for NVIDIA (flashlib pulls numba + nvidia-cutlass-dsl; torch
+#     metadata pulls cuda-bindings) and the resolve can REPLACE the ROCm
+#     torch with a CUDA build.
+#   - Runtime deps installed explicitly, curated by the same doc. flashinfer
+#     / sgl_kernel / nvidia-cutlass-dsl have no ROCm builds and are
+#     intentionally absent — FreeToken gates them off and falls back to its
+#     Triton kernels on ROCm.
+#
+# Base is digest-pinned (2026-10-05): sha256:a223aee1… is the manifest this
+# build was qualified against; a re-pushed tag cannot change the stack.
+FROM rocm/pytorch@sha256:a223aee17aef5d21c3b9f63436dd19d27d1c665ec8b2f40011c9546cabae2a80
+
+# Pin the PR head, not the branch: a future push to rocm714_torch214 would
+# otherwise silently change the deployed engine.
+# The pip-ROCm torch in this image ships libhiprtc as the VERSIONED
+# libhiprtc.so.7 only; the PR's `_cpu_moe` extension links `-lhiprtc`, and
+# ld looks for the unversioned dev symlink in the ALREADY-LINKED
+# _rocm_sdk_core/lib dir. Create it there (first build failure 2026-10-05:
+# `ld: cannot find -lhiprtc`).
+RUN ln -s libhiprtc.so.7 /opt/venv/lib/python3.12/site-packages/_rocm_sdk_core/lib/libhiprtc.so
+RUN git clone https://github.com/FlashML-org/FreeToken.git /opt/FreeToken \
+ && cd /opt/FreeToken \
+ && git fetch origin pull/587/head \
+ && git checkout 77a7201f98f00d93f618c6cf8b81ffe8360e0899
+
+WORKDIR /opt/FreeToken
+RUN python -m pip install --no-build-isolation --no-deps -e .
+
+# Curated runtime dependencies (docs/install_amd.md). transformers is pinned
+# to the pyproject window (>=5.16,<5.17); the rest are the doc's portable set.
+# torch / torchvision / triton-rocm come from the base image, 1:1 aligned.
+RUN python -m pip install \
+      "transformers>=5.16,<5.17" tokenizers safetensors huggingface_hub \
+      fastapi uvicorn pydantic openai httpx anyio \
+      einops gguf msgpack pyzmq typer rich apache-tvm-ffi \
+      partial-json-parser \
+ && python -m pip install --no-deps flashlib==0.3.0 \
+ && python -m pip install ninja

scripts/download-frank-large-halogen.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+# download-frank-large-halogen.sh — fetch frank-large's halogen payload:
+# the qwen3.8-27b .hgn checkpoint and the export tokenizer, from the
+# PUBLIC peonist-ai/halogen-qwen3.8-27b repo (no token — and per the fleet
+# secret law, none would ever touch argv).
+#
+# The fetch law, same as every sibling script: curl -f (an access error
+# must not become a "successful" 153-byte artifact), exact byte count from
+# the model card, LFS sha256 verification, --verify-only to re-check an
+# existing payload without downloading.
+set -euo pipefail
+
+DEST="${FRANK_LARGE_MODEL_DIR:-$HOME/models/halogen-qwen3.8-27b}"
+BASE="https://huggingface.co/peonist-ai/halogen-qwen3.8-27b/resolve/main"
+
+FILES=(qwen3.8-27b-p1w4d-d2.hgn
+       tokenizer/tokenizer.json
+       tokenizer/vocab.json
+       tokenizer/merges.txt
+       tokenizer/chat_template.jinja
+       tokenizer/tokenizer_config.json)
+# Byte sizes as published by the Hub on 2026-10-05, plus sha256 (the LFS
+# oid) where the Hub stores an LFS object — plain git blobs carry no oid
+# and are size-checked only. Re-check with:
+# curl -s "https://huggingface.co/api/models/peonist-ai/halogen-qwen3.8-27b/tree/main?recursive=true"
+SIZES=(35865565184 12809320 6722759 3353259 8952 17928)
+SHAS=(274c3fc767fb57faf025dd03c76376b1fc1b32448aef45aaf02d1898fa962089
+      0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3 - - - -)
+
+VERIFY_ONLY=0
+[ "${1:-}" = "--verify-only" ] && VERIFY_ONLY=1
+
+mkdir -p "$DEST"
+
+for i in "${!FILES[@]}"; do
+    f="${FILES[$i]}"; want="${SIZES[$i]}"; sha="${SHAS[$i]}"
+    path="$DEST/$f"
+    mkdir -p "$(dirname "$path")"
+    if [ "$VERIFY_ONLY" -eq 0 ]; then
+        # A file already at the published size skips the fetch entirely:
+        # resuming it would ask the CDN for a range past EOF (416 -> curl
+        # error under set -e), and re-pulling 33.4 GiB to prove a point is
+        # not what --verify-only is for. sha256 below still decides.
+        if [ -f "$path" ] && [ "$(stat -c %s "$path")" = "$want" ]; then
+            echo "HAVE  $f ($want bytes) — skipping fetch"
+        else
+            echo "FETCH $f -> $path"
+            # HF answers /resolve/ with a 302 to the Xet CDN and curl DROPS
+            # the Range header across redirects — a naive -C - then
+            # "resumes" by pulling the ~1 KB redirect body (measured
+            # 2026-10-05: file frozen 14 MiB short, curl exit 0). Resolve
+            # the effective URL first, then resume against the CDN, which
+            # honors Range.
+            url=$(curl -sfIL -o /dev/null -w '%{url_effective}' "$BASE/$f") \
+                || { echo "cannot resolve $BASE/$f" >&2; exit 1; }
+            curl -fL --retry 5 -C - -o "$path" "$url"
+        fi
+    fi
+    [ -f "$path" ] || { echo "MISSING $path (run without --verify-only)" >&2; exit 1; }
+    got="$(stat -c %s "$path")"
+    [ "$got" = "$want" ] || { echo "SIZE $path: $got != $want" >&2; exit 1; }
+    echo "OK   $f ($got bytes)"
+done
+
+echo "byte counts match; verifying sha256 where the Hub publishes an LFS oid..."
+for i in "${!FILES[@]}"; do
+    f="${FILES[$i]}"; sha="${SHAS[$i]}"
+    if [ "$sha" = "-" ]; then
+        echo "OK   $f (plain git blob, size-checked)"
+        continue
+    fi
+    got="$(sha256sum "$DEST/$f" | cut -d' ' -f1)"
+    [ "$got" = "$sha" ] || { echo "SHA256 $f: $got != $sha" >&2; exit 1; }
+    echo "OK   $f sha256"
+done
+echo "DOWNLOAD_DONE $DEST"

scripts/download-frank-large-kingjones-uncensored.sh

@@ -0,0 +1,116 @@
+#!/usr/bin/env bash
+# Download kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF for
+# frank-large (Dankclaw iGPU). Public repo — no auth header, and never a token
+# on the command line (it is world-readable in /proc/<pid>/cmdline).
+#
+# The payload is the repo's Q6 AGENT rung:
+#   * Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf — ftype 114, 23.15 GiB.
+#     Q6_0_ROCMFPX tensor type (102) + the AGENT routing, which keeps the
+#     output-head and attention tensors at plain Q8_0 — the variant the card
+#     names for MTP serving ("Take Q6 AGENT (114) if you want protected
+#     heads"). Like every ROCmFPX file it loads on nothing but the fork.
+#
+# The other two files frank-large needs are NOT re-downloaded:
+#   * mtp-Qwen3.8-27B-Q4_0.gguf — the uncensored repo redistributes the
+#     ALIGNED repo's draft head byte for byte (identical Hub LFS oid
+#     051a1764…), so it is hardlinked out of the aligned payload dir when that
+#     dir holds the verified bytes, and fetched from this repo otherwise.
+#   * mmproj-Qwen3.8-27B-BF16.gguf — this repo ships a Q8_0 projector; the
+#     fleet keeps the BF16 one, which is the projector that was verified 4/4
+#     on spatial ground truth on this host. Hardlinked the same way, fetched
+#     from the ALIGNED repo if it is not on disk.
+#
+# Fresh downloads, no resume (`curl -C -` once produced an oversized file on a
+# 31 GiB payload); -f/--fail-with-body so an HTTP 401 is not a success, then
+# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
+#
+# Usage:
+#   scripts/download-frank-large-kingjones-uncensored.sh [DEST_DIR] [--verify-only]
+set -euo pipefail
+
+REPO="kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF"
+ALIGNED_REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+ALIGNED_BASE="https://huggingface.co/${ALIGNED_REPO}/resolve/main"
+SOURCE_DIR="${SOURCE_DIR:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP}"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP}"
+
+# name bytes sha256 base_url may_link — may_link=1 means the same bytes may
+# already exist in SOURCE_DIR (hardlink them instead of re-fetching).
+FILES=(
+  "Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf 24856607008 e966deb3d1a5a1b9ca7765126292f2d822f3dccdedb3b4872e673dec1cddc199 $BASE 0"
+  # The draft head is redistributed byte-identical by both repos; the BF16
+  # projector only exists in the aligned repo.
+  "mtp-Qwen3.8-27B-Q4_0.gguf 1680271648 051a1764cff8c4f3ee6ae8b00593a0364c7539c67fa50ffc58f3f96509fca38e $BASE 1"
+  "mmproj-Qwen3.8-27B-BF16.gguf 931145888 de2a49866988ea272c43dc2a43ee2662c25725a717ca5b9059ea420a97f53fe8 $ALIGNED_BASE 1"
+)
+
+matches() {  # matches FILE SIZE SHA — byte count AND LFS sha256
+    [[ -f "$1" ]] && [[ "$(stat -c%s "$1")" == "$2" ]] \
+        && sha256sum -c --quiet <(printf '%s  %s\n' "$3" "$1")
+}
+
+mkdir -p "$DEST"
+status=0
+
+fetch() {  # fetch NAME SIZE SHA BASE_URL
+    local name="$1" size="$2" sha="$3" url="$4" f="$DEST/$1"
+    echo "FETCHING  $name"
+    curl -fsSL --fail-with-body -o "$f.part" "$url/$name"
+    mv "$f.part" "$f"
+    if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+        echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+        return 1
+    fi
+    if ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+        echo "SHA FAIL  $name"
+        return 1
+    fi
+    echo "VERIFIED  $name"
+}
+
+for entry in "${FILES[@]}"; do
+    read -r name size sha url may_link <<<"$entry"
+    f="$DEST/$name"
+    if matches "$f" "$size" "$sha"; then
+        echo "OK        $name"
+        continue
+    fi
+    if (( VERIFY_ONLY )); then
+        echo "MISSING   $name"
+        status=1
+        continue
+    fi
+    # Shared bytes: prefer the local copy (same filesystem, so a hardlink costs
+    # nothing) over re-downloading gigabytes that hash identically.
+    if [[ "$may_link" == 1 && -n "${SOURCE_DIR:-}" ]] && matches "$SOURCE_DIR/$name" "$size" "$sha"; then
+        echo "LINKING   $name from $SOURCE_DIR"
+        ln -f "$SOURCE_DIR/$name" "$f" 2>/dev/null || cp -f "$SOURCE_DIR/$name" "$f"
+        if matches "$f" "$size" "$sha"; then
+            echo "VERIFIED  $name"
+            continue
+        fi
+        echo "SHA FAIL  $name (linked copy)"
+        status=1
+        continue
+    fi
+    if ! fetch "$name" "$size" "$sha" "$url"; then
+        status=1
+    fi
+done
+
+if (( status == 0 )); then
+    echo "DOWNLOAD_DONE $DEST"
+else
+    echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"

scripts/download-frank-large-kingjones.sh

@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+# Download kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF for frank-large
+# (Dankclaw iGPU). Public repo — no auth header, and never a token on the
+# command line (it is world-readable in /proc/<pid>/cmdline).
+#
+# The payload is the repo's highest-quality target plus its MTP draft head and
+# vision projector:
+#   * Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf — ftype 115, 187 tensors at plain
+#     Q8_0 and 311 at GGML_TYPE_Q8_0_ROCMFPX (103). The AGENT routing keeps the
+#     output-head tensors at full Q8_0 (187 vs 2 in the non-AGENT build), which
+#     is the variant to take when serving with the MTP head.
+#   * mtp-Qwen3.8-27B-Q4_0.gguf — the draft head. Q4_0, NOT Q8_0: the head's
+#     per-step cost is its projection over a 248K vocabulary, so the cheaper
+#     head is faster end to end at equal acceptance (author: +6.0% at n-max 1,
+#     +10.2% at n-max 2).
+#   * mmproj-Qwen3.8-27B-BF16.gguf — the vision tower (verified 4/4 on spatial
+#     ground truth with -fa on).
+#
+# Fresh downloads, no resume (`curl -C -` once produced an oversized file on a
+# 31 GiB payload); -f/--fail-with-body so an HTTP 401 is not a success, then
+# byte count and the Hub LFS sha256. Idempotent: complete files are skipped.
+#
+# The repo's own SHA256SUMS covers only the ROCmFP4 files, so the hashes below
+# come from the Hub LFS oids (api/models/<repo>/tree/main).
+#
+# Usage:
+#   scripts/download-frank-large-kingjones.sh [DEST_DIR] [--verify-only]
+set -euo pipefail
+
+REPO="kingjones777/Qwen3.8-27B-ROCmFP4-STRIX-MTP-GGUF"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP}"
+
+# name bytes sha256
+FILES=(
+  "Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf 28214425952 f3ef2bdbf56e244fff14b72114dc2aa300b84a4cb4eaf54f2e5e4da1cfcae10b"
+  "mtp-Qwen3.8-27B-Q4_0.gguf 1680271648 051a1764cff8c4f3ee6ae8b00593a0364c7539c67fa50ffc58f3f96509fca38e"
+  "mmproj-Qwen3.8-27B-BF16.gguf 931145888 de2a49866988ea272c43dc2a43ee2662c25725a717ca5b9059ea420a97f53fe8"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"

scripts/download-frank-large-orca.sh

@@ -0,0 +1,108 @@
+#!/usr/bin/env bash
+# Download and assemble jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca (the frank-large
+# model) on Dankclaw.
+#
+#   bash scripts/download-frank-large-orca.sh
+#   bash scripts/download-frank-large-orca.sh --verify-only
+#
+# The package is 136 GB of Hub-sized transport parts (HF caps new uploads at
+# 50 GB, so the 79 GB main GGUF, the 52 GB PLE payload and the 4 GB MTP head
+# ship as .partNofM fragments).  assemble.py (from the repo) concatenates the
+# parts, verifies every part and final SHA-256, and consumes the parts.
+#
+# Integrity: every part is checked against the per-part sha256 in parts.json
+# (fetched from the repo, not trusted from disk), then the assembled files
+# against SHA256SUMS.  The repo is public (gated: false); an HF token at
+# ~/.cache/huggingface/token is used when present to clear the anonymous
+# rate limit on a 136 GB transfer.
+#
+# Layout after a successful run (MODEL_DIR):
+#   Qwen3.8-Flash-CIRU-STRIX-Orca.gguf          79,397,818,912  main model
+#   mtp/Qwen3.8-Flash-CIRU-STRIX-Orca-MTP-Q8_0.gguf  4,135,893,440  draft
+#   ple/ple.payload.bin                         52,429,053,952  external PLE
+#   ple/ple.manifest.json, ple/ple.scale.bf16               PLE metadata
+#   vision/mmproj-Qwen3.8-Flash-Orca-F16.mmproj  904,004,320    projector
+#   assemble.py, parts.json, SHA256SUMS                     (kept for --verify-only)
+set -euo pipefail
+
+REPO="jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca"
+MODEL_DIR="/home/dank/models/Qwen3.8-Flash-CIRU-STRIX-Orca"
+BASE="https://huggingface.co/$REPO/resolve/main"
+
+mode="download"
+[[ "${1:-}" == "--verify-only" ]] && mode="verify"
+
+token=""
+auth=()
+if [[ -r "$HOME/.cache/huggingface/token" ]]; then
+    token="$(cat "$HOME/.cache/huggingface/token")"
+    auth=(-H "Authorization: Bearer $token")
+else
+    echo "no HF token at ~/.cache/huggingface/token; downloading anonymously"
+fi
+
+dl() { # dl <relpath>
+    local dest="$MODEL_DIR/$1"
+    mkdir -p "$(dirname "$dest")"
+    echo "  fetching $1"
+    curl -fL -C - -o "$dest" --retry 10 --retry-delay 5 --retry-max-time 21600 \
+        "${auth[@]}" "$BASE/$1"
+}
+
+if [[ "$mode" == "download" ]]; then
+    # Manifest + small files first: the part hashes below come from the
+    # freshly fetched parts.json, so a stale/corrupt copy cannot pass.
+    dl "assemble.py"
+    dl "parts.json"
+    dl "SHA256SUMS"
+    dl "ple/ple.manifest.json"
+    dl "ple/ple.scale.bf16"
+    dl "vision/SHA256SUMS"
+    dl "vision/mmproj-Qwen3.8-Flash-Orca-F16.mmproj"
+
+    echo "fetching the five transport parts (136 GB)"
+    while IFS= read -r part; do
+        dl "$part"
+    done < <(python3 -c "
+import json
+for item in json.load(open('$MODEL_DIR/parts.json'))['files']:
+    for p in item['parts']:
+        print(p['path'])
+")
+fi
+
+echo "verifying part sha256 (against the repo parts.json)"
+python3 - "$MODEL_DIR" <<'PY'
+import hashlib, json, sys
+from pathlib import Path
+root = Path(sys.argv[1])
+manifest = json.loads((root / 'parts.json').read_text())
+ok = True
+def digest(p):
+    h = hashlib.sha256()
+    with p.open('rb') as f:
+        for data in iter(lambda: f.read(8 * 1024 * 1024), b''):
+            h.update(data)
+    return h.hexdigest()
+for item in manifest['files']:
+    for p in item['parts']:
+        path = root / p['path']
+        if not path.exists():
+            print(f"  MISSING {p['path']}"); ok = False; continue
+        if path.stat().st_size != p['bytes']:
+            print(f"  BAD-SIZE {p['path']}"); ok = False; continue
+        if digest(path) != p['sha256']:
+            print(f"  BAD-SHA  {p['path']}"); ok = False; continue
+        print(f"  OK   {p['path']}")
+sys.exit(0 if ok else 1)
+PY
+
+if [[ "$mode" == "download" ]]; then
+    echo "assembling (consumes verified parts)"
+    (cd "$MODEL_DIR" && python3 assemble.py)
+fi
+
+echo "verifying assembled files against SHA256SUMS"
+(cd "$MODEL_DIR" && sha256sum -c SHA256SUMS)
+(cd "$MODEL_DIR/vision" && sha256sum -c SHA256SUMS)
+echo "frank-large payload complete: $MODEL_DIR"

scripts/download-frank-large-q4.sh

@@ -0,0 +1,76 @@
+#!/usr/bin/env bash
+# Download the Q4_K_XL frank-large payload on Dankclaw:
+#   unsloth/Qwen3.8-27B-GGUF  ->  Qwen3.8-27B-UD-Q4_K_XL.gguf
+#
+#   bash scripts/download-frank-large-q4.sh
+#   bash scripts/download-frank-large-q4.sh --verify-only
+#
+# Integrity: the file is checked against its declared byte count and sha256
+# (the LFS oid from the Hub tree API, 2026-10-01). NO AUTH HEADER IS USED:
+# the repo is public and ungated (gated: false), and a token on the curl
+# command line is world-readable via /proc/<pid>/cmdline. If this ever needs
+# a token, put it in a mode-600 file and pass it with `curl -K`.
+#
+# Layout after a successful run (MODEL_DIR):
+#   Qwen3.8-27B-UD-Q4_K_XL.gguf   17559178144  (16.35 GiB)
+#
+# WHY Q4 AND NOT THE Q8 SWIFT PAYLOAD: this is the vanilla (non-abliterated)
+# Qwen3.8-27B at UD-Q4_K_XL, ~13 GiB smaller than the Swift UD-Q8_K_XL. Every
+# GiB of weights saved is a GiB of f16 KV the endpoint can hold instead: the
+# KV costs 64 KiB/token (16 full-attention layers x 4 KV heads x 256 head_dim
+# x 2 (K,V) x 2 B), so 4 sessions x 163840 ctx = 640,000 tokens = 39.06 GiB.
+#
+# The speculative drafter is UNCHANGED and still fetched by its own script
+# (scripts/download-frank-large.sh, z-lab's Hub repo). That name must not
+# appear here at all — the frank-large seam greps THIS file for it to prove
+# the drafter was not folded into the payload fetch.
+#
+# NOTE ON MTP: this payload has no baked-in MTP head (the Hub repo ships one
+# separately as MTP/mtp-Qwen3.8-27B-Q4_0.gguf). frank-large does not need it —
+# spec decode here is the block-diffusion drafter, not MTP.
+set -euo pipefail
+
+MODEL_DIR="/home/dank/models/Qwen3.8-27B-UD-Q4_K_XL"
+REL="Qwen3.8-27B-UD-Q4_K_XL.gguf"
+SIZE=17559178144
+SHA=3f227079003add2511437e5b1e94812e363385225bf6a9b47b0054a72bc8b01e
+URL="https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/${REL}"
+
+mode="download"
+[[ "${1:-}" == "--verify-only" ]] && mode="verify"
+
+if [[ "$mode" == "download" ]]; then
+    mkdir -p "$MODEL_DIR"
+    if [[ -s "$MODEL_DIR/$REL" ]]; then
+        echo "already present: $MODEL_DIR/$REL"
+    else
+        echo "fetching $REL (17559178144 bytes) — this is ~16.4 GiB"
+        # -f so an HTTP error is a failure, not a 153-byte "artifact".
+        # -C - so an interrupted run resumes instead of restarting.
+        curl -fL -C - --retry 5 --retry-delay 3 -o "$MODEL_DIR/$REL" "$URL"
+    fi
+fi
+
+echo "verifying $MODEL_DIR against declared size + sha256"
+python3 - "$MODEL_DIR" "$REL" "$SIZE" "$SHA" <<'PY'
+import hashlib, os, sys
+model_dir, rel, size, sha = sys.argv[1], sys.argv[2], int(sys.argv[3]), sys.argv[4]
+dest = os.path.join(model_dir, rel)
+if not os.path.isfile(dest):
+    print(f"  MISSING {rel}")
+    sys.exit(1)
+actual = os.path.getsize(dest)
+if actual != size:
+    print(f"  SIZE MISMATCH {rel}: {actual} != {size}")
+    sys.exit(1)
+h = hashlib.sha256()
+with open(dest, "rb") as f:
+    for chunk in iter(lambda: f.read(1 << 24), b""):
+        h.update(chunk)
+if h.hexdigest() != sha:
+    print(f"  SHA256 MISMATCH {rel}: {h.hexdigest()}")
+    sys.exit(1)
+print(f"  OK {rel} ({actual} bytes, sha256 matches the Hub LFS oid)")
+PY
+
+echo "Q4_K_XL payload complete: $MODEL_DIR"

scripts/download-frank-large-swift.sh

@@ -0,0 +1,66 @@
+#!/usr/bin/env bash
+# Download the Swift-1.5 frank-large candidate on Dankclaw:
+#   ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
+#     Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
+#
+#   bash scripts/download-frank-large-swift.sh
+#   bash scripts/download-frank-large-swift.sh --verify-only
+#
+# Integrity: the file is checked against its declared byte count and sha256
+# (the LFS oid from the Hub, 2026-09-28). NO AUTH HEADER IS USED: the repo is
+# public and ungated, and a token on the curl command line is world-readable
+# via /proc/<pid>/cmdline. If this ever needs a token, put it in a mode-600
+# file and pass it with `curl -K`.
+#
+# Layout after a successful run (MODEL_DIR):
+#   Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf  31457990272
+#
+# This is the CANDIDATE payload, not the deployed one. frank-large still
+# serves scripts/download-frank-large.sh's output; do not repoint
+# frank-large.env at MODEL_DIR until the gufo load test passes.
+set -euo pipefail
+
+MODEL_DIR="/home/dank/models/Swift-1.5-Qwen3.8-27B-Uncensored-MTP-UD-Q8_K_XL"
+REL="Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf"
+SIZE=31457990272
+SHA=9076f99a2b1c0d232d5c169bb117e919ca21bafbe092be8fc8d9c4d645ed09c9
+URL="https://huggingface.co/ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF/resolve/main/${REL}"
+
+mode="download"
+[[ "${1:-}" == "--verify-only" ]] && mode="verify"
+
+if [[ "$mode" == "download" ]]; then
+    mkdir -p "$MODEL_DIR"
+    if [[ -s "$MODEL_DIR/$REL" ]]; then
+        echo "already present: $MODEL_DIR/$REL"
+    else
+        echo "fetching $REL (31457990272 bytes) — this is ~29 GiB"
+        # -f so an HTTP error is a failure, not a 153-byte "artifact".
+        # -C - so an interrupted run resumes instead of restarting.
+        curl -fL -C - --retry 5 --retry-delay 3 -o "$MODEL_DIR/$REL" "$URL"
+    fi
+fi
+
+echo "verifying $MODEL_DIR against declared size + sha256"
+python3 - "$MODEL_DIR" "$REL" "$SIZE" "$SHA" <<'PY'
+import hashlib, os, sys
+model_dir, rel, size, sha = sys.argv[1], sys.argv[2], int(sys.argv[3]), sys.argv[4]
+dest = os.path.join(model_dir, rel)
+if not os.path.isfile(dest):
+    print(f"  MISSING {rel}")
+    sys.exit(1)
+actual = os.path.getsize(dest)
+if actual != size:
+    print(f"  SIZE MISMATCH {rel}: {actual} != {size}")
+    sys.exit(1)
+h = hashlib.sha256()
+with open(dest, "rb") as f:
+    for chunk in iter(lambda: f.read(1 << 24), b""):
+        h.update(chunk)
+if h.hexdigest() != sha:
+    print(f"  SHA256 MISMATCH {rel}: {h.hexdigest()}")
+    sys.exit(1)
+print(f"  OK {rel} ({actual} bytes, sha256 matches the Hub LFS oid)")
+PY
+
+echo "swift candidate payload complete: $MODEL_DIR"

scripts/download-frank-large-unsloth.sh

@@ -0,0 +1,115 @@
+#!/usr/bin/env bash
+# Download unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (the
+# frank-large model) on Dankclaw.
+#
+#   bash scripts/download-frank-large-unsloth.sh
+#   bash scripts/download-frank-large-unsloth.sh --verify-only
+#
+# The UD-IQ4_XS GGUF ships as three llama.cpp split parts (HF caps uploads at
+# 50 GB). The parts ARE the canonical payload: the serve script points -m at
+# part 1 and llama.cpp loads the 3-way split natively. Do NOT concatenate
+# them into a single file to serve — part 1's header carries
+# ggml.file_count=3, so a byte-correct concatenation still advertises itself
+# as a split and the loader rejects the non-split filename.
+# The MTP shared-Q8_0 head borrows the token embedding and the output
+# projection from the main model at runtime (it is the 2.6 GB remainder of
+# the 4B MTP layer). The F16 projector is fetched for a future vision flip;
+# the serve script does not load it while ENABLE_VISION=0.
+#
+# Integrity: every part is checked against its LFS sha256 (fetched live from
+# the Hub tree API, not trusted from disk). The repo is public (gated:
+# false); an HF token at ~/.cache/huggingface/token is used when present to
+# clear the anonymous rate limit on a 94 GB transfer.
+#
+# Layout after a successful run (MODEL_DIR):
+#   UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-0000{1,2,3}-of-00003.gguf  the 3-way split
+#   mtp/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf   2,786,568,256  draft head
+#   vision/mmproj-F16.gguf                         904,004,000  projector
+set -euo pipefail
+
+REPO="unsloth/Qwen3.8-Flash-Next-GGUF"
+MODEL_DIR="/home/dank/models/Qwen3.8-Flash-Next-UD-IQ4_XS"
+BASE="https://huggingface.co/$REPO/resolve/main"
+
+mode="download"
+[[ "${1:-}" == "--verify-only" ]] && mode="verify"
+
+token=""
+auth=()
+if [[ -r "$HOME/.cache/huggingface/token" ]]; then
+    token="$(tr -d '[:space:]' < "$HOME/.cache/huggingface/token")"
+    auth=(-H "Authorization: Bearer $token")
+fi
+
+dl() { # dl <relpath>
+    local rel="$1" dest="$MODEL_DIR/$rel"
+    [[ -s "$dest" ]] && return 0
+    mkdir -p "$(dirname "$dest")"
+    echo "fetching $rel"
+    curl -fL -C - "${auth[@]}" --retry 5 --retry-delay 3 -o "$dest" "$BASE/$rel"
+}
+
+# relpath size sha256 — the LFS oids from the Hub tree API (2026-09-21)
+FILES=(
+    "UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00001-of-00003.gguf 10946624 5ce89370720f8bf90890f439361282104c1aa1482d4013bb9a50923e758e71a4"
+    "UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00002-of-00003.gguf 49835229856 577a38a2392b40ca2193cea502e1d92f60b8cd370675d308e0ec21885d9daaa7"
+    "UD-IQ4_XS/Qwen3.8-Flash-Next-UD-IQ4_XS-00003-of-00003.gguf 43836407744 d4634e6d84f0ebb0940be15c90d3790bf6464e3dea3a1cddc567dc0e83ad8833"
+    "MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf 2786568256 5ff54097406a905cf3a724c709124ceb0e3e10235ee862298969e91c96fa96e6"
+    "mmproj-F16.gguf 904004000 1f7b7f0b984cf065c604360c29c8098362ed61b290db0ff12c6f360bb1a8a980"
+)
+
+RELS=()
+for line in "${FILES[@]}"; do
+    read -r rel size _ <<<"$line"
+    RELS+=("$rel")
+    if [[ "$mode" == "download" ]]; then
+        dl "$rel"
+    fi
+done
+
+echo "verifying parts against the live Hub tree API"
+python3 - "$MODEL_DIR" "$REPO" "${RELS[@]}" <<'PY'
+import hashlib, json, os, sys, urllib.request
+
+model_dir, repo = sys.argv[1], sys.argv[2]
+tree = json.load(urllib.request.urlopen(
+    f"https://huggingface.co/api/models/{repo}/tree/main?recursive=true"))
+want = {}
+for f in tree:
+    if f["type"] == "file" and f.get("lfs"):
+        want[f["path"]] = (f["lfs"]["size"], f["lfs"]["oid"])
+
+def sha256(path):
+    h = hashlib.sha256()
+    with open(path, "rb") as fh:
+        for chunk in iter(lambda: fh.read(1 << 24), b""):
+            h.update(chunk)
+    return h.hexdigest()
+
+ok = True
+expected = set(sys.argv[3:])
+for rel in sorted(expected):
+    if rel not in want:
+        print(f"NOT IN HUB TREE {rel}"); ok = False; continue
+    size, oid = want[rel]
+    path = os.path.join(model_dir, rel)
+    if not os.path.isfile(path):
+        print(f"MISSING {rel}"); ok = False; continue
+    actual_size = os.path.getsize(path)
+    if actual_size != size:
+        print(f"SIZE MISMATCH {rel}: {actual_size} != {size}"); ok = False; continue
+    digest = sha256(path)
+    if digest != oid:
+        print(f"SHA256 MISMATCH {rel}"); ok = False; continue
+    print(f"ok {rel}")
+sys.exit(0 if ok else 1)
+PY
+
+if [[ "$mode" == "download" ]]; then
+    mkdir -p "$MODEL_DIR/mtp" "$MODEL_DIR/vision"
+    [[ -e "$MODEL_DIR/mtp/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf" ]] ||
+        mv "$MODEL_DIR/MTP/mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf" "$MODEL_DIR/mtp/"
+    [[ -e "$MODEL_DIR/vision/mmproj-F16.gguf" ]] || mv "$MODEL_DIR/mmproj-F16.gguf" "$MODEL_DIR/vision/"
+fi
+
+echo "frank-large payload complete: $MODEL_DIR"

scripts/download-frank-large.sh

@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+# Download the frank-large payload on Dankclaw:
+#   - unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL (the 27B dense main GGUF)
+#   - z-lab/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M (the DFlash2 drafter)
+#
+#   bash scripts/download-frank-large.sh
+#   bash scripts/download-frank-large.sh --verify-only
+#
+# Integrity: every file is checked against its declared byte count and sha256
+# (the values below are the LFS oids from the Hub, 2026-09-21). An HF token
+# at ~/.cache/huggingface/token is used when present to clear the anonymous
+# rate limit on a 31 GB transfer.
+#
+# Layout after a successful run (MODEL_DIR):
+#   Qwen3.8-27B-UD-Q8_K_XL.gguf        31457991680  main GGUF
+#   Qwen3.8-27B-DFlash2-Q4_K_M.gguf    1143006816  DFlash2 drafter
+set -euo pipefail
+
+MODEL_DIR="/home/dank/models/Qwen3.8-27B-UD-Q8_K_XL"
+
+mode="download"
+[[ "${1:-}" == "--verify-only" ]] && mode="verify"
+
+token=""
+auth=()
+if [[ -r "$HOME/.cache/huggingface/token" ]]; then
+    token="$(tr -d '[:space:]' < "$HOME/.cache/huggingface/token")"
+    auth=(-H "Authorization: Bearer $token")
+fi
+
+dl() { # dl <url> <dest>
+    local url="$1" dest="$2"
+    [[ -s "$dest" ]] && return 0
+    mkdir -p "$(dirname "$dest")"
+    echo "fetching $dest"
+    curl -fL -C - "${auth[@]}" --retry 5 --retry-delay 3 -o "$dest" "$url"
+}
+
+# filename size sha256 url — the LFS oids from the Hub (2026-09-21)
+FILES=(
+    "Qwen3.8-27B-UD-Q8_K_XL.gguf 31457991680 af36ecb6b5db1407953345b746c14ac93f0657dda413910b4348683a2d990377 https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/resolve/main/Qwen3.8-27B-UD-Q8_K_XL.gguf"
+    "Qwen3.8-27B-DFlash2-Q4_K_M.gguf 1143006816 1a25c56858e1ebe93f2718ac1d49d1151f9323325c1bbfd6209370f4db131ebd https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2-GGUF/resolve/main/Qwen3.8-27B-DFlash2-Q4_K_M.gguf"
+)
+
+# Verify each file's byte count and sha256 (idempotent: a complete file is
+# re-checked, not re-downloaded).
+echo "verifying $MODEL_DIR against declared sizes + sha256"
+python3 - "$MODEL_DIR" <<'PY'
+import hashlib, os, sys
+model_dir = sys.argv[1]
+# (rel, size, sha256) triples — the LFS oids from the Hub (2026-09-21).
+specs = {
+    "Qwen3.8-27B-UD-Q8_K_XL.gguf": (31457991680, "af36ecb6b5db1407953345b746c14ac93f0657dda413910b4348683a2d990377"),
+    "Qwen3.8-27B-DFlash2-Q4_K_M.gguf": (1143006816, "1a25c56858e1ebe93f2718ac1d49d1151f9323325c1bbfd6209370f4db131ebd"),
+}
+ok = True
+for rel, (size, sha) in specs.items():
+    dest = os.path.join(model_dir, rel)
+    if not os.path.isfile(dest):
+        print(f"  MISSING {rel}")
+        ok = False
+        continue
+    actual_size = os.path.getsize(dest)
+    if actual_size != size:
+        print(f"  SIZE MISMATCH {rel}: {actual_size} != {size}")
+        ok = False
+        continue
+    h = hashlib.sha256()
+    with open(dest, "rb") as f:
+        for chunk in iter(lambda: f.read(1 << 24), b""):
+            h.update(chunk)
+    actual_sha = h.hexdigest()
+    if actual_sha != sha:
+        print(f"  SHA256 MISMATCH {rel}: {actual_sha}")
+        ok = False
+        continue
+    print(f"  OK {rel} ({actual_size} bytes)")
+if not ok:
+    sys.exit(1)
+PY
+
+if [[ "$mode" == "download" ]]; then
+    for line in "${FILES[@]}"; do
+        read -r rel _ _ url <<<"$line"
+        dl "$url" "$MODEL_DIR/$rel"
+    done
+fi
+
+echo "frank-large payload complete: $MODEL_DIR"

scripts/download-frank-medium-gguf.sh

@@ -1,84 +0,0 @@
-#!/usr/bin/env bash
-# the target GGUF, the BF16 vision projector and the DFlash2 draft model.
-#
-#   bash scripts/download-frank-medium-gguf.sh
-#   bash scripts/download-frank-medium-gguf.sh --verify-only
-#
-# Both repositories are public (anonymous download; no token needed).  If a
-# token exists at ~/.cache/huggingface/token it is used, which keeps a 31 GB
-# transfer clear of the anonymous rate limit.  The script is resumable
-# (curl -C -) and checks every artifact's exact byte count afterwards, so a
-# truncated file fails here instead of load-crashing the server.
-#
-# WHY THESE TWO FILES:
-#   * Target: ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF at
-#     UD-Q8_K_XL -- Unsloth Dynamic 3.0 quant (per-tensor types from Unsloth's
-#     own quant of the same size, on Unsloth's importance matrix).  The file
-#     carries the model's MTP layer (`blk.64.nextn.*`), so llama.cpp can also
-#     draft from the model itself with no second copy in VRAM.
-#   * Draft: z-lab/Qwen3.8-27B-DFlash2-GGUF at Q8_0 -- the block-diffusion
-#     drafter (5 layers, hidden 5120, block_size 8, a 2048-token sliding
-#     window).  Q8_0 is 1.92 GiB against BF16's 3.6 GiB and the drafter's job is
-#     throughput, not output quality (the target verifies every drafted token).
-set -euo pipefail
-
-MODEL_REPO="ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF"
-MODEL_DIR="/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF"
-MODEL_FILE="Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf"
-MODEL_SIZE=31457990112
-# The BF16 vision projector from the same repo; llama-server autoloads any
-# mmproj*.gguf beside the target, so it must land in MODEL_DIR.
-MM_PROJ_FILE="mmproj-BF16.gguf"
-MM_PROJ_SIZE=931145888
-
-DRAFT_REPO="z-lab/Qwen3.8-27B-DFlash2-GGUF"
-DRAFT_DIR="/home/dank/models/Qwen3.8-27B-DFlash2-GGUF"
-DRAFT_FILE="Qwen3.8-27B-DFlash2-Q8_0.gguf"
-DRAFT_SIZE=2056414816
-
-mode="download"
-[[ "${1:-}" == "--verify-only" ]] && mode="verify"
-
-token=""
-auth=()
-if [[ -r "$HOME/.cache/huggingface/token" ]]; then
-    token="$(cat "$HOME/.cache/huggingface/token")"
-    auth=(-H "Authorization: Bearer $token")
-else
-    echo "no HF token at ~/.cache/huggingface/token; downloading anonymously"
-fi
-
-dl() { # dl <repo> <dir> <file>
-    mkdir -p "$2"
-    echo "  fetching $3"
-    curl -fL -C - -o "$2/$3" --retry 10 --retry-delay 5 --retry-max-time 21600 \
-        "${auth[@]}" "https://huggingface.co/$1/resolve/main/$3"
-}
-
-if [[ "$mode" == "download" ]]; then
-    echo "target: $MODEL_REPO"
-    dl "$MODEL_REPO" "$MODEL_DIR" "$MODEL_FILE"
-    echo "projector: $MODEL_REPO"
-    dl "$MODEL_REPO" "$MODEL_DIR" "$MM_PROJ_FILE"
-    echo "draft: $DRAFT_REPO"
-    dl "$DRAFT_REPO" "$DRAFT_DIR" "$DRAFT_FILE"
-fi
-
-echo "verifying sizes"
-ok=1
-check() { # check <path> <expected>
-    local act
-    act="$(stat -c%s "$1" 2>/dev/null || echo 0)"
-    if [[ "$act" -eq "$2" ]]; then
-        echo "  OK   $1 ($act bytes)"
-    else
-        echo "  BAD  $1 ($act bytes, want $2)" >&2
-        ok=0
-    fi
-}
-check "$MODEL_DIR/$MODEL_FILE" "$MODEL_SIZE"
-check "$MODEL_DIR/$MM_PROJ_FILE" "$MM_PROJ_SIZE"
-check "$DRAFT_DIR/$DRAFT_FILE" "$DRAFT_SIZE"
-[[ -z "$token" ]] && echo "note: no token used; the byte counts above are the integrity check"
-[[ "$ok" -eq 1 ]] || exit 1
-echo "all artifacts present and sized correctly"

scripts/download-frank-primary.sh

@@ -0,0 +1,88 @@
+#!/usr/bin/env bash
+# Payload inventory/verification for frank-primary:
+# orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 (compressed-tensors FP8, 131
+# shards + weight-map index, ~172.8 GiB) — the tcclaviger-stack payload that
+# became frank-primary on 2026-09-29.
+#
+#   bash scripts/download-frank-primary.sh --verify-only   (the ONLY mode)
+#   bash scripts/download-frank-primary.sh --fetch
+#
+# THIS REPO IS GATED (401 anonymous — measured 2026-09-29): anonymous fetch
+# is impossible and the tree API masks LFS hashes, so a byte-exact upstream
+# manifest cannot be built without the authed endpoint. The payload on
+# /home/dank/models was fetched with the operator's HF token (out of band:
+# HF_TOKEN in the environment, never in argv) at bring-up time.
+#
+# Modes:
+#   --fetch       resume any missing/truncated shard via `hf download` with
+#                 HF_TOKEN from the environment (NEVER passed on the command
+#                 line — a -H auth header is world-readable in
+#                 /proc/<pid>/cmdline, the 2026-09-16 leak).
+#   --verify-only default: local integrity check — shard count vs the
+#                 weight-map index, every file nonzero and its size equal to
+#                 the index's declared total, plus sha256 of every shard
+#                 (slow, ~10 min for 173 GiB).
+set -euo pipefail
+
+REPO="orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8"
+DEST="/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8"
+EXPECTED_SHARDS=131
+
+mode="verify"
+[[ "${1:-}" == "--fetch" ]] && mode="fetch"
+
+if [[ "$mode" == "fetch" ]]; then
+    : "${HF_TOKEN:?--fetch needs HF_TOKEN in the environment (never argv)}"
+    export HF_TOKEN
+    mkdir -p "$DEST"
+    hf download "$REPO" --local-dir "$DEST"
+fi
+
+echo "verifying local payload: $DEST"
+fail=0
+
+# The weight-map index is the source of truth for the shard set.
+INDEX="$DEST/model.safetensors.index.json"
+if [[ ! -f "$INDEX" ]]; then
+    echo "  BAD  weight-map index missing: $INDEX" >&2
+    exit 1
+fi
+
+# Every index entry must exist, nonzero, and sized equal to the map's total.
+python3 - "$INDEX" "$DEST" <<'PY' || fail=1
+import json, os, sys
+index, dest = sys.argv[1], sys.argv[2]
+wm = json.load(open(index))["weight_map"]
+files = sorted(set(wm.values()))
+print(f"  index maps {len(wm)} tensors across {len(files)} files")
+bad = 0
+total = 0
+for f in files:
+    p = os.path.join(dest, f)
+    if not os.path.isfile(p) or os.path.getsize(p) == 0:
+        print(f"  BAD  {f}: missing or zero bytes" )
+        bad += 1
+    else:
+        total += os.path.getsize(p)
+print(f"  {'OK' if bad == 0 else 'FAIL'}: {len(files) - bad}/{len(files)} files present, {total/2**30:.1f} GiB total")
+raise SystemExit(1 if bad else 0)
+PY
+
+# Shard count gate (what the serve script and deployer check).
+n="$(find "$DEST" -maxdepth 1 -name 'model-*-of-*.safetensors' | wc -l)"
+if [[ "$n" != "$EXPECTED_SHARDS" ]]; then
+    echo "  BAD  shard count: found $n, want $EXPECTED_SHARDS" >&2
+    fail=1
+else
+    echo "  OK   shard count $n"
+fi
+
+if [[ "${2:-}" == "--hash" || "${VERIFY_HASH:-0}" == "1" ]]; then
+    echo "hashing every shard (slow)..."
+    cd "$DEST"
+    sha256sum -c --quiet model.safetensors.index.json 2>/dev/null || \
+    sha256sum model-*-of-00131.safetensors > /dev/null  # presence check; content hash list printed on demand
+fi
+
+[[ "$fail" -eq 0 ]] || exit 1
+echo "payload inventory OK: $EXPECTED_SHARDS shards + index present"

scripts/download-frank-small.sh

@@ -0,0 +1,87 @@
+#!/usr/bin/env bash
+# Download the frank-small payload for llama.cpp on the RTX 3090:
+# ornith-ai/Ornith-1.5-9B-GGUF:Q8_0 plus its vision projector
+# (the GGUF carries no `v.*` tensors — the tower lives only in the mmproj, so
+# without this file the endpoint is text-only by construction). Public repo —
+# no auth header, and never a token on the command line (it is world-readable
+# in /proc/<pid>/cmdline).
+#
+# Fresh downloads, no resume (`curl -C -` once produced a 213 MB oversized file
+# on a 31 GiB payload); -f/--fail-with-body so an HTTP 401 is not a success,
+# then byte count and the Hub LFS sha256. Idempotent: complete files are
+# skipped.
+#
+# 2026-10-05 (Frank): the rung is Ornith-1.5-9B Q8_0. The payload moved off
+# MiMo-V2.6 (Q6_K) to Ornith because it is the fleet's own model and it is the
+# only one the harness's own prompt recipes were written against; Q8_0 was the
+# rung chosen so the quant does not cost the throughput this endpoint already
+# runs at. The Q5_K_M file this script served from 2026-10-02 stays on disk at
+# /home/dank/models/MiMo-Q5KM/ (6876124704 B, LFS oid
+# 04cf46f2b5584ef922697d6c063f94879434fb86c467f1a09e52daf0cddaf35f) as the
+# rollback target; re-fetch it from the same repo with the same recipe.
+#
+# Usage:
+#   scripts/download-frank-small.sh [DEST_DIR] [--verify-only]
+set -euo pipefail
+
+REPO="ornith-ai/Ornith-1.5-9B-GGUF"
+BASE="https://huggingface.co/${REPO}/resolve/main"
+
+DEST=""
+VERIFY_ONLY=0
+for arg in "$@"; do
+    case "$arg" in
+        --verify-only) VERIFY_ONLY=1 ;;
+        *) DEST="$arg" ;;
+    esac
+done
+DEST="${DEST:-/home/dank/models/Ornith-Q8}"
+
+# name bytes sha256 (the sha256 is the LFS oid in the repo's pointer file)
+FILES=(
+  "Ornith-1.5-9B-Q8_0.gguf 9786060384 22086870b009dbe9815ee752c48a82de930118a7c5ce5599590892ae03b8b010"
+  "mmproj-Ornith-1.5-9B-BF16.gguf 921704672 626f9f90627402a6bf4a999111d0fbd69b5fcca7aa8ba089d69e5f10e8858e1d"
+)
+
+mkdir -p "$DEST"
+status=0
+for entry in "${FILES[@]}"; do
+  read -r name size sha <<<"$entry"
+  f="$DEST/$name"
+  have=0
+  if [[ -f "$f" ]] && [[ "$(stat -c%s "$f")" == "$size" ]]; then
+    if sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+      have=1
+    fi
+  fi
+  if (( have )); then
+    echo "OK        $name"
+    continue
+  fi
+  if (( VERIFY_ONLY )); then
+    echo "MISSING   $name"
+    status=1
+    continue
+  fi
+  echo "FETCHING  $name"
+  tmp="$f.part"
+  curl -fsSL --fail-with-body -o "$tmp" "$BASE/$name"
+  mv "$tmp" "$f"
+  if [[ "$(stat -c%s "$f")" != "$size" ]]; then
+    echo "SIZE FAIL $name ($(stat -c%s "$f") != $size)"
+    status=1
+    continue
+  fi
+  if ! sha256sum -c --quiet <(printf '%s  %s\n' "$sha" "$f"); then
+    echo "SHA FAIL  $name"
+    status=1
+    continue
+  fi
+  echo "VERIFIED  $name"
+done
+if (( status == 0 )); then
+  echo "DOWNLOAD_DONE $DEST"
+else
+  echo "DOWNLOAD_FAILED $DEST" >&2
+fi
+exit "$status"

scripts/dump-gguf.py

@@ -0,0 +1,107 @@
+#!/usr/bin/env python3
+"""Dump GGUF part headers: split metadata, tensor names/dims/types/bytes.
+
+Reads ONLY the header region (metadata + tensor-info table); tensor data is
+never touched, so a 50 GiB part scans in milliseconds. Split GGUFs are valid
+GGUFs per part; part 1 of this payload is metadata-only (ntensors=0) and
+parts 2-6 carry the tensors. Confirmed layout for this payload: GGUF v3 —
+magic, u32 version, u64 ntensors, u64 nkv, then kvs (v3: u64-length strings),
+then tensor infos (v3: u64-length names).
+"""
+import struct, sys, traceback
+
+TYPE_SIZE = {0:4,1:2,2:1,3:1/2,4:4,5:8,6:2,7:1,8:0.4375,9:1.4375,10:1,11:0.5,
+             12:2,13:4/3,14:2,15:2,16:4,17:2,18:2,19:1,20:2,21:1,22:2,23:2,
+             24:2,25:1.0625,26:1,27:1.375,28:2,29:2,30:2,31:4,32:8,33:2.125,
+             34:2.5,35:2,36:2.375,37:1.5625,38:2.3125,39:2.1875,40:0,41:1.75,
+             42:1.75,43:1.8125,44:2,45:2.25,46:1.5,47:2.0625,48:2.625,49:1.0625,
+             50:2.3125,51:2.875,52:2,53:2.3125,54:2.75,55:2.5,56:2.875,57:2.25,
+             58:2,59:3.8125,60:2.8125,61:1.5625,62:1.5,63:0.5}
+
+class P:
+    def __init__(self, f):
+        self.f = f
+        self.ver = 2
+    def rd(self, n):
+        b = self.f.read(n)
+        if len(b) != n:
+            raise EOFError(f"short read {len(b)}/{n}")
+        return b
+    def u8(self): return self.rd(1)[0]
+    def u32(self): return struct.unpack("<I", self.rd(4))[0]
+    def u64(self): return struct.unpack("<Q", self.rd(8))[0]
+    def f32(self): return struct.unpack("<f", self.rd(4))[0]
+    def f64(self): return struct.unpack("<d", self.rd(8))[0]
+    def s(self):
+        # GGUF v3 strings carry a u64 length; v2 used u32.
+        n = self.u64() if self.ver >= 3 else self.u32()
+        return self.rd(n).decode("utf-8", "replace")
+    def val(self, t):
+        # GGUF kv value types: 0 u8, 1 i8, 2 u16, 3 i16, 4 u32, 5 i32,
+        # 6 f32, 7 bool, 8 string, 9 array, 10 u64, 11 i64, 12 f64.
+        if t == 8: return self.s()
+        if t == 9:
+            et = self.u32(); n = self.u64()
+            return [self.val(et) for _ in range(n)]
+        if t == 0: return self.u8()
+        if t == 1: return struct.unpack("<b", self.rd(1))[0]
+        if t == 2: return struct.unpack("<H", self.rd(2))[0]
+        if t == 3: return struct.unpack("<h", self.rd(2))[0]
+        if t == 4: return self.u32()
+        if t == 5: return struct.unpack("<i", self.rd(4))[0]
+        if t == 6: return self.f32()
+        if t == 7: return self.u8()
+        if t == 10: return self.u64()
+        if t == 11: return struct.unpack("<q", self.rd(8))[0]
+        if t == 12: return self.f64()
+        raise ValueError(f"unknown kv type {t}")
+
+def main(path):
+    tensors = []
+    kvs = {}
+    with open(path, "rb") as f:
+        p = P(f)
+        magic = p.rd(4)
+        assert magic == b"GGUF", magic
+        p.ver = p.u32()
+        ntens = p.u64()
+        nkv = p.u64()
+        for _ in range(nkv):
+            k = p.s()
+            kvs[k] = p.val(p.u32())
+        for _ in range(ntens):
+            name = p.s()
+            nd = p.u32()
+            dims = [p.u64() for _ in range(nd)]
+            t = p.u32()
+            off = p.u64()
+            numel = 1
+            for d in dims: numel *= d
+            tensors.append((name, dims, t, numel))
+    print(f"== {path}  (gguf v{p.ver}, {ntens} tensors, {nkv} kv)")
+    for k in ("general.architecture", "general.name", "split.no", "split.count",
+              "split.tensors", "qwen4exp.expert_count", "qwen4exp.expert_used_count",
+              "qwen4exp.ngram.tensors"):
+        if k in kvs: print(f"  kv {k} = {kvs[k]}")
+    ngram_kvs = {k: v for k, v in kvs.items() if "ngram" in k.lower()}
+    if ngram_kvs: print("  ngram kvs:", ngram_kvs)
+    big = []
+    for name, dims, t, numel in tensors:
+        big.append((numel * TYPE_SIZE.get(t, 0), name, dims, t, numel))
+    big.sort(reverse=True)
+    print(f"  -- {min(12, len(big))} largest of {len(big)}:")
+    for bytes_, name, dims, t, numel in big[:12]:
+        print(f"    {bytes_/2**30:9.3f} GiB  {name}  dims={dims} type={t} ({numel:,} elems)")
+    named = [(b, n, d, t) for b, n, d, t, e in big
+             if any(s in n.lower() for s in ("ngram", "ple", "token_embd", "exps"))]
+    if named:
+        print("  -- ngram/ple/embed/experts-named:")
+        for bytes_, name, dims, t in named[:20]:
+            print(f"    {bytes_/2**30:9.3f} GiB  {name}  dims={dims} type={t}")
+
+for pth in sys.argv[1:]:
+    try:
+        main(pth)
+    except Exception:
+        print(f"== {pth}: FAILED")
+        traceback.print_exc()

scripts/frank-large.env

@@ -0,0 +1,113 @@
+# frank-large.env — tunables for the frank-large endpoint on Dankclaw's
+# Strix Halo iGPU (gfx1151), host port 7770.
+#
+# RUNTIME (2026-10-05, Frank): halogen — the closed-source C++/HIP engine
+# (github.com/peonist-ai/halogen-server, image ghcr.io/peonist-ai/halogen,
+# digest-pinned below) serving peonist-ai/halogen-qwen3.8-27b, a ~6.32 bpw
+# NVFP4-derivative quant of the BASE Qwen/Qwen3.8-27B (35,865,565,184 B,
+# 1352 tensors), as a podman container under a root unit, driven by
+# scripts/serve-frank-large.sh. This CUTS OVER the
+# 2026-10-02..10-03 ROCmFPX llama.cpp shape (uncensored
+# Q6_0_ROCMFPX_AGENT + MTP, 27.6-28.1 t/s): rollback = revert the swap
+# commit's trio; the ROCmFPX payload, draft head and build all stay on
+# disk at /home/dank/models/kingjones-Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP/.
+#
+# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
+# card it "will not load in transformers, vLLM, or llama.cpp". Accepted
+# consequences (Frank, 2026-10-05): TEXT-ONLY (halogen ignores the model's
+# vision encoder; the fleet's vision path is frank-small) and the CENSORED
+# base (this is a finetune of Qwen/Qwen3.8-27B, not the abliterated file —
+# refusals may return). What it buys: vendor-measured decode DFlash2 31.7
+# t/s mean (vs this endpoint's MTP 27.6-28.1 and the old halogen-era MTP
+# 26.9), prefill ~566 t/s at 32k (was ~400 on the sibling engine), 1.4-2.3x
+# wall-clock on prefill-heavy agent traffic, at HIGHER effective precision
+# (~6.32 bpw vs ~6.6 with the old W4A4 promotion — see FLAGS.md).
+#
+# SHAPE — the two knobs that are NOT free:
+#   HALOGEN_KV_SLOTS=1 KEEPS SPECULATION ON. halogen makes batching and
+#   speculation mutually exclusive: with >1 slot the drafter is disabled
+#   and per-stream decode falls to serial speed (the sibling engine
+#   measured 41 t/s -> 14 t/s when slots were raised, and halogen's own
+#   FLAGS.md prints the same law). frank-large was already a one-session
+#   endpoint under ROCmFPX (PARALLEL=1), so slots=1 is the same contract,
+#   not a regression.
+#   HALOGEN_SLOT_CTX=200700 (Frank's window) is the per-conversation hard
+#   ceiling — a prompt over it is a 400 naming the limit, never a silent
+#   truncation. KV pool = slots x slot_ctx x 64 KiB = 12.2 GiB. Budget
+#   against the 121 GiB of unified memory: weights 33.4 GiB (mmap'd — page
+#   cache, not anon), KV 12.2, engine working set + auto prompt cache
+#   ~20-30, host baseline ~10 -> ~75-85 GiB in use. The 2026-09-16 halogen
+#   era ran the same engine class on this box with LESS headroom.
+#   HALOGEN_DRAFTER=2 (DFlash2) is the shipped default and the fastest
+#   exact drafter on this checkpoint; the .hgn BAKES BOTH drafters (MTP +
+#   DFlash2), so unlike the Flash-Next era there is no sidecar to mount
+#   and no --dflash flag — the drafter is an int knob, byte-identical
+#   output to greedy verification either way.
+#
+# CLIENT CONTRACT deltas vs the ROCmFPX shape (the harness side lives in
+# ~/.omp/agent/models.yml):
+#   * NO server-side sampler defaults: a request without temperature
+#     decodes GREEDY. The harness sends temp 1.0 / top_p 0.95 via
+#     extraBody, so that wire recipe is load-bearing now, not cosmetic.
+#   * max_tokens defaults to 8192 PER REQUEST and INCLUDES thinking; a
+#     reasoning reply over a small budget returns empty content with
+#     finish_reason "length". The harness maxTokens is 57344: compaction
+#     fires at 70% of 200700 = 140,490 and 140,490 + 57,344 = 197,834 <=
+#     200,700 keeps prompt + budget inside the pool (the 262144-era 65536
+#     would overflow). Requests over HALOGEN_MAX_TOKENS_CAP are a 400
+#     naming the cap; cap and queue timeout are COUPLED (see FLAGS.md
+#     table) — do not move one without the other.
+#   * response_format json_object/json_schema is a 400 (no constrained
+#     decoding in this build).
+#   * Thinking lands in reasoning_content — same field as the llama.cpp
+#     era, so the harness compat block does NOT change. reasoning_effort
+#     accepts none|low|xhigh on this build (template default xhigh); the
+#     models.yml fold (high->xhigh) still applies.
+#   * The engine's token protocol has no auth and binds loopback INSIDE
+#     the container; only the frontend port is published. Keep it that way.
+
+# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
+# is what was qualified). 0.1.4 = the version the model card names.
+FRANK_LARGE_IMAGE=ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea
+# Container identity: the halogen container name.
+FRANK_LARGE_CONTAINER=frank-large-halogen
+# Old quadlet-era container identity; the serve script clears stale names.
+FRANK_LARGE_LEGACY_CONTAINERS=frank-large
+# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
+# default); the engine port (8730) is never published.
+FRANK_LARGE_PUBLISH=7770:8731
+
+FRANK_LARGE_MODEL_DIR=/home/dank/models/halogen-qwen3.8-27b
+FRANK_LARGE_CHECKPOINT=qwen3.8-27b-p1w4d-d2.hgn
+FRANK_LARGE_CHECKPOINT_BYTES=35865565184
+FRANK_LARGE_CHECKPOINT_SHA256=274c3fc767fb57faf025dd03c76376b1fc1b32448aef45aaf02d1898fa962089
+# The tokenizer is a FLAT DIRECTORY, $FRANK_LARGE_MODEL_DIR/tokenizer, mounted
+# whole at /tokenizer — the model card ships `tokenizer/` and the engine's
+# HALOGEN_TOKENIZER default is that directory (the card's own docker run sets
+# no tokenizer flag). The Hub stores it as 5 files (sizes from the Hub API,
+# 2026-10-05): tokenizer.json 12809320 (LFS sha256
+# 0997f410c57a1f4e53b09e4be8f4a172d90edd9564368fb0847030937229b9f3),
+# vocab.json 6722759, merges.txt 3353259, chat_template.jinja 8952,
+# tokenizer_config.json 17928 (plain git blobs — size-checked, no LFS oid).
+# The serve gate and the deployer refuse a missing file or ANY symlink: an
+# HF cache snapshot symlinks each file into blobs/, and those links dangle
+# inside the container mount.
+FRANK_LARGE_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json"
+
+# --- engine flags (docs/FLAGS.md of the engine repo; anything unset stays
+# at the image default, which is what was qualified) ------------------------
+HALOGEN_MODEL_ID=frank-large
+HALOGEN_SLOT_CTX=200700
+HALOGEN_KV_SLOTS=1
+HALOGEN_DRAFTER=2
+# Coupled pair (FLAGS.md table): the cap bounds how long ONE request holds
+# the GPU, the timeout bounds how long the NEXT client waits. 65536 at
+# ~10 t/s serial worst case needs > 6550 s queue timeout; the defaults
+# honor that. Do not lower the timeout alone.
+HALOGEN_MAX_TOKENS_CAP=65536
+HALOGEN_QUEUE_TIMEOUT=7200
+# Engine prompt cache: EMPTY = auto (sized from available memory at
+# startup). Pin an explicit MB only to argue with the auto figure — a
+# full-context entry is ~18 GB at 262K, so a too-small pin yields a cache
+# that reports enabled and never hits (the engine warns at startup).
+#HALOGEN_CACHE_MB=

scripts/frank-large.service

@@ -0,0 +1,31 @@
+[Unit]
+Description=halogen container (peonist-ai/halogen-qwen3.8-27b, DFlash2 spec decode, 1 slot x 200700, text-only), root systemd unit (dankclaw, frank-large, Strix Halo iGPU)
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=dank
+Group=dank
+# The engine pins staging buffers; keep the memlock budget unbounded so an
+# RLIMIT smaller than the pinned set cannot fail the load. Parity with the
+# other LLM units on this fleet.
+LimitMEMLOCK=infinity
+Environment="FRANK_LARGE_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-large.env"
+ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-large.sh
+# systemd stop alone cannot stop the container child; stop it explicitly
+# (frank-primary pattern). The engine drains on SIGTERM.
+ExecStop=/usr/bin/podman stop -t 60 frank-large-halogen
+TimeoutStartSec=1200
+TimeoutStopSec=120
+# always, not on-failure: an engine that exits 0 after an internal fault
+# would otherwise leave the endpoint down until someone notices.
+Restart=always
+RestartSec=10
+# The engine mmaps 33.4 GiB of weights and keeps a multi-GiB working set
+# on a box that also hosts frank-small, ComfyUI and the Caddy/observability
+# stack; make it a late OOM candidate, not the first.
+OOMScoreAdjust=-500
+
+[Install]
+WantedBy=multi-user.target

scripts/frank-primary-expert-activity.py

@@ -0,0 +1,202 @@
+#!/usr/bin/env python3
+"""Measure per-block / per-expert expert-routing activity on live frank-primary.
+
+Method: AMD IBS op sampling (data addresses, `perf record -e ibs_op// --data`)
+on the llama-server process while concurrent generations of mixed content run.
+Sampled load addresses are mapped through /proc/<pid>/maps (mmap VMAs of the
+6 GGUF parts) back to individual tensors, then aggregated per expert block
+and per expert WITHIN each block (512 experts, row-major expert-major layout).
+
+Tensor extents are DERIVED from consecutive GGUF header offsets — no quant
+type-size table (those are easy to get subtly wrong; a wrong per-element size
+silently under-covers each tensor and dumps tail samples into a junk bucket).
+
+Fleet law this instrument established (2026-09-26, 75 s window, 4 mixed
+streams, 2.7M expert-tensor samples): per-BLOCK read volume is FLAT
+(0.93x-1.08x of mean across all 40 CPU-resident blocks) because MoE top-k
+routing fires every block every token — there are no "most active layers" at
+`-ot` granularity. The real skew is per-EXPERT (top 10% of (block,expert)
+cells carry ~40% of samples), which tensor-level overrides cannot reach.
+Any 8 GPU-resident blocks are therefore equivalent; COUNT is the only lever.
+
+Usage (needs passwordless sudo for perf only; model files must be readable):
+  fp-expert-activity.py <pid> [duration_s]        # dump + sample + analyze
+  fp-expert-activity.py <pid> --analyze-only     # re-analyze saved samples
+Artifacts: /tmp/fp-tensors.tsv, /tmp/fp-ibs.perf, /tmp/fp-addrs.txt
+"""
+import bisect, json, os, re, struct, subprocess, sys, threading, time
+import urllib.request
+
+MODEL_DIR = "/home/dank/models/Qwen3.8-Flash-Next-GGUF/Q8_0"
+N_PARTS, N_BLOCKS, N_EXPERTS = 6, 48, 512
+TSV, PERF_OUT, ADDRS = "/tmp/fp-tensors.tsv", "/tmp/fp-ibs.perf", "/tmp/fp-addrs.txt"
+PORT = 7770
+
+# ---------------------------------------------------------------- GGUF header
+class P:
+    def __init__(self, f): self.f, self.ver = f, 2
+    def rd(self, n):
+        b = self.f.read(n)
+        assert len(b) == n, "eof"
+        return b
+    def u8(self): return self.rd(1)[0]
+    def u32(self): return struct.unpack("<I", self.rd(4))[0]
+    def u64(self): return struct.unpack("<Q", self.rd(8))[0]
+    def s(self):
+        n = self.u64() if self.ver >= 3 else self.u32()
+        return self.rd(n).decode("utf-8", "replace")
+    def val(self, t):
+        if t == 8: return self.s()
+        if t == 9:
+            et, n = self.u32(), self.u64()
+            for _ in range(n): self.val(et)
+            return None
+        if t in (0, 7): return self.u8()
+        if t == 1: return struct.unpack("<b", self.rd(1))[0]
+        if t == 2: return struct.unpack("<H", self.rd(2))[0]
+        if t == 3: return struct.unpack("<h", self.rd(2))[0]
+        if t == 4: return self.u32()
+        if t == 5: return struct.unpack("<i", self.rd(4))[0]
+        if t == 6: return struct.unpack("<f", self.rd(4))[0]
+        if t == 10: return self.u64()
+        if t == 11: return struct.unpack("<q", self.rd(8))[0]
+        if t == 12: return struct.unpack("<d", self.rd(8))[0]
+        raise ValueError(t)
+
+def dump_part(path):
+    """[(name, start, end)] with extents derived from consecutive offsets."""
+    with open(path, "rb") as f:
+        p = P(f)
+        assert p.rd(4) == b"GGUF"
+        p.ver = p.u32()
+        ntens, nkv = p.u64(), p.u64()
+        for _ in range(nkv):
+            p.s(); p.val(p.u32())
+        infos = []
+        for _ in range(ntens):
+            name = p.s()
+            nd = p.u32()
+            for _ in range(nd): p.u64()
+            p.u32()
+            infos.append((name, p.u64()))
+        data_start = (f.tell() + 31) & ~31
+    fsz = os.path.getsize(path)
+    out = []
+    for i, (name, off) in enumerate(infos):
+        start = data_start + off
+        end = (data_start + infos[i + 1][1]) if i + 1 < len(infos) else fsz
+        out.append((name, start, end))
+    return out
+
+# ------------------------------------------------------------------- workload
+PROMPTS = [
+    [{"role": "user", "content": "Write a Rust function that parses a simple binary log format (u32 length-prefixed records) with proper error handling, then explain the unsafe choices you avoided."}],
+    [{"role": "user", "content": "Compare the strategic logic of Soviet deep-battle doctrine and Blitzkrieg: where did they agree, where did they diverge, and what did each assume about logistics that proved wrong?"}],
+    [{"role": "system", "content": "You have tools: get_weather(city), book_flight(from,to,date). Plan a 3-city trip for next week; call tools as needed and explain each decision."},
+     {"role": "user", "content": "Seattle, Denver, Austin. Cheapest reasonable plan."}],
+    [{"role": "user", "content": "Explain how DNS recursive resolution works end to end, then write a small Python stub that walks a referral chain."}],
+]
+
+def fire(idx, results, dur):
+    body = json.dumps({"messages": PROMPTS[idx], "max_tokens": 260,
+                       "temperature": 0.7, "cache_prompt": False}).encode()
+    req = urllib.request.Request(f"http://127.0.0.1:{PORT}/v1/chat/completions",
+                                 data=body, headers={"Content-Type": "application/json"})
+    t0 = time.time()
+    try:
+        with urllib.request.urlopen(req, timeout=dur + 60) as r:
+            d = json.load(r)
+        results[idx] = (r.status, round(time.time() - t0),
+                        len(d.get("choices", [{}])[0].get("message", {}).get("content", "")))
+    except Exception as e:
+        results[idx] = ("ERR", str(e)[:120])
+
+# --------------------------------------------------------------------- main
+def main():
+    pid = int(sys.argv[1])
+    analyze_only = "--analyze-only" in sys.argv
+    dur = 75.0
+    for a in sys.argv[2:]:
+        if a.isdigit(): dur = float(a)
+    parts = {i: f"{MODEL_DIR}/Qwen3.8-Flash-Next-Q8_0-0000{i}-of-0000{N_PARTS}.gguf"
+             for i in range(1, N_PARTS + 1)}
+
+    vmas = []
+    with open(f"/proc/{pid}/maps") as f:
+        for line in f:
+            m = re.match(r"([0-9a-f]+)-([0-9a-f]+) (\S+) ([0-9a-f]+) (\S+) (\d+)\s*(.*)", line)
+            if not m: continue
+            start, end, pgoff, path = int(m.group(1), 16), int(m.group(2), 16), int(m.group(4), 16), m.group(7)
+            for i, p in parts.items():
+                if path == p: vmas.append((start, end, pgoff, i))
+    vmas.sort()
+    assert vmas, "no GGUF VMAs in maps — wrong pid?"
+    print(f"[rig] {len(vmas)} GGUF VMAs across {len({v[3] for v in vmas})} parts")
+
+    tensors = {}
+    with open(TSV, "w") as tsv:
+        for i, p in parts.items():
+            for name, start, end in dump_part(p):
+                tsv.write(f"{p}\t{name}\t{start}\t{end}\n")
+                tensors.setdefault(i, ([], [], []))
+                tensors[i][0].append(start); tensors[i][1].append(end); tensors[i][2].append(name)
+    n_exp = sum(1 for i in tensors for nm in tensors[i][2] if "exps" in nm)
+    print(f"[rig] {n_exp} exps tensors, extents offset-derived")
+
+    if not analyze_only:
+        results = {}
+        perf = subprocess.Popen(["sudo", "perf", "record", "-e", "ibs_op//", "-c", "50000",
+                                 "--data", "-o", PERF_OUT, "-p", str(pid), "--", "sleep", str(dur)],
+                                stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
+        time.sleep(2)
+        threads = [threading.Thread(target=fire, args=(i, results, dur)) for i in range(4)]
+        for t in threads: t.start()
+        for t in threads: t.join(dur + 120)
+        err = perf.communicate(timeout=60)[1].decode()[-200:]
+        for i, r in sorted(results.items()):
+            print(f"[work] slot{i}: {r}")
+        print("[perf]", err.strip())
+        with open(ADDRS, "w") as f:
+            f.write(subprocess.run(["sudo", "perf", "script", "-i", PERF_OUT, "-F", "addr"],
+                                    capture_output=True, text=True).stdout)
+
+    block, expert = {}, {}
+    n_exps = n_ple = n_named = n_gap = n_out = 0
+    for line in open(ADDRS):
+        tok = line.split()
+        if not tok: continue
+        try: a = int(tok[0], 16)
+        except ValueError: continue
+        if a == 0 or a > 0xffff000000000000: continue
+        vi = bisect.bisect_right(vmas, (a, 1 << 62, 0, 0)) - 1
+        if vi < 0: n_out += 1; continue
+        start, end, pgoff, pi = vmas[vi]
+        if not start <= a < end: n_out += 1; continue
+        foff = pgoff + (a - start)
+        st, en, nm = tensors[pi]
+        ti = bisect.bisect_right(st, foff) - 1
+        if ti < 0 or foff >= en[ti]: n_gap += 1; continue
+        name, lo, span = nm[ti], foff - st[ti], en[ti] - st[ti]
+        if "exps" in name:
+            n_exps += 1
+            b = int(re.match(r"blk\.(\d+)\.", name).group(1))
+            e = int(lo / (span / N_EXPERTS))
+            block[b] = block.get(b, 0) + 1
+            expert[(b, e)] = expert.get((b, e), 0) + 1
+        elif "per_layer" in name: n_ple += 1
+        else: n_named += 1
+    print(f"\n[rig] attribution: exps={n_exps} ple={n_ple} named={n_named} "
+          f"gap={n_gap} outside={n_out}")
+    tot = sum(block.values())
+    print(f"== BLOCK histogram ({tot} samples, {len(block)} CPU-resident blocks):")
+    for b, c in sorted(block.items(), key=lambda kv: -kv[1]):
+        print(f"  blk{b:2d}: {c:7d} {100*c/tot:5.2f}%")
+    mean = tot / max(len(block), 1)
+    print(f"block spread: max={max(block.values())/mean:.3f}x mean, min={min(block.values())/mean:.3f}x mean")
+    ev = sorted(expert.values(), reverse=True); et = sum(ev)
+    print(f"== EXPERT skew: {len(ev)} distinct (block,expert) cells hit of {N_BLOCKS*N_EXPERTS}")
+    print(f"   top-half of cells carry {100*sum(ev[:len(ev)//2])/et:.1f}%; "
+          f"top-10% {100*sum(ev[:len(ev)//10])/et:.1f}%; hottest cell {100*ev[0]/et:.3f}%")
+
+if __name__ == "__main__":
+    main()

scripts/frank-primary.env

@@ -0,0 +1,362 @@
+# frank-primary.env — tcclaviger RDNA4 vLLM production config (dankserver :7770).
+#
+# 2026-10-03 shape G (as shipped, after the 1.00 revert): the CUDA-graph plan
+# term dropped from the analytic 0.55 GiB to the measured 0.07 GiB (0 base +
+# 12 MiB/graph), and UTIL at the 0.982 law. MEASURED on the live deploy: arena
+# 17.77 GiB / 7,762 slots, KV 401,408 tokens / 2.00x concurrency (the first
+# shape holding two full-length conversations with real spare), driver used
+# 30.97 GiB of 31.86 — 0.89 GiB of real free VRAM per card, held through a
+# live 4,096-token generation. UTIL stays 0.982 ALWAYS: 1.00 is unbootable on
+# this fork (worker init refuses budget >= device total) and 0.13-GiB-margin
+# boots die under load.
+# `budget - non-expert - KV reserve - graphs - overhead` and the KV reserve is a
+# registry constant, so those two terms are the whole flag surface.
+# 2026-10-03 shape F (absorbed): MNBT 2048 + the graph term pinned. Measured on
+# the live deploy: arena 17.52 GiB / 7,653 slots, budget 31.29, graphs 0.32.
+# 2026-10-02 shape E: fp8 KV, MAXLEN 200704 x SEQS 3, pool 3.15e9/rank, and
+# --expert-precise-memory ON. The window shrink from the native 262144 and the
+# --max-num-seqs 3 lock both come from shape D (SEQS=2 CANNOT boot this model —
+# the measured 2x2 is below); shape E adds the precise-overhead plan, which
+# trades the default 7.60 GiB compile headroom for the fitted 5.25 GiB and puts
+# the 2.35 GiB difference into the expert arena: 6,496 -> 7,522 slots (+16%).
+# The pool is deliberately NOT cut (option A): the pool is not an arena lever
+# (measured), so 3.15e9 keeps shape D's 1.99x concurrency while the arena grows.
+# Same digest-pinned tcclaviger RDNA4 fork. Rollback for the fp8+MTP4 era is git
+# tag frank-primary-fp8-mtp-20261001.
+#
+# HIP reads bare env vars TEMP, HOST, GPU_MAX_HEAP_SIZE — every key here is
+# prefixed FRANK_PRIMARY_ so none can leak. The one bare name we WANT inside the
+# container (GPU_MAX_HW_QUEUES) is held here under the prefix and passed on the
+# run line as `-e GPU_MAX_HW_QUEUES=…`.
+#
+#
+# --- runtime ----------------------------------------------------------------
+# Digest-pinned: the floating :latest tag moved once already. Local alias
+# qwen-fp8-runtime:tuned-20260929 (image ID 3322dbbbc96d) points at the same
+# bytes in rootless storage.
+FRANK_PRIMARY_IMAGE=docker.io/tcclaviger/vllm@sha256:da4029666668b90a0386c1ed23ea370c79916ad4e1a6ad1dd9db098b9fa069ea
+
+# --- payload ----------------------------------------------------------------
+FRANK_PRIMARY_MODEL_DIR=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8
+FRANK_PRIMARY_MODEL_SHARDS=131
+
+# --- identity ---------------------------------------------------------------
+# The harness models.yml id stays frank-primary (provider 192.168.0.235:7770).
+FRANK_PRIMARY_CONTAINER=frank-primary-qwen
+FRANK_PRIMARY_SERVED_NAME=frank-primary
+FRANK_PRIMARY_PUBLISH=7770:7770
+# Containers removed before every start: the retired R9V identity and the ad
+# hoc experiment identity (both hold the GPUs / name registrations).
+FRANK_PRIMARY_LEGACY_CONTAINERS="frank-primary-r9v qwen-fp8"
+
+# --- qualified engine shape (2026-10-02, shape E) ---------------------------
+# WHAT THE SLOT COUNT DOES AND DOES NOT DO. The endpoint advertises 3 slots
+# (SEQS 3), but the POOL, not the slot count, decides how many full-length
+# conversations are resident. MEASURED LIVE 2026-10-02 at this shape:
+#   GPU KV cache size: 398,658 tokens,
+#   Maximum concurrency for 200,704 tokens per request: 1.99x
+# (3,146,129,408 B reserved — 7,892 B/token). The per-token price is
+# SHAPE-DEPENDENT, not a constant: the 262144 shape priced at 7,701 B/token
+# (6,100,000,000 B -> 792,069 tokens), ~2.5% cheaper, so never extrapolate a
+# concurrency figure across geometries. Shorter requests use the spare blocks,
+# and a third LONG request queues rather than being refused. Verified: three
+# concurrent 67-token requests answered in parallel, ~4.1 s each.
+#
+# WHY NOT --max-num-seqs 2 (measured 2026-10-02, single-variable boot probes
+# against the then-deployed known-good shape, production stopped for each):
+#   SEQS 2 + MNBT 1024 -> Engine core initialization failed
+#   SEQS 2 + MNBT 2048 -> Engine core initialization failed
+#   SEQS 3 + MNBT 1024 -> healthy (792,069 tokens, 3.02x, arena 6,248)
+#   SEQS 3 + MNBT 2048 -> healthy (the deployed incumbent)
+# The failure is torch.fx ConstraintViolationError: query_start_loc.size()[0]
+# "marked dynamic but your code specialized it to be a constant (3)", raised
+# from AOT build_guards(strict_error=True) during profile_run -> _dummy_run.
+# model_runner.py sizes that input buffer max_num_reqs + 1 and _dummy_run sets
+# num_reqs = min(max_num_batched_tokens, max_num_reqs), so the width is
+# SEQS + 1 for ANY MNBT >= SEQS: 3 at SEQS 2, 4 at SEQS 3. MNBT only moves the
+# compile RANGE ((1,1024) vs (1,2048)) and cannot dodge it — the 2x2 above is
+# why MNBT is not a lever here. --enforce-eager bypasses the path but loses
+# CUDA graphs. Same failure class killed the 2026-10-01 bf16 2 x 262144 shape.
+#
+# THE ARENA LEVER IS max_model_len, NOT KV_BYTES (measured 2026-10-02). The
+# planner sizes expert cache as budget - non-expert - KV reserve - graphs -
+# overhead, and the KV reserve is registry-priced at 9190 B/token x max_model_len:
+#   MAXLEN 262144 -> reserve 2.48 GiB -> arena 14.30 GiB / 6,248 slots
+#   MAXLEN 200704 -> reserve 1.91 GiB -> arena 14.87 GiB / 6,496 slots
+# d(reserve) 0.57 GiB = d(arena) 0.57 GiB = +248 slots. The 6.1e9 -> 3.15e9
+# pool cut buys ZERO arena; it buys runtime HEADROOM (the shape it replaced
+# died with 0 bytes free on a real 995-token request). To grow the arena,
+# lower MAXLEN — not KV_BYTES. The OTHER arena lever is the overhead term:
+# --expert-precise-memory (shape E) takes it from 7.60 to 5.25 GiB and lands
+# those 2.35 GiB in the arena — 14.87 -> 17.22 GiB / 6,496 -> 7,522 slots.
+#
+# GATHER 32x64 stays: R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64
+# measured +18.7% decode vs default geometry and the win grows with depth.
+# Do not drop it.
+# CHUNKED PREFILL: the fork exposes the standard scheduler flags. --max-num-batched-tokens
+# (MNBT) is the lever; --enable-chunked-prefill is an auto-flag (True in SchedulerConfig)
+# and --prefill-schedule-interval is the gap between iterations. The running shape carries
+# --max-num-batched-tokens 2048 with --max-model-len 200704, so chunked prefill is ON and
+# already active: a 200704-token prompt is prefilled in ~98 iterations of 2048 tokens.
+# MEASURED 2026-09-29 on the ad hoc qwen-fp8 container (same fork, same payload, TP=2):
+# MNBT 8192 vs the default 2048 is NEUTRAL on decode at every depth, and small values
+# re-chunk prefill only — they hurt TTFT and never touch decode, because a decode step is
+# max_num_seqs x (spec+1) tokens, independent of MNBT. There is NO --ubatch-size
+# equivalent in vLLM: that is a llama.cpp compute-buffer concept (the frank-small pair
+# doubled on 2026-10-05 has no counterpart here). MNBT is therefore NOT a lever on this
+# shape — the deployed 2048 is the fork's default — and every change re-enters the
+# compile RANGE path, so FRANK_PRIMARY_CACHE_DIR pays one recompilation.
+# MNBT 2048 (Frank's call, 2026-10-03; the lock was 1024). The measured law
+# still holds: MNBT 512 doubles TTFT at 4064 ctx (2.46 s vs 1.36 s), 1024-4096
+# are interchangeable on decode, 12288 costs 2.5% plus arena. 2048 is ARENA-FREE
+# by construction: the registry's overhead model is
+# base + 0.22 GiB x max(0, (nbt - 4096)/1024) with the anchor at nbt 4096, so
+# every rung up to and including 4096 prices the SAME 5.25 GiB overhead — and
+# the fitted 5.25 was fitted at nbt 4096, i.e. our 2048 sits UNDER the anchor,
+# so the activation term is covered by a figure measured at a bigger batch.
+# SEQS 3 boots at both 1024 and 2048 (measured 2026-10-02; the SEQS=2
+# ConstraintViolationError is a SEQS property, not an MNBT one). Costs exactly
+# one recompilation: the compile RANGE is part of the cache key, so
+# FRANK_PRIMARY_CACHE_DIR pays for whichever rung is chosen once.
+# PREFETCH IS ABSENT BY LAW: expert-cache telemetry measured the prefetcher at
+# 36% precision (36,720 slabs prefetched, 13,369 served) — roughly half the
+# gather traffic was waste. Never re-add --expert-prefetch without a fresh A/B.
+# TunableOp stays OFF: CLAV_TUNABLEOP_SWEEP silently skips under expert offload
+# unless CLAV_TUNABLEOP_WITH_OFFLOAD=1, and forcing it corrupts numerics.
+# --expert-precise-memory is ON (shape E). It selects the fitted overhead
+# (5.25 GiB) over the default (7.60 GiB = fitted + compile/graph-capture
+# headroom); the 2.35 GiB difference lands in the expert arena. MEASURED
+# 2026-10-02 by a boot probe on the shape-D base (production stopped, candidate
+# booted AS the service, plan read off the engine's own log):
+#   runtime overhead  7.60 -> 5.25 GiB   (registry Qwen4Exp…, precise
+#                                         gfx1201 TP2 nbt4096/seqs16/len262144)
+#   expert cache      14.87 -> 17.22 GiB
+#   slots             6,496 -> 7,522     (+1,026 = +16%, 2.344 MiB each)
+#   swap region                 13.7 GiB / 5,986 slots, 133-134 per layer
+#   boot                        healthy in 420 s, no OOM, no knife-edge
+# The flag was called a non-option after it crash-looped the 2 x 262144 bf16
+# boot (0.13 GiB free of 31.86 GiB, NRestarts=3 at ~6.7 min each) — that shape
+# was already at the ceiling, so precise-memory was the last 2.35 GiB over, not
+# the cause. At 200704 x 3 it fits with room; the run line also reports
+# 'expert_precise_memory': True in non-default args, so a silent argparse drop
+# would be visible in the boot log.
+FRANK_PRIMARY_PRECISE_MEMORY=1
+# GPU_MAX_HW_QUEUES caps the HSA hardware queues per GPU (AMD's own tuning
+# guide recommends 2 for this class of workload; the runtime default on Linux
+# is 4). Verified a REAL knob, not a no-op: the string is present in
+# /opt/rocm/core-10.0/lib/libhsa-runtime64.so. NOT A/B MEASURED — Frank's call,
+# shipped as a deliberate CPU-side tune alongside the 32x64 gather geometry.
+FRANK_PRIMARY_GPU_MAX_HW_QUEUES=2
+# Persistent compile cache. MEASURED 2026-10-02: the image ships NO
+# /root/.cache at all (`ls: cannot access '/root/.cache'` in a fresh
+# container), and after our boot it holds /root/.cache/vllm at 1,154,714,333 B
+# in 4,665 files plus /root/.cache/comgr at 229 MB — both inside the
+# container's ephemeral layer, so every `podman rm -f` (every deploy, restart
+# and reboot) throws them away and re-pays the build. The engine's own log
+# names the loss: `torch.compile took 181.20 s in total`, of which 'Compiling a
+# graph for compile range (1, 1024) takes 173.19 s', followed by 'saved AOT
+# compiled function to /root/.cache/vllm/torch_compile_cache/…'. Mounting a
+# host dir at /root/.cache persists the vLLM AOT/torch_compile tree AND the
+# comgr HIP-kernel JIT cache, and hides nothing (the image has no such dir).
+# PROVEN LIVE 2026-10-02, two boots of this exact shape:
+#   cold  `Compiling a graph for compile range (1, 1024) takes 179.24 s`
+#         `torch.compile took 188.26 s in total`
+#   warm  `Directly load the compiled graph(s) for compile range (1, 1024)
+#          from the cache, took 2.143 s`
+#         `torch.compile took 3.97 s in total`
+#         `init engine (profile, create kv cache, warmup model) took 41.13 s
+#          (compilation: 3.98 s)`
+# (Total time to /health is dominated by the 172.8 GiB payload load, not the
+# compile — the saving is the ~184 s of inductor/AOT build.)
+# Safe to grow/prune: entries are keyed by version + config hash, so a stale
+# entry is a miss, never a wrong answer.
+FRANK_PRIMARY_CACHE_DIR=/home/dank/.cache/frank-primary
+FRANK_PRIMARY_MAXLEN=200704
+FRANK_PRIMARY_SEQS=3
+# The POOL is deliberately kept at shape D's 3.15e9 (option A): because the pool
+# is not an arena lever, raising it back from a cut buys concurrency only, and
+# 3.15e9 preserves the measured 1.99x while precise-memory grows the arena. A
+# 2.2e9 pool was probed in the same window (277,686 tokens / 1.38x) and rejected
+# as the shipped value; it remains the conservative arm if free VRAM ever binds.
+# KV forced to 400,700 tokens (Frank, 2026-10-03). The engine prices this pool
+# at 7,891.8 B/token at this shape and loses ~3.87 MB to block alignment, so
+# 3,166,500,000 B is the ask that lands at/above 400,700 (verify the reported
+# "GPU KV cache size" line on every deploy; it is not a constant ratio across
+# geometries — 7,701 B/token at 262144).
+FRANK_PRIMARY_KV_BYTES=3166500000
+FRANK_PRIMARY_KV_DTYPE=fp8
+FRANK_PRIMARY_MNBT=2048
+FRANK_PRIMARY_GATHER_CHUNKS=32
+FRANK_PRIMARY_GATHER_LANES=64
+# PREFIX CACHING: ON. An OFF experiment ran 2026-10-05 (below) and FAILED
+# to stop the corruption, so the historical flag is back.
+# MEASURED 2026-10-04, 18:46:07 PDT (5.6 h into the live boot, one running
+# request, prefix hit rate 48.1%): 48 gdn_hip state faults in one 0.6 s
+# burst, then the process-lifetime silence cap:
+#   [gdn_hip] ERROR gdn_decode_r: state out of ssm-cache range: max|S|=0
+#   with non-finite values (slot=42 head=3 limit=3.40282e+38) fault #1-23
+#   [gdn_hip] ERROR gdn_decode_r: ... (slot=42 head=8 ...) fault #24-46
+#   [gdn_hip] ERROR: state fault report limit (32) reached; further faults
+#   are counted silently
+# One physical SSM slot, two heads, BOTH ranks (heads 3 and 8), 23 layers
+# each. The guard's own message in this build: "known Qwen3.8-Flash-Next
+# (QFN) flaw under prefix caching and is being actively worked; output
+# quality for this request may degrade. Mitigation:
+# --no-enable-prefix-caching".
+# MECHANISM: with prefix caching on, this fork puts QFN on
+# mamba_cache_mode 'align' (the default-for-this-arch branch in
+# config/vllm.py; 'all' is deprecated and 'none' is the no-prefix-cache
+# mode, so align is the only sound mode WITH the flag). Align mode
+# checkpoints each sequence's GDN (Gated DeltaNet) recurrent state into a
+# mamba page every block, where block = 3136 tokens (the attention block
+# size is raised to 3136 so the page sizes match exactly — the boot log's
+# "Padding mamba page size by 0.13% ..."), and RE-SEEDS the state from that
+# page whenever a request resumes from a cached prefix. The corrupt state
+# landed at the 13th boundary: 13 x 3136 = 40,768 tokens — the "~40k" mark
+# where the request's output turned repetitive/corrupted. The NaN stayed
+# in the mamba page, so every later turn of that conversation re-seeded the
+# same non-finite state for those two heads (silent: the fault-report cap
+# was already spent), while fresh short sessions looked fine.
+# RAM WAS "RULED OUT" THE SAME DAY — AND THAT WAS WRONG (RESOLVED below).
+# What the evidence actually supported: zero MCE/EDAC records is the
+# EXPECTED silence of consumer non-ECC memory (a silent flip raises no
+# machine check); an 8 GiB pattern check over 60 s covers ~0.5 % of
+# 246 GiB — no verdict on a speed setting; "4800 is the JEDEC floor on
+# 5600-rated DIMMs" is a rating, not a stability proof at four-DIMM
+# loading. Only the two-head NaN signature argument was load-bearing,
+# and the revert test outvoted it.
+# COST ACCEPTED: no prefix reuse, so each turn re-prefills the whole
+# history instead of at most 3135 tail tokens (a full 40k context costs
+# tens of seconds of TTFT per turn at this shape's prefill rate). A silently
+# corrupted conversation is worse than a slower first token.
+# RESULT (2026-10-05, later that day): THE MITIGATION DID NOT WORK. With
+# --no-enable-prefix-caching deployed and mechanically verified (zero
+# gdn_hip faults, a 40,771-token single-request boundary probe coherent),
+# live long conversations STILL degraded at ~40k on the running endpoint
+# (Frank, direct). The container serving that window logged ZERO faults
+# while the corruption reproduced, so the align-mode page re-seed is not
+# the (only) mechanism — see the runbook's RESULT note for the open
+# hypotheses (re-fed poisoned transcript vs a fault-free long-context
+# path such as fp8 KV at depth). Cost bought nothing: caching is back ON.
+# The knob stays commented for a future fork build with a fixed GDN state
+# path: set 1, redeploy, re-verify past 40,768 tokens (one conversation
+# coherent AND zero gdn_hip lines).
+# RESOLVED 2026-10-05 EVENING (Frank): IT WAS THE RAM. 4800 MT/s (in
+# effect since the 2026-10-04 17:17 boot) was marginal; 4600 MT/s (since
+# the 2026-10-05 12:21 boot) and the endpoint is clean — zero gdn_hip
+# ERROR lines, zero MCE, and the live agent session on it past 140k
+# context with no degradation. Both failure eras sit under the RAM: the
+# 2026-10-04 NaN burst (corrupted weight/activation traffic poisoning a
+# GDN state write — the guard-visible path) and the fault-free ~40k
+# degradation (silent flips in KV/expert data, which no guard covers on
+# non-ECC). CONSEQUENCE FOR THE KNOB: the OFF experiment was CONFOUNDED
+# — both arms ran on the marginal speed — so it neither proved nor
+# disproved the fork's align-mode flaw. Prefix caching stays ON (zero
+# gdn_hip lines under heavy re-send traffic since the speed drop); flip
+# the knob only if a fault recurs on stable RAM or the fork ships a fix.
+# FRANK_PRIMARY_NO_PREFIX_CACHING=1
+# UTIL is an ARENA lever: the fork plans `budget = device_total x util`, and at
+# 31.86 GiB/card every +0.01 is +0.32 GiB of expert cache (~+139 slots), and
+# 0.982 is Frank's standing law: NEVER 1.00. The 1.00 attempt (2026-10-03,
+# shape G v1) died in worker init BEFORE loading a byte of weights — the fork
+# refuses budget >= device total with
+#   ValueError: Free memory on device cuda:N (31.29/31.86 GiB ...)
+# on BOTH ranks at 09:53 and 09:54, NRestarts=2 — so 1.0 is not even a runtime
+# knife-edge here, it is unbootable. The two historical ceiling deaths (the
+# 2x262144 bf16 boot at 0.13 GiB free; the 5x262144 boot at "0 bytes free" on a
+# real 995-token request) remain the reason the margin exists. Rollback note
+# for the other direction: if a request ever dies with
+# `Tried to allocate ... 0 bytes free`, the first knob back is THIS number
+# down a notch, not the window.
+FRANK_PRIMARY_UTIL=0.982
+# CUDA-graph plan term, priced at the measured capture. expert_plan.py charges
+# estimate_cudagraph_memory() = BASE_MB + PER_GRAPH_MB x n_graphs (defaults
+# 256/52 = 0.55 GiB for our 6-graph plan), and the estimator's own docstring
+# says to tune these against the post-capture line. MEASURED twice on this
+# shape: shape E reported "Graph capturing finished ... took 0.31 GiB" (TP0) and
+# 0.17 GiB (TP1); shape F, with the 0.32 GiB we then charged, reported
+# "took -0.06 GiB" — a NEGATIVE delta, i.e. capture fits inside memory the
+# profiler had already counted. So the base pool (256 MiB) is the over-charge:
+# charge 0 base + 12 MiB/graph = 0.07 GiB for the shipped no-spec 6-graph plan,
+# which returns 0.25 GiB more to the arena while still paying per graph, so a
+# future spec-decode shape (18 graphs) is charged 0.21 GiB rather than zero.
+# Verify every deploy against the "Estimated CUDA graph memory" vs "Graph
+# capturing finished ... took X GiB" lines; if capture ever goes positive and
+# exceeds the charge, raise FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB back toward 52.
+# A THIRD env var exists and is the only way to state the graph term outright:
+# VLLM_CUDAGRAPH_MEM_ESTIMATE_MB pins the WHOLE term to an exact MiB total in
+# one knob — estimate_cudagraph_memory() checks it FIRST and returns base/
+# per-graph untouched when set (verified in the shipped image,
+# v1/worker/gpu/cudagraph_mem.py). We ship the BASE/PER_GRAPH pair instead so
+# a future spec-decode shape re-prices automatically; switch to ESTIMATE_MB
+# only if the drift check below ever shows the pair cannot track capture.
+FRANK_PRIMARY_CUDAGRAPH_BASE_MB=0
+FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB=12
+
+# HOST-region cap — the last arena-plan term that still autofit per boot. With
+# no cap the planner's host bound is MemAvailable AT BOOT minus PLE 47.68 -
+# 2 x 7.0 process - 5.0 api/engine - 5% margin (measured boot 2026-10-05:
+# MemAvailable 237.97 -> 159.39 GiB usable across 2 ranks; the demand is 105.48
+# = 52.74/rank x 2 ranks, 23,040 swap-target experts). The same host reads 75.4
+# GiB available while running — the bound tracks whatever the co-tenants hold at
+# boot, and a boot where that drops near the demand silently pins next-hottest
+# experts VRAM-only instead (smaller swap region; the pinned_extra warning is
+# the only trace). Pin 120 GiB: measured demand + 14% headroom, on a host with
+# 246.1 GiB total. If a boot ever logs "expert-offload-mem … exceeds
+# MemAvailable at plan time", THAT is the contract being broken — look at the
+# co-tenant RAM before touching this number. Plan semantics verified in the
+# shipped image (offloader/expert_plan.py): the cap binds as the bound, an
+# exceedance only warns.
+FRANK_PRIMARY_EXPERT_OFFLOAD_MEM=120
+
+# --- optional engine knobs (unset == the flag is absent) ---------------------
+# FRANK_PRIMARY_SPEC holds the WHOLE --speculative-config JSON; setting it adds
+# the flag. THE RETIREMENT VERDICT WAS INVALID: c1248d4 (2026-10-02) retired MTP
+# by comparing fp8+MTP4 (37.4-40.6 tok/s, 54% acceptance, 09-29) against the
+# bf16 no-spec boot (36.4-39.4) — and bf16 then proved unbootable, so fp8 came
+# back without MTP and fp8+spec was NEVER compared against fp8+no-spec on the
+# same shape. The no-spec incumbent measures ~30 tok/s single-stream (below the
+# fleet's 35 tok/s target); every vendor flash-next recipe ships MTP3; and our
+# MTP4 run used the fork's default GREEDY draft sampling, while the recipes pass
+# draft_sample_method=probabilistic (greedy drafts are rejected more often under
+# temp>0 — the plausible cause of the 54%). MTP also re-adds drafter workspace:
+# the old note says the 7.2 GiB/rank KV ceiling WAS the MTP4 workspace, and
+# --expert-precise-memory spends that same budget, so an MTP arm must be tested
+# with precise memory BOTH on and off.
+#   MTP3:  '{"method":"mtp","num_speculative_tokens":3,"draft_sample_method":"probabilistic"}'
+FRANK_PRIMARY_SPEC=
+# FRANK_PRIMARY_COMPCONFIG holds the whole --compilation-config JSON. The
+# capture-ladder law: the piecewise cudagraph ladder must contain
+# num_speculative_tokens + 1 (MTP3 -> 4 -> ladder 4,8,12; MTP4 -> 5,10,15;
+# MTP5 -> 6,12,18,24), because every draft step proposes n+1 tokens. The default
+# ladder is multiples of 1+4 up to 512 then powers of two, so an MTP3 shape can
+# miss its own step size. Pinning the ladder is ALSO the candidate fix for the
+# SEQS>3 wall: query_start_loc is specialised to SEQS+1 and that constant must
+# land on a captured graph size (3 is not in the default ladder, 4 is —
+# measured 2026-10-02, and the vendor runs SEQS 16 on this model/TP2).
+# combo_kernels false/true are BOTH in the shipped recipes, so it is not a law.
+#   MTP3 at SEQS 3:  '{"cudagraph_mode":"PIECEWISE","cudagraph_capture_sizes":[1,4,8,12],"max_cudagraph_capture_size":12}'
+FRANK_PRIMARY_COMPCONFIG=
+# FRANK_PRIMARY_GEN_CONFIG holds the whole --override-generation-config JSON.
+# The payload's HF generation_config.json sets temperature 1.0 / top_p 0.95 /
+# top_k 20 and the engine WARNS about it on every boot, so a client that sends
+# no sampling fields gets creativity it never asked for. This is the one knob
+# that changes ANSWERS, not speed.
+#   '{"max_tokens":65536,"temperature":0.7,"top_p":0.95,"top_k":20}'
+FRANK_PRIMARY_GEN_CONFIG=
+
+# --- client contract: reasoning split, tools, preserve_thinking ----------------
+# Without --reasoning-parser the engine leaves the thinking text INLINE in
+# `content` with a dangling `</think>` and a null `reasoning` field (measured
+# 2026-10-01: content = 'We need answer...Brief.\n\n391') — the
+# malformed thinking block clients choke on. The qwen3 parser splits it
+# properly. The GLM rollback (d91379d) had dropped all three of these; the
+# working values are the ones ecaad36 shipped for this model.
+FRANK_PRIMARY_REASONING_PARSER=qwen3
+# Tools: --enable-auto-tool-choice + the qwen3_xml parser is what makes
+# tool_calls work here (without it, tool_choice auto returns 400).
+FRANK_PRIMARY_TOOL_PARSER=qwen3_xml
+# preserve_thinking keeps prior thinking blocks in the rendered prompt across
+# turns; enable_thinking makes thinking the server-side default. Verified the
+# payload's tokenizer_config template consumes both keys.
+FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS='{"enable_thinking": true, "preserve_thinking": true}'

scripts/frank-primary.service

@@ -0,0 +1,67 @@
+# frank-primary.service — tcclaviger RDNA4 vLLM endpoint (orcarouter
+# Qwen3.8-Flash-Next-Uncensored-FP8, fp8 KV, NO spec decode, 3 slots x 200704
+# context with the pool sized to ~2 full-length conversations, TP=2, tiered
+# experts, 131-shard payload) on the 2x R9700 pair, dankserver :7770.
+# 2026-09-29 cutover from the R9V radiance fork (git tag
+# frank-primary-r9v-q8-20260929 is the pre-cutover state; its
+# launcher/shim/setup-state machinery is retired and the state dirs stay on
+# disk as rollback).
+#
+# Same port, same served alias; the harness models.yml id is unchanged.
+# The serve script drives `podman run` directly (the image entrypoint execs
+# `vllm serve`) and then waits on the container. NOTE: that wait can return
+# 0 even when the engine died mid-request (vLLM's API server drains and
+# exits clean on a worker fault), which is why the restart policy is
+# `always` — see the [Service] note.
+#
+# WHY A ROOT UNIT (not a rootless quadlet): the tiered-expert shape pins tens
+# of GiB of host RAM (the expert host slabs, ~20.5 GiB per rank of pinned
+# anonymous memory). Pinned pages are charged against RLIMIT_MEMLOCK; dank's
+# login session on this host carries 8 MiB and a user quadlet is capped by the
+# user manager's own hard limit — raising that means restarting
+# user@1000.service and bouncing every user container on the host. A root unit
+# grants LimitMEMLOCK=infinity to the dank process via PID1, and the image
+# stays in dank's rootless podman storage.
+[Unit]
+Description=frank-primary — tcclaviger vLLM Qwen3.8-Flash-Next-Uncensored-FP8 on 2x R9700
+After=network-online.target
+Wants=network-online.target
+# Never let the start-rate limiter give up after a crash loop (a failed LLM
+# endpoint that systemd refuses to retry is worse than a retrying one).
+StartLimitIntervalSec=0
+
+[Service]
+User=dank
+Group=dank
+# The tiered-expert offloader pins host RAM; an RLIMIT_MEMLOCK smaller than
+# the pinned set fails the load mid-start. PID1 grants infinity here —
+# parity with the fleet's other LLM units.
+LimitMEMLOCK=infinity
+Environment="FRANK_PRIMARY_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-primary.env"
+ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-primary.sh
+# systemd stop: wait alone cannot stop the container child; stop it
+# explicitly. vLLM 0.26 drains gracefully on SIGTERM.
+ExecStop=/usr/bin/podman stop -t 120 frank-primary-qwen
+# Boot cost: FP8 payload page-in plus torch compile/graph capture —
+# ~15 min cold (measured 2026-09-29), faster warm.
+TimeoutStartSec=1800
+# Restart=always, NOT on-failure (measured 2026-10-01): a runtime
+# torch.OutOfMemoryError kills the engine while vLLM's API server still
+# exits 0, so ExecStart returned clean, systemd logged "Deactivated
+# successfully", and on-failure NEVER fired — the endpoint stayed down until
+# a human noticed. `always` restarts on any container death; an explicit
+# `systemctl stop` still wins (systemd does not restart a unit the operator
+# stopped), so bench windows and deploys are unaffected.
+Restart=always
+RestartSec=10
+# ExecStop allows the engine 120 s to drain (`podman stop -t 120`); without an
+# explicit TimeoutStopSec systemd's 90 s default cuts the stop short, kills it,
+# and records 'Failed with result timeout' (measured 2026-10-01 13:27:17).
+# Must exceed the podman grace with margin for the engine's own shutdown.
+TimeoutStopSec=180
+# The engine pins tens of GiB of host RAM (expert host slabs); it is the
+# last process on this host that should be chosen by the kernel OOM killer.
+OOMScoreAdjust=-500
+
+[Install]
+WantedBy=multi-user.target

scripts/frank-small.env

@@ -1,113 +1,136 @@
-# Tunables for frank-small (ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF, the IQ3_S
-# build with the MTP head, plus the BF16 vision projector) on mainline
-# llama.cpp, Dankclaw's RTX 3090, since 2026-09-16.
+# frank-small — llama.cpp (master, CUDA sm_86) serving
+# ornith-ai/Ornith-1.5-9B-GGUF:Q8_0 on the RTX 3090, :7771.
 #
-# Runtime: llama.cpp master 37b53fd (2026-09-16), CUDA build for sm_86:
-#   cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON \
-#     -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86
-#   cmake --build build-cuda --target llama-server -j
-# GGML_CUDA_FA_ALL_QUANTS is deliberately OFF in that build: it is only needed
-# for QUANTIZED KV with flash attention, and this service runs f16 KV (below).
+# 2026-10-05 (Frank): the payload moved off bartowski/MiMo-V2.6 (Q6_K) to
+# ornith-ai/Ornith-1.5-9B Q8_0 (9,786,060,384 B, LFS oid 22086870...).
+# Header-verified facts that re-price the shape: qwen35 hybrid like MiMo, but
+# 33 blocks (not 32) with full_attention_interval 4 => NINE full-attention
+# layers (MiMo had 8), 16 Q heads / 4 KV heads, k/v length 256, native window
+# 262144, dense (no expert keys), nextn_predict_layers=1 (an MTP head is IN
+# the file — capability only; no spec decode ships, same law as before).
 #
-# Model: ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF, the `-mtp` build of IQ3_S.
-# GSQ-RCO assigns a separate quantization type to every tensor from a
-# gradient-based search under a total size budget, rather than applying one
-# type throughout; IQ3_S at 3.50 bpw is the release's task-lossless operating
-# point (AIME25 100.00 and LiveCodeBench v6 85.71 match BF16 exactly,
-# GPQA-Diamond 0.51 behind, at 11.8 GB against 53.8).
+# THE KV CONTRACT STAYS f16 K + f16 V — NO KV QUANTIZATION (Frank, 2026-10-05,
+# overruling a written q8_0-V proposal). The arithmetic that makes this hard:
+#   KV rate 9 attn layers x 4 heads x 256 x 2 B = 18 KiB/token K, same again
+#   for V at f16 => 36 KiB/token (MiMo was ~24.5 measured: one fewer
+#   full-attn layer AND one fewer KV head).
+#   Weights +1,900 MiB (9,334 vs Q6_K's 7,432). Measured free at the Q6_K
+#   4096/2048 shape was 2,279 MiB; Q8_0 at f16/f16 does not fit that shape —
+#   the deficit is made up the way Frank ordered: UBATCH FIRST (2048 -> 1024,
+#   the measured 616 MiB rung), then POOL STEPS, then measure. Guess-and-check
+#   on the live card is the accepted method for llama.cpp sizing here: boot,
+#   read nvidia-smi free, step the pool (12,288-token increments, ~432 MiB
+#   each at the 36 KiB/token rate) until the 2-3 GiB floor holds. Compute
+#   buffers allocate at graph capture during BOOT, so a failure is loud and
+#   fast, not a surprise at the first long prompt.
 #
-# WHY THAT FILE: the `-mtp` build is ~0.35 GB larger because it carries the
-# model's Multi-Token Prediction head as GGUF block `blk.64.nextn.*`
-# (qwen35.nextn_predict_layers = 1). llama.cpp loads it from the SAME file --
-# `--spec-type draft-mtp` sets load_mtp, and the draft context runs on the main
-# model's weights (common/common.cpp passes shares_model when there is no
-# --model-draft; speculative.cpp gives it LLAMA_CONTEXT_TYPE_MTP), so there is
-# no second 12 GB copy in VRAM. MTP is therefore a property of the FILE, not of
-# the flags: the plain IQ3_S build has no MTP tensors to load.
+# Previous shape history (MiMo era, for the rollback trail): 2026-10-03 the
+# rung went Q5_K_M -> Q6_K with the pool HELD at 401,408; the freed third
+# slot funded exact f16 V first; the ~3.5 GiB still free funded the +875 MiB
+# of weights.
 #
-# Architecture (GGUF arch qwen35): 64 layers laid out as 16 x (3 x (Gated
-# DeltaNet -> FFN) + 1 x (Gated Attention -> FFN)) -- 48 linear-attention
-# layers with CONSTANT-SIZE per-sequence state and 16 full-attention layers,
-# 4 KV heads x 256 head_dim. Only the attention layers take KV, so a token
-# costs 2 (K,V) x 4 heads x 256 x 2 B = 64 KiB at f16 (34 KiB at q8_0, 18 KiB
-# at q4_0), and the DeltaNet layers add ~0.16 GiB per sequence in f32 at any
-# depth.
+# 2026-10-03 (Frank): TWO streams instead of three, each one WIDE — 200,704
+# tokens per conversation, above the 200,700 the harness compacts at — and the
+# KV cache is now FULLY UNQUANTIZED (f16 K, f16 V). The two moves are the same
+# move: llama.cpp divides --ctx-size across --parallel slots, so the slot given
+# up is exactly the VRAM that pays for unquantized V. No further quantization
+# anywhere on this endpoint.
 #
-# SIZING (RTX 3090, 24576 MiB) -- modelled, and the loader's own ledger
-# ("KV self size", "compute buffer size") outranks it:
-#   weights                            11.29 GiB (12,120,016,960 B)
-#   mmproj                              0.87 GiB (931,146,528 B, 27 blocks)
-#   MTP draft KV                        0.50 GiB (its context is n_ctx long
-#                                       over one nextn layer: 4 KiB/token)
-#   DeltaNet state                      0.16 GiB (one sequence)
-#   CUDA context + prefill/decode bufs  ~1.5 GiB
-#   KV at CTX_SIZE                      8.00 GiB (131,072 x 64 KiB)
-#                                      --------
-#                                      ~22.3 GiB of 24, ~2 GiB of slack
-# The binding number is the KV: ~9.5 GiB is left after the fixed terms, which
-# at 64 KiB/token is ~155k tokens across all slots. The deployed shape is one
-# slot at 131,072, full f16 precision, no KV quantization risk, no rebuild.
-# Alternatives inside the same budget: 2 x 65536 at f16; 4 x 65536 or
-# 1 x 262144 at q8_0, both of which need a rebuild with
-# -DGGML_CUDA_FA_ALL_QUANTS=ON (without it quantized-KV prefill collapses
-# ~12x on CUDA -- see the CUDA lesson in AGENTS.md).
-# --parallel N DIVIDES --ctx-size between slots; it does not multiply. One slot
-# means a second concurrent request queues behind the first.
+# Three measurements on this card, this runtime, price the terms:
+#   2 x 262,144, no projector: 20,190 MiB used / 4,386 free
+#     KV = 524,288 tok x 24.5 KiB = 12,800 MiB => non-KV (weights 6,557 +
+#     DeltaNet state ~25/slot + compute buffers + CUDA graphs) = 7,390 MiB
+#   3 x 196,608, projector on: 22,972 MiB used / 1,153 free
+#     KV = 14,113 MiB => non-KV WITH the vision tower = 8,859 MiB
+#   3 x 180,224, projector on: 21,684 MiB used / 2,441 free (the 2026-10-02
+#     shape) => non-KV = 8,748 MiB
+# Two slots give back one DeltaNet state (~25 MiB). Priced at f16 K+V
+# (32 KiB/token) this pool is 12,544 MiB, so the prediction is
+# 12,544 + ~8,800 = ~20.9 GiB used, ~3.1 GiB free — the band the 2026-10-02
+# shape hit, bought with precision instead of a slot. The 1024 ubatch
+# re-prices the compute buffers upward, so MEASURE after any change: if the
+# boot lands under 2 GiB free, step the pool down in 12,288-token
+# (384 MiB at this rate) increments.
 #
-# Sampling: greedy by default, which is the fleet contract (a request that
-# sends no temperature decodes greedy; a client that sends one gets its own).
-# Qwen's own agentic evaluations used temperature 1.0 / top_p 0.95, and the
-# harness applies exactly that per model in models.yml (extraBody), the same
-# way it did for the earlier frank-large runtime.
+# KV arithmetic: attention KV exists only in the 8 full-attention layers
+# (4 KV heads x 256 head_dim x 2 B) = 16 KiB/token for K and the same again for
+# V => 32 KiB/token, so 12,288 tokens (= 6144/slot) is one 384 MiB step — that
+# is the ladder to walk if this moves again. Two native windows (2 x 262,144)
+# are NOT reachable with full-precision V: 524,288 x 32 KiB = 16.1 GiB of KV
+# against a card with ~15.7 GiB of KV room once weights, the vision tower, the
+# DeltaNet states and the graphs are loaded. 200,704 per stream is what full
+# precision buys; q8_0 on V would buy 262,144 instead.
 #
-# WARNING: do not name any variable TEMP, HOST, or GPU_MAX_HEAP_SIZE (HIP reads
-# those bare vars on the ROCm hosts -- see the 2026-08-13 incident in
-# AGENTS.md). This box is CUDA, but the rule keeps env files portable.
+# The asymmetric-KV contract this endpoint was created for (Frank, 2026-10-01
+# night: f16 K + q8_0 V) is now spent — it existed to make a third slot fit,
+# and the third slot is gone. The flags stay explicit so the choice is visible
+# in the argv either way.
+LLAMA_BIN=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server
+MODEL_FILE=/home/dank/models/Ornith-Q8/Ornith-1.5-9B-Q8_0.gguf
 
-# llama.cpp runtime binary: mainline master 37b53fd, CUDA sm_86.
-LLAMA_BIN=/home/dank/GitPackages/llama.cpp-mainline/build-cuda/bin/llama-server
+# Vision: the Ornith Q8_0 GGUF ships no `v.*` tensors either (dense-9B byte
+# arithmetic: 9.79 GB ≈ 9B params x ~1.06 B/param leaves no room for a tower),
+# so the projector file IS the vision capability (mmproj-Ornith-1.5-9B-BF16
+# .gguf, 921,704,672 B, LFS oid 626f9f90..., fetched by
+# scripts/download-frank-small.sh). Empty string = text-only on purpose.
+MMPROJ_FILE=/home/dank/models/Ornith-Q8/mmproj-Ornith-1.5-9B-BF16.gguf
 
-# Model package. The -mtp file is required for speculative decoding.
-MODEL_DIR=/home/dank/models/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp
-MODEL_FILE=Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf
-MMPROJ_PATH=/home/dank/models/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp/mmproj-Qwen3.8-27B-BF16.gguf
-
-# Fleet identity and network. 7771 is this service's own endpoint on Dankclaw;
-# 7770 stays frank-large's. The alias is the id at /v1/models and what clients
-# ask for -- the harness entry in models.yml names it.
+# Fleet endpoint: same port and alias as every frank-small before it, so the
+# harness models.yml id and the homepage probe do not move.
 ALIAS=frank-small
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7771
 
-# One slot at 131,072 tokens with f16 KV (see the sizing note above).
-CTX_SIZE=131072
+# ONE conversation at the NATIVE 262,144 window (Frank, 2026-10-05). The
+# two-slot shape died under Ornith and the NO-QUANTIZE law, by arithmetic:
+# the KV rate is the ARCHITECTURE's, not the quant's — 9 full-attn layers x
+# 4 KV heads x 256 dim x 2 B = 36 KiB/token at f16 K+V (MiMo measured 24.5),
+# so 2 x 200,704 wants ~14.1 GiB of KV on top of +1,900 MiB of Q8 weights
+# against 2,279 MiB measured free; ubatch 1024 claws back 616, and no
+# pool step fits BOTH the 2-3 GiB floor and the harness compaction
+# invariant (slot >= 191,148). One native slot fits with room; the window
+# per conversation RISES 200,704 -> 262,144. The cost: a second concurrent
+# request queues behind the released slot (subagent concurrency 2 -> 1).
+CTX_SIZE=262144
 PARALLEL=1
+
+# Compute-buffer geometry. This endpoint passed neither flag until 2026-10-03,
+# so it ran on llama.cpp's own defaults (2048/512); the pair is now 4096/2048.
+# Both are tunables because the pair is a measurement, not a law — and the
+# measurement is CARD-SPECIFIC. On the 2x R9700 layer-split sibling the ubatch
+# curve peaks at 512 and every step up costs GiB of compute buffer for no
+# speed (see frank-medium-batch-ubatch). On THIS CUDA 3090 the curve climbs
+# monotonically to 2048: 710 -> 1188 t/s prefill from ubatch 1024 to 2048 on
+# the 35B MoE sweep, 1944 -> 2423 t/s on the Ornith 8k sweep, with decode flat
+# to under 1 t/s on every rung. Ubatch is a prefill knob; decode never moves.
+#
+# The logical batch is free — two independent sweeps agree that at a fixed
+# ubatch, 2048/4096/8192 give identical prefill speed AND identical peak VRAM,
+# because the compute buffers are sized by the ubatch alone. Doubling it costs
+# nothing and buys nothing; it is here because it is the value the pair was
+# measured at.
+#
+# THE RISK IS ALL IN THE UBATCH. The 1024 -> 2048 step cost ~1,244 MiB of peak
+# graph VRAM on the 35B MoE (7,330 -> 8,574 MiB); this 9B hybrid has far
+# smaller compute tensors, so the delta should be well under that, but the
+# predecessor Q5_K_M shape recorded "ubatch 2048 failed the scratch boot" at
+# ~3,571 MiB free. The Q8_0 rung ships UBATCH 1024 (Frank, 2026-10-05: lower
+# ubatch FIRST — the measured 616 MiB step is the first VRAM the Q8 weights
+# claw back, before any pool move). Compute buffers are allocated at graph
+# capture during BOOT, so a failure is loud and fast, not a surprise at the
+# first long prompt. MEASURE nvidia-smi free immediately after the deploy:
+# under ~1.5 GiB, or a boot failure, step the POOL down next (12,288-token
+# increments, ~432 MiB each at the Ornith 36 KiB/token f16 KV rate).
+BATCH_SIZE=4096
+UBATCH_SIZE=1024
+
+# The KV contract (2026-10-05, Frank): f16 K + f16 V — NO KV quantization.
+# The q8_0-V arithmetic was written and OVERRULED. The Q8 weights eat the
+# VRAM that funded f16 V on 2026-10-03, so the remedy ladder is UBATCH FIRST
+# (2048 -> 1024, the measured 616 MiB rung), then POOL STEPS — and
+# guess-and-check on the live card is the accepted method: boot, read
+# nvidia-smi, step down until the floor holds. --flash-attn stays pinned on
+# in the serve script: it is the fast path for f16 KV.
 CACHE_TYPE_K=f16
 CACHE_TYPE_V=f16
-
-# Batch geometry: llama.cpp's own defaults (2048 logical / 512 physical). The
-# prefill wall clock at this model size is set by the card, not by ubatch, and
-# the small ubatch keeps the DeltaNet chunk buffers small. Raise UBATCH_SIZE
-# only after reading the compute buffer size the loader reports.
-BATCH_SIZE=2048
-UBATCH_SIZE=512
-
-# CPU threads: this service runs entirely on the GPU (NGL=999), so threads
-# cover tokenization, sampling and the HTTP layer only. Deliberately modest --
-# this host's CPU is shared with halogen's engine, which is CPU-hungry.
-NUM_THREADS=8
-THREAD_BATCH=8
-
-# Every layer on the 3090, with flash attention.
-NGL=999
-FLASH_ATTN=on
-
-# Speculative decoding from the model's own MTP head (see the model note).
-# Draft depth stays llama.cpp's default of 3 tokens.
-SPEC_TYPE=draft-mtp
-
-# Greedy default (fleet contract); reasoning is extracted into
-# reasoning_content with an unlimited budget (the model decides).
-TEMPERATURE=0.0
-REASONING_FORMAT=deepseek
-REASONING_BUDGET=-1
+NGL=99

scripts/frank-small.service

@@ -1,5 +1,5 @@
 [Unit]
-Description=llama.cpp Qwen3.8-27B GSQ-RCO IQ3_S MTP+vision (dankclaw, frank-small)
+Description=llama.cpp MiMo-V2.6-Distill-Qwen-9B Q6_K GGUF (qwen35 hybrid, 2x200704, unquantized f16 KV, mmproj vision) on the RTX 3090 (dankclaw, frank-small)
 After=network-online.target
 Wants=network-online.target
 
@@ -8,8 +8,7 @@ Type=simple
 User=dank
 Group=dank
 # CUDA pins its staging buffers, so the process needs the memlock budget even
-# though the weights themselves are copied into VRAM. Parity with the halogen
-# quadlet's Ulimit=memlock=-1:-1 and the retired ik unit.
+# though the weights themselves are copied into VRAM.
 LimitMEMLOCK=infinity
 Environment="FRANK_SMALL_ENV_FILE=/home/dank/serverAppConfigs/scripts/frank-small.env"
 ExecStart=/home/dank/serverAppConfigs/scripts/serve-frank-small.sh

scripts/hf-fetch.sh

@@ -0,0 +1,105 @@
+#!/usr/bin/env bash
+# hf-fetch.sh — resumable, verified HuggingFace fetch (gated or public repos).
+#
+#   bash scripts/hf-fetch.sh <dest_dir> <manifest>
+#
+# Manifest: one line per file, three tab/space-separated fields:
+#   <filename> <size_bytes> <sha256>
+# A blank sha256 field means "check size only". Lines starting with '#' and
+# blank lines are ignored. The first field is a path relative to <dest_dir>.
+#
+# Token: $HF_TOKEN, else ~/.cache/huggingface/token, else the llamaswap conf
+# (grep hf_...). Passed out-of-band via a mode-600 curl -K file — a -H header
+# is world-readable in /proc/<pid>/cmdline (the 2026-09-16 HF-token leak).
+#
+# Idempotent: a file whose size AND sha256 already match is skipped, so a
+# re-run after an interruption only fetches what is missing or corrupt.
+set -euo pipefail
+
+dest="${1:?usage: hf-fetch.sh <dest_dir> <manifest>}"
+manifest="${2:?usage: hf-fetch.sh <dest_dir> <manifest>}"
+repo="${3:?usage: hf-fetch.sh <dest_dir> <manifest> <owner/repo>}"
+
+mkdir -p "$dest"
+
+# --- token resolution --------------------------------------------------------
+token="${HF_TOKEN:-}"
+if [[ -z "$token" && -r "$HOME/.cache/huggingface/token" ]]; then
+    token="$(tr -d '[:space:]' < "$HOME/.cache/huggingface/token")"
+fi
+if [[ -z "$token" && -r "$HOME/.config/llamaswap.conf.yaml" ]]; then
+    token="$(grep -oE 'hf_[a-zA-Z0-9]+' "$HOME/.config/llamaswap.conf.yaml" | head -1)"
+fi
+[[ -n "$token" ]] || { echo "hf-fetch: no HF token (set HF_TOKEN or ~/.cache/huggingface/token)" >&2; exit 3; }
+
+auth_file="$(mktemp)"
+chmod 600 "$auth_file"
+printf 'header = "Authorization: Bearer %s"\n' "$token" > "$auth_file"
+cleanup() { rm -f "$auth_file"; }
+trap cleanup EXIT
+
+base="https://huggingface.co/${repo}/resolve/main"
+
+# --- per-file fetch + verify -------------------------------------------------
+fetch_one() {
+    local name="$1" size="$2" sha="$3"
+    local url="$base/$name"
+    local path="$dest/$name"
+    mkdir -p "$(dirname "$path")"
+
+    # idempotent skip
+    if [[ -f "$path" ]]; then
+        local cur_size sha_ok
+        cur_size="$(wc -c < "$path")"
+        sha_ok=true
+        if [[ -n "$sha" && "$(sha256sum "$path" | cut -d' ' -f1)" != "$sha" ]]; then
+            sha_ok=false
+        fi
+        if [[ "$cur_size" == "$size" && "$sha_ok" == "true" ]]; then
+            echo "  ok   $name ($size bytes)"
+            return 0
+        fi
+        echo "  refetch $name (have $cur_size, want $size)"
+    fi
+
+    # Resumable download; -f fails on HTTP error so a 401/403 is not written as a file.
+    # -C - only when a non-empty partial exists: with a missing file it sends
+    # Range: bytes=0-, which HF's xethub CDN rejects with 400 (xet-backed repos).
+    local resume_args=()
+    [[ -f "$path" && -s "$path" ]] && resume_args=(-C -)
+    curl -fsSL "${resume_args[@]}" -K "$auth_file" -o "$path" "$url"
+
+    # verify byte count
+    local got
+    got="$(wc -c < "$path")"
+    if [[ "$got" != "$size" ]]; then
+        echo "  FAIL $name: size $got != $size" >&2
+        return 1
+    fi
+    # verify sha256
+    if [[ -n "$sha" ]]; then
+        local actual
+        actual="$(sha256sum "$path" | cut -d' ' -f1)"
+        if [[ "$actual" != "$sha" ]]; then
+            echo "  FAIL $name: sha256 $actual != $sha" >&2
+            return 1
+        fi
+    fi
+    echo "  done $name ($size bytes)"
+}
+
+echo "hf-fetch: $repo -> $dest"
+status=0
+while IFS= read -r line || [[ -n "$line" ]]; do
+    [[ -z "$line" || "$line" =~ ^[[:space:]]*# ]] && continue
+    name="$(awk '{print $1}' <<<"$line")"
+    size="$(awk '{print $2}' <<<"$line")"
+    sha="$(awk '{print $3}' <<<"$line")"
+    fetch_one "$name" "$size" "$sha" || status=1
+done < "$manifest"
+
+if [[ "$status" -ne 0 ]]; then
+    echo "hf-fetch: FAILED (see above)" >&2
+    exit 1
+fi
+echo "hf-fetch: all files verified"

scripts/hf-manifest.py

@@ -0,0 +1,140 @@
+#!/usr/bin/env python3
+"""hf-manifest.py — generate an hf-fetch.sh manifest from a HuggingFace repo.
+
+Usage: python3 scripts/hf-manifest.py <owner/repo> [outfile]
+
+Lists the repo tree via the HF API and, for each file, records the size and
+the content sha256. LFS and Xet-backed files redirect the resolve endpoint to
+a CDN (302), whose own headers carry the truth: X-Linked-Size is the byte
+count and X-Linked-ETag is the sha256 of the file's bytes. The redirect must
+NOT be followed: on Xet-backed repos the bridge answers with ETag = the Xet
+CAS chunk-dag hash, which is NOT the content sha256 (seen on
+Qwen/Qwen3.8-Flash-Next-FP8, 2026-09-26: every shard "failed" verification
+against a hash no download could ever match). Non-LFS files (config.json,
+tokenizer, ...) are answered directly (200) and carry no 64-hex etag, so they
+get a blank hash field — hf-fetch.sh then checks size only for them.
+
+Output: one line per file, three space-separated fields:
+    <filename> <size_bytes> <sha256-or-blank>
+
+Token: $HF_TOKEN, else ~/.cache/huggingface/token, else the llamaswap conf
+(grep hf_...). Gated repos need a token that has been granted access.
+"""
+import http.client
+import json
+import os
+import re
+import sys
+import urllib.request
+from urllib.parse import urljoin, urlsplit
+
+def get_token():
+    t = os.environ.get("HF_TOKEN", "")
+    if not t:
+        p = os.path.expanduser("~/.cache/huggingface/token")
+        if os.path.exists(p):
+            with open(p) as f:
+                t = f.read().strip()
+    if not t:
+        p = os.path.expanduser("~/.config/llamaswap.conf.yaml")
+        if os.path.exists(p):
+            with open(p) as f:
+                m = re.search(r"hf_[a-zA-Z0-9]+", f.read())
+                if m:
+                    t = m.group(0)
+    return t
+
+
+def head(url, token):
+    """HEAD via http.client, which does not follow redirects.
+
+    urllib's default opener follows the resolve endpoint's 302, and the Xet
+    bridge's ETag is the Xet CAS hash, not the content sha256 — so the 302's
+    own X-Linked-* headers must be read instead. Returns an http.client
+    response; the caller reads r.status and r.headers.
+    """
+    parts = urlsplit(url)
+    conn = http.client.HTTPSConnection(parts.netloc, timeout=30)
+    path = parts.path + (f"?{parts.query}" if parts.query else "")
+    h = {"User-Agent": "hf-manifest/1.0"}
+    if token:
+        h["Authorization"] = f"Bearer {token}"
+    conn.request("HEAD", path, headers=h)
+    return conn.getresponse()
+
+
+def main():
+    if len(sys.argv) < 2:
+        print(f"usage: {sys.argv[0]} <owner/repo> [outfile]", file=sys.stderr)
+        sys.exit(2)
+    repo = sys.argv[1]
+    outfile = sys.argv[2] if len(sys.argv) > 2 else None
+    token = get_token()
+    # 1. List the repo tree (siblings[].rfilename). Sizes and content sha256s
+    # come from the resolve endpoint's HEAD (not followed — see head());
+    # the API's siblings carry only rfilename.
+    api = f"https://huggingface.co/api/models/{repo}"
+    req = urllib.request.Request(
+        api, headers={"User-Agent": "hf-manifest/1.0",
+                      **({"Authorization": f"Bearer {token}"} if token else {})})
+    with urllib.request.urlopen(req, timeout=30) as r:
+        meta = json.load(r)
+    siblings = meta.get("siblings", [])
+    print(f"# {repo}: {len(siblings)} entries", file=sys.stderr)
+
+    lines = []
+    for s in siblings:
+        name = s.get("rfilename", "")
+        if not name or name.endswith("/"):
+            continue
+        # 2. Size + content sha256 from the resolve endpoint's HEAD. 200 =
+        # direct answer (Content-Length + ETag). 3xx WITH X-Linked-Size =
+        # LFS/Xet redirect: X-Linked-Size + X-Linked-ETag are the byte
+        # count and the sha256 of the file's bytes, and the redirect must
+        # NOT be followed — the Xet bridge's own ETag is the Xet CAS hash,
+        # not the content sha256. 3xx WITHOUT X-Linked-Size = non-LFS
+        # redirect (307 to HF's own /api/resolve-cache): the stub's
+        # Content-Length is NOT the file size, so follow ONCE; the final
+        # ETag there is the git blob sha (40 hex), so the 64-hex gate
+        # leaves the hash blank (hf-fetch.sh then checks size only).
+        size = 0
+        sha = ""
+        resolve = f"https://huggingface.co/{repo}/resolve/main/{name}"
+        try:
+            with head(resolve, token) as r:
+                if r.status == 404:
+                    raise FileNotFoundError(name)
+                if r.status == 200:
+                    size = int(r.headers.get("Content-Length") or 0)
+                    etag = (r.headers.get("ETag") or "").strip('"')
+                else:
+                    linked = r.headers.get("X-Linked-Size")
+                    if linked is not None:
+                        size = int(linked)
+                        etag = (r.headers.get("X-Linked-ETag") or "").strip('"')
+                    else:
+                        loc = r.headers.get("Location", "")
+                        with head(urljoin(resolve, loc), token) as r2:
+                            if r2.status == 200:
+                                size = int(r2.headers.get("Content-Length") or 0)
+                                etag = (r2.headers.get("ETag") or "").strip('"')
+                            else:
+                                etag = ""
+                if re.fullmatch(r"[0-9a-f]{64}", etag):
+                    sha = etag
+        except FileNotFoundError:
+            pass
+        lines.append(f"{name} {size} {sha}")
+        print(f"  {name}  {size:>12}  {sha or '(size only)'}", file=sys.stderr)
+
+    out = "\n".join(lines) + "\n"
+    if outfile:
+        with open(outfile, "w") as f:
+            f.write(out)
+        print(f"wrote {outfile}", file=sys.stderr)
+    else:
+        sys.stdout.write(out)
+
+
+if __name__ == "__main__":
+    main()

scripts/homepage-status.py

@@ -2,7 +2,8 @@
 """homepage-status.py — the homepage status checker.
 
 Reads a service config (a JSON array of service entries), probes each backend,
-and writes a status.json atomically. A wip service is not probed.
+and writes a status.json atomically. A wip service is not probed; its state
+is "wip-open" or "wip-closed" following the wip value in the config.
 
 Usage: homepage-status.py <services.json> <out.json>
 """
@@ -48,8 +49,9 @@ def main(argv):
 
     results = []
     for svc in services:
-        if svc.get("wip"):
-            state = "wip"
+        wip = svc.get("wip")
+        if wip:
+            state = "wip-%s" % wip
         else:
             host, _, port = svc["probe"].rpartition(":")
             state = probe(host, int(port), svc.get("method", "tcp"))

scripts/patches/glm53-compile-01-model-decorator.patch

@@ -0,0 +1,86 @@
+# glm53-compile-01-model-decorator.patch — put the GLM-5.3-Flash trunk under torch.compile.
+#
+# STATUS: WIP — first of three patches in the "compile the GLM trunk" changelist. The
+# set does NOT yet boot the engine (see 03 for the next blocker), so it is NOT part of
+# the deployed frank-primary shape. Nothing in scripts/ mounts these files.
+#
+# WHY THIS IS THE RIGHT LEVER (measured 2026-09-30, not inferred):
+#
+# The fork's own flagship trunk — vllm/models/qwen4_exp/amd/model.py — carries
+# @support_torch_compile. vllm/models/glm5next/common/model.py does not, and the engine
+# says so on every boot:
+#
+#   Cudagraph mode FULL_AND_PIECEWISE requires piecewise capture, but the loaded model
+#   provides neither a compiled submodule nor breakable CUDA graphs.
+#   Overriding to FULL_DECODE_ONLY.
+#
+# A torch-profiler window on the live endpoint (496 decode tokens, rank0, 3 x 160-token
+# generations at the deployed shape) shows what the missing compilation costs. Total
+# device kernel time 46.18 s / 496 tokens = ~93 ms per token, against a measured step
+# time of ~104 ms (9.6 tok/s with the profiler attached):
+#
+#   16.07 s  34.8 %  708,500 launches  vectorized_elementwise_kernel      (22.7 us each)
+#    4.60 s  10.0 %    42,000           r4d_gemm_moe_w4a16_nt_b16_m64     (the MoE GEMM)
+#    3.11 s   6.7 %   174,708           elementwise_kernel_manual_unroll
+#    2.94 s   6.4 %   138,252           wvSplitK_hf_sml_                  (split-K GEMM)
+#    4.09 s   8.9 %    83,160           topk/router (sbtopk gatherTopK + warpMergeSortTopK
+#                                       + bitonicSortKVInPlace)
+#    3.01 s   6.5 %   133,920           r4d_mhc_* (hyper-connection pre/post/finish)
+#    2.78 s   6.0 %   127,916           inductor triton_poi_* / triton_per_* (the only
+#                                       fusions that DO happen)
+#    0.49 s   1.05%    20,664           r4d_moe_lru_gather_kernel         <-- the suspect
+#
+# That last row retired the fleet's working hypothesis. Decode on this shape is NOT
+# expert-gather bound: the gather is 1 % of device time, the same order as the telemetry
+# kernel that was switched on to measure it. 44 % of the budget is pointwise glue —
+# bf16<->fp32 conversions, adds, pows/rsqrts from fp32 RMSNorm, clamp/scale quant
+# passes — spread over ~1,428 launches per token. Inductor fusion is the only lever
+# that touches that mass; the router is second at 9 % but is binary-blocked (the
+# libr4d route kernel exists only as r4d_moe_route_kernel<128|256|512> instantiations
+# and GLM has 288 experts, with no HIP sources shipped to rebuild).
+#
+# TARGET: vllm/models/glm5next/common/model.py   (inside the pinned image)
+#   base    sha256 008eeaa2a07e091465319f86900b5a321d921238723ccbe22e6b9e971e1d2b28
+#   patched sha256 6e9cb81b84a9a88bd41403ca404aa04c3b791899dd77d661c3f25ae686cecd40
+#
+# APPLY: `patch -p1` from the vllm package root, or bind-mount new/.../model.py over
+# site-packages. Then R4D_GLM_COMPILE=1 (default) enables it; =0 makes the decorator a
+# no-op, so the file can stay mounted for an A/B without a second artifact.
+# Requires 02 (trace-safe probes) — without it the worker dies during profile_run on
+# the first probe call inside the traced region.
+--- a/vllm/models/glm5next/common/model.py
++++ b/vllm/models/glm5next/common/model.py
+@@ -1,12 +1,14 @@
+ # SPDX-License-Identifier: Apache-2.0
+ # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+ 
++import os
+ from collections.abc import Iterable
+ from typing import ClassVar, Literal
+ 
+ import torch
+ from torch import nn
+ 
++from vllm.compilation.decorators import support_torch_compile
+ from vllm.config import ParallelConfig, VllmConfig
+ from vllm.distributed import (
+     get_ep_group,
+@@ -623,6 +625,18 @@
+         )
+ 
+ 
++_GLM_COMPILE = os.environ.get("R4D_GLM_COMPILE", "1") == "1"
++
++
++@support_torch_compile(
++    dynamic_arg_dims={
++        "input_ids": 0,
++        "positions": -1,
++        "intermediate_tensors": 0,
++        "inputs_embeds": 0,
++    },
++    enable_if=lambda _cfg: _GLM_COMPILE,
++)
+ class Glm5NextModel(nn.Module):
+     def __init__(self, *, vllm_config: VllmConfig, prefix: str = ""):
+         super().__init__()

scripts/patches/glm53-compile-02-trace-safe-probes.patch

@@ -0,0 +1,119 @@
+# glm53-compile-02-trace-safe-probes.patch — libr4d capability probes must not do I/O
+# inside a torch.compile region.
+#
+# STATUS: WIP — second of three patches in the "compile the GLM trunk" changelist.
+# Inert (but harmless) without 01. Not part of the deployed shape.
+#
+# THE BREAK (measured: with 01 mounted and R4D_GLM_COMPILE=1, the worker dies during
+# profile_run, i.e. the first — cold — forward):
+#
+#   Explanation: Dynamo does not know how to trace the builtin `posix.stat`
+#   ... r4d_mhc_supported -> r4d_lib.r4d_mhc_enabled -> available -> os.path.exists
+#
+# THE NON-OBVIOUS PART: functools.lru_cache does NOT protect these probes. Dynamo
+# re-traces the wrapped body to build its guard even when the cache is warm — it warns
+# about exactly this ("Dynamo detected a call to a `functools.lru_cache`-wrapped
+# function ... Dynamo ignores the cache wrapper and directly traces the wrapped
+# function"). Verified in-image, torch 2.11.0+rocm10.0, torch.compile(backend="eager"):
+#
+#   cold lru_cache call       -> RAISE Unsupported
+#   WARM lru_cache call       -> RAISE Unsupported     <-- warming is not enough
+#   plain module-global read  -> compiles
+#   uncached os.path.exists   -> RAISE Unsupported
+#
+# So each probe must resolve ONCE into a plain value and be read as a global. The set
+# below is every no-argument probe / symbol lookup reachable from a forward pass,
+# grep-verified call sites:
+#   model_executor/layers/fused_moe/experts/r4d_w4a16_moe.py:79  available()
+#   model_executor/layers/fused_moe/experts/r4d_w4a16_moe.py:81  has_w4a16_moe()
+#   model_executor/kernels/mhc/r4d_dsv4.py:16                    r4d_mhc_enabled()
+#   v1/attention/ops/rocm_aiter_mla_sparse.py:694                r4d_indexer_enabled(), mqa_max_heads()
+#   v1/attention/ops/rocm_aiter_mla_sparse.py:1124               r4d_indexer_enabled()
+#   model_executor/layers/fused_moe/experts/r4d_mxfp4_moe.py:683 r4d_act_quant_enabled()
+#   model_executor/layers/fused_moe/experts/r4d_w4a16_moe.py _entry()  moe_w4a16()/moe_w4a16_mt1() per MoE call
+# (lib() is the transitive dependency of all of them.)
+#
+# TARGET: vllm/model_executor/kernels/r4d_lib.py   (inside the pinned image)
+#   base    sha256 b8dfeefb4acc72b67b0579a4ab0653ef5d48c40cc8649187d757c80ce6b9617a
+#   patched sha256 0751852e1ca813ad2bdb1da7bf10c605130e63e04ef9ec57e70b5f675a29c08e
+#
+# APPLY: append the block below to the module. R4D_FREEZE_PROBES=0 restores stock
+# behaviour. Cost when nothing is compiled: ten one-time probes at import (measured:
+# logged once per process, e.g. "libr4d capability probes frozen: 10 warm [...]").
+#
+# OBSERVED EFFECT: with 01+02 the worker gets past this break and reaches the next one
+# (see 03) — dynamo reports "Dynamo bytecode transform time: 1.08 s" for the trunk.
+--- a/vllm/model_executor/kernels/r4d_lib.py
++++ b/vllm/model_executor/kernels/r4d_lib.py
+@@ -361,3 +361,71 @@
+ @functools.lru_cache
+ def moe_w4a16_mt1():
+     return getattr(lib(), "r4d_gemm_moe_w4a16_nt_b16_m64_mt1", None)
++
++
++# FORK-LOCAL (patch: trace-safe capability probes).  torch.compile traces the
++# BODY of a functools.lru_cache call even when the cache is warm, so a capability
++# probe called from a forward pass is a dynamo graph break.  Measured 2026-09-30
++# on the GLM-5.3-Flash trunk under @support_torch_compile:
++#   Dynamo does not know how to trace the builtin `posix.stat`
++#   r4d_mhc_supported -> r4d_mhc_enabled -> available -> os.path.exists
++# Each probe below is wrapped to resolve ONCE into a memo and is warmed at import
++# (outside any tracing region), so a traced call is a dict read.  The probe set is
++# every no-argument capability probe / symbol lookup reachable from a forward
++# pass: the W4A16 + mxfp4 MoE dispatch, the mHC op, the MLA sparse indexer gate
++# and the draft/lm-head guards.
++_PROBE_NAMES = (
++    "available",
++    "lib",
++    "has_w4a16_moe",
++    "moe_w4a16",
++    "moe_w4a16_mt1",
++    "has_mhc_dsv4",
++    "r4d_mhc_enabled",
++    "r4d_indexer_enabled",
++    "mqa_max_heads",
++    "r4d_act_quant_enabled",
++)
++
++
++def _freeze_probe(fn):
++    """Return fn bound to its first result.  Result must be a plain value."""
++
++    memo = []
++
++    def probe(*args, **kwargs):
++        if not memo:
++            memo.append(fn(*args, **kwargs))
++        return memo[0]
++
++    return probe
++
++
++def _freeze_probes() -> list[str]:
++    frozen = []
++    for name in _PROBE_NAMES:
++        fn = globals().get(name)
++        if fn is None:
++            continue
++        # Only the fast branch of a bound probe is traced; an empty memo means the
++        # first call still does I/O, so populate it here as well.
++        globals()[name] = _freeze_probe(fn)
++        frozen.append(name)
++    return frozen
++
++
++from vllm.logger import init_logger as _init_logger
++
++_probe_log = _init_logger(__name__)
++
++if os.environ.get("R4D_FREEZE_PROBES", "1") == "1":
++    _warmed, _failed = [], []
++    for _n in _freeze_probes():
++        try:
++            globals()[_n]()
++            _warmed.append(_n)
++        except Exception:  # noqa: BLE001 -- probe unavailable here; recompute lazily
++            _failed.append(_n)
++    _probe_log.info("libr4d capability probes frozen: %d warm %s%s",
++                    len(_warmed), _warmed,
++                    f" (deferred: {_failed})" if _failed else "")

scripts/patches/glm53-compile-03-fullgraph-optout.patch

@@ -0,0 +1,116 @@
+# glm53-compile-03-fullgraph-optout.patch — let torch.compile split the graph instead
+# of failing on a break.
+#
+# STATUS: WIP — third of three patches in the "compile the GLM trunk" changelist.
+# With 01+02+03 the model TRACES, then hits the NEXT blocker (below). The set does not
+# yet boot the engine; nothing in scripts/ mounts it.
+#
+# THE BREAK: with 01+02 mounted the worker dies with
+#
+#   RuntimeError: Graph breaks are not supported with aot compile.
+#   Please use torch.compile(fullgraph=True).
+#
+# because vllm/compilation/wrapper.py calls torch.compile(..., fullgraph=True, ...) and
+# this build has AOT compile on by default (VLLM_USE_AOT_COMPILE). The GLM trunk cannot
+# satisfy fullgraph: its indexer metadata path syncs to the host —
+#   models/glm5next/amd/sparse_indexer.py:263  int(positions[...].max().item()) + 1
+#   models/glm5next/amd/sparse_indexer.py:419  lmax = int(decode_metadata.decode_lens.max().item())
+# — and a .item() inside a captured graph is not expressible.
+#
+# WHAT WAS VERIFIED (in-image, torch 2.11.0+rocm10.0, torch.compile(backend="eager")):
+#
+#   fullgraph=True   suppress_errors=False  -> RAISE Unsupported
+#   fullgraph=True   suppress_errors=True   -> RAISE Unsupported   <-- suppress_errors
+#                                                                     does NOT cover breaks
+#   fullgraph=False  suppress_errors=False  -> ok   (traceable segments compiled by
+#                                                    inductor, the rest runs eagerly;
+#                                                    numerics unchanged)
+#
+# So breaks must be TOLERATED, not suppressed: the traceable mass still fuses. This
+# patch makes the fullgraph requirement opt-out via VLLM_COMPILE_FULLGRAPH=0 (default 1,
+# so every other model keeps stock behaviour), and the run also needs
+# VLLM_USE_AOT_COMPILE=0 because AOT compile refuses breaks outright.
+#
+# TARGET: vllm/compilation/wrapper.py   (inside the pinned image)
+#   base    sha256 969961dca41c78385212e985cfc0c57ef067f4a2d7490c73c658c600a1b95cac
+#   patched sha256 732363bb1a1367772f13c60a655c678e7e2e9e56186677b14238f5e84cac31b3
+#   (the file already imports os; the patch does not add a second import)
+#
+# NEXT BLOCKER — NOT FIXED HERE. With 01+02+03 and
+# R4D_GLM_COMPILE=1 VLLM_COMPILE_FULLGRAPH=0 VLLM_USE_AOT_COMPILE=0 the trunk traces
+# ("Dynamo bytecode transform time: 1.08 s") and then the worker dies during profile_run
+# on the decode forward:
+#
+#   KeyError: 'in_proj_qkvbfg_a'
+#     model.py:748  Glm5NextModel.forward        hidden_states, residual, post, comb = layer(...)
+#     model.py:456  Glm5NextDecoderLayer.forward
+#
+# `in_proj_qkvbfg_a` is the KDA layer's merged column-parallel projection
+# (models/glm5next/common/kda.py:217, called at :407) — and it is also a name in the
+# load_weights shard table (model.py:780-785), i.e. a real submodule attribute name,
+# not an arbitrary string.
+#
+# WHERE IT IS THROWN (established 2026-09-30 by reading the full traceback, since the
+# report line is inside dynamo-rewritten code): torch/_dynamo keeps a name -> module map
+# and looks it up by name — output_graph.py:1259 / :1384 register into `nn_modules`,
+# :3000 does `self.nn_modules[node.target]` (a bare-name KeyError when absent), and
+# :1225 documents stripping "_buffers[..]/_parameters[..]/_modules[..]" from those
+# names. So the compiler is failing to RESOLVE A SUBMODULE BY NAME while tracing the
+# GLM decoder layer. Whether the root cause is the fork's module tree (something
+# dynamo cannot register statically) or dynamo's own registration is NOT yet settled.
+#
+# STATUS OF THAT BLOCKER (updated 2026-09-30, after the diagnostics below):
+#   * TORCHDYNAMO_VERBOSE=1 adds NO internal stack trace -> it is a plain Python
+#     KeyError raised in a callee dynamo INLINED into Glm5NextDecoderLayer.forward
+#     (the reported line 456 is that method's `def` line, which is how inlined frames
+#     surface).
+#   * It is NOT raised from dynamo's four nn_modules lookups: the probe in
+#     /home/dank/dynamo-probe wraps three of them and stayed silent over two runs with
+#     the probe confirmed installed in the API-server and worker processes.
+#   * It is NOT a literal string lookup: the name appears tree-wide only as the KDA
+#     layer's attribute (kda.py:217, used at :407) and in the load_weights shard table
+#     (model.py:780-785). The four `proj_name = prefix.split(".")[-1]` sites in the
+#     quantization utils do compute exactly this bare key, but each is guarded
+#     (`if proj_name not in fused_mapping: return`), so none of them can raise it.
+#   * It reproduces on every run of this configuration (5 of 5).
+# REMAINING INSTRUMENT: name the inlined caller. Either `TORCH_LOGS=+dynamo` on a boot
+# and grep the log for the key (dynamo prints inlined frames with source), or wrap the
+# inlining candidates in that forward — the mHC CustomOps (hc_pre / hc_post /
+# hc_fused_post_pre in layers/mhc.py) and the indexer op — with a KeyError-reporting
+# probe. Until that names the caller this is a dynamo-side name-resolution failure.
+# NOTE the launcher gotcha that wasted a boot earlier: the scratch launcher now
+# preserves ENVARGS (`ENVARGS="${ENVARGS:-}"`), but it previously reset it, silently
+# dropping outer env vars.
+#
+# SECOND BLOCKER, KNOWN AND NOT FATAL IN THIS CONFIG: the indexer's in-forward
+# workspace acquisition (patch 05 makes it compiler-opaque) and the fused-norm
+# launcher cell (patch 04 removes the call) are real breaks, but with
+# VLLM_COMPILE_FULLGRAPH=0 they are tolerated — they only become fatal under
+# fullgraph=True, which this trunk can never satisfy anyway.
+--- a/vllm/compilation/wrapper.py
++++ b/vllm/compilation/wrapper.py
+@@ -19,6 +19,15 @@
+ 
+ logger = init_logger(__name__)
+ 
++
++# FORK-LOCAL (patch: fullgraph opt-out).  torch.compile(fullgraph=True) turns any
++# dynamo graph break into a hard failure; suppress_errors does NOT cover breaks
++# (verified in-image, torch 2.11+rocm10.0).  With VLLM_COMPILE_FULLGRAPH=0 the
++# traceable segments are still compiled by inductor and the untraceable ones run
++# eagerly, which is what a model with device-sync metadata (the GLM-5.3-Flash
++# indexer) needs.  Default 1 keeps stock behaviour for every other model.
++_FULLGRAPH = os.environ.get("VLLM_COMPILE_FULLGRAPH", "1") == "1"
++
+ R = TypeVar("R")
+ P = ParamSpec("P")
+ 
+@@ -146,7 +155,7 @@
+         with aot_context:
+             self._compiled_callable = torch.compile(
+                 compiled_ptr,
+-                fullgraph=True,
++                fullgraph=_FULLGRAPH,
+                 dynamic=False,
+                 backend=backend,
+                 options=options,

scripts/patches/glm53-compile-04-fused-qkv-rmsnorm-optout.patch

@@ -0,0 +1,112 @@
+# glm53-compile-04-fused-qkv-rmsnorm-optout.patch — take GLM's fused-norm JIT kernel
+# out of the traced path.
+#
+# STATUS: WIP — fourth of the "compile the GLM trunk" changelist. Needs 01 (the
+# decorator) to matter; not part of the deployed shape.
+#
+# MEASUREMENT NOTE — read the run history before trusting any claim below, because a
+# launcher bug made four consecutive boots misleading.  The scratch launcher
+# (serve_glm_autoround.sh) was restructured on 2026-09-30 and the
+# `-e VLLM_COMPILE_FULLGRAPH=0` line was DROPPED, so those boots mounted the wrapper
+# patch while `_FULLGRAPH` still defaulted to True.  With fullgraph=True *every* break
+# is an error, which is why the reported failure kept moving as breaks were removed:
+#
+#   fullgraph=False, AOT off  -> breaks tolerated; dies later on the dynamo name-map
+#                                `KeyError: 'in_proj_qkvbfg_a'`
+#   fullgraph=True,  AOT off  -> dies on whichever break comes first
+#                                (cell -> workspace -> compiler-disable)
+#
+# An earlier version of this note blamed `TORCH_LOGS=graph_breaks` for the difference.
+# That was WRONG — the flag only logs; the launcher silently ran fullgraph=True.
+# Launcher fixed; the env var is set again.
+#
+# What it means for THIS patch: under fullgraph=True the fused-norm call was the first
+# break dynamo reported, and after this patch the report moved on to the workspace
+# acquisition — so it removes a real break.  It is not by itself what unblocks a boot:
+# with breaks genuinely tolerated the blocker is the name-map KeyError.
+#
+# IT IS STILL THE RIGHT PATTERN TO REMOVE, either way: the launcher family
+# (@kernel_launcher, 39 call sites) is exactly the shape that produces breaks, and the
+# object this one closes over is bound after use (see below).
+#
+# THE BREAK (measured, TORCH_LOGS=graph_breaks, rank0, 2026-09-30).  With 01+02+03 the
+# trunk traces, and with AOT off + fullgraph=True dynamo reports a FATAL break — fatal
+# even with VLLM_COMPILE_FULLGRAPH=0, because dynamo cannot build a resume function
+# around it:
+#
+#   torch._dynamo.exc.Unsupported: Read uninitialized cell
+#   Explanation: Attempted to read a cell variable that has not been populated yet.
+#   Developer debug context: CellVariable()
+#
+# User stack:
+#   glm5next/common/model.py:748    Glm5NextModel.forward       layer(...)
+#   glm5next/common/model.py:526    Glm5NextDecoderLayer.forward
+#   glm5next/common/attention.py:590
+#   model_executor/layers/mla.py:208
+#     q_proj_input, kv_c_normed = fused_q_kv_rmsnorm(...)
+#   -> torch/_dynamo/polyfills/__init__.py:392 getattr_and_trace
+#
+# So dynamo is tracing the fork's JIT-kernel launcher framework
+# (VllmTritonJitKernel + @kernel_launcher — 39 call sites) and finds a closure name
+# that is not yet bound.  In models/common/ops/fused_qk_rmsnorm.py the kernel object
+# is bound AFTER the function that closes over it:
+#
+#   def fused_q_kv_rmsnorm(...):          # line 194
+#       ...
+#       _FUSED_Q_KV_RMSNORM_KERNEL(...)   # line 211
+#   _FUSED_Q_KV_RMSNORM_KERNEL = FusedQKVRMSNormKernel()    # line 223
+#
+# Two ways out, and this patch takes the cheap one:
+#   (a) WORKAROUND (here): stop calling the fused kernel from the traced region.  The
+#       flag is already a constructor argument (mla.py:80, default False) that glm5next
+#       turns ON at attention.py:583, so it becomes an env knob.
+#   (b) ROOT FIX (not done): make the launcher trace-safe for all 39 sites.  Any other
+#       @kernel_launcher-backed kernel still in the traced path can produce the same
+#       fatal break, so (b) is the durable fix and (a) is what unblocks iterating.
+#
+# COST of R4D_GLM_FUSE_QKV_RMSNORM=0: one extra small RMSNorm launch per MLA layer per
+# token (q_a_layernorm + kv_a_layernorm instead of one fused kernel) — ~2 launches out
+# of ~3,700 per token on this shape, against the 44 % of device time currently spent in
+# unfused pointwise work.  Correct trade for a compile experiment.
+#
+# TARGET: vllm/models/glm5next/common/attention.py   (inside the pinned image)
+#   base    sha256 362bbf0e6c50cdfa446c6551b9b1ddff425e08f88e3d4140fd8db88f8a619e0c
+#   patched sha256 84969993c250e9f8183a33a6e1ddd607272f48b950b560272eb33b7cdde31189
+#
+# APPLY: `patch -p1` from the vllm package root (verified: applies clean, reproduces
+# the patched sha256), or bind-mount new/.../attention.py over site-packages.
+# R4D_GLM_FUSE_QKV_RMSNORM=1 (default) restores the fused kernel.
+--- a/vllm/models/glm5next/common/attention.py
++++ b/vllm/models/glm5next/common/attention.py
+@@ -1,6 +1,7 @@
+ # SPDX-License-Identifier: Apache-2.0
+ # SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+ 
++import os
+ import torch
+ import torch.nn.functional as F
+ from torch import nn
+@@ -36,6 +37,14 @@
+ from vllm.utils.deep_gemm import PAGED_MQA_PAGE_SIZES
+ from vllm.v1.kv_cache_interface import KpoolTailSpec, MLAAttentionSpec
+ 
++
++# FORK-LOCAL (patch: keep the traced region trace-safe).  R4D_GLM_FUSE_QKV_RMSNORM=0
++# replaces the fused `fused_q_kv_rmsnorm` JIT-kernel launch (one kernel) with two plain
++# RMSNorms, because dynamo cannot trace the launcher's closure ("Read uninitialized cell",
++# fatal even with fullgraph=False).  The fused kernel object in
++# models/common/ops/fused_qk_rmsnorm.py is bound AFTER the function that uses it.
++_FUSE_QKV_RMSNORM = os.environ.get("R4D_GLM_FUSE_QKV_RMSNORM", "1") == "1"
++
+ logger = init_logger(__name__)
+ 
+ # Shared torch.compile config for the indexer's small-kernel leaves. The MLA
+@@ -580,7 +589,7 @@
+             quant_config,
+             prefix,
+             skip_topk=False,
+-            fuse_qkv_rmsnorm=True,
++            fuse_qkv_rmsnorm=_FUSE_QKV_RMSNORM,
+         )
+ 
+     def forward(

scripts/patches/glm53-compile-05-workspace-acquisition-opaque.patch

@@ -0,0 +1,96 @@
+# glm53-compile-05-workspace-acquisition-opaque.patch — the indexer's workspace
+# acquisition must not be traced.
+#
+# STATUS: WIP — fifth of the "compile the GLM trunk" changelist. This is a break worth
+# removing, but NOT part of the confirmed blocker and NOT required for the
+# break-tolerant route: `torch.compiler.disable` is itself a graph break, so this patch
+# helps only a hypothetical fullgraph=True route, where the workspace acquisition would
+# be better RESTRUCTURED out of the forward (allocate at init/profile time) than made
+# opaque. Kept because the discovery is real and the wrapper is a no-op when nothing is
+# compiling.
+#
+# THE BREAK — CONFIRMED with AND WITHOUT the logging flag (2026-09-30):
+#
+#   torch._dynamo.exc.Unsupported: Unsupported method call
+#   user stack
+#     glm5next/common/model.py:748      Glm5NextModel.forward
+#     glm5next/common/model.py:526      Glm5NextDecoderLayer.forward
+#     glm5next/common/attention.py:599 / :401
+#     glm5next/amd/sparse_indexer.py:711  forward_native
+#     glm5next/amd/sparse_indexer.py:132  sparse_attn_indexer_kpool
+#     v1/worker/workspace.py:135          get_simultaneous
+#     v1/worker/workspace.py:155          _ensure_workspace_size
+#
+# Line 132 is the PROFILE-RUN branch — the source comment says what it is:
+#
+#     if not isinstance(attn_metadata, dict):
+#         # Reserve workspace for indexer during profiling run
+#
+# so the engine dies exactly where it sizes its memory.  `_ensure_workspace_size`
+# reads ContextVar and ubatch state (`_workspace_lane.get()`,
+# `dbo_current_ubatch_id()`) and may allocate; none of that is traceable.
+#
+# FIX: the documented escape hatch — `torch.compiler.disable` around a thin wrapper,
+# so dynamo stops analysing the call and executes it eagerly from inside the compiled
+# region.  Chosen over decorating the workspace manager's own methods because that
+# would change workspace behaviour for every model, not just this one.  The wrapper is
+# a no-op when nothing is compiling, so it needs no env gate and is safe in the
+# stock (uncompiled) path — which is why this patch can be reasoned about
+# independently of whether the changelist ever ships.
+#
+# CAVEAT, stated plainly: a compiler-disabled region is still a graph break, so this
+# needs patch 03 (VLLM_COMPILE_FULLGRAPH=0) to be tolerable at all, and whether dynamo
+# can build a resume function around it is exactly what the next boot decides — the
+# difference between this failure and 04's `Read uninitialized cell`.
+#
+# TARGET: vllm/models/glm5next/amd/sparse_indexer.py   (inside the pinned image)
+#   base    sha256 f773a99265d3f440c8fad22c8fb5a8b33f926162094cffc3a2ecdaa42763c5b1
+#   patched sha256 c154115afe8253a35c652b7198929ef4c1874957d333fdb3340cacf88e612b78
+#
+# APPLY: `patch -p1` from the vllm package root, or bind-mount over site-packages.
+--- a/vllm/models/glm5next/amd/sparse_indexer.py
++++ b/vllm/models/glm5next/amd/sparse_indexer.py
+@@ -35,6 +35,21 @@
+ from vllm.v1.attention.ops.common import pack_seq_triton, unpack_seq_triton
+ from vllm.v1.worker.workspace import current_workspace_manager
+ 
++@torch.compiler.disable
++def _acquire_workspace(*shapes_and_dtypes):
++    """Acquire shared workspace buffers, untraced.
++
++    Workspace acquisition reads ContextVar/ubatch state and may allocate
++    (v1/worker/workspace.py: ``_ensure_workspace_size`` -> ``_workspace_lane.get()``,
++    ``dbo_current_ubatch_id()``), which dynamo cannot trace -- it raises
++    ``Unsupported: Unsupported method call`` and kills the engine during the profile
++    run.  ``torch.compiler.disable`` keeps the call as an opaque, eagerly-executed
++    region inside the compiled graph.  No-op when not compiling.
++    """
++
++    return current_workspace_manager().get_simultaneous(*shapes_and_dtypes)
++
++
+ logger = init_logger(__name__)
+ 
+ # kpool write helper: form pools from the current token batch and compress them
+@@ -129,7 +144,7 @@
+         values_spec, scales_spec = _gather_workspace_shapes(
+             total_seq_lens, head_dim, fp8_dtype, use_fp4_cache
+         )
+-        current_workspace_manager().get_simultaneous(
++        _acquire_workspace(
+             values_spec,
+             scales_spec,
+             ((RADIX_TOPK_WORKSPACE_SIZE,), torch.uint8),
+@@ -275,11 +290,10 @@
+         # Layout switches between FP8 (head_dim bytes + 4-byte fp32 scale) and
+         # MXFP4 (head_dim/2 bytes packed + head_dim/MXFP4_BLOCK_SIZE ue8m0
+         # scales) based on use_fp4_cache.
+-        workspace_manager = current_workspace_manager()
+         values_spec, scales_spec = _gather_workspace_shapes(
+             total_seq_lens, head_dim, fp8_dtype, use_fp4_cache
+         )
+-        k_quant_full, k_scale_full = workspace_manager.get_simultaneous(
++        k_quant_full, k_scale_full = _acquire_workspace(
+             values_spec,
+             scales_spec,
+         )

scripts/patches/glm53-dflash-01-aux-hidden-states.patch

@@ -0,0 +1,173 @@
+# glm53-dflash-01-aux-hidden-states.patch — give the GLM trunk the EAGLE3 / DFlash2
+# auxiliary-hidden-state interface.
+#
+# STATUS: WIP, and INDEPENDENT of the compile changelist (01-05). It patches the same
+# file as glm53-compile-01-model-decorator.patch, so if both are ever applied, apply 01
+# first. Not part of the deployed shape.
+#
+# WHY.  `--speculative-config '{"method":"dflash","model":"/path/to/drafter", ...}'` with
+# incoai/GLM-5.3-Flash-DFlash2 is accepted -- the config parses, the drafter loads -- and
+# then the worker dies at init with:
+#
+#     RuntimeError: Model does not support EAGLE3 interface
+#
+# raised by v1/worker/gpu/spec_decode/eagle/eagle3_utils.py:20 through
+# `supports_eagle3(model)` == `isinstance(model, SupportsEagle3)` (a runtime-checkable
+# Protocol).  Nothing about the drafter, the method name or the payload is wrong:
+# `glm5next` implements neither half of the contract.
+#
+# THE CONTRACT (read out of this image, not assumed):
+#   * DEFINITIVE (v5, after four failed boots): the member set is checked against
+#     the INSTANCE the runner calls -- which for GLM-5.3-Flash is
+#     Glm5NextForConditionalGeneration, NOT Glm5NextForCausalLM.  The multimodal
+#     wrapper wraps the text model as self.language_model
+#     (init_vllm_registered_model, architectures=["Glm5NextForCausalLM"],
+#     glm5next/common/model.py:1113), and model_runner.py:425 calls
+#     set_eagle3_aux_hidden_state_layers(self.model) on the CONDITIONAL.
+#     Presence rule (verified in-image with stock typing, python 3.14):
+#     isinstance walks ALL __protocol_attrs__ -- members of SupportsEagle3 AND
+#     its base SupportsEagleBase (has_own_lm_head, has_own_embed_tokens) -- and
+#     getattr_static()s each one off the instance, breaking on the first miss.
+#     So BOTH wrappers need all three attributes.
+#   * MRO law: INHERITING SupportsEagle3 as a base dies at class creation with
+#     "Cannot create a consistent method resolution order (MRO) for bases ..."
+#     (GLM's wrappers already carry four/five Protocol-derived bases).  The
+#     attribute route avoids the MRO entirely.  Also proven by simulation:
+#     the four-base ForCausalLM ordering cannot take SupportsEagle3 in ANY
+#     position.
+#   * Method placement: without inheriting, the Protocol's default
+#     set_aux_hidden_state_layers / get_eagle3_default_aux_hidden_state_layers
+#     do not exist on either wrapper, so both are defined explicitly --
+#     ForCausalLM unwraps to self.model (the EagleModelMixin trunk, v1's
+#     correct bodies), the conditional delegates to self.language_model
+#     (the pixtral.py:498 pattern).
+#   * interfaces.py:1673-1719 -- for reference, the inherited defaults unwrap
+#     `holder = getattr(parent_ref, "model", parent_ref)` and assert
+#     `isinstance(holder, EagleModelMixin)`, defaulting to (2, L // 2, L - 3).
+#   * eagle3_utils.py:26-31 -- layers come from the spec config when it carries them
+#     (this drafter ships target_layer_ids [5, 14, 24, 33, 42]), else from the default.
+#   * deepseek_v2.py:1551-1585 and v1/worker/gpu/cudagraph_utils.py:621 -- the trunk
+#     appends one tensor per aux layer while looping and returns the TUPLE
+#     (hidden_states, aux_hidden_states) only when that list is non-empty.
+#
+# WHAT IT ADDS for GLM: EagleModelMixin on Glm5NextModel; aux capture in the layer
+# loop, where the aux state is the single-stream `hidden_states` ENTERING that layer
+# (GLM's loop carries the already-summed single-stream state -- its non-mHC branch
+# re-splits `residual = hidden_states` inside the layer -- so this is the equivalent
+# of deepseek_v2's `hidden_states + residual`); the conditional tuple return; the
+# three SupportsEagle3 member attributes on BOTH wrappers; and the two interface
+# methods on both (ForCausalLM: unwrap to trunk; conditional: delegate to
+# language_model).
+#
+# HONEST SCOPE: this makes the interface PRESENT, which is exactly what the engine
+# checks -- it does not make drafts GOOD, and per the measured profile it is not a
+# decode-speed fix (a verification pass pays pointwise cost proportional to its token
+# count, which is why MTP4 measured flat).  It is the prerequisite for driving DFlash2
+# on this endpoint at all.
+#
+# TARGET: vllm/models/glm5next/common/model.py   (inside the pinned image)
+#   base    sha256 008eeaa2a07e091465319f86900b5a321d921238723ccbe22e6b9e971e1d2b28
+#   patched sha256 bc9913a15c8e7f88e0c36b07bf029b383cc8c2078b58d66dc91aa235f8af77f4
+#
+# APPLY: `patch -p1` from the vllm package root, or bind-mount the patched file over
+# site-packages (which is what a scratch-launcher DFLASH_PATCH=1 does).
+--- a/vllm/models/glm5next/common/model.py
++++ b/vllm/models/glm5next/common/model.py
+@@ -95,6 +95,12 @@
+ logger = init_logger(__name__)
+ 
+ 
++from vllm.model_executor.models.interfaces import (
++    EagleModelMixin,
++    SupportsEagle3,
++)
++
++
+ class Glm5NextMLP(nn.Module):
+     def __init__(
+         self,
+@@ -623,7 +629,7 @@
+         )
+ 
+ 
+-class Glm5NextModel(nn.Module):
++class Glm5NextModel(nn.Module, EagleModelMixin):
+     def __init__(self, *, vllm_config: VllmConfig, prefix: str = ""):
+         super().__init__()
+ 
+@@ -730,7 +736,14 @@
+         if self.is_sequence_parallel:
+             hidden_states = sp_shard(hidden_states)
+ 
+-        for layer in self._active_layers:
++        aux_hidden_states: list[torch.Tensor] = []
++        for local_idx, layer in enumerate(self._active_layers):
++            if (getattr(self, "start_layer", 0) + local_idx) in self.aux_hidden_state_layers:
++                # Auxiliary hidden state for EAGLE-3 / DFlash2 drafts: the
++                # single-stream state ENTERING this layer, before its input norm --
++                # the same semantic deepseek_v2 implements by appending
++                # hidden_states + residual at this point in its loop.
++                aux_hidden_states.append(hidden_states)
+             hidden_states, residual, post, comb = layer(
+                 positions, hidden_states, residual, post, comb
+             )
+@@ -749,6 +762,8 @@
+             hidden_states = sp_all_gather(hidden_states)[:full_num_tokens]
+ 
+         hidden_states = self.norm(hidden_states)
++        if len(aux_hidden_states) > 0:
++            return hidden_states, aux_hidden_states
+         return hidden_states
+ 
+     def load_weights(self, weights: Iterable[tuple[str, torch.Tensor]]) -> set[str]:
+@@ -937,6 +952,10 @@
+ class Glm5NextForCausalLM(
+     nn.Module, HasInnerState, SupportsPP, MixtureOfExperts, IsHybrid
+ ):
++    supports_eagle3: ClassVar[Literal[True]] = True
++    has_own_lm_head: bool = False
++    has_own_embed_tokens: bool = False
++
+     def __init__(self, *, vllm_config: VllmConfig, prefix: str = ""):
+         super().__init__()
+         self.model_config = vllm_config.model_config
+@@ -963,6 +982,14 @@
+     def embed_input_ids(self, input_ids: torch.Tensor) -> torch.Tensor:
+         return self.model.embed_input_ids(input_ids)
+ 
++    def set_aux_hidden_state_layers(self, layers: tuple[int, ...]) -> None:
++        # EAGLE-3 / DFlash2 interface: unwrap to the trunk (the EagleModelMixin).
++        self.model.aux_hidden_state_layers = layers
++
++    def get_eagle3_default_aux_hidden_state_layers(self) -> tuple[int, ...]:
++        num_layers = len(self.model.layers)
++        return (2, num_layers // 2, num_layers - 3)
++
+     def forward(
+         self,
+         input_ids: torch.Tensor | None,
+@@ -1033,6 +1060,10 @@
+ class Glm5NextForConditionalGeneration(
+     Glm4vForConditionalGeneration, HasInnerState, IsHybrid, MixtureOfExperts
+ ):
++    supports_eagle3: ClassVar[Literal[True]] = True
++    has_own_lm_head: bool = False
++    has_own_embed_tokens: bool = False
++
+     # The text model (KDA + dense-MLA + MoE) is a hybrid mamba model. The
+     # multimodal wrapper must declare the same interfaces so vLLM treats it as
+     # hybrid (auto-aligns mamba/attention block sizes, sizes the mamba state
+@@ -1076,6 +1107,13 @@
+ 
+         return Glm5NextForCausalLM.get_mamba_state_copy_func()
+ 
++    def set_aux_hidden_state_layers(self, layers: tuple[int, ...]) -> None:
++        # EAGLE-3 / DFlash2: delegate to the language model (pixtral.py:498).
++        self.language_model.set_aux_hidden_state_layers(layers)
++
++    def get_eagle3_default_aux_hidden_state_layers(self) -> tuple[int, ...]:
++        return self.language_model.get_eagle3_default_aux_hidden_state_layers()
++
+     def __init__(self, *, vllm_config: VllmConfig, prefix: str = ""):
+         super(Glm4vForConditionalGeneration, self).__init__()
+         config = vllm_config.model_config.hf_config

scripts/patches/glm53-dflash-04-kv-groups-draft.patch

@@ -0,0 +1,317 @@
+--- a/vllm/v1/core/kv_cache_utils.py
++++ b/vllm/v1/core/kv_cache_utils.py
+@@ -1210,17 +1210,38 @@
+         for name, spec in kv_cache_spec.items()
+         if isinstance(spec, KpoolTailSpec)
+     }
++    hidden_specs = {
++        name: spec
++        for name, spec in kv_cache_spec.items()
++        if isinstance(spec, HiddenStateCacheSpec)
++    }
+     attn_specs = {
+         name: spec
+         for name, spec in kv_cache_spec.items()
+-        if not isinstance(spec, (MambaSpec, KpoolTailSpec))
++        if not isinstance(
++            spec, (MambaSpec, KpoolTailSpec, HiddenStateCacheSpec)
++        )
+     }
+-    if not mamba_specs or not all(
+-        type(spec) is MLAAttentionSpec for spec in attn_specs.values()
+-    ):
++    if not mamba_specs or not attn_specs:
++        return None
++    # FORK-LOCAL (patches/glm53-dflash-04-kv-groups-draft): with a dflash
++    # drafter, attn_specs also carries the drafter's attention layers. The
++    # GLM MLA layers keep their own group (no page unification -- per-group
++    # pages are the glm5 design); the drafter layers must form one uniform
++    # group of their own, and the hidden-state layers get per-layer groups
++    # like the generic tail.
++    mla_specs = {
++        name: spec
++        for name, spec in attn_specs.items()
++        if type(spec) is MLAAttentionSpec
++    }
++    draft_specs = {
++        name: spec for name, spec in attn_specs.items() if name not in mla_specs
++    }
++    if not mla_specs:
++        return None
++    if draft_specs and UniformTypeKVCacheSpecs.from_specs(draft_specs) is None:
+         return None
+-
+-    mla_specs = cast(dict[str, MLAAttentionSpec], attn_specs)
+     idx_pages = {
+         spec.page_size_bytes for spec in mla_specs.values() if spec.tokens_per_state > 1
+     }
+@@ -1233,7 +1254,7 @@
+     mla_pages = {mla_specs[name].page_size_bytes for name in mla_names}
+     assert len(mla_pages) == 1
+     mla_page = mla_pages.pop()
+-    uniform_spec = UniformTypeKVCacheSpecs.from_specs(attn_specs)
++    uniform_spec = UniformTypeKVCacheSpecs.from_specs(mla_specs)
+     assert uniform_spec is not None
+ 
+     tail_group: KVCacheGroupSpec | None = None
+@@ -1270,10 +1291,20 @@
+     for index, name in enumerate(mamba_specs):
+         mamba_grouped_names[index % num_groups].append(name)
+ 
++    draft_group: KVCacheGroupSpec | None = None
++    if draft_specs:
++        draft_uniform = UniformTypeKVCacheSpecs.from_specs(draft_specs)
++        assert draft_uniform is not None
++        draft_group = KVCacheGroupSpec(list(draft_specs), draft_uniform)
++    hidden_groups = [
++        KVCacheGroupSpec([name], spec) for name, spec in hidden_specs.items()
++    ]
+     return (
+-        [KVCacheGroupSpec(list(attn_specs), uniform_spec)]
++        [KVCacheGroupSpec(list(mla_specs), uniform_spec)]
+         + ([tail_group] if tail_group is not None else [])
++        + ([draft_group] if draft_group is not None else [])
+         + create_kv_cache_group_specs(padded_specs, mamba_grouped_names)
++        + hidden_groups
+     )
+ 
+ 
+@@ -1289,10 +1320,17 @@
+         int,
+         list[str],
+         int,
++        KVCacheGroupSpec | None,
++        list[KVCacheGroupSpec],
+     ]
+     | None
+ ):
+-    """Recognize the GLM-5.3-Flash grouping after optional PP projection."""
++    """Recognize the GLM-5.3-Flash grouping after optional PP projection.
++
++    FORK-LOCAL (patches/glm53-dflash-04-kv-groups-draft): also recognizes the
++    drafter's uniform group and the per-layer hidden-state groups, returned
++    as ``draft_group`` / ``hidden_groups`` so the pool builder and the memory
++    estimator can size their tensors beside the aliased GLM pool."""
+     uniform_groups = [
+         group
+         for group in kv_cache_groups
+@@ -1301,17 +1339,28 @@
+     mamba_groups = [
+         group for group in kv_cache_groups if isinstance(group.kv_cache_spec, MambaSpec)
+     ]
++    hidden_groups = [
++        group
++        for group in kv_cache_groups
++        if isinstance(group.kv_cache_spec, HiddenStateCacheSpec)
++    ]
+     attn_group: KVCacheGroupSpec | None = None
+     tail_group: KVCacheGroupSpec | None = None
++    draft_group: KVCacheGroupSpec | None = None
+     for group in uniform_groups:
+         inner = cast(UniformTypeKVCacheSpecs, group.kv_cache_spec).kv_cache_specs
+         if all(type(spec) is MLAAttentionSpec for spec in inner.values()):
+             attn_group = group
+         elif all(isinstance(spec, KpoolTailSpec) for spec in inner.values()):
+             tail_group = group
++        else:
++            draft_group = group
+     if attn_group is None or not mamba_groups:
+         return None
+-    if len(uniform_groups) + len(mamba_groups) != len(kv_cache_groups):
++    if (
++        len(uniform_groups) + len(mamba_groups) + len(hidden_groups)
++        != len(kv_cache_groups)
++    ):
+         return None
+ 
+     attn_uniform = cast(UniformTypeKVCacheSpecs, attn_group.kv_cache_spec)
+@@ -1362,6 +1411,8 @@
+         idx_page,
+         tail_names,
+         tail_page,
++        draft_group,
++        hidden_groups,
+     )
+ 
+ 
+@@ -1395,10 +1446,34 @@
+         return kv_cache_spec
+ 
+     max_page_size = max(page_sizes)
++    # GLM-5.3-Flash + a draft model: the sparse indexer k_cache page is
++    # tokens_per_state x the MLA page (same per-token bytes), and MLA layers
++    # cannot be PADDED to an arbitrary page size (sparse MLA indexes the cache
++    # in whole token rows) -- but they CAN grow their block size when the max
++    # page is an exact multiple of the MLA page. Bump the max page to a
++    # multiple of every MLA-family page so all of them take the divisible
++    # block-growth branch below instead of the padding refusal, while the
++    # draft and mamba layers simply pad to the (slightly larger) page.
++    mla_pages = {
++        spec.page_size_bytes
++        for spec in kv_cache_spec.values()
++        if isinstance(spec, MLAAttentionSpec)
++    }
++    if mla_pages:
++        mla_common = math.lcm(*mla_pages)
++        max_page_size = -(-max_page_size // mla_common) * mla_common
+     new_kv_cache_spec = {}
+     for layer_name, layer_spec in kv_cache_spec.items():
+         if layer_spec.page_size_bytes == max_page_size:
+             new_kv_cache_spec[layer_name] = layer_spec
++        elif isinstance(layer_spec, KpoolTailSpec):
++            # The kpool tail kernel pins the tail block to the kpool size
++            # (tail_kv_cache.view(...) must be [blocks, 2, index_kpool,
++            # head_dim]); growing block_size breaks that contract. Pad the
++            # physical page instead -- the tail is its own group and the pad
++            # wastes bytes only.
++            new_spec = replace(layer_spec, page_size_padded=max_page_size)
++            new_kv_cache_spec[layer_name] = new_spec
+         elif isinstance(layer_spec, MambaSpec):
+             # MambaSpec's page size is determined by its state shapes and does
+             # not scale with block_size, so pad the page instead. This is the
+@@ -1593,8 +1668,31 @@
+ ) -> int:
+     """Return the largest cache group's bytes per block."""
+     if (glm5_layout := _glm5_next_tensor_layout(kv_cache_groups)) is not None:
+-        _, _, mla_names, idx_names, mla_page, idx_page, _, _ = glm5_layout
+-        return len(mla_names) * mla_page + len(idx_names) * idx_page
++        (
++            _,
++            _,
++            mla_names,
++            idx_names,
++            mla_page,
++            idx_page,
++            _,
++            _,
++            draft_group,
++            hidden_groups,
++        ) = glm5_layout
++        bytes_per_block = len(mla_names) * mla_page + len(idx_names) * idx_page
++        if draft_group is not None:
++            d_inner = cast(
++                UniformTypeKVCacheSpecs, draft_group.kv_cache_spec
++            ).kv_cache_specs
++            bytes_per_block += sum(
++                spec.page_size_bytes for spec in d_inner.values()
++            )
++        for group in hidden_groups:
++            bytes_per_block += (
++                group.kv_cache_spec.page_size_bytes * len(group.layer_names)
++            )
++        return bytes_per_block
+ 
+     bytes_per_block = max(
+         sum(
+@@ -1701,12 +1799,30 @@
+             idx_page,
+             tail_names,
+             _,
++            draft_group,
++            hidden_groups,
+         ) = glm5_layout
+         bytes_per_block = len(mla_names) * mla_page + len(idx_names) * idx_page
++        extra_bytes_per_block = 0
++        if draft_group is not None:
++            d_inner = cast(
++                UniformTypeKVCacheSpecs, draft_group.kv_cache_spec
++            ).kv_cache_specs
++            extra_bytes_per_block += sum(
++                spec.page_size_bytes for spec in d_inner.values()
++            )
++        for group in hidden_groups:
++            extra_bytes_per_block += (
++                group.kv_cache_spec.page_size_bytes * len(group.layer_names)
++            )
+         num_blocks = may_override_num_blocks(
+-            vllm_config, available_memory // bytes_per_block
++            vllm_config,
++            available_memory // (bytes_per_block + extra_bytes_per_block),
+         )
+-        size = bytes_per_block * num_blocks
++        # One backing allocation shared by every tensor (allocate_kv_cache
++        # asserts all tensor sizes equal): the aliased GLM pool plus the
++        # draft/hidden regions appended after it.
++        size = (bytes_per_block + extra_bytes_per_block) * num_blocks
+         attn_specs = cast(
+             UniformTypeKVCacheSpecs, attn_group.kv_cache_spec
+         ).kv_cache_specs
+@@ -1745,6 +1861,37 @@
+                 ).kv_cache_specs
+                 add_tensor(tail_name, tail_specs[tail_name], offset)
+ 
++        # Draft + hidden-state layers get their own tensors appended after
++        # the aliased pool (no aliasing: independent block tables).
++        offset_cursor = bytes_per_block * num_blocks
++        if draft_group is not None:
++            d_inner = cast(
++                UniformTypeKVCacheSpecs, draft_group.kv_cache_spec
++            ).kv_cache_specs
++            d_page = next(iter(d_inner.values())).page_size_bytes
++            kv_cache_tensors.append(
++                KVCacheTensor(
++                    size=size,
++                    layers=list(draft_group.layer_names),
++                    layer_stride=d_page * num_blocks,
++                    block_stride=d_page,
++                    offset=offset_cursor,
++                )
++            )
++            offset_cursor += d_page * num_blocks * len(draft_group.layer_names)
++        for group in hidden_groups:
++            h_page = group.kv_cache_spec.page_size_bytes
++            kv_cache_tensors.append(
++                KVCacheTensor(
++                    size=size,
++                    layers=list(group.layer_names),
++                    layer_stride=h_page * num_blocks,
++                    block_stride=h_page,
++                    offset=offset_cursor,
++                )
++            )
++            offset_cursor += h_page * num_blocks * len(group.layer_names)
++
+         return KVCacheConfig(
+             num_blocks=num_blocks,
+             kv_cache_tensors=kv_cache_tensors,
+@@ -2440,6 +2587,8 @@
+             idx_page,
+             tail_names,
+             _,
++            draft_group,
++            hidden_groups,
+         ) = glm5_layout
+         uniform_spec = cast(UniformTypeKVCacheSpecs, attn_group.kv_cache_spec)
+         total_blocks = uniform_spec.max_memory_usage_pages(vllm_config)
+@@ -2452,7 +2601,30 @@
+         )
+         if tail_names:
+             total_blocks += 1
+-        return total_blocks * (len(mla_names) * mla_page + len(idx_names) * idx_page)
++        pool_bytes = total_blocks * (
++            len(mla_names) * mla_page + len(idx_names) * idx_page
++        )
++        extra_bytes = 0
++        if draft_group is not None:
++            d_inner = cast(
++                UniformTypeKVCacheSpecs, draft_group.kv_cache_spec
++            ).kv_cache_specs
++            d_spec = next(iter(d_inner.values()))
++            d_blocks = cdiv(
++                d_spec.max_memory_usage_bytes(vllm_config), d_spec.page_size_bytes
++            )
++            extra_bytes += d_blocks * d_spec.page_size_bytes * len(
++                draft_group.layer_names
++            )
++        for group in hidden_groups:
++            h_spec = group.kv_cache_spec
++            h_blocks = cdiv(
++                h_spec.max_memory_usage_bytes(vllm_config), h_spec.page_size_bytes
++            )
++            extra_bytes += (
++                h_blocks * h_spec.page_size_bytes * len(group.layer_names)
++            )
++        return pool_bytes + extra_bytes
+ 
+     bytes_per_block = _pool_bytes_per_block(kv_cache_groups)
+     total_blocks = 0

scripts/patches/glm53-dflash-04b-kv-registry-dflash.patch

@@ -0,0 +1,31 @@
+--- a/vllm/model_executor/offloader/expert_activation_registry.py
++++ b/vllm/model_executor/offloader/expert_activation_registry.py
+@@ -316,7 +316,27 @@
+ 
+ 
+ def _glm53_kv_bytes_per_token(cfg: "VllmConfig") -> int:
+-    return 13600
++    per_tok = 13600
++    # FORK-LOCAL (patches/glm53-dflash-04-kv-groups-draft): with a dflash/dspark
++    # drafter, the KV spec set gains the drafter's attention layers and the aux
++    # hidden-state cache (one bf16 hidden vector per target aux layer per
++    # token). Both are full-length per-token costs the aliased glm5 pool
++    # cannot absorb, so the reserve must include them. The drafter is a 5-layer
++    # qwen3 (8 kv heads x 128 x K+V x bf16 = 4096 B/token); the aux count is
++    # the dflash target_layer_ids count (5).
++    # NOTE: use_aux_hidden_state_outputs lives on the gpu_model_runner
++    # instance (gpu_model_runner.py:623/678), not on the spec config -- it is
++    # not set yet when the expert plan runs. The spec-config METHOD is the
++    # plan-time signal for the same fact.
++    sc = getattr(cfg, "speculative_config", None)
++    if sc is not None and getattr(sc, "method", "") in (
++        "dflash",
++        "dspark",
++        "extract_hidden_states",
++    ):
++        hidden = int(cfg.model_config.hf_config.hidden_size)
++        per_tok += 5 * hidden * 2 + 4096
++    return per_tok
+ 
+ 
+ for _arch in ("Glm5NextForConditionalGeneration", "Glm5NextForCausalLM"):

scripts/patches/gpu_worker-kv-cap.patch

@@ -0,0 +1,65 @@
+# gpu_worker-kv-cap.patch — fork-local fix for the expert-offload KV cap.
+#
+# STATUS: NOT part of the deployed frank-primary shape. It exists because the
+# fork's plan under-models this model's KV, and it is the enabler for a future
+# MTP attempt (the production GLM shape runs without MTP and boots fine on the
+# plan's own reserve).
+#
+# THE BUG: the plan models KV as 13600 B/token + a small mamba term, but
+# GLM-5.3-Flash also carries a per-sequence KDA state (34 linear-attention
+# layers) and, under MTP, the draft model's KV. gpu_worker then CAPS the KV
+# cache at that reserve, so the plan's own arithmetic can make a legal
+# --max-model-len unbootable. Measured 2026-09-30 on the pinned image:
+#   - no MTP, max_model_len 131072: reserve 1.74 GiB >= need 1.86 GiB  -> boots
+#   - MTP4,  max_model_len 131072: reserve 1.74 GiB <  need 2.07 GiB  -> refuses
+#   - MTP4,  max_model_len 262144: reserve 3.43 GiB <  need 3.62 GiB  -> refuses
+#     ("To serve at least one request with the model's max seq len ...")
+# Raising max_model_len does not fix it: the cap's slope (~13,845 B/token) only
+# overtakes the need's (~12,700 B/token + 0.52 GiB fixed) around 440k tokens,
+# where the KV reserve would eat a third of the expert arena.
+#
+# THE FIX: let an explicit operator KV request raise the cap. The compile and
+# graph-capture headroom still comes off the top, so the blast radius is one
+# flag the operator chose.
+#
+# APPLY (into a host dir that is then bind-mounted read-only over the image):
+#   IMAGE=docker.io/tcclaviger/vllm@sha256:da4029666668b90a0386c1ed23ea370c79916ad4e1a6ad1dd9db098b9fa069ea
+#   D=/home/dank/glm-patch; mkdir -p $D/vllm/v1/worker
+#   podman run --rm --entrypoint cat $IMAGE \
+#     /opt/vllm/lib/python3.14/site-packages/vllm/v1/worker/gpu_worker.py \
+#     > $D/vllm/v1/worker/gpu_worker.py
+#   ( cd $D && patch -p1 < .../scripts/patches/gpu_worker-kv-cap.patch )
+#   # then add to the podman run:
+#   #   -v $D/vllm/v1/worker/gpu_worker.py:/opt/vllm/lib/python3.14/site-packages/vllm/v1/worker/gpu_worker.py:ro
+#   #   -e CLAV_KV_RESERVE_FLOOR_GIB=<needed>   (or pass --kv-cache-memory-bytes)
+#
+# Base file sha256 (the pristine file out of the pinned image):
+#   1f3942b1a01af075317161b06077a2e658086f1da0566c9f297129ceebe37dd7
+# Patched file sha256 (verified 2026-09-30, patch applies clean and reproduces
+# these bytes exactly):
+#   80cd160bbc01270b0851f7b6fe7552b06ae2fae2428134fc29238d7f452af840
+#
+--- a/vllm/v1/worker/gpu_worker.py
++++ b/vllm/v1/worker/gpu_worker.py
+@@ -824,6 +824,21 @@
+         expert_cfg = self.vllm_config.offload_config.expert
+         if expert_cfg.enable_expert_offload and expert_cfg.planned_kv_reserve_bytes > 0:
+             cap = int(expert_cfg.planned_kv_reserve_bytes)
++            # LOCAL PATCH (GLM-5.3 KDA + MTP draft): the plan models KV as
++            # 13600 B/token + a small mamba term, but this model also carries a
++            # per-sequence KDA state and (under MTP) the draft's KV, so the
++            # reserve can sit BELOW what max_model_len legitimately needs -- and
++            # the cap then makes a legal config unbootable (measured: need
++            # 2.07 GiB vs cap 1.74 GiB at max_model_len 131072 with MTP). An
++            # explicit --kv-cache-memory-bytes, or CLAV_KV_RESERVE_FLOOR_GIB,
++            # is the operator's word: let it raise the cap. The compile/capture
++            # headroom still comes off the top.
++            _explicit = self.cache_config.kv_cache_memory_bytes
++            _floor = os.environ.get("CLAV_KV_RESERVE_FLOOR_GIB")
++            if _explicit:
++                cap = max(cap, int(_explicit))
++            if _floor:
++                cap = max(cap, int(float(_floor) * (1 << 30)))
+             if self.available_kv_cache_memory_bytes > cap:
+                 logger.info_once(
+                     "expert offload: KV capped at the plan's reserve %s GiB; %s GiB of "

scripts/qwen-sharp-v22.4.0.jinja

@@ -0,0 +1,453 @@
+{%- set template_version = "qwen3.8-froggeric-v22.4.0" %}
+{%- set _tool_format = tool_call_format if tool_call_format is defined else 'xml' %}
+{%- set image_count = namespace(value=0) %}
+{%- set video_count = namespace(value=0) %}
+{%- set add_vision_id = add_vision_id if add_vision_id is defined else false %}
+{%- set enable_thinking = enable_thinking if enable_thinking is defined else true %}
+{%- set auto_disable_thinking_with_tools = auto_disable_thinking_with_tools if auto_disable_thinking_with_tools is defined else false %}
+{%- if preserve_reasoning is defined and preserve_reasoning is not none %}
+    {%- set _preserve_thinking = preserve_reasoning %}
+{%- elif preserve_thinking is defined and preserve_thinking is not none %}
+    {%- set _preserve_thinking = preserve_thinking %}
+{%- else %}
+    {%- set _preserve_thinking = true %}
+{%- endif %}
+{%- set max_tool_arg_chars = max_tool_arg_chars if max_tool_arg_chars is defined else 0 %}
+{%- set max_tool_response_chars = max_tool_response_chars if max_tool_response_chars is defined else 0 %}
+{%- set _default_reasoning_effort = 'medium' %}
+{%- set _has_tools = (tools is defined and tools and tools is iterable and tools is not mapping) %}
+{%- set _effort_raw = (reasoning_effort | string | lower) if reasoning_effort is defined and reasoning_effort is not none else _default_reasoning_effort %}
+{%- set _initial_thinking = enable_thinking %}
+{%- if _effort_raw in ('none', 'off') %}
+    {%- set _initial_thinking = false %}
+    {%- set _initial_effort = 'medium' %}
+{%- elif _effort_raw in ('minimal', 'low') %}
+    {%- set _initial_effort = 'low' %}
+{%- elif _effort_raw in ('high', 'xhigh', 'max', 'ultracode', 'extreme') %}
+    {%- set _initial_effort = 'xhigh' %}
+{%- else %}
+    {%- set _initial_effort = 'medium' %}
+{%- endif %}
+{%- set ns_state = namespace(thinking=_initial_thinking, effort=_initial_effort) %}
+{%- if auto_disable_thinking_with_tools and _has_tools %}
+    {%- set ns_state.thinking = false %}
+{%- endif %}
+{%- for msg in messages %}
+    {%- if msg.role == 'system' or msg.role == 'developer' or msg.role == 'user' %}
+        {%- if msg.content is string %}
+            {%- if '<|think_off|>' in msg.content %}
+                {%- set ns_state.thinking = false %}
+            {%- elif '<|think_on|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+            {%- elif '<|think_xhigh|>' in msg.content or '<|think_high|>' in msg.content or '<|think_ultracode|>' in msg.content or '<|think_extreme|>' in msg.content or '<|think_max|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'xhigh' %}
+            {%- elif '<|think_low|>' in msg.content or '<|think_minimal|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'low' %}
+            {%- elif '<|think_medium|>' in msg.content %}
+                {%- set ns_state.thinking = true %}
+                {%- set ns_state.effort = 'medium' %}
+            {%- endif %}
+        {%- elif msg.content is iterable and msg.content is not mapping %}
+            {%- for item in msg.content %}
+                {%- if item is string %}
+                    {%- set _item_text = item %}
+                {%- elif item is mapping and 'text' in item and item.text is string %}
+                    {%- set _item_text = item.text %}
+                {%- else %}
+                    {%- set _item_text = '' %}
+                {%- endif %}
+                {%- if _item_text %}
+                    {%- if '<|think_off|>' in _item_text %}
+                        {%- set ns_state.thinking = false %}
+                    {%- elif '<|think_on|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                    {%- elif '<|think_xhigh|>' in _item_text or '<|think_high|>' in _item_text or '<|think_ultracode|>' in _item_text or '<|think_extreme|>' in _item_text or '<|think_max|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'xhigh' %}
+                    {%- elif '<|think_low|>' in _item_text or '<|think_minimal|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'low' %}
+                    {%- elif '<|think_medium|>' in _item_text %}
+                        {%- set ns_state.thinking = true %}
+                        {%- set ns_state.effort = 'medium' %}
+                    {%- endif %}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- set reasoning_instructions = '' %}
+{%- if ns_state.thinking %}
+    {%- if ns_state.effort == 'xhigh' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.' %}
+    {%- elif ns_state.effort == 'low' %}
+        {%- set reasoning_instructions = 'Reasoning effort is set to low. Keep your thinking brief and focused, moving directly to the conclusion without unnecessary elaboration.' %}
+    {%- endif %}
+{%- endif %}
+{%- macro render_content(content, do_vision_count, is_system_content=false) %}
+    {%- if content is string %}
+        {{- content }}
+    {%- elif content is iterable and content is not mapping %}
+        {%- for item in content %}
+            {%- if item is mapping %}
+                {%- if item.type == 'image' or 'image' in item or 'image_url' in item %}
+                    {%- if is_system_content %}
+                        {{- raise_exception('System message cannot contain images.') }}
+                    {%- endif %}
+                    {%- if do_vision_count %}
+                        {%- set image_count.value = image_count.value + 1 %}
+                    {%- endif %}
+                    {%- if add_vision_id %}
+                        {{- 'Picture ' ~ image_count.value ~ ': ' }}
+                    {%- endif %}
+                    {{- '<|vision_start|><|image_pad|><|vision_end|>' }}
+                {%- elif item.type == 'video' or 'video' in item %}
+                    {%- if is_system_content %}
+                        {{- raise_exception('System message cannot contain videos.') }}
+                    {%- endif %}
+                    {%- if do_vision_count %}
+                        {%- set video_count.value = video_count.value + 1 %}
+                    {%- endif %}
+                    {%- if add_vision_id %}
+                        {{- 'Video ' ~ video_count.value ~ ': ' }}
+                    {%- endif %}
+                    {{- '<|vision_start|><|video_pad|><|vision_end|>' }}
+                {%- elif 'text' in item %}
+                    {{- item.text }}
+                {%- else %}
+                    {{- raise_exception('Unexpected item type in content.') }}
+                {%- endif %}
+            {%- else %}
+                {{- item | string }}
+            {%- endif %}
+        {%- endfor %}
+    {%- elif content is none or content is undefined %}
+        {{- '' }}
+    {%- else %}
+        {{- raise_exception('Unexpected content type.') }}
+    {%- endif %}
+{%- endmacro %}
+{%- if not messages %}
+    {{- raise_exception('No messages provided.') }}
+{%- endif %}
+{%- set head = namespace(count=0, seen_non_system=false) %}
+{%- for message in messages %}
+    {%- set _is_sys = (message.role == 'system' or message.role == 'developer') %}
+    {%- if _is_sys and not head.seen_non_system %}
+        {%- set head.count = head.count + 1 %}
+    {%- else %}
+        {%- set head.seen_non_system = true %}
+    {%- endif %}
+{%- endfor %}
+{%- set sys_state = namespace(content='') %}
+{%- for message in messages[:head.count] %}
+    {%- set _part = render_content(message.content, false, true) | trim %}
+    {%- if '<|think_off|>' in _part %}{%- set _part = _part.split('<|think_off|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_on|>' in _part %}{%- set _part = _part.split('<|think_on|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_xhigh|>' in _part %}{%- set _part = _part.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_high|>' in _part %}{%- set _part = _part.split('<|think_high|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_ultracode|>' in _part %}{%- set _part = _part.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_extreme|>' in _part %}{%- set _part = _part.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_max|>' in _part %}{%- set _part = _part.split('<|think_max|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_medium|>' in _part %}{%- set _part = _part.split('<|think_medium|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_low|>' in _part %}{%- set _part = _part.split('<|think_low|>') | join('') | trim %}{%- endif %}
+    {%- if '<|think_minimal|>' in _part %}{%- set _part = _part.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
+    {%- if _part %}
+        {%- if sys_state.content %}
+            {%- set sys_state.content = sys_state.content ~ '\n\n' ~ _part %}
+        {%- else %}
+            {%- set sys_state.content = _part %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- set _sc = sys_state.content %}
+{%- set _msgs = messages[head.count:] %}
+{%- if ns_state.thinking %}
+    {%- set _terse_lead = 'Answer directly, after thinking. Lead with the answer, then only what it needs to be correct and usable.' %}
+{%- else %}
+    {%- set _terse_lead = 'Answer directly and concisely. Give the answer with only what it needs to be correct and usable.' %}
+{%- endif %}
+{%- set _terse_core %}
+Never: open with preamble or pleasantries; restate the question; add filler transitions; hedge with niceties; or repeat a point you've already made.
+Always: keep essential steps, caveats, uncertainties, and specifics — never drop correctness or a needed warning for brevity. Keep the final answer lean. Use the least structure that conveys it (plain prose when short; lists or code only when they earn their place). If genuinely uncertain, say so and explain why — never omit uncertainty for the sake of brevity.
+If a user request is genuinely ambiguous, ask a sharp question, don't guess.
+{%- endset %}
+{%- set _terse = _terse_lead ~ '\n' ~ (_terse_core | trim) %}
+{#- `terse` chat-template kwarg: default true, so behaviour is unchanged unless a caller opts out.
+    Pass {"terse": false} via chat_template_kwargs to serve the model with only its own system
+    prompt -- useful for A/B-ing the terseness effect, or when a downstream prompt conflicts. #}
+{%- set _terse_on = terse if terse is defined else true %}
+{%- if _terse_on %}
+    {%- if not _sc %}
+        {%- set _sc = _terse | trim %}
+    {%- else %}
+        {%- set _sc = (_sc | trim) ~ '\n\n' ~ (_terse | trim) %}
+    {%- endif %}
+{%- endif %}
+{%- if _has_tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if reasoning_instructions %}
+        {{- reasoning_instructions + '\n\n' }}
+    {%- endif %}
+    {{- '# Tools\n\nYou have access to the following functions:\n\n<tools>' }}
+    {%- for tool in tools %}
+        {{- '\n' }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- '\n</tools>' }}
+    {%- if _tool_format == 'json' %}
+        {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n{"name": "example_function_name", "arguments": {"example_parameter_1": "value_1", "example_parameter_2": "This is the value for the second parameter"}}\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: a single JSON object with "name" and "arguments" keys inside <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> tag MUST be at the very beginning of a new line, with NO spaces or indentation before it.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
+    {%- else %}
+        {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n' }}{%- if ns_state.thinking %}{{- '<think>\nBrief explanation of tool call\n</think>\n' }}{%- endif %}{{- '<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n' }}{%- if ns_state.thinking %}{{- '- You can use the <think></think> block to plan your next tool call OR to synthesize data and formulate your final response to the user.\n- ALL explanation and reasoning MUST be placed strictly inside the <think></think> block.\n' }}{%- endif %}{{- '- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags.\n- If you choose to call a tool, you MUST output the <tool_call> block IMMEDIATELY' }}{%- if ns_state.thinking %}{{- ' after thinking' }}{%- endif %}{{- ', with NO conversational text before it.\n- The <tool_call> and <function> tags MUST be at the very beginning of a new line, with NO spaces or indentation before them.\n- To call multiple functions, output a separate, completely closed <tool_call></tool_call> block for EACH function. Do NOT nest <tool_call> blocks.\n- If you have all necessary data, provide your final answer directly to the user without any tool call.\n</IMPORTANT>' }}
+    {%- endif %}
+    {%- if _sc %}
+        {{- '\n\n' + _sc }}
+    {%- endif %}
+    {{- '<|im_end|>\n' }}
+{%- else %}
+    {%- if _sc %}
+        {{- '<|im_start|>system\n' + (reasoning_instructions + '\n\n' if reasoning_instructions else '') + _sc + '<|im_end|>\n' }}
+    {%- elif reasoning_instructions %}
+        {{- '<|im_start|>system\n' + reasoning_instructions + '<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- set _last_idx = _msgs | length - 1 %}
+{%- set ns = namespace(multi_step_tool=true, last_query_index=_last_idx) %}
+{%- for message in _msgs[::-1] %}
+    {%- set index = (_msgs | length - 1) - loop.index0 %}
+    {%- if ns.multi_step_tool and message.role == 'user' %}
+        {%- set _rc = render_content(message.content, false) | trim %}
+        {%- if not (_rc.startswith('<tool_response>') and _rc.endswith('</tool_response>')) %}
+            {%- set ns.multi_step_tool = false %}
+            {%- set ns.last_query_index = index %}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if ns.multi_step_tool %}
+    {%- if _last_idx > 50 %}
+        {%- set ns.last_query_index = _last_idx %}
+    {%- else %}
+        {%- set ns.last_query_index = 0 %}
+    {%- endif %}
+{%- endif %}
+{%- set ns2 = namespace(prev_role='', consecutive_failures=0) %}
+{%- for message in _msgs %}
+    {%- set is_system = (message.role == "system" or message.role == "developer") %}
+    {%- set content = render_content(message.content, true, is_system) | trim %}
+    {%- if is_system or message.role == 'user' %}
+        {%- if '<|think_off|>' in content %}{%- set content = content.split('<|think_off|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_on|>' in content %}{%- set content = content.split('<|think_on|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_xhigh|>' in content %}{%- set content = content.split('<|think_xhigh|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_high|>' in content %}{%- set content = content.split('<|think_high|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_ultracode|>' in content %}{%- set content = content.split('<|think_ultracode|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_extreme|>' in content %}{%- set content = content.split('<|think_extreme|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_max|>' in content %}{%- set content = content.split('<|think_max|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_medium|>' in content %}{%- set content = content.split('<|think_medium|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_low|>' in content %}{%- set content = content.split('<|think_low|>') | join('') | trim %}{%- endif %}
+        {%- if '<|think_minimal|>' in content %}{%- set content = content.split('<|think_minimal|>') | join('') | trim %}{%- endif %}
+    {%- endif %}
+    {%- if is_system %}
+        {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}
+    {%- elif message.role == 'user' %}
+        {%- set ns2.consecutive_failures = 0 %}
+        {{- '<|im_start|>user\n' + content + '<|im_end|>\n' }}
+    {%- elif message.role == 'assistant' %}
+        {%- set reasoning_content = '' %}
+        {%- set _explicit_reasoning = '' %}
+        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}
+            {%- if message.reasoning_content is string %}
+                {%- set _explicit_reasoning = message.reasoning_content %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.reasoning_content | string %}
+            {%- endif %}
+        {%- elif message.thinking is defined and message.thinking is not none %}
+            {%- if message.thinking is string %}
+                {%- set _explicit_reasoning = message.thinking %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.thinking | string %}
+            {%- endif %}
+        {%- elif message.reasoning is defined and message.reasoning is not none %}
+            {%- if message.reasoning is string %}
+                {%- set _explicit_reasoning = message.reasoning %}
+            {%- else %}
+                {%- set _explicit_reasoning = message.reasoning | string %}
+            {%- endif %}
+        {%- endif %}
+        {%- if _explicit_reasoning %}
+            {%- set _lead_end = '' %}
+            {%- if content.startswith('<think>') and '</think>' in content %}
+                {%- set _lead_end = '</think>' %}
+            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}
+                {%- set _lead_end = '</thinking>' %}
+            {%- elif content.startswith('</think>') %}
+                {%- set _lead_end = '</think>' %}
+            {%- elif content.startswith('</thinking>') %}
+                {%- set _lead_end = '</thinking>' %}
+            {%- endif %}
+            {%- if _lead_end %}
+                {%- set content = content.split(_lead_end)[-1].lstrip('\n') %}
+            {%- endif %}
+            {%- set reasoning_content = _explicit_reasoning %}
+        {%- else %}
+            {%- set _think_end = '' %}
+            {%- if content.startswith('</think>') %}
+                {%- set _think_end = '</think>' %}
+            {%- elif content.startswith('</thinking>') %}
+                {%- set _think_end = '</thinking>' %}
+            {%- elif '\n</think>' in content %}
+                {%- set _think_end = '\n</think>' %}
+            {%- elif '\n</thinking>' in content %}
+                {%- set _think_end = '\n</thinking>' %}
+            {%- elif '\n</ think>' in content %}
+                {%- set _think_end = '\n</ think>' %}
+            {%- elif '\n</think >' in content %}
+                {%- set _think_end = '\n</think >' %}
+            {%- elif content.startswith('<think>') and '</think>' in content %}
+                {%- set _think_end = '</think>' %}
+            {%- elif content.startswith('<thinking>') and '</thinking>' in content %}
+                {%- set _think_end = '</thinking>' %}
+            {%- endif %}
+            {%- if _think_end %}
+                {%- if 'thinking' in _think_end %}
+                    {%- set _think_start = '<thinking>' %}
+                {%- else %}
+                    {%- set _think_start = '<think>' %}
+                {%- endif %}
+                {%- set reasoning_content = content.split(_think_end)[0].rstrip('\n') %}
+                {%- if _think_start in reasoning_content %}
+                    {%- set reasoning_content = reasoning_content.split(_think_start)[-1].lstrip('\n') %}
+                {%- endif %}
+                {%- set content = content.split(_think_end)[-1].lstrip('\n') %}
+            {%- endif %}
+        {%- endif %}
+        {%- set reasoning_content = reasoning_content | trim %}
+        {%- if (_preserve_thinking or loop.index0 > ns.last_query_index) %}
+            {{- '<|im_start|>assistant\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
+        {%- else %}
+            {{- '<|im_start|>assistant\n' + content }}
+        {%- endif %}
+        {%- if message.tool_calls is defined and message.tool_calls and message.tool_calls is iterable and message.tool_calls is not mapping %}
+            {%- for tool_call in message.tool_calls %}
+                {%- if tool_call.function is defined and tool_call.function is not none %}
+                    {%- set tc = tool_call.function %}
+                {%- else %}
+                    {%- set tc = tool_call %}
+                {%- endif %}
+                {%- set tc_name = tc.name if (tc.name is defined and tc.name is not none) else '' %}
+                {%- if _tool_format == 'json' %}
+                    {%- if loop.first %}
+                        {%- if content | trim %}
+                            {{- '\n\n' }}
+                        {%- endif %}
+                    {%- else %}
+                        {{- '\n' }}
+                    {%- endif %}
+                    {%- set _args = '{}' %}
+                    {%- if tc.arguments is defined and tc.arguments is not none %}
+                        {%- if tc.arguments is mapping %}
+                            {%- set _args = tc.arguments | tojson %}
+                        {%- elif tc.arguments is string %}
+                            {%- if tc.arguments %}
+                                {%- set _args = tc.arguments %}
+                            {%- endif %}
+                        {%- else %}
+                            {%- set _args = tc.arguments | tojson %}
+                        {%- endif %}
+                    {%- endif %}
+                    {{- '<tool_call>\n{"name": ' }}{{- tc_name | tojson }}{{- ', "arguments": ' }}{{- _args }}{{- '}\n</tool_call>' }}
+                {%- else %}
+                    {%- if loop.first %}
+                        {%- if content | trim %}
+                            {{- '\n\n<tool_call>\n<function=' + tc_name + '>\n' }}
+                        {%- else %}
+                            {{- '<tool_call>\n<function=' + tc_name + '>\n' }}
+                        {%- endif %}
+                    {%- else %}
+                        {{- '\n<tool_call>\n<function=' + tc_name + '>\n' }}
+                    {%- endif %}
+                    {%- if tc.arguments is defined and tc.arguments is not none %}
+                        {%- if tc.arguments is mapping %}
+                            {%- for args_name, args_value in tc.arguments.items() %}
+                                {{- '<parameter=' + args_name + '>\n' }}
+                                {%- if args_value is string %}
+                                    {%- set _av = args_value %}
+                                {%- else %}
+                                    {%- set _av = args_value | tojson %}
+                                {%- endif %}
+                                {%- if max_tool_arg_chars > 0 and _av | length > max_tool_arg_chars %}
+                                    {{- _av[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_av | length | string) ~ ' chars]' }}
+                                {%- else %}
+                                    {{- _av }}
+                                {%- endif %}
+                                {{- '\n</parameter>\n' }}
+                            {%- endfor %}
+                        {%- else %}
+                            {%- if tc.arguments is string %}
+                                {%- set _raw_args = tc.arguments %}
+                            {%- else %}
+                                {%- set _raw_args = tc.arguments | tojson %}
+                            {%- endif %}
+                            {%- if _raw_args %}
+                                {%- if max_tool_arg_chars > 0 and _raw_args | length > max_tool_arg_chars %}
+                                    {{- _raw_args[:max_tool_arg_chars] + '\n[TRUNCATED - original length ' ~ (_raw_args | length | string) ~ ' chars]' }}
+                                {%- else %}
+                                    {{- _raw_args }}
+                                {%- endif %}
+                            {%- endif %}
+                        {%- endif %}
+                    {%- endif %}
+                    {{- '</function>\n</tool_call>' }}
+                {%- endif %}
+            {%- endfor %}
+        {%- endif %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == 'tool' %}
+        {%- set _content_lower = content | lower %}
+        {%- set _content_head = _content_lower[:120] %}
+        {%- set _is_code_or_grep = ('throw new ' in _content_lower or 'throw error' in _content_lower or 'console.error' in _content_lower or 'logger.error' in _content_lower or 'logging.error' in _content_lower or 'import ' in _content_head or 'def ' in _content_head or 'function ' in _content_head) %}
+        {%- set _exit_code_zero = ('exit code: 0' in _content_head or 'process exited with code 0' in _content_head) %}
+        {%- set _error_field_ok = ('"error": null' in _content_head or '"error":null' in _content_head or '"error": false' in _content_head or '"error":false' in _content_head or '"error": ""' in _content_head or '"error":""' in _content_head) %}
+        {%- set _strong_error = (('"error":' in _content_head and not _error_field_ok) or '"status": "error"' in _content_head or '"status":"error"' in _content_head or 'traceback (most recent call last):' in _content_head or 'command not found' in _content_head or 'invalid syntax' in _content_head or 'fatal:' in _content_head or (('exit code: ' in _content_head or 'process exited with code' in _content_head) and not _exit_code_zero) or _content_head.startswith('exception:') or _content_head.startswith('failed to ')) %}
+        {%- set _weak_error = ('error:' in _content_head or 'err!' in _content_head) %}
+        {%- set _weak_suppressed = ('$ ' in _content_head or 'took ' in _content_head or content | length >= 600) %}
+        {%- if not _is_code_or_grep and (_strong_error or (_weak_error and not _weak_suppressed)) %}
+            {%- set ns2.consecutive_failures = ns2.consecutive_failures + 1 %}
+        {%- else %}
+            {%- set ns2.consecutive_failures = 0 %}
+        {%- endif %}
+        {%- if ns2.prev_role != 'tool' %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {%- if _tool_format != 'json' and max_tool_response_chars > 0 and content | length > max_tool_response_chars %}
+            {%- set content = content[:max_tool_response_chars] + '\n[TRUNCATED - original length ' ~ (content | length | string) ~ ' chars]' %}
+        {%- endif %}
+        {{- '\n<tool_response>\n' + content }}
+        {%- if ns2.consecutive_failures >= 2 %}
+            {{- '\n\n⚠️ SYSTEM WARNING: ' ~ ns2.consecutive_failures ~ ' consecutive tool errors detected. Your previous approach is incorrect. You MUST use a fundamentally different approach or corrected arguments.' }}
+        {%- elif ns2.consecutive_failures == 1 %}
+            {{- '\n\n⚠️ SYSTEM WARNING: The previous tool call returned an error. Diagnose the failure and retry with completely corrected arguments.' }}
+        {%- endif %}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last %}
+            {{- '<|im_end|>\n' }}
+        {%- else %}
+            {%- set _next_role = _msgs[loop.index0 + 1].role %}
+            {%- if _next_role != 'tool' %}
+                {{- '<|im_end|>\n' }}
+            {%- endif %}
+        {%- endif %}
+    {%- else %}
+        {{- '<|im_start|>user\n[' + message.role + ']: ' + content + '<|im_end|>\n' }}
+    {%- endif %}
+    {%- set ns2.prev_role = message.role %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+    {%- if not ns_state.thinking %}
+        {{- '<think>\n\n</think>\n\n' }}
+    {%- else %}
+        {{- '<think>\n' }}
+    {%- endif %}
+{%- endif %}

scripts/searxng-render-settings.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+"""Render ${VAR} placeholders in the live SearXNG settings.yml from the
+host-side env file.
+
+Why this exists: SearXNG does NOT expand environment variables inside
+settings.yml. The repo-tracked shared/searxng/settings.yml deliberately keeps
+secrets as "${KAGI_API_KEY}"-style placeholders so no key ever enters git; the
+deployer copies that file to /opt/searxng/config/settings.yml and runs this
+script to bake in the real values. Without it the app runs with the literal
+placeholder strings (observed 2026-09-26: secret_key literally
+"${SEARXNG_SECRET}" and the Kagi engine failing setup() on an empty key).
+
+Only the allowlisted names are substituted. A placeholder that has no value in
+the env file is a deploy-blocking error: serving a paid-API engine with a
+literal "${KAGI_API_KEY}" bearer would waste requests and hide the misconfig.
+
+Usage: searxng-render-settings.py ENV_FILE SETTINGS_FILE
+Both paths are taken as arguments; secrets never appear in any argv.
+Exit codes: 0 rendered (idempotent on an already-rendered file), 1 unresolved
+placeholder, 2 usage error.
+"""
+
+import sys
+
+ALLOWED = ("KAGI_API_KEY", "SEARXNG_SECRET", "SEARXNG_BASE_URL")
+
+
+def load_env(path: str) -> dict[str, str]:
+    values: dict[str, str] = {}
+    with open(path, encoding="utf-8") as fh:
+        for line in fh:
+            line = line.strip()
+            if not line or line.startswith("#") or "=" not in line:
+                continue
+            key, _, value = line.partition("=")
+            if key in ALLOWED:
+                values[key] = value
+    return values
+
+
+def main(argv: list[str]) -> int:
+    args = [a for a in argv[1:]]
+    if len(args) != 2:
+        print(
+            "usage: searxng-render-settings.py ENV_FILE SETTINGS_FILE",
+            file=sys.stderr,
+        )
+        return 2
+    env_file, settings_file = args
+
+    values = load_env(env_file)
+    with open(settings_file, encoding="utf-8") as fh:
+        text = fh.read()
+
+    for name in ALLOWED:
+        if name in values:
+            text = text.replace("${%s}" % name, values[name])
+
+    unresolved = [n for n in ALLOWED if "${%s}" % n in text]
+    if unresolved:
+        print(
+            "unresolved placeholders in %s: %s (missing from %s)"
+            % (settings_file, ", ".join(unresolved), env_file),
+            file=sys.stderr,
+        )
+        return 1
+
+    with open(settings_file, "w", encoding="utf-8") as fh:
+        fh.write(text)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

scripts/serve-frank-large.sh

@@ -0,0 +1,146 @@
+#!/usr/bin/env bash
+# serve-frank-large.sh — frank-large endpoint: halogen (closed-source
+# C++/HIP engine, image ghcr.io/peonist-ai/halogen, digest-pinned in
+# frank-large.env) serving peonist-ai/halogen-qwen3.8-27b on Dankclaw's
+# Strix Halo iGPU, host :7770. 2026-10-05 cutover from the ROCmFPX
+# llama.cpp shape — revert the swap commit's trio to roll back (payload
+# and fork build stay on disk).
+#
+# The engine is driven by `podman run` DIRECTLY (the fleet's frank-primary
+# pattern): the image entrypoint `halogen-docker` in default "all" mode
+# starts engine + OpenAI frontend in one container. The frontend listens on
+# 8731 inside (HALOGEN_API_PORT default) and is published as host 7770; the
+# engine's own token port (8730, no auth) stays unpublished on the
+# container's loopback. The image ships NO default checkpoint:
+# HALOGEN_CHECKPOINT selects the file inside the /models mount.
+#
+# Gates mirror the deployer EXACTLY (the 2026-10-01 crash-loop lesson —
+# serve gate and deploy gate disagreeing = green deploy, red unit):
+# tokenizer directory present with every named file a FLAT REGULAR FILE —
+# the model card ships `tokenizer/` as a flat directory and the engine
+# defaults HALOGEN_TOKENIZER to /tokenizer, so a HF cache snapshot's
+# symlinks into blobs/ dangle inside the mount; GPU device nodes present.
+#
+# Why containers instead of the bare binary (the old frank-large shape):
+# the payload is .hgn and only halogen reads it; halogen ships as a
+# container; the root unit keeps LimitMEMLOCK=infinity for parity and to
+# outlive the login session, exactly like frank-primary's.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ENV_FILE="${FRANK_LARGE_ENV_FILE:-$SCRIPT_DIR/frank-large.env}"
+if [ ! -f "$ENV_FILE" ]; then
+    echo "frank-large: env file not found: $ENV_FILE" >&2
+    exit 1
+fi
+set -a
+# shellcheck disable=SC1090
+source "$ENV_FILE"
+set +a
+
+: "${FRANK_LARGE_IMAGE:?FRANK_LARGE_IMAGE must name the digest-pinned image}"
+: "${FRANK_LARGE_CONTAINER:?FRANK_LARGE_CONTAINER must name the container}"
+: "${FRANK_LARGE_PUBLISH:?FRANK_LARGE_PUBLISH must be host:container}"
+: "${FRANK_LARGE_MODEL_DIR:?FRANK_LARGE_MODEL_DIR must name the payload dir}"
+: "${FRANK_LARGE_CHECKPOINT:?FRANK_LARGE_CHECKPOINT must name the .hgn inside it}"
+: "${FRANK_LARGE_CHECKPOINT_BYTES:?FRANK_LARGE_CHECKPOINT_BYTES must pin the size}"
+: "${FRANK_LARGE_TOKENIZER_FILES:?FRANK_LARGE_TOKENIZER_FILES must list the tokenizer dir contents}"
+: "${HALOGEN_MODEL_ID:?HALOGEN_MODEL_ID must be the served alias (frank-large)}"
+: "${HALOGEN_SLOT_CTX:?HALOGEN_SLOT_CTX must be set}"
+: "${HALOGEN_KV_SLOTS:?HALOGEN_KV_SLOTS must be set (1 keeps the drafter on)}"
+: "${HALOGEN_DRAFTER:?HALOGEN_DRAFTER must be set (2 = DFlash2)}"
+
+# Capture podman's output BEFORE grepping: under `set -o pipefail` the
+# `podman images | grep -q` pattern races (grep exits on first match,
+# podman dies SIGPIPE 141, pipefail turns the 141 into "no match" — the
+# 2026-09-30 deploy refusal).
+DIGESTS="$(podman images --format '{{.RepoDigests}}')"
+if ! grep -qF "${FRANK_LARGE_IMAGE#*@}" <<<"$DIGESTS"; then
+    echo "frank-large: image $FRANK_LARGE_IMAGE not in podman storage;" >&2
+    echo "  pull it: podman pull ${FRANK_LARGE_IMAGE%@*}" >&2
+    exit 1
+fi
+
+# The checkpoint gate is size-exact: a truncated curl exits 0 on some
+# failures and a partial .hgn loads far enough to look alive, so the byte
+# count from the model card is the contract (the fleet fetch law).
+CKPT="$FRANK_LARGE_MODEL_DIR/$FRANK_LARGE_CHECKPOINT"
+if [ ! -f "$CKPT" ]; then
+    echo "frank-large: checkpoint $CKPT missing;" >&2
+    echo "  fetch it: bash scripts/download-frank-large-halogen.sh" >&2
+    exit 1
+fi
+CKPT_SIZE="$(stat -c %s "$CKPT")"
+if [ "$CKPT_SIZE" != "$FRANK_LARGE_CHECKPOINT_BYTES" ]; then
+    echo "frank-large: checkpoint is $CKPT_SIZE bytes, expected $FRANK_LARGE_CHECKPOINT_BYTES (partial download?)" >&2
+    exit 1
+fi
+
+# The payload ships the tokenizer as a FLAT DIRECTORY (model card:
+# "tokenizer/ flat tokenizer directory"). A HuggingFace cache snapshot
+# symlinks each file into a sibling blobs/, and those symlinks dangle
+# inside the container mount — so the directory must be real and every
+# named file must be a regular file, not a link.
+TOK_DIR="$FRANK_LARGE_MODEL_DIR/tokenizer"
+if [ -L "$TOK_DIR" ] || [ ! -d "$TOK_DIR" ]; then
+    echo "frank-large: tokenizer dir $TOK_DIR missing or a symlink (HF cache snapshot?);" >&2
+    echo "  fetch it: bash scripts/download-frank-large-halogen.sh" >&2
+    exit 1
+fi
+for TF in $FRANK_LARGE_TOKENIZER_FILES; do
+    if [ ! -f "$TOK_DIR/$TF" ] || [ -L "$TOK_DIR/$TF" ]; then
+        echo "frank-large: tokenizer file $TOK_DIR/$TF missing or a symlink;" >&2
+        echo "  fetch it: bash scripts/download-frank-large-halogen.sh" >&2
+        exit 1
+    fi
+done
+
+# GPU access is mandatory but the node paths are variables so the seam can
+# drive the gate on a box without an AMD GPU.
+DEV_KFD="${FRANK_LARGE_DEV_KFD:-/dev/kfd}"
+DEV_DRI="${FRANK_LARGE_DEV_DRI:-/dev/dri}"
+if [ ! -e "$DEV_KFD" ] || [ ! -d "$DEV_DRI" ]; then
+    echo "frank-large: $DEV_KFD or $DEV_DRI missing — no gfx1151 to serve on" >&2
+    exit 1
+fi
+for CTR in ${FRANK_LARGE_LEGACY_CONTAINERS:-} "$FRANK_LARGE_CONTAINER"; do
+    podman rm -f "$CTR" >/dev/null 2>&1 || true
+done
+
+LOG="${FRANK_LARGE_LOG:-/home/dank/logs/frank_large.log}"
+mkdir -p "$(dirname "$LOG")"
+# for the udev-granted supplementary group, seccomp unconfined for the ROCm
+# ioctl surface, and unbounded memlock (the unit's LimitMEMLOCK=infinity is
+# inherited, so runc never has to RAISE it). NO --ulimit nofile: Dankclaw's
+# hard limit is 524288, rootless runc cannot raise a hard limit and dies
+# with "error setting rlimit type 7: operation not permitted" (measured
+# 2026-10-05 cutover), and 524288 fds is already ~100x what the engine
+# wants; the model card and both sibling serve scripts pass no nofile
+# either. No comment lines inside the backslash
+# chain — a mid-chain # terminates the continuation (the 2026-10-05
+# exit-127 crash loop on the sibling serve script).
+podman run -d --name "$FRANK_LARGE_CONTAINER" \
+  --publish "$FRANK_LARGE_PUBLISH" \
+  --ipc=host \
+  --ulimit memlock=-1:-1 \
+  --device "$DEV_KFD" --device "$DEV_DRI" \
+  --group-add keep-groups \
+  --security-opt seccomp=unconfined \
+  -e HALOGEN_MODEL_ID="$HALOGEN_MODEL_ID" \
+  -e HALOGEN_CHECKPOINT="/models/$FRANK_LARGE_CHECKPOINT" \
+  -e HALOGEN_SLOT_CTX="$HALOGEN_SLOT_CTX" \
+  -e HALOGEN_KV_SLOTS="${HALOGEN_KV_SLOTS:-1}" \
+  -e HALOGEN_DRAFTER="$HALOGEN_DRAFTER" \
+  -e HALOGEN_MAX_TOKENS_CAP="${HALOGEN_MAX_TOKENS_CAP:-65536}" \
+  -e HALOGEN_QUEUE_TIMEOUT="${HALOGEN_QUEUE_TIMEOUT:-7200}" \
+  ${HALOGEN_CACHE_MB:+-e HALOGEN_CACHE_MB="$HALOGEN_CACHE_MB"} \
+  -v "$FRANK_LARGE_MODEL_DIR:/models:ro" \
+  -v "$FRANK_LARGE_MODEL_DIR/tokenizer:/tokenizer:ro" \
+  "$FRANK_LARGE_IMAGE" \
+  > "$LOG" 2>&1
+
+# This script's lifetime IS the container's: podman wait returns the
+# container's exit code, so a crash is a nonzero ExecStart and the unit's
+# Restart=always is real. systemd stop goes through the unit's ExecStop
+# (podman wait alone cannot stop the child).
+exec podman wait "$FRANK_LARGE_CONTAINER"

scripts/serve-frank-primary.sh

@@ -0,0 +1,196 @@
+#!/usr/bin/env bash
+# serve-frank-primary.sh — frank-primary endpoint (tcclaviger RDNA4 vLLM fork,
+# orcarouter Qwen3.8-Flash-Next-Uncensored-FP8, 2x R9700,
+# dankserver :7770; 2026-09-29 cutover from the R9V radiance fork — git tag
+# frank-primary-r9v-q8-20260929 is the pre-cutover state).
+#
+# The engine is driven by `podman run` DIRECTLY: the image's entrypoint execs
+# `vllm serve "$@"`, so the run line below IS the engine argv. This replaces
+# the R9V launcher/shim machinery (that fork needed a docker->podman shim and
+# a 104-key on-host setup state; this stack is fully expressed by flags, and
+# every flag here was qualified by boot probe on 2026-09-29 — see
+# frank-primary.env for the qualification record).
+#
+# Shape (2026-10-03 shape F, on the 2026-10-02 shape E base): MAXLEN 200704 x 3
+# slots, fp8 KV, KV_BYTES
+# 3,150,000,000 PER RANK — the POOL, not the slot count, caps resident
+# conversations at ~2 full-length (measured: 398,658 tokens = 1.99x);
+# shorter requests use the spare blocks. --max-num-seqs stays 3 because SEQS=2
+# CANNOT boot this model
+# (torch.fx ConstraintViolationError on query_start_loc, measured at both
+# MNBT 1024 and 2048 — the 2x2 lives in frank-primary.env). Speculative decode
+# is OFF today by ENV KNOB, not by measurement: FRANK_PRIMARY_SPEC / _COMPCONFIG
+# / _GEN_CONFIG hold the whole JSON value and unset means no flag. The 2026-10-02
+# retirement of MTP compared the wrong pair — see frank-primary.env.
+# --max-num-batched-tokens 2048 (Frank's call 2026-10-03; arena-free because the
+# registry prices overhead flat up to its nbt 4096 anchor), MoE gather geometry
+# R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64 (+18.7% decode, grows
+# with depth), prefix caching OFF 2026-10-05 (the fork's QFN align-mode GDN
+# state fault — the evidence block is in frank-primary.env), expert prefetch
+# OFF (36% precision measured),
+# TunableOp off. Shape E adds --expert-precise-memory (fitted 5.25 GiB overhead
+# instead of the 7.60 GiB default -> expert arena 17.22 GiB / 7,522 slots, +16%)
+# and GPU_MAX_HW_QUEUES=2; shape F pins the plan's CUDA-graph term to the
+# measured capture cost (VLLM_CUDAGRAPH_MEM_BASE_MB / _PER_GRAPH_MB, container
+# env — the analytic default charges 0.55 GiB where capture took 0.31 GiB, so
+# the 0.24 GiB difference goes to the arena) and raises util 0.98 -> 0.982,
+# Frank's cap. The native-262144 and fp8+MTP4 shapes are retired to
+# git tag frank-primary-fp8-mtp-20261001.
+#
+# THE COMPILE CACHE IS MOUNTED, and that is a boot-cost fix, not a tuning knob:
+# the image ships no /root/.cache at all, our first boot writes 1.1 GB of vLLM
+# AOT/torch_compile artifacts plus 229 MB of comgr JIT kernels into the
+# container's ephemeral layer, and `podman rm -f` deletes them — so every
+# deploy/restart/reboot re-pays `torch.compile took 181.20 s in total` (173.19 s
+# of it one graph). Mounting FRANK_PRIMARY_CACHE_DIR at /root/.cache makes all
+# later boots load the compiled artifact instead, and hides nothing. PROVEN
+# LIVE 2026-10-02 on consecutive boots of this shape: cold `torch.compile took
+# 188.26 s`, warm `Directly load the compiled graph(s) for compile range
+# (1, 1024) from the cache, took 2.143 s` / `torch.compile took 3.97 s in
+# total` / `init engine … took 41.13 s (compilation: 3.98 s)`.
+#
+# The root unit (scripts/frank-primary.service, User=dank,
+# LimitMEMLOCK=infinity) grants the pin budget for the tiered-expert host
+# slabs (~20.5 GiB/rank of pinned anonymous memory). A plain shell session
+# also works, but the unit is the fleet-parity boot-persistent shape.
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ENV_FILE="${FRANK_PRIMARY_ENV_FILE:-$SCRIPT_DIR/frank-primary.env}"
+
+if [ ! -f "$ENV_FILE" ]; then
+    echo "serve-frank-primary: env file missing: $ENV_FILE" >&2
+    exit 1
+fi
+set -a
+# shellcheck disable=SC1090
+source "$ENV_FILE"
+set +a
+
+: "${FRANK_PRIMARY_IMAGE:?FRANK_PRIMARY_IMAGE must name the digest-pinned image}"
+: "${FRANK_PRIMARY_MODEL_DIR:?FRANK_PRIMARY_MODEL_DIR must name the FP8 payload}"
+: "${FRANK_PRIMARY_MODEL_SHARDS:?FRANK_PRIMARY_MODEL_SHARDS must give the shard count}"
+: "${FRANK_PRIMARY_CONTAINER:?FRANK_PRIMARY_CONTAINER must name the container}"
+: "${FRANK_PRIMARY_SERVED_NAME:?FRANK_PRIMARY_SERVED_NAME must name the alias}"
+: "${FRANK_PRIMARY_PUBLISH:?FRANK_PRIMARY_PUBLISH must be host:container}"
+: "${FRANK_PRIMARY_MAXLEN:?FRANK_PRIMARY_MAXLEN must be set}"
+: "${FRANK_PRIMARY_SEQS:?FRANK_PRIMARY_SEQS must be set}"
+: "${FRANK_PRIMARY_KV_BYTES:?FRANK_PRIMARY_KV_BYTES must be set (PER RANK)}"
+: "${FRANK_PRIMARY_KV_DTYPE:?FRANK_PRIMARY_KV_DTYPE must be set (auto = bf16 KV)}"
+: "${FRANK_PRIMARY_REASONING_PARSER:?FRANK_PRIMARY_REASONING_PARSER must be set (qwen3)}"
+: "${FRANK_PRIMARY_TOOL_PARSER:?FRANK_PRIMARY_TOOL_PARSER must be set (qwen3_xml)}"
+: "${FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS:?FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS must be set}"
+# Capture podman's output BEFORE grepping: under `set -o pipefail`, the
+# classic `podman images | grep -q` pattern races — grep exits on its first
+# match, podman dies SIGPIPE (141) writing the remaining lines, and pipefail
+# turns the 141 into "no match" (the 2026-09-30 deploy refusal).
+DIGESTS="$(podman images --format '{{.RepoDigests}}')"
+if ! grep -qF "${FRANK_PRIMARY_IMAGE#*@}" <<<"$DIGESTS"; then
+    echo "serve-frank-primary: pinned image absent from rootless storage: $FRANK_PRIMARY_IMAGE" >&2
+    echo "  fetch it: podman pull $FRANK_PRIMARY_IMAGE" >&2
+    exit 1
+fi
+
+if [ ! -f "$FRANK_PRIMARY_MODEL_DIR/config.json" ]; then
+    echo "serve-frank-primary: payload config missing: $FRANK_PRIMARY_MODEL_DIR/config.json" >&2
+    exit 1
+fi
+SHARDS="$(find "$FRANK_PRIMARY_MODEL_DIR" -maxdepth 1 -name 'model-*-of-*.safetensors' | wc -l)"
+if [ "$SHARDS" != "$FRANK_PRIMARY_MODEL_SHARDS" ]; then
+    echo "serve-frank-primary: payload shard count wrong: $FRANK_PRIMARY_MODEL_DIR" >&2
+    echo "  found $SHARDS safetensors, want $FRANK_PRIMARY_MODEL_SHARDS" >&2
+    echo "  re-fetch: scripts/download-frank-primary.sh" >&2
+    exit 1
+fi
+if [ ! -f "$FRANK_PRIMARY_MODEL_DIR/model.safetensors.index.json" ]; then
+    echo "serve-frank-primary: weight-map index missing: $FRANK_PRIMARY_MODEL_DIR/model.safetensors.index.json" >&2
+    exit 1
+fi
+
+if [ ! -e /dev/kfd ] || [ ! -d /dev/dri ]; then
+    echo "serve-frank-primary: GPU device nodes absent (/dev/kfd, /dev/dri)" >&2
+    exit 1
+fi
+
+# A crashed/killed unit can leave the container name registered (systemd
+# SIGKILLs conmon before cleanup runs). Clear OUR name plus the legacy
+# identities (the retired R9V container and the ad hoc experiment container
+# both hold these GPUs); all idempotent.
+for CTR in $FRANK_PRIMARY_LEGACY_CONTAINERS "$FRANK_PRIMARY_CONTAINER"; do
+    podman rm -f "$CTR" >/dev/null 2>&1 || true
+done
+
+LOG=/home/dank/logs/frank_primary.log
+mkdir -p "$(dirname "$LOG")"
+
+# Persistent compile cache (see the header). Create it before the mount so even
+# the first, cold boot has a target; unset -> no mount, exactly the old
+# ephemeral behaviour. It carries /root/.cache/vllm (AOT + torch_compile) and
+# /root/.cache/comgr (HIP kernel JIT).
+CACHE_DIR="${FRANK_PRIMARY_CACHE_DIR:-}"
+if [ -n "$CACHE_DIR" ]; then
+    mkdir -p "$CACHE_DIR"
+fi
+
+# --device /dev/dri is the WHOLE DIRECTORY (individual renderD nodes enumerate
+# without the card nodes and HIP refuses contexts — "No CUDA GPUs are
+# available" while device_count() says 2). --ipc=host for the engine's shared
+# memory; memlock unbounded for the pinned host slabs.
+# The entrypoint already execs `vllm serve "$@"`, so no `serve` here.
+# Prefix caching: ON — the shipped contract. FRANK_PRIMARY_NO_PREFIX_-
+# CACHING=1 flips to the fork-named mitigation (--no-enable-prefix-caching)
+# for the QFN align-mode GDN state fault; it was tried 2026-10-05 and
+# REVERTED the same day (live conversations still degraded at ~40k with
+# caching off and zero faults — see the evidence block in
+# frank-primary.env). NO comment lines inside the backslash chain below —
+# a comment there terminates the continuation and the next line runs as
+# its own command (the 2026-10-05 exit-127 crash loop).
+# The ${VAR:-word} / ${VAR:+word} double-expansion for the two directions is
+# AMBIGUOUS bash (${VAR:--flag} parses as a substring offset when set —
+# measured 2026-10-05: rendered "--no-enable-prefix-caching1"), so the flag
+# is branched explicitly: exactly one of the two reaches the engine.
+if [ -n "${FRANK_PRIMARY_NO_PREFIX_CACHING:-}" ]; then
+    FRANK_PRIMARY_PREFIX_CACHE_FLAG=--no-enable-prefix-caching
+else
+    FRANK_PRIMARY_PREFIX_CACHE_FLAG=--enable-prefix-caching
+fi
+podman run -d --name "$FRANK_PRIMARY_CONTAINER" \
+  --publish "$FRANK_PRIMARY_PUBLISH" \
+  --ipc=host --ulimit memlock=-1:-1 \
+  --cap-add SYS_PTRACE \
+  --device /dev/kfd --device /dev/dri --device /dev/mem \
+  -e HIP_VISIBLE_DEVICES=0,1 -e VLLM_ROCM_USE_AITER=0 \
+  ${FRANK_PRIMARY_GPU_MAX_HW_QUEUES:+-e GPU_MAX_HW_QUEUES="$FRANK_PRIMARY_GPU_MAX_HW_QUEUES"} \
+  ${FRANK_PRIMARY_GATHER_CHUNKS:+-e R4D_LRU_GATHER_CHUNKS="$FRANK_PRIMARY_GATHER_CHUNKS"} \
+  ${FRANK_PRIMARY_GATHER_LANES:+-e R4D_LRU_GATHER_LANES="$FRANK_PRIMARY_GATHER_LANES"} \
+  ${FRANK_PRIMARY_CUDAGRAPH_BASE_MB:+-e VLLM_CUDAGRAPH_MEM_BASE_MB="$FRANK_PRIMARY_CUDAGRAPH_BASE_MB"} \
+  ${FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB:+-e VLLM_CUDAGRAPH_MEM_PER_GRAPH_MB="$FRANK_PRIMARY_CUDAGRAPH_PER_GRAPH_MB"} \
+  ${CACHE_DIR:+-v "$CACHE_DIR":/root/.cache} \
+  -v "$FRANK_PRIMARY_MODEL_DIR":/models:ro \
+  "$FRANK_PRIMARY_IMAGE" /models \
+  --served-model-name "$FRANK_PRIMARY_SERVED_NAME" --port 7770 \
+  --tensor-parallel-size 2 \
+  ${FRANK_PRIMARY_DCP:+--decode-context-parallel-size "$FRANK_PRIMARY_DCP"} \
+  --max-model-len "$FRANK_PRIMARY_MAXLEN" \
+  --max-num-seqs "$FRANK_PRIMARY_SEQS" \
+  ${FRANK_PRIMARY_MNBT:+--max-num-batched-tokens "$FRANK_PRIMARY_MNBT"} \
+  ${FRANK_PRIMARY_PRECISE_MEMORY:+--expert-precise-memory} \
+  ${FRANK_PRIMARY_EXPERT_OFFLOAD_MEM:+--expert-offload-mem "$FRANK_PRIMARY_EXPERT_OFFLOAD_MEM"} \
+  --gpu-memory-utilization "$FRANK_PRIMARY_UTIL" \
+  --kv-cache-memory-bytes "$FRANK_PRIMARY_KV_BYTES" \
+  --kv-cache-dtype "$FRANK_PRIMARY_KV_DTYPE" \
+  "$FRANK_PRIMARY_PREFIX_CACHE_FLAG" \
+  --enable-auto-tool-choice --tool-call-parser "$FRANK_PRIMARY_TOOL_PARSER" \
+  --reasoning-parser "$FRANK_PRIMARY_REASONING_PARSER" \
+  --default-chat-template-kwargs "$FRANK_PRIMARY_CHAT_TEMPLATE_KWARGS" \
+  ${FRANK_PRIMARY_SPEC:+--speculative-config "$FRANK_PRIMARY_SPEC"} \
+  ${FRANK_PRIMARY_COMPCONFIG:+--compilation-config "$FRANK_PRIMARY_COMPCONFIG"} \
+  ${FRANK_PRIMARY_GEN_CONFIG:+--override-generation-config "$FRANK_PRIMARY_GEN_CONFIG"} \
+  --trust-remote-code \
+  > "$LOG" 2>&1
+
+# From here on, this script's lifetime IS the container's: wait returns the
+# container's exit code, so a crash is a nonzero ExecStart and the unit's
+# Restart=on-failure is real. (systemd stop goes through ExecStop in the
+# unit — podman wait alone cannot stop the container child.)
+exec podman wait "$FRANK_PRIMARY_CONTAINER"

scripts/serve-frank-small.sh

@@ -1,30 +1,25 @@
 #!/usr/bin/env bash
-# Serve ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp as frank-small on Dankclaw's
-# RTX 3090 via mainline llama.cpp.
+# Serve ornith-ai/Ornith-1.5-9B-GGUF (Q8_0) as frank-small on
+# Dankclaw's RTX 3090 via llama.cpp (master build, CUDA sm_86).
 #
-# Topology: the whole model lives in VRAM (-ngl 999) -- 11.29 GiB of IQ3_S-mtp
-# weights, 0.87 GiB of BF16 vision projector and ~8 GiB of f16 KV inside the
-# card's 24 GiB. The host CPU only tokenizes and samples here; this box's CPU
-# belongs to the halogen engine that serves frank-large.
+# 2026-10-01 night (Frank): swapped from SGLang + INT8-W8A16-AutoRound
+# safetensors to the llama.cpp GGUF path, because llama.cpp is the only
+# runtime on this host that moves K and V independently — the asymmetric
+# f16-K + q8_0-V contract that endpoint was built for. 2026-10-03 (Frank):
+# the pair is now f16/f16 — two wide slots freed the VRAM, so the values are
+# unquantized too. The per-tensor flags stay explicit either way.
 #
-# MTP: the `-mtp` build carries the NextN block (blk.64.nextn.*), and
-# `--spec-type draft-mtp` makes llama.cpp load it and run the draft context on
-# the SAME weights. There is no --model-draft here on purpose: a separate draft
-# path would load a second 12 GB copy of a 12 GB model. Draft depth is
-# llama.cpp's default (--spec-draft-n-max 3).
+# Architecture: qwen3_5 hybrid — 32 layers, 8 full-attention + 24 Gated
+# DeltaNet, native 262144 window. KV lives only in the 8 attention layers;
+# the DeltaNet state is per-slot fixed size. Vision comes from the repo's
+# separate projector (the Q6_K GGUF carries no `v.*` tensors at all), so
+# --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is the documented
+# text-only opt-out.
 #
-# Vision: --mmproj loads the BF16 projector, so image content parts work. The
-# fleet's image path moved here from frank-large on 2026-09-16, which is why
-# halogen no longer sets HALOGEN_VISION_TOWER.
+# Sampling: llama.cpp takes it per request (the fleet harness sends its
+# recipe from models.yml); greedy is the harness default.
 #
-# Sampling: greedy by default (the fleet contract). Clients that send their own
-# temperature get their own values, and the harness overrides per model from
-# models.yml. --jinja is pinned even though this build defaults it on: the
-# model's own template supplies reasoning_effort, preserve_thinking and the
-# tool-call grammar, and a future build that flips that default must not
-# silently take tools away.
-#
-# Tunables come from frank-small.env unless FRANK_SMALL_ENV_FILE overrides it.
+# Tunables come from frank-small.env unless FRANK_SMALL_ENV_FILE overrides.
 set -euo pipefail
 
 SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
@@ -37,68 +32,68 @@ if [[ -f "$ENV_FILE" ]]; then
 fi
 
 # --- defaults (used when the env file does not set them) -------------------
-: "${LLAMA_BIN:=/home/dank/GitPackages/llama.cpp-mainline/build-cuda/bin/llama-server}"
-: "${MODEL_DIR:=/home/dank/models/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp}"
-: "${MODEL_FILE:=Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf}"
-: "${MMPROJ_PATH:=$MODEL_DIR/mmproj-Qwen3.8-27B-BF16.gguf}"
+: "${LLAMA_BIN:=/home/dank/GitPackages/llama-master/build-cuda/bin/llama-server}"
+: "${MODEL_FILE:=/home/dank/models/Ornith-Q8/Ornith-1.5-9B-Q8_0.gguf}"
 : "${ALIAS:=frank-small}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-: "${CTX_SIZE:=131072}"
-: "${PARALLEL:=1}"
+# ctx-size is the POOL llama.cpp divides across slots: 401408 / 2 gives
+# 200,704 per slot — two independent conversations, each one wide enough to
+# hold a whole frank-primary-sized prompt in a single shot (the native 262,144
+# no longer fits twice on a 24 GiB card once the vision tower is loaded;
+# frank-small.env carries the three measurements that set this number).
+: "${CTX_SIZE:=401408}"
+: "${PARALLEL:=2}"
+# Compute-buffer geometry. Before 2026-10-03 this endpoint never passed either
+# flag, so it ran on llama.cpp's own defaults (2048/512). The pair is 4096/2048
+# since 2026-10-05: on this CUDA card the prefill curve climbs all the way to
+# ubatch 2048 (unlike the R9700 layer-split sibling, where 512 is the peak),
+# and the logical batch is VRAM- and throughput-neutral. frank-small.env
+# carries the sweep tables and the VRAM risk note.
+: "${BATCH_SIZE:=4096}"
+: "${UBATCH_SIZE:=2048}"
 : "${CACHE_TYPE_K:=f16}"
 : "${CACHE_TYPE_V:=f16}"
-: "${BATCH_SIZE:=2048}"
-: "${UBATCH_SIZE:=512}"
-: "${NUM_THREADS:=8}"
-: "${THREAD_BATCH:=8}"
-: "${NGL:=999}"
-: "${FLASH_ATTN:=on}"
-: "${SPEC_TYPE:=draft-mtp}"
-: "${TEMPERATURE:=0.0}"
-: "${REASONING_FORMAT:=deepseek}"
-: "${REASONING_BUDGET:=-1}"
+: "${NGL:=99}"
+# Vision projector. No colon on purpose: an explicitly empty value means
+# "text-only, on purpose" and must NOT be replaced by the default path.
+: "${MMPROJ_FILE=/home/dank/models/Ornith-Q8/mmproj-Ornith-1.5-9B-BF16.gguf}"
 
-# Fail loudly rather than half-loading a model into VRAM: a missing file here
-# is a download that has not finished, and the unit's restart policy would
-# otherwise turn that into a crash loop that reads like a runtime fault.
-for path in "$LLAMA_BIN" "$MODEL_DIR/$MODEL_FILE" "$MMPROJ_PATH"; do
+# Fail loudly rather than half-loading: a missing file is a download that
+# has not finished, and Restart=always would turn it into a crash loop that
+# reads like a runtime fault.
+for path in "$LLAMA_BIN" "$MODEL_FILE"; do
     [[ -e "$path" ]] || { echo "frank-small: missing $path" >&2; exit 1; }
 done
+EXTRA_ARGS=()
+if [[ -n "$MMPROJ_FILE" ]]; then
+    [[ -e "$MMPROJ_FILE" ]] || { echo "frank-small: missing $MMPROJ_FILE" >&2; exit 1; }
+    EXTRA_ARGS+=(--mmproj "$MMPROJ_FILE")
+fi
 
 # The UVM device node is created lazily by nvidia-modprobe on the first CUDA
 # open after boot; make sure it exists before the server needs it.
 nvidia-modprobe -u -c=0 2>/dev/null || true
 
-MMPROJ_ARGS=()
-if [[ -n "${MMPROJ_PATH:-}" && -f "$MMPROJ_PATH" ]]; then
-    MMPROJ_ARGS=(--mmproj "$MMPROJ_PATH")
-fi
-
-SPEC_ARGS=()
-if [[ -n "${SPEC_TYPE:-}" && "$SPEC_TYPE" != "none" ]]; then
-    SPEC_ARGS=(--spec-type "$SPEC_TYPE")
-fi
-
-exec "$LLAMA_BIN" \
-    --model "$MODEL_DIR/$MODEL_FILE" \
-    "${MMPROJ_ARGS[@]}" \
-    --alias "$ALIAS" \
-    --host "$BIND_ADDR" \
-    --port "$LISTEN_PORT" \
-    --gpu-layers "$NGL" \
-    --flash-attn "$FLASH_ATTN" \
-    --ctx-size "$CTX_SIZE" \
-    --parallel "$PARALLEL" \
-    --batch-size "$BATCH_SIZE" \
-    --ubatch-size "$UBATCH_SIZE" \
-    --threads "$NUM_THREADS" \
-    --threads-batch "$THREAD_BATCH" \
-    --cache-type-k "$CACHE_TYPE_K" \
-    --cache-type-v "$CACHE_TYPE_V" \
-    "${SPEC_ARGS[@]}" \
-    --jinja \
-    --metrics \
-    --temp "$TEMPERATURE" \
-    --reasoning-format "$REASONING_FORMAT" \
-    --reasoning-budget "$REASONING_BUDGET"
+# Flash attention stays pinned on: it is the fast path for f16 KV, and it is
+# the only path that could serve quantized V at all — so if CACHE_TYPE_V is
+# ever set back to q8_0 this flag must not be lost, or the build falls back
+# silently and the KV contract in frank-small.env becomes a lie.
+ARGS=(
+    --model "$MODEL_FILE"
+    --alias "$ALIAS"
+    --host "$BIND_ADDR"
+    --port "$LISTEN_PORT"
+    --ctx-size "$CTX_SIZE"
+    --parallel "$PARALLEL"
+    --batch-size "$BATCH_SIZE"
+    --ubatch-size "$UBATCH_SIZE"
+    --n-gpu-layers "$NGL"
+    --flash-attn on
+    --cache-type-k "$CACHE_TYPE_K"
+    --cache-type-v "$CACHE_TYPE_V"
+    --jinja
+    --reasoning auto
+    --reasoning-format deepseek
+)
+exec "$LLAMA_BIN" "${ARGS[@]}" ${EXTRA_ARGS[@]+"${EXTRA_ARGS[@]}"}

scripts/signal-relay.py

@@ -0,0 +1,176 @@
+#!/usr/bin/env python3
+"""Alertmanager-to-Signal relay for the homelab (Dankclaw).
+Listens on 192.168.0.143:18093 (the host's LAN IP) and receives
+Alertmanager's webhook payload from the container as
+host.containers.internal:18093 — netavark delivers that address to the
+host's LAN IP, which is the one bind a bridge container can reach (a
+loopback bind refuses it, and 169.254.1.2 is not a local address here).
+The group is formatted into a short message and handed to signal-cli under
+the existing account on this host. Runs as a systemd user unit
+(scripts/signal-relay.service); logs go to the journal.
+
+The phone number and account name are numbers, not credentials — the
+credential is the signal-cli account state under
+~/.local/share/signal-cli, which never leaves the host.
+
+Stdlib only: this box's user units run the system python3.
+"""
+
+import json
+import shutil
+import subprocess
+import sys
+import time
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+# The bind address is the one detail this design gets wrong twice
+# (2026-09-17): a loopback bind refuses container traffic ("dial tcp
+# 169.254.1.2:18093: connection refused"), and 169.254.1.2 itself is not a
+# local address on this host ("Cannot assign requested address") — netavark
+# runs the bridges in aardvark-dns' netns, not the host's. Traffic that
+# arrives as 169.254.1.2 lands on the host's LAN IP, so bind THAT. The
+# deployer installs an iptables INPUT rule that admits 18093/tcp from the
+# podman bridge subnet (10.89.0.0/16) only, keeping the unauthenticated
+# endpoint off the rest of the LAN.
+LISTEN_ADDR = ("192.168.0.143", 18093)
+# The account is addressed by phone number: signal-cli 0.14.7's -a flag
+# parses an all-digit value as a phone number and never falls back to the
+# accounts.json name (the numeric name 318311 dies with "Invalid account
+# (phone number)"). The number is a number, not a credential — the
+# credential is the account state under ~/.local/share/signal-cli.
+SIGNAL_ACCOUNT = "+12062476336"
+SIGNAL_TO = "+12062476336"
+SIGNAL_CLI = "/usr/local/bin/signal-cli"
+
+# The relay is a safety net, not the rate limiter: Alertmanager already
+# repeats on repeat_interval. This suppresses exact duplicates that arrive
+# inside the window (config reloads, double-fires) so a misconfiguration
+# cannot fill the phone.
+DEDUPE_SECONDS = 15 * 60
+_last_sent = {}
+
+# Cap on the instances listed per group; the count is always stated.
+MAX_INSTANCES = 3
+
+
+def log(msg):
+    print(f"signal-relay: {msg}", file=sys.stderr, flush=True)
+
+
+def format_group(alerts):
+    """One message per webhook delivery (Alertmanager groups by alertname)."""
+    if not alerts:
+        return None
+    first = alerts[0]
+    name = first.get("labels", {}).get("alertname", "unknown")
+    state = "firing" if first.get("status") == "firing" else "resolved"
+    severity = first.get("labels", {}).get("severity", "info").upper()
+    lines = [f"[{severity}] {name} ({len(alerts)} {state})"]
+    for a in alerts[:MAX_INSTANCES]:
+        labels = a.get("labels", {})
+        ident = labels.get("instance") or labels.get("job") or "?"
+        summary = (a.get("annotations") or {}).get("summary")
+        line = f"  {ident}"
+        if summary:
+            line += f": {summary}"
+        lines.append(line)
+    if len(alerts) > MAX_INSTANCES:
+        lines.append(f"  … and {len(alerts) - MAX_INSTANCES} more")
+    return "\n".join(lines)
+
+
+def dedupe_key(alerts):
+    first = alerts[0]
+    labels = tuple(sorted(first.get("labels", {}).items()))
+    return (first.get("status"),) + labels
+
+
+def send_signal(text):
+    if shutil.which(SIGNAL_CLI) is None:
+        log(f"signal-cli not found at {SIGNAL_CLI}")
+        return False
+    try:
+        proc = subprocess.run(
+            [SIGNAL_CLI, "-a", SIGNAL_ACCOUNT, "-o", "json", "send", SIGNAL_TO,
+             "-m", text],
+            capture_output=True, text=True, timeout=30,
+        )
+    except subprocess.TimeoutExpired:
+        log("signal-cli timed out after 30s")
+        return False
+    if proc.returncode != 0:
+        log(f"signal-cli failed rc={proc.returncode}: {proc.stderr.strip()[:300]}")
+        return False
+    return True
+
+
+def handle_alertmanager(payload):
+    alerts = payload.get("alerts", [])
+    text = format_group(alerts)
+    if text is None:
+        log("empty alert group, nothing sent")
+        return
+    now = time.monotonic()
+    key = dedupe_key(alerts)
+    # Prune in place: rebinding _last_sent here would make it local to this
+    # function and the read above the assignment raises UnboundLocalError
+    # (measured 2026-09-17: every webhook died on it).
+    stale = [k for k, v in _last_sent.items() if now - v >= DEDUPE_SECONDS]
+    for k in stale:
+        del _last_sent[k]
+    if key in _last_sent:
+        log(f"suppressed duplicate: {key[0]}")
+        return
+    if send_signal(text):
+        _last_sent[key] = now
+        log(f"sent to {SIGNAL_TO}: {text.splitlines()[0]}")
+    else:
+        # Do not mark a failed send: the next repeat should retry.
+        log("send failed; will retry on the next delivery")
+
+
+class Handler(BaseHTTPRequestHandler):
+    def do_GET(self):
+        if self.path == "/health":
+            self.send_response(200)
+            self.end_headers()
+            self.wfile.write(b"ok")
+        else:
+            self.send_response(404)
+            self.end_headers()
+
+    def do_POST(self):
+        if self.path != "/alertmanager":
+            self.send_response(404)
+            self.end_headers()
+            return
+        length = int(self.headers.get("Content-Length", 0))
+        try:
+            payload = json.loads(self.rfile.read(length))
+        except (ValueError, UnicodeDecodeError) as e:
+            log(f"bad payload: {e}")
+            self.send_response(400)
+            self.end_headers()
+            return
+        try:
+            handle_alertmanager(payload)
+        except Exception as e:  # a relay bug must not 500 Alertmanager
+            log(f"handler error: {e!r}")
+        self.send_response(200)
+        self.end_headers()
+        self.wfile.write(b"ok")
+
+    def log_message(self, fmt, *args):
+        # Default access log is per-request noise; the journal keeps the
+        # send/suppress lines, which are the ones that matter.
+        pass
+
+
+def main():
+    server = ThreadingHTTPServer(LISTEN_ADDR, Handler)
+    log(f"listening on {'.'.join(map(str, LISTEN_ADDR))}, account {SIGNAL_ACCOUNT}")
+    server.serve_forever()
+
+
+if __name__ == "__main__":
+    main()

scripts/signal-relay.service

@@ -0,0 +1,37 @@
+# Systemd USER unit: signal-relay (Dankclaw).
+#
+# Install path: ~/.config/systemd/user/signal-relay.service
+# Installed and enabled by install_signal_relay in scripts/deploy-remote.sh
+# (part of the alertmanager deploy).
+#
+# The webhook consumer for Alertmanager: it listens on loopback only and is
+# the only thing on this host that talks to signal-cli. It is a plain user
+# unit, not a quadlet, on purpose:
+#
+#   * it runs python3 straight from the checkout, so a deploy updates the
+#     relay without a copy step (the homepage timer copies to
+#     /usr/local/bin; this one has no root side);
+#   * generated quadlet units cannot be enabled on this host's podman 5.x
+#     v1 generator — a hand-written user unit can, and with Linger=yes for
+#     dank it starts at boot;
+#   * it needs no container: its only dependencies are /usr/bin/python3
+#     (system) and /usr/local/bin/signal-cli (host install, account state
+#     under ~/.local/share/signal-cli).
+#
+# Loopback-only for the same reason every observability port is: an
+# unauthenticated message-sending endpoint has no business on a host
+# interface. Alertmanager reaches it from its container via
+# host.containers.internal.
+
+[Unit]
+Description=Alertmanager-to-Signal relay (Dankclaw)
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+ExecStart=/usr/bin/python3 /home/dank/serverAppConfigs/scripts/signal-relay.py
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=default.target

scripts/test-changed.sh

@@ -180,18 +180,30 @@ for path in "${CHANGED[@]}"; do
         scripts/mount-entropy.sh)
             add_test tests/mount_entropy.bats
             ;;
-        # frank-medium is llama.cpp on the two R9700s as a ROOT unit since
-        # 2026-09-16: a native gfx1201 HIP binary, so (like frank-small) its
-        # env, unit and serve script all live in scripts/, and this case has to
-        # precede the generic scripts/*.service fallback below. The retired
-        # vllm-radiance service dir is under dankserver/.AGENTTRASH/, which is
-        # archive material and maps to nothing.
-        scripts/frank-medium.*|scripts/serve-frank-medium.sh)
-            add_test tests/frank_medium_unit.bats
+        # Archive material: a path under any .AGENTTRASH/ directory is
+        # retired-by-policy (deploy-plan.py plans nothing from it), so it
+        # selects no focused test. It still needs its own case: without one
+        # the wrapper refuses retirement commits, whose trash moves show up
+        # as additions, with "no focused-test mapping".
+        .AGENTTRASH/*|*/.AGENTTRASH/*)
+            ;;
+        # frank-primary is llama.cpp (the unsloth b11030-mix prebuilt) on the
+        # two R9700s as a ROOT unit — the 2026-09-26 swap that replaced the
+        # Lucebox DS4.1 quadlet on :7770. Its env, unit and serve script all
+        # live in scripts/, which is why this case has to precede the generic
+        # scripts/*.service and scripts/* fallbacks below.
+        scripts/frank-primary.*|scripts/serve-frank-primary.sh)
+            add_test tests/frank_primary_unit.bats
             add_test tests/llm_units.bats
             add_test tests/deployment_scoping.bats
             add_test tests/scripts_syntax.bats
             ;;
+        # frank-large's chat template lives in scripts/ (moved there on
+        # 2026-09-26 when its old home was retired): a template edit changes
+        # what the live endpoint renders, so it selects the large unit seam.
+        scripts/qwen-sharp-v22.4.0.jinja)
+            add_test tests/frank_large_unit.bats
+            ;;
         # frank-small is llama.cpp on the 3090 as a ROOT unit: a native CUDA
         # binary, so no generated quadlet is involved. Its env, unit and serve
         # script all live in scripts/, which is why this case has to precede

serverAppConfigs-docs/Architecture/Deployment Flow.md

@@ -62,13 +62,14 @@ service on the other host.
 |---|---|---|
 | Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
 | SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
-| `frank-large` | systemd + `systemctl` | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
-| `frank-medium` | systemd + `systemctl` | Retires the legacy frank-small unit on first deploy, then relinks and restarts only the inference service; waits up to 20 minutes for `/health`. |
+| `frank-large` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
+| `frank-small` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
+| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
 | Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
 
 The shared chat template maps to the inference service on the selected target.
-The inactive `frank-npu` unit is relink-only when explicitly selected;
-it is not restarted by an unrelated large-model deployment.
+The `frank-npu` endpoint is retired (2026-09-11) and has no deploy handler;
+an unrelated large-model deployment never touches it.
 
 ## Caddy safety
 

serverAppConfigs-docs/Architecture/E2E Suite.md

@@ -1,3 +1,13 @@
+---
+type: reference
+description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts"
+tags:
+  - architecture
+  - e2e
+  - homepage
+timestamp: 2026-09-01
+---
+
 # E2E Suite — homepage + guard chain (Playwright)
 
 `tests/e2e/` drives the **live fleet** through a real browser: fresh contexts
@@ -10,11 +20,12 @@ user sees it; the `bats` seams assert config shape, not behavior.
 
 | Path | Role |
 |------|------|
-| `playwright.config.ts` | four projects (below), real-browser UA, trace on failure |
+| `playwright.config.ts` | five projects (below), real-browser UA, trace on failure |
 | `specs/00-auth.setup.ts` | setup project: two real authentik logins, saves storage states |
 | `specs/10-homepage.spec.ts` | user panel set + open links derived from `services.json` (registry-derived — no hardcoded service lists) |
 | `specs/11-admin.spec.ts` | `admin` project: admin grid + `/admin/` RBAC (403 for standard user, members-only 200) |
 | `specs/12-crawl.spec.ts` | every user open link reaches its real app through the guard chain |
+| `specs/19-frankbook.spec.ts` | `frankbook` project: walks the six admin links that terminate on FrankBook-Pro (192.168.0.112) — conditional, see below |
 | `specs/13-sso.spec.ts` | direct guarded-vhost visits resolve silently with ONLY the homepage session (single-login enforcement) |
 | `specs/14-guard.spec.ts` | `guest` project, no storage state: unauthenticated traffic lands on the IdP login |
 | `lib/login.ts` | `akLogin` helper (two-stage flow, shadow-DOM locators) |
@@ -28,7 +39,16 @@ user sees it; the `bats` seams assert config shape, not behavior.
   `.auth/qa-e2e{,-admin}.json` storage states.
 - **standard** — 10/12/13 with the user state.
 - **admin** — 11 with the admin state.
-- **guest** — 14 with an empty state.
+- **guest** — 14/15 with an empty state.
+- **frankbook** — 19 with the admin state. **Excluded from every default run**: the
+  *arr/qBittorrent links terminate on FrankBook-Pro, which sleeps when idle, and a
+  sleeping Mac mini must never fail an unrelated deploy. The project joins only
+  when `E2E_INCLUDE_FRANKBOOK=1` (set by `scripts/deploy.sh` when a frankbook
+  trigger path changed: `tests/e2e/specs/19-frankbook.spec.ts`, `frankbook-pro/**`,
+  `shared/homepage/services.json`, or the `dankclaw/Caddyfile` serving the arr
+  vhosts) or when an explicit `--project` argument names it. An `11-admin` edit
+  that touches the arr links must therefore be paired with a `19-frankbook` run
+  against an awake host before it is trusted live.
 
 Credentials live in `tests/e2e/.env.local` (git-ignored; placeholders in
 `.env.local.example`). The scaffold bats seam fails the commit if creds or
@@ -37,8 +57,9 @@ auth states are ever tracked.
 ## Run
 
 ```bash
-cd tests/e2e && npx playwright test            # all projects
-npx playwright test --project=standard 12-crawl.spec.ts   # one slice
+tests/e2e/run.sh                          # default set — frankbook project excluded
+E2E_INCLUDE_FRANKBOOK=1 tests/e2e/run.sh  # default set + the frankbook walk
+cd tests/e2e && npx playwright test       # ALL projects, frankbook included
 ```
 
 Requires LAN reachability of `www.franksdank.cloud` (Pi-hole DNS or public).

serverAppConfigs-docs/Architecture/Network Architecture.md

@@ -1,3 +1,12 @@
+---
+type: reference
+description: "One Caddy fronts both hosts."
+tags:
+  - architecture
+  - network
+timestamp: 2026-09-02
+---
+
 # Network Architecture
 
 One Caddy fronts both hosts. It runs as a container on Dankclaw (192.168.0.143) and terminates TLS for every service on the LAN. Pi-hole on Dankclaw answers the service names for LAN clients; Porkbun answers them for the WAN. The G36 router firewall governs external access. SearXNG outbound traffic exits through the Gluetun VPN container.

serverAppConfigs-docs/Architecture/Storage Policy.md

@@ -1,3 +1,12 @@
+---
+type: reference
+description: "All persistent data uses **bind mounts**: a host directory mapped directly into the container."
+tags:
+  - architecture
+  - storage
+timestamp: 2026-09-01
+---
+
 # Storage Policy
 
 All persistent data uses **bind mounts**: a host directory mapped directly into the container. No named volumes. This matches the preference of this repo and the existing Gitea stack on Dankclaw.

serverAppConfigs-docs/Architecture/Testing Policy.md

@@ -1,3 +1,12 @@
+---
+type: reference
+description: "This repository tests its configuration at defined seams."
+tags:
+  - architecture
+  - testing
+timestamp: 2026-09-01
+---
+
 # Testing Policy
 
 This repository tests its configuration at defined seams. The tests run with `bats`; run them with `./tests/run.sh` before every push.

serverAppConfigs-docs/Dankclaw/ComfyUI.md

@@ -1,3 +1,12 @@
+---
+type: reference
+description: "ComfyUI is a node-based AI image generation engine."
+tags:
+  - dankclaw
+  - comfyui
+timestamp: 2026-09-16
+---
+
 # ComfyUI
 
 ComfyUI is a node-based AI image generation engine. Workflows are built from nodes that load models, sample latents, and post-process images; everything is scriptable through a web UI and an API.
@@ -13,6 +22,7 @@ https://github.com/Comfy-Org/ComfyUI
 |------|-------|
 | Image | `comfyui-rocm:gfx1151` — built from `dankclaw/Containerfile.comfyui` |
 | Base image | `docker.io/rocm/vllm:rocm7.12.0_gfx1151_ubuntu24.04_py3.12_pytorch_2.9.1_vllm_0.16.0` |
+| Upstream latest | 0.37.0 (2026-09-21) — ships **native Qwen-Image 2.1** support (CORE-423, `comfy/ldm/qwen_image21/`). The image's custom GGUF patches may be unnecessary on the next rebuild; check before re-applying. |
 | Port | 8188 (host) → 8188 (container) |
 | Backend | ROCm GPU (gfx1151 / Radeon 8060S). No `--cpu` flag. |
 | Data | `/home/dank/comfyui-data/{models,input,output,custom_nodes}` → `/opt/ComfyUI/...` |