docs: wrap up the 2026-09-19 fleet cutover

frank-large -> CIRU v4.4.1 Orca (new runbook), frank-medium -> the deployed
orcarouter Q8_0 / DFlash2 Q4_K_M / balanced 32:33 / 327680 state. The d374ab9
frank-medium deploy updated only scripts/tests and left AGENTS.md, the runbook
and both indexes on the predecessor (ajgazin Swift UD-Q8_K_XL, 35:30, 393216).

- AGENTS.md: frank-large section (CIRU Orca, root unit, runtime gate, PLE, MTP)
  + frank-medium section and topology row to the deployed shape
- new runbook frank-large-ciru-orca.md
- frank-medium runbook header/intro: orcarouter Q8_0 swap, 32:33, 327680;
  predecessor kept as calibration history
- LLM index.md: intro + mermaid (halogen->CIRU, GSQ-RCO->Gemma4, Swift->orcarouter),
  add ciru-orca, retire halogen
- top-level index.md: add ciru-orca link (docs-index seam), retire halogen,
  fix frank-medium entry

AGENTS.md

@@ -12,7 +12,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
   Calibre-Web, Jellyfin, Plex) plus the `frank-medium` LLM service.
 - **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
   via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
-  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (halogen-flash-server on
+  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (CIRU v4.4.1 llama.cpp on
   the gfx1151 iGPU) and `frank-small` (llama.cpp on the RTX 3090). Caddy on
   Dankclaw reverse-proxies both hosts.
 - **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
@@ -24,8 +24,8 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
-|Dankserver|`frank-medium`|mainline llama.cpp master fb27a525 (root systemd unit, gfx1201 HIP build)|ajgazin Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP UD-Q8_K_XL with the model's own MTP head (DFlash2 sidecar kept on disk, one env change away), 2x Radeon AI PRO R9700 in ONE layer-split process (35:30 — skewed toward the x16-slot card; the other sits behind a Gen4 x1 chipset path), 393216 ctx over 2 slots of 196608, batch 6144x512, vision (BF16 mmproj)|7770|
-| Dankclaw | `frank-large` | halogen-flash-server 0.5.8 (rootless Quadlet) | peonist-ai Qwen3.8-Flash-Next w4b `.hgn` on the gfx1151 iGPU (Strix Halo LPDDR), 200192 ctx per conversation over one 400384-position KV pool / 2 slots, text only | 7770 |
+|Dankserver|`frank-medium`|mainline llama.cpp master fb27a525 (root systemd unit, gfx1201 HIP build)|orcarouter Qwen3.8-27B-Uncensored Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), 2x Radeon AI PRO R9700 in ONE layer-split process (32:33 balanced; ROCm0 Gen4 x16, ROCm1 Gen4 x1 chipset path), 327680 ctx over 2 slots of 163840, batch 6144x512, f16 KV + 16 GiB host-RAM prompt cache, vision (BF16 mmproj)|7770|
+| Dankclaw | `frank-large` | CIRU v4.4.1 llama.cpp (QSA window hotfix, root systemd unit, gfx1151 HIP build) | jcbtc Qwen3.8-Flash-CIRU-STRIX-Orca on the gfx1151 iGPU (Strix Halo LPDDR), 262144 ctx single stream (1 slot), MTP depth 4, PLE 4096 MiB, text only | 7770 |
 | Dankclaw | `frank-small` | mainline llama.cpp master 37b53fd (root systemd unit) | HauhauCS Gemma4-12B-QAT-Uncensored-Balanced Q4_K_M (BF16 mmproj + MTP assistant sidecar, `draft-mtp,ngram-mod`) on the RTX 3090, 1048576 pool / 4 slots of 262,144, q8_0 KV, the fleet's vision endpoint | 7771 |
 
 Retired endpoints (`frank-npu`, and every prior runtime) are in
@@ -68,8 +68,8 @@ All edits flow through git. Never edit files on a remote machine directly.
 The planner maps changed paths to services; the remote runner restarts only
 the selected service and records its revision outside the checkout.
 Documentation-only changes are a no-op. LLM services are selected only by
-their own files — frank-large by `dankclaw/frank-large/quadlet/`, frank-small
-and frank-medium by their unit/env/serve trio in `scripts/` — so unrelated
+their own files — all three by their unit/env/serve trio in `scripts/` — so
+unrelated
 container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
 runs on a host without passwordless sudo; the value never enters the repo.
 
@@ -207,9 +207,8 @@ archive.
 
 **STOP the production service BEFORE benchmarking** — these servers serve
 real users, and a bench beside a live service starves the GPU and corrupts
-results. Use the runtime's own harness: `llama-bench` (llama.cpp), `vllm
-bench` (vLLM), and halogen-flash-server's own `bench`/`sweep` modes for
-frank-large, which is the active runtime there again. Rolling your own bench
+results. Use the runtime's own harness: `llama-bench` for every llama.cpp
+endpoint (all three are llama.cpp now). Rolling your own bench
 requires asking first.
 Scratch ports are not used on LLM hosts — a
 test process cannot share the single GPU anyway; validate against the service
@@ -227,11 +226,8 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
 merely wedged kernel usually still answers ARP.
 
 1. Stop and mask (self-healing would restart it):
-   - Dankclaw `frank-large` (rootless Quadlet again since 2026-09-16):
-     `systemctl --user stop frank-large`, then move its
-     `~/.config/containers/systemd/frank-large.container` aside and
-     `systemctl --user daemon-reload` — the quadlet generator re-creates that
-     unit from the file on every reload, so the source is the lever, not `mask`.
+   - Dankclaw `frank-large` (root unit, CIRU llama.cpp on the iGPU):
+     `sudo systemctl stop frank-large && sudo systemctl mask frank-large`
    - Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
      `sudo systemctl stop frank-small && sudo systemctl mask frank-small`
    - Dankserver `frank-medium` (system unit):
@@ -264,42 +260,45 @@ provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
 `models.yml`, then fix every `modelRoles` entry that named the old endpoint —
 a role pointing at a provider the file does not define fails at call time. As
 of 2026-09-16 the image path is `frank-small` (`input: [text, image]`), and
-`frank-large` is text-only because its halogen deployment loads no vision
-tower. There is **no audio** input anywhere in the fleet: that capability left
-with the FLM NPU endpoint on 2026-09-11.
+`frank-large` is text-only because its CIRU deployment loads no vision
+projector (the F16 mmproj is on disk, one env flip away). There is **no audio**
+input anywhere in the fleet: that capability left with the FLM NPU endpoint on
+2026-09-11.
 
-### frank-large — halogen-flash-server (Dankclaw, ACTIVE)
+### frank-large — CIRU v4.4.1 Orca (Dankclaw, ACTIVE)
 
-halogen-flash-server 0.5.8 as a **rootless Quadlet again since 2026-09-16**
-(`dankclaw/frank-large/quadlet/frank-large.container`), serving
-`peonist-ai/halogen-qwen3.8-flash-next` (115.55 GiB w4b `.hgn` checkpoint + a
-quality overlay that re-fetched to 2.40 GiB on 2026-09-16) on port 7770 from
-the gfx1151 iGPU. Two conversations of 200,192 tokens resident (pool 400,384,
-2 slots), greedy default, reasoning extracted to `reasoning_content`, and **no
-vision tower** — that moved to frank-small. It replaced ik_llama.cpp, which had
-replaced it for two days.
+CIRU v4.4.1 (the QSA window hotfix) as a **root systemd unit** (`scripts/
+frank-large.service`, `User=dank`), serving `jcbtc/Qwen3.8-Flash-CIRU-STRIX-
+Orca` (79.4 GiB main GGUF + a 4.1 GiB MTP head + a 52.4 GiB PLE payload, all
+sha256-verified) on port 7770 from the gfx1151 iGPU. One stream, the model's
+FULL 262,144 trained window (`PARALLEL_SLOTS=1`; the MTP drafter needs exactly
+one slot in this runtime), MTP depth 4, PLE cache 4096 MiB, reasoning extracted
+to `reasoning_content`, and **no vision projector** (the F16 mmproj is on disk,
+`ENABLE_VISION=0` — one env flip to enable). It replaced halogen-flash-server,
+which had replaced ik_llama.cpp.
 
-The engine **fetches its own weights on first start** (`HALOGEN_DOWNLOAD`, into
-the `/home/dank/models/halogen` bind mount): clearing that directory and
-restarting the unit is the supported way to pick up upstream updates, and it
-needs the directory to exist **empty** — a missing directory fails the
-container with `Error: statfs /home/dank/models/halogen: no such file or
-directory` and the unit loops on `Restart=on-failure` until `mkdir -p` brings
-it back. The sidecars are versioned in place, so a stale copy is invisible from
-the file name (both overlays were ~95 MB larger upstream than the copies this
-host had been serving).
+The runtime is a **private host build** (`/home/dank/GitPackages/ciru-runtime-
+v4.4.1-qsa-hotfix/`), not a container image: a gfx1151 HIP build of the CIRU
+llama.cpp fork with its own bundled ROCm SDK in a venv (`.venv-rocm/`), and the
+binary resolves the SDK libs via RUNPATH. The launcher (`scripts/ciru/
+run-orca-server.sh` → `run-server.sh`) has a **runtime gate**: it exits 2 unless
+`${repo_root}/runtime/hip/lib/libamdhip64.so` and `runtime/rocr/lib/
+libhsa-runtime64.so` exist. The serve script (`scripts/serve-frank-large.sh`)
+satisfies that gate idempotently by `ln -sfn`-ing both to the venv's unversioned
+devel symlinks, then `exec`s the launcher with `--chat-template-kwargs
+'{"preserve_thinking":true}'` (Qwen3.8's template needs this to carry thinking
+blocks across turns; a request-level value still wins).
 
-Measured boot ledger (2026-09-16, `HALOGEN_VERBOSE=1` ships in the unit):
-device budget `121 − 67.7 − 20 = 33.5 GiB` against a need of `~27.2 GiB`, so
-the configured pool was **not clamped**; 68.0 GiB of weights locked + 11.0 GiB
-of KV pool + 16.5 GiB of working memory = 95.4 GiB in all, leaving 14.7 GiB
-(475 contiguous 2 MiB blocks) for everything else: frank-small on the 3090 and
-~28 co-tenant containers. The engine printed its co-tenant stall NOTE and
-logged 4,350 compaction stalls (285 failed) while reserving the pool, so read
-that as "tight and fragmented", not "out of memory".
+The serve script validates launcher → built binary → venv runtime → main GGUF →
+MTP head → PLE payload + manifest + scale, and exits with a pointer message if
+any is missing. `PLE_CACHE_MIB=4096` is the vendor production-profile default;
+the 52.4 GiB PLE is mmap'd and paged, not resident.
 
-Runbook: `serverAppConfigs-docs/LLM/halogen-flash-server.md`.
-Rollback: [[frank-large-ik-uncensored]].
+The halogen Quadlet (`dankclaw/frank-large/quadlet/`) was retired to
+`.AGENTTRASH/` — one owner per endpoint, now the scripts/ trio.
+
+Runbook: `serverAppConfigs-docs/LLM/frank-large-ciru-orca.md`.
+Rollback: [[halogen-flash-server]].
 
 ### frank-small — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
 
@@ -327,25 +326,25 @@ ubatch 2048 OOMs the draft's pp buffer, and why draft depth 4 + poll
 Runbook: `serverAppConfigs-docs/LLM/frank-small-gemma4-12b.md`.
 Rollback: [[frank-small-qwen38-gsq-rco]].
 
-### frank-medium — llama.cpp layer split, MTP drafter (Dankserver, ACTIVE)
+### frank-medium — llama.cpp layer split, DFlash2 drafter (Dankserver, ACTIVE)
 
 Mainline llama.cpp at **`fb27a525`** (a gfx1201 HIP build, `Root=`-less root
 systemd unit `scripts/frank-medium.service`) serving
-`ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF` at **UD-Q8_K_XL**
-(29.29 GiB) with the model's OWN **MTP head** (`blk.<n>.nextn.*` in the file;
-the z-lab DFlash2 `Q8_0` sidecar stays on disk and in the fetch script — one
-`SPEC_TYPE` change brings the faster chain back), in **one process across both
-R9700s**, `--split-mode layer` at a **35:30 tensor split** — one layer moved
-to ROCm0 on 2026-09-17 because the cards' PCIe paths are not symmetric:
-ROCm0 (03:00.0) is CPU-root-port Gen4 x16 (~31.5 GB/s) while ROCm1
-(0b:00.0) rides the B650 chipset at Gen4 x1 (~1.97 GB/s), and with the
-drafter, output layers and heavier compute buffers pinned to the
-last-layers card, the bottlenecked card was running 98% while the x16 card
-sat at 40%.  36:29 would cross a full-attention layer and fails both the
-1 GiB floor and the fit slack, so 35:30 is the farthest safe shift at this
-pool.  393216 ctx over **2 slots** of 196608, batch 6144x512, vision
-via the BF16 mmproj.  DSpark was
-put up in the drafter's place the same day and reverted: it declares no sliding
+`orcarouter/Qwen3.8-27B-Uncensored-GGUF` at **Q8_0** (27.0 GiB) with the
+**z-lab DFlash2 `Q4_K_M` drafter** (1.06 GiB). The target file also carries the
+model's own MTP head (`blk.<n>.nextn.*`, `nextn_predict_layers=1`), so
+`draft-mtp` is one `SPEC_TYPE` change away — the drafter file stays on disk. In
+**one process across both R9700s**, `--split-mode layer` at a **32:33 tensor
+split** — balanced (operator call 2026-09-19, the most even split 65 layers
+allow; ROCm1, the last-layers card that also carries the drafter and output
+layers, holds the extra one). The predecessor's 35:30 skew (2026-09-17) leaned
+work onto the x16 card because ROCm1 (0b:00.0) rides the B650 chipset at Gen4
+x1 (~1.97 GB/s) while ROCm0 (03:00.0) is CPU-root-port Gen4 x16 (~31.5 GB/s);
+the balanced point trades that lean for symmetry and was accepted. 327680 ctx
+over **2 slots** of 163840 — inside the checkpoint's trained 262144 window, so
+no rope extrapolation (unlike the predecessor's 393216) — batch 6144x512, f16
+KV, a 16384 MiB host-RAM prompt cache, vision via the BF16 mmproj. DSpark was
+tried in the drafter's place on 2026-09-16 and reverted: it declares no sliding
 window, so its draft KV spanned the whole context (5.0 GiB) and cost ROCm1 four
 layers of the split.
 
@@ -706,7 +705,7 @@ Active:
 
 | Service | Runbook |
 |---------|---------|
-| frank-large (halogen-flash-server, Dankclaw) | `serverAppConfigs-docs/LLM/halogen-flash-server.md` |
+| frank-large (CIRU v4.4.1 Orca, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-ciru-orca.md` |
 | frank-small (llama.cpp Gemma4-12B QAT, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-gemma4-12b.md` |
 | frank-medium (llama.cpp + MTP, Dankserver) | `serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
@@ -718,6 +717,7 @@ Retired (rollback targets and history):
 |---------|------|--------|---------|
 | kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
 | ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-large`) | Dankclaw | 2026-09-15 → 09-16 | `frank-large-ik-uncensored.md` |
+| halogen-flash-server Qwen3.8-Flash-Next w4b (`frank-large`) | Dankclaw | 2026-09-16 → 09-19 | `halogen-flash-server.md` |
 | ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`frank-small`) | Dankclaw | 2026-09-16 → 09-17 | `frank-small-qwen38-gsq-rco.md` |
 | vLLM Gemma4-12B w4a16 (`frank-small`) | Dankclaw | 2026-09-19 (same-day revert) | `frank-small-vllm-gemma4.md` |
 | agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |

serverAppConfigs-docs/LLM/frank-large-ciru-orca.md

@@ -0,0 +1,219 @@
+---
+type: reference
+title: CIRU v4.4.1 Orca on Dankclaw (frank-large)
+description: The active frank-large runbook — CIRU v4.4.1 (QSA window hotfix) serving jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca on the Dankclaw Strix Halo iGPU: the private host build, the runtime gate, the payload, the deployed config, MTP and PLE, the client contract, and the failure modes worth knowing.
+tags:
+  - llm
+  - ciru
+  - dankclaw
+  - frank-large
+  - strix-halo
+  - gfx1151
+  - qwen4exp
+  - rocm
+  - mtp
+timestamp: 2026-09-19
+---
+
+# CIRU v4.4.1 Orca on Dankclaw (frank-large)
+
+> **ACTIVE since 2026-09-19.** It replaced halogen-flash-server (see
+> [[halogen-flash-server]], the current rollback), which had replaced
+> ik_llama.cpp (see [[frank-large-ik-uncensored]]). Deployed as a **root
+> systemd unit** (`scripts/frank-large.service`, `User=dank`), port 7770, on the
+> same gfx1151 iGPU the halogen quadlet used. The RTX 3090 it does not touch
+> belongs to [[frank-small-gemma4-12b]].
+
+CIRU is a llama.cpp fork tuned for the Qwen3.8-Flash-Next ("qwen4exp")
+architecture on AMD gfx1151 (Strix Halo). v4.4.1 carries a **QSA window
+hotfix** that is what makes vision and the MTP drafter work on this build; the
+pre-applied hotfix source ships in the model repo under
+`runtime/v4.4.1/qsa-window-hotfix/`. Like every other fleet endpoint it is a
+llama.cpp runtime — `llama-bench` is its own harness.
+
+## Shape
+
+One root unit, one process, one published port:
+
+| half | port | reachable |
+|---|---|---|
+| llama-server (OpenAI front-end + engine, single process) | 7770 | published as host **7770** |
+
+Unlike the halogen predecessor (two ports: an unauthenticated engine on 8730
+and an OpenAI front-end on 8731), CIRU is a single `llama-server` process —
+there is no separate engine port to keep private.
+
+`/v1/models` reports the loaded model name, **`Qwen3.8-Flash-CIRU-STRIX-Orca`**
+(the launcher sets no `--alias`). The endpoint serves **whatever model name a
+request names** — llama.cpp does not reject an unknown id — so the harness's
+`frank-large` label in `models.yml` works without a rename.
+
+## Runtime
+
+A **private host build**, not a container image. The v4.4.1 source with the QSA
+window hotfix, built on the host against the private ROCm SDK that
+`scripts/ciru/setup-linux-amd.sh` installs into the checkout's own venv:
+
+| path | what |
+|---|---|
+| `/home/dank/GitPackages/ciru-runtime-v4.4.1-qsa-hotfix/` | the checkout (source + build + venv) |
+| `build-gfx1151-sdk/bin/llama-server` | the built binary |
+| `.venv-rocm/` | the private ROCm SDK venv (~5.7 GB); the binary resolves its libs through RUNPATH |
+| `scripts/ciru/run-orca-server.sh` | the launcher (`exec`'d by the serve script) |
+| `profiles/strix-halo-production.env` | the 36-variable kernel bundle + PLE pager defaults, sourced by the launcher AFTER `frank-large.env` (every line is `${VAR:-default}`, so a value set in `frank-large.env` wins) |
+
+### The runtime gate
+
+`run-orca-server.sh` sources `run-server.sh`, which **exits 2** unless
+`${repo_root}/runtime/hip/lib/libamdhip64.so` and `runtime/rocr/lib/
+libhsa-runtime64.so` exist (unversioned, no suffix). It prepends both to
+`LD_LIBRARY_PATH` and `ldd`-checks that the built `libllama-common.so` carries
+the "Qwen4Exp MTP image position" marker and the HIP lib carries the
+`flash_attn_index_mask_clear/set/empty` symbols before exec.
+
+The build links the venv through RUNPATH, so those two files are an existence
+gate, not a real dependency. `serve-frank-large.sh` satisfies the gate
+idempotently: it finds the venv's unversioned devel symlinks
+(`.venv-rocm/lib/python*/site-packages/_rocm_sdk_devel/lib/`) and `ln -sfn`-s
+both into `runtime/{hip,rocr}/lib/`. **The first deploy of this endpoint failed
+exactly here** — the build linked the venv but `runtime/` did not exist, so the
+launcher exited 2 and the unit crash-looped until the serve script created the
+symlinks.
+
+## Payload
+
+`scripts/download-frank-large-orca.sh` assembles `jcbtc/Qwen3.8-Flash-CIRU-
+STRIX-Orca` (HuggingFace, public and ungated) into
+`/home/dank/models/Qwen3.8-Flash-CIRU-STRIX-Orca/`, sha256-verified. The
+launcher refuses to start without the main GGUF, the MTP head and the PLE
+sidecar:
+
+| file | bytes | role |
+|---|---|---|
+| `Qwen3.8-Flash-CIRU-STRIX-Orca.gguf` | 79,397,818,912 | the main model (Orca variant) |
+| `mtp/Qwen3.8-Flash-CIRU-STRIX-Orca-MTP-Q8_0.gguf` | 4,135,893,440 | the MTP drafter head |
+| `ple/ple.payload.bin` | 52,429,053,952 | the per-layer-embedding (PLE) payload — mmap'd and paged, not resident |
+| `ple/ple.manifest.json` | — | PLE manifest |
+| `ple/ple.scale.bf16` | — | PLE scale |
+| `vision/mmproj-Qwen3.8-Flash-Orca-F16.mmproj` | 904,004,320 | the F16 vision projector (on disk, `ENABLE_VISION=0`) |
+
+The main GGUF ships as **transport parts** that the download script assembles
+and verifies; the PLE is the large one (52.4 GB) and is paged by the engine's
+own workers, not held in RAM.
+
+## Deployed config
+
+Tunables live in `scripts/frank-large.env`; the launcher owns the flag list
+(`-sm none` one iGPU no split, `-ngl all`, f16 KV, flash attention,
+cont-batching, the PLE sidecar, the MTP drafter, the production kernel-bundle
+profile). The departures worth naming:
+
+| flag | value | why |
+|---|---|---|
+| `CONTEXT_SIZE` | `262144` | the model's FULL trained window (the operator's "full context" call); the GGUF carries no rope scaling |
+| `PARALLEL_SLOTS` | `1` | one stream — and the MTP drafter **requires exactly one slot** in this runtime, so the operator's ask and the drafter's constraint are the same number |
+| `ENABLE_MTP` | `1` | the MTP drafter, depth 4 (the Orca variant's own `mtp_depth_default`) |
+| `ENABLE_VISION` | `0` | text-only for now; the F16 projector is on disk, one flip to enable. The fleet's image path is [[frank-small-gemma4-12b]] either way |
+| `PLE_CACHE_MIB` | `4096` | the PLE pager's hot window kept resident; the vendor production-profile default. The 52.4 GB payload is mmap'd and paged around it |
+| `HOST` | `0.0.0.0` | the launcher consumes it for `--host`; the value doubles as a JIT-cache directory name, which is why `0.0.0.0` is safe (HIP reads `HOST` bare — see the AGENTS.md note) |
+
+`TEMP`, `HOST` and `GPU_MAX_HEAP_SIZE` are the three names to avoid for any
+new tunable: HIP reads them bare.
+
+## MTP
+
+The Multi-Token Prediction drafter is a **separate Q8_0 head** (not inside the
+main GGUF), loaded by the launcher. It requires exactly one slot, which is why
+`PARALLEL_SLOTS=1`. Depth 4 is the Orca variant's default.
+
+Verified at deploy (2026-09-19): the per-task `draft acceptance = 0.65625
+(21/32), mean len = 3.62` — the drafter is engaged and accepting, not silently
+standing down.
+
+## PLE
+
+The per-layer-embedding payload (52.4 GB) is the Qwen3.8-Flash-Next
+architecture's signature: token embeddings live outside the main weight set and
+are paged on demand by the engine's own workers (`GGML_QWEN4EXP_PLE_WORKERS=16`,
+`GGML_QWEN4EXP_PLE_IO_WORKERS=32` in the production profile). `PLE_CACHE_MIB=4096`
+is the hot window kept resident; the rest is mmap'd file cache. **Do not treat
+the 52.4 GB as resident memory** — `free` and the dashboards will not show it
+pinned, and that is correct.
+
+## Client contract
+
+The differences from a stock llama.cpp endpoint that bite:
+
+- **`reasoning_content`** carries the reasoning trace (the standard OpenAI
+  field). The serve script passes `--chat-template-kwargs
+  '{"preserve_thinking":true}'` so prior thinking blocks are kept across turns;
+  a request-level `chat_template_kwargs` still wins.
+- **`reasoning_effort` is gated.** Qwen3.8's template accepts only
+  `xhigh`, `medium`, `low` and **500s on anything else** (not a clamp). The
+  harness folds every level onto that set in `models.yml`
+  (`reasoningEffortMap`: minimal/low→low, medium→medium, high/max→xhigh) —
+  subagents run at `high`, which is why the map exists.
+- **Text only.** `ENABLE_VISION=0`, so image content parts are refused. The
+  fleet's image path is [[frank-small-gemma4-12b]].
+- **Sampling.** The server has no sampler defaults of its own; the harness
+  applies the per-model recipe from `models.yml` (temp 1.0 / top_p 0.95 / top_k
+  20 / min_p 0.0). A client that stays silent gets the launcher's default.
+- **`tools` works** — the launcher passes `--jinja`, and the Qwen3.8 template
+  supplies the tool-call grammar. Verified at deploy: a tool call returns
+  `tool_calls`.
+
+## Deploy and operate
+
+```bash
+./scripts/deploy.sh dankclaw --service frank-large
+```
+
+The handler pulls the checkout (ff-only), (re)links the unit, and gates on
+`/health`. First boot is the slow one (model load ~46 s + graph capture); a
+warm restart is fast.
+
+```bash
+sudo systemctl status frank-large            # the root unit
+sudo journalctl -u frank-large -n 60         # startup account: runtime check, model load, "listening on http://0.0.0.0:7770", MTP draft loaded
+curl -s http://127.0.0.1:7770/health         # ok
+curl -s http://127.0.0.1:7770/v1/models      # Qwen3.8-Flash-CIRU-STRIX-Orca
+```
+
+**Stop and mask before any GPU experiment** (the AGENTS.md bench protocol):
+`sudo systemctl stop frank-large && sudo systemctl mask frank-large`. The
+frank-large serve process is a `llama-server` **and so is frank-small on the
+same host** — never `pkill -x llama-server`. If you must kill by pattern, use
+`pkill -f 'Qwen3.8-Flash-CIRU-STRIX-Orca'` (the model path is unique to this
+endpoint and does not appear in the deploying shell's own command line).
+
+## Troubleshooting
+
+| symptom | cause |
+|---|---|
+| launcher exits 2: "The v4.4 HIP/ROCr runtime is missing" | `runtime/{hip,rocr}/lib/*.so` absent — the serve script's `ln -sfn` did not run, or the venv was moved. Re-run the serve script or rebuild: `bash $RUNTIME_DIR/scripts/ciru/setup-linux-amd.sh --install-host-deps` |
+| "missing SDK venv runtime under .../.venv-rocm" | the venv's unversioned devel symlinks are gone — rebuild the runtime |
+| "missing target model / missing payload" | a download did not finish — re-run `scripts/download-frank-large-orca.sh` |
+| loads, `/health` ok, first completion slow (~45-60 s) | the PLE pager pulling the hot window in from file cache — not a crash, do not restart over it |
+| a subagent 500s naming `reasoning_effort` | the Qwen3.8 template gate — the harness `reasoningEffortMap` must fold the level onto {xhigh, medium, low} |
+| `free` / dashboards do not show 52 GB pinned | correct — the PLE is mmap'd file cache, not resident; only `PLE_CACHE_MIB` is hot |
+
+Retrieval: this is a **root** unit, so `journalctl -u frank-large` is the
+durable record (no `--user`, no `sudo` needed for the journal once the unit is
+readable). There is no container log — there is no container.
+
+## Rollback
+
+The predecessor is intact: the halogen payload stays at
+`/home/dank/models/halogen` (retired, not deleted), and the halogen quadlet is
+archived under `.AGENTTRASH/`. Rolling back is a `git revert` of this change set
+plus a deploy — **but the root unit and the halogen quadlet both claim port
+7770**, so the handler's port-free gate covers that race. See
+[[halogen-flash-server]].
+
+The `amd_iommu=off` kernel argument is separate and reverts on its own — see
+[[dankclaw-iommu-translation-off]].
+
+## Citations
+
+- [jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca](https://huggingface.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca) — weights, MTP head, PLE payload, and the v4.4.1 QSA window-hotfix source tarball
+- [CIRU runtime v4.4.1](https://github.com/ciru) — `scripts/ciru/run-server.sh` (runtime gate), `profiles/strix-halo-production.env` (kernel bundle), `docs/BUILD_LINUX.md`

serverAppConfigs-docs/LLM/frank-medium-llamacpp-dflash2.md

@@ -1,7 +1,7 @@
 ---
 type: configuration
-title: frank-medium llama.cpp Runbook (MTP drafter)
-description: The ACTIVE frank-medium deployment since 2026-09-16 — mainline llama.cpp on Dankserver serving ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP at UD-Q8_K_XL with the model's own MTP head (the z-lab DFlash2 sidecar kept on disk, one env change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a 35:30 split (skewed toward the x16-slot card; see the 2026-09-17 part 3 section), a 393216-position KV pool (196608 per conversation over two slots), vision via the BF16 mmproj.
+title: frank-medium llama.cpp Runbook (DFlash2 drafter)
+description: The ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
 tags:
   - frank-medium
   - dankserver
@@ -12,16 +12,19 @@ tags:
   - gfx1201
   - r9700
   - layer-split
-timestamp: 2026-09-17
+timestamp: 2026-09-19
 ---
 
 # frank-medium — llama.cpp (DFlash2 drafter)
 
-The ACTIVE frank-medium deployment since 2026-09-16: **mainline llama.cpp**
-serving `ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF` at `UD-Q8_K_XL`
-with the **z-lab DFlash2** block-diffusion drafter, as **one process across both
-Radeon AI PRO R9700 cards**, layer-split. DSpark was tried in the drafter's
-place for a few hours the same day and reverted — see *The DSpark excursion*.
+The ACTIVE frank-medium deployment since 2026-09-16 (model swapped to
+orcarouter Q8_0 on 2026-09-19): **mainline llama.cpp**
+serving `orcarouter/Qwen3.8-27B-Uncensored-GGUF` at `Q8_0`
+with the **z-lab DFlash2** `Q4_K_M` block-diffusion drafter, as **one process across both
+Radeon AI PRO R9700 cards**, layer-split at a balanced 32:33. The predecessor
+model (ajgazin Swift-Qwen3.8-27B UD-Q8_K_XL, 35:30 skew, 393216 pool) is the
+calibration history in the sizing tables below. DSpark was tried in the drafter's
+place on 2026-09-16 and reverted — see *The DSpark excursion*.
 It replaced the vllm-radiance MXFP4 replica pair + routing proxy
 ([[frank-medium-radiance-mxfp4]], the record and the PCIe-atomics evidence) in
 the same day the pair went live. It is also the box's **only** LLM runtime: the

serverAppConfigs-docs/LLM/index.md

@@ -1,13 +1,13 @@
 # LLM Fleet — Knowledge Base
 
-Documentation for the LLM inference fleet on the two hosts (2026-09-16):
+Documentation for the LLM inference fleet on the two hosts (2026-09-19):
 **mainline llama.cpp on Dankserver** serving frank-medium as one layer-split
-process across both Radeon AI PRO R9700 cards (ajgazin Swift-Qwen3.8-27B-
-Uncensored-Dynamic-MTP UD-Q8_K_XL + the z-lab DFlash2 drafter), and TWO
-endpoints on Dankclaw — halogen-flash-server (frank-large, Qwen3.8-Flash-Next
-w4b on the gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small,
-ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S with MTP and vision on the RTX 3090, port
-7771).
+process across both Radeon AI PRO R9700 cards (orcarouter Qwen3.8-27B-
+Uncensored Q8_0 + the z-lab DFlash2 Q4_K_M drafter, balanced 32:33), and TWO
+endpoints on Dankclaw — the CIRU v4.4.1 runtime (frank-large, jcbtc
+Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151 iGPU, port 7770) and mainline
+llama.cpp (frank-small, HauhauCS Gemma4-12B QAT Q4_K_M with MTP and vision on
+the RTX 3090, port 7771).
 
 > 2026-08-22: the former `vllmScripts` repository is merged into the
 > serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -23,22 +23,22 @@ graph LR
 gfx1201, 32 GiB each]
         B2[llama.cpp, layer split
 frank-medium, port 7770]
-        C[Swift-Qwen3.8-27B-Uncensored UD-Q8_K_XL
-ONE process on both cards, 262144 ctx / 1 slot, DFlash2]
+        C[orcarouter Qwen3.8-27B Q8_0 + DFlash2 Q4_K_M
+ONE process on both cards, 2x163840 ctx, balanced 32:33]
         A --> B2 --> C
     end
 
     subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
         E2[RTX 3090 (PCIe x4 dock) +
 gfx1151 iGPU 64 GiB unified memory]
-        E[halogen-flash-server 0.5.8
+        E[CIRU v4.4.1 runtime
 frank-large, port 7770]
-        F[peonist-ai Qwen3.8-Flash-Next w4b
-2 x 200192 ctx over one 400384-position pool]
+        F[jcbtc Qwen3.8-Flash-CIRU-STRIX-Orca w4b
+262144 ctx / 1 slot, MTP depth 4 + PLE]
         G[mainline llama.cpp 37b53fd
 frank-small, port 7771]
-        H[Qwen3.8-27B GSQ-RCO IQ3_S
-131072 ctx / 1 slot, MTP + vision]
+        H[Gemma4-12B QAT Q4_K_M
+4 x 262144 ctx, MTP + vision + audio]
         E2 --> E --> F
         E2 --> G --> H
     end
@@ -57,11 +57,12 @@ frank-small, port 7771]
 ## Active Deployment
 
 - [[frank-medium-llamacpp-dflash2]] — the ACTIVE frank-medium runbook since
-  2026-09-16: mainline llama.cpp serving
-  ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF at UD-Q8_K_XL with the
-  z-lab DFlash2 Q8_0 drafter on Dankserver's 2x Radeon AI PRO R9700, as ONE
-  layer-split process behind :7770 (tensor parallelism is impossible on this
-  host — the second card cannot dispatch an RCCL collective)
+  2026-09-16 (model swapped 2026-09-19): mainline llama.cpp serving
+  orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M
+  drafter on Dankserver's 2x Radeon AI PRO R9700, as ONE layer-split process
+  behind :7770 (tensor parallelism is impossible on this host — the second card
+  cannot dispatch an RCCL collective), balanced 32:33 over two 163,840-token
+  slots
 - [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-16, one day after it was
   deployed: the vllm-radiance MXFP4 stack (stilldeadcode/vllm-radiance:0.9.3 +
   ggz14/radiance-vllm-mxfp4 patches) serving
@@ -81,12 +82,17 @@ frank-small, port 7771]
   (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
-- [[halogen-flash-server]] — ACTIVE frank-large runbook again since 2026-09-16
-  (rootless Quadlet, port 7770): peonist-ai halogen-flash-server 0.5.8 serving
-  peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn + quality
-  overlay) on the gfx1151 iGPU — two 200k conversations over one
+- [[frank-large-ciru-orca]] — ACTIVE frank-large runbook since 2026-09-19
+  (root systemd unit, port 7770): the CIRU v4.4.1 runtime (private host build,
+  QSA hotfix) serving jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151
+  iGPU — one 262,144-token conversation, MTP depth 4, 4096 MiB PLE cache,
+  runtime-root gate, client contract, troubleshooting
+- [[halogen-flash-server]] — RETIRED 2026-09-19 (replaced by
+  [[frank-large-ciru-orca]]; rollback target): peonist-ai halogen-flash-server
+  0.5.8 serving peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b
+  .hgn + quality overlay) on the gfx1151 iGPU — two 200k conversations over one
   400,384-position pool, deployed flags, memory model, client contract,
-  troubleshooting
+  troubleshooting (2026-09-16 → 09-19)
 - [[frank-small-gemma4-12b]] — ACTIVE frank-small runbook since 2026-09-17
   (root systemd unit, port 7771): mainline llama.cpp serving
   HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced Q4_K_M on the RTX
@@ -140,8 +146,8 @@ frank-small, port 7771]
 - [[kairic-edge]] — the retired Kairic Edge IU4 experiment
   (deployed and reverted 2026-08-22)
 - [[Dankserver System Specs]] — the 2x R9700 box (frank-medium)
-- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on
-  halogen-flash-server since 2026-09-16, frank-small on the RTX 3090)
+- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on the
+  CIRU v4.4.1 runtime since 2026-09-19, frank-small on the RTX 3090)
 - [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
   on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
   the retirement record — the IOMMU/SVA err=-19 root cause and where

serverAppConfigs-docs/index.md

@@ -116,12 +116,12 @@ The LLM inference fleet on the two hosts. Full index:
 - [[deployment]] — unified deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
 - [[frank-medium-llamacpp-dflash2]] — the ACTIVE frank-medium runbook since
-  2026-09-16: mainline llama.cpp (gfx1201 HIP build) serving
-  ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF at UD-Q8_K_XL with
-  the DFlash2 drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF at Q8_0) on
-  Dankserver's 2x Radeon AI PRO R9700 as one layer-split process behind the
-  :7770 endpoint — no tensor parallelism, because the second card cannot
-  dispatch an RCCL collective
+  2026-09-16 (model swapped 2026-09-19): mainline llama.cpp (gfx1201 HIP build)
+  serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the DFlash2
+  drafter (z-lab/Qwen3.8-27B-DFlash2-GGUF at Q4_K_M) on Dankserver's 2x Radeon
+  AI PRO R9700 as one layer-split process behind the :7770 endpoint, balanced
+  32:33 over two 163,840-token slots — no tensor parallelism, because the
+  second card cannot dispatch an RCCL collective
 - [[frank-medium-radiance-mxfp4]] — RETIRED 2026-09-16: the vllm-radiance
   MXFP4 stack serving just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq on
   Dankserver's 2x Radeon AI PRO R9700 as two single-GPU replicas behind a
@@ -145,11 +145,17 @@ The LLM inference fleet on the two hosts. Full index:
   agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
   Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
   rollback record)
-- [[halogen-flash-server]] — the ACTIVE frank-large runbook again since
-  2026-09-16 (rootless Quadlet, port 7770): peonist-ai/halogen-flash-server
-  0.5.8 serving peonist-ai/halogen-qwen3.8-flash-next (w4b `.hgn` + quality
-  overlay) on the gfx1151 iGPU — two 200k conversations over one
-  400,384-position pool, deployed flags, memory model, client contract
+- [[frank-large-ciru-orca]] — the ACTIVE frank-large runbook since 2026-09-19
+  (root systemd unit, port 7770): the CIRU v4.4.1 runtime (private host build,
+  QSA hotfix) serving jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca w4b on the gfx1151
+  iGPU — one 262,144-token conversation, MTP depth 4, 4096 MiB PLE cache,
+  runtime-root gate, client contract
+- [[halogen-flash-server]] — RETIRED 2026-09-19 (replaced by
+  [[frank-large-ciru-orca]]; rollback target, 2026-09-16 → 09-19):
+  peonist-ai/halogen-flash-server 0.5.8 serving peonist-ai/halogen-qwen3.8-
+  flash-next (w4b `.hgn` + quality overlay) on the gfx1151 iGPU — two 200k
+  conversations over one 400,384-position pool, deployed flags, memory model,
+  client contract
 - [[frank-large-ik-uncensored]] — the RETIRED 2026-09-16 frank-large runbook
   and rollback target (2026-09-15 → 09-16): ik_llama.cpp serving
   mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090 +