docs: the vault and AGENTS.md describe two Dankclaw endpoints

halogen-flash-server is ACTIVE again with its 2026-09-16 measured ledger (budget
33.5 GiB against a need of 27.2 — no clamp — and 14.7 GiB left for everything
else), ik_llama.cpp is retired with its measurements kept as the record of why,
and the new frank-small runbook covers the -mtp file, the 64 KiB/token KV
arithmetic, the MTP contract, and the reasoning_effort trap that cost me every
subagent on the endpoint.

AGENTS.md: topology, Active Services, runbook index, and three new llama.cpp
lessons (MTP is a property of the file; --jinja is what `tools` needs; a model's
template can reject the harness's thinking level). Three pre-existing defects
found while editing it: a truncated sentence mid-file, a harness path that does
not exist on either server, and the benchmarking stop/mask recipe still calling
frank-large a root unit.

Truthfulness notes worth keeping: the co-tenant stall NOTE prints again, so the
claim that it "no longer prints" is gone and the three observed block counts are
recorded instead of a guessed threshold; and the Observability page says out
loud why frank-large has no scrape job rather than leaving a silent absence.

AGENTS.md

@@ -12,8 +12,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
   Calibre-Web, Jellyfin, Plex) plus the `frank-medium` LLM service.
 - **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
   via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
-  Alloy, Pi-hole) plus the `frank-large` LLM service. Caddy on Dankclaw
+  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (halogen-flash-server on
-  reverse-proxies both hosts.
+  the gfx1151 iGPU) and `frank-small` (llama.cpp on the RTX 3090). Caddy on
+  Dankclaw reverse-proxies both hosts.
 - **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
   (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) as quadlets inside
   its podman machine; all egress rides the Gluetun WireGuard netns. Deployed
@@ -24,9 +25,10 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
 | Dankserver | `frank-medium` | vllm-radiance 0.9.3 (ggz14/radiance-vllm-mxfp4 patches), 2x single-GPU rootless Quadlets behind an adaptive-routing proxy | just1moremodel Qwen3.8-27B-Uncensored-MXFP4-awq + Qwen3.8-27B-DFlash2-FP8 drafter, 2x Radeon AI PRO R9700 (one card per replica, TP=1), 114688 ctx / 4 seqs each | 7770 |
-| Dankclaw | `frank-large` | ik_llama.cpp master d85a1d7 (root systemd unit) | mradermacher Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M GGUF, RTX 3090 (dense/attn/KV) + LPDDR (experts/PLE) hybrid, 262144 ctx / 1 slot, vision via mmproj | 7770 |
+| Dankclaw | `frank-large` | halogen-flash-server 0.5.8 (rootless Quadlet) | peonist-ai Qwen3.8-Flash-Next w4b `.hgn` on the gfx1151 iGPU (Strix Halo LPDDR), 200192 ctx per conversation over one 400384-position KV pool / 2 slots, text only | 7770 |
+| Dankclaw | `frank-small` | mainline llama.cpp master 37b53fd (root systemd unit) | ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S (`-mtp` + BF16 mmproj) on the RTX 3090, 131072 ctx / 1 slot, MTP speculative decoding, the fleet's vision endpoint | 7771 |
 
-Retired endpoints (`frank-npu`, `frank-small`, and every prior runtime) are in
+Retired endpoints (`frank-npu`, and every prior runtime) are in
 the [runbook index](#runbook-index). The full swap history lives in
 `serverAppConfigs-docs/LLM/index.md` — do not grow it here.
 
@@ -66,9 +68,10 @@ All edits flow through git. Never edit files on a remote machine directly.
 The planner maps changed paths to services; the remote runner restarts only
 the selected service and records its revision outside the checkout.
 Documentation-only changes are a no-op. LLM services are selected only by
-their own unit/env/serve files or the shared chat template — unrelated
+their own files — frank-large by `dankclaw/frank-large/quadlet/`, frank-small
-container changes cannot interrupt inference. Set `FRANK_SUDO_PW` for headless
+by its unit/env/serve trio in `scripts/` — so unrelated container changes
-runs on a host without passwordless sudo; the value never enters the repo.
+cannot interrupt inference. Set `FRANK_SUDO_PW` for headless runs on a host
+without passwordless sudo; the value never enters the repo.
 
 ### Deployment completeness (MANDATORY)
 
@@ -194,8 +197,9 @@ archive.
 **STOP the production service BEFORE benchmarking** — these servers serve
 real users, and a bench beside a live service starves the GPU and corrupts
 results. Use the runtime's own harness: `llama-bench` (llama.cpp), `vllm
-bench` (vLLM), llama-bench for ik_llama.cpp (the halogen `bench`/`sweep` modes
+bench` (vLLM), and halogen-flash-server's own `bench`/`sweep` modes for
-belong to a retired runtime). Rolling your own bench requires asking first.
+frank-large, which is the active runtime there again. Rolling your own bench
+requires asking first.
 Scratch ports are not used on LLM hosts — a
 test process cannot share the single GPU anyway; validate against the service
 itself.
@@ -212,8 +216,13 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
 merely wedged kernel usually still answers ARP.
 
 1. Stop and mask (self-healing would restart it):
-   - Dankclaw `frank-large` (root unit since the 2026-09-15 ik cutover):
+   - Dankclaw `frank-large` (rootless Quadlet again since 2026-09-16):
-     `sudo systemctl stop frank-large && sudo systemctl mask frank-large`
+     `systemctl --user stop frank-large`, then move its
+     `~/.config/containers/systemd/frank-large.container` aside and
+     `systemctl --user daemon-reload` — the quadlet generator re-creates that
+     unit from the file on every reload, so the source is the lever, not `mask`.
+   - Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
+     `sudo systemctl stop frank-small && sudo systemctl mask frank-small`
    - Dankserver `frank-medium` (system unit):
      `sudo systemctl stop frank-medium && sudo systemctl mask frank-medium`
 2. Bench.
@@ -225,47 +234,71 @@ deletes the unit file from `/etc/systemd/system/`, not just the symlink.
 
 ## Model Config (harness)
 
-The harness reaches the fleet through `/home/dank/.omp/agent/models.yml`
+The harness reaches the fleet through `.omp/agent/models.yml` in the
-(outside the repo). Each provider is one host endpoint; each model `id` is
+operator's home — `/home/blank/.omp/agent/models.yml` on the dev box, which is
-the served alias: `frank-medium` → `192.168.0.235:7770`, `frank-large` →
+where the harness runs; there is no copy on the servers. Each provider is one
-`192.168.0.143:7770`. Per-role routing lives in
+HOST:PORT; each model `id` is the served alias: `frank-medium` →
-`/home/dank/.omp/agent/config.yml` under `modelRoles`.
+`192.168.0.235:7770`, `frank-large` → `192.168.0.143:7770`, `frank-small` →
+`192.168.0.143:7771`. Per-role routing lives beside it in `config.yml` under
+`modelRoles`.
 
 Resolution rules: the default model resolves as a `defaultProvider` +
 `defaultModel` pair (the provider must serve the model); a subagent override
-uses a bare model `id`, which resolves only when unique across providers.
+uses a bare model `id`, which resolves only when unique across providers. Two
+ports on one host need two providers — the file cannot express "same host,
+different port".
 
 When renaming a service, retiring an endpoint, or moving a model: update the
 provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
 `models.yml`, then fix every `modelRoles` entry that named the old endpoint —
-a role pointing at a removed provider fails at call time. `frank-large`
+a role pointing at a provider the file does not define fails at call time. As
-declares `input: [text, image]` (its mmproj vision tower). There is **no audio
+of 2026-09-16 the image path is `frank-small` (`input: [text, image]`), and
+`frank-large` is text-only because its halogen deployment loads no vision
+tower. There is **no audio** input anywhere in the fleet: that capability left
+with the FLM NPU endpoint on 2026-09-11.
 
 ## Active Services
 
-### frank-large — ik_llama.cpp (Dankclaw)
+### frank-large — halogen-flash-server (Dankclaw, ACTIVE)
 
-ik_llama.cpp master `d85a1d7` as a **root systemd unit** (repo
+halogen-flash-server 0.5.8 as a **rootless Quadlet again since 2026-09-16**
-`scripts/frank-large.service` symlinked into `/etc/systemd/system/`;
+(`dankclaw/frank-large/quadlet/frank-large.container`), serving
-`Restart=always`), serving `mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`
+`peonist-ai/halogen-qwen3.8-flash-next` (115.55 GiB w4b `.hgn` checkpoint + the
-(111 GiB GGUF + mmproj-f16) on port 7770 with 262144 ctx / 1 slot, greedy
+2.31 GiB quality overlay) on port 7770 from the gfx1151 iGPU. Two conversations
-default, reasoning on (deepseek), vision via mmproj. Topology: the RTX 3090
+of 200,192 tokens resident (pool 400,384, 2 slots), greedy default, reasoning
-(PCIe x4 in the M.2 dock) carries dense/attention/KV; ALL MoE expert tensors
+extracted to `reasoning_content`, and **no vision tower** — that moved to
-and the ~33 GiB PLE table stay in LPDDR via CPU buffers. Deployed flags in
+frank-small. It replaced ik_llama.cpp, which had replaced it for two days.
-`scripts/frank-large.env` / `scripts/serve-frank-large.sh`; the serve script
-refuses to start without `GGML_CUDA_NO_PINNED_WEIGHTS=1` (the pinned-host
-buffer incident starved co-tenants into unreachability until a power cycle).
 
-Measured envelope (2026-09-15/16 campaign): prefill ~192 tok/s (1.8k-32k
+Measured boot ledger (2026-09-16, `HALOGEN_VERBOSE=1` ships in the unit):
-prompts, CPU VNNI compute ceiling), decode ~3.8-4.3 tok/s — gated by ~100
+device budget `121 − 67.7 − 20 = 33.5 GiB` against a need of `~27.2 GiB`, so
-CPU↔GPU backend splits per token (GPU 95% idle, launch/sync-bound; thread
+the configured pool was **not clamped**; 68.0 GiB of weights locked + 11.0 GiB
-count irrelevant). The halogen-era 30 tok/s decode bar is out of reach on
+of KV pool + 16.5 GiB of working memory = 95.4 GiB in all, leaving 14.7 GiB
-this model+hardware; the runbook carries the full evidence chain, the
+(475 contiguous 2 MiB blocks) for everything else: frank-small on the 3090 and
-mainline-llama.cpp comparison (1.0 tok/s — ruled out), and the tuning levers.
+~28 co-tenant containers. The engine printed its co-tenant stall NOTE and
+logged 4,350 compaction stalls (285 failed) while reserving the pool, so read
+that as "tight and fragmented", not "out of memory".
 
-Runbook: `serverAppConfigs-docs/LLM/frank-large-ik-uncensored.md`.
+Runbook: `serverAppConfigs-docs/LLM/halogen-flash-server.md`.
-Rollback: [[halogen-flash-server]] (quadlet archived in
+Rollback: [[frank-large-ik-uncensored]].
-`~/.AGENTTRASH/20260915-ik-swap/` on dankclaw).
+
+### frank-small — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
+
+Mainline llama.cpp master `37b53fd` as a **root systemd unit** (repo
+`scripts/frank-small.service`, `relink_unit`ed into `/etc/systemd/system/`),
+serving `ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF` at IQ3_S — the `-mtp` build, so
+the model's own Multi-Token Prediction head drives speculative decoding — with
+the BF16 mmproj, on port 7771, 131,072 ctx / 1 slot, f16 KV, greedy default.
+Deployed flags are in `scripts/frank-small.env`; the serve script refuses to
+start if the weights, the projector or the binary are missing.
+
+Verified at deploy (2026-09-16): `/v1/models` reports `frank-small`; a chat
+completion returns content plus `reasoning_content`; a tool call returns
+`tool_calls`; the slot timing lines show `draft acceptance = 0.66–0.94, mean
+len ≈ 3` (MTP engaged); an image request is answered from the projector while
+the halogen side refuses images with a named reason; 22,362 MiB of the card's
+24,576 MiB is in use; `n_slots = 1, n_ctx_slot = 131072`.
+
+Runbook: `serverAppConfigs-docs/LLM/frank-small-qwen38-gsq-rco.md`.
 
 ### frank-medium — vllm-radiance MXFP4 (Dankserver, ACTIVE)
 
@@ -400,6 +433,33 @@ linked runbooks.
 - **FastMTP sidecar**: build must be commit `4df29be4` + the HauhauCS patch;
   stock binaries fail the draft load with a tensor-shape mismatch.
 
+**llama.cpp serving**
+
+- **MTP is a property of the FILE, not of the flags.** The `-mtp` GGUF build
+  carries the head as `blk.<n>.nextn.*` (`qwen35.nextn_predict_layers = 1`);
+  `--spec-type draft-mtp` then sets `load_mtp`, and the draft context runs on
+  the MAIN model's weights (`shares_model`), so there is no second copy of a
+  12 GiB model in VRAM and no `-md` argument. The plain build has no MTP
+  tensors at all, and pointing `--model-draft` at the file would load it twice.
+- **`--jinja` is what `tools` needs.** On 2026-09-16 a subagent died with
+  `500 tools param requires --jinja flag`, from the ik_llama.cpp runtime that
+  then served frank-large: every tool-using agent on that endpoint was broken
+  and the failure read like a model fault. Mainline llama.cpp 0.4.1 defaults
+  `--jinja` on and the serve script pins it; halogen also accepts `tools` and
+  returns `tool_calls`. Check tool support before routing a role at an endpoint.
+- **Quantized KV on CUDA needs `GGML_CUDA_FA_ALL_QUANTS=ON`** — the same
+  option and the same ~12x prefill collapse described in the ROCm note above.
+  f16 KV needs nothing, which is why frank-small runs f16.
+- **A model's own template can REJECT the harness's thinking level.** Qwen3.8's
+  template accepts only `xhigh`, `medium` and `low` for `reasoning_effort` and
+  raises on anything else — HTTP 500, not a clamp — so the harness's levels
+  must be folded onto that set in `models.yml` (`compat.reasoningEffortMap`).
+  A role whose level is unmapped fails on its FIRST request, and the failure
+  names Jinja, not the model: every subagent on `frank-small` died with
+  `Unexpected reasoning effort high` until `high: xhigh` was added on
+  2026-09-16. Halogen accepts the wider set, so the two entries legitimately
+  differ.
+
 **Ops**
 
 - Never `pkill -f` a pattern that appears in your own command line — it kills
@@ -430,7 +490,8 @@ Active:
 
 | Service | Runbook |
 |---------|---------|
-| frank-large (ik_llama.cpp, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-ik-uncensored.md` |
+| frank-large (halogen-flash-server, Dankclaw) | `serverAppConfigs-docs/LLM/halogen-flash-server.md` |
+| frank-small (llama.cpp Qwen3.8-27B GSQ-RCO, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-qwen38-gsq-rco.md` |
 | frank-medium (vllm-radiance MXFP4, Dankserver) | `serverAppConfigs-docs/LLM/frank-medium-radiance-mxfp4.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow + incidents | `serverAppConfigs-docs/LLM/deployment.md` |
@@ -440,7 +501,7 @@ Retired (rollback targets and history):
 | Runtime | Host | Window | Runbook |
 |---------|------|--------|---------|
 | kingjones STRIX Qwen3.8-Flash-Next ROCmFP4 | Dankclaw | 2026-08-30 → 09-11 | `kingjones-strix-rocmfp4.md` |
-| halogen-flash-server (`frank-large`) | Dankclaw | → 2026-09-15 | `halogen-flash-server.md` |
+| ik_llama.cpp Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M (`frank-large`) | Dankclaw | 2026-09-15 → 09-16 | `frank-large-ik-uncensored.md` |
 | agentionai ROCmFP4-FAST imatrix | Dankclaw | 2026-08-29 → 08-30 | `agentionai-fast-imatrix.md` |
 | eemin Qwen3.8-27B ROCmI4 W4A4 MTP | Dankclaw | 2026-08-18 → 08-25 | `eemin-rocmi4-mtp.md` |
 | Kairic Edge IU4 | Dankclaw | 2026-08-22 (same-day revert) | `kairic-edge.md` |

serverAppConfigs-docs/Infrastructure/Observability.md

@@ -86,24 +86,34 @@ The rule is the same as for every other service: the only off-host path is a gua
 | `comfyui` | `comfyui:8188` | UI probe, no native exporter |
 | `services` | `192.168.0.143:8081`, `:3005` | SearXNG, OpenWebUI up/down probes |
 | `frank-medium` | `192.168.0.235:7770` | vLLM on Dankserver — 418 `vllm:` series |
-| `frank-large` | `192.168.0.143:7770` | llama.cpp on Dankclaw — `llamacpp:` counters |
+| `frank-small` | `192.168.0.143:7771` | llama.cpp on Dankclaw's RTX 3090 — `llamacpp:` counters, including the spec-decode pair |
 
-The two LLM endpoints were added on 2026-09-16, after a night spent diagnosing
+The LLM endpoints were added on 2026-09-16, after a night spent diagnosing
-frank-medium by hand. vLLM exports exactly what that diagnosis needed:
+frank-medium by hand, and re-cut the same day when Dankclaw split into two
-`vllm:kv_cache_usage_perc` (how full the per-card pool is), 
+endpoints. vLLM exports exactly what that diagnosis needed:
+`vllm:kv_cache_usage_perc` (how full the per-card pool is),
 `vllm:num_requests_waiting` (queue depth), the request-latency histograms, and
 the prefix-cache counters `vllm:prefix_cache_hits_total` /
 `vllm:prefix_cache_queries_total`, whose rates give the reuse rate that says
 whether a pinned conversation is being served warm or re-prefilled cold. What
-those numbers mean on this hardware is in [[frank-medium-radiance-mxfp4]].
+those numbers mean on this hardware is in [[frank-medium-radiance-mxfp4]];
+llama.cpp's counters (prompt and decode throughput, KV fill, and the
+`spec_decode_*` pair that proves MTP is running) are read against
+[[frank-small-qwen38-gsq-rco]].
 
-Both are scraped on their LAN addresses, never on the guarded vhosts:
+**`frank-large` is deliberately NOT scraped.** The halogen engine publishes
-Prometheus cannot solve an Anubis challenge.
+`/health`, `/cache` and `/v1/*` and no Prometheus text endpoint at all, so a
+target there would be red by construction and would fire `LlmEndpointDown`
+forever — the same class of defect as the page-only probes below. Its liveness
+is on the homepage (a loopback probe) and in the engine's own boot ledger.
+
+Both scraped endpoints are reached on their LAN addresses, never through the
+guarded vhosts: Prometheus cannot solve an Anubis challenge.
 
 For services without a native exporter, use the Prometheus **blackbox exporter** in probe mode to assert up/down and HTTP latency. The homepage status timer probes the same endpoints from the host.
 
-**Four jobs read "down" and three of them always will** (found 2026-09-16 while
+**Four targets read "down" — three jobs, and every one of them is structural**
-auditing why the target list was not green):
+(found 2026-09-16 while auditing why the target list was not green):
 
 | job | error | what it means |
 |-----|-------|---------------|

serverAppConfigs-docs/LLM/deployment.md

@@ -58,25 +58,30 @@ on the selected host. The following files select the corresponding service:
 - Documentation, tests, deployment tooling, and unrelated container changes
   do not select either inference service.
 
-The target-side `scripts/deploy-remote.sh` relinks the selected unit into
+The target-side `scripts/deploy-remote.sh` installs the selected service: for a
-`/etc/systemd/system/`, reloads systemd, enables it, restarts it, and waits for
+**root unit** it relinks the repo file into `/etc/systemd/system/`, reloads
-that service's health endpoint (`http://127.0.0.1:7770/health` for the two
+systemd, enables it and restarts it (frank-small); for a **Quadlet** it retires
-inference services; `/v1/version` for the NPU sight+sound endpoint — FLM has no
+the predecessor, stops the service's own unit, waits for the published port to
-`/health` route). The wait budget is 20 minutes because cold model loads can be
+clear, installs the `.container` and restarts the generated unit (frank-large).
-slow. A failed handler does not advance its per-service state.
+Each handler then waits on that service's health endpoint —
+`http://127.0.0.1:7770/health` for frank-large, `http://127.0.0.1:7771/health`
+for frank-small. The wait budget is 20 minutes because cold model loads are
+slow: halogen pins ~68 GiB of weights before it serves anything. A failed
+handler does not advance its per-service state.
 
 ## Unit and artifact paths
 
-Both units point into the stable target checkout:
+The units point into the stable target checkout:
-
 ```text
 /home/dank/serverAppConfigs/scripts/frank-medium.service
-/home/dank/serverAppConfigs/scripts/frank-large.service
+/home/dank/serverAppConfigs/scripts/frank-small.service
 ```
 
 The unit files and serve scripts validate their model/runtime artifacts before
-starting. Keep `REMOTE_DIR` stable; moving the checkout requires updating both
+starting. `frank-large` is a Quadlet under `dankclaw/frank-large/quadlet/`
-the unit absolute paths and the deployment tests in the same change.
+rather than a unit in `scripts/`; `frank-small` keeps the unit/env/serve trio
+above. Keep `REMOTE_DIR` stable; moving the checkout requires updating both the
+unit absolute paths and the deployment tests in the same change.
 
 ## NPU unit
 
@@ -105,23 +110,26 @@ change in that service's inputs.
 
 If the LLM handler fails after the target pull:
 
-1. Inspect `sudo journalctl -u frank-medium -n 200` or
+1. Inspect `journalctl -u frank-small -n 200` (root unit) or
-   `journalctl -u frank-large -n 200`.
+   `journalctl --user -u frank-large -n 200` (rootless Quadlet).
-2. Confirm the model files, custom runtime binary, and chat template are
+2. Confirm the model files, the runtime binary and (for frank-large) the
-   readable by `dank`.
+   container image are present and readable by `dank`.
 3. Retry only the affected service:
 
    ```bash
    ./scripts/deploy.sh dankserver --service frank-medium
-   # or
    ./scripts/deploy.sh dankclaw --service frank-large
+   ./scripts/deploy.sh dankclaw --service frank-small
    ```
-
 4. Verify the matching health endpoint before declaring recovery.
 
-Do not stop the other host's inference service during recovery. For benchmark
+Do not stop the other host's inference service during recovery. The two
-work, follow the separate stop/benchmark/restart procedure in [[Testing Policy]]
+Dankclaw endpoints are independent processes on different devices: restarting
-and the repository rules.
+frank-large does not disturb frank-small, and the reverse holds too — with one
+exception worth knowing, that the deployer waits for port 7770 to be released
+before it starts the replacement container. For benchmark work, follow the
+separate stop/benchmark/restart procedure in [[Testing Policy]] and the
+repository rules.
 
 ## See Also
 

serverAppConfigs-docs/LLM/frank-large-ik-uncensored.md

@@ -7,10 +7,15 @@ timestamp: 2026-09-16T03:30:00-07:00
 
 # frank-large — ik_llama.cpp, Qwen3.8-Flash-Next-Uncensored i1-Q4_K_M
 
-**ACTIVE since 2026-09-15.** Replaced [[halogen-flash-server]] (rollback target;
+**RETIRED 2026-09-16, after one day.** Replaced by [[halogen-flash-server]],
-quadlet unit archived in `~/.AGENTTRASH/20260915-ik-swap/halogen-quadlet` on
+which was the runtime before it as well — the fleet went ik → halogen → back.
-dankclaw). Serves `mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`
+Its unit, env, serve script and the Qwen-Sharp template are archived in the repo
-(111 GiB GGUF + mmproj-f16) at `192.168.0.143:7770`.
+under `scripts/.AGENTTRASH/*.ik-llama.cpp` and `*.ik-llama-qwen-sharp`, and the
+payload, build and chat template stay on disk. The measurements below are not a
+stale plan: they are the complete account of what this model costs on this
+hardware, and they are why the fleet went back. It served
+`mradermacher/Qwen3.8-Flash-Next-Uncensored` (111 GiB GGUF + mmproj-f16) at
+`192.168.0.143:7770` while it ran.
 
 ## Shape
 

serverAppConfigs-docs/LLM/frank-small-qwen38-gsq-rco.md

@@ -0,0 +1,204 @@
+---
+type: reference
+title: frank-small on Dankclaw — llama.cpp, Qwen3.8-27B GSQ-RCO IQ3_S
+description: The active frank-small runbook — mainline llama.cpp on Dankclaw's RTX 3090 serving ISTA-DASLab Qwen3.8-27B-GSQ-RCO at IQ3_S with the model's own MTP head and the BF16 vision projector: identity, KV arithmetic, deployed flags, the MTP contract, the client contract, and what to check when it misbehaves.
+tags:
+  - llm
+  - frank-small
+  - dankclaw
+  - llama.cpp
+  - rtx3090
+  - qwen35
+  - mtp
+  - vision
+timestamp: 2026-09-16
+---
+
+# frank-small — llama.cpp on the RTX 3090
+
+> **ACTIVE since 2026-09-16.** Created when Dankclaw's inference was re-split:
+> [[halogen-flash-server]] took the gfx1151 iGPU back for `frank-large`, which
+> freed the RTX 3090 for this endpoint. Two endpoints, one host: `frank-large`
+> on `:7770`, `frank-small` on `:7771`.
+
+## Identity
+
+| what | value |
+|---|---|
+| runtime | mainline llama.cpp master `37b53fd` (2026-09-16), CUDA build for sm_86 |
+| binary | `/home/dank/GitPackages/llama.cpp-mainline/build-cuda/bin/llama-server` |
+| unit | **root systemd unit** `frank-small.service` (repo `scripts/frank-small.service`, `relink_unit`ed into `/etc/systemd/system/`) |
+| flags | `scripts/frank-small.env`; entrypoint `scripts/serve-frank-small.sh` |
+| endpoint | `192.168.0.143:7771`, alias `frank-small` |
+| model | `ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF`, the `IQ3_S-mtp` build + BF16 mmproj |
+
+It is a root unit rather than a Quadlet because the runtime is a native CUDA
+binary, not a container: there is no image to install and no generated unit to
+avoid. Boot persistence is the unit's `[Install] WantedBy=multi-user.target`;
+`Restart=on-failure` / `RestartSec=10` make it self-healing; `LimitMEMLOCK=infinity`
+covers CUDA's pinned staging buffers.
+
+Build recipe (the deployed binary):
+
+```bash
+cmake -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON \
+  -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86
+cmake --build build-cuda --target llama-server -j
+```
+
+`GGML_CUDA_FA_ALL_QUANTS` is deliberately **off**: it is only needed for
+*quantized* KV with flash attention (see the CUDA lesson in AGENTS.md — without
+it, quantized-KV prefill collapses ~12x), and this service runs f16 KV.
+
+## The model, and why the `-mtp` build specifically
+
+GSQ-RCO is a **per-tensor mixed-precision** allocation produced by a
+gradient-based search under a total size budget, rather than one quantization
+type applied throughout. IQ3_S at 3.50 bpw is the release's task-lossless
+operating point: AIME25 100.00 and LiveCodeBench v6 85.71 match BF16 exactly,
+GPQA-Diamond is 0.51 behind — at 11.8 GB against 53.8 GB.
+
+Two files are required, and the second suffix matters:
+
+| file | size | why |
+|---|---|---|
+| `Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp.gguf` | 12,120,016,960 B | the `-mtp` build **is** the MTP build: it carries the head as `blk.64.nextn.*` (`qwen35.nextn_predict_layers = 1`) |
+| `mmproj-Qwen3.8-27B-BF16.gguf` | 931,146,528 B | the vision encoder + projector (`clip.projector_type = qwen3vl_merger`, 27 blocks) |
+
+Both live in `/home/dank/models/Qwen3.8-27B-GSQ-RCO-IQ3_S-mtp/`, fetched with
+`huggingface_hub` (which verifies the LFS sha256 and resumes).
+
+**MTP is a property of the file, not of the flags.** `--spec-type draft-mtp`
+sets `load_mtp`, and the draft context runs on the *main model's* weights
+(`common/common.cpp` passes `shares_model` when no `--model-draft` is given;
+`common/speculative.cpp` gives it `LLAMA_CONTEXT_TYPE_MTP`). So there is no
+second copy of a 12 GiB model in VRAM, and `--model-draft` would be wrong here —
+it would load the whole file again.
+
+## Architecture and the KV arithmetic
+
+GGUF arch `qwen35`: 64 layers laid out as `16 x (3 x (Gated DeltaNet -> FFN) +
+1 x (Gated Attention -> FFN))`. Only the 16 full-attention layers take KV, so a
+token costs
+
+```
+2 (K,V) x 4 heads x 256 head_dim x 2 B = 64 KiB at f16   (34 KiB q8_0, 18 KiB q4_0)
+```
+
+and the 48 DeltaNet layers add ~0.16 GiB per sequence of constant-size state in
+f32, at any depth. The MTP draft context is `n_ctx` long over one nextn layer,
+so it costs 4 KiB/token of its own.
+
+## Sizing (RTX 3090, 24576 MiB)
+
+| term | GiB |
+|---|---|
+| weights | 11.29 |
+| mmproj | 0.87 |
+| MTP draft KV (131,072 x 4 KiB) | 0.50 |
+| DeltaNet state (one sequence) | 0.16 |
+| CUDA context + prefill/decode buffers | ~1.5 |
+| **KV, one slot at 131,072 x 64 KiB** | **8.00** |
+| **modelled total** | **~22.3 of 24** |
+
+The binding number is the KV: about 9.5 GiB is left after the fixed terms, which
+is ~155k tokens at 64 KiB/token. Alternatives inside the same budget: `2 x
+65536` at f16, or `4 x 65536` / `1 x 262144` at q8_0 — the quantized variants
+need a rebuild with `-DGGML_CUDA_FA_ALL_QUANTS=ON`. **`--parallel N` divides
+`--ctx-size` between the slots**; it does not multiply.
+
+Measured at deploy: **22,362 MiB** in use of 24,576, `n_slots = 1`,
+`n_ctx_slot = 131072`.
+
+## Deployed flags
+
+| flag | value | why |
+|---|---|---|
+| `CTX_SIZE` / `PARALLEL` | `131072` / `1` | one conversation per slot, full f16 precision, no KV quantization risk |
+| `CACHE_TYPE_K/V` | `f16` | quantizing would need the FA_ALL_QUANTS build for no benefit here |
+| `NGL` | `999` | every layer on the card |
+| `FLASH_ATTN` | `on` | |
+| `SPEC_TYPE` | `draft-mtp` | the model's own head; draft depth stays llama.cpp's default of 3 |
+| `BATCH_SIZE` / `UBATCH_SIZE` | `2048` / `512` | the defaults; ubatch 512 keeps the DeltaNet chunk buffers small |
+| `NUM_THREADS` / `THREAD_BATCH` | `8` / `8` | the work is on the GPU; the host CPU belongs to halogen's engine |
+| `TEMPERATURE` | `0.0` | greedy default (fleet contract) — clients that send their own values get them |
+| `REASONING_FORMAT` / `REASONING_BUDGET` | `deepseek` / `-1` | thinking into `reasoning_content`, unlimited budget (the model decides) |
+| `--jinja` | pinned | the GGUF's own template supplies `reasoning_effort`, `preserve_thinking` and the tool-call grammar. This build defaults it on; pinning it means a future default change cannot silently take tools away |
+| `--metrics` | on | Prometheus scrapes `/metrics` here (`frank-small` job) |
+
+## Client contract
+
+- **One slot.** A second concurrent request queues; it is not served in
+  parallel, and agent fan-out through this endpoint serializes. Widen with
+  `PARALLEL` (which divides the context) if that becomes the wrong trade.
+- **Thinking counts against the budget.** A request whose budget runs out
+  mid-thought returns `finish_reason: "length"` with an empty `content` and the
+  partial reasoning in `reasoning_content` — observed live on a vision request
+  at `max_tokens: 64`. Give reasoning models a real budget.
+- **Tools work** (`tool_calls` returned; verified with a function-call round
+  trip) because `--jinja` serves the model's own template. This is also what
+  fixed subagents fleet-wide: the ik runtime that served `frank-large` until
+  2026-09-16 rejected every request carrying `tools` with
+  `500 tools param requires --jinja flag`.
+- **Images are accepted** as OpenAI `image_url` content parts with a `data:`
+  URL or bare base64. This is the fleet's only image path: halogen's
+  `frank-large` runs with no tower and refuses images with a named reason.
+- **`reasoning_effort` accepts only `xhigh`, `medium`, `low`.** The template
+  raises on anything else — a 500, not a clamp — so the harness folds its own
+  levels onto that set in `compat.reasoningEffortMap` (`high: xhigh`,
+  `max: xhigh`). An unmapped level fails on the first request.
+- `/v1/models` reports `frank-small`; `/health` answers 200 once the model is
+  loaded and the projector is up.
+
+## Verified at deploy (2026-09-16)
+
+- `systemctl is-enabled/is-active frank-small.service` → `enabled` / `active`.
+- `/v1/models` → `frank-small`; a chat completion returns `content` **and**
+  `reasoning_content`.
+- A tool call returns `tool_calls` with the expected arguments.
+- Slot timing lines show `draft acceptance = 0.66-0.94, mean len = 2.95-3.83`
+  — MTP is engaged, not merely configured.
+- An image request returns the right colour answer from the projector.
+- Harness routing: role `smol` lands here, role `slow` on `frank-large`, and a
+  real subagent (the `task` role) completes a tool-using turn.
+
+## Operate
+
+```bash
+systemctl status frank-small.service          # root unit
+journalctl -u frank-small.service -n 60       # load + per-request timing
+curl -s http://127.0.0.1:7771/health
+curl -s http://127.0.0.1:7771/v1/models
+curl -s http://127.0.0.1:7771/metrics | grep -E 'spec_decode|kv_cache_usage'
+nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv
+```
+
+`/metrics` carries `llamacpp:spec_decode_num_{drafts,draft_tokens,accepted_tokens}_total`,
+which is the cheapest way to confirm MTP is running, and
+`llamacpp:kv_cache_usage_ratio` for how full the slot is.
+
+## Troubleshooting
+
+| symptom | cause |
+|---|---|
+| unit restarts immediately, `frank-small: missing <path>` in the journal | the model, the projector or the binary is absent — the serve script refuses to start rather than half-load; finish the download |
+| `500 … Jinja Exception: Unexpected reasoning effort <level>` | a client sent an effort the template does not accept; map it (see the client contract) |
+| `500 tools param requires --jinja flag` | impossible here (`--jinja` is pinned) — you are looking at a *different* endpoint, most likely the retired ik runtime's logs |
+| `finish_reason: "length"` with empty content | the budget ran out inside the thinking block; raise `max_tokens` |
+| a request that waits, then answers normally | the single slot was busy (or the KV is nearly full); raise `PARALLEL` or expect serialization |
+| CUDA OOM at load | another process holds the 3090, or the KV was widened past ~9.5 GiB; check `nvidia-smi` and re-derive from the sizing table |
+
+## Rollback
+
+`frank-small` is a new endpoint, so rollback is retirement:
+`systemctl disable --now frank-small.service`, move
+`/etc/systemd/system/frank-small.service` into `~/.AGENTTRASH/`, and remove the
+harness provider + the `smol`/`tiny`/`task`/`vision` role entries. The old
+`frank-small` (FreeToken on Dankserver, retired 2026-09-15) is a different
+service with a different card — see [[freetoken-frank-small]].
+
+## Citations
+
+- [ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF](https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF) — the quantized checkpoints, the `-mtp` builds and the projector
+- [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) — base model: 64 layers, Gated DeltaNet/Gated Attention layout, 262,144 native context
+- [ggml-org/llama.cpp](https://github.com/ggml-org/llama.cpp) — `--spec-type draft-mtp`, the MTP context type, and the Qwen3-VL projector path

serverAppConfigs-docs/LLM/halogen-flash-server.md

@@ -16,10 +16,14 @@ timestamp: 2026-09-13
 
 # halogen-flash-server on Dankclaw (frank-large)
 
-> **ACTIVE since 2026-09-11.** Replaced the llama.cpp / kingjones30-ROCmFPX
+> **ACTIVE again since 2026-09-16, and retuned.** It replaced the llama.cpp /
-> STRIX deployment (see [[kingjones-strix-rocmfp4]], now the rollback target).
+> kingjones30-ROCmFPX STRIX deployment on 2026-09-11 (see
-> Deployed as a rootless Quadlet container,
+> [[kingjones-strix-rocmfp4]]), was displaced for two days by ik_llama.cpp (see
-> `dankclaw/frank-large/quadlet/frank-large.container`, port 7770.
+> [[frank-large-ik-uncensored]], the current rollback), and came back as the
+> shape the fleet asked for: two 200k-token conversations rather than the
+> native-context pair. Deployed as a rootless Quadlet container,
+> `dankclaw/frank-large/quadlet/frank-large.container`, port 7770. The RTX 3090
+> it does not touch now belongs to [[frank-small-qwen38-gsq-rco]].
 
 The engine is a from-scratch serving runtime for the Qwen3.8-Flash-Next
 architecture on AMD's gfx1151 (Strix Halo) — the same silicon this host runs.
@@ -77,12 +81,12 @@ upstream's defaults:
 | flag | value | why |
 |---|---|---|
 | `HALOGEN_MODEL_ID` | `frank-large` | fleet identity; no client rename |
-| `HALOGEN_CTX` | `262144` | the model's native context, and what both clients declare |
+| `HALOGEN_CTX` | `200192` | **set 2026-09-16**: exactly half the pool, so two conversations of the full window are always resident. Not a round 200,000 — pool/2 is. The model's native window is 262,144, so this is a real reduction in the per-conversation ceiling, and a client asking for more gets a 400 (never a silent truncation). The harness declares 200192 for `frank-large` |
-| `HALOGEN_KV_POOL_POSITIONS` | `401408` | **trimmed 2026-09-13** from the 524,288 the doubling deployed: one full 262k conversation resident beside a ~139k one, or three at 131k, ~11.0 GiB of KV as reserved. Measured against the doubled pool the host went from 7.1 GiB / 310 contiguous 2 MiB blocks to 9.1 / 1396, and the co-tenant stall NOTE no longer prints. `HALOGEN_KV_POOL_FIT` still guards the boot, so read the ledger rather than trusting this column |
+| `HALOGEN_KV_POOL_POSITIONS` | `400384` | **retuned 2026-09-16** (from 401,408): 2 x 200,192. Reserves ~11.0 GiB of KV, confirmed by the boot ledger as `400384 positions, ~27.2 GiB (16.2 GiB of it for everything but the pool, at max_tok 24576)` against a 33.5 GiB device budget — **no clamp**. Slots still oversubscribe the pool for any real request (admission reserves prompt + `max_tokens`), so a near-full pair that is both generating is pool-bound. `HALOGEN_KV_POOL_FIT` guards the boot: read the ledger, not this column |
-| `HALOGEN_KV_SLOTS` | `3` | Concurrency cap — **lowered from 4 on 2026-09-12** to the real fleet size. Each slot is 111.0 MiB of O(1) state, so this freed ~111 MiB (held 103.5 → 103.4 GiB). Slots share the ONE pool and **oversubscribe** it (3 × 262144 > 401408), so admission is pool-bound: one full-length conversation plus a ~139k second, or three at 131k. This trims simultaneous requests, not how much context fits |
+| `HALOGEN_KV_SLOTS` | `2` | **lowered from 3 on 2026-09-16** — one slot per half of the pool, and two is the fleet's shape now that Dankclaw serves two endpoints. Each slot is 111.0 MiB of O(1) state, so this hands ~111 MiB back as well |
-| `HALOGEN_MAX_TOK` | `32768` | upstream's default, kept. The arena is 20.6 GiB here against 12.2 at 16384; it fit alongside the doubled pool only because the reserve dropped to 15, and it fits the trimmed 401,408 pool with room to spare. At 65536 the arena alone exceeds the budget and the server **will not boot** |
+| `HALOGEN_MAX_TOK` | `24576` | **halved from the 32768 default on 2026-09-16** for frank-small, its co-tenant on the same host. The arena's variable term goes `8.40 x 2 → 8.40 x 1.5`, handing ~4.2 GiB back at roughly a 4-5 % prefill cost — the same lever at 16,384 costs 8-9 %. The vendor's own notes call 24,576 the largest arena that allocated with headroom to spare, so it stays inside their tested range. 65536 still cannot boot. It sizes a per-call PREFILL CHUNK, not the prompt ceiling: a 200k conversation is prefilled in arena-sized pieces |
-| `HALOGEN_HOST_RESERVE_GIB` | `15` | **lowered from 24 (vendor default 20) on 2026-09-12** to buy the doubled pool. The only term you own in the device budget (24 → 29.5 GiB, 15 → 38.5 GiB). Against the doubled pool the ceiling was 16 — the engine needed budget ≥ 36.5, and 18 gave 35.5, silently re-clamping; at 401,408 the need is ~33.4 so the ceiling loosens, but a bigger reserve still buys nothing. Matched pair with the pool. It is **not** an allocation: raising it cannot return memory to the host, only take the pool away |
+| `HALOGEN_HOST_RESERVE_GIB` | `20` | **raised from 15 with the same retune**, because the arena shrank: `budget = 121.2 − 67.7 − 20 = 33.5 GiB` against a measured need of ~27.2, so the fit check kept ~6 GiB of margin and did not clamp. It is **not** an allocation: raising it cannot return memory to the host, only take the pool away (see the ceiling section below) |
-| `HALOGEN_VISION_TOWER` | `1` | image input; the fleet's only image path since [[frank-npu]] was retired. Charges 1.97 GiB against the budget — the term missing from my first estimate |
+| `HALOGEN_VISION_TOWER` | *unset* | **removed 2026-09-16.** The tower charges 1.97 GiB against the same device budget as the pool and the arena, and the fleet's image path moved to [[frank-small-qwen38-gsq-rco]] (llama.cpp with `--mmproj`). This server refuses image content parts, and `models.yml` declares `frank-large` text-only to match |
 | `HALOGEN_DOWNLOAD` | `peonist-ai/halogen-qwen3.8-flash-next` | first-start fetch |
 | `HALOGEN_VERBOSE` | `1` | verbose engine logging — the fullest startup account, on permanently so an incident report needs no restart-into-debug |
 
@@ -104,10 +108,13 @@ Three numbers explain the whole box:
   **overstate free memory by the size of the model**. The server prints the
   corrected figure at startup; believe that line, not Grafana.
 - **~28 KiB per KV position** (measured: 262,144 positions = 6,656 MiB of KV,
-  ~7.2 GiB with block scratch), plus the prefill arena (16.7 GiB at
+  ~7.2 GiB with block scratch), plus the prefill arena. The 2026-09-16 ledger
-  `HALOGEN_MAX_TOK=32768`, 8.4 GiB at 16384). The engine's device-side budget
+  reads the two together in the engine's own words: `400384 positions, ~27.2
-  stops at roughly 47 GiB on a 128 GB machine *with the weights pinned* — and
+  GiB (16.2 GiB of it for everything but the pool, at max_tok 24576)` — so
-  lower on a machine carrying more besides this server, which Dankclaw is.
+  ~11.0 GiB of pool and 16.2 GiB of arena, working slack and (no longer) a
+  vision tower. The engine's device-side budget stops at roughly 47 GiB on a
+  128 GB machine *with the weights pinned* — and lower on a machine carrying
+  more besides this server, which Dankclaw is.
 - **`HALOGEN_HOST_RESERVE_GIB`** is what is left for the page cache the lookup
   table is read through. Too little and long prompts crawl with the disk busy
   and the process in uninterruptible sleep — that is a cache problem, not a
@@ -128,6 +135,8 @@ startup account, including three deliberate experiments:
 | 524288 | 65536 | 24 | — | 37.3 | 46.5 | 29.5 | **OOM — will not boot** |
 | 524288 | 32768 | **15** | 14.4 | 20.6 | 37.0 | **38.5** | **deployed — both features** |
 | 524288 | 32768 | 16 | 14.4 | 20.6 | 37.0 | 37.5 | fits, footprint unchanged — see the ceiling below |
+| **400384** | **24576** | **20** | **11.0** | **16.4** | **27.2** | **33.5** | **deployed 2026-09-16 — ledger need 27.2, no clamp** |
+| 401408 | 32768 | 15 | 11.0 | 20.6 | 33.4 | 38.5 | 2026-09-13 → 09-16, superseded by the retune |
 
 ### The sizing rule
 
@@ -183,11 +192,14 @@ The device budget is not the whole story. Memory position by configuration
 | 524288 / 16384, reserve 24 | 94.6 | 13.8 GiB | 3742 |
 | 524288 / 32768, reserve **16** | **103.5** | **7.1 GiB** | 293 |
 | **401408 / 32768, reserve 15** | **100.0** | **9.1 GiB** | **1396** |
+| **400384 / 24576, reserve 20** | **95.4** | **14.7 GiB** | **475** |
 
 At the doubled setting (2026-09-12) the server held 103.5 of 121 GiB and the
-engine volunteered a note it had not printed before (it no longer prints at
+engine volunteered the stall note. It did not print it at the 401,408 trim, and
-the 401,408 trim — 9.1 GiB / 1396 blocks sits below whatever threshold
+it prints it again after the 2026-09-16 retune. Three data points, and they do
-triggers it):
+not line up with a "free memory below X" reading: 310 blocks printed, 1396 did
+not, 475 did. So treat the note as "the machine is tight" and read the
+`host memory left` line rather than waiting to be told:
 
 ```
 halogen: NOTE this server now holds most of the machine, and 310 contiguous
@@ -326,25 +338,33 @@ deployment:
 **plus** `max_tokens` positions when it is admitted; if the pool cannot hold it
 the request waits in arrival order, and after the queue timeout (3600 s) it
 gets a `503 engine_busy`. A prompt large enough that prompt + budget exceeds
-the pool can therefore never be admitted at all. With the pool at 401,408
+the pool can therefore never be admitted at all. The 2026-09-16 shape is a
-(since 2026-09-13) the per-conversation ceiling is still `HALOGEN_CTX`
+400,384-position pool with `HALOGEN_CTX` 200,192 and two slots: two
-(262,144) — even a full-context prompt plus the 65,536 cap is 327,680 ≤
+conversations at the ceiling fit the pool exactly, so a pair that is both
-401,408. The pool binds **how many** conversations stay resident: at 524,288
+near-full and both generating is pool-bound — the second waits for the first to
-two full-length ones fit; at 401,408 it is one full-length beside a ~139k
+release its region. The pool binds **how many** conversations stay resident,
-second, or three at 131k. Under the pre-09-12 262,144 pool the two numbers
+not how long one may be; the older 524,288 / 401,408 numbers live in the ledger
-were identical, which is why four concurrent 65k chats were contending for
+tables above.
-one conversation's worth of positions.
 
 ## Vision
 
-With the tower loaded, `/v1/chat/completions` accepts the usual OpenAI image
+**None here since 2026-09-16.** `HALOGEN_VISION_TOWER` is deliberately unset:
-content part carrying a `data:` URL or bare base64 — **`http(s)` URLs are
+the tower is a 1.97 GiB charge against the same device budget as the pool and
-refused deliberately**, because fetching one would make the server issue
+the arena, and the fleet's image path moved to [[frank-small-qwen38-gsq-rco]],
-outbound requests to wherever a client pointed it. One image costs about 2,040
+which loads the BF16 projector alongside its own weights. This server refuses
-tokens of context and 5.5 / 11.8 / 25.3 s at 1280x800 / 1920x1080 / 2560x1440;
+image content parts and says why, rather than ignoring them:
-text at 12 pt and above is read exactly, and a bigger frame past that point
+
-adds empty area rather than detail. Audio and video have no path here at all —
+```json
-that capability left with [[frank-npu]].
+{"error": {"message": "this server does not accept images: the engine was
+started without a vision tower (HALOGEN_VISION_TOWER)", "type":
+"invalid_request_error"}}
+```
+
+The llama.cpp side handles images the same way this engine used to: a `data:`
+URL or bare base64, about 2,040 tokens of context per image, and `http(s)` URLs
+refused deliberately (fetching one would make the server issue outbound
+requests to wherever a client pointed it). Audio and video have no path
+anywhere in the fleet — that left with [[frank-npu]].
 
 ## Deploy and operate
 
@@ -352,17 +372,27 @@ that capability left with [[frank-npu]].
 ./scripts/deploy.sh dankclaw --service frank-large
 ```
 
-The handler retires the llama.cpp-era **root** unit first (stops it, drops its
+The handler retires the ik-era **root** unit first (stop, disable, move
-enable symlinks, moves `/etc/systemd/system/frank-large.service` into the home
+`/etc/systemd/system/frank-large.service` into
-trash — never `rm`), then pulls `ghcr.io/peonist-ai/halogen-flash-server:0.5.8`,
+`~/.AGENTTRASH/20260916-halogen-restore/host-units` — never `rm`; this host's
-re-tags it to the `localhost/` alias the unit references, installs the Quadlet,
+`disable` may delete the file outright, which the handler now reports), stops
-and restarts the generated unit. Boot persistence comes from
+the Quadlet's own unit if it is running, then pulls
-`[Install] WantedBy=default.target` in the unit file plus linger on the deploy
+`ghcr.io/peonist-ai/halogen-flash-server:0.5.8`, re-tags it to the
-user — generated units **cannot** be `systemctl --user enable`d on this host.
+`localhost/` alias the unit references, and **waits for port 7770 to be free**
+before installing and starting anything.
 
-The health gate is bootstrap-aware: while the checkpoint is absent the
+That last gate is not paranoia. On the first cutover attempt the ik server
-container is downloading and `/health` cannot answer, so the handler waits for
+logged "Deactivated successfully", and pasta — rootless podman's userspace
-the file instead of burning its budget and reporting a failure that is not one.
+network stack, which binds the published port at container CREATE — still
+reported `Address already in use` a second later, exiting the unit with code
+126. Only `Restart=on-failure` recovered it, minutes later. A deploy that
+starts a doomed container has not deployed anything, so `wait_port_free` polls
+every socket on the port (listener, accepted, TIME_WAIT) and fails loudly with
+the socket table if it never clears.
+
+Boot persistence comes from `[Install] WantedBy=default.target` in the unit
+file plus linger on the deploy user — generated units **cannot** be
+`systemctl --user enable`d on this host.
 
 ```bash
 systemctl --user status frank-large.service     # the generated unit
@@ -404,15 +434,15 @@ in root's storage and answers "no such container".
 
 ## Rollback
 
-The llama.cpp era is intact: the STRIX payload stays at
+The predecessor is intact: the ik payload stays at
-`/home/dank/models/kingjones-Qwen3.8-Flash-Next-ROCmFP4-STRIX`, the
+`/home/dank/models/mradermacher-Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M`, the
-`kingjones30/ROCmFPX` build stays at
+ik build at `/home/dank/GitPackages/ik_llama.cpp/build`, and its unit, env,
-`/home/dank/GitPackages/kingjones-ROCmFPX/build`, and the retired unit, env and
+serve script and the Qwen-Sharp template are archived in the repo under
-serve script are in `serverAppConfigs/.AGENTTRASH/` (suffixed `.llama.cpp`).
+`scripts/.AGENTTRASH/*.ik-llama.cpp` and `*.ik-llama-qwen-sharp`. Rolling back
-Rolling back is a `git revert` of this change set plus a deploy — **but the
+is a `git revert` of this change set plus a deploy — **but the Quadlet must be
-Quadlet unit must be removed from `~/.config/containers/systemd/` first**
+removed from `~/.config/containers/systemd/` first**, or the restarted root
-(otherwise the restarted root unit and the still-installed container both
+unit and the still-installed container both claim port 7770. The port gate in
-claim port 7770). See [[kingjones-strix-rocmfp4]].
+the handler now covers exactly that race. See [[frank-large-ik-uncensored]].
 
 The `amd_iommu=off` kernel argument that came with this change is separate and
 reverts on its own — see [[dankclaw-iommu-translation-off]].

serverAppConfigs-docs/LLM/index.md

@@ -1,10 +1,11 @@
 # LLM Fleet — Knowledge Base
 
-Documentation for the LLM inference fleet on the two hosts
+Documentation for the LLM inference fleet on the two hosts (2026-09-16):
-(2026-09-16): charlie12345/ROCmFPX llama.cpp on Dankserver (frank-medium,
+vllm-radiance on Dankserver (frank-medium, just1moremodel
-kingjones777 Qwen3.8-27B-ROCmFPX-Q8_0-AGENT on 2x Radeon AI PRO R9700) and
+Qwen3.8-27B-Uncensored-MXFP4-awq on 2x Radeon AI PRO R9700) and TWO endpoints on
-ik_llama.cpp on Dankclaw (frank-large, mradermacher Qwen3.8-Flash-Next-
+Dankclaw — halogen-flash-server (frank-large, Qwen3.8-Flash-Next w4b on the
-Uncensored i1-Q4_K_M on the RTX 3090 + Strix Halo hybrid).
+gfx1151 iGPU, port 7770) and mainline llama.cpp (frank-small, ISTA-DASLab
+Qwen3.8-27B-GSQ-RCO IQ3_S with MTP and vision on the RTX 3090, port 7771).
 
 > 2026-08-22: the former `vllmScripts` repository is merged into the
 > serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -18,26 +19,31 @@ graph LR
     subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
         A[2x Radeon AI PRO R9700
 gfx1201, 32 GiB each]
-        B2[ROCmFPX llama.cpp
+        B2[vllm-radiance MXFP4
 frank-medium, port 7770]
-        C[Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
+        C[Qwen3.8-27B-Uncensored-MXFP4-awq
-layer split, 131k ctx / 1 slot, MTP]
+2 single-GPU replicas, 114688 ctx / 4 seqs, DFlash2]
         A --> B2 --> C
     end
 
     subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
         E2[RTX 3090 (PCIe x4 dock) +
 gfx1151 iGPU 64 GiB unified memory]
-        E[ik_llama.cpp master d85a1d7
+        E[halogen-flash-server 0.5.8
 frank-large, port 7770]
-        F[mradermacher Qwen3.8-Flash-Next-Uncensored
+        F[peonist-ai Qwen3.8-Flash-Next w4b
-i1-Q4_K_M GGUF, 262k ctx / 1 slot, vision]
+2 x 200192 ctx over one 400384-position pool]
+        G[mainline llama.cpp 37b53fd
+frank-small, port 7771]
+        H[Qwen3.8-27B GSQ-RCO IQ3_S
+131072 ctx / 1 slot, MTP + vision]
         E2 --> E --> F
+        E2 --> G --> H
     end
 
     CLI[LLM Clients / API] -->|HTTP :7770| B2
     CLI -->|HTTP :7770| E
-```
+    CLI -->|HTTP :7771| G
 
 ## Servers
 
@@ -59,23 +65,31 @@ i1-Q4_K_M GGUF, 262k ctx / 1 slot, vision]
   2026-09-15): charlie12345/ROCmFPX serving
   kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF; kept as the record of the
   runtime it replaced
-- [[freetoken-frank-small]] — RETIRED 2026-09-15 with the RTX 3090: the
+- [[freetoken-frank-small]] — RETIRED 2026-09-15, when the RTX 3090 moved to
-  FreeToken v0.1.2 frank-small runbook (batsclamp Huihui-Qwen3.6-35B-A3B-
+  frank-large: the FreeToken v0.1.2 frank-small runbook (batsclamp
-  abliterated-FP8), kept as the retirement record
+  Huihui-Qwen3.6-35B-A3B-abliterated-FP8), kept as the retirement record. The
+  NAME was reused on 2026-09-16 by a different service on Dankclaw — see
+  [[frank-small-qwen38-gsq-rco]]
 - [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep
   (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
-- [[frank-large-ik-uncensored]] — ACTIVE frank-large runbook since
+- [[halogen-flash-server]] — ACTIVE frank-large runbook again since 2026-09-16
-  2026-09-15: ik_llama.cpp (root systemd unit) serving mradermacher/
+  (rootless Quadlet, port 7770): peonist-ai halogen-flash-server 0.5.8 serving
-  Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090 (dense/attention/
+  peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn + quality
-  KV) + LPDDR (experts/PLE) hybrid; measured performance envelope, the
+  overlay) on the gfx1151 iGPU — two 200k conversations over one
-  decode root-cause chain, tuning levers, and the co-tenancy traps
+  400,384-position pool, deployed flags, memory model, client contract,
-- [[halogen-flash-server]] — RETIRED 2026-09-15 (rollback target): the
+  troubleshooting
-  peonist-ai halogen-flash-server 0.5.8 runbook serving
+- [[frank-small-qwen38-gsq-rco]] — ACTIVE frank-small runbook since 2026-09-16
-  peonist-ai/halogen-qwen3.8-flash-next (Qwen3.8-Flash-Next w4b .hgn +
+  (root systemd unit, port 7771): mainline llama.cpp serving
-  quality overlay), rootless Quadlet container on Dankclaw port 7770 —
+  ISTA-DASLab/Qwen3.8-27B-GSQ-RCO IQ3_S on the RTX 3090, with the model's own
-  deployed flags, memory model, client contract, troubleshooting
+  Multi-Token Prediction head for speculative decoding and the BF16 projector
+  for the fleet's image path
+- [[frank-large-ik-uncensored]] — RETIRED 2026-09-16 (rollback target): the
+  ik_llama.cpp runbook (root systemd unit, 2026-09-15 → 09-16) serving
+  mradermacher/Qwen3.8-Flash-Next-Uncensored-i1-Q4_K_M on the RTX 3090
+  (dense/attention/KV) + LPDDR (experts/PLE) hybrid; its measured envelope and
+  decode root-cause chain are the record of why the fleet went back to halogen
 - [[dankclaw-iommu-translation-off]] — why Dankclaw boots with
   `amd_iommu=off` since 2026-09-11 (13-16 % prefill, measured upstream),
   what it cost (the NPU endpoint), what was deliberately not copied from
@@ -100,8 +114,8 @@ i1-Q4_K_M GGUF, 262k ctx / 1 slot, vision]
 - [[kairic-edge]] — the retired Kairic Edge IU4 experiment
   (deployed and reverted 2026-08-22)
 - [[Dankserver System Specs]] — the 2x R9700 box (frank-medium)
-- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large,
+- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on
-  on halogen-flash-server since 2026-09-11)
+  halogen-flash-server since 2026-09-16, frank-small on the RTX 3090)
 - [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
   on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
   the retirement record — the IOMMU/SVA err=-19 root cause and where

serverAppConfigs-docs/log.md

@@ -541,3 +541,83 @@ timestamp: 2026-09-14
   Active Services section + bench-policy unit type. Committed 795b0b3;
   tests 300 ok with the single pre-existing not-ok-73 (comfyui selector
   bug, unrelated).
+
+## 2026-09-16 — Dankclaw re-split: halogen returns as frank-large, llama.cpp takes the 3090 as frank-small
+- `/dankclaw/frank-large/quadlet/frank-large.container` — halogen-flash-server
+  restored as the active runtime (the 09-15 ik swap never removed the file) and
+  retuned to the shape Frank asked for: `HALOGEN_CTX` 262144 → **200192**,
+  `HALOGEN_KV_POOL_POSITIONS` 401408 → **400384** (exactly 2 × 200192),
+  `HALOGEN_KV_SLOTS` 3 → **2**, `HALOGEN_MAX_TOK` 32768 → **24576** and
+  `HALOGEN_HOST_RESERVE_GIB` 15 → **20** (arena −4.2 GiB, for the co-tenant),
+  `HALOGEN_VISION_TOWER` **unset** (the tower moved to frank-small). Live ledger:
+  budget `121 − 67.7 − 20 = 33.5 GiB` against a need of `27.2`, so **no clamp**;
+  68.0 GiB weights + 11.0 KV + 16.5 working = 95.4 GiB held, 14.7 GiB left for
+  everything else (475 contiguous 2 MiB blocks). The co-tenant stall NOTE is
+  back, and the boot logged 4,350 compaction stalls (285 failed) — tight and
+  fragmented, not out of memory.
+- `/scripts/frank-small.env`, `frank-small.service`, `serve-frank-small.sh` (new)
+  — mainline llama.cpp `37b53fd` on the RTX 3090 serving
+  `ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF` at IQ3_S: the `-mtp` build (the head is
+  `blk.64.nextn.*` in the SAME file, so `--spec-type draft-mtp` runs the draft
+  context on the main model's weights — no second copy, no `-md`) plus the BF16
+  projector, 131072 ctx / 1 slot, f16 KV, `--jinja` and `--metrics` pinned, port
+  7771, root systemd unit. 22,362 MiB of the card's 24,576 in use. Verified:
+  `/health`, chat with `reasoning_content`, a `tool_calls` round trip, slot
+  timing `draft acceptance 0.66–0.94`, and an image answered from the projector.
+- `/scripts/deploy-remote.sh` — `deploy_llm_large` flipped back to halogen
+  (retire the ik ROOT unit into
+  `~/.AGENTTRASH/20260916-halogen-restore/host-units`, pull + re-tag the image,
+  **`wait_port_free 7770`**, install the Quadlet, start) and `deploy_llm_small`
+  added (root unit, model-file gate, health on 7771). The port gate exists
+  because the FIRST cutover attempt failed exactly there: pasta reported
+  `Address already in use` about a second after the ik server exited, the unit
+  died with code 126, and only `Restart=on-failure` recovered it. A redeploy now
+  stops this service's own quadlet before waiting, or the gate would deadlock
+  against itself.
+- `/scripts/deploy-plan.py`, `/scripts/deploy.sh`, `/scripts/test-changed.sh` —
+  frank-small added to `SERVICE_ORDER` and the Dankclaw inventory (the stale
+  `dankclaw:frank-npu` entry dropped with it); the changed-test mapping for
+  `dankclaw/frank-large/*` pointed at `tests/frank_large_quadlet.bats`, retired
+  on 09-15, so the wrapper had been failing on that path ever since — now
+  `tests/frank_large_unit.bats`.
+- Retired to `scripts/.AGENTTRASH/`: the ik trio (`frank-large.env`,
+  `frank-large.service`, `serve-frank-large.sh`), `chat-template.jinja` (only the
+  ik runtime read it), and four benchmark harnesses bound to retired runtimes.
+- `/tests/frank_large_unit.bats` rewritten for the Quadlet (the `pool == 2 × ctx`
+  invariant, no vision tower, the port gate) and `/tests/frank_small_unit.bats`
+  added (root-unit shape, MTP-from-the-file contract, KV budget vs the card, port
+  separation). One assertion failed on its first run because the negative grep
+  matched a COMMENT in the serve script — the test now strips comments before
+  grepping the effective flags.
+- `shared/homepage/services.json`, `shared/prometheus/prometheus.yml`,
+  `rules/llm-fleet.yml`, `grafana/.../llm-fleet.json` — a frank-small card at
+  `127.0.0.1:7771` and a `frank-small` scrape job at `192.168.0.143:7771`; the
+  `frank-large` job **removed**, because the halogen engine has no `/metrics` at
+  all (its routes are `/health`, `/cache` and `/v1/*`), so a target there would
+  have been red by construction; the dashboard's endpoint picker had been sourced
+  from a vLLM-only metric, so a llama.cpp job could never be selected. Live: 11
+  targets, 7 up, 4 down (comfyui, gitea, services — all structural), and all
+  three LLM homepage cards up.
+- Harness (`~/.omp/agent/models.yml` + `config.yml`, dev box) — third provider
+  `llamacpp (dankclaw-3090)` for `:7771`; `frank-large` ctx 262144 → 200192 and
+  `input: [text]`; roles `smol`, `tiny`, `task`, `vision` → frank-small (the
+  image path and the small/fast roles), `slow`/`plan`/`designer`/`advisor` →
+  frank-large. Two failures found by driving it, both fixed: (1) frank-small's
+  template accepts only `xhigh|medium|low` for `reasoning_effort` and 500s on
+  anything else, and subagents run at `high` — every subagent on that endpoint
+  died until `compat.reasoningEffortMap` folded the levels onto that set; (2) the
+  ik runtime that had been serving `frank-large` rejected every request carrying
+  `tools` (`500 tools param requires --jinja flag`), which is what killed a
+  monitoring subagent earlier in the day. Both endpoints now answer tool calls,
+  and a subagent completes a tool-using turn.
+- Docs: `frank-small-qwen38-gsq-rco.md` (new runbook),
+  `halogen-flash-server.md` re-activated with the 09-16 ledger,
+  `frank-large-ik-uncensored.md` marked retired, LLM index, deployment flow,
+  Observability and AGENTS.md topology/sections/lessons. Three AGENTS.md defects
+  found on the way: a truncated sentence mid-file ("There is **no audio"), a
+  harness path that does not exist (`/home/dank/.omp/agent/models.yml` — the
+  harness runs on the dev box), and the benchmarking stop/mask recipe still
+  describing frank-large as a root unit.
+- Deploy: `./scripts/deploy.sh dankclaw --service frank-large --service
+  frank-small` — per-service health gates, then the e2e gate PASSED (31 passed,
+  1 flaky-retried). Suite: 323 seams ok, 0 failures.