docs: frank-large halogen runbook, AGENTS.md + indices for the swap

New runbook frank-large-halogen27b-dflash2.md; LLM + vault indices re-pointed, ROCmFPX q6agent runbook marked RETIRED (rollback target); AGENTS.md topology row, detail section, image-capability paragraph (two endpoints accept images now), bench-law prose and runbook index updated.
2026-10-05 16:29:11 -07:00
parent fece3ffdd0
commit 0900551317
5 changed files with 219 additions and 102 deletions
@@ -12,8 +12,8 @@ merge absorbed the former `vllmScripts` repo, history preserved).
  Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service.
 - **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
  via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
-  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (ROCmFPX llama.cpp
-  fork on the gfx1151 iGPU) and `frank-small` (llama.cpp CUDA on the RTX 3090). Caddy on
+  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (halogen container on
+  the gfx1151 iGPU) and `frank-small` (llama.cpp CUDA on the RTX 3090). Caddy on
  Dankclaw reverse-proxies both hosts.
 - **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
  (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) as quadlets inside
@@ -25,7 +25,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
 | Dankserver | `frank-primary` | tcclaviger RDNA4 vLLM fork (`docker.io/tcclaviger/vllm@sha256:da4029666…`, digest-pinned, plain `podman run`, root systemd unit) | `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (~172.8 GiB, 131 shards), TP=2, expert offload automatic (fork plans it), prefetch OFF, **fp8 KV** (`--kv-cache-dtype fp8`), **no speculative decode**, MNBT 2048 (arena-free: the registry prices overhead flat up to its nbt 4096 anchor), MoE gather 32×64, **`--expert-precise-memory`** (fitted 5.25 GiB overhead, not the 7.60 GiB default), **`VLLM_CUDAGRAPH_MEM_BASE_MB=0` / `_PER_GRAPH_MB=12`** (the plan's CUDA-graph term priced at the measured capture — 0.31 GiB then a NEGATIVE −0.06 GiB delta, so the 256 MiB analytic base was fiction → graph term 0.07 GiB), **`GPU_MAX_HW_QUEUES=2`**, **mounted cache dir** at `/root/.cache` (the image ships no `/root/.cache` at all; unmounted, every boot re-pays `torch.compile took 181.20 s`), **3 slots × 200704 ctx** at KV_BYTES 3,166,500,000/rank (~400,700 KV tokens at the engine's 7,891.8 B/token here) — the **pool, not the slot count, caps resident conversations** (measured live: `GPU KV cache size: 401,408 tokens, Maximum concurrency for 200,704 tokens per request: 2.00×`; the 3.15e9 pool below it measured 398,658/1.99×), and shorter requests use the spare blocks. **`--max-num-seqs` must stay 3: SEQS=2 cannot boot this model** — torch.fx `ConstraintViolationError` on `query_start_loc`, measured at BOTH MNBT 1024 and 2048. **util stays 0.982 BY LAW**: at `le=1` the worker init refuses `budget == total` with `ValueError: Free memory on device cuda:N (31.29/31.86 GiB)` on both ranks before a single weight loads — measured crash loop 2026-10-03. Arena at shape G: **17.77 GiB / 7,762 slots measured**, KV 401,408 tokens / 2.00× concurrency, 0.89 GiB of real free VRAM per card held through a live 4,096-token generation (the ~3.5 GiB of driver-reported free VRAM at shape E was estimator slack — the registry's 9190 B/token KV reserve vs the real 7,892, and the 5.25 GiB overhead vs far less — NOT allocatable), and the client contract wired: `--reasoning-parser …
-| Dankclaw | `frank-large` | **ROCmFPX llama.cpp fork** (`kingjones30/ROCmFPX` @ `36e9acd`, gfx1151, built by `scripts/build-rocmfpx.sh`; bare `llama-server`, root systemd unit) | `kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF` **`Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`** (23.15 GiB, ftype 114 ≈ 7.44 bpw: 353 F32 + 159 `Q8_0_ROCMFPX` + 339 `GGML_TYPE_Q6_0_ROCMFPX` tensors — only this fork can read them — with 64 protected output-head tensors), **MTP spec decode from the repo's own `mtp-Qwen3.8-27B-Q4_0.gguf` (n-max 4, pinned to ROCm0)**, **1 session × 262,144 (native window)** with **f16 KV**, the BF16 mmproj for vision, thinking on with the card's recipe (temp 1.0 / top_p 0.95 / top_k 20). Measured 2026-10-03: prose decode **27.6–28.1 t/s** at 4,813-token depth (draft acceptance 396/456 = 0.868, mean acceptance length ≈ 4.4), schema-echo 31.4 t/s at 100 %; `ngram-mod` chained onto MTP measured **−15 % on prose** and was rejected (the chain disables the hybrid's recurrent-state rollback). Cutover 2026-10-03 from the aligned Q8 file (17.0 t/s, acceptance 0.462 — rollback payload still on disk) | 7770 |
+| Dankclaw | `frank-large` | **halogen container** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | `peonist-ai/halogen-qwen3.8-27b` **`qwen3.8-27b-p1w4d-d2.hgn`** (33.4 GiB; the `.hgn` format loads in NO other engine — llama.cpp, vLLM and transformers all refuse it), **DFlash2 spec decode (`HALOGEN_DRAFTER=2`)**, **1 slot × 200,700 ctx** — `HALOGEN_KV_SLOTS=1` is load-bearing: batching XOR speculation, slots>1 silently disables the drafter (the sibling engine measured 41 → 14 t/s), d2 KV (2× fp8, 64 KiB/token → 12.2 GiB pool, engine sizes it and prints a boot ledger naming any clamp), **TEXT-ONLY** (the engine ignores the vision encoder — vision is frank-small's job), **censored base** (the ROCmFPX AGENT rollback is the refusal escape hatch), **no server-side sampler: absent temperature is GREEDY**, so the card's thinking recipe (temp 1.0 / top_p 0.95) rides the wire in `models.yml`. Publisher card 31.7 t/s mean across eight text workloads vs the retired ROCmFPX endpoint's measured 27.6–28.1. Prompt over the slot is a 400 naming the limit, never a truncation; `max_tokens` over `HALOGEN_MAX_TOKENS_CAP` is a 413. Cutover 2026-10-05|7770|
 | Dankclaw | `frank-small` | llama.cpp master, CUDA sm_86 build (root systemd unit, relinked) | bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** (9B dense qwen3_5 hybrid, 7.26 GiB GGUF, no MTP weights so no spec decode) + the repo's f16 mmproj for **vision**, on the RTX 3090, **2 × 200,704** (pool 401,408 — NOT the native window) with **fully unquantized KV (f16 K + f16 V)** under flash attention, **batch 4096 / ubatch 2048** (doubled 2026-10-05: measured +4.6 % prefill at 8,928 tokens for 616 MiB), 21,846 used / **2,279 free** of 24,576 MiB, 94.4 tok/s decode. Resized 2026-10-03 (3 × 180,224 + q8_0 V → 2 × 200,704 + f16 V → + Q6_K rung: the freed slots paid for exact values AND the higher quant; the rung costs a measured −10 % decode — weight-bandwidth-bound); rollback `MODEL_FILE=…/MiMo-Q5KM/…Q5_K_M.gguf` (buys the 10 % back) or `CTX_SIZE=540672 PARALLEL=3 CACHE_TYPE_V=q8_0` (Q5 only) | 7771 |

 Retired endpoints (`frank-npu`, and every prior runtime) are in
@@ -223,7 +223,7 @@ Scratch ports are not used on LLM hosts — a
 test process cannot share the single GPU anyway; validate against the service
 itself.

-(A gufo-era law; frank-large is ROCmFPX llama.cpp since 2026-10-02, but the rollback tag keeps it live.) **`gufo bench` cannot rank speculative-decode settings, and will invert the ranking.**
+(A gufo-era law; frank-large ran ROCmFPX llama.cpp 2026-10-02 → 10-05 and is the halogen container since, but the rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode settings, and will invert the ranking.**
 Measured 2026-09-28 on frank-large at depth 32768: its corpus
 reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
 draft widths, i.e. the same highly-predictable text every run, and 93 % of
@@ -249,7 +249,7 @@ one: `ip neigh show <ip>` from a peer — FAILED means off/NIC dead, while a
 merely wedged kernel usually still answers ARP.

 1. Stop and mask (self-healing would restart it):
-   - Dankclaw `frank-large` (root unit, vLLM podman on the iGPU):
+   - Dankclaw `frank-large` (root unit, halogen podman on the iGPU):
     `sudo systemctl stop frank-large && sudo systemctl mask frank-large`
   - Dankclaw `frank-small` (root unit, llama.cpp on the 3090):
     `sudo systemctl stop frank-small && sudo systemctl mask frank-small`
@@ -285,9 +285,10 @@ When renaming a service, retiring an endpoint, or moving a model: update the
 provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
 `models.yml`, then fix every `modelRoles` entry that named the old endpoint —
 a role pointing at a provider the file does not define fails at call time. As
-of 2026-10-02 **all three endpoints accept images** (`input: [text, image]`):
-`frank-large` through the BF16 mmproj that ships inside its GGUF payload,
-`frank-primary` through a BF16 mmproj loaded explicitly (this build never
+of 2026-10-05 **two of the three endpoints accept images** (`input: [text,
+image]`) — `frank-large` went TEXT-ONLY with the halogen swap (the engine
+ignores the vision encoder; the BF16-mmproj era ended 2026-10-05) — while
+`frank-primary` takes them through a BF16 mmproj loaded explicitly (this build never
 autoloads projectors), and `frank-small` through
 `mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf` passed as `--mmproj` — the MiMo
 GGUF itself carries no vision tensors. `frank-small`'s `contextWindow` is
@@ -304,84 +305,59 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
 audio encoder, and left again with the Gemma4 retirement on 2026-10-01
 (the MiMo tower is vision-only).

-### frank-large — ROCmFPX llama.cpp, Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (Dankclaw, ACTIVE)
+### frank-large — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE)

-The **kingjones30/ROCmFPX llama.cpp fork** at commit
-`36e9acd40e10a87cd3c3ef8ec734668757dc8520`, built for gfx1151 by
-`scripts/build-rocmfpx.sh` into `/home/dank/GitPackages/kingjones-ROCmFPX/build`,
-as a **root systemd unit** (`scripts/frank-large.service`, `User=dank`,
-`Restart=always`) running a bare `llama-server` (no container) on port 7770
-from the Strix Halo iGPU. Boot to healthy: **~15 s**.
+The **halogen engine** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`,
+digest-pinned, published tag 0.1.4) as a podman container driven by
+`scripts/serve-frank-large.sh` under a **root systemd unit**
+(`scripts/frank-large.service`, `User=dank`, `Restart=always`, `ExecStop=podman
+stop -t 120 frank-large-halogen`, the script `exec podman wait` so the restart
+policy is real) on port 7770 from the Strix Halo iGPU. Engine stdout goes to
+`/home/dank/logs/frank_large.log` — **the boot ledger (KV pool, drafter state,
+any clamp) lives there, not in the journal**. Serving
+`peonist-ai/halogen-qwen3.8-27b`: `qwen3.8-27b-p1w4d-d2.hgn` (35,880,329,102 B
+≈ 33.4 GiB, p1 weights / w4 activations / d2 KV) + `frank-large.tokenizer.gguf`
+(1,293,343 B). Cutover 2026-10-05 from the ROCmFPX llama.cpp fork
+([[frank-large-rocmfpx-q6agent-uncensored-mtp]], measured 27.6–28.1 t/s prose),
+chosen for the publisher's 31.7 t/s DFlash2 mean.

-**The runtime is not interchangeable — it is chosen by the FILE.**
-`kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF`'s protected-heads
-rung, `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf` (24,856,607,008 B,
-sha256 `e966deb3…`, ftype 114 ≈ 7.44 bpw), stores **339 tensors of type 102,
-`GGML_TYPE_Q6_0_ROCMFPX`, plus 159 type-103 `Q8_0_ROCMFPX`** — quantizations
-that exist in no other loader: stock llama.cpp and gufo both refuse the file at
-load (`unknown tensor type`). The AGENT construction keeps 64 output-head
-tensors at plain precision (the protected-heads design of the aligned repo's
-Q8 file, one quant rung down), and the uncensored tuning is what Frank swapped
-to on 2026-10-03. Vision comes from the BF16 `mmproj-Qwen3.8-27B-BF16.gguf`
-(931,145,888 B — hardlinked from the aligned payload dir, byte-identical).
+**The format is the lock.** `.hgn` loads in nothing else — the runtime is
+chosen by the FILE, as ROCmFPX's tensor types were before it. This is NOT the
+2026-09 `halogen-flash-server` image (different repo, Flash-Next w4b era); the
+halogen-server repo restarted versioning at 0.1.x.

-**The pin is load-bearing.** The 2026-08-25 ROCmFPX builds segfaulted in
-`ggml_compute_forward_set_rows` on exactly this file; `build-rocmfpx.sh`
-checks the commit out by SHA, so a fork-side force-push cannot silently
-re-break the endpoint.
+**Two product regressions, both accepted by Frank before deploy:** the
+endpoint is **text-only** (README honest limits: the vision encoder is
+ignored — frank-small owns images now) and the checkpoint is a finetune of the
+**censored** `Qwen/Qwen3.8-27B` base (refusals can return; the rollback is one
+commit-revert away with the AGENT payload still on disk).

-**Spec decode is the performance feature, and it is measured.** The draft is
-the repo's own `mtp-Qwen3.8-27B-Q4_0.gguf` (1,680,271,648 B, sha256 `051a1764…`,
-`nextn_predict_layers = 1`), pinned to the same card:
-`--spec-type draft-mtp --spec-draft-n-max 4 --spec-draft-n-min 0
--spec-draft-p-min 0.0 --spec-draft-device ROCm0 --spec-draft-ngl 99`.
-On the ALIGNED Q8 file (2026-10-02, bench windows, 2,202-token prompt, temp 0):
-**MTP 17.00–17.14 t/s vs 7.60–7.66 with the draft stripped — +123 %**,
-acceptance 0.462. On the UNCENSORED Q6 file (2026-10-03, production service,
-prose at 4,813-token depth): **27.57–28.10 t/s decode, draft acceptance
-396/456 = 0.868** (mean acceptance length ≈ 4.4 — drafting at nearly the full
-n-max 4), schema-echo traffic 31.44 t/s at 639/639 = 1.000. The drafter is
-materially better on this file; treat the card's 0.95 as a file property, not a
-throughput promise. **`ngram-mod` chained onto MTP was measured and REJECTED
-(2026-10-03):** `--spec-type draft-mtp,ngram-mod` cost −15 % on prose
-(23.40 t/s) and −7 % on verbatim block-copy even with the matcher demonstrably
-firing — the boot log says why: `recurrent state rollback is not compatible
-with 'ngram-mod' - disabling rollback support`, and the hybrid's MTP
-verification needs that rollback. Keep `draft-mtp` alone.
+**Engine laws that shape the config** (all in `scripts/frank-large.env` with
+evidence): `HALOGEN_KV_SLOTS=1` — batch XOR speculation, slots>1 turns the
+drafter off silently. `HALOGEN_SLOT_CTX=200700` — the per-conversation HARD
+ceiling (400 naming the limit, never truncation); `models.yml`
+`contextWindow: 200700` must match or the harness discovers the wall as a 400.
+`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled
+pair; the harness `maxTokens: 57344` stays under the cap (over-cap is a 413).
+`HALOGEN_DRAFTER=2` = DFlash2. **No server-side sampler**: an absent
+temperature is greedy, so the thinking recipe exists only in `models.yml`
+`extraBody`. `response_format` is unsupported (no constrained decoding).
+Thinking lands in **`reasoning_content`** (harness field unchanged); the same
+Qwen3.8 template rejects `reasoning_effort` outside `xhigh|medium|low`, so the
+`models.yml` fold stays mandatory.

-**Shape and budget:** ONE session at the native 262,144 window, unquantized
-f16 KV (~64 KiB/token → ~16 GiB, affordable inside the iGPU's ~114 GiB GTT
-aperture), `--fit off --flash-attn on --direct-io --ctx-checkpoints 16
--cache-ram 8192 --cache-idle-slots`. Measured GTT ~61 GiB under live load
-(host baseline 16.7 GiB). Prompt cache verified (Q8 era): a repeat pass over
-the same 6k prompt replays in 0.5 s.
+Fetch: `scripts/download-frank-large-halogen.sh` into
+`/home/dank/models/halogen-qwen3.8-27b/` (`-f` + byte count + sha256 +
+`--verify-only`, no token on the command line). The deployer and serve script
+both refuse a **symlinked or directory tokenizer** — an HF snapshot layout
+passes `test -f` but fails the engine's open at request time.

-**Client contract (llama.cpp, live-verified 2026-10-02):** `--jinja` is on
-(without it `chat_template_kwargs` are silently ignored), tools are native
-(`get_weather` → `{"city":"Seattle"}`), thinking lands in
-**`reasoning_content`**, `reasoning_effort` accepts ONLY
-`xhigh|medium|low` (anything else is HTTP 500 — `models.yml` folds `high` and
-`max` to `xhigh`), and thinking is enabled server-side with the card's
-recipe (temp 1.0 / top_p 0.95 / top_k 20 / min_p 0) plus
-`--reasoning-format deepseek --reasoning-budget -1`. Two traps: a request
-whose whole budget is consumed by reasoning returns `content: null` (bench
-clients must fall back to `reasoning_content` or they crash *after*
-measuring), and the fork logs
-`MTP boundary missing for seq_id=… ; resyncing` under concurrent draft
-resets — self-recovering, not a fault.
-
-Fetch: `scripts/download-frank-large-kingjones-uncensored.sh` (hardlinks the
-byte-identical MTP head and projector from the aligned payload dir when
-present, otherwise fetches; `-f` + byte count + LFS sha256 verified,
-`--verify-only`, no token on the command line).
-
-Runbook: `serverAppConfigs-docs/LLM/frank-large-rocmfpx-q6agent-uncensored-mtp.md`.
-Rollback: the ALIGNED Q8 file (the previous shape) remains on disk at
-`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/` — revert the
-2026-10-03 swap commit (unit/env/serve trio) and redeploy. The older gufo tag
-`frank-large-gufo-q4kxl-20261001` is now a **re-download** rollback: its
-Q4_K_XL payload and DFlash2 drafter were removed from disk 2026-10-03
-(`scripts/build-gufo.sh` stays for the rebuild).
+Runbook: `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md`.
+Rollback: revert the swap commit and redeploy — the ROCmFPX payload
+(`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`), the fork build
+and `scripts/build-rocmfpx.sh` (pin `36e9acd`) all remain on disk, seam-tested.
+The older gufo tag `frank-large-gufo-q4kxl-20261001` stays a **re-download**
+rollback (payloads removed 2026-10-03; `scripts/build-gufo.sh` stays).

 ### frank-small — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)

@@ -1005,7 +981,7 @@ Active:

 | Service | Runbook |
 |---------|---------|
-| frank-large (ROCmFPX llama.cpp fork, Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP4 + vision, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
+| frank-large (halogen container, Qwen3.8-27B .hgn + DFlash2, 1×200,700, text-only, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md` |
 | frank-small (llama.cpp CUDA, MiMo-V2.6-Distill-Qwen-9B Q6_K, 2×200,704 f16 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-mimo-q6k-2x200704.md` |
 | frank-primary (tcclaviger vLLM, Qwen3.8-Flash-Next FP8, Dankserver) | `serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
@@ -1015,6 +991,7 @@ Retired (rollback targets and history):

 | Runtime | Host | Window | Runbook |
 |---------|------|--------|---------|
+| ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
 | ROCmFPX llama.cpp fork Qwen3.8-27B Q8_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-02 → 10-03 (rollback target, payload on disk) | `frank-large-rocmfpx-q8agent-mtp.md` |
 | gufo v0.1.1 Qwen3.8-27B UD-Q8_K_XL/UD-Q4_K_XL + DFlash2 (`frank-large`) | Dankclaw | 2026-09-28 → 10-01, again 10-01 → 10-02 (payloads removed from disk 2026-10-03; rollback = re-download) | `frank-large-gufo-qwen38-27b.md` |
 | vLLM TheRock MiMo-V2.6-Distill-Qwen-9B BF16 + cross-model MTP (`frank-large`) | Dankclaw | 2026-10-01 (same-day rollback to gufo) | `frank-large-vllm-mimo-v26-bf16.md` |
@@ -0,0 +1,119 @@
+# frank-large — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE 2026-10-05)
+
+frank-large is the Strix Halo iGPU endpoint on Dankclaw (`:7770`). Since
+2026-10-05 it runs the **halogen** engine — `ghcr.io/peonist-ai/halogen` at a
+pinned digest, launched by `scripts/serve-frank-large.sh` under a root
+systemd unit as a podman container — serving
+**`peonist-ai/halogen-qwen3.8-27b`** (`qwen3.8-27b-p1w4d-d2.hgn`, 35,880,329,102 B
+≈ 33.4 GiB) with the **DFlash2** drafter at **one slot x 200,700 ctx**.
+
+It replaced the ROCmFPX llama.cpp fork endpoint ([[frank-large-rocmfpx-q6agent-uncensored-mtp]]),
+which remains the rollback target with its payload on disk.
+
+## Why swap
+
+|Metric|ROCmFPX Q6_0_ROCMFPX_AGENT + MTP (retired)|halogen qwen3.8-27b + DFlash2 (active)|
+|---|---|---|
+|Prose decode|27.6–28.1 t/s measured (4.8k depth, acceptance 0.868)|31.7 t/s mean across eight text workloads (publisher card, bf16 reference hardware)|
+|Copy-echo decode|31.4 t/s|publisher reports echo inside the same 31.7 mean|
+|Context|1 x 262,144|1 x 200,700|
+|Format|GGUF (fork-private tensor types 102/103)|`.hgn` — loads in NO other engine|
+
+The swap was accepted with eyes open on two product regressions, both
+confirmed by Frank before deploy:
+
+1. **Text-only.** halogen loads the language tower only; README "honest
+   limits": *"The vision encoder is ignored; image inputs are not supported."*
+   The mmproj-era vision role moves wholly to frank-small.
+2. **Censored base.** The checkpoint is a finetune of the censored
+   `Qwen/Qwen3.8-27B` base, built to make its DFlash2 head strong. The
+   ROCmFPX AGENT file was an *Uncensored* build; refusals can return.
+
+## Shape and why
+
+`scripts/frank-large.env` carries every tunable with its evidence; the load-
+bearing numbers:
+
+- **`HALOGEN_KV_SLOTS=1` + `HALOGEN_DRAFTER=2`** — the engine offers batch OR
+  speculation, never both (FLAGS.md: "Batching and speculation are exclusive:
+  a step batches when more than one conversation is running, so the drafter
+  stays off with more than one slot"). Slots>1 silently loses DFlash2; the
+  sibling halogen engine measured 41 -> 14 t/s when slots went up. frank-large
+  was a one-conversation endpoint before (PARALLEL=1), so the contract holds.
+- **`HALOGEN_SLOT_CTX=200700`** — the per-conversation HARD ceiling: a longer
+  prompt is a 400 naming the limit, never a truncation. The harness side is
+  wired to it: `models.yml` `contextWindow: 200700`, and OMP compacts at 70%
+  = 140,490, leaving 60,210 of the slot for the reply.
+- **`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200`** — coupled
+  pair (FLAGS.md table): a request at the cap must not outlive the timeout.
+  65536 > the harness's 57,344 `maxTokens` so an eager request never meets
+  the engine's 413.
+- **KV math:** 1 slot x 200,700 x 64 KiB/token (d2 KV = 2x fp8) = 12.2 GiB
+  pool, inside the ~65 GiB left after 33.4 GiB mmap'd weights + engine working
+  set + host baseline on the 121.2 GiB box. The engine sizes the pool itself
+  and **prints a boot ledger naming any clamp** — the post-deploy check reads
+  that ledger, not a formula.
+- **No server-side sampler:** an absent temperature is GREEDY. The card's
+  thinking recipe (temp 1.0 / top_p 0.95) lives in `models.yml` `extraBody`
+  — on the wire only.
+
+## Client contract deltas vs llama.cpp
+
+|Behavior|halogen|
+|---|---|
+|Prompt over slot|400 naming the limit (no truncation)|
+|`max_tokens` over cap|413, max_tokens over queue timeout|503s for the queue|
+|`response_format`|NOT supported (no constrained decoding)|
+|thinking field|`reasoning_content` (same as before)|
+|`reasoning_effort`|same Qwen3.8 template; outside {xhigh,medium,low} rejected — models.yml fold stays|
+|tools|server-side tool calls advertised — round-trip MUST be re-probed after deploy|
+|image input|ignored/error — route to frank-small|
+
+## Deploy mechanics
+
+- **Image:** `ghcr.io/peonist-ai/halogen@sha256:dc0a39a0016d6cfc58a197978febaafdf8d28403f724ded6111d98b5fb7ac0ea`
+  (published tag 0.1.4). The deployer gates the exact digest exists in podman
+  storage before dispatch. NOT the 2026-09 `halogen-flash-server` repo — that
+  was the Flash-Next w4b era, different image, retired.
+- **Payload:** `scripts/download-frank-large-halogen.sh` (fetch law: `curl -f`,
+  exact byte count, LFS/hub sha256, `--verify-only`, token never in argv) into
+  `/home/dank/models/halogen-qwen3.8-27b/`. The tokenizer bundle
+  (`frank-large.tokenizer.gguf`) must be a FLAT regular file — the deployer
+  and serve script both refuse a symlink or directory: an HF symlinked/snapshot
+  layout passes `test -f` but fails the engine's open at request time.
+- **Unit:** `scripts/frank-large.service` (root, `User=dank`, relinked into
+  `/etc/systemd/system/`), `ExecStart` = serve script, `ExecStop` = `podman
+  stop -t 120 frank-large-halogen`, script `exec podman wait` so
+  `Restart=always` is real. Engine stdout goes to
+  `/home/dank/logs/frank_large.log` (the boot ledger lives there, not in the
+  journal).
+- **Cutover:** `deploy_llm_large` in `scripts/deploy-remote.sh` sweeps the
+  legacy bare `llama-server --model /home/dank/models/kingjones-*` process and
+  the legacy `frank-large` container name by NAME (never `pkill -f
+  llama-server` — that hits frank-small), retires the old quadlet if the 2026-09
+  halogen era's unit lingers, waits port 7770 free, relinks, health-gates.
+- **Bench law:** stop+mask the unit before any GPU experiment on this box
+  (AGENTS.md); `/health` answers 503 while the 33.4 GiB payload loads — that
+  is loading, not failure.
+
+## Verification (post-deploy checklist)
+
+1. `grep -iE 'clamp|kv|pool|drafter|dflash' /home/dank/logs/frank_large.log`
+   — ledger shows the pool at the asked shape, DFlash2 active, NO clamp line.
+2. `curl -s http://192.168.0.143:7770/health` -> 200.
+3. Real completion with `reasoning_effort` + temp/top_p from the wire recipe;
+   `reasoning_content` populated; decode rate from the timings.
+4. Spec decode fires: multi-token step behavior visible in rate; compare
+   against DRAFTER=0 arm if in doubt (never against gufo — bench law).
+5. Harness round-trip: an OMP call through `llamacpp (dankclaw)/frank-large`
+   incl. a tool call (V1TOOLS).
+6. Prompt > 200,700 tokens returns the 400 naming the limit (contract test).
+
+## Rollback
+
+Revert the frank-large swap commit and redeploy `dankclaw --service frank-large`:
+the ROCmFPX payload (`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`),
+the fork build (`/home/dank/GitPackages/kingjones-ROCmFPX/build`), and its
+`scripts/build-rocmfpx.sh` pin (commit `36e9acd`, seam-tested) all remain on
+disk. The halogen payload can stay: it costs nothing while the unit points
+elsewhere.
@@ -17,7 +17,11 @@ timestamp: 2026-10-03

 # frank-large — ROCmFPX Uncensored Q6_0_ROCMFPX_AGENT + MTP (gfx1151)

-> **ACTIVE since 2026-10-03.** Cut over from
+> **RETIRED 2026-10-03 → 2026-10-05 (rollback target; payload, fork build and
+> build script all on disk).** Replaced by [[frank-large-halogen27b-dflash2]]
+> (DFlash2, publisher 31.7 t/s mean vs the 27.6–28.1 measured here). This page
+> remains the contract record for the rollback. Original banner follows.
+> ACTIVE since 2026-10-03. Cut over from
 > [[frank-large-rocmfpx-q8agent-mtp]] (ALIGNED Q8_0_ROCMFPX_AGENT, 17.0 t/s)
 > on Frank's request: the *uncensored* variant of the same protected-heads
 > family, one quant rung down. The swap turned out to be a strict upgrade on
@@ -14,11 +14,11 @@ mounted cache dir** (the image ships no `/root/.cache` at all; unmounted, every
 `torch.compile took 181.20 s`). `--max-num-seqs` must stay 3:
 SEQS 2 cannot compile this model. Rollback frank-primary-fp8-mtp-20261001),
 and Dankclaw
-runs two endpoints — **ROCmFPX llama.cpp (frank-large, port 7770:
-kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
-Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT, a ftype-114 file only that fork can
-read, with the repo's MTP head drafting 4 tokens at 27.6-28.1 t/s decode, ONE
-session at 262,144, f16 KV, vision via the BF16 projector)** and
+runs two endpoints — **halogen container (frank-large, port 7770:
+ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
+/halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
+engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
+lives on frank-small)** and
 **llama.cpp CUDA (frank-small, port 7771: MiMo-V2.6-Distill-Qwen-9B Q6_K
 GGUF, 2 x 200,704, fully unquantized f16 KV, on the RTX 3090)**.

@@ -45,10 +45,10 @@ expert-precise-memory arena 7,522 slots, /root/.cache mounted]
    subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
        E2[RTX 3090 (PCIe x4 dock) +
 gfx1151 iGPU 114 GiB GTT]
-        E[ROCmFPX llama.cpp gfx1151, pinned fork build
+        E[halogen container gfx1151, digest-pinned image
 frank-large, port 7770]
-        F[Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP-Q4_0 head
-1 x 262144 ctx, f16 KV, draft n-max 4, mmproj vision]
+        F[halogen-qwen3.8-27b .hgn p1w4d-d2 + DFlash2 drafter
+1 x 200700 ctx, d2 KV, text-only]
        G[llama.cpp master CUDA sm_86
 frank-small, port 7771]
        H[MiMo-V2.6 Q5_K_M GGUF
@@ -71,16 +71,17 @@ frank-small, port 7771]

 ## Active Deployment

- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — **ACTIVE since 2026-10-03**
-  (root systemd unit, port 7770): the pinned kingjones30/ROCmFPX fork serving
-  the uncensored protected-heads rung `Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf`
-  (23.15 GiB, ftype 114 = 159 type-103 + 339 GGML_TYPE_Q6_0_ROCMFPX tensors,
-  64 protected output heads) with the repo's `mtp-Qwen3.8-27B-Q4_0.gguf` head
-  drafting 4 tokens, one 262,144 window, f16 KV, the BF16 projector for
-  vision. Measured 27.6-28.1 t/s prose decode at 86.8 % draft acceptance; the
-  ngram-mod chain onto MTP was measured and REJECTED (-15 %: it disables the
-  hybrid's recurrent state rollback). Rollback: the aligned Q8 file, still on
-  disk
+- [[frank-large-halogen27b-dflash2]] — **ACTIVE since 2026-10-05**
+  (root systemd unit + podman, port 7770): the digest-pinned
+  `ghcr.io/peonist-ai/halogen` image serving `peonist-ai/halogen-qwen3.8-27b`
+  (`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB) with the DFlash2 drafter on ONE slot
+  of 200,700 ctx (slots>1 disables speculation — engine law). Publisher card
+  31.7 t/s mean across eight text workloads. Accepted tradeoffs: TEXT-ONLY
+  (the engine ignores the vision encoder; frank-small owns images) and the
+  CENSORED base model (the ROCmFPX AGENT file was Uncensored by construction).
+  No server-side sampler — the thinking recipe rides the wire in models.yml.
+  Rollback: revert the swap commit; the ROCmFPX payload + fork build +
+  `build-rocmfpx.sh` pin all remain on disk
 - [[frank-small-llama-mimo-q6k-2x200704]] — **ACTIVE since 2026-10-03**
  (root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving
  bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090, TWO
@@ -202,6 +203,14 @@ frank-small, port 7771]
  (bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
 - [[deployment]] — deploy and update process for both servers
 - [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
+- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — **RETIRED 2026-10-03 →
+  10-05 (rollback target; payload, fork build and build script on disk)**:
+  the pinned ROCmFPX fork serving the uncensored protected-heads
+  Q6_0_ROCMFPX_AGENT rung with the repo's MTP-Q4_0 head drafting 4 tokens,
+  one 262,144 window, f16 KV; measured 27.6-28.1 t/s prose decode at 86.8 %
+  acceptance. Replaced by the halogen DFlash2 endpoint; the ngram-mod-onto-MTP
+  rejection (-15 %, recurrent-state rollback) stays a live lesson for this
+  architecture.
 - [[frank-large-rocmfpx-q8agent-mtp]] — **RETIRED 2026-10-02 → 10-03 (the
  rollback target; payload still on disk)**: the same pinned ROCmFPX fork
  serving the ALIGNED `Q8_0_ROCMFPX_AGENT` (ftype 115, 17.0 t/s, MTP
@@ -213,8 +213,16 @@ The LLM inference fleet on the two hosts. Full index:
  agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB,
  Q4_0_ROCMFP4_FAST) + own MTP draft, build-qwen38 (retired 2026-08-30;
  rollback record)
- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — the ACTIVE frank-large
-  runbook since 2026-10-03 (root systemd unit, port 7770): the pinned
+- [[frank-large-halogen27b-dflash2]] — the ACTIVE frank-large runbook since
+  2026-10-05 (root systemd unit + podman, port 7770): the digest-pinned
+  ghcr.io/peonist-ai/halogen image serving peonist-ai/halogen-qwen3.8-27b
+  (qwen3.8-27b-p1w4d-d2.hgn, 33.4 GiB — a .hgn that loads in no other
+  engine) with the DFlash2 drafter on ONE slot of 200,700 ctx (slots>1
+  disables speculation — engine law); TEXT-ONLY and censored base, both
+  accepted tradeoffs; no server-side sampler, so the thinking recipe rides
+  the wire
+- [[frank-large-rocmfpx-q6agent-uncensored-mtp]] — RETIRED 2026-10-03 → 10-05
+  (rollback target; payload, fork build and build script on disk): the pinned
  kingjones30/ROCmFPX fork (scripts/build-rocmfpx.sh) serving kingjones777/
  Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF
  **Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT** — ftype 114, 339 tensors of