docs: frank-large flash-server v2 cutover — runbook, index, AGENTS.md

- New runbook: frank-large-flash-server-v2-swift15.md (engine, shape,
  memory model, engine laws, boot ledger, probes, rollback)
- LLM/index.md: active entry + retired halogen 27B entry
- Vault index: link the new runbook from the LLM Fleet section
- AGENTS.md: fleet topology table, Model Config section,
  Runbook Index, retired table (halogen 27B → 10-06)

AGENTS.md

@@ -12,8 +12,9 @@ merge absorbed the former `vllmScripts` repo, history preserved).
   Calibre-Web, Jellyfin, Plex) plus the `frank-primary` LLM service.
 - **Dankclaw** (192.168.0.143) — AI/search/dev/observability stack (SearXNG
   via Gluetun, ComfyUI, OpenWebUI, Hermes, Gitea, Grafana, Prometheus, Loki,
-  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (halogen container on
+  Alloy, Pi-hole) plus TWO LLM services: `frank-large` (flash-server
-  the gfx1151 iGPU) and `frank-small` (llama.cpp CUDA on the RTX 3090). Caddy on
+  container on the gfx1151 iGPU) and `frank-small` (llama.cpp CUDA on the
+  RTX 3090). Caddy on
   Dankclaw reverse-proxies both hosts.
 - **FrankBook-Pro** (192.168.0.112) — macOS mini running the *arr stack
   (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) as quadlets inside
@@ -25,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
 | Target | Service | Runtime | Model | Port |
 |--------|---------|---------|-------|------|
 | Dankserver | `frank-primary` | **radiance** (StillDeadcode v1.0.7 line; image `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned INDEX digest, 287 MiB, `libr4d` gfx1201 kernels, plain `podman run`, root systemd unit) | `Qwen/Qwen3.8-Flash-Next-FP8` official (172.8 GB, 131 shards, sha256-verified) converted **no-recipe** to `qwen4exp-fp8-native.rad` (177,109 MiB measured = 173.0 GiB) — **native modelopt block-FP8, no requantisation** — on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, `--expert-vs-cache-ratio 0.82` = the 99 %-hit point), **PLE 47.7 GiB pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV 7.32 KiB/token**, **MTP spec decode depth 3**, *…
-| Dankclaw | `frank-large` | **halogen container** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | `peonist-ai/halogen-qwen3.8-27b` **`qwen3.8-27b-p1w4d-d2.hgn`** (35,865,565,184 B; the `.hgn` format loads in NO other engine — llama.cpp, vLLM and transformers all refuse it), **DFlash2 spec decode (`HALOGEN_DRAFTER=2`)**, **1 slot × 200,700 ctx** — `HALOGEN_KV_SLOTS=1` is load-bearing: batching XOR speculation, slots>1 silently disables the drafter (measured: 8 slots at serial ~6 t/s/stream), d2 KV (2× fp8, 64 KiB/token → 12.2 GiB pool, engine sizes it and prints a boot ledger naming any clamp), **TEXT-ONLY** (the engine ignores the vision encoder — vision is frank-small's job), **censored base** (the ROCmFPX AGENT rollback is the refusal escape hatch), **no server-side sampler: absent temperature is GREEDY**, so the card's thinking recipe (temp 1.0 / top_p 0.95) rides the wire in `models.yml`. Publisher card 31.7 t/s mean across eight text workloads vs the retired ROCmFPX endpoint's measured 27.6–28.1. Prompt over the slot AND `max_tokens` over `HALOGEN_MAX_TOKENS_CAP` are both 400s naming the limit, never truncations. Serve passes NO `--ulimit nofile` (rootless runc cannot raise Dankclaw's 524288 hard limit — see Key Lessons). Cutover 2026-10-05|7770|
+| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the head per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 200,192 ctx** (KV pool 400,384 positions ≈ 10.7 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold) | 7770 |
 | Dankclaw | `frank-small` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF **RCO-IQ3_XXS -mtp** (27B dense qwen3_5 hybrid, 10,442,827,392 B; a per-tensor allocation mix of stock types; the MTP head in the file **drafted** — **spec ON** (draft-mtp depth 3) since 2026-10-06 evening, ngram skipped on the recurrent-rollback law) + the publisher's own **BF16 mmproj** for **vision**, on the RTX 3090, **2 × 131,072** (Frank's 128k cap each; the tightest fit in the endpoint's history — **489 MiB measured free**, the 2-3 GiB floor knowingly violated) with **q8_0 K + q8_0 V (FP8 class, 34 KiB/token)** under flash attention, batch 4096 / ubatch 512; **63.3 t/s recipe decode** (MTP acceptance 49.8 %; 48.0 no-spec). Supersedes the same-morning IQ3_S-mtp payloads: the two-slot no-spec shape and the one-slot MTP shape (both 2026-10-06; payload on disk).|7771|
 
 Retired endpoints (`frank-npu`, and every prior runtime) are in
@@ -234,7 +235,10 @@ Scratch ports are not used on LLM hosts — a
 test process cannot share the single GPU anyway; validate against the service
 itself.
 
-(A gufo-era law; frank-large ran ROCmFPX llama.cpp 2026-10-02 → 10-05 and is the halogen container since, but the rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode settings, and will invert the ranking.**
+(A gufo-era law; frank-large ran ROCmFPX llama.cpp 2026-10-02 → 10-05, the
+halogen 27B container 2026-10-05 → 10-06, and is flash-server since, but the
+rollback tags keep it live.) **`gufo bench` cannot rank speculative-decode
+settings, and will invert the ranking.**
 Measured 2026-09-28 on frank-large at depth 32768: its corpus
 reports `acceptance=0.933 ± 0.000` with an **identical trace sha256** across
 draft widths, i.e. the same highly-predictable text every run, and 93 % of
@@ -296,9 +300,10 @@ When renaming a service, retiring an endpoint, or moving a model: update the
 provider block, model `id`, `contextWindow`, `maxTokens`, `input` in
 `models.yml`, then fix every `modelRoles` entry that named the old endpoint —
 a role pointing at a provider the file does not define fails at call time. As
-of 2026-10-05 **two of the three endpoints accept images** (`input: [text,
+of 2026-10-06 **all three endpoints accept images** (`input: [text,
-image]`) — `frank-large` went TEXT-ONLY with the halogen swap (the engine
+image]`) — `frank-large` returned to vision with the flash-server v2 swap
-ignores the vision encoder; the BF16-mmproj era ended 2026-10-05) — while
+(the v2 vision tower loads beside the checkpoint; the text-only halogen
+27B window closed 2026-10-06) — while
 `frank-primary` takes them natively (radiance runs the checkpoint's
 27-layer Qwen-VL tower — patching, pad-token expansion and 3-part rotary
 are the engine core; `--mm-max-patches` available),
@@ -318,82 +323,93 @@ endpoint on 2026-09-11, returned 2026-09-17 with the Gemma4 projector's
 audio encoder, and left again with the Gemma4 retirement on 2026-10-01
 (the MiMo tower is vision-only).
 
-### frank-large — halogen container, Qwen3.8-27B + DFlash2 (Dankclaw, ACTIVE)
+### frank-large — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, ACTIVE)
 
-The **halogen engine** (`ghcr.io/peonist-ai/halogen@sha256:dc0a39a0…`,
+The **flash-server engine** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`,
-digest-pinned, published tag 0.1.4) as a podman container driven by
+digest-pinned, published tag 0.16.2, 3.6 GB image) as a podman container
-`scripts/serve-frank-large.sh` under a **root systemd unit**
+driven by `scripts/serve-frank-large.sh` under a **root systemd unit**
 (`scripts/frank-large.service`, `User=dank`, `Restart=always`,
-`ExecStop=podman
+`ExecStop=podman stop -t 60 frank-large-halogen`, the script runs `podman
-stop -t 120 frank-large-halogen`, the script runs `podman run` FOREGROUND —
+run` FOREGROUND — the container's exit code is the service's exit so
-the container's exit code is the service's exit so `Restart=always` stays
+`Restart=always` stays real, and the engine ledger lands on disk) on port
-real, and the engine ledger lands on disk) on port 7770 from the Strix Halo
+7770 from the Strix Halo iGPU. Engine stdout goes to
-iGPU. Engine stdout goes to `/home/dank/logs/frank_large.log` — **the boot
+`/home/dank/logs/frank_large.log` — **the boot ledger (weights resident, KV
-ledger (KV pool, prompt cache, drafter state, any clamp) lives there, not in
+pool, MTP head, prompt cache, any clamp) lives there, not in the journal**.
-the journal** (the 2026-10-05 `-d` + `podman wait` shape captured only the
+Cutover 2026-10-06 from the halogen 27B-dense era
-container ID — fixed 2026-10-06).
+([[frank-large-halogen27b-dflash2]], retired the same window).
-Serving
-`peonist-ai/halogen-qwen3.8-27b`: `qwen3.8-27b-p1w4d-d2.hgn` (35,865,565,184 B
-≈ 33.4 GiB, sha256 `274c3fc7…`, 1352 tensors; p1 weights / w4 activations /
-d2 KV) + the flat `tokenizer/` directory (5 files). Cutover 2026-10-05 from the ROCmFPX llama.cpp fork
-([[frank-large-rocmfpx-q6agent-uncensored-mtp]], measured 27.6–28.1 t/s prose),
-chosen for the publisher's 31.7 t/s DFlash2 mean.
 
-**The format is the lock.** `.hgn` loads in nothing else — the runtime is
+Serving `Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`:
-chosen by the FILE, as ROCmFPX's tensor types were before it. This is NOT the
+`qwen38-flash-next-v2-swift15-abliterated.hgn` (66,687,678,464 B ≈ 62.1
-2026-09 `halogen-flash-server` image (different repo, Flash-Next w4b era); the
+GiB, sha256 `9c880b8b…` — a compacted r2 of the card's 89,775,868,032 B
-halogen-server repo restarted versioning at 0.1.x.
+append-only original; the abliteration workflow appends and the repo's
+tooling drops dead padding, live tensors byte-identical) + the **REQUIRED**
+n-gram lookup table `qwen38-flash-next-ngram.hgn` (51,200,246,144 B,
+peonist @5cc17cea) + the vision tower `qwen38-flash-next-vision.hgn`
+(897,916,416 B, @e053f488) + the flat `tokenizer/` directory (6 files,
+fetched fresh from peonist). Cutover 2026-10-06 from the halogen
+27B-dense shape, chosen by Frank for the ABLITERATED base.
 
-**Two product regressions, both accepted by Frank before deploy:** the
+**The format is the lock.** The v2 `.hgn` loads ONLY in the
-endpoint is **text-only** (README honest limits: the vision encoder is
+flash-server line — the 27B-era halogen 0.1.4 container refuses it, so the
-ignored — frank-small owns images now) and the checkpoint is a finetune of the
+engine moved with the model. The MTP head is INSIDE the checkpoint
-**censored** `Qwen/Qwen3.8-27B` base (refusals can return; the rollback is one
+(`HALOGEN_MTP_HEAD` stays unset; the separate `qwen38-flash-next-mtp.hgn`
-commit-revert away with the AGENT payload still on disk).
+sidecar is deliberately not fetched).
 
-**Engine laws that shape the config** (all in `scripts/frank-large.env` with
+**Product facts vs the 27B era, both regressions FIXED:** the base is
-evidence): `HALOGEN_KV_SLOTS=1` — batch XOR speculation, slots>1 turns the
+**abliterated** (the 27B was a censored-base finetune; refusals were an
-drafter off silently. `HALOGEN_SLOT_CTX=200700` — the per-conversation HARD
+accepted trade there) and **vision is ON** (the 27B engine silently
-ceiling (400 naming the limit, never truncation); `models.yml`
+ignored its encoder — frank-small owned images for exactly one day at this
-`contextWindow: 200700` must match or the harness discovers the wall as a 400.
+endpoint).
-`HALOGEN_MAX_TOKENS_CAP=65536` + `HALOGEN_QUEUE_TIMEOUT=7200` are a coupled
-pair; the harness `maxTokens: 57344` stays under the cap (over-cap is a 400
-naming the limit — README, never a 413). Thinking tokens COUNT against the
-budget: exhausted mid-thought is `finish_reason: "length"` with empty
-`content` and the reasoning in `reasoning_content`.
-`HALOGEN_DRAFTER=2` = DFlash2. **No server-side sampler**: an absent
-temperature is greedy, so the thinking recipe exists only in `models.yml`
-`extraBody`. `response_format` is unsupported (no constrained decoding).
-Thinking lands in **`reasoning_content`** (harness field unchanged); the same
-Qwen3.8 template rejects `reasoning_effort` outside `xhigh|medium|low`, so the
-`models.yml` fold stays mandatory.
-`HALOGEN_CACHE_MB=16384` pins the prompt cache (2026-10-06): auto is OFF
-on this box — the floor is 2 full-context entries (36.9 GB) and the boot
-ledger had 35.7 GB free — so every deep request re-prefilled its whole
-79-81k context (84.9-179.8 s per request for ~500-650 new tokens per
-turn) until the pin. Probe-verified: a 17.3k cold prefill of 27.0 s
-resumes in 1.7 s on the next turn (cache_n 16,384). Decode-at-depth is
-structural: measured 5.5 t/s at 46k → 3.3-3.8 at 80k vs 30-50 shallow
-(the KV pool reads at ~8 GB/s effective vs ~180-240 for the weights);
-`docs/FLAGS.md` documents no decode-affecting lever and 0.1.4 is the
-latest engine. The boot WARNING "smaller than ONE full-context entry
-(18.4 GB at kMaxCtx=262144)" is a false positive for our slot: the
-deepest possible entry is 200,700 × ~70 KB ≈ 14 GB, inside the pin, and
-the 262k entry it fears cannot exist (over-slot is a 400).
 
-Fetch: `scripts/download-frank-large-halogen.sh` into
+**Engine laws that shape the config** (all in `scripts/frank-large.env`
-`/home/dank/models/halogen-qwen3.8-27b/` (`-f` + byte count + LFS sha256 +
+with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
-`--verify-only`, no token on the command line; resume goes against the
+policy disables the MTP head for whole requests after low acceptance and
-RESOLVED CDN URL because curl drops `Range` across HF's 302). The deployer
+HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
-and serve script both refuse a missing tokenizer file or **any symlink** —
+for it (accepted). `HALOGEN_CTX=200192` is the PER-SLOT hard ceiling (400
-an HF cache snapshot symlinks each entry into `blobs/` and those links
+naming the limit, never truncation); `models.yml` `contextWindow: 200192`
-dangle inside the container mount.
+must match, and compaction at 70 % = 140,134 + 57,344 budget = 197,478 fits.
+`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
+law does NOT carry over; batching and speculation coexist (the card's
+quickstart ships 2 slots with the head), so 2 is concurrent capacity.
+`HALOGEN_KV_POOL_POSITIONS=400384` (explicit 2 × ctx; ~28 KiB/position ⇒
+~10.7 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
+harness always sends 57,344); `HALOGEN_MAX_TOKENS_CAP=65536` +
+`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
+**A server-side sampler EXISTS** (`HALOGEN_TEMPERATURE/TOP_P/TOP_K` =
+1.0/0.95/20, the card's thinking recipe) — an absent temperature is no
+longer greedy; `models.yml` extraBody carries the same values, consistent
+not load-bearing. Prompt cache is ON by default (`PROMPT_CACHE=2`): a warm
+100k follow-up costs ~2 s vs ~88 s cold — the 27B engine's manual
+`HALOGEN_CACHE_MB` pin era is over. `response_format` is unsupported.
+Thinking lands in **`reasoning_content`**; the v2 chat template rejects
+`reasoning_effort` outside `xhigh|medium|low`, so the `models.yml` fold
+stays mandatory. Thinking tokens COUNT against the budget (empty content +
+`finish_reason: "length"` when exhausted mid-thought). Vision: OpenAI
+`image_url` content parts only; ≥256 px probe images or the model answers
+"video".
 
-Runbook: `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md`.
+**Memory ledger** (121.2 GiB unified; the referee is the boot ledger):
-Rollback: revert the swap commit and redeploy — the ROCmFPX payload
+weights ~62 + KV ~10.7 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
-(`/home/dank/models/kingjones-Qwen3.8-27B-ROCmFP4-STRIX-MTP/`), the fork build
+~2 ≈ 82.7 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~18 GiB
-and `scripts/build-rocmfpx.sh` (pin `36e9acd`) all remain on disk, seam-tested.
+margin over baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd
-The older gufo tag `frank-large-gufo-q4kxl-20261001` stays a **re-download**
+page cache beside the checkpoint — the card's quickstart arithmetic does
-rollback (payloads removed 2026-10-03; `scripts/build-gufo.sh` stays).
+not bill it as resident, but READ the boot ledger and re-size if the
+engine prints a clamp.
+
+Fetch: `scripts/download-frank-large-flashnext.sh` into
+`/home/dank/models/halogen-flashnext-v2/` (`-f` + exact byte counts + LFS
+sha256 + `--verify-only`, no token on the command line; resume goes
+against the RESOLVED CDN URL because curl drops `Range` across HF's 302).
+The serve gates mirror the deployer EXACTLY (byte-exact checkpoint +
+n-gram + vision tower, flat tokenizer with all six files as regular
+files) — the 2026-10-01 crash-loop lesson.
+
+Runbook:
+`serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md`.
+Rollback: revert the swap commit and redeploy — the 27B payload
+(`/home/dank/models/halogen-qwen3.8-27b/`), its fetch script and the
+halogen 0.1.4 image digest all remain on disk; the ROCmFPX door behind
+that (`scripts/build-rocmfpx.sh`, pin `36e9acd`) is untouched.
 
 ### frank-small — llama.cpp on the RTX 3090 (Dankclaw, ACTIVE)
 
@@ -997,7 +1013,7 @@ Active:
 
 | Service | Runbook |
 |---------|---------|
-| frank-large (halogen container, Qwen3.8-27B .hgn + DFlash2, 1×200,700, text-only, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md` |
+| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×200,192, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
 | frank-small (llama.cpp CUDA, Swift-1.5-27B Uncensored RCO IQ3_XXS-mtp, 2×131,072 draft-mtp q8_0 KV, Dankclaw) | `serverAppConfigs-docs/LLM/frank-small-llama-swift15-unc-iq3xxs-mtp-131072.md` |
 | frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` |
 | Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
@@ -1007,6 +1023,7 @@ Retired (rollback targets and history):
 
 | Runtime | Host | Window | Runbook |
 |---------|------|--------|---------|
+| halogen 0.1.4 Qwen3.8-27B p1w4d-d2 + DFlash2 (`frank-large`) | Dankclaw | 2026-10-05 → 10-06 (rollback target — payload + fetch script + image digest on disk; owns the prompt-cache pin and FA-tile lessons) | `frank-large-halogen27b-dflash2.md` |
 | ROCmFPX llama.cpp fork Qwen3.8-27B Uncensored Q6_0_ROCMFPX_AGENT + MTP (`frank-large`) | Dankclaw | 2026-10-03 → 10-05 (rollback target, payload + fork build on disk) | `frank-large-rocmfpx-q6agent-uncensored-mtp.md` |
 | tcclaviger vLLM fork Qwen3.8-Flash-Next-Uncensored FP8, TP=2, shape G (`frank-primary`) | Dankserver | 2026-09-29 → 10-05 (rollback target — tag `frank-primary-vllm-shapeG-20261003`; trio + env lore in the tag, image in rootless storage) | `dankserver-qwen38-fp8-expert-offload-tuning.md` |
 | FreeToken PR #587 ROCm, Qwen3.8-Flash-Next-Uncensored-FP8, ONE R9700 (`frank-primary`) | Dankserver | 2026-10-05 → 10-05 (rollback target — tag `frank-primary-freetoken-pr587-20261005`; trio + Dockerfile in the tag, image `localhost/frank-primary-freetoken:pr587-77a7201` and orcarouter payload on disk) | `frank-primary-freetoken-pr587.md` |

serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md

@@ -0,0 +1,178 @@
+# frank-large — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, Strix Halo iGPU)
+
+**ACTIVE since 2026-10-06.** Rollback = revert the swap commit's trio
+(`scripts/frank-large.env`, `scripts/serve-frank-large.sh`,
+`scripts/frank-large.service`); the retired 27B payload stays on disk at
+`/home/dank/models/halogen-qwen3.8-27b/` and its fetch script
+(`scripts/download-frank-large-halogen.sh`) stays in the repo.
+
+## Why this shape exists
+
+Frank's ask: replace the frank-large model with
+`https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`.
+That payload is an **abliterated Swift1.5 finetune of peonist's
+Qwen3.8-Flash-Next v2 checkpoint in the v2 `.hgn` format** — a file that
+loads ONLY in the **halogen-flash-server** engine line, NOT in the halogen
+0.1.4 container that served the 27B dense checkpoint. The model move is
+therefore a **full engine migration**, and the accepted-product facts flip
+twice in the fleet's favour:
+
+- the base is **abliterated** (the 27B halogen endpoint was a finetune of
+  the censored `Qwen/Qwen3.8-27B`; refusals were an accepted regression there),
+- **vision is back** — the v2 vision tower loads beside the checkpoint and
+  the engine runs it (the 27B halogen engine silently ignored its encoder),
+- **MTP speculative decoding returns** with the head inside the checkpoint
+  itself (the 27B shipped a DFlash2 drafter instead; comparable purpose,
+  different file plumbing),
+- the card claims **53–57 t/s greedy decode, ~1,640 t/s prefill at 34k, ~72 %
+  fewer thinking tokens** than the censored 27B (905 mean), and **262,144
+  native context**. Publisher-card numbers — verify against live traffic
+  before quoting further (see Verification).
+
+The engine is **closed-source**, ships as `ghcr.io/peonist-ai/
+halogen-flash-server:0.16.2` (the version the model card's quickstart pins),
+digest-pinned:
+
+```
+ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
+```
+
+(3.6 GB image; RepoDigest captured at pull 2026-10-06. Card-tested fallback
+if a future pin misbehaves: 0.15.1.)
+
+## The payload — a triangle of files, all sha256-verified at fetch
+
+| File | Bytes | Source repo @ revision | sha256 (LFS) |
+|---|---|---|---|
+| `qwen38-flash-next-v2-swift15-abliterated.hgn` | 66,687,678,464 | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated @ main (`69a85db`) | `9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037` |
+| `qwen38-flash-next-ngram.hgn` (REQUIRED) | 51,200,246,144 | peonist-ai/halogen-qwen3.8-flash-next @ `5cc17cea` | `3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe` |
+| `qwen38-flash-next-vision.hgn` (enabled) | 897,916,416 | peonist-ai/halogen-qwen3.8-flash-next @ `e053f488` | `d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10` |
+| `tokenizer/` (6 files, flat dir) | 22.4 MB total | peonist @ `e053f488` | `tokenizer.json` is LFS `0997f410…` (byte-identical to the 27B-era copy); the rest are plain git blobs, size-checked |
+
+Payload notes worth keeping:
+
+- The deployed checkpoint is a **compacted r2** of the card's append-only
+  original (89,775,868,032 B → 66,687,678,464 B): the abliteration workflow
+  appends and the repo's `tooling/` compact drops dead padding; live tensors
+  are byte-identical, the LFS sha256 is the contract, and the 24 GiB saving
+  is why it fits.
+- **`HALOGEN_MTP_HEAD` stays UNSET**: the MTP head is INSIDE the v2
+  checkpoint (the separate `qwen38-flash-next-mtp.hgn` sidecar is for a
+  different assembly and is deliberately not fetched).
+- The **n-gram lookup table is REQUIRED**. The engine reads it beside the
+  checkpoint (documented fallback); the card's quickstart also names it
+  explicitly via `HALOGEN_NGRAM_TABLE`, and the serve script passes both —
+  the env var is belt-and-braces over the documented beside-the-checkpoint
+  default, and both mechanisms resolve to the same file.
+- The tokenizer is fetched fresh from peonist @ `e053f488` into
+  `/home/dank/models/halogen-flashnext-v2/tokenizer/` — same flat-directory,
+  no-symlink law as every other payload (an HF cache snapshot's symlinks
+  dangle inside the container mount; both the serve gate and the deployer
+  refuse them).
+
+Fetch: `scripts/download-frank-large-flashnext.sh` (`--verify-only` to
+re-check; `-f` + exact byte counts + LFS sha256 + resolve-the-CDN-then-`-C -`
+resume law throughout; no token — the repos are public).
+
+## Shape — the knobs that are not free
+
+`scripts/frank-large.env` carries the full rationale inline; the table:
+
+| Env | Value | Why |
+|---|---|---|
+| `HALOGEN_CTX` | 200192 | per-slot HARD ceiling (a longer prompt is a 400 naming the limit, never truncation). Harness compaction: 70 % = 140,134; 140,134 + 57,344 = 197,478 ≤ 200,192 |
+| `HALOGEN_KV_SLOTS` | 2 | the 27B engine's "slots>1 disables the drafter" law does NOT carry over — on this engine batching and speculation coexist (the card's quickstart ships 2 slots WITH the head). 2 slots = concurrent capacity |
+| `HALOGEN_KV_POOL_POSITIONS` | 400384 | explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ ~10.7 GiB KV |
+| `HALOGEN_HOST_RESERVE_GIB` | 20 | keeps the host honest (the w4b-era lesson) |
+| `HALOGEN_SPEC_ADAPT` | **0** | MANDATORY (card): the default adaptive policy turns the MTP head off for whole requests after low acceptance and **halves decode**. The head stays on; temp-1.0 traffic pays some acceptance for it — accepted trade |
+| `HALOGEN_MAX_TOK` / `HALOGEN_MAX_TOKENS_CAP` / `HALOGEN_QUEUE_TIMEOUT` | 8192 / 65536 / 7200 | 8192 is only the per-request DEFAULT (the harness always sends 57,344); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s) |
+| `HALOGEN_TEMPERATURE` / `TOP_P` / `TOP_K` | 1.0 / 0.95 / 20 | the card's thinking recipe as SERVER-SIDE defaults — the engine HAS a sampler now, so an absent temperature is no longer greedy; models.yml extraBody matches it (consistent, not load-bearing) |
+| `HALOGEN_WEIGHTS_LOCK` | 1 | card's belt-and-braces; the engine pins weights by default on v2 (`HALOGEN_FLASH_PIN_TRUNK=0` is a w4b-only knob and is REFUSED by v2) |
+| `HALOGEN_MODEL_ID` / publish | frank-large / 7770:8731 | unchanged aliases; engine port 8730 never published |
+| MTP depth | default 2 | FLAGS.md: 2 is best all-round, 3 for code — Frank's traffic is agent/prose |
+
+**Memory ledger** (121.2 GiB unified, the referee is the boot ledger):
+weights ~62 GiB resident + KV ~10.7 + arena ~8 (`3.8 + 8.4 × 8192/16384`)
++ vision ~2 ≈ **82.7 GiB engine-side**, + 20 host reserve = ~103 with
+~18 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram
+table rides as mmap'd page cache beside the checkpoint — the card's own
+quickstart arithmetic does not bill it as resident, but WATCH the ledger
+at boot and re-size if the engine prints a clamp. Prompt cache is ON by
+default (`PROMPT_CACHE=2`): a follow-up turn at 100k ctx costs ~2 s
+instead of ~88 s cold — the big-TTFT win the 27B engine needed a manual
+`HALOGEN_CACHE_MB` pin for is built in here.
+
+## Engine contract facts (FLAGS.md of the flash-server repo)
+
+- OpenAI chat **and** `/v1/responses` **and** `/v1/messages` **and**
+  `/v1/completions`; tool calls verified on the card; `response_format`
+  still unsupported (no constrained decoding).
+- Thinking → **`reasoning_content`** (same field as every engine this
+  endpoint has run); the v2 chat template rejects `reasoning_effort`
+  outside `xhigh|medium|low` — the models.yml fold (high→xhigh) stays
+  mandatory.
+- Vision: OpenAI `image_url` content parts only (the fleet's standard
+  contract; the legacy top-level `images` field is the frank-small trap).
+  ≥256 px probe images or the model answers "video".
+- `HALOGEN_MAX_TOKENS_CAP` over-cap is a 400 naming the limit — never a
+  truncation. `max_tokens` INCLUDES thinking.
+- All flags read once at startup; every flag is BITWISE except
+  `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`,
+  `HALOGEN_INDEXER_BUDGET`, `HALOGEN_COMPOSABLE_CONTEXT`. Unset
+  `HALOGEN_CHECKPOINT` defaults to `/models/qwen38-flash-next-v2.hgn`
+  with the lookup table beside it.
+- Prompt-lookup drafting (3,3 window) rides beside the MTP head by
+  default and is byte-identical to greedy verification where it fires.
+- Watchdog: `HALOGEN_ENGINE_WATCHDOG_S=180` with defer-logic for memory
+  stalls — on a shared host the defaults are the safe choice; the
+  restart policy stays real through the root unit's `Restart=always`.
+
+Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it); `amdgpu.noretry=-1`
+(auto) — for this IP auto resolves to retry-on, the same effective state
+the 27B engine ran on. Only revisit if the ledger shows SVM complaints.
+
+## Mechanics
+
+- **Root systemd unit** (`scripts/frank-large.service`, `User=dank`,
+  `LimitMEMLOCK=infinity`, `Restart=always`, `ExecStop=podman stop -t 60
+  frank-large-halogen`), driven by `scripts/serve-frank-large.sh` — the
+  fleet's `podman run`-direct pattern. The script runs the container
+  FOREGROUND so the boot ledger (weights resident, pool fit, MTP head,
+  prompt cache, any clamp) lands in `/home/dank/logs/frank_large.log`.
+- Serve gates mirror the deployer EXACTLY (2026-10-01 crash-loop law):
+  image digest in podman storage, byte-exact checkpoint + n-gram table +
+  vision tower, flat tokenizer with all six files as regular files,
+  GPU nodes present.
+- Engine stdout goes to the log, NOT the journal. The FIRST request after
+  a (re)start can sit silent ~45–60 s on this engine class (health
+  `engine_unresponsive`, 503s) and self-recovers — do not restart over it.
+
+## Verification checklist (run after any redeploy)
+
+1. Boot ledger greps in `/home/dank/logs/frank_large.log`: resident
+   weights (~62 GiB), KV pool line, MTP/drafter line, prompt-cache line,
+   **no clamp named**.
+2. `curl http://192.168.0.143:7770/health` → 200.
+3. Text completion probe: reasoning lands in `reasoning_content`,
+   `timings` honest (generate ≥128 tokens; at 8 the inter_token number is
+   first-token noise).
+4. Vision probe: a 512×512 solid-red PNG through an `image_url` part →
+   "Red" (≥256 px, one word expected).
+5. Spec decode: `draft_n`/`draft_n_accepted` present in timings on
+   real-shaped traffic (SPEC_ADAPT=0 means the head is always engaged).
+
+## Client contract (harness)
+
+`~/.omp/agent/models.yml` — provider `llamacpp (dankclaw)` @
+`http://192.168.0.143:7770/v1`: `contextWindow: 200192`,
+`maxTokens: 57344`, `input: [text, image]`,
+`reasoningContentField: "reasoning_content"`, the `high→xhigh` effort
+fold, `extraBody` temperature 1.0 / top_p 0.95. Compaction still rides
+frank-small (the `compactionModel` steering is unchanged).
+
+## Rollback
+
+Revert the swap commit and redeploy — the 27B trio is intact on disk
+(payload at `~/models/halogen-qwen3.8-27b/`, fetch script, and the halogen
+0.1.4 image digest still in podman storage). The older ROCmFPX rollback
+door (`build-rocmfpx.sh` pin) is untouched.

serverAppConfigs-docs/LLM/index.md

@@ -10,11 +10,12 @@ experts in a 40 GiB/rank pinned host pool + VRAM cache (ratio 0.82), the
 47.7 GiB PLE table pinned to host RAM, fp8 KV, MTP spec decode depth 3,
 2 × 200,000 first boot. Rollback tags frank-primary-freetoken-pr587-20261005
 and frank-primary-vllm-shapeG-20261003),
-and Dankclaw runs two endpoints — **halogen container (frank-large, port 7770:
+and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
-ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
+(frank-large, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
-/halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
+digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
-engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
+Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
-lives on frank-small)** and
+MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
+ctx, vision ON)** and
 **llama.cpp CUDA (frank-small, port 7771: ajgazin Swift-1.5-Qwen3.8-27B
 Uncensored Dynamic-MTP RCO-IQ3_XXS GGUF — 27B (the MTP head in the file
 IS drafted, draft-mtp depth 3), TWO 131,072 conversations at Frank's
@@ -45,10 +46,10 @@ PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 3,
     subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
         E2[RTX 3090 (PCIe x4 dock) +
 gfx1151 iGPU 114 GiB GTT]
-        E[halogen container gfx1151, digest-pinned image
+        E[flash-server container gfx1151, digest-pinned image
 frank-large, port 7770]
-        F[halogen-qwen3.8-27b .hgn p1w4d-d2 + DFlash2 drafter
+        F[flash-server v2 abliterated Swift1.5 .hgn + ngram table + vision tower
-1 x 200700 ctx, d2 KV, text-only]
+2 x 200192 ctx, MTP always-on, image input]
         G[llama.cpp master CUDA sm_86
 frank-small, port 7771]
         H[Swift-1.5-27B Uncensored RCO-IQ3_XXS-mtp GGUF
@@ -71,30 +72,37 @@ frank-small, port 7771]
 
 ## Active Deployment
 
-- [[frank-large-halogen27b-dflash2]] — **ACTIVE since 2026-10-05**
+- [[frank-large-flash-server-v2-swift15]] — **ACTIVE since 2026-10-06**
   (root systemd unit + podman, port 7770): the digest-pinned
-  `ghcr.io/peonist-ai/halogen` image serving `peonist-ai/halogen-qwen3.8-27b`
+  `ghcr.io/peonist-ai/halogen-flash-server` 0.16.2 engine serving the
-  (`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB) with the DFlash2 drafter on ONE slot
+  Quat3rnion ABLITERATED Swift1.5 finetune of peonist's
-  of 200,700 ctx (slots>1 disables speculation — engine law). Publisher card
+  Qwen3.8-Flash-Next v2 checkpoint (`qwen38-flash-next-v2-swift15-abliterated.hgn`,
-  31.7 t/s mean across eight text workloads. Accepted tradeoffs: TEXT-ONLY
+  66.7 GB compacted r2 of the card's 89.8 GB append-only original,
-  (the engine ignores the vision encoder; frank-small owns images) and the
+  sha256-verified) with the REQUIRED n-gram lookup table (51.2 GB,
-  CENSORED base model (the ROCmFPX AGENT file was Uncensored by construction).
+  @peonist 5cc17cea) and the vision tower (897.9 MB, @e053f488) beside
-  No server-side sampler — the thinking recipe rides the wire in models.yml.
+  it. TWO slots of 200,192 ctx, MTP head ALWAYS ON (`HALOGEN_SPEC_ADAPT=0`
-  Rollback: revert the swap commit; the ROCmFPX payload + fork build +
+  — the adaptive default turns the head off per request and halves
-  `build-rocmfpx.sh` pin all remain on disk
+  decode), server-side sampler at the card's 1.0/0.95/20, coupled
-  Tuned 2026-10-06: `HALOGEN_CACHE_MB=16384` pins the prompt cache —
+  cap/timeout 65536/7200, prompt cache ON by default (the 27B engine's
-  auto was OFF at the 36.9 GB two-entry floor, so every deep request
+  manual `HALOGEN_CACHE_MB` pin era is over — warm 100k turn ≈ 2 s vs
-  re-prefilled its 79-81k context (85-180 s each); a 17.3k cold prefill
+  ≈ 88 s cold). Vision ON (image_url parts only). Card: 53-57 t/s
-  of 27.0 s now resumes in 1.7 s on the warm turn (cache_n 16,384). The
+  greedy, ~1,640 t/s prefill at 34k, ~72 % fewer thinking tokens.
-  serve script captures the engine ledger on disk (foreground `podman
+  Rollback: revert the swap commit; the 27B payload + fetch script stay
-  run`; the `-d` era wrote only the container ID). Decode-at-depth is
+  on disk
-  structural: 3.3-3.8 t/s at 80k vs 30-50 shallow; no supported flag
+- [[frank-large-halogen27b-dflash2]] — **RETIRED 2026-10-05 → 10-06
-  touches it.
+  (rollback target; payload + fetch script + image on disk)**: the
-  The FA tile lever from upstream issue #12 (`HALOGEN_FA_TQ=256`) was
+  digest-pinned `ghcr.io/peonist-ai/halogen` 0.1.4 engine serving
-  A/B'd at 80k on 2026-10-06: decode ±0.3 % (rejected), prefill −9.5 %
+  `peonist-ai/halogen-qwen3.8-27b` (`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB)
-  on this topology — the default tile stands. The deep harness roles
+  — DFlash2 drafter, ONE 200,700 slot, text-only (the engine ignored the
-  (`slow`, `advisor`) rerouted to frank-primary; frank-large keeps
+  vision encoder), censored base. Owns the measured lessons: the prompt
-  OpenWebUI and shallow callers.
+  cache pin (`HALOGEN_CACHE_MB=16384` — auto was OFF at the 36.9 GB
+  two-entry floor; a 17.3k cold prefill of 27.0 s resumed in 1.7 s warm),
+  the FA tile rejection (`HALOGEN_FA_TQ=256`: prefill −9.5 %, decode
+  ±0.3 %), and deep decode at 3.3-3.8 t/s@80k being structural. The
+  foreground-`podman run` ledger capture and the deep-role rerouting
+  (`slow`/`advisor` → frank-primary) carried forward. Superseded by the
+  MODEL swap: the abliterated v2 .hgn loads only in flash-server, so the
+  engine moved with the model
 - [[frank-small-llama-swift15-unc-iq3xxs-mtp-131072]] — **ACTIVE since
   2026-10-05 (evening); UNCECERSED RCO-IQ3_XXS with MTP back on since
   2026-10-06 evening** (root systemd unit, port 7771): llama.cpp master
@@ -400,7 +408,7 @@ frank-small, port 7771]
   (deployed and reverted 2026-08-22)
 - [[Dankserver System Specs]] — the 2x R9700 box (frank-primary)
 - [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on
-  the unsloth b11030 runtime since 2026-09-21, frank-small on the RTX 3090)
+  the flash-server v2 runtime since 2026-10-06, frank-small on the RTX 3090)
 - [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
   on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
   the retirement record — the IOMMU/SVA err=-19 root cause and where