docs: frank-large flash-server v2 cutover — runbook, index, AGENTS.md

- New runbook: frank-large-flash-server-v2-swift15.md (engine, shape,
  memory model, engine laws, boot ledger, probes, rollback)
- LLM/index.md: active entry + retired halogen 27B entry
- Vault index: link the new runbook from the LLM Fleet section
- AGENTS.md: fleet topology table, Model Config section,
  Runbook Index, retired table (halogen 27B → 10-06)

serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md

@@ -0,0 +1,178 @@
+# frank-large — halogen-flash-server 0.16.2, Qwen3.8-Flash-Next v2 Swift1.5 ABLITERATED (Dankclaw, Strix Halo iGPU)
+
+**ACTIVE since 2026-10-06.** Rollback = revert the swap commit's trio
+(`scripts/frank-large.env`, `scripts/serve-frank-large.sh`,
+`scripts/frank-large.service`); the retired 27B payload stays on disk at
+`/home/dank/models/halogen-qwen3.8-27b/` and its fetch script
+(`scripts/download-frank-large-halogen.sh`) stays in the repo.
+
+## Why this shape exists
+
+Frank's ask: replace the frank-large model with
+`https://huggingface.co/Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated`.
+That payload is an **abliterated Swift1.5 finetune of peonist's
+Qwen3.8-Flash-Next v2 checkpoint in the v2 `.hgn` format** — a file that
+loads ONLY in the **halogen-flash-server** engine line, NOT in the halogen
+0.1.4 container that served the 27B dense checkpoint. The model move is
+therefore a **full engine migration**, and the accepted-product facts flip
+twice in the fleet's favour:
+
+- the base is **abliterated** (the 27B halogen endpoint was a finetune of
+  the censored `Qwen/Qwen3.8-27B`; refusals were an accepted regression there),
+- **vision is back** — the v2 vision tower loads beside the checkpoint and
+  the engine runs it (the 27B halogen engine silently ignored its encoder),
+- **MTP speculative decoding returns** with the head inside the checkpoint
+  itself (the 27B shipped a DFlash2 drafter instead; comparable purpose,
+  different file plumbing),
+- the card claims **53–57 t/s greedy decode, ~1,640 t/s prefill at 34k, ~72 %
+  fewer thinking tokens** than the censored 27B (905 mean), and **262,144
+  native context**. Publisher-card numbers — verify against live traffic
+  before quoting further (see Verification).
+
+The engine is **closed-source**, ships as `ghcr.io/peonist-ai/
+halogen-flash-server:0.16.2` (the version the model card's quickstart pins),
+digest-pinned:
+
+```
+ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
+```
+
+(3.6 GB image; RepoDigest captured at pull 2026-10-06. Card-tested fallback
+if a future pin misbehaves: 0.15.1.)
+
+## The payload — a triangle of files, all sha256-verified at fetch
+
+| File | Bytes | Source repo @ revision | sha256 (LFS) |
+|---|---|---|---|
+| `qwen38-flash-next-v2-swift15-abliterated.hgn` | 66,687,678,464 | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated @ main (`69a85db`) | `9c880b8ba48deabfa06172eff1e7572e4fe465e09e97faec6e50c3f6fdcac037` |
+| `qwen38-flash-next-ngram.hgn` (REQUIRED) | 51,200,246,144 | peonist-ai/halogen-qwen3.8-flash-next @ `5cc17cea` | `3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe` |
+| `qwen38-flash-next-vision.hgn` (enabled) | 897,916,416 | peonist-ai/halogen-qwen3.8-flash-next @ `e053f488` | `d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10` |
+| `tokenizer/` (6 files, flat dir) | 22.4 MB total | peonist @ `e053f488` | `tokenizer.json` is LFS `0997f410…` (byte-identical to the 27B-era copy); the rest are plain git blobs, size-checked |
+
+Payload notes worth keeping:
+
+- The deployed checkpoint is a **compacted r2** of the card's append-only
+  original (89,775,868,032 B → 66,687,678,464 B): the abliteration workflow
+  appends and the repo's `tooling/` compact drops dead padding; live tensors
+  are byte-identical, the LFS sha256 is the contract, and the 24 GiB saving
+  is why it fits.
+- **`HALOGEN_MTP_HEAD` stays UNSET**: the MTP head is INSIDE the v2
+  checkpoint (the separate `qwen38-flash-next-mtp.hgn` sidecar is for a
+  different assembly and is deliberately not fetched).
+- The **n-gram lookup table is REQUIRED**. The engine reads it beside the
+  checkpoint (documented fallback); the card's quickstart also names it
+  explicitly via `HALOGEN_NGRAM_TABLE`, and the serve script passes both —
+  the env var is belt-and-braces over the documented beside-the-checkpoint
+  default, and both mechanisms resolve to the same file.
+- The tokenizer is fetched fresh from peonist @ `e053f488` into
+  `/home/dank/models/halogen-flashnext-v2/tokenizer/` — same flat-directory,
+  no-symlink law as every other payload (an HF cache snapshot's symlinks
+  dangle inside the container mount; both the serve gate and the deployer
+  refuse them).
+
+Fetch: `scripts/download-frank-large-flashnext.sh` (`--verify-only` to
+re-check; `-f` + exact byte counts + LFS sha256 + resolve-the-CDN-then-`-C -`
+resume law throughout; no token — the repos are public).
+
+## Shape — the knobs that are not free
+
+`scripts/frank-large.env` carries the full rationale inline; the table:
+
+| Env | Value | Why |
+|---|---|---|
+| `HALOGEN_CTX` | 200192 | per-slot HARD ceiling (a longer prompt is a 400 naming the limit, never truncation). Harness compaction: 70 % = 140,134; 140,134 + 57,344 = 197,478 ≤ 200,192 |
+| `HALOGEN_KV_SLOTS` | 2 | the 27B engine's "slots>1 disables the drafter" law does NOT carry over — on this engine batching and speculation coexist (the card's quickstart ships 2 slots WITH the head). 2 slots = concurrent capacity |
+| `HALOGEN_KV_POOL_POSITIONS` | 400384 | explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ ~10.7 GiB KV |
+| `HALOGEN_HOST_RESERVE_GIB` | 20 | keeps the host honest (the w4b-era lesson) |
+| `HALOGEN_SPEC_ADAPT` | **0** | MANDATORY (card): the default adaptive policy turns the MTP head off for whole requests after low acceptance and **halves decode**. The head stays on; temp-1.0 traffic pays some acceptance for it — accepted trade |
+| `HALOGEN_MAX_TOK` / `HALOGEN_MAX_TOKENS_CAP` / `HALOGEN_QUEUE_TIMEOUT` | 8192 / 65536 / 7200 | 8192 is only the per-request DEFAULT (the harness always sends 57,344); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s) |
+| `HALOGEN_TEMPERATURE` / `TOP_P` / `TOP_K` | 1.0 / 0.95 / 20 | the card's thinking recipe as SERVER-SIDE defaults — the engine HAS a sampler now, so an absent temperature is no longer greedy; models.yml extraBody matches it (consistent, not load-bearing) |
+| `HALOGEN_WEIGHTS_LOCK` | 1 | card's belt-and-braces; the engine pins weights by default on v2 (`HALOGEN_FLASH_PIN_TRUNK=0` is a w4b-only knob and is REFUSED by v2) |
+| `HALOGEN_MODEL_ID` / publish | frank-large / 7770:8731 | unchanged aliases; engine port 8730 never published |
+| MTP depth | default 2 | FLAGS.md: 2 is best all-round, 3 for code — Frank's traffic is agent/prose |
+
+**Memory ledger** (121.2 GiB unified, the referee is the boot ledger):
+weights ~62 GiB resident + KV ~10.7 + arena ~8 (`3.8 + 8.4 × 8192/16384`)
++ vision ~2 ≈ **82.7 GiB engine-side**, + 20 host reserve = ~103 with
+~18 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram
+table rides as mmap'd page cache beside the checkpoint — the card's own
+quickstart arithmetic does not bill it as resident, but WATCH the ledger
+at boot and re-size if the engine prints a clamp. Prompt cache is ON by
+default (`PROMPT_CACHE=2`): a follow-up turn at 100k ctx costs ~2 s
+instead of ~88 s cold — the big-TTFT win the 27B engine needed a manual
+`HALOGEN_CACHE_MB` pin for is built in here.
+
+## Engine contract facts (FLAGS.md of the flash-server repo)
+
+- OpenAI chat **and** `/v1/responses` **and** `/v1/messages` **and**
+  `/v1/completions`; tool calls verified on the card; `response_format`
+  still unsupported (no constrained decoding).
+- Thinking → **`reasoning_content`** (same field as every engine this
+  endpoint has run); the v2 chat template rejects `reasoning_effort`
+  outside `xhigh|medium|low` — the models.yml fold (high→xhigh) stays
+  mandatory.
+- Vision: OpenAI `image_url` content parts only (the fleet's standard
+  contract; the legacy top-level `images` field is the frank-small trap).
+  ≥256 px probe images or the model answers "video".
+- `HALOGEN_MAX_TOKENS_CAP` over-cap is a 400 naming the limit — never a
+  truncation. `max_tokens` INCLUDES thinking.
+- All flags read once at startup; every flag is BITWISE except
+  `HALOGEN_CK_OVERLAY`, `HALOGEN_MATMUL_TUNING_FILE`,
+  `HALOGEN_INDEXER_BUDGET`, `HALOGEN_COMPOSABLE_CONTEXT`. Unset
+  `HALOGEN_CHECKPOINT` defaults to `/models/qwen38-flash-next-v2.hgn`
+  with the lookup table beside it.
+- Prompt-lookup drafting (3,3 window) rides beside the MTP head by
+  default and is byte-identical to greedy verification where it fires.
+- Watchdog: `HALOGEN_ENGINE_WATCHDOG_S=180` with defer-logic for memory
+  stalls — on a shared host the defaults are the safe choice; the
+  restart policy stays real through the root unit's `Restart=always`.
+
+Kernel: `CONFIG_HSA_AMD_SVM=y` (CachyOS ships it); `amdgpu.noretry=-1`
+(auto) — for this IP auto resolves to retry-on, the same effective state
+the 27B engine ran on. Only revisit if the ledger shows SVM complaints.
+
+## Mechanics
+
+- **Root systemd unit** (`scripts/frank-large.service`, `User=dank`,
+  `LimitMEMLOCK=infinity`, `Restart=always`, `ExecStop=podman stop -t 60
+  frank-large-halogen`), driven by `scripts/serve-frank-large.sh` — the
+  fleet's `podman run`-direct pattern. The script runs the container
+  FOREGROUND so the boot ledger (weights resident, pool fit, MTP head,
+  prompt cache, any clamp) lands in `/home/dank/logs/frank_large.log`.
+- Serve gates mirror the deployer EXACTLY (2026-10-01 crash-loop law):
+  image digest in podman storage, byte-exact checkpoint + n-gram table +
+  vision tower, flat tokenizer with all six files as regular files,
+  GPU nodes present.
+- Engine stdout goes to the log, NOT the journal. The FIRST request after
+  a (re)start can sit silent ~45–60 s on this engine class (health
+  `engine_unresponsive`, 503s) and self-recovers — do not restart over it.
+
+## Verification checklist (run after any redeploy)
+
+1. Boot ledger greps in `/home/dank/logs/frank_large.log`: resident
+   weights (~62 GiB), KV pool line, MTP/drafter line, prompt-cache line,
+   **no clamp named**.
+2. `curl http://192.168.0.143:7770/health` → 200.
+3. Text completion probe: reasoning lands in `reasoning_content`,
+   `timings` honest (generate ≥128 tokens; at 8 the inter_token number is
+   first-token noise).
+4. Vision probe: a 512×512 solid-red PNG through an `image_url` part →
+   "Red" (≥256 px, one word expected).
+5. Spec decode: `draft_n`/`draft_n_accepted` present in timings on
+   real-shaped traffic (SPEC_ADAPT=0 means the head is always engaged).
+
+## Client contract (harness)
+
+`~/.omp/agent/models.yml` — provider `llamacpp (dankclaw)` @
+`http://192.168.0.143:7770/v1`: `contextWindow: 200192`,
+`maxTokens: 57344`, `input: [text, image]`,
+`reasoningContentField: "reasoning_content"`, the `high→xhigh` effort
+fold, `extraBody` temperature 1.0 / top_p 0.95. Compaction still rides
+frank-small (the `compactionModel` steering is unchanged).
+
+## Rollback
+
+Revert the swap commit and redeploy — the 27B trio is intact on disk
+(payload at `~/models/halogen-qwen3.8-27b/`, fetch script, and the halogen
+0.1.4 image digest still in podman storage). The older ROCmFPX rollback
+door (`build-rocmfpx.sh` pin) is untouched.

serverAppConfigs-docs/LLM/index.md

@@ -10,11 +10,12 @@ experts in a 40 GiB/rank pinned host pool + VRAM cache (ratio 0.82), the
 47.7 GiB PLE table pinned to host RAM, fp8 KV, MTP spec decode depth 3,
 2 × 200,000 first boot. Rollback tags frank-primary-freetoken-pr587-20261005
 and frank-primary-vllm-shapeG-20261003),
-and Dankclaw runs two endpoints — **halogen container (frank-large, port 7770:
-ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
-/halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
-engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
-lives on frank-small)** and
+and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
+(frank-large, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
+digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
+Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
+MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
+ctx, vision ON)** and
 **llama.cpp CUDA (frank-small, port 7771: ajgazin Swift-1.5-Qwen3.8-27B
 Uncensored Dynamic-MTP RCO-IQ3_XXS GGUF — 27B (the MTP head in the file
 IS drafted, draft-mtp depth 3), TWO 131,072 conversations at Frank's
@@ -45,10 +46,10 @@ PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 3,
     subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
         E2[RTX 3090 (PCIe x4 dock) +
 gfx1151 iGPU 114 GiB GTT]
-        E[halogen container gfx1151, digest-pinned image
+        E[flash-server container gfx1151, digest-pinned image
 frank-large, port 7770]
-        F[halogen-qwen3.8-27b .hgn p1w4d-d2 + DFlash2 drafter
-1 x 200700 ctx, d2 KV, text-only]
+        F[flash-server v2 abliterated Swift1.5 .hgn + ngram table + vision tower
+2 x 200192 ctx, MTP always-on, image input]
         G[llama.cpp master CUDA sm_86
 frank-small, port 7771]
         H[Swift-1.5-27B Uncensored RCO-IQ3_XXS-mtp GGUF
@@ -71,30 +72,37 @@ frank-small, port 7771]
 
 ## Active Deployment
 
-- [[frank-large-halogen27b-dflash2]] — **ACTIVE since 2026-10-05**
+- [[frank-large-flash-server-v2-swift15]] — **ACTIVE since 2026-10-06**
   (root systemd unit + podman, port 7770): the digest-pinned
-  `ghcr.io/peonist-ai/halogen` image serving `peonist-ai/halogen-qwen3.8-27b`
-  (`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB) with the DFlash2 drafter on ONE slot
-  of 200,700 ctx (slots>1 disables speculation — engine law). Publisher card
-  31.7 t/s mean across eight text workloads. Accepted tradeoffs: TEXT-ONLY
-  (the engine ignores the vision encoder; frank-small owns images) and the
-  CENSORED base model (the ROCmFPX AGENT file was Uncensored by construction).
-  No server-side sampler — the thinking recipe rides the wire in models.yml.
-  Rollback: revert the swap commit; the ROCmFPX payload + fork build +
-  `build-rocmfpx.sh` pin all remain on disk
-  Tuned 2026-10-06: `HALOGEN_CACHE_MB=16384` pins the prompt cache —
-  auto was OFF at the 36.9 GB two-entry floor, so every deep request
-  re-prefilled its 79-81k context (85-180 s each); a 17.3k cold prefill
-  of 27.0 s now resumes in 1.7 s on the warm turn (cache_n 16,384). The
-  serve script captures the engine ledger on disk (foreground `podman
-  run`; the `-d` era wrote only the container ID). Decode-at-depth is
-  structural: 3.3-3.8 t/s at 80k vs 30-50 shallow; no supported flag
-  touches it.
-  The FA tile lever from upstream issue #12 (`HALOGEN_FA_TQ=256`) was
-  A/B'd at 80k on 2026-10-06: decode ±0.3 % (rejected), prefill −9.5 %
-  on this topology — the default tile stands. The deep harness roles
-  (`slow`, `advisor`) rerouted to frank-primary; frank-large keeps
-  OpenWebUI and shallow callers.
+  `ghcr.io/peonist-ai/halogen-flash-server` 0.16.2 engine serving the
+  Quat3rnion ABLITERATED Swift1.5 finetune of peonist's
+  Qwen3.8-Flash-Next v2 checkpoint (`qwen38-flash-next-v2-swift15-abliterated.hgn`,
+  66.7 GB compacted r2 of the card's 89.8 GB append-only original,
+  sha256-verified) with the REQUIRED n-gram lookup table (51.2 GB,
+  @peonist 5cc17cea) and the vision tower (897.9 MB, @e053f488) beside
+  it. TWO slots of 200,192 ctx, MTP head ALWAYS ON (`HALOGEN_SPEC_ADAPT=0`
+  — the adaptive default turns the head off per request and halves
+  decode), server-side sampler at the card's 1.0/0.95/20, coupled
+  cap/timeout 65536/7200, prompt cache ON by default (the 27B engine's
+  manual `HALOGEN_CACHE_MB` pin era is over — warm 100k turn ≈ 2 s vs
+  ≈ 88 s cold). Vision ON (image_url parts only). Card: 53-57 t/s
+  greedy, ~1,640 t/s prefill at 34k, ~72 % fewer thinking tokens.
+  Rollback: revert the swap commit; the 27B payload + fetch script stay
+  on disk
+- [[frank-large-halogen27b-dflash2]] — **RETIRED 2026-10-05 → 10-06
+  (rollback target; payload + fetch script + image on disk)**: the
+  digest-pinned `ghcr.io/peonist-ai/halogen` 0.1.4 engine serving
+  `peonist-ai/halogen-qwen3.8-27b` (`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB)
+  — DFlash2 drafter, ONE 200,700 slot, text-only (the engine ignored the
+  vision encoder), censored base. Owns the measured lessons: the prompt
+  cache pin (`HALOGEN_CACHE_MB=16384` — auto was OFF at the 36.9 GB
+  two-entry floor; a 17.3k cold prefill of 27.0 s resumed in 1.7 s warm),
+  the FA tile rejection (`HALOGEN_FA_TQ=256`: prefill −9.5 %, decode
+  ±0.3 %), and deep decode at 3.3-3.8 t/s@80k being structural. The
+  foreground-`podman run` ledger capture and the deep-role rerouting
+  (`slow`/`advisor` → frank-primary) carried forward. Superseded by the
+  MODEL swap: the abliterated v2 .hgn loads only in flash-server, so the
+  engine moved with the model
 - [[frank-small-llama-swift15-unc-iq3xxs-mtp-131072]] — **ACTIVE since
   2026-10-05 (evening); UNCECERSED RCO-IQ3_XXS with MTP back on since
   2026-10-06 evening** (root systemd unit, port 7771): llama.cpp master
@@ -400,7 +408,7 @@ frank-small, port 7771]
   (deployed and reverted 2026-08-22)
 - [[Dankserver System Specs]] — the 2x R9700 box (frank-primary)
 - [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on
-  the unsloth b11030 runtime since 2026-09-21, frank-small on the RTX 3090)
+  the flash-server v2 runtime since 2026-10-06, frank-small on the RTX 3090)
 - [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
   on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
   the retirement record — the IOMMU/SVA err=-19 root cause and where