docs: frank-large flash-server v2 cutover — runbook, index, AGENTS.md

- New runbook: frank-large-flash-server-v2-swift15.md (engine, shape,
  memory model, engine laws, boot ledger, probes, rollback)
- LLM/index.md: active entry + retired halogen 27B entry
- Vault index: link the new runbook from the LLM Fleet section
- AGENTS.md: fleet topology table, Model Config section,
  Runbook Index, retired table (halogen 27B → 10-06)
This commit is contained in:
Frank
2026-10-06 03:35:27 -07:00
parent 20d7182f4b
commit 2d89bf97cc
3 changed files with 312 additions and 109 deletions
+40 -32
View File
@@ -10,11 +10,12 @@ experts in a 40 GiB/rank pinned host pool + VRAM cache (ratio 0.82), the
47.7 GiB PLE table pinned to host RAM, fp8 KV, MTP spec decode depth 3,
2 × 200,000 first boot. Rollback tags frank-primary-freetoken-pr587-20261005
and frank-primary-vllm-shapeG-20261003),
and Dankclaw runs two endpoints — **halogen container (frank-large, port 7770:
ghcr.io/peonist-ai/halogen @ pinned digest serving peonist-ai
/halogen-qwen3.8-27b, a 33.4 GiB .hgn checkpoint that loads in no other
engine, DFlash2 spec decode, ONE slot at 200,700 ctx, text-only — vision
lives on frank-small)** and
and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
(frank-large, port 7770: ghcr.io/peonist-ai/halogen-flash-server @ pinned
digest serving the Quat3rnion ABLITERATED Swift1.5 finetune of
Qwen3.8-Flash-Next v2, a ~62 GiB v2 .hgn that loads in no other engine,
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots at 200,192
ctx, vision ON)** and
**llama.cpp CUDA (frank-small, port 7771: ajgazin Swift-1.5-Qwen3.8-27B
Uncensored Dynamic-MTP RCO-IQ3_XXS GGUF — 27B (the MTP head in the file
IS drafted, draft-mtp depth 3), TWO 131,072 conversations at Frank's
@@ -45,10 +46,10 @@ PLE 47.7 GiB pinned RAM, fp8 KV 7.32 KiB/tok, MTP depth 3,
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
E2[RTX 3090 (PCIe x4 dock) +
gfx1151 iGPU 114 GiB GTT]
E[halogen container gfx1151, digest-pinned image
E[flash-server container gfx1151, digest-pinned image
frank-large, port 7770]
F[halogen-qwen3.8-27b .hgn p1w4d-d2 + DFlash2 drafter
1 x 200700 ctx, d2 KV, text-only]
F[flash-server v2 abliterated Swift1.5 .hgn + ngram table + vision tower
2 x 200192 ctx, MTP always-on, image input]
G[llama.cpp master CUDA sm_86
frank-small, port 7771]
H[Swift-1.5-27B Uncensored RCO-IQ3_XXS-mtp GGUF
@@ -71,30 +72,37 @@ frank-small, port 7771]
## Active Deployment
- [[frank-large-halogen27b-dflash2]] — **ACTIVE since 2026-10-05**
- [[frank-large-flash-server-v2-swift15]] — **ACTIVE since 2026-10-06**
(root systemd unit + podman, port 7770): the digest-pinned
`ghcr.io/peonist-ai/halogen` image serving `peonist-ai/halogen-qwen3.8-27b`
(`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB) with the DFlash2 drafter on ONE slot
of 200,700 ctx (slots>1 disables speculation — engine law). Publisher card
31.7 t/s mean across eight text workloads. Accepted tradeoffs: TEXT-ONLY
(the engine ignores the vision encoder; frank-small owns images) and the
CENSORED base model (the ROCmFPX AGENT file was Uncensored by construction).
No server-side sampler — the thinking recipe rides the wire in models.yml.
Rollback: revert the swap commit; the ROCmFPX payload + fork build +
`build-rocmfpx.sh` pin all remain on disk
Tuned 2026-10-06: `HALOGEN_CACHE_MB=16384` pins the prompt cache —
auto was OFF at the 36.9 GB two-entry floor, so every deep request
re-prefilled its 79-81k context (85-180 s each); a 17.3k cold prefill
of 27.0 s now resumes in 1.7 s on the warm turn (cache_n 16,384). The
serve script captures the engine ledger on disk (foreground `podman
run`; the `-d` era wrote only the container ID). Decode-at-depth is
structural: 3.3-3.8 t/s at 80k vs 30-50 shallow; no supported flag
touches it.
The FA tile lever from upstream issue #12 (`HALOGEN_FA_TQ=256`) was
A/B'd at 80k on 2026-10-06: decode ±0.3 % (rejected), prefill −9.5 %
on this topology — the default tile stands. The deep harness roles
(`slow`, `advisor`) rerouted to frank-primary; frank-large keeps
OpenWebUI and shallow callers.
`ghcr.io/peonist-ai/halogen-flash-server` 0.16.2 engine serving the
Quat3rnion ABLITERATED Swift1.5 finetune of peonist's
Qwen3.8-Flash-Next v2 checkpoint (`qwen38-flash-next-v2-swift15-abliterated.hgn`,
66.7 GB compacted r2 of the card's 89.8 GB append-only original,
sha256-verified) with the REQUIRED n-gram lookup table (51.2 GB,
@peonist 5cc17cea) and the vision tower (897.9 MB, @e053f488) beside
it. TWO slots of 200,192 ctx, MTP head ALWAYS ON (`HALOGEN_SPEC_ADAPT=0`
— the adaptive default turns the head off per request and halves
decode), server-side sampler at the card's 1.0/0.95/20, coupled
cap/timeout 65536/7200, prompt cache ON by default (the 27B engine's
manual `HALOGEN_CACHE_MB` pin era is over — warm 100k turn ≈ 2 s vs
≈ 88 s cold). Vision ON (image_url parts only). Card: 53-57 t/s
greedy, ~1,640 t/s prefill at 34k, ~72 % fewer thinking tokens.
Rollback: revert the swap commit; the 27B payload + fetch script stay
on disk
- [[frank-large-halogen27b-dflash2]] — **RETIRED 2026-10-05 → 10-06
(rollback target; payload + fetch script + image on disk)**: the
digest-pinned `ghcr.io/peonist-ai/halogen` 0.1.4 engine serving
`peonist-ai/halogen-qwen3.8-27b` (`qwen3.8-27b-p1w4d-d2.hgn`, 33.4 GiB)
— DFlash2 drafter, ONE 200,700 slot, text-only (the engine ignored the
vision encoder), censored base. Owns the measured lessons: the prompt
cache pin (`HALOGEN_CACHE_MB=16384` — auto was OFF at the 36.9 GB
two-entry floor; a 17.3k cold prefill of 27.0 s resumed in 1.7 s warm),
the FA tile rejection (`HALOGEN_FA_TQ=256`: prefill −9.5 %, decode
±0.3 %), and deep decode at 3.3-3.8 t/s@80k being structural. The
foreground-`podman run` ledger capture and the deep-role rerouting
(`slow`/`advisor` → frank-primary) carried forward. Superseded by the
MODEL swap: the abliterated v2 .hgn loads only in flash-server, so the
engine moved with the model
- [[frank-small-llama-swift15-unc-iq3xxs-mtp-131072]] — **ACTIVE since
2026-10-05 (evening); UNCECERSED RCO-IQ3_XXS with MTP back on since
2026-10-06 evening** (root systemd unit, port 7771): llama.cpp master
@@ -400,7 +408,7 @@ frank-small, port 7771]
(deployed and reverted 2026-08-22)
- [[Dankserver System Specs]] — the 2x R9700 box (frank-primary)
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large on
the unsloth b11030 runtime since 2026-09-21, frank-small on the RTX 3090)
the flash-server v2 runtime since 2026-10-06, frank-small on the RTX 3090)
- [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
on Dankclaw (Gemma4 E4B vision+audio pack, XDNA2, port 7771), kept as
the retirement record — the IOMMU/SVA err=-19 root cause and where